Aakash GuptaYou'll be left Behind as an AI PM If You Don't Use ChatGPT Apps
CHAPTERS
- 0:00 – 3:10
Why ChatGPT apps are a massive new distribution channel
Colin frames ChatGPT apps as an underrated opportunity: apps can live inside the chat experience, giving products a new way to reach users. Aakash sets the episode goal—translating hypey announcements into practical implications for product builders and PMs.
- •ChatGPT apps enable interactive app experiences directly within a conversation
- •Distribution upside: being present where users already are
- •Early framing: what this means specifically for product builders and AI PMs
- 3:10 – 4:40
Defining ChatGPT apps and the coming “App Store” experience
They define ChatGPT apps as branded, interactive experiences embedded in ChatGPT rather than simple text responses. The discussion covers current discoverability limitations and the likely evolution toward a browsable marketplace similar to iOS/Android app stores.
- •Apps bring a company’s UI/UX and workflows into ChatGPT
- •Current state: apps are somewhat hidden; discovery is limited
- •Future state: a full marketplace where users browse/install apps
- 4:40 – 6:46
Automatic surfacing, high-intent traffic, and enterprise control over the experience
Colin explains how ChatGPT may surface relevant apps even when users haven’t installed them, acting like tool-routing for tasks. They discuss why enterprises care: AI-driven traffic is lower volume but higher conversion, and apps provide a deterministic way to show up and control brand experience.
- •ChatGPT may proactively suggest an app when a query matches its capabilities
- •LLM-driven referrals often have higher intent and conversion
- •Apps provide deterministic presence vs. ‘whac-a-mole’ SEO/web-search inclusion
- •Example: Target cart-building inside ChatGPT with checkout handoff to Target site
- 6:46 – 8:36
What small builders can make: utilities to full-feature mini-products
They move from enterprise examples to what ‘regular people’ can build—small utilities like pinned to-do lists or spreadsheet-like tools, plus richer experiences with maps, search, and integrations. Colin emphasizes that limitations are mostly about early-stage maturity, not capability.
- •Utility apps: to-do lists, spreadsheets, lightweight helpers pinned in chat
- •Richer apps: maps, navigation, search, backend integrations
- •Few strict capability limits; current apps are early and often bare-bones
- 8:36 – 12:41
Architecture deep dive: MCP + tools + widgets/UI rendering
Colin breaks down the technical foundation: Model Context Protocol (MCP) enables chat agents to call external tools over the internet. On top of tool outputs, OpenAI-style widgets/UI are fetched and rendered in-chat, enabling interactive experiences beyond plain text.
- •MCP (Model Context Protocol) connects LLMs to external tools/services
- •ChatGPT decides whether a request should use an app/tool
- •Tools expose actions (e.g., browse listings, book listing) with parameters
- •Widgets/UI layer: tool can return a UI endpoint rendered in the chat
- 12:41 – 16:09
The easy way vs. hard way to build: Chippy vs. DIY MCP stack
Colin introduces Chippy, his platform that scaffolds the MCP server and app-building workflow to reduce iteration friction. He contrasts it with the DIY approach: hosting your own MCP server, writing tool definitions/UI, bundling code for ChatGPT rendering, and handling guidelines like full-screen vs. inline modes.
- •Chippy scaffolds MCP server + tool-building workflow to speed prototyping
- •Preview/test loop inside Chippy avoids repeated rebundling + re-uploading
- •DIY path requires hosting, tool definitions, UI, bundling, and platform guidelines knowledge
- 16:09 – 19:53
Connecting an app to ChatGPT + the three ways apps get invoked
They walk through connecting a Chippy app to ChatGPT via a connector URL, naming, and settings. Colin explains invocation modes: explicitly typing the app name, selecting it from the app list, or having ChatGPT infer and invoke it automatically based on the user query—where discoverability and prompt matching become crucial.
- •Developer testing flow: add connector URL, name, configure auth, connect
- •Invocation methods: name-typed, manually selected/tagged, or auto-inferred by ChatGPT
- •Automatic invocation creates a need for evaluation and iteration on triggers
- 19:53 – 26:29
Live build: healthcare reviews manager (plan mode → tools → UI)
Aakash chooses healthcare, and Colin proposes a hospital review management app. Using plan mode, they generate a toolset (view reviews, share reviews, analytics), discuss PM prototyping value, and inspect the initial UI/UX and mock data approach.
- •Idea selection: hospital/surgeon reviews influence revenue and trust
- •Plan mode generates a structured tool plan before execution
- •Three-tool design: view, share, analytics dashboards/cards
- •PM value: prototyping specs quickly to communicate with engineering/design
- 26:29 – 29:12
Testing in ChatGPT: tool calls, data quirks, and connector visibility
They connect the healthcare app to ChatGPT and run prompts to trigger different tools. The demo reveals a data/UX glitch (mock data flicker), but they successfully show multiple tool calls and highlight that connectors expose the available tools directly in ChatGPT settings.
- •End-to-end test: connect ‘Healthcare Reviews’ and run prompts
- •Tool selection in practice can be imperfect; iterative debugging required
- •ChatGPT connector UI reveals the tools exposed by the app (analytics/share/view)
- 29:12 – 34:01
Observability + evals: building a golden set (direct/indirect/negative)
Colin demonstrates the observability logs—user prompt, tool chosen, parameters—and explains how that enables an eval workflow. They define OpenAI-aligned eval types (direct, indirect, negative) and add an indirect eval case from the live prompt to start forming a golden set.
- •Observability shows: user prompt, chosen tool, parameters, and outputs
- •Evals are essential for ‘should this prompt trigger this tool?’ questions
- •Eval taxonomy: direct (named app), indirect (describes outcome), negative (unrelated)
- •Golden set grows from real interactions and logged prompts
- 34:01 – 40:19
Closing the loop: auto vs. manual evals and fixing tool metadata
They run an auto eval and see a mismatch: the prompt ‘I want to share a review’ should map to the share tool, but ChatGPT invoked view reviews due to ambiguous descriptions. Colin shows how to fix it by editing the tool description (removing ‘sharing’) and explains that descriptions can include examples and operational guidance, not just keywords.
- •Auto evals provide fast directional feedback but may differ from live ChatGPT
- •Failure analysis: tool descriptions can accidentally bias selection
- •Fix strategy: refine tool metadata/descriptions to improve routing
- •Descriptions can include examples and usage guidance beyond simple ‘SEO-like’ keywords
- 40:19 – 43:49
PM role debate: prototyping supports core PM work (not role bloat)
Aakash raises the concern that PM responsibilities keep expanding, citing a viral framework of core PM activities. Colin argues AI prototyping is a supporting skill—like using Figma—enhancing stakeholder/customer communication rather than replacing foundational PM work.
- •AI prototyping isn’t a separate PM responsibility; it supports existing ones
- •Analogy: Figma proficiency helps PMs communicate and iterate faster
- •Prototyping boosts fidelity in stakeholder/customer conversations
- •Core PM skills remain essential; prototyping is leverage, not a substitute
- 43:49 – 48:18
Strategic guidance: benefits, who should build, and how to decide priority
They map the benefits of ChatGPT apps into skills/career learning and enterprise growth outcomes, emphasizing high-intent distribution. Colin recommends a cross-functional pod (PM/design/engineering) and gives decision criteria: try the form factor, watch competitor adoption and surfacing behavior, and consider retention/re-engagement via micro-app touchpoints.
- •Primary enterprise benefit: growth via high-intent ChatGPT distribution
- •Team model: pod with designer, engineers (auth/tooling), and PM (strategy/evals/insights)
- •Decision criteria: experiment, observe real app surfacing, benchmark competitors
- •Potential retention angle: users interact with brand without logging into main app
- 48:18 – 55:29
Ideas for solopreneurs + the MCP advantage across multiple assistants
They brainstorm solo-builder app ideas: embedded utilities where ChatGPT collaborates (spreadsheets, task lists), and ‘good-enough’ alternatives to standalone AI products (e.g., presentations like Gamma). Colin notes MCP’s broader ecosystem: building once can unlock distribution across ChatGPT and other MCP-supporting platforms (Claude in progress, plus tools like Cursor/Lovable), though Gemini support is uncertain.
- •Best solo ideas: embedded collaboration utilities that reduce context switching
- •‘Good enough’ apps can win via distribution inside ChatGPT even vs. specialized tools
- •E-commerce and content-creation are natural fits (Target/Amazon hypotheticals, Canva)
- •MCP enables cross-platform potential: one integration, many assistants (ChatGPT/Claude/etc.)
- •Caveat: ecosystem is early; success depends on marketplace execution and adoption
- 55:29 – 1:02:22
Colin’s solopreneur year: bets, stack, and why he prefers VS Code + Claude Code
Aakash shifts to Colin’s personal journey: a year of solopreneurship with multiple experiments (podcast, SaaS apps, prototyping tools). Colin shares how he allocates time between operations and new bets, his goal of balancing software with teaching, and his practical dev stack—including why he uses VS Code with Claude Code/Codex over Cursor.
- •Time split: maintaining the business vs. running new product bets
- •Experimentation mindset: try fast, drop what won’t reach top-tier execution
- •Tech stack highlights: Replit for UX exploration, VS Code + Claude Code/Codex for codegen, Neon DB, Render hosting, Git/GitHub
- •Tool preference: prioritizes codegen quality over ‘bells and whistles’
