Aakash GuptaYou'll be left Behind as an AI PM If You Don't Use ChatGPT Apps
CHAPTERS
Why ChatGPT apps are a major opportunity for product builders
Aakash frames the episode as a practical breakdown of what the emerging “ChatGPT App Store” means for builders, beyond the headline demos. Colin previews why embedded apps inside chat can create new distribution and product surface areas for teams and solo builders.
- •Goal: move past hype/use-case montages to builder-relevant implications
- •Chat-based apps as a new distribution channel (potentially underrated)
- •What PMs should understand now before the marketplace fully opens
What “ChatGPT apps” are (and how the App Store concept works)
Colin defines ChatGPT apps as branded, designed interactive experiences that run inside a ChatGPT conversation—more than text outputs or web results. They let companies control user experience, guide interactions, and sometimes hand users off to complete actions in the company’s own product.
- •Embedded UI + interaction directly inside the chat thread
- •Companies control branding and workflow vs relying on generic web search answers
- •App-store-like browsing/installing is expected to evolve from today’s limited discoverability
- •Apps can also deep-link users back to the native product for completion steps (e.g., checkout)
Discovery, intent, and why enterprises care (Target/Expedia examples)
They unpack why discovery—even if currently ‘hidden’—can become powerful once ChatGPT starts surfacing relevant apps automatically. Colin explains why AI-referred users can convert better and why enterprises want a deterministic way to appear inside ChatGPT rather than playing “Whac-A-Mole” with SEO.
- •Automatic surfacing: ChatGPT may propose an app even if user didn’t install it
- •AI traffic can have higher intent and higher conversion despite lower volume
- •Deterministic presence vs volatile web-search ranking battles
- •Example pattern: build cart in-chat, finish purchase in first-party app/site (Target)
What regular builders can ship: utilities and micro-apps inside chat
Colin predicts a familiar curve: big brand apps plus small utility apps (like early iOS “flashlight apps”). He outlines lightweight but sticky ideas—spreadsheets, to-do lists pinned to the top, and other tools that create repeat usage within ChatGPT.
- •Utility apps: spreadsheet-like collaboration, to-do/task checklist UI, pinned tools
- •More complex apps: maps, navigation, search, integrations—few limits long-term
- •Early apps are bare-bones because the platform is early; complexity will increase
- •Opportunity for niche workflows where chat needs structured UI to be useful
Architecture deep dive: MCP + tool calling + UI widgets
Colin explains the underlying mechanism: Model Context Protocol (MCP) lets an AI client discover and call external tools. OpenAI’s UI “widgets” layer (now being incorporated into MCP) adds the ability to render interactive interfaces inside chat instead of only returning text/data.
- •MCP (from Anthropic): a standard for agents to call tools over the internet
- •Flow: user request → model decides to use a tool → fetch tool list → call tool with parameters
- •Caching of tool schemas/availability; refresh when needed
- •Widgets/UI: tool returns data plus UI code/URL so ChatGPT can render an interface in-chat
Building a ChatGPT app: the easy path (Chippy) vs the hard path (DIY)
Colin contrasts building with his platform (Chippy) versus spinning up your own MCP server and UI bundling pipeline. Chippy is positioned as an iteration-friendly environment that previews the chat experience without repeatedly rebundling and reconnecting inside ChatGPT.
- •Easy path: Chippy spins up MCP server + tool scaffolding optimized for ChatGPT apps
- •Preview/testing loop built-in: simulated chat + UI rendering before connecting to ChatGPT
- •Hard path: host MCP server, define tools, build UI, handle bundling/packaging for rendering
- •Extra complexity: form factors (full screen vs inline), OpenAI guidelines, auth, deployment
Live setup: connecting an MCP app and ways users invoke it
They walk through connecting an app via a generated MCP URL and enabling it in ChatGPT settings. Colin explains three invocation modes: typing the app name, manually selecting/tagging it, or relying on ChatGPT to choose it automatically for relevant prompts.
- •Developer workflow: create connector, paste MCP URL, configure auth/name
- •Invocation modes: name-based, manual selection, or automatic relevance-based routing
- •“Automatic invocation” becomes the distribution battleground (showing up on relevant queries)
- •Marketplace status: public publishing not yet open; sharing via URL works for testing
Live build: a healthcare reviews manager app (tools + UI)
Aakash chooses a healthcare domain and they build a hospital reviews app with three tools: view reviews, share reviews, and review analytics. They discuss PM value: using these apps as prototypes/specs and learning the full AI product loop quickly.
- •Domain selection: hospital/surgeon reviews as revenue-relevant use case
- •Tool design: viewing, sharing, analytics as separate callable capabilities
- •Plan mode to draft tool breakdown before generating implementation
- •PM angle: prototype-to-spec workflow; engineers can productionize later
Observability and evals: measuring whether the right tool gets called
Colin demonstrates logs/observability showing the user prompt, tool called, and parameters selected by ChatGPT. He introduces OpenAI’s eval categories—direct, indirect, and negative—to systematically test discoverability and prevent irrelevant tool calls.
- •Observability: inspect prompts, tool routing, parameters, and outputs
- •Eval types: direct (names app), indirect (describes outcome), negative (should not call tool)
- •Build a “golden set” from real interactions and log expected behavior
- •Tool-routing quality is a core metric: does the model choose the intended tool?
Improving performance with eval feedback: fix tool metadata/descriptions
They run an “auto eval” and see a mismatch: the prompt “I want to share a review” should trigger the share tool but calls the view tool due to ambiguous wording. Colin shows how small edits to tool descriptions (metadata) can improve routing behavior and recommends iterating via tests rather than guessing.
- •Auto evals provide fast directional feedback but may differ from in-product ChatGPT behavior
- •Failure analysis: the word “sharing” in the view tool description caused misrouting
- •Fix approach: refine tool descriptions, add clarity and examples where needed
- •Broader lesson: iterate with evals instead of overthinking a PRD in advance
PM role debate: prototyping is a skill that amplifies core PM work
Aakash raises concerns about endlessly expanding PM responsibilities (citing Itamar Gilad’s framework). Colin argues AI prototyping isn’t a new responsibility category; it’s a skill (like Figma) that strengthens communication, stakeholder alignment, and customer discovery—when used with intent.
- •Reframe: prototyping supports core PM work (alignment, discovery), not a separate job duty
- •Comparable to earlier tools (Balsamiq/Figma) for visualizing ideas quickly
- •Risk: “vibe coding” without purpose won’t replace fundamentals like research and strategy
- •Best use: faster feedback loops and clearer communication with stakeholders/customers
Strategy mind map: benefits, who should build, and when it matters
They map benefits across personal skill-building and enterprise growth. Colin expects pods (PM/design/engineering) to own the effort, with PMs focusing on why/priority and ongoing iteration (evals, analytics, incremental shipping) to capture AI-driven demand and retention.
- •Primary enterprise benefit: growth—get in front of high-intent ChatGPT users
- •Team structure: cross-functional pod (design form factor + engineering complexity + PM prioritization)
- •PM decision criteria: test form factors, watch competitor adoption, assess retention/re-engagement value
- •Operational loop: ship small, evaluate usage, improve tool calling and outcomes
Ideas for solo builders + cross-platform bet: MCP beyond ChatGPT
Colin suggests building embedded collaborative utilities where structured UI beats pure chat (spreadsheets, presentations, whiteboards). He highlights distribution as the advantage—even if feature depth lags incumbents—and notes MCP makes apps portable to other clients (Claude, Cursor, Lovable), reducing single-platform risk.
- •Good solo ideas: structured collaborative UIs where ChatGPT can co-edit with users
- •Compete via distribution: “good enough” embedded alternatives can win adoption
- •E-commerce and content creation are natural fits (cart building, mini Canva/Figma-like flows)
- •MCP portability: potential to work across ChatGPT, Claude (in progress), and other MCP-enabled tools
Colin’s solopreneur year: experimentation, stack, and tool choices
Aakash shifts to Colin’s creator/builder journey: balancing operations with new bets, quickly dropping ideas that won’t reach excellence, and aiming for a durable software product alongside teaching. Colin shares his stack (Replit for UX exploration; VS Code + Claude Code/Codex, Neon, Render) and why he prefers VS Code over Cursor.
- •Time split: maintaining the business vs testing new commercial/interest-driven bets
- •Experiments: multiple SaaS apps, two prototyping tools, RAG exploration, podcast trial
- •Stack: VS Code + Claude Code/Codex, Git/GitHub, Neon (DB), Render (hosting), vendors like Voyage for embeddings
- •Tool philosophy: prioritize code-gen quality over IDE “bells and whistles”
Wrap-up: platform readiness, cautious optimism, and calls to action
They close with a balanced view: the form factor is promising, but adoption depends on marketplace/discovery execution by OpenAI. Aakash recaps the episode as a masterclass for PMs and encourages engagement and subscriptions.
- •Still early: few launch partners; success hinges on marketplace + discovery UX
- •Expectation: clarity within a few months whether it becomes a major platform
- •Recap: why PMs should learn now to avoid being behind as AI PMs
- •Outro: follow/subscribe, newsletter bundle promos, audience feedback invitation
