Aakash Gupta

You'll be left Behind as an AI PM If You Don't Use ChatGPT Apps

Colin Matthews is back with the definitive guide to ChatGPT apps. MCP protocol explained, live app building demo, and eval strategies. Plus: why every PM should learn this new distribution channel. Full Writeup: https://www.news.aakashg.com/p/colin-matthews-podcast Transcript: https://www.aakashg.com/chatgpt-apps-guide-colin-matthews/ Chippy: https://chippy.build/ ---- Timestamps: 0:00 - Intro 3:09 - What Are ChatGPT Apps? 8:25 - Architecture & How They're Built 10:32 - Ads 11:24 - Building First App 19:52 - Live Demo: Healthcare App 30:18 - Ads 33:12 - Improving with Evals 40:19 - PM Role & Prototyping Debate 52:01 - Ideas for Solo Builders 54:38 - Colin's Solopreneur Year 1:01:26 - Outro ---- 🏆 Thanks to our sponsors: 1. Colin's ChatGPT Apps Course: Next cohort starts February 13th on Maven - https://bit.ly/4qd2ryx 2. Vanta: Automate compliance, Get $1,000 with my link: https://www.vanta.com/lp/demo-1k?utm_campaign=1k_offer&utm_source=product-growth&utm_medium=podcast 3. Land PM Job: 12-week experience to master getting a PM job - https://www.landpmjob.com/ 4. Naya One: Accelerate AI adoption in financial services - https://nayaone.com/ 5. Mobbin: The world's largest mobile & web design library - Get 20% off: https://mobbin.com/?via=aakash ---- Key Takeaways: 1. ChatGPT apps = MCP + widgets - The Model Context Protocol (invented by Anthropic) lets AI agents call external tools. OpenAI added UI widgets on top to create embedded app experiences directly in chat. 2. 900M weekly active users = massive distribution opportunity - This is the new SEO. Early data shows 26% higher conversion from AI traffic vs traditional search. Every enterprise will eventually build here. 3. You're building for multiple platforms - MCP works across ChatGPT, Claude (coming soon), Cursor, and other AI tools. Build once, distribute everywhere. Gemini doesn't support it yet. 4. Apps get called based on tool descriptions - Your metadata matters. Like SEO but for LLMs. Run evals to test if correct prompts trigger your tools. Iterate on descriptions to improve discovery. 5. Three eval categories: direct, indirect, negative - Direct: user names your app. Indirect: user describes outcome. Negative: irrelevant request shouldn't trigger your tool. Test all three systematically. 6. PMs should prototype but engineers ship production - Use tools like Chippy to prototype quickly and test concepts. Show stakeholders real interactions. Engineering team builds the production version. 7. Enterprise-first, solo builders second - Large companies (Target, Uber, Canva) are early adopters chasing distribution. But huge opportunity for indie builders once public marketplace launches. 8. Best opportunities: embedded collaboration tools - Spreadsheets, task lists, whiteboards where ChatGPT can partner with you. Not just search results—actual interactive experiences. 9. Error analysis on observability logs is critical - Track what prompts triggered which tools with what parameters. Look for mismatches between expected and actual behavior. Iterate tool descriptions. 10. Marketplace launching by end of 2024/early 2025 - Currently only launch partners can publish. Public marketplace coming soon means anyone can ship apps and reach ChatGPT's massive user base. ---- 👨‍💻 Where to find Colin Matthews: LinkedIn: https://www.linkedin.com/in/colinmatthews-pm/?originalSubdomain=ca Newsletter: https://blog.techforproduct.com/ 👨‍💻 Where to find Aakash: Twitter: https://www.x.com/aakashg0 LinkedIn: https://www.linkedin.com/in/aagupta/ Newsletter: https://www.news.aakashg.com #chatgptapps #aipm ---- 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 200K+ listeners. 🔔 Subscribe and turn on notifications to get more videos like this.

CMColin MatthewsguestAGAakash Guptahost

Jan 22, 2026·1h 2m·Watch on YouTube ↗

📖 CHAPTERS

1. 1

   Why ChatGPT apps are a major opportunity for product builders

   Aakash frames the episode as a practical breakdown of what the emerging “ChatGPT App Store” means for builders, beyond the headline demos. Colin previews why embedded apps inside chat can create new distribution and product surface areas for teams and solo builders.

2. 2

   What “ChatGPT apps” are (and how the App Store concept works)

   Colin defines ChatGPT apps as branded, designed interactive experiences that run inside a ChatGPT conversation—more than text outputs or web results. They let companies control user experience, guide interactions, and sometimes hand users off to complete actions in the company’s own product.

3. 3

   Discovery, intent, and why enterprises care (Target/Expedia examples)

   They unpack why discovery—even if currently ‘hidden’—can become powerful once ChatGPT starts surfacing relevant apps automatically. Colin explains why AI-referred users can convert better and why enterprises want a deterministic way to appear inside ChatGPT rather than playing “Whac-A-Mole” with SEO.

4. 4

   What regular builders can ship: utilities and micro-apps inside chat

   Colin predicts a familiar curve: big brand apps plus small utility apps (like early iOS “flashlight apps”). He outlines lightweight but sticky ideas—spreadsheets, to-do lists pinned to the top, and other tools that create repeat usage within ChatGPT.

5. 5

   Architecture deep dive: MCP + tool calling + UI widgets

   Colin explains the underlying mechanism: Model Context Protocol (MCP) lets an AI client discover and call external tools. OpenAI’s UI “widgets” layer (now being incorporated into MCP) adds the ability to render interactive interfaces inside chat instead of only returning text/data.

6. 6

   Building a ChatGPT app: the easy path (Chippy) vs the hard path (DIY)

   Colin contrasts building with his platform (Chippy) versus spinning up your own MCP server and UI bundling pipeline. Chippy is positioned as an iteration-friendly environment that previews the chat experience without repeatedly rebundling and reconnecting inside ChatGPT.

7. 7

   Live setup: connecting an MCP app and ways users invoke it

   They walk through connecting an app via a generated MCP URL and enabling it in ChatGPT settings. Colin explains three invocation modes: typing the app name, manually selecting/tagging it, or relying on ChatGPT to choose it automatically for relevant prompts.

8. 8

   Live build: a healthcare reviews manager app (tools + UI)

   Aakash chooses a healthcare domain and they build a hospital reviews app with three tools: view reviews, share reviews, and review analytics. They discuss PM value: using these apps as prototypes/specs and learning the full AI product loop quickly.

9. 9

   Observability and evals: measuring whether the right tool gets called

   Colin demonstrates logs/observability showing the user prompt, tool called, and parameters selected by ChatGPT. He introduces OpenAI’s eval categories—direct, indirect, and negative—to systematically test discoverability and prevent irrelevant tool calls.

10. 10

    Improving performance with eval feedback: fix tool metadata/descriptions

    They run an “auto eval” and see a mismatch: the prompt “I want to share a review” should trigger the share tool but calls the view tool due to ambiguous wording. Colin shows how small edits to tool descriptions (metadata) can improve routing behavior and recommends iterating via tests rather than guessing.

11. 11

    PM role debate: prototyping is a skill that amplifies core PM work

    Aakash raises concerns about endlessly expanding PM responsibilities (citing Itamar Gilad’s framework). Colin argues AI prototyping isn’t a new responsibility category; it’s a skill (like Figma) that strengthens communication, stakeholder alignment, and customer discovery—when used with intent.

12. 12

    Strategy mind map: benefits, who should build, and when it matters

    They map benefits across personal skill-building and enterprise growth. Colin expects pods (PM/design/engineering) to own the effort, with PMs focusing on why/priority and ongoing iteration (evals, analytics, incremental shipping) to capture AI-driven demand and retention.

13. 13

    Ideas for solo builders + cross-platform bet: MCP beyond ChatGPT

    Colin suggests building embedded collaborative utilities where structured UI beats pure chat (spreadsheets, presentations, whiteboards). He highlights distribution as the advantage—even if feature depth lags incumbents—and notes MCP makes apps portable to other clients (Claude, Cursor, Lovable), reducing single-platform risk.

14. 14

    Colin’s solopreneur year: experimentation, stack, and tool choices

    Aakash shifts to Colin’s creator/builder journey: balancing operations with new bets, quickly dropping ideas that won’t reach excellence, and aiming for a durable software product alongside teaching. Colin shares his stack (Replit for UX exploration; VS Code + Claude Code/Codex, Neon, Render) and why he prefers VS Code over Cursor.

15. 15

    Wrap-up: platform readiness, cautious optimism, and calls to action

    They close with a balanced view: the form factor is promising, but adoption depends on marketplace/discovery execution by OpenAI. Aakash recaps the episode as a masterclass for PMs and encourages engagement and subscriptions.