Aakash GuptaYou'll be left Behind as an AI PM If You Don't Use ChatGPT Apps
Aakash Gupta and Colin Matthews on chatGPT apps: MCP-powered embedded tools PMs must learn now.
In this episode of Aakash Gupta, featuring Colin Matthews and Aakash Gupta, You'll be left Behind as an AI PM If You Don't Use ChatGPT Apps explores chatGPT apps: MCP-powered embedded tools PMs must learn now ChatGPT apps embed interactive, branded mini-app UIs directly inside a chat, giving companies more deterministic control than relying on AI web search links alone.
At a glance
WHAT IT’S REALLY ABOUT
ChatGPT apps: MCP-powered embedded tools PMs must learn now
- ChatGPT apps embed interactive, branded mini-app UIs directly inside a chat, giving companies more deterministic control than relying on AI web search links alone.
- The underlying architecture uses MCP (Model Context Protocol) for tool calling plus UI “widgets,” enabling ChatGPT to fetch tool schemas, invoke functions with parameters, and render app interfaces in-chat.
- Colin demonstrates building a healthcare reviews manager app rapidly with his tool Chippy, then connecting it to ChatGPT via an MCP URL to test real tool invocation behavior.
- The episode emphasizes evaluation workflows—direct, indirect, and negative trigger tests—to improve discoverability and correctness by iterating on tool descriptions and metadata.
- They argue AI prototyping is a complementary PM skill (like basic Figma fluency) that accelerates stakeholder alignment and learning rather than replacing core PM responsibilities.
IDEAS WORTH REMEMBERING
7 ideasChatGPT apps are a new distribution channel with higher-intent traffic.
They can be surfaced inline during relevant user queries, and companies can deliver branded experiences that may convert better than traditional SEO traffic.
MCP is the core “USB-C” connector for agent-to-tool interoperability.
ChatGPT (or other agents) requests available tool definitions, calls the appropriate tool with parameters, and uses returned data/UI instructions to render an embedded experience.
UI widgets turn tool-calling from text output into interactive software.
Instead of only returning raw data, apps can return a widget/UI reference so users interact visually (maps, lists, dashboards) inside chat.
Prototyping speed matters because iteration friction kills learning.
Colin built Chippy to avoid the slow loop of rebundling UI, redeploying, reconnecting, and retesting inside ChatGPT for every small change.
Tool descriptions are product-critical metadata, not documentation fluff.
ChatGPT’s tool selection can hinge on wording; a single word like “sharing” in a “view reviews” description caused the wrong tool to be invoked.
Evals for tool triggering are mandatory to achieve reliable discoverability.
Using direct/indirect/negative eval sets helps ensure correct tool calls, prevents irrelevant activation, and provides a repeatable optimization loop.
PMs shouldn’t treat ‘vibe coding’ as an extra responsibility bucket.
AI prototyping is framed as a skill that supports core PM work—alignment, customer conversations, and faster validation—similar to lightweight Figma usage.
WORDS WORTH SAVING
5 quotesChatGPT apps are basically a way for companies to bring in their own designs... directly into ChatGPT.
— Colin Matthews
Underlying ChatGPT apps is this protocol called MCP, or Model Context Protocol.
— Colin Matthews
This is a really underrated way to get more distribution.
— Colin Matthews
Theoretically… if I say something alluding to it… ChatGPT may decide to use my app.
— Colin Matthews
You can’t really necessarily predict how ChatGPT is going to interpret the way that you wrote your tool description… you should just test it and run evals.
— Colin Matthews
QUESTIONS ANSWERED IN THIS EPISODE
5 questionsIn practice, what signals does ChatGPT use to auto-surface an app (without naming/tagging it), and how much can builders influence that beyond tool descriptions?
ChatGPT apps embed interactive, branded mini-app UIs directly inside a chat, giving companies more deterministic control than relying on AI web search links alone.
What are the most effective patterns for writing tool descriptions (examples, constraints, few-shot usage) to reduce misfires like “share” triggering “view”?
The underlying architecture uses MCP (Model Context Protocol) for tool calling plus UI “widgets,” enabling ChatGPT to fetch tool schemas, invoke functions with parameters, and render app interfaces in-chat.
How should a team design negative evals to prevent accidental activation when many tools exist—what’s a good coverage strategy?
Colin demonstrates building a healthcare reviews manager app rapidly with his tool Chippy, then connecting it to ChatGPT via an MCP URL to test real tool invocation behavior.
Where do you draw the line between what stays inside the embedded UI versus what should “pop out” to the company’s native app (like Target’s cart-to-checkout flow)?
The episode emphasizes evaluation workflows—direct, indirect, and negative trigger tests—to improve discoverability and correctness by iterating on tool descriptions and metadata.
What are the current technical gotchas with authentication for ChatGPT apps, and what secure defaults should enterprises adopt?
They argue AI prototyping is a complementary PM skill (like basic Figma fluency) that accelerates stakeholder alignment and learning rather than replacing core PM responsibilities.
EVERY SPOKEN WORD
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome