Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

Aakash GuptaAakash Gupta

How to Talk to 100,000 Customers with AI (Ex-Bolt CPO)

Greg Greiner ran product at Bolt. Now he's co-founder and CEO of Woven, and in this episode he builds an AI research agent in Claude Code that emails every customer who drops off your funnel, then adds an eval loop live so the agent can…

Greg GreinerguestAakash Guptahost
Nov 13, 202658mWatch on YouTube

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

Great conversations run long. Their best ideas don't have to.

uListen is free on iPhone, iPad, Android, Chrome, and Claude.

See how

CHAPTERS

Jump to publisher timestamps

Chapter summaries

  1. From 5–10 customer chats to AI-scale conversations

    Greg Greiner contrasts how many customers a product leader could realistically talk to in a month versus what becomes possible with AI. He frames the goal as building a scaled version of the PM that can hold real, two-way conversations and return learnings as insights.

    • •Greg says he used to be satisfied talking to a handful of customers per month, but AI enables massive scale
    • •He describes using AI as a “scaled representation” of a PM to engage customers and report back insights
    • •Aakash frames the episode as moving from transcript synthesis to proactive, automated customer research
  2. What AI research on drop-offs can reveal (and fix) in the funnel

    Greg walks through an example report showing what happens when AI reaches out to people who drop off mid-journey. He explains how the approach can produce both conversion impact and structured insight into reasons for abandonment, including which issues are easy to resolve versus deeper product problems.

    • •Greg explains reaching out to people who abandoned purchasing a kit and measuring business impact
    • •The report surfaces the most frequent concerns at a specific funnel step and ties them to outcomes
    • •He distinguishes concerns solvable via messaging (like shipping questions) from harder issues (like subscription confusion)
    • •Layering conversion outcomes on top of concern categories helps prioritize what to fix in product vs messaging
  3. How the conversations are triggered and why it’s richer than surveys

    The conversation shifts to the mechanics: AI outreach begins after a user provides contact info and then drops off. Greg describes it as an evolution of surveys because the agent can ask follow-ups, attempt to resolve concerns, and observe how customers react in real time.

    • •Greg explains outreach can start after users enter email/phone and then abandon the flow
    • •He says scale can reach tens or hundreds of thousands of conversations per month
    • •Aakash compares it to in-app surveys; Greg agrees but emphasizes follow-up questioning
    • •Greg highlights the ability to probe a customer’s initial concern and see their reaction after responses
  4. Building a DIY outreach agent in Claude Code (data + instructions)

    Greg explains how he built a quick example using existing tools rather than a dedicated product. He outlines the core ingredients: a customer data source, clear conversation instructions, and company context/policies to keep outreach personal and accurate.

    • •Greg says many teams can replicate the approach with Claude Code rather than buying a dedicated tool
    • •The agent ingests customer data (demo uses a CSV with drop-off points and usage metadata)
    • •He notes the data source could also be a CDP or data warehouse
    • •A Claude.md file provides instructions for how to approach the conversation and format emails
    • •He adds company context, policies, and escalation paths so the agent stays aligned
  5. Demo walkthrough: triggering emails and running fast back-and-forth threads

    Greg demonstrates the agent sending personalized emails to customers who dropped off in a trial and then responding quickly to replies. He emphasizes that responsiveness matters: delays reduce reply rates and conversion, making AI useful for maintaining momentum in longer-running conversations.

    • •Greg triggers the outreach skill and explains it finds customers not yet contacted and personalizes emails
    • •He shows an example thread with a user response and the agent replying within minutes
    • •Greg says quick responses increase the value of outreach; delays cause reply rates to drop sharply
    • •He ties fast, accurate responses to better extraction of insight and improved conversion outcomes
  6. Why email beats in-app chat for drop-off research (and the reply-rate range)

    Aakash challenges the channel choice, and Greg argues email is especially effective for abandoned users who are no longer in the app. He also notes operational advantages: email is easy to roll out, widely used, and can produce higher response rates than on-site chat for research campaigns.

    • •Greg says abandoned users aren’t in the app anymore, making email a better reach channel
    • •He warns in-app research prompts can distract from the user’s main goal and potentially hurt conversion
    • •Email is easy to scale because customers already use it daily
    • •Greg reports research campaigns can see roughly 20–40% response rates, often higher than on-site chat
  7. What drives 20% vs 40% responses, and how to write the first email

    Greg explains that response rates depend on campaign type, funnel stage, and how personal and relevant the outreach feels. He recommends simple, direct questions and avoiding long, survey-like emails so the conversation can unfold over multiple back-and-forth messages.

    • •Greg says later-funnel users with more intent tend to reply more than early-funnel contacts
    • •He describes an experiment for trial abandon that produced a response rate in the mid-30s
    • •Personalization and relevance in the initial message materially affect replies
    • •He recommends plain-text style with focused questions
    • •He cautions against packing too many questions into one email; let the customer steer and follow up
  8. The four lifecycle email touchpoints (including incentives at trial end)

    Greg lays out a set of key journey points where email outreach can gather insight and influence behavior. He highlights an end-of-trial outreach as an especially good moment to combine learning with incentives for sharing feedback, which can increase reply rates and potentially recover conversions.

    • •Greg identifies early sign-up drop-off (email entered, onboarding not completed) as high volume/low intent
    • •He suggests check-ins during the trial to set users up for success and assess progress
    • •He recommends reaching out to trial non-converters as a key insight and conversion opportunity
    • •Greg says incentives offered for sharing insights can significantly increase reply rates
    • •He argues incentives tend to attract users already on the fence, enabling both objection-handling and conversion
  9. Operational setup: sending from Gmail vs marketing tools, and owning a research mail system

    Aakash asks how this works in practice and who owns the sending infrastructure. Greg says a PM can prototype from a regular inbox for smaller scale, but larger scale may require partnering with marketing or using a dedicated email service, and that research emails are often sent from separate systems.

    • •Greg says he ran the demo from a standard Gmail inbox, which limits scale but enables solo testing
    • •For larger audiences, he suggests partnering with marketing to use existing tooling or setting up a dedicated email service
    • •He notes research-oriented emails are commonly sent via separate systems rather than marketing-owned flows
    • •Aakash summarizes the stack idea: triggers by lifecycle stage plus AI replies in the PM’s voice
  10. Prompting Claude Code: MCP setup, data triggers, and iterative debugging

    Aakash asks for practical scaffolding prompts and how a PM can get started. Greg describes giving Claude the context, connecting to an email-sending mechanism, pointing it at trigger data, and iterating on issues like threading and tone until it works well enough to test on real customers.

    • •Greg describes providing context to set up outreach from an inbox using a Gmail integration, and mentions alternatives
    • •He says you must decide where trigger data comes from and define the outreach criteria (example: users 10 days into trial)
    • •He notes the first version typically needs tweaking (email threading, inappropriate language)
    • •Greg says basic instructions can yield a working prototype quickly, then improved through tests
  11. Adding an eval loop for groundedness before auto-sending replies

    Aakash raises the need for online evaluation to ensure responses are grounded in real documentation and aligned to brand and objection-handling. Greg shows how to add an eval step after classification for auto-send replies, and how to regenerate or draft for review when the eval fails.

    • •Aakash highlights the need to measure groundedness, brand voice, and alignment to real sources before customer-facing sends
    • •Greg proposes an eval step that runs before an auto-sent reply is sent
    • •He demonstrates an instruction: evaluate groundedness in documentation; if it fails, rerun generation and save a draft for review
    • •Greg emphasizes building a loop so failures don’t just halt automation
    • •He notes email tolerates longer response times, making eval loops more feasible than real-time chat
  12. What Woven changes: scale, training via simulations, and guardrails that rerun until passing

    Aakash asks what happens if someone doesn’t want to build this themselves. Greg describes how Woven operationalizes the same core workflow at higher scale, adds simulated examples for feedback-driven improvement, and applies global and custom guardrails with reruns until a response passes.

    • •Greg says Woven manages complexity and supports greater scale and more conversion-oriented campaigns
    • •Outreach can be driven by integrations with marketing automation platforms or a CDP
    • •He describes using simulated examples to gather feedback and improve responses without manual prompt back-and-forth
    • •Greg says Woven provides global guardrails (including groundedness and formatting) plus custom guardrails from customer feedback
    • •He describes rerunning responses until they pass guardrails and notes teams can set up new engagement points quickly once practiced
  13. When AI outreach isn’t the right fit: early PMF and high-touch relationship building

    Greg explains cases where PMs should keep direct, human-led conversations central. He points to lower-scale contexts, earlier product-market-fit stages, and situations where deeper relationship building requires more context than an agent can have.

    • •Greg says the choice depends on scale and where you are in the product-market-fit journey
    • •He argues low-scale environments benefit from direct conversations and deeper relationship-building
    • •He notes non-text conversations can be important for richer understanding
    • •Greg says AI outreach fits better later, when value is established and the goal is to maximize and scale learning and conversion
  14. The broader AI research agenda: combining automation with human discovery

    Aakash zooms out to a broader research strategy beyond email, and Greg describes scaling interviews as well. They agree AI workflows should amplify—not replace—human-to-human research, by making direct conversations more informed and focusing time on what matters most.

    • •Greg positions email outreach as part of a broader AI research strategy with a steady drumbeat of insight
    • •He points to tools that automate voice-based research conversations and suggests using email as an entry point to find high-signal users
    • •Aakash outlines a mix: regular human customer interactions plus AI-enabled programs and analysis of customer communication
    • •Greg agrees automation makes human conversations more powerful by reducing time spent on basics and increasing preparation
    • •He describes distributing research outputs via reports and Slack to keep insights where teams work
  15. AI and the future PM role: fewer engineers per pod, less process, more cohesion at speed

    Greg shares how AI changes product operations and team dynamics, arguing that output can increase dramatically with smaller teams. He emphasizes that PMs must resist process-heavy busywork, keep product cohesive amid rapid shipping, and manage broader responsibilities as output accelerates.

    • •Greg reflects that PM-to-engineer ratios can be too high and shares experience managing a large engineer surface area
    • •He argues small engineering pods can produce what much larger teams used to produce, changing how PMs must operate
    • •Greg says PMs need to context-switch quickly and fill gaps across roles to keep output cohesive
    • •He identifies meetings, approvals, and stakeholder-heavy hoops as common traps and says PMs must push back and automate prep where possible
    • •He warns against shipping everything requested without coherence, even when speed is high
  16. Internal AI vs customer-facing AI: quality bar, sandbox testing, and handling model changes

    Aakash asks how building for internal use differs from deploying AI directly to customers. Greg explains the higher quality demands for customer-facing automation and the need for rigorous guardrails, sample datasets, sandbox tests, and ongoing QA—especially when updating models frequently.

    • •Greg says internal tools can tolerate lower quality because a human reviews outputs before they’re public
    • •Customer-facing AI requires stronger guardrails and ongoing measurement against them
    • •He describes using trusted sample data and running generations outside production to catch regressions
    • •Greg says even a model version bump needs a more rigorous process when outputs go directly to customers
    • •He notes frequent iteration requires scalable, confident testing workflows
  17. AI product skills: probabilistic testing, action-permission checks, and learning from traces

    Greg explains that fundamentals remain, but testing changes when outputs are probabilistic. He describes combining deterministic tests with LLM-judge evals for complex criteria like groundedness and permissioning, and emphasizes analyzing traces both when the system fails and when it surprisingly succeeds.

    • •Greg says the fundamentals don’t change, but guardrails and evals become critical with probabilistic systems
    • •He contrasts deterministic tests with LLM-as-judge evals for nuanced checks like groundedness
    • •He highlights the need to prevent the AI from taking actions it wasn’t permitted to take by linking actions to explicit instructions
    • •Aakash underscores examining traces; Greg agrees and extends it to studying successful traces too
    • •Greg suggests learning from rare successful conversions in difficult topics to improve the approach
  18. Inside an AI-native startup workflow: debate first, then transcripts to PRDs and tasks

    Aakash asks what an AI-native startup looks like internally. Greg describes a process where the team spends more time debating what to build, captures discussions via transcripts and diagrams, and then uses AI to turn those artifacts into PRDs and task plans, reducing time spent on manual documentation.

    • •Greg says building quickly is easier; the hardest part remains building the right thing
    • •He describes small-team discussions to define the problem, challenges, and desired experience
    • •They capture a transcript and a diagram (whiteboard or similar) as the core artifact
    • •An AI skill turns those artifacts into a detailed PRD and creates project tasks
    • •Greg says this shifts time from writing documents to substantive product debates and decision-making
  19. Adapting AI-native practices to larger companies and existing process requirements

    Aakash asks what can be adopted in bigger orgs with established standards. Greg says teams can keep required outputs like standardized PRDs while changing how they get there—using smaller decision-making groups, recorded discussions, and automation to produce the expected formats more efficiently.

    • •Greg says large companies can still adopt much of the workflow even if output formats are fixed
    • •He suggests using a subset of key contributors for open-ended definition conversations
    • •Automation can convert discussion artifacts into the standardized documents needed for approvals
    • •He says adoption depends on expected outputs and having the right people for product-oriented discussions

Publisher timestamps

  1. 0:00Intro
  2. 1:36What AI research on every drop-off can tell you
  3. 6:32How to build your own research agent in Claude Code
  4. 7:00Ads
  5. 8:52Demo: an agent that emails customers who drop off
  6. 12:29Why email beats in-app chat, and 20% to 40% reply rates
  7. 16:03The four emails to send across the customer journey
  8. 24:15Adding an eval loop to the agent, live
  9. 31:23What Woven adds, and when to talk to customers yourself
  10. 35:46The full AI research agenda for a PM
  11. 40:28Why one PM can now run a team that ships like 25 engineers
  12. 45:45AI for your team vs AI for your customers
  13. 52:03Inside an AI-native startup, from whiteboard to PRD

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.