Skip to content
Aakash GuptaAakash Gupta

The #1 Skill PMs Need in 2025: AI Product Discovery Masterclass by World’s Leading Authority

What happens to product discovery when AI can generate prototypes in minutes, synthesize interviews in seconds, and give you feedback before your coffee gets cold? Does it make discovery obsolete… or more important than ever? In this episode with Teresa Torres, legendary author of Continuous Discovery Habits, who has trained over 17,000 PMs across 100 countries. She pulls back the curtain on: - Why most customer interviews fail and how to fix them with story-based interviewing - The real difference between testing your idea and testing your assumptions - How to keep your Opportunity Solution Tree alive and evolving - The five skills every PM needs to build AI features that actually work If you’ve ever wanted to master continuous discovery and AI product development without drowning in fluff or hype… then this podcast is for you. Transcript: https://www.news.aakashg.com/p/teresa-torres-podcast Timestamps: Teresa's Background - 0:00 Story-Based Interviewing - 3:20 Fake Discovery Signs - 4:08 Assumption Testing - 4:39 Continuous Discovery Framework - 5:35 AI Changes Discovery - 8:01 AI Synthesis Concerns - 9:21 AI Prototyping Era - 12:45 Ads - 15:45 AI Prototyping Workflow - 17:32 Common Interview Mistakes - 22:24 Interview Synthesis - 24:26 OST Updates - 28:53 Discovery Theater - 30:52 Ads - 32:15 Real Product Management - 34:03 AI Product Discovery - 35:29 Context Engineering - 39:16 Orchestration Explained - 42:03 Error Analysis - 46:01 Observability & Traces - 46:05 Claude Code Demo - 49:15 Business Numbers - 52:56 Thanks to our sponsors: 1. Miro: The innovation workspace is your team’s new canvas - https://miro.com/innovation-workspace/?irclickid=VCiVcr1RbxycTNSy1219xzQHUkpxGiT7VWmDzE0&utm_source=Test%20partner%20account%20miro&utm_medium=cpa&utm_campaign=&utm_affiliate_network=impact&utm_custom=Aakash&irgwc=1 2. Jira Product Discovery: Build the right thing - https://www.atlassian.com/software/jira/product-discovery 3. Parlance Labs: Practical consulting that improves your AI - https://parlance-labs.com 4. Product Faculty's #1 AI PM Certification with OpenAI's Product Lead (get $500 off) - https://maven.com/product-faculty/ai-product-management-certification?promoCode=AAKASH25 Takeaways: 1. If nothing in your backlog changes and you never kill ideas, you're doing fake discovery. Real discovery should constantly reshape your product direction. 2. Stop asking "would you use this?" Instead ask "tell me about the last time you solved this problem" to get reliable, actionable insights. 3. When delivery becomes free through AI, discovery becomes MORE important to avoid overwhelming customers with incoherent features. 4. Break your ideas into underlying assumptions and test those individually rather than building full prototypes first. 5. AI can handle 60-80% of interview synthesis, but you lose critical context and differentiated insights in that missing 20-40%. 6. Building AI products is like teaching humans - you need the right context at the right time, not everything at once. 7. AI product discovery is heavily focused on observing traces, identifying error patterns, and iterating on prompts and orchestration. 8. Weekly customer interviews should load your brain with user context, making you a better human LLM for product decisions. 9. Map customer stories to opportunity spaces and update your OST every 3-4 interviews to keep discovery actionable. 10. Teresa rewrote her entire AI interview coach evaluation system in one week using Claude Code without writing a single line herself.RetryClaude can make mistakes. Please double-check responses. 👨‍💻 Where to find Teressa: LinkedIn: https://www.linkedin.com/in/teresatorres/ X (Twitter): https://x.com/ttorres Website: https://www.producttalk.org/?srsltid=AfmBOopiWRDhn3IXM55mP320CUnE6THriNiviDHcZvk1ToAYXp6c3FDj Courses & Mentorship: https://learn.producttalk.org/? Book: Continuous Discovery Habits: https://www.amazon.com/Continuous-Discovery-Habits-Discover-Products/dp/1736633309 👨‍💻 Where to find Aakash: Twitter: https://www.twitter.com/aakashg0 LinkedIn: https://www.linkedin.com/in/aagupta/ Instagram: https://www.instagram.com/aakashg0/ #ai #productdiscovery 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 180K listeners. Hosted by Aakash Gupta, who spent 16 years in PM, rising to VP of product, this 2x/ week show covers product and growth topics in depth. 🔔 Subscribe and like the video to support our content! And turn on the bell for notifications.

Aakash GuptahostTeresa Torresguest
Aug 12, 202556mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:002:10

    Discovery in the age of AI: why it matters more when “delivery is free”

    1. AG

      What does discovery look like in the age of AI? Does it change everything or does it change nothing?

    2. TT

      You know, I've been getting asked a lot, like when delivery is free, do we still need to do discovery? And I actually think when delivery is free, discovery becomes more important.

    3. AG

      In today's episode, I sat down with Teresa Torres, the legendary author of Continuous Discovery Habits. This is a book that I myself have read multiple times, marked up multiple times. So many PMs are doing customer interviews, yet their products and their features fail. Why? She has worked with over 17,000 PMs across the world in over 100 countries, so she brings the insight you need to improve your discovery, not just for regular features with AI, but for AI features.

    4. TT

      Here's the challenge I see with prompt engineering. We all have experience like chatting with ChatGPT or Claude, and we're in a conversation. If we get that first prompt wrong, we can immediately refine. But when you're building a product, the prompt can't be refined by you. Once it's live in your product, there's no refinement. It's a one shot.

    5. AG

      So what are the signs that PMs are doing fake discovery?

    6. TT

      Nothing in their backlog changes. They don't kill any ideas. There's a lot of discovery theater out there.

    7. AG

      Before you go, Teresa, I have to ask, how big is the business of Teresa Torres?

    8. TT

      [laughs] Yeah, so I'm a [beep]

    9. AG

      Teresa, welcome to the podcast.

    10. TT

      Thanks for having me. I'm excited to do this.

    11. AG

      As I was saying off air, you are on my S tier of guests along with Marty Cagan. You are the two guests I wanted most when I dreamed of starting this podcast, and that's because I think you have probably advised more PMs on discovery than anyone else in the world. What would you say the number is at now?

    12. TT

      Yeah, it kind of depends on how we count. So when I was coaching teams directly, I would work with about 30 teams a year. I did that for over a decade, so probably over 300 teams. And what that means is like weekly calls for, for multiple months. So that was sort of in depth. Through the Product Talk Academy, we have over 17,000 students, which is pretty mind-blowing, and they come from over 100 countries.

    13. AG

      Wow. 100 countries.

    14. TT

      Yeah.

  2. 2:103:16

    Why customer interviews fail: shifting from solution feedback to story-based learning

    1. AG

      I didn't even know PM was practiced in 100 countries. So you have really seen the whole world of product discovery. And it's interesting because so many PMs are doing customer interviews, yet their products and their features fail. Why?

    2. TT

      Yeah, this is a complicated topic. I think there's a lot of reasons for this. I think the primary reason is that we're not that good at interviewing. So a lot of teams, they go into interviews with the intent of exploring their solution and getting feedback on their solution. That's not really the best way to get feedback on our solutions. Our goal in our customer interviews should be to learn about our customers. Um, even if we know that's the goal of the interviews and we don't talk about our solutions, we tend to ask really unreliable questions like, "What do you like and dislike about different things?" "Tell me about your experience broadly." And so one of the things that I teach, I introduce this idea in the book, we teach it through all of our programs, is this idea of story-based interviewing. So how do I talk to you and collect a reliable story about your past experience? So I learn about what you actually do and not what you think you do, not what you aspire to do, but, like, in reality, what did you do recently so that I can make sure that I'm building a product that fits in your, in your lived world.

  3. 3:165:27

    Better questions, better signal: practical story prompts + assumption testing overview

    1. AG

      So it sounds like people are asking too many hypothetical questions. Here's a prototype, would you like to use this? What would be the better way for them to approach that conversation?

    2. TT

      Yeah, so, like, let's look at this in tiers. So the first is a lot of people present a solution and say, "Would you use this?" That's terrible.

    3. AG

      [laughs]

    4. TT

      Unreliable feedback. We're not good at predicting our future behavior. Also, humans wanna be nice, so we're gonna say, "Yeah, of course I would use that." And even if we're, we think we're being honest, it's actually, uh, we're optimistic about our future time and about what we might do in the future, so we might actually genuinely think we're gonna use it, but it doesn't mean we are. There's actually better ways to test our solutions. So I really like assumption testing when we're evaluating solutions, which is a whole different activity from interviewing. We can get into that if you want. What I like to use interviews for is, let me just learn about you. And so what I wanna do, sort of the next level, like, people learn, okay, I should ask an open-ended question. So they'll be like, "Tell me about your experience with my product." The challenge with that type of question, it is open-ended. I might learn a lot about you, but what I'm gonna learn is what you think you do, not necessarily what you actually do. And so to fix that, I wanna ask you, tell me about, about the last time you used the product or, even better, tell me about the last time you solved the problem the product was designed to solve.

    5. AG

      That is exactly what I've learned through hard trial and error and reading and rereading [laughs] this book. It actually works, folks. And you briefly mentioned assumption testing. Can you give us the 30-second overview of what that is?

    6. TT

      Yeah, so it's this idea of we tend to fall into the trap of big idea testing, which means we have to do all the design up front and then usability test it or we have to build it and A/B test it. The challenge with those strategies, they're great to have in our toolbox. We want to do them eventually. But when we're doing them in discovery, we're learning after we did all the work if the idea would work or not. I prefer to learn if something's gonna work or not before I do all the work. And so the key to that is to break the idea down into its underlying assumptions, so what needs to be true in order for this idea to work, and then to test those particulars. Um, and we te- we can tend to test assumptions much quicker. We don't have to do all the design work. We certainly don't have to do all the engineering work. And we can start to collect data on whether, whether an idea would work or not before we do all the work.

  4. 5:277:50

    Continuous Discovery Habits: outcomes, Opportunity Solution Trees, and compare-and-contrast decisions

    1. AG

      Okay. And I think this comes together, although you can correct me if I'm wrong, in the Continuous Discovery Habits system. Can you break down what exactly is continuous discovery?

    2. TT

      Yeah. So this framework I developed to help newly empowered product teams figure out what in the world to do. So let's, let's talk about that for a second. I think historically, product teams have been asked to deliver specific features, usually by specific dates. We ta-

    3. AG

      [laughs]

    4. TT

      We call these roadmaps. Sometimes teams are creating those roadmaps themselves and they've had to deal a little bit with wh-What do I put on my roadmap? But for a lot of product managers, their stakeholders are putting things on their roadmap, and their job is literally to just deliver. And so what I saw happen is companies started to shift from a output focus to a outcome focus. So now they're saying, "Okay, teams, we get it. The future's uncertain. We don't know what you should build, but you need to reduce churn, or you need to increase retention, or you need to drive engagement." And these teams are like, "What? You've always told me what to build. I don't know how to do this." And so the Continuous Discovery habits are about how do we answer these evergreen, wide open challenges like engagement and retention, or even customer acquisition. And so it starts with having a clear idea of what your outcome is. That's typically a metric. Can be something like increase engagement. Um, usually it's more specific than that. We're defining the types of engagement we want. But that's good enough for now. And then the habit on the left here is we're interviewing, and we're interviewing week over week with the goal of understanding our customers. So we're not testing our solutions. We're not evaluating our solutions in our interviews. We're really trying to understand who are we building for. We're setting up a good building with cadence. And then the visual in the middle, I should have started there, is an Opportunity Solution Tree. This is a visual that I designed to help people keep track of their messy discovery work. So it starts with an outcome at the top. We're interviewing to uncover the opportunity space. Opportunities are unmet customer needs, pain points, and desires. So as we interview, as we learn about what people did in the past, we're gonna uncover friction. We're gonna capture all that in the opportunity space. Eventually, hopefully not after too long, we're gonna choose a target opportunity. I really am a advocate of compare and contrast decisions when evaluating solutions. So we're looking at multiple solutions for the same target opportunity, and then we're using assumption testing to break those solutions down into their underlying assumptions and then assumption testing to evaluate which one looks like a winner.

  5. 7:5012:36

    AI’s impact on discovery workflows: thought partner, interviewing, and synthesis tradeoffs

    1. AG

      So this diagram is timeless. I still refer people back to the core Continuous Discovery loop. But how has AI changed Continuous Discovery?

    2. TT

      I think it depends on who you ask, and I really am s- conflicted about this. So let's talk about two different paths of how AI impacts this. There's the path of how does AI affect how I do my day-to-day job, and then there's the path of how does my job change when I'm building AI products. So in the first path, I love AI as a thought partner. So, like, even if I'm defining outcomes, I might be chatting with Claude or ChatGPT about my outcome and how I might better frame it or how I could better measure it. It's probably not telling me what my outcome is because it needs a lot of business context for that. I mean, you know, now people set up projects with all their business context, and maybe it could help with that. But typically, our outcomes are coming from our executives, and so maybe it's we're using it to refine that activity. I do know some teams that are starting to experiment with having AI interview their customers for you. I actually think this ... We live in a world where this is very possible. I'm a little bit concerned about what it says to our customers, like, we really wanna learn about you, but not enough to spend our own time doing it. I also have some concern about, like, one of the benefits of interviewing beyond what we learn from our customers. Just this act of having firsthand exposure to our customers helps us build empathy for them.

    3. AG

      Yeah.

    4. TT

      It helps us see the gap in how we think and how they think, and I don't think, like, reading a transcript is gonna get you that benefit. So I'm still a big advocate of humans talking to humans. One area where I'm really torn on is on synthesis. So I know a lot of teams are starting to use AI for synthesis. Um, here's what I like about this. I know a lot of teams that do no synthesis. They conduct interviews, the notes go into a folder, they never look at them again. If that's what you're doing, I think AI can help. My caution is I've experimented a lot with, like, my synthesis ver- versus, like, Claude's synthesis or ChatGPT's synthesis, and it misses a lot. It's r- you have to work really hard to get it specific to help it really identify opportunities. I've been running these experiments. I will probably release tools in this space eventually. It's, like, 60 to 80% good, and I worry about what we lose in that 20 to 40%. I also worry about what we lose when humans aren't in the data. I think it's r- I think our brains change when we spend that much time in our data.

    5. AG

      Mm-hmm.

    6. TT

      So I'm not gonna say never on that synthesis part. In fact, the way I'm starting to think about it is for everyd- ever- green ch- like, our everyday mundane things that we work on, maybe we don't need to go deep, and AI synthesis is more than adequate. For our differentiators, for the things that really differentiate us from our competitors in the market, I'm probably gonna go deep and still do my human synthesis. And then I know David Bland is experimenting a ton with AI in helping you identify assumptions and even helping you identify assumption tests, and I think there's a ton of potential there, too. I just ... What I fall back on is I'm a big believer in AI augmenting human work, not replacing human work. And so I really caution people that when they start to use AI in discovery to really think about how does it accelerate but not replace what you're doing.

    7. AG

      Yeah, that's what I keep advising people. Everybody keeps asking me, "Aakash, should I be using AI to synthesize and report out on my discovery?" And I feel like what it's doing is it's preventing you from improving your own brain as an LLM, right? We keep wanting to load our brain up with the right context, and the magic of Continuous Discovery was that you were loading it up with this beautiful context from your users, of actually talking to their us- your users. And if you're outsourcing all of that, especially the interviewing part, you might lose some of that context building.

    8. TT

      Yeah. I know that, like, especially interview synthesis is so time-consuming, and it takes a lot of work. I'm actually really interested in this as a problem space of, like, how can AI speed up that work? What I'm wary of is using AI to completely automate that work. And it ... There's a good reason for this, right? Like, we all have access to the same AI tools. Like, what's eventually gonna be your differentiator if we're just all outsourcing it to the same tools? Now, you could argue if you're conducting better interviews and you have better inputs, yes, that is a differentiator, but I also think there's something that we can ... that, that humans will do. Not just that humans will do it better than AI.But how humans change when we do the work. That's the part I'm most worried about losing. So what I'm interested in is, like, what are the tools that help us do it better and faster, but still enable us to get our hands dirty and to do the work?

    9. AG

      Yeah. Maybe AI can do 60 to 80% of it, but let's make sure that we then go in on top and do some of the work, do some of the context building to enhance it and also to create differentiated insights.

    10. TT

      Yeah, absolutely. So that... And that's, like, the AI, how does it affect our workflow? Do you wanna get into, like, how does discovery change when we're building AI products?

  6. 12:3617:35

    AI prototyping and the “feature factory” risk: discovery must provide discipline

    1. AG

      I wanna get there in just a second, but I wanna stay on the workflow one more second because there's one other element of AI I wanna talk about, which is AI prototyping. And some podcast guests have called it the golden age of the feature factory because executives, you know, they can just come up with a prototype, they can send it your way, and they can say, "Hey, can you please do a couple user tests and then ship this into production next week?"

    2. TT

      Yeah.

    3. AG

      And that's actually a reality for PMs I'm talking to these days. So how do you correctly pull AI prototyping into these discovery workflows?

    4. TT

      Yeah. Okay, so I'm gonna start by I am a huge fan of AI prototyping. It really feels like magic. I actually just interviewed 17 product people about how they're using Lovable at work, and we'll be publishing this blog post at, uh, on August 13th. So a huge deep dive on what are people doing at work with AI prototyping. You know, I've been getting asked a lot, like, when delivery is free, do we still need to do discovery? And I actually think when delivery is free, discovery becomes more important because if we're building anything and everything and just shoving it into our product, we are going to drive our customers nuts. There's gonna be no product coherence. They're gonna have change fatigue. We're gonna have feature bloat. And what comes to mind is have you seen the Simpsons episode where Homer got to design his own car?

    5. AG

      Oh, I have to hear this story. I haven't.

    6. TT

      Yeah. This is the best analogy I have for this. So I forget the details 'cause I saw this probably 20, 30 years ago. But Homer Simpson got to design his own car, and you can imagine it's Homer Simpson, right? So his chair is, like, a big recliner, and he's got cup holders everywhere for his beers, and he's got donut holders. And, like, it's just a ridiculous car.

    7. AG

      A7.

    8. TT

      It looks silly. It probably doesn't even drive. Like, he's just thinking about all the things that, like, he thinks matter but actually don't matter, 'cause actually getting from point A to point B is what matters most in a car. And I actually think this is the biggest, the best analogy for what might happen if we just let anybody tack on features to our product. So I think, like, what I love about AI prototyping, we can now assumption test so quickly, like, literally at the push of a button. We can do much higher fidelity assumption testing really quickly. But I don't think it changes our discovery work other than it speeds up a really important step. I think we still wanna be really considerate about what do we put in our backlog and what are we actually releasing to customers because the biggest impact of change on our cust- is on our customers. It's not on our engineers. Like, sure, we used to maybe do d- discovery to, like, save our engineers time from building the wrong thing, but the real goal of discovery was to not exhaust our customers w- with the wrong features and, and constant change.

    9. AG

      So take me through, I guess, the life cycle, right? Um, typically, we would do... We would, like, explore the problem space. We would come up with our assumptions. We would test our assumptions. Then finally, the designer might... It might be worth their time to create a prototype. Then we'd put the prototype in front of them. Nowadays, people are, are just jumping straight to the prototyping. What is the actual life cycle it should be? When should you bring the AI prototyping into the process, and how exactly should you use? Today's episode is brought to you by Miro. Let me ask you something. How many tools are you juggling just to get a single project across the finish line? One for brainstorming, another for planning, something else for tracking tickets. That's where Miro comes in. It becomes an all-in-one collaboration workspace. Whether you're consolidating user research from several interviews, developing and synthesizing product briefs or a wireframe, or project managing development, Miro brings everyone into the same space. It's fast, intuitive, and fully loaded with features like project templates, two-way Jira sync, and integration with software like draw.io and PlantUML. Miro's AI features can be used to synthesize elements in a board to develop a ready-to-review product requirements document in seconds. If you're tired of tab overload and scattered workflows, try Miro. Head to miro.com and see why over 90 million users choose Miro to guide from idea to outcome. Today's episode is brought to you by Jira Product Discovery. If you're like most product managers, you're probably in Jira tracking tickets and managing the backlog. But what about everything that happens before delivery? Jira Product Discovery helps you move your discovery, prioritization, and even roadmapping work out of spreadsheets and into a purpose-built tool designed for product teams. Capture insights, prioritize what matters, and create roadmaps you can easily tailor for any audience. And because it's built to work with Jira, everything stays connected from idea to delivery. Used by product teams at Canva, Deliveroo, and even The Economist, check out why and try it for free today at atlassian.com/product-discovery. That's A-T-L-A-S-S-I-A-N.com/product-discovery. Jira Product Discovery, build the right thing.

  7. 17:3521:19

    Where AI prototypes belong: assumption-test particulars, not whole ideas

    1. TT

      Yeah, I like it in the context of assumption testing. So what does that mean? It means I already have an outcome. I've already conducted, let's say, three to four interviews. I have a draft of my opportunity space. It's gonna continue to evolve as I keep interviewing. I've chosen a target opportunity. Ideally, I've brainstormed multiple solutions for that target opportunity, so we're setting up a good compare and contrast decision. Now, with AI prototyping, you might say, "Teresa, I can actually prototype all three in a day, so why wouldn't I just do that?"Here's the challenge with whole idea testing. Okay, great, we just made it free basically to almost free to prototype three ideas. So I've got three high fidelity interactive prototypes. Why can't I just go test those? Well, first of all, testing a whole idea with a customer takes a long time. It's a big usability test. There's like... And by a long time, it might be 20 minutes per customer, but that's a long time, and I'll explain why in a second. So we're already fatiguing our customers because we're going through 20-minute sessions. We're doing a lot of them. And then what do we get in that feedback session? We're getting, like, random bits of feedback on different parts of the idea based on what resonated or didn't resonate with that customer. It's not very structured. If instead I take my ideas and I break them down into underlying assumptions. So let's say that... I'll give you a recent example from something I'm building. I'm building an interview coach. It gives my students feedback on how they conducted an interview. When I was, like, building out this solution, I had some, like, technical challenges. So my students conduct interviews in a Zoom breakout room during class. Zoom doesn't let me record every breakout room. That's already a technical challenge. I need recordings of the breakout room. I need to go from recording to transcript, and then I need to go from transcript to the coach. So when I'm, like, testing this idea, will it work, I have a number of steps that I have to test. Will students, like, stay in the main room long enough for me to give them permission to record? When they go to their breakout room, will they remember to record? Uh, will they be able to find the recording on their computer? Will they be willing to submit it to my extract an SRT file tool? Will they then copy... Like, it's, there's a lot of friction in this process. It's kind of a nightmare. Thank you, Zoom. And so when I'm testing this idea, I don't just release it out there and hope they do all the things that they do because I don't get visibility into what step broke down, but I can test each individual assumption. So I can first, like, with one group, just be like, "Okay, we're gonna record our practice sessions today. Here's how it's gonna work." And I'm gonna see, like, did I have to go pull people out of their breakout rooms to get them back to the main room? 'Cause that's the only place I can give them permission, right? And so, like, I'm, I'm methodically testing each of these individual pieces, and so it tells me exactly where the breakdown is and exactly what I might need to iterate on. Versus if I just released a product and students didn't actually submit a transcript, I would know, have no idea where in that process it failed, especially 'cause I'm using third-party tools. I can't instrument Zoom, right? Um, so th- so what you... Even though AI prototyping, like, now makes it really easy to create a prototype, I still wanna create a prototype for a particular so I know exactly where in my process the breakdown happened. So I'm t- um, testing each particular in isolation.

    2. AG

      Okay, so to recap that back to see if I got it right, where we wanna use AI prototypes is after we already have identified the opportunity space, after we've brainstormed several solutions, and we don't wanna just prototype the whole solution. We wanna prototype a particular element of that solution in order to assumption test around that element of the solution.

    3. TT

      Exactly.

  8. 21:1922:12

    Common AI prototyping mistakes: shiny objects without a customer problem

    1. AG

      And what are people doing wrong with AI prototyping tools? What do you most often see them doing? [laughs]

    2. TT

      You know, I love... Well, the first thing is a lot of people are just starting with solution ideas. They don't have any idea what customer problem they're solving, and this is really rampant across all product management, right? A lot of us, like, we think of it as, like, shiny object syndrome. My id- my job is to have a creative idea. Um, and this, I think, is a really big misunderstanding of product management. Like, our job is to solve customer needs, and so we should always be working in the context of a customer problem, a customer pain point, a customer desire that we're working to address. And I think AI prototyping makes it even easier to be shiny object, to fall prey to shiny object syndrome, 'cause I can literally just type, like, "I had this half-baked idea," and now you can make it real. Um, and I think it takes a little bit of discipline to be like, "No, hold on. What are we doing this for?"

  9. 22:1224:18

    Interviewing skill that changes everything: excavating the story (not guessing)

    1. AG

      So staying a little bit more on these fundamentals of continuous discovery for any feature, what are the most common bad interview questions that PMs ask outside of the hypotheticals? What are the common mistakes that your interview coach or you, when you're coaching people, are fixing for them?

    2. TT

      Yeah, so let's assume we're conducting a story-based interview. So people know already to collect a story, so they're starting with something like, "Tell me about the last time you used L- Lovable." What's hard is that people aren't good at telling their stories naturally, and so people underestimate how much work goes into the skill of, we use the verb excavating the story. So I love this verb because it's a really good analogy, right? Like, typically, we excavate artifacts with, like, small hand tools and, like, gently pull out the archite- ar- the artifact from, like, a dig site. This is actually a great analogy for interviewing because in a conversation, there's this, like, 50/50 norm. I say something, you say something. I say something, you say something. In an interview, we actually wanna break that norm. We want the participant doing most of the talking, and so we have to communicate, no, we really want all the detail, right? So if I say, like, "Tell me about the last time you watched Netflix," you're gonna be like, "I watched a movie last night after dinner." And I have to actually situate you in that moment, like, oh, it was after dinner. Okay, so let's put you back in that moment when you're finishing dinner. Tell me about the decision to decide to watch something, and then walk me through the whole narrative of, like, you went from the dining room table to the living room and turned on the TV, and what device did you use? And, like, I'm gonna pull out the whole narrative. Whereas what the biggest mistake people make is they start with the story-based questionBut because they don't know how to excavate the story, they don't know how to pull out that narrative, they start guessing what happened. And so their questions are guesses of like, "Oh, did something go wrong while you were watching?" Or, "Did you look at recommendations?" Right? Instead of just using, like, temporal prompts, "What did you do first? What came next?" So there's sort of this skill of digging out a story that I think a lot of people underestimate.

  10. 24:1827:23

    Single-interview synthesis: the Interview Snapshot (experience map + opportunities)

    1. AG

      And once they're done with an interview, what is the right way to capture the insights from that particular interview?

    2. TT

      Yeah. Okay. So I wanna clarify, I teach one form of interviewing, story-based interviewing. The reason why I teach this is while it is a skill and it does require practice, the idea is simple, so we can all grok it, right? It's also, I think, the best form of interviewing for continuous interviews. For a product team that is interviewing a customer every week, it's allowing us to continuously invest in our understanding of customers. Now, the reason why I had to caveat that is when we start talking about how do we synthesize this interview, the goal of this synthesis is supporting week-over-week product work. So this is different from, like, interviews that your user research team might conduct or how they might synthesize a dozen interviews at once. So I wanna just clarify that. But in this context, if I'm on a product team, I'm doing an interview every week, what I wanna be doing is I wanna be capturing what I'm learning from each interview. So I distinguish, distinguish between single interview synthesis and then a cross-interview synthesis. So this interview snapshot that we're looking at is a, it's just a one-page template, and it's designed to help you synthesize a single interview. And we're doing a few things here. We're starting with the, the biggest thing here is at the bottom. It's actually this experience map. So if I'm collecting a story, I wanna identify what are the key moments in that story and how do I capture them in an experience map. This is gonna help me remember the story over time. It's, it's gonna allow me, at a glance, to look at this snapshot and be like, "Oh, yeah, I remember that story." It's also gonna help me find patterns across stories, and it's gonna help me give structure to the opportunity space on a opportunity solution tree. I'd say the next most important thing on this snapshot is the list of opportunities. This is what's making our interviews actionable. So what opportunities, what unmet customer needs, pain points, and desires did I hear in this interview? And I'm capturing as many as I can. And then the rest of the elements here are just helping with, like, the quick facts are sort of my customer segment data. So how do I put this specific story in the right context? Is this an engaged customer? Is it a new customer? Are they a large enterprise? Are they s- a small business? These quick facts are gonna vary business to business, but you're settling on half a dozen, maybe up to a dozen kinda quick facts that help you situate the story in a specific context. Um, the photo and the quote, again, are memory aids. So I like to pull out a salient quote that just reminds me what did I learn in this interview. This is kind of a memory trick. Like, I will tell you, I still remember interviews I did years ago because-

    3. AG

      Mm-hmm

    4. TT

      ... of the salient quote. So it's just another way to quickly look at the interview snapshot and be like, "Oh, yeah, I remember that person." Um, and then of course, the insights is just other notes that we've captured that we're not, that they're not quite actionable yet. We're not sure where they go, but we don't wanna lose them.

  11. 27:2328:40

    Cross-interview synthesis: building the OST with a “super experience map”

    1. AG

      So this is for a single interview. How does it look for the other scenario?

    2. TT

      Yeah. When we're synthesizing across interviews, this is where I like to do this in the context of my opportunity solution tree. So I'm starting with my outcome. I'm looking across my interview snapshots, which not all my opportunities for my snap go on my tree. My outcome acts as a filter. So which of these opportunities do I think have the potential to drive the outcome? And those are the opportunities that I'm moving over to my tree. And then I'm looking, I'm using the experience maps to give structure to the opportunity space. So what I teach teams is once you have three or four interviews done, you can start to look at your experience maps for the individual stories, and then what you're looking for is the experience map that encompasses all of the stories. So it's like a super experience map. And then the moments in that super experience map map to your top level opportunities. And this is really important because it guarantees that each branch in your tree is distinct because each branch represents a different moment in a story, and so the needs and the pain points that, that show up will be specific to that moment. This helps to reduce dependencies between our opportunities and allows us to work on one opportunity at a time.

  12. 28:4030:52

    Maintaining a living OST: cadence, iteration, and stakeholder communication

    1. AG

      You're speaking to a part that I think is very nuanced and is part of the skill that people are gonna develop as they do more continuous discovery, which is how to update the opportunity solution tree. So can you go in a little bit more depth for us? Like, how are we adding things here? How are we crossing them off? What is the typical cadence looking like to refine this OST?

    2. TT

      Yeah. So the first thing I'll highlight is this really is a living document. It's not a one-time activity. It does have prerequisites. You need a outcome. I recommend you start with at least three to four story-based customer interviews with snapshots for those interviews. So you've already done your experience mapping. You've already identified your opportunities. And then from there, you can start to work on your first draft of the opportunity space, and I cannot emphasize enough it is a first draft. And so again, we're finding those top level opportunities based on the key moments across our stories. We can then take all the individual opportunities we're hearing that are related to our outcome and just group them under the moment in which they occurred. So we can say for this opportunity, what moment in the story did it c- did it emerge? And then for each branch, we're gonna start to look for relationships between the opportunities. Which ones are children? Which ones are parents? We probably have to mi- uh, we probably have to add some parents to give structure to the opportunity space. Um-I, you know, let's talk about a ideal timeline. Like, if I'm starting a quarter with a new outcome, in week one I'm probably front-loading three to four interviews, so by the end of week one I have a draft of my opportunity space. That sets me up in week two. I can choose a target opportunity. I can start brainstorming solutions. I can start assumption testing in, as soon as week two. I'm gonna continue to interview every week, and every three to four interviews I'm gonna revisit my opportunity space. So if I'm working on a quarterly basis, if I'm doing one interview a week, then I'm roughly revising my opportunity space every three to four weeks.

    3. AG

      Beautiful. I don't think enough people are grokking that it's a iterative document, and it's also a communication tool for your stakeholders.

    4. TT

      Absolutely.

  13. 30:5235:23

    Discovery theater and the weekly-interviews debate: what ‘real’ discovery looks like

    1. AG

      So what are the signs that PMs are doing fake discovery?

    2. TT

      Uh, nothing in their backlog changes. They don't kill any ideas. They always end up building what they started with. They're not considering multiple solutions. I mean, this, there's a lot of discovery theater out there, which some of this is just lack of know-how. I don't think teams are like, like intentionally putting up a sh- a facade. I've actually never seen a team that's like, like cheating the system. I think it's more like there's, like, personal work we have to do to make this work. Like, we have to be able to set aside our ego a little bit. We have to distance ourselves from our favorite idea. We have to come into a interview with curiosity, especially when it's the seventh interview on the same topic. We have to recognize that each customer is unique and actively dig for what's unique about this customer. And I think plenty of product managers and e- everybody else on the product trio aren't getting the coaching they need or the support they need to do that work. Plenty of organizational contexts don't give them the space to do that. We still reward people for being right and for having strong opinions, and so we're not rewarding them for being curious and, and we're not rewarding failures that indicate we actually learned something new. So I don't put this on pr- all entirely on product managers. I think a lot of our organizational context doesn't support this way of working. But I do think individuals can make more progress than they think they can even if their organization doesn't support it.

    3. AG

      Is your AI product burning budget without results? Drowning in frameworks but can't improve your LLM performance? That's exactly why Parlance Labs exists, today's podcast partner. They're an engineer-only AI consulting firm led by Hammel Hussein. No slide decks, just practical solutions. They've helped 30-plus companies like Honeycomb, dbt Labs, and even LangChain dramatically improve their AI through systematic evaluation. Here's what makes them different: they don't want dependency. They teach your team to evaluate AI systems and optimize performance so you can fire them when ready. Ready to stop guessing and start improving your AI? Visit parlance-labs.com. That's P-A-R-L-A-N-C-E dash L-A-B-S dot com or email consulting@parlance-labs.com. Before we dive deeper, let's talk about something every PM faces, getting alignment on product decisions. You know that feeling when you're trying to explain a user flow to engineering or justify a design choice to leadership and you're just describing it with your hands? That's where Mobbin comes in. Mobbin is the world's largest library of real-world mobile and web app designs from industry leading apps like Airbnb, Uber, and Pinterest. Instead of spending hours taking screenshots or hunting for inspiration, you can instantly find exactly how successful products handle onboarding, paywalls, checkout flows, whatever you're facing. Over 1.7 million product builders use Mobbin to benchmark against best-in-class products and show their teams proven solutions. Whether you need to convince stakeholders there's a better way to handle user activation or research how top apps approach feature discovery, Mobbin gives you the visual proof to back up your product decisions. Check out mobbin.com/aakash. That's M-O-B-B-I-N.com/A-A-K-A-S-H and get 20% off your first year. If you're not talking to customers weekly, are you really doing real product management?

    4. TT

      Yeah, in the past I've said no, and I've been very black and white about this, but I've softened my answer, and I think it's because there's a lot of toxic messages in our industry right now where if you're not doing all the right things, you're not a real product manager. Here's my answer now. If you're doing what your company expects you to do, you are a product manager. Like, fundamentally, that's our floor. Like, your job is to do what your company expects you to do, period. Can we do more than that? I, I think so, and I really think everybody has more agency than they realize. And so I wanna encourage people to, like, step into this even if they're not getting organizational support for it, but I don't wanna tell them what they're doing is wrong. I feel like something has changed recently. I feel like it's everybody's building their personal brand and, like, LinkedIn has become this toxic dump of everybody telling you you're doing your job wrong, and I really don't wanna contribute to that. I think we could ... All of us have room for improvement and there's more that we could be doing, but fundamentally our job is to do what our company expects us to do.

    5. AG

      I love that message. There's way too many people telling you if you wanna be a great PM versus good PM, I think it's like, yes, Ben Horowitz wrote a great piece of content 25 years ago-

    6. TT

      Yeah

    7. AG

      ... and it resonated then, but can we please move on from that messaging? [laughs]

    8. TT

      Yeah.

  14. 35:2348:59

    Discovery for AI features: the new product work—context engineering, orchestration, observability, evals

    1. AG

      So I wanna touch on the other side now of AI and product discovery. What does product discovery look like for an AI feature?

    2. TT

      Yeah. I'm just learning about this myself, so I'll s- share that, let's see, it's late July. I am about four months into building my first AI product, and I ... It's completely opened my eyes to just how different product management is for a AI product. So let's first start with the differences of, like, like, what's different, and then we can get into, like, where does discovery fit in there.So I think there's a few things that are different, and I think what's hard for me about this topic is I'm still struggling to separate what's product work and what's engineering work. And I really think AI products are gonna blur our roles a lot more than they have in the past, which I'm actually really excited about 'cause I love it when our roles blur. Um, I've always been a boundary sp- spanner, and so I think I wanna see more of that. But, like, let's talk about this. I'm actually working on a blog post where I'm just doing this for my own sense making, trying to understand, like, what are the big components of a AI product that are different from, like, a deterministic pro- product or feature. And so the first one is, I know people are starting to use this term context engineering, which I like a little bit better than prompt engineering. Here's the challenge I see with prompt engineering. We all have experience, like, chatting with ChatGPT or Claude, and we're in a conversation. If we get that first prompt wrong, we can immediately refine. And so when we think prompt engineering, we're like, "Oh, we know how to do that 'cause we talk to these tools all day, every day." But when you're building a product, the prompt can't be refined by you in the... I mean, it can be as you're developing, but, like, once it's live in your product, there's no refinement. It's a one-shot. So that prompt has to work. So that... There's this skill of, like, how do we write prompts? How do we instruct an LLM to reliably do the same thing over thousands and thousands of instances? And I think people underestimate how hard this is, and this is a very different problem than, like, I'm just chatting with ChatGPT. So I think that's one piece, and we can get into that and, like, what d- how discovery can inform that. I think a second piece is always in product work, we have this task of decomposition. We have this vision of this big feature and, like, how are we gonna get there over time? But I think with AI products, the decom- composition task is twofold. How are we gonna get there over time, but also how are we orchestrating... Like, how do we break up the tasks so the LLMs can be good at it? So, like, I went through this on my own learning journey. My interview coach started as one prompt, and now it's seven prompts, and it's quickly growing. And so then it, it introduces, like, this sorta o- orchestration question, which is a little bit engineering, but I think it's also product. And this is where we get into things like which tools should we use. Are we using MCP servers? Are we using RAG? Like, there's sort of this messy... I'm just gonna call it orchestration. And then there's a third piece of, like, observability. Are we collecting traces? Are we doing that in a ethical data practice way? Are we informing our customer? But we have to store traces. And for those that don't know, a trace is just a LLM prompt plus the response, so all the back and forth between the LLM and the end user. And then I think the last piece of this is, like, how are we evaluating quality, and this is where evals come in. And so I think broadly... And then there's the ongoing, like, maintenance of a feature, which I think looks very different than a deterministic feature. So that's kind of the five buckets that I've kind of started to noodle on of this framework of how are AI products different, and I think discovery can inform each of those. But let me pause there before I get into that.

    3. AG

      So let me recap. Context engineering, orchestration, observability, quality, and maintenance.

    4. TT

      Yep.

    5. AG

      Did I get that right?

    6. TT

      Yep.

    7. AG

      So let's start with a little bit of a deeper dive on context engineering. How does that change, and how should we- discovery be playing into that?

    8. TT

      Yeah. So I was really surprised when I s- so but just to give a little bit of context, the AI product that I'm in building is a interview coach. So in our courses, people practice interviews. They submit their transcripts, and a AI coach gives them really detailed feedback. Like, it pulls out excerpts. It gives them coaching tips. And when I started building this, I was... When I first started experimenting, I was literally in a Claude project. I uploaded a lot of my course content. We already had a rubric in the course that we gave to students so they could give feedback on each other's interviews, and I had this really big insight that context engineering in the context of a LLM product is very similar to teaching humans a skill, right? So to teach a LLM to do a thing is very similar to teach a human how to do a thing. It's really about what context do you give them and when, so they're able to do the skill well. And so I... Like, now I'm really interested in, like, how do we get all of our natural teachers involved in helping us build AI products, but that's a little bit of a side tangent. It's really, like, a lot of us start out with, like, "Oh, I'm just gonna give it everything it could possibly need." And the challenge with that is that LLMs get confused.

    9. AG

      Yeah.

    10. TT

      I know we talk about, like, million token context windows or 200,000 token context windows, but, like, when you start to bump up against that limit or an even long before you hit that limit, the LLM is not good at following everything in that context.

    11. AG

      Yeah.

    12. TT

      And so one of the keys of this first piece is, how do I give it the right context at the right time? And this is why, like, MCP servers and RAG and, like, all these tools that are helping us feed in the right context at the right time are becoming more pervasive. And this is also a big part of the agentic flows is, how do you equip the agent to pull in the right information it needs, either through tools or through MCP servers or even RAG as a tool? And if people aren't familiar with those terms, like, four months ago, all of this was like a word... like acronym soup in my brain. Really, it's just like an MCP server, a tool is like an agent can request information from another tool, right? And that tool could be local in your system, or it could be through a third party, which is where a MCP server comes in. And then RAG is just, um, a ha- it... Like, you could have a database of documents, and it's, it's a retrieving, thinking about it like a search. It's just retrieving the right context for that query, for that input. And so I put all of those in that context when... like, that context engineering step of, like, where is the right information to help the LLM do this task really well?

    13. AG

      The next step you highlighted is orchestration, and not many people talk about this. Even I'm a little unclear in terms of where does discovery come in there?

    14. TT

      Yeah. Okay. So actually, we didn't really get into discovery on the prompt piece, and I think where discovery informs the prompt piece, and this will also inform orchestration.Is I can't, like, I can't teach an LLM how to do a thing that's supposed to help a customer if I don't know what the customer needs, right? So fundamentally, like all... Start with my... I'll use my interview coach as an example. How do I give a student feedback on their interviews if I don't know what mistakes students are making, right? So, like, I have my teaching knowledge. I know what a good interview looks like, and I can start there, but, uh, as soon as I hit real data and I start seeing real student interviews, I start to learn, like, "Oh, they're making these mistakes, not these mistakes." So now my, my coach needs to know about those types of mistakes and needs to be prepared to give feedback on those types of mistakes. So even just how I'm instructing the coach in that context engineering piece is really informed by the opportunities I'm identifying in my interviews or by looking at students' work. When we get into orchestration, this is where I recommend people start with the simplest solution to start, and then when they s- through their observability, which is the next step, when they start to identify errors, then that is gonna maybe affect their orchestration. So I'll give an example of this. My interview coach started literally as one prompt. It was just one long document of here's how we grade an interview. And I have a rubric where I grade interviews on seven different dimensions, and what I found was that the coach was starting to get confused across the dimensions. So, like, it'd be grading one dimension, but it would be pulling in instructions from another dimension. And I was like, "Okay, this is... It's great that my prompt fits in the context window, but it's getting so big the LLM isn't keeping it organized and keeping it straight." And I think this is a general thing that I see people write about a lot, is that LLMs perform better when we give them a simple task.

    15. AG

      Yeah.

    16. TT

      So I was like, "Okay, I have seven dimensions. I'm gonna, I'm gonna break each of those dimensions into their own LLM call." So now I have a workflow. So what started as one prompt now becomes a workflow where I take the same transcript and I send it to seven different LLM calls, and then I have to orchestrate the response. So I'm collating the responses before I send it to the student. So that's a... A workflow is just a series of orchestrated LLM calls. Um, so that's one example of orchestration. Another example of orchestration is I could see in the long run, like, I might move to a agent model where the agent is smart enough to look at the transcript, maybe knows a little bit about my student's history. Maybe I'm gonna pull in, like, the student's history through something like RAG or an- or a tool, and it's gonna look at both and make a decision about what's the dimension that's most important to give feedback on. And instead of giving feedback on all dimensions, it gives feedback on the dimension that's most important to the student. That would... I would put that in the orchestration bucket. Like, how am I orchestrating how the LLM is gonna respond to this input? And so where does discovery come into play there? It's... I think orchestration and con- context engineering are really closely tied. It's how do we understand what opportunities we're trying to address, and how are we giving the LLM the context and small enough tasks that it can meet those opportunities clearly and adequately.

    17. AG

      Okay. And what I'm hearing here is a really important focus on error analysis then, where discovery and error analysis, it seems like, are almost synonymous in the case of building AI features. And as you set up this good observability and you're doing your evals for quality and you're maintaining the system, you're going back into the context engineering and the orchestration to actually change things. Is that a fair summary?

    18. TT

      Yeah, so this is let's get into this third bucket, bucket of observability. So most people may not realize this, 'cause I didn't realize this, and I think this is a huge ethical concern. Most AI products are logging your traces. So what does that mean? It means when you interact with the AI feature, they are creating a record of the inputs and outputs. This is good for the developer. I actually think we need to be much more transparent about this, so I think this is a part where our discovery best practices of being upfront about what we're storing in our data practices is very relevant here. I think we need to be informing people this, and maybe not buried it in a terms of service. But this is a critical step. We do have to observe what these non-deterministic tools are doing, so we need a way to log at least a percentage of our traces, if not all of our traces, so we can start to look at them. And you mentioned error analysis. This is where we're gonna look at our traces, some percentage of our traces, and we're gonna, we're gonna literally have humans review them and tag them: Were there errors? And I have been doing this a ton for my interview coach, where I literally look at the transcript, I look at the coach's response, and I write notes about what, what would I h- what would I as the instructor have done differently. And I do this... I typically, transcripts are long, so I typically work in batches of, like, 50, and then once I've done a batch of 50, I'm looking across my annotations and I'm looking for what are my common errors. And that's telling me, that's my feedback loop into do I need to change my context engineering? Do I need to update my prompts, or do I need to update my orchestration? Um, and I've actually had errors that have driven both of those types of changes. And then eventually, error analysis leads to evals. So some of my errors I can fix by just making prompt changes, and they go away. Some of my errors, maybe I change a prompt and it fixes a, a little bit, but then they come back. So for errors that are kind of persistent, I'm gonna write an eval. And an eval is just code. It can be code. It can be a data set. It can be another LLM. It's a way to say, "How do I know how often this error is appearing in my traces?" So it's an automated way of saying, "How do I know how often this error is showing up in my traces?" I prefer to use code-based evals or LLM-as-judge evals. And so what that means is once I identify a error, if it persists, if it's not easy to make it go away, I write a eval-To de- to detect the error, and then what that allows me to do is now I can A/B test changes. So I can have a s- I can have a set of transcripts that I use to, to run experiments. I can detect an error, and then I can change the way either my context engineering or my orchestration, and then I can run that set of transcripts on the old way and the new way, and then the evals is grading is the new way better. That was a lot, so I'm happy to pause for questions if there are some questions.

  15. 48:5952:53

    Claude Code for PMs: AI-assisted engineering to ship evals and fixes faster

    1. AG

      Yeah. Well, because we have limited time, folks, we did an episode with Hamel Hussein and Shreya Shanker on evals, if you wanna go a little bit deeper on that. I wanna ask Teresa really badly, because she's been writing about Claude Code-

    2. TT

      Yeah

    3. AG

      ... to just give us the 30-second introduction to Claude Code. Why... What is it? Most PMs are really scared of using it. Can they use it?

    4. TT

      Yeah. Okay, so we're recording this on a Monday. I will tell you, I am seven days into using Claude Code. But let me tell you what I did in those seven days. So about two weeks ago, I noticed an error in my interview coach. The error was a orchestration error, so by splitting my prompts into seven individual prompts, what it meant is the prompt, like, one prompt didn't have any context for the other dimensions. And what I was seeing was they were all using the same excerpts. And so one section would say, "This is a great question," in the context of my dimension, and then in another section, it'd be like, "This is a problematic question," in the context of my dimension, and it was leading to really confusing feedback to my customers. And so first I had to write a eval to detect the error, how often was this error happening, and then I had to identify a solution, and then I had to A/B test the solution. Okay, before a week ago, I did all of my evals in a notebook, in a Jupyter Notebook, and I kind of did my own, like, A/B testing ad hoc. Okay. Here's what I did starting last Monday with Claude Code. So I installed Claude Code. It's a terminal... You install it in your terminal. You talk to it just like it's Claude, but the difference is, is in your terminal it can see all your files. So I also started using VS Code a week ago. It's been a big week. So I started using a proper IDE, so I'm using Claude in the context of my development environment, which is VS Code, and I basically said, "I've identified a problem with my interview coach," and it can see all my interview coach code, "and the first thing I wanna do is detect it. And so, like, help me design a eval for this." And it gave me, sadly, a really complex design, which I got stuck on and spent, like, the first three days trying to get s- work on that problem, and then I went to sleep, and I woke up the next morning, and I came up with a way simpler solution. And I was like, "Hey, Claude, why don't we do this simpler solution?" And it was like, "Great idea." Thank you, sycophant LLM. And it literally wrote all the code for me. Now, I know how to code, and I know how to read code, and so my rule when I let Claude code for me is I review all the code, and I make sure I understand all of it, and that it's actually doing what it does, because Claude Code does make mistakes. So, like, you have to be a babysitter. But it literally wrote all of the code for my evals. It, for that eval, it wrote all of the code for my proposed fix. It also wrote code, like a testing harness for the A/B test. Like, it just did everything. I don't think I wrote a line of code all week. I reviewed a lot of code, and I was, like, the architect, because Claude's solutions were often very complex. Um, but if I think... Yeah, I don't think I wrote a single line of code. But I released a major update to my interview coach and a brand-new eval, and it was the first time I did a eval that spanned LLM calls, so it was, like, a more complicated eval than I had done, and I didn't write a line of code.

    5. AG

      There you go, guys. She started with it a week ago, and she's already making huge changes. I cannot emphasize enough if you are listening to this podcast to get over the barrier of, hey, it's in the terminal, even if, unlike Teresa, you're not great at understanding and reading code, and just get started with this tool. I think it's a game changer. I think it's even better than Cursor. Before you go, Teresa, I have to ask, because, you know, there's so many creators who are gonna be listening to this podcast who have been looking up to you. Maybe you

  16. 52:5356:25

    Closing: Teresa’s business, course formats, and wrap-up

    1. AG

      were one of the inspirations for them to even become a product creator. How big is the business of Teresa Torres?

    2. TT

      [laughs] Yeah. So I'm a company of one. Kind of a company of two. I have a full-time admin in the Philippines. She's technically a contractor, so from a US standpoint, I'm a company of one, but I have a number of contractors who help with my business. We... In a typical year, we have 2,000 to 3,000 students a year in our courses. I know, like, in the era of Maven, that might not sound very big, but we teach... All of our courses are 20 to 50 students each. We try to keep them small. We want a high instructor-to-student ratio. And for me, it's not really about, like... I mean, I know, like, people talk about, like, I've worked with more people in discovery than anybody else, blah, blah, blah. It's not about size. It really is about impact. Like, I know it's hard to build discovery habits, and I really... All of our programs, we've designed them to change behavior, and I think it's very different some, from some of these edutainment products that are out there. I know there's amazing classes on Maven. I'm not dissing Maven. I took the AI Evals class on Maven with Hamel and Shreya, and I absolutely loved it, and I strongly recommend it. But I, we're, we follow a little bit of a different process with our cohort courses. We practice, we focus a lot on really hands-on practice and a lot of support. So we're at, we've, we're at, we're at, it looks like, just over 17,000 students.

    3. AG

      Amazing, and how much does one of those courses cost?

    4. TT

      We have a n- we have a few different formats. So we have our, like, Product Discovery Fundamentals course, which covers all of the discovery habits, but think about it as a introductory course. When you cover all the habits, we can't go deep in any habit.That's a six-week course. It has 12 hours of live instruction and it's $1,795 US. We then have a series of deep dive courses that are, they're each on a single habit. And so those are our skill-based classes. They're designed to build skill really quickly. So for example, we have one on continuous interviewing. This is where the interview coach shows up. And it's you round after round after round of practicing interviews. Our goal is for you to leave the class feeling really comfortable to do a real customer interview. Um, those courses are all $799. And then we are starting to convert some of our curriculum into on-demand courses. We have one today called Customer Recruiting for Continuous Discovery. We're about to launch our second one, Story-Based Customer Interviewing, probably by September. And those are $259.

    5. AG

      Okay, folks. You can do the math on that if you wanna figure out what her business is. Teresa, as I said at the beginning, this was a true dream come true, and I think you over-delivered on the responses. Your fluency about AI is mind-blowing. Thank you so, so much for being on the podcast.

    6. TT

      Thanks for having me. This was a lot of fun.

    7. AG

      And I always believe in wishing in the future, so I'm gonna go ahead and put it out there to everybody. I hope we have a round two. Support this episode so Teresa just has to come back. Thank you all for watching, and see you in the next episode. I really hope you guys enjoyed that episode. It would mean a ton to me and the team if you could please subscribe on YouTube, follow on Apple and Spotify podcasts, and leave a rating and review. Those ratings and reviews really help grow the show and help other people discover the show, and they help fund the production so that we can do bigger and better productions. Can't wait to share the next episode with you. Until then, see you later.

Episode duration: 56:30

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode wGuRuuPuYNQ

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.