Skip to content
Aakash GuptaAakash Gupta

Complete Course: AI Product Discovery

Tanguy Crusson is one of my favourite product management voices in the world because this guy really gets what it takes to build valuable products that users love to user and positively impact the business growth. We could talk a million things with him but I’m keeping it to what I like the most about his work - Product Discovery. You’ll also learn: - How they built Jira Product Discovery from a slide deck prototype → 18,000 customers - What most PMs get wrong about discovery (and how to fix it) - Why Tanguy hasn’t written a PRD in 5 years If you’ve ever thought, “Damn, I wish I actually knew how to do product discovery right…”, this episode is for you. 🎥 Timestamps: Preview & Intro — 00:00:00 Ideal Discovery Process - 00:00:24 Stage 1: Wonder - 00:02:12 Jira Product Discovery (JPD) Roadmap - 00:04:44 Ad (JPD) - 00:09:21 Ad: AIPM Certification with OpenAI PM — 00:10:16 Summarising Wonder Stage - 00:11:03 The Best Investment You Could Make as PM - 00:12:33 Most Important Area to Look for when You Join a Team - 00:19:14 Ad: Vanta Compliance & Security — 00:26:21 Ad: AI Evals Course for PMs & Engineers — 00:27:25 Why This System Will Revolutionise the Work for 75% of you - 00:28:25 Stage 2: Explore - 00:28:59 How to Build What Matters - 00:31:18 Stage 3: Make – The Growth Funnel vs. Safety Funnel - 00:36:00 Legendary Atlassian Ad for PMs - 00:40:20 If he had Zero Customers, What Would He Do - 00:42:31 How The Process Looks Like Between Make & Impact Stages - 00:47:07 Stage 4: Impact - 00:54:33 Outro: 00:57:14 ---- Podcast transcript: https://www.news.aakashg.com/p/tanguy-crusson-podcast 💼 Check out our sponsors: 1. Jira Product Discovery: Plan with purpose, ship with confidence - https://www.atlassian.com/software/jira/product-discovery 2. Product Faculty: Get $500 off the AI PM certification with code AAKASH25 - https://maven.com/product-faculty/ai-product-management-certification?promoCode=AAKASH25 3. Vanta: Automate compliance, security, and trust with AI (Get $1,000 with our link) - https://www.vanta.com/lp/demo-1k?utm_campaign=1k_offer&utm_source=product-growth&utm_medium=podcast 4. The AI Evals Course for PMs & Engineers :Get $800 off with this link - https://maven.com/parlance-labs/evals?promoCode=ag-product-growth 👀 Where to Find Tanguy LinkedIn: https://www.linkedin.com/in/tanguy-crusson-99832a/ 👨‍💻 Where to find Aakash: Twitter: https://www.twitter.com/aakashg0 LinkedIn: https://www.linkedin.com/in/aagupta/ Instagram: https://www.instagram.com/aakashg0/ 🔑 Key Takeaways: 1. Discovery isn’t a phase, it’s a system. Atlassian runs product discovery continuously, not just “before development.” It’s embedded across problem finding, prototyping, building, and post-launch. 2. Use video, not documents, to communicate user pain. Instead of writing long research summaries, PMs compile 10-minute reels of real customer interviews. Watching raw emotion builds urgency and alignment. 3. Start with ~10 users, not thousands. Atlassian validates ideas with small, focused user groups. It's faster, cheaper, and more revealing than wide surveys or launches. 4. Prototype with whatever is fastest. From AI tools like V0 to basic Figma slides, the goal is speed. You don’t need polished UIs, you need fast feedback on core concepts. 5. Strong user reactions guide investment. When users say “I need this now,” that’s a green light. Mild interest or polite nods? That’s a warning to dig deeper. 6. Build only once you have real pull. They don’t move into development (“Make” stage) until a prototype has strong qualitative validation. Code follows conviction. 7. PMs rotate weekly to tag and analyze feedback. Every week, one PM owns triaging incoming feedback, tagging it to ideas, and surfacing themes. Discovery is part of the rhythm — not a side project. 8. Real discovery requires exposure, not summaries. Dashboards, sanitized reports, and secondhand quotes are not enough. PMs must stay close to raw user input — live or recorded. #productdiscovery #ai #atlassianjira #atlassian 🧠 About Product Growth: The world's largest podcast focused solely on product + growth, with over 175K listeners. Hosted by Aakash Gupta, who spent 16 years in PM, rising to VP of product, this 2x/ week show covers product and growth topics in depth. 🔔 Subscribe and like the video to support our content! And turn on the bell for notifications.

Aakash GuptahostTanguy Crussonguest
Jul 8, 202557mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. AG

    Of tasks that are high leverage for a PM, product discovery is number one. And AI is allowing PMs to do much better and more product discovery. Today, I sit down with Tanguy Crusson, who is one of the preeminent thinkers on product discovery as head of product for Jira Product Discovery for the last five years. And today's episode will blow your mind on a new way of working. Can you walk us through step by step an ideal discovery process?

  2. TC

    Uh, yeah. So ideal, uh, no.

  3. AG

    [chuckles]

  4. TC

    Uh, but an example, definitely. Uh, and what I'll use is the example for how we do things in the Jira Product Discovery team. So the, um, uh, quote-unquote, "process" that we follow ourselves has four main... I'd call them stages. It looks like waterfall. It's not really waterfall. Things keep moving from one stage to the other based on what we learn. But generally speaking, whenever we tackle a new problem area, that's what we follow. When we created Jira Product Discovery, the product, the whole product, we followed stages in this process as well. So it, it applies for us at the macro level, and it applies also at the micro level when we work on specific features. The reason we created this process was that it's very, uh, complicated to talk about the stages of things with everyone at Atlassian. Uh, so whenever we talk about an idea and this idea is, it's super fuzzy, we don't- we're not even sure what problem area we're doing, w-we need to give it a name because people might assume it's further down the track otherwise, and maybe we'll come to them with a specific ask to help depending on the stage. So for example, if we are making something, we might come with an ask from a dependencies to help us build something. Versus if we're exploring solutions, uh, and we're just about to exit that stage, it means that we might be asking for funding or things like that, and things like that, right? So we've kind of set those four, uh, stages as vocabulary across the whole of, uh, Atlassian pretty much. Everyone knows at Atlassian what we're doing when we say we've got something in Wonder. So we've got Wonder, Explore, Make, and Impact. Wonder is all about problem exploration. We're not exactly sure what we're doing, but we've got a lot of customers currently that are talking about in their road mapping process, how they should be working with dependencies and working with capacity, and those things seem to be tied together, for example. And so what we do is we get in and do, uh, about a dozen, uh, user interviews and to try and unpack that and to go with customers and try, try to understand how these things fit together. And the conversations will usually be about an hour long, and we do a few of them until we feel like, "Okay, the same themes are coming back," and so on and so forth. Once we've got a good, uh, uh, understanding, uh, of that, we go into solution explorations. And our goal here is not just to go into a, you know, a room, create designs, and then only come out when the designs are finished. No. The goal is basically to quickly find ways to test stuff with users until they tell us, "Yes, this would actually solve my problem." I have a few examples of that. Then once we have enough validation that something is worth investing into, that's when a team makes a pitch to go into Make, which is, "Okay. You know what? We've got all the validation that we need here. It's enough. Now we should really start working with a lot more customers. So currently we've got ten that are interested. We'd like to get to a hundred, and progressively to a thousand," for example. And so that's what we call Make, which is we're committing to building something. We've got enough validation we think for it, and now it's really about making it happen. Maybe we just do a first version first. Maybe we do some more iterations later. So we try to shape that there. And there's a few iterations that, that go in Make. And then Impact is usually when, uh, okay, the problem has translated into something that is in the product, and now we're measuring to see how well it performs. Uh, because it might be working well for a small number of users, but we need to make it work well with a super large number of users. And so we call that Impact. This is usually a few months after we launch things. We do a-- We sit down, and we write down an impact page to go, "This is actually how it's performing. We recommend just putting it on hold for now, it's doing fine, or doing one more iteration, or we actually need to have a roadmap over a longer period of time to actually have it work to its full potential. What should we do? Should we retire it? Should we leave it to a small number of customers, or should we, um, invest more?" So that's the process: problem, solution, building it for real, and actually iterating on it, uh, until we've got, uh, we've-- we hit the desired goals for it. So, um, probably best that I go into... So that's Jira Product Discovery, and that's our Jira Product Discovery project. So in there we do everything from discussing, uh, feedback that we receive from customers, uh, classifying them into, okay, those are big bets, uh, versus iterations on current experiences versus like small pebbles, which are tiny, uh, UX improvements that, uh, any engineer can come in and pick up, uh, whenever they've got some spare, spare time and want to do something different. Uh, and it's also where we have our roadmaps. So going from the roadmap that we have at the highest level, uh, and that we share with our leadership focused on bets and solutions, and then roadmaps from, uh, specific teams, for example, that would go more into solutions, uh, that ladder up to specific bets and specific iterations, right? So there's, um, uh... What we do in the Jira Product Discovery team is we, uh, actually are on a tation- a rotation, the PM team, to look at user feedback. Uh, so we're, uh, four PMs, meaning every week, uh, there's one of us four that for the week looks at all the feedback and tags this feedback based on what we learn to, uh, as insights to ideas. We discuss that every week all together. I think next week is actually my turn.Uh, and, uh, then, uh, we sit down, uh, more or less every month to discuss, okay, so what's changed really that means that we should revise, review our roadmaps? But let's assume that we have said, "Okay, you know what? That stuff came often enough. We need to look into it." Uh, what we do is we'd start a Wonder Stage, which is let's look at-- Let's, let's unpack the problem. So I've got here, um, in product management jobs to be done, which is one of our bets. We help product operations teams and administration teams roll out JPD, Jira Product Discovery, at scale, and it's part of the jobs to be done theme, where we focus on specific personas, uh, PMs, product operations, administration. So in there, there's something called global fields. It was not called global fields before. It was a messy area, uh, which was called global entities, which was we were finding... People kept talking to us about having many Jira Product Discovery spaces, projects, and needing to be able to work with all of them, uh, at scale. And it was, um, initially the i- we had a-- we had discussed a bunch of solutions, and we were like, "You know what? It's super hazy." Hermos, Hermos Ndunga, one of the, uh, product managers in, in Jira Product Discovery, "Why don't you go and actually figure out what, what that all means?"

  5. AG

    Mm-hmm.

  6. TC

    And she went, ran a Wonder Stage. And this is a typical outcome from a Wonder Stage, which is a page that tries to find, um, basically who we're trying to help and the types of problems that they face. So she had decided for this particular exploration to focus on specific personas, so product ops, program management, VPs, stakeholders, and PMs. And then what we do for all of them, what I ask the PMs is, "All right. I don't care about big, lengthy pages that explain to me how big a problem is. Instead, give me all up less than ten minutes of videos of customers talking about their problems, and I want to see the different facets of it." Right? So it has to be probably at least ten customers, and they have to be talking about their problems in ways that makes us understand the different facets of these problems.

  7. AG

    Mm-hmm.

  8. TC

    And so that's what Hermos does really, really well. She starts and runs a little bit of, uh, research. It lasts for a couple of weeks, and she comes back with a page like this, where you see there's one, uh, problem area here, there's a second problem area here, and there's a third one there. Uh, and then, uh, that's it. And literally, we have a meeting with all the product managers, and we watch this together, silent reading and watching videos for ten minutes, and then we discuss. And usually, within half an hour after that, we're like, "Oh, yeah, yeah, that sounds important," or, "Ah, it needs a bit more, uh, exploration and whatnot." But that's really for us what Wonder means. It doesn't have to be something which is, uh, like you... we've surveyed, uh, thousands of users. We've done this, uh, comprehensive, uh, super scientific approach to all this. It's really no. Like, we've got a few customers that have been voicing this. Let's go in there and understand how they're feeling that pain. And we bring that to the rest of the team, so they understand where the pain comes from. So that's for us, that for us is Wonder. Uh, any, any questions you'd have before we go into Explore?

  9. AG

    Today's episode is brought to you by Jira Product Discovery. If you're like most product managers, you're probably in Jira tracking tickets and managing the backlog. But what about everything that happens before delivery? Jira Product Discovery helps you move your discovery, prioritization, and even road mapping work out of spreadsheets and into a purpose-built tool designed for product teams. Capture insights, prioritize what matters, and create road maps you can easily tailor for any audience. And because it's built to work with Jira, everything stays connected from idea to delivery. Used by product teams at Canva, Deliveroo, and even The Economist. Check out why and try it for free today at atlassian.com/product-discovery. That's A-T-L-A-S-S-I-A-N .com/product-discovery. Jira Product Discovery, build the right thing. Today's episode is brought to you by the AIPM Certification on Maven, run by Miqdad Jaffer, who is a product leader at OpenAI. This is not your typical course. It's eight weeks of live cohort-based learning with a leader at one of the top companies in tech. OpenAI just doesn't stop shipping, and this is your chance to learn how. Run along with product faculty and Mo Ali, the course has a four point nine rating with a hundred and thirty-three reviews. Former students come from companies like OpenAI, Shopify, Stripe, Google, and Meta. The best part? Your company can probably cover the cost. So if you want to get five hundred dollars off, use my code AAKASH25 and head to maven.com/product-faculty. That's M-A-V-E-N .com/P-R-O-D-U-C-T dash F-A-C-U-L-T-Y. It seems a lot like ethnographic research, where instead of just doing a large survey, you're trying to go really deep and add texture to the problem, really understand how does that affect their jobs to be done.

  10. TC

    Yeah, that's correct. And we try to, um, we try to do as much as we can in, uh, plain English, raw emotions for those users. It's like, okay, so, you know, let's explain it to the team in the customer's own words how the pain is coming out, removing all the varnish there is around it. Let's try n-not to make it sound too smart. Let's try not to make it, um, so compelling with a lot of, uh, superlative and obj-adjectives to describe it. Instead, let's just hear it from them. And what we found is people resonate a lot with that. They'll be convinced, uh, abs-- in an abstract sense by reading a page that's beautifully written, and they'll be like, "Wow, that's smart." But it doesn't trigger the urgency and the need for action that, uh, watching a videoFrom a customer does, and it, it's even better when we invite, for example, engineers with us on some of these conversations, and the engineers hear it firsthand from the customers. The response from them is like, "Okay, there's an innovation, innovation week next week. Uh, I'm gonna try a solution for that." And they really go all in, and that's exactly the energy that we want around these problems, right? But that just-

  11. AG

    So are there any-

  12. TC

    Yeah

  13. AG

    ... tips for how to conduct this conversation as a PM?

  14. TC

    Yeah. Honestly, if there's one investment you could make that could change your life as a PM is find a real user researcher, someone who does that as a trade, and ask them for training. I used to think I was amazing at user interviews. I was saying a lot of smart things, and people would concur with a lot of them. And then I had, um, um, a researcher, Georgie Bottomley, uh, she was at Atlassian, and she watched about 20 user interviews, and she said, "Yeah, you... I know you're very proud of those 50 user interviews you did. Dude, I can tell you, you didn't learn a thing. You're leading all these conversations. You're pushing users to, in directions that they would not have taken on their own, so you're not really learning anything." And what she did is to basically give me a simple interview script with a lot of silence to follow, which is basically just, you know, ask them to introduce themselves, ask them to describe what they do in their company, and from then on, you don't have an interview script. You rebound on what they say, and you might be poke in one direction if they are going there, but you do not ask leading questions to take them somewhere. And so if you're trying to validate if product managers struggle with dealing with feedback, you don't never bring the word feedback in the conversation. Instead, you let them s- you, you try to see if they go there. And if they go there, then you can start to poke. And the other feedback she gave me is never interrupt a user who's talking. So some... Now, um, after... Like I took all her feedback on board and started doing more interviews, and she's like, "Wow, I'm proud of you. Three minutes you didn't say a thing." Uh, and there's a few other tips like this, like never give options. You know, it might be uncomfortable, but if someone-- If you ask someone, uh, for example, "What's the first thing that you do in the morning?" We're humans. We're trying to make the other person feel comfortable. We're gonna give options. "What's the first thing you do in the morning? Do you drink coffee? Do you have a shower? Do you brush your teeth or... " When you try to interview users and you're trying to learn from them, the way you'd ask that is, "What's the first thing that you do in the morning?" And you just sit on your hands-

  15. AG

    [laughs]

  16. TC

    ... and you just wait, and there's an uncomfortable silence, and it's just there with you. You feel like it's forever, and then they start talking. But the way they start talking is based on, it's, it's from their perspective. You didn't give them options they can just latch on to and, and reply yes to, 'cause they just want to please you by default. So things like this are just tips and tricks. I'm not trying to tell you that's exactly what would work for you. More like get training from a researcher. It will drastically change everything that you do afterwards. Otherwise, I can promise you that you are going to have many conversations, and it's gonna prove what you think is right.

  17. AG

    And this document seems really well written. What is the right way to sit down and put a document like this together?

  18. TC

    Um, I think mostly, like the way Amos does it is the text doesn't matter. The only thing that matters is the videos. I, I barely glance at the text. Um, I watch the videos, and what she's doing is to find ways to tell a story by selecting snippets of, uh, in the video that progressively ladder up to give me the feeling that I've looked at this via all the different facets. She's got basically 10 minutes to summarize probably about 10 to 15 hours of conversations. And so typically what happens is it's, it's a lot on... It's a lot of the work that you do interviewing users, and immediately after those conversations, like finding the highlights that you think like really, uh, took you to an aha moment. So I wouldn't focus so much time in the writing of the document, but a lot more time in, um, making sure that as you go through the customer conversations, you tag the moments that gave you this aha moment to see if you can reproduce that with the people you're going to share the video with, right? And making sure that it, it tells a story when you put all of them together is quite important, so you basically are taking people with you. By the end of reading this document, I was telling Amos, "Oh, okay, I see. You would like to build this." And she was like, "Yes." [laughs] But I was, I was... It, it was so clear by watching this video, and it might be a trick to convince me, but she was convinced, right? Uh, that it was the right thing to do after doing all these interviews. So that's why like it's, uh, it should be a no-brainer to the people watching this. It should be really clear. There's a moment in when you start those conversations where you just don't know. There's a... It, it can last for a long time. You're, you're navigating, and you feel like you're going backwards some days. Like some things you were thinking, some customers went that way, some customers went completely another way. That's completely normal, and my advice here for people is just embrace that messiness. It's fine. It's actually those things that, you know, if you, if you let it sit for a while, eventually you're gonna see through. It's like if you take a bottle of water and you put sand in it and you shake everything. That's what we do when we do a lot of user interviews. And then you, you put it on the table and you, you wait. And eventually the sand will settle, and you're gonna see through. It's exactly the type of, of things that I, I ask the PMs, like wait till you, you, you've hit that clarityLike you have another conversation and you're like, "Oh, I feel like I'm having the same conversation again." Good. That's the moment where you actually write this page. It should not take you more than two, three hours all up to write that page, basically, because you're already clear.

  19. AG

    So you stop the interviews once you're converging on insights, once that sand is starting to settle. And I think one thing that might get in people's way is just the tools itself, because they'll say, "I've never clipped a video and put a video together-- several clips together. How do you guys do that?"

  20. TC

    Oh, it's, it's really simple these days. I don't know how many tools there is out there. Uh, for us, uh, we've tried many. Uh, we use, uh, Loom for putting them together, and we also use, uh, Dovetail. Uh, so Dovetail is where a lot of our, uh-- most of our user interviews go, and then it creates a transcript, and then, uh, we can query it for, you know, we can query transcript for specific themes, but also we can create snippets, and then we put them in, in, uh, in, um, in what I think they call that an insight, uh, which would have a bunch of video reels put one after the other. Like the-- There's one... Okay. My advice to every PM as well is to s-- like when you join a team and there's no... People don't talk to customers a lot, and they don't know how to do stuff like that, your first job is to put that in place, right? So the finding, like recruiting, identifying, recruiting, um, setting time with customers, um, extracting, like running, running the interviews, extracting video clips from it should be something that takes you no thinking whatsoever. Uh, so for us, for example, I go in Pendo, I run a little segment. I know which customer I'd like to talk to for specific behaviors if it's data related about a specific feature. If not, I will go into Dovetail, or I would go into our Jira Service Management cube where we receive raw feedback. I'd run a little query, and I-- like within a few minutes, I have a-- I've got a few names of people I can talk to. I also have tooling to be able to send them, uh, emails with a quick intro that's as little fluff as possible. "Hey, you've been using Jira Product Discovery a lot lately. I'm part of the team. Would love to chat about, uh, how we can make it better for you." That kind of stuff. Make it human simple. Two-liners tops with a Calendly link. Uh, they click on it, they book time. It's connected with my calendar. I don't have to send three options. They reply, we reply. No, that stuff is gone. Then whenever I join this meeting, uh, the, the-- like I know that it's going to go to Dovetail. I've got my, uh, recorder, uh, uh, coming with me on the call as well, so I know it's all going to be recorded, so I can extract snippets after the call. Um, like all that stuff, all that mechanics is really something that you should put in place because as a PM, AI or no AI, you're gonna beat every other competitor if you learn faster than them and if you know more about your customers. And it's, it's, it's, it's worth a lot-- like ten times more than any automatic insights you can get from any, uh, model today. Which-- And so use the AI to get you in the right conversations, is the way I would say it. Use the AI to extract s-snippets from videos. Watch as many of them as you can. So for example, sometimes I go in Gong and say, I mentioned Jira Product Discovery, say, Premium, because we just launched it.

  21. AG

    Mm-hmm.

  22. TC

    And I see where it's mentioned, and boom, I watch all the video clips. That kind of stuff, right? That's a lot-- That f- that's what will give you an unfair advantage because you're n- you're gonna keep learning, and it's gonna keep giving you perspectives that you didn't have. So that's, um, uh, that tooling, really important. And so for us, we've got, uh, Dovetail here. We've got a community group where users can come in, and they can ask questions. We receive maybe ten questions a day here sometimes. Um, and they ask questions. Like it can be as simple as, uh, something like this, where, uh, like someone was saying, "Oh, there's a bug. I used to be able to filter on this, I c- I can't." And, uh, just took an example for the... It happened yesterday. And uh, today I was like, "Oh, shit, yes, we're looking into it." And then a few hours later, "Okay, we fixed it for you." That kind of stuff gets users engaged, and then they come back, they ask more questions. So that we can then let them know when, uh, for example, we're experimenting with new features. So I, for example, posted this feature, which is now part of the Jira Product Discovery Premium plan, but basically started just with a Loom where I was like, "Hey, we're thinking of this. What do you think?" And so, uh, you can see here there was like two thousand views on it, lots of, uh, um, comments and reactions, uh, and, uh, even at the bottom of the post, like either excitement or questions. I've got four pages of, of comments for it.

  23. AG

    Wow.

  24. TC

    Right? Um, da, da, da. So we use it for that, for answering people's questions, for, um, uh, addressing their bugs. So it's quite useful at the scale that we're at, which is 18,000 customers today. We still are able to stay on top of that, you know, with a PM team of four, uh, with a weekly rotation. Um, we do have Pendo, where we basically, uh, can in-app ask questions to users. Uh, we use that for CSAT, but we also use that for things like, "We're looking at option A versus B. What do you think?" Uh, and sometimes it can help settle an argument. We get like, I don't know, a hundred answers, and we're like, "Eh, see?" And we look at that, but we often look at the commentary. We al- always ask them like, "Tell us more." And they give us commentary. So whenever we hesitate between two things, we just ask, and then we get something back. That's pretty useful. Needs that-- need to implement this tooling, but once you have it, it's really amazing. Uh, we do, uh, we do talk with, uh, customers on Slack. So I've got, uh, probably about twenty customers, uh, that I'm-- I have on speed dial. So they're all in this folder right here, which is collapsed. And Medibank is one example. You can see here, 5th of May. It's like, what? Uh, two weeks ago. They had an issue using Jira Product Discovery with Miro with a new feature we built. We helped them with that, and since then, there's been other questions, answers, questions, answers, and then stuff that we ask, ask them to do as well sometimes.So basically, as close to the ground as possible is what I, I would, uh, recommend for, uh, for things like this, right? So we-- And we've got, uh, this, uh, this thing where, so you're in Jira Product Discovery, and you might have feedback for us. You click that, you just give us your feedback. This feedback goes into a Jira service management queue, and we can triage all this, and we tag the new things that we haven't thought about to ideas as insights. And we can also use Rovo here to give us like, okay, so in the last six months, what do people say? Uh, uh, which is a way to use the AI to help us like look at things at the macro level sometimes. Every... Anytime I start in a new team, that's the first thing we set up, which is all these touch points to make it so customers just literally have to pick up the phone, and they can talk to us. Every PM that first joins the team say, "It's never gonna scale." And I'm like, "I don't care." [laughs] Because it's... We're meant to be swamped with this stuff. That's what will make us think. Not if we're comfortable, and we have little summaries that our research team gives us, and it's all very well organized, and there's, "We should do just one, uh, just A and B and C." It doesn't work like that. What we know is we ship products, there's existing features, but they also want new things, and there's also reliability things we need to worry about. And there's some customer segments it works for them, some it doesn't. Some, uh, would care about specific details, others just don't care. And it's, uh, just like this diversity and this, uh, is, is how I progressively get an understanding of what our users need. The reason I, I can say I think, uh, I've got, uh, some credibility in talking about this is, as we scaled from zero to 18,000 customers, our CSAT has stayed at above eighty-five the whole way.

  25. AG

    Wow.

  26. TC

    And that's, that's really hard to reach a score like this. It's the first time in my career I actually get that, and that's where a lot of our growth comes from too.

  27. AG

    Trust isn't just earned, it's demanded. Whether you're a startup founder navigating your first audit or a seasoned professional scaling your GRC program, proving your commitment to security has never been more critical or more complex. That's where Vanta comes in. Businesses use Vanta to establish trust by automating compliance needs across over thirty-five frameworks like SOC 2 and ISO twenty-seven oh oh one. Centralize security workflows, complete questionnaires up to five times faster, and proactively manage vendor risk. Vanta can help you start or scale your security program by connecting you with auditors and experts to conduct your audit and set up your security program quickly. Plus, with automation and AI throughout the platform, Vanta gives you time back so you can focus on building your company. Join over nine thousand global companies like Atlassian, Quora, and Factory who use Vanta to manage risk and prove security in real time. For a limited time, my listeners get one thousand dollars off Vanta at vanta.com/aakash. That's V-A-N-T-A.com/A-A-K-A-S-H for one thousand dollars off. AI evals are one of the most important skills for PMs, and I know you know they matter. The question is, are you doing them right? Most teams are winging it with basic metrics and hoping for the best. Meanwhile, the teams that actually ship reliable AI, they've cracked the code on systematic evaluation. Today's episode is brought to you by the AI Evals for Engineers and PMs course by Hamel Hussein and Shreya Shankar. This live Maven course will teach you the battle-tested frameworks from Hamel and Shreya, who are the engineers behind GitHub Copilot's evaluation system and twenty-five plus production AI implementations. Four weeks, live instruction. Next cohort starts July twenty-first. Start shipping AI that actually works. Enroll at maven.com with my code AG-PRODUCT-GROWTH for over eight hundred dollars off. That's AG-PRODUCT-GROWTH. I think just implementing this system [clears throat] will be a revolutionary way to work for seventy-five percent of people listening, but I cannot vouch for this more. It's about seeping the customer knowledge into your own LLM brain, right? And this is how you do it. You're getting as close to the feedback as possible.

  28. TC

    Yeah. And always try to frame it from the lens of the customer with their own words and vocabulary and so on and so forth, which actually takes me-- We've been spending a lot of time in that first stage, wonder. Explore. Explore. How do people... How do I, do I see our customers, or a lot of our customers do, do this? Often for the less mature PM teams, what happens is a long process of designing the solution, followed by a spec, followed by a long process of building it. And in that long process of building it, they use sprints. And yes, that makes sense. But Agile was actually meant to cover both of these things together, which is you actually iterate on the solution, and you iterate on making it at the same time, and you might be uncomfortable for a super long time as this thing is shaping up, and that's exactly what it's meant to help you achieve. So, but let's start with explore. So what do we mean by prototypes and validation here? So what I ask the PMs is create... Like, you can work on prototypes, expose them to the... You know, you've got ten customers here in your wonder page, great. Get the same ten people in a s- in a video after you've presented your solutions, and get them to explain how that's solving it for them. That's what I want as to be ab- for you to be able to exit explore with enough confidence that we should invest a lot in it. So it's basically Hermance going back with these customers, iterating on potential solutions. And here she's like, "You know what? We're really onto something here. Here's the proof."And it can be things that are very lo-fi and simple. For example, I'm going to zoom back to five years ago. We were exploring concepts for what turned out to be Jira Product Discovery. And the first prototype we put in front of users was actually that one, which is nothing more but a slide.

  29. AG

    Mm-hmm.

  30. TC

    And because we were hearing them talk about all these problems, and we were like, "Phwoar, there's a lot there. Like, PMs are having a tough time. What should we help with?" And so we, we, we tried, pff, I don't know, countless iterations of that slide, where we had different pillars. We put them in different orders. This was probably the last one, where everything we realized was centered on prioritization, and everything else was in service of it. Uh, but there were key components we needed to get right that integrate well with it. But what we did is we would create a slide like this, and we would join a call with, um, a PM who would explain their, uh, their life and what they were struggling with, and then we'd pop... show that slide and go, "So what do you think? Uh, this is a solution we're building. How, how would it help you?" And we'd get them to engage with this and, uh, uh, and give us feedback. "Ah, that's interesting because, uh, this bit is, uh, uh, quite important. Like this bit is quite important for us, but really the stuff I'm struggling with is that," for example, which is where we landed in a lot of conversations. And we found a lot of users to work with them were able to very crisply define what their problems were. And so one of them, Brent, which is one of our Lighthouse customers, we can come back to that later, was saying, "You know, this is really what it boils down to. I work in a sales-driven organization, and I need to get people rallied behind my priorities. And it's really hard, but I really enjoy when we get to the end of this process and there's clarity into what we're building for everyone. It's, it's really joyful when we get there." And we're like, "You know what? We want to help Brent." And Brent was v-validated this one, and then we found other customers, and eventually, we narrowed down on that. But that was, that was the prototype. That was the first one. So we're like, "Okay, cool. We need to focus on prioritization. How are we going to help them focus on prioritization?" Initially, we were like, "You know what? We need to help them with prioritization because they're struggling with working with all these other teams. So we're going to give them a place where they can have all the feedback going in, and they can, they can prioritize this feedback." And, uh, look, it looks like, um, Gmail, I guess. It's basically an inbox of feedback. We presented that to a few customers. So this one, this one was probably a week spent in Figma discussing potential options. And then we presented that to users, and they were like, "Oh, yeah. That's, um, that's interesting. Yeah, I could, um, I could use something like that." Brent was saying the same stuff. And I was like, "That's not quite the response I was expecting." Okay. We explored a few other concepts. We went back to Brent, and we presented that. Again, very simple. It's just like capture, prioritize, deliver, which are the pillars we described in the slide. And then we were centering on ideas here with, uh, laddering up to goals, user impact, strategic value, and with, uh, some kind of pri- of a priority score. We presented that to Brent and a bunch of other users, and they were like, "When can I get this? I'm struggling right now. I've got this conversation coming up with my CPO. We're trying to present our roadmap. The CPO keeps arguing for us to build a particular feature. We keep thinking we need them to step up, right? Focus on go- focus on goals. Let us do the work of, of defining what should be in the product. I need this." That's when we knew we were onto something, right? We didn't write a single line of code there. Uh, the main reason is, like there's no technical complexity in building something like this in Jira. Right? We've got all the assets that we need there. So it was basically one PM... two PMs, one designer. We were iterating on this concept. We built a full prototype at some point, and we validated a lot of stuff this way. That was before we wrote a single line of code. Today, the way we do this is we go in Lovable, we upload a screenshot from Figma, and we turn it into a prototype, and then we get, uh, users in course playing with it. Trying to make it lo-fi still in, uh, in, um, in Figma because we still want... Uh, sorry, in, uh, in Lovable because we still want people to understand it's not a final product and get them to think.

Episode duration: 57:48

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode 6oottgko7pQ

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome