Aakash Gupta10 Years After the Lean Product Playbook: PM in the Age of AI
EVERY SPOKEN WORD
90 min read · 17,880 words- 0:00 – 1:31
Problem space vs. solution space risks with AI prototyping
- AGAakash Gupta
Is there a risk that people jump too quickly into the solution space and don't adequately investigate the problem and solidify that they're solving the right problem with these tools?
- DODan Olsen
Before vibe coding, you know, solution-space thinking is very prevalent. People start talking about, "We should build this feature. We should build this product." Sales teams are asking for features. Stakeholders are asking for features. Clients are asking for features. All that discussion is actually in the solution space. A lot of PMs are spending their time just managing the Scrum process, and they don't have time to do discovery. And now it's easier than ever to create a solution, which is kinda interesting if it's like, hey, if designing it and coding it is no longer the bottleneck, then it's like, well, what text you put in is the only thing that makes any difference.
- AGAakash Gupta
What specifically has AI changed? This feels pretty timeless.
- DODan Olsen
Yeah, it is pretty timeless. I mean, you know, at the end of the day, you still have to understand your customers, and AI's not gonna tell you, you know, about your customers.
- AGAakash Gupta
Dan Olsen, I have been following your work for over 15 years. What has changed in the past 10 years?
- DODan Olsen
I think really two main things have changed. One is ... The other big thing ...
- AGAakash Gupta
How has AI changed product management?
- DODan Olsen
AI has pretty much changed every step in the product management process. It can help you explore new opportunities that are out there in the market. It can help you identify segments. It can help you brainstorm feature ideas. It can help you do competitive analysis. But probably the most fundamental area it's changed the most is ...
- AGAakash Gupta
You're a legend in the product management community for your work on The Lean Product Playbook. What was the thesis?
- DODan Olsen
Yeah, the thesis was basically there was this term-
- 1:31 – 3:05
The Lean Product Playbook thesis: a process for product–market fit
- AGAakash Gupta
Really quickly, I think a crazy stat is that more than 50% of you listening are not subscribed. If you can subscribe on YouTube, follow on Apple or Spotify podcasts, my commitment to you is that we'll continue to make this content better and better. And now on to today's episode. Dan Olsen, I have been following your work for over 15 years. You're a legend in the product management community for your work on The Lean Product Playbook. For those of us who have forgotten in the last 10 years what that was about, what was the thesis?
- DODan Olsen
Yeah, the thesis was basically that the, the, there was this term, product-market fit, you know, defined by Marc Andreessen back in 2007, but there was no, like, s- rigorous framework or systematic process for how to do it. And through my consulting and speaking, I realized that I could create that process, and that's the lean product process, which is a simple six-step process that, you know, thousands of companies have followed at this point to try to achieve product-market fit. Yeah, so this is the six-step process. Uh, basically, you start with defining your target customer. You figure out what are their underserved needs. Those two layers form the market part of product-market fit. And then from there, you define your product's value proposition. You figure out what your feature set should be. That's where MVP thinking comes in, so you don't over-scope it. You figure out your UX. You design the UX. You prototype it, and then you go and test it with customers to see how you're doing with product-market fit. And you iterate through a learning loop to try to achieve product-market fit or decide if you have to pivot or, worst case, you know, you pull the plug. But that's basically the process at a high level, um, for how to achieve product-market fit.
- 3:05 – 3:44
What changed in 10 years: PM maturity and the AI wave
- AGAakash Gupta
What has changed in the past 10 years?
- DODan Olsen
I think really two main things have changed. One is when the book came out, product management wasn't as well understood as it is now or valued. And, um, you know, um, in the last 10 years, it's really exploded. People are way more product manager jobs. People appreciate what product managers do. And so, uh, there's more appreciation for the techniques and frameworks in the book, and more and more companies are actually applying it. The other big thing, which we're gonna go deep into today, is AI, obviously. AI, we've, we've been fortunate to live through-- I've been fortunate to live through a few different disruptive waves of the internet, you know, mobile, uh, and now we've got AI, and so it definitely is impacting how teams are creating products, and I'm excited to get into that with you.
- 3:44 – 5:44
Where AI helps—and where judgment still wins in PM work
- AGAakash Gupta
What specifically has AI changed? This feels pretty timeless.
- DODan Olsen
Yeah, it is pretty timeless. I mean, you know, at the end of the day, you still have to understand your customers, and AI's not gonna tell you, you know, about your customers, uh, you know. Um, you still have to figure out how to segment your market. You have to figure out your persona. It can help you brainstorm and come up with, you know, potential ideas and segmentation attributes. But at the end of the day, you still have to get out of the building and talk to people and, uh, to do your discovery research, basically. So it can help you synthesize discovery research, but you've gotta come up with it. Um, and then the next thing is you need to-- Once you've identified the customer problems, you've gotta prioritize which ones are the biggest opportunities. That's where my importance versus satisfaction framework comes in. And an AI's not gonna be able to do that for you. You're gonna have to use judgment. And, um, you know, I played around with GenAI. It's really good for brainstorming and creating convergent, like, divergent ideas, but it's not good w- as good when it comes time to eval- like, kinda converge and evaluate and prioritize ideas. And so that's where you still need to h- you, you still need to do that. And then your value prop. You know, I've tried a couple times to get, uh, ChatGPT to create a product strategy. They kinda sound good on paper, but when you scratch the surface, they're, the, the, the substance isn't really there as much. So I feel like that's the case. And then defining your feature set, that's another place it can help. It can help you brainstorm features, right? You say, "Hey, I wanna solve these problems for customers." It can come up with some really cool feature ideas. And then the biggest place, though, is in that creating the UX, creating the prototype. And we'll get into details of that. But with AI today, you can do it a lot quicker, and if you don't have a designer, you can actually do it now, where before you might not have been able to even do it at all.
- AGAakash Gupta
I think that really is the unlock, right? Is with AI prototyping tools and with vibe coding tools, what used to be PMs sitting in a dock writing things up and then begging their designer for some resources or some time-
- DODan Olsen
Yeah
- AGAakash Gupta
... to do some explorations off of a napkin sketch, PMs are now more empowered than ever-To go ahead and come up with some explorations in the solution space.
- 5:44 – 8:32
Old prototyping workflow and the persistent ‘design gap’ on teams
- DODan Olsen
Yes, it's true. PMs really couldn't do a whole lot without designers. So let me walk you through the old flow. Um, I learned early on in my product career how important UX design is. So in my book, I have a whole chapter on UX design, and I like to describe the different artifacts that people could use and, like, in a preferred order. So I like to categorize them by interactivity and fidelity, as you can see here. We usually start with a product brief or a PRD, right? Some textual document. Um, and then we start doing hand sketches with our teams. We iterate those until we're happy with them, and then we can move on to clickable wireframes if we want. Um, low fidelity. They're low fidelity. They're usually black and white, a tool like Balsamiq. You can test with those. But really the best place in the old workflow to test was when you had clickable or tappable mock-ups.
- AGAakash Gupta
Yeah.
- DODan Olsen
Right? These days you use a tool like Figma to do that. In the old days, you might use InVision with whatever asset your designer gave you. And I've-- You know, for a lot of my clients, I-- this is where the, the rubber met the road. We would do waves and waves of user tests at this stage, and then once we worked things through and iterated to the point where we had confidence in product-market fit, then we would go code the product, and of course, we'd test the line product. But basically, in order to do that flow, you needed to have someone who could prototype on your team.
- AGAakash Gupta
Yeah.
- DODan Olsen
And that brings up something I've been talking about for a long, long time, which is the design gap on many teams. If we just take the high level activities that a product team needs to do to create a product, and we say it's basically defining the product, which is mainly PM's job, designing the product, which typically falls on the designer, and the develop, then basically there's different levels of UX maturity within a team and different org structures that you see. The most common one and level one is basically, "Hey, we've got developers," and that's it.
- AGAakash Gupta
Yeah.
- DODan Olsen
There are a lot of startups that are like that. Actually, a lot of AI startups these days are like that too, because at the end of the day, if you don't have coders, you're not gonna build anything. So that's like the least UX savvy level. The next level up is, hey, in addition to engineering, we have PM, but neither one of them is really a UX expert, right? That's the idea. You're gonna see... You can see when you see products from these kind of companies, you see the lack of UX design come through in their product. Now maybe, you know, one of these two can lean in and help try to cover some or most of that gap. Maybe the product manager, you know, has picked up some front-end skills, some design skills, or maybe you have a front-end coder who's picked up design, right? They might be able to get by, and maybe between the two of them, they can kinda cover it. Frankly, in the early days of Intuit, where I started my product manager career, that's how they got, got by. Before design was a thing, that's how they got by.
- AGAakash Gupta
Mm.
- DODan Olsen
The PMs and the engineers thought enough about UI to get it right that our product was easy, easy to use. But the real thing you wanna have, obviously, is the triad, the trio that we talk about. You've got product management, engineering, and UX who can do that. So, you know, unless you really had level five, it was really hard to like rapidly create good prototypes. The cool thing is with vibe coding, it's completely changed that. One, if you don't have-- If you're, if you're living with a design gap, you as a PM now can actually create those prototypes and you can create them quickly. And even if you
- 8:32 – 12:02
Vibe coding and the new workflow: text-to-live prototypes for everyone
- DODan Olsen
had a designer, we know that designers are busy. Um, they get-- They're on multiple projects, so your project might not be a priority for them. You can now take it on yourself and get it done, and so you can get to a prototype more quickly, even if you have a designer. And so the new workflow is, you know, pretty-- It, it's pretty interesting, is you can just... You know, you're starting with text. Again, you're starting with text, you know, whether it's a product brief or a PRD. That's gonna be the input into your vibe coding tool, and now you're gonna get a live prototype. You know, before vibe coding, people would create what we call HTML prototypes. It would be full on HTML, CSS, maybe a little JavaScript, but no back end if they wanted to really kinda have it be really high fidelity. Nowadays, with vibe coding tools, you can totally have a front end, and you can even have a back end tool. And so you can get quickly to that live prototype to test your idea. And that's one of the key things about the lean product process, Aakash, is you wanna get through all those bottom layers and get to the prototype as quickly as you can, 'cause it's by putting that prototype in front of customers and getting feedback where the real learning and iteration towards product-market fit happens.
- AGAakash Gupta
Yeah. I think that the biggest unlock for people was just testing [chuckles] those clickable prototypes that we were building, whether they were in Balsamiq or Figma. But now what we're seeing with AI prototyping tools like V0... We just had the CPO on the podcast. Lovable-
- DODan Olsen
Yep
- AGAakash Gupta
... Bolt, we had the CEO on the podcast. With those tools is that any of the designer, the PM, the engineer, heck, I've been hearing stories about salespeople and marketers-
- DODan Olsen
Right. Right
- AGAakash Gupta
... can create live prototypes.
- DODan Olsen
Totally. Yeah, that's true. It empowers everyone to do it. Um, you know, and we can talk about, you know, basically-- Especially if there's different use cases too, right? If you're creating a brand-new concept, then it's great for that, 'cause you're starting from scratch. You have no existing design system. You have no existing code base. So you can create it. So it's great for those kind of conceptual new concept ideas. Um, you know, obviously, if you have a designer, the design's probably gonna be better if a designer's involved, uh, versus not. These things will generate kinda plain vanilla UI. And they are getting better and better. You know, I, I've used several of these. I use Lovable and Bolt. I was just... You know, yesterday-- two days ago, I was at this brainstorming session with a client, and the team had come up with this idea, and I s- I wasn't even-- didn't even say anything to them. I just pulled out my laptop and pulled up Lovable, and next thing I was like, "Here's a prototype." And they're all like, "Whoa," you know? So it's a great way to get your ideas out quickly, um, it-- which is super exciting. Now, there is a separate use case where it's like, okay, I've got an existing product. We have an... You know. So one, if you have design systems you need to live with, um, then you can, you know, you, you need to figure that out. Although I was just playing around with some of these tools, and one of them actually let you kind of import your existing design system if you wanted to, which was pretty cool. Um, you know, so it's, it's very interesting. So it's interesting 'cause the, the unit of work before, like at a high level, it was like product managers created text, right? Designers created Figmas, and then developers created code from the Figmas. So the biggest design work product was a Figma board, right? Well, now, you know, that doesn't have to be the design product anymore. You can go straight from text to a live prototype. You can still go to Figma. The funny thing is to see theSee the workflows. You can still go to Figma or back from Figma if you want to. You know, it's kinda interesting that everyone's still plugging and playing with Figma for the most part. Um, and then at the back end, when it's time to integrate with your code, if it's, if you have existing code, then that's, that's a whole other thing. Um, but just to illustrate and get to a prototype and test a product idea, it's really fast and it's great.
- AGAakash Gupta
[upbeat music]
- 12:02 – 14:18
Sponsors break
- AGAakash Gupta
This episode is brought to you by WorkOS. If you're building a SaaS app, at some point, your customers will start asking for enterprise features like SAML authentication and SCIM provisioning. That's where WorkOS comes in, making it fast and painless to add enterprise features to your app. Their APIs are easy to understand so that you can ship quickly and get back to building other features. Today, hundreds of companies are already powered by WorkOS, including ones you probably know, like Vercel, Webflow, and Loom. WorkOS also recently acquired Warrant, the fine-grained authentication service. Warrant's product is based on a groundbreaking authorization system called Zanzibar, which was originally designed for Google to power Google Docs and YouTube. This enables fast authorization checks at enormous scale while maintaining a flexible model that can be adapted to even the most complex use cases. If you're currently looking to build role-based access control or other enterprise features like single sign-on, SCIM, or user management, you should consider WorkOS. It's a drop-in replacement for Auth0 and supports up to one million monthly active users for free. Check it out at workos.com/lenny to learn more. That's WorkO.S. .com/lenny. Today's episode is brought to you by Jira Product Discovery. If you're like most product managers, you're probably in Jira tracking tickets and managing the backlog. But what about everything that happens before delivery? Jira Product Discovery helps you move your discovery, prioritization, and even roadmapping work out of spreadsheets and into a purpose-built tool designed for product teams. Capture insights, prioritize what matters, and create roadmaps you can easily tailor for any audience. And because it's built to work with Jira, everything stays connected from idea to delivery. Used by product teams at Canva, Deliveroo, and even The Economist, check out why and try it for free today at atlassian.com/product-discovery. That's A-T-L-A-S-S-I-A-N .com/product-discovery. Jira Product Discovery. Build the right thing.
- 14:18 – 17:33
AI raises the floor, commoditizes UX, and makes differentiation harder
- AGAakash Gupta
Is there a risk that people jump too quickly into the solution space and don't adequately investigate the problem and solidify that they're solving the right problem with these tools?
- DODan Olsen
I, I, I'm so glad you mentioned problem space. My, uh, my heart is very happy right now. That's one of the things I'm very passionate about. And yeah, you're right. I mean, before vibe coding, you know, solution space thinking is very prevalent. Um, you know, people start talking about, "We should build this feature. We should build this product." Sales teams are asking for features. You know, stakeholders are asking for features. Clients are asking for features. All that discussion is actually in the solution space. And so if we go back to that high level define, design, discover or develop, a lot of PMs are spending their time just managing the Scrum process, and they don't have time to do discovery. And so they end up just kind of building what people ask for, and then it's kinda like ready, fire, aim. You never validated it was a true customer problem. So I do think, one, is our solution space thinking was already prevalent, and now it's easier than ever to create a solution, which is kinda interesting if it's like, hey, if designing it and coding it is no longer the bottleneck, um, then it's like, well, what text you put in is the only thing that makes any difference, right? [chuckles] And how well thought out is who the customer is and what their problems are and what features we need in order to support that. Um, so I do think it will lead towards solution space thinking. The other thing I thought a lot about too is I think it's gonna lead to, um, a higher challenge for differentiation, right? If everybody's using the same tools, you know, it's kinda like raising the bar. You're, you're not gonna see as many bad UX websites if everyone's using this 'cause it kinda follows general principles. You're not gonna get some amazing UI. You're gonna get some plain vanilla UI. You can ask it to copy the UI of another website. So you can copy people. You can create plain vanilla stuff. But if you really wanna innovate in UX, which we can talk about, go a little deeper in next if we want, you're still gonna have to design. But basically, if anybody can create any idea, then what's the differentiation at that point, right? And the only real differentiations are you're solving a problem, a real problem that's an important problem. You're solving it in a way that's better. Um, you have a data advantage, right? That's the other thing people are talking about as they realize AI, kind of these tools are kind of becoming commoditized, then the proprietary data that you have becomes more important as well. So, so I think you're gonna see, [tsking] uh, increasing, you know, challenge in differentiating one product from the other when anybody can create a product pretty easily.
- AGAakash Gupta
Yeah. The way I like to think about it is that AI is really good at pulling people up to the average.
- DODan Olsen
Yeah.
- AGAakash Gupta
You can even see this in a platform like LinkedIn. Like, a lot of the bad content that used to be out there, those people are plugging their content into ChatGPT or Claude, and they're bringing it up to average.
- DODan Olsen
Yep. Yep.
- AGAakash Gupta
But you don't see those people winning, right?
- DODan Olsen
No.
- AGAakash Gupta
The people who are winning-
- DODan Olsen
No
- AGAakash Gupta
... are the people who are, you know, crafting it on their own. It doesn't sound like the-- What I like to c- I think of ChatGPT text as, like, brown, you know? It's like the average of everybody's writing style.
- DODan Olsen
[laughs]
- AGAakash Gupta
Right? There's, there's no-
- DODan Olsen
Yeah
- AGAakash Gupta
... clear, like, red or blue or pink-
- DODan Olsen
Yeah
- AGAakash Gupta
... or, like, different style there-
- DODan Olsen
Yes
- AGAakash Gupta
... that you're getting that pointed voice, right? That's why everybody loves Marty Cagan's articles because it's his voice. He's not writing with AI.
- DODan Olsen
Yeah.
- AGAakash Gupta
And I think that with design, that's the new bar that we're setting with these-
- DODan Olsen
Yeah
- 17:33 – 26:37
How PMs should collaborate with designers (and avoid turf wars)
- AGAakash Gupta
... AI prototyping tools, is that you need to create that differentiated design. So what is the right sequencing? You know, when should a PM be experimenting with prototyping tools, and when should they bring the designer into the process? How do they make sure that that solution phase, which is now totally transformed, is done in a way that both PMs and designers will be happy?
- DODan Olsen
Yeah. Well, I'd love to pick up on what you said, 'cause the way I think about how, like, how is AI helping people? What is it doing? I think of a bell curve, like you said, right? And basically, what it's doing is people that were at the bottom of the bell curve, the floor of the bell curve is now a lot higher. The floor is rising basically, right? Instead of the floor is lava, the floor is rising as gen AI gets better and better, right? And I love the quotes where I was like, "Hey, this is the worst GPT you're ever gonna use in your life." It just keeps getting better. So the floor-- You're right. So the people that were, like, the bottom decile, now they're performing in the second decile or, you know, third decile or something, right? So it's like there's, you know, it's, it's getting rid of that bad low end of the curve, which is related also to the whole speculation about, "Hey, what's it gonna do for jobs?" I remember about nine months ago, people were like-- or a year ago, "Oh, it's gonna kill PM jobs." And in my head I'm like, "I don't think it is." If I think about the jobs that are most at risk, I think it's the developers, the junior front-end developers, because they're just-- At the end of the day, code is text, and if they're in the bottom decile, right, uh, of junior developer coders, and this stuff can crank out third decile, fourth decile stuff, then, you know, it's gonna replace that. And, um, yeah, so that's, that's it. And I think to answer your question about when to bring in a designer, let's get into the, you know, l- I, I think this is where it's important to double-click on UX design. So let me share you my framework. Like I mentioned, when I got into PM, um, I, I realized that, you know, UX design is super important. I mean, it's just really, really important. And, uh, and so I made it a point to learn a lot about it. I basically-- I could write, to put it in the parlance of reasoning here, I could write the best text ever. I could write the best requirements document, the best user stories, but if we fall down on the UX... I think about it like an iceberg, because like an iceberg, there's a part that's above the water that's visible, and it's what most people see and react to, and that's the visual design. That's the part that's above the water. You know, we're all looking at... That's why people love high-fidelity prototypes or live coding things, 'cause it's real pixels, it's high fidelity, it's got the colors, it's got the fonts, it's got the images. But when you use a product that's got product-market fit and it's got a great user experience and is easy to use, it's because the team has, like, taken into account and done a really good job with these more foundational layers below the waterline that aren't as obvious. Starting at the bottom, it's conceptual design. Like, what's the conceptual design, overall design for this, for this product, right? A lot of e-commerce things, the conceptual design is just a catalog plus a shopping cart. It's a very common conceptual design. Uber's conceptual design wasn't necessarily that it was just map-centric. It was like... You know, the thing about a conceptual design, you can design it. You could de- you could describe it verbally. It's like, hey, let's show the user in the middle of the map, let's show the cars on the map where they're driving around, and when the user hails a car, then let's show the car going to them, right? There's 1,000 different ways to implement that visual design, but that's the conceptual design. The next layer up is information architecture. And so information architecture, interaction design, these are the things that these cool coding tools are probably not gonna do as well as, you know, they're not gonna do as well as a top-notch designer for sure, right? Over time, they're gonna get better, like we're saying. But information architecture is, you know, is the way you're structuring the information and organizing it l- makes sense. Like when you go to look for something, is it where you expect it to be? Does the navigation make sense? Do the labels make sense? And then interaction design is basically anything you can click or tap and the flows, right? And so that's the one of the things that comes to mind to me when I vibe code. Like, if you don't tell it the flow, then it, it's probably not gonna get the flow right if you've got a multi-screen, multi-page, right? If you're d- designing one screen, there's not really much flow. But if you've got a multi-step workflow or a multiple feature product where they've got to navigate between it, you gotta think about this. And that brings up something that's really interesting is, like, I always feel like, yes, you should start in the problem space. When you get to the solution space, it's so funny. You see this with all gen AI. You make a request in your prompt, and it gives you something you didn't expect. Like, "Oh, I forgot to say make the grass green," or whatever. And then you, then you edit it, right? And then you're like, "Oh, I forgot to say make the sky blue." And so it's funny. By going back and forth, by going to the solution space, it helps you go, "Oh, I forgot to-- I forgot this one problem space requirement." So it can really be helpful to actually help you explore jointly the solution and problem space to get better requirements and better specs, if you will.
- AGAakash Gupta
Mm-hmm.
- DODan Olsen
So, but to answer your point, you know, if you're building some straightforward pattern, you know, like some e-commerce shopping flow, there are best practices out there. You might try to innovate. If you wanna innovate, then you should get a designer. But if it's pretty, pretty straightforward or you're following an existing design system, that's where you can kinda stay in the guardrails and just kind of replicate what you already have in your product. But if you're really trying to create a better user experience, a better information architecture, better interaction design, or even a novel visual design, then that's when you wanna get the human touch of, like, a 10X designer involved in that. So I think that, you know, uh, theoretically, uh, timing-wise, a PM could go prototype the general functionality and flow with vibe coding, understanding that, hey, this is not, this is not anywhere near the, the final UI per se. It's just getting us the flow and the functionality down perhaps.
- AGAakash Gupta
So what I'm hearing is that there might be a reduced need to bring a product designer into every feature that's shipped.
- DODan Olsen
I agree with you. I mean, right now, the, you know, the, the good news is, again, like, just like product management has risen, the appreciation that to develop a, a, a great product you need a developer, a designer, and a product manager, and ideally they're all very skilled in their domains, and they are ideally collaborating very well together. That's a high expectation. What a function of that now is most teams do have a designer, and to your point, the designer is involved with any UI creation, right? And therefore, they are potentially a bottleneck for any UI creation. And so I think kinda combining what we're talking about, perhaps to your point, the more plain, you know, trivial stuff can be done without them, and we can reserve their time for the stuff that really does require more creative thinking, like creating a new flow or creating, you know, some, some cool new feature.
- AGAakash Gupta
I also think that there will probably need to be a bit of an evolution within the product design field, where I think that it has been a little bitLike you said, a bottleneck for what a team can create. And I think that this next iteration of product design, we're gonna see more and more that product designers have to be able to work with a simple feature and maybe build that out using prompts. You know, we see things like Figma Make, right?
- DODan Olsen
Yep.
- AGAakash Gupta
Where it takes your existing design system, they just prompt it out so that they can do that really quick.
- DODan Olsen
Yep.
- AGAakash Gupta
And then when they actually need to more deeply apply the double design, double diamond design process, they can do that for bigger features.
- DODan Olsen
Yeah. Yeah. Well, it, it's interesting. Yeah, so Figma launched Figma Make, right? So you can go straight from your Figma. The funny thing is everybody else-- There's a lot of people... Figma is kinda the, was the dominant player in the old workflow, right? And there's, that's not gonna change overnight. And so a lot of these tools plug and play very well with Figma. And Figma themselves are trying to innovate and, and make it, you know, make it be the case that, like the last thing they would wanna see probably is designers having reasons to not be in Figma, right? That would be bad for their business. And so I think-- And they have a, a robust plugins and extensions, you know, platform and things like that. So to the extent they can support these, like I'm saying, like I'm, you know, when you double-click and get deep into it, it's like, okay, we need to build a new feature. It's relatively straightforward, but we need it to comply with our design system. You know, and certain companies have really r-r- uh, you know, robust, well-thought-out, nice design systems. Well, if a, if a new PM comes in and says, "I'm just gonna create a vibe coding prototype," and it looks nothing like it, that's gonna be a problem, right? So but like I said, you're already seeing people, people recognizing this. The industry's recognizing that, hey... A-and this is a broader case. Take a step back. You know, everyone's excited about GenAI, and when you play around with GenAI, like I did the whole action figure thing that everybody was doing, and I, I decided to do a Star Wars figure 'cause I'm a Star Wars fan. And what I found was it was good, but when you wanted to change one thing, it would like change the rest of it somewhat. And I, I view it like rolling dice. I call it a re-roll. Like y- every time you make an edit, it would like re-roll the whole thing, and you, it might fix the one thing, but it's like whack-a-mole. It would mess something else out, right? And so kind of a general philosophical concept that I'm pushing these days is now that we have GenAI that's generating new stuff, we need a good edit UI, uh, edit AI. We need a good edit AI, right? That's what we need. Because-- And that's the thing is, hey, cool, honor my design system, but just generate this stuff in it. So you're seeing tools like being able to support that use case, recognizing that, yeah, vibe coding's cool when you have starting from scratch if you're a startup or a brand-new product. But for these companies that have thousands of lines of existing code, robust design systems, how do we use those tools there? I think the support is getting there, and I think that's where the designers, hopefully design tooling ecosystem will also support that, which I think it will.
- 26:37 – 36:27
Tool landscape: coding assistants, prototype generators, and ‘reverse prototyping’
- AGAakash Gupta
So what are the specific tools you'd recommend for vibe prototyping?
- DODan Olsen
Yeah, I mean, I think, you know, it's interesting. Uh, the cha- space is changing very quickly. Um, but as of right now, there are kind of like the more hardcore, like if you are a coder and know how to code, there are tools, you know, like Cursor that are in that kinda camp. And there are other ones, like if you're more like, "Well, I don't really know how to code. I wanna, um, care more about prototyping," you know, there's things like Lovable and Bolt that are kinda live in that camp more, right? And, uh, and then there's still-- And there's another category, and those are great. Like I said, if you wanna build a new product from scratch, you can put in a prompt. It'll create a new product from scratch. Everything's cool. The other thing is, like imagine you're a PM, you don't have a designer, and you need to modify or enhance an existing product or feature. You know, in the old days, you might take a screenshot of it and then do what I call Frankenstein it, like hack it up. [chuckles] You know, like put in some weird... You try to make it like-
- AGAakash Gupta
Oh, yeah
- DODan Olsen
... look as good as you can, but you're adding button, janky buttons and dropdowns. Like it gets the job done. But the cool thing now is to support that specific use case, where instead of starting with text, you can start with a screen grab of an existing product, and then it creates basically a Figma-like artboard from that, right? And so then you can go, "All right, I'm gonna change this text. I'm gonna add a button here. I'm gonna do this." And then you can, just like with Figma, you can create a interactive prototype, so when somebody taps here, it goes here and goes there. So the tools in that specific category are Visily, Uizard, which got acquired by Miro, Magic Patterns, and like UXPilot. Those, those like, they're kind of, you know... Um, and you know, like Magic Patterns will actually generate code for you. The other ones will generate the artboard for you. You can then export them to Figma. So again, there's multiple workflows depending on where you wanna take what they create. But they're focusing more on that use case of, hey, I wanna, you wanna create an editable... [lip smack] You know, you don't have the original Figma. If you had the original Figma file, you'd go, and y- that would mean you probably had a designer. You'd go and edit that. So if you kinda live in a Figma-centric world, your workflow's probably still gonna rely on Figma-centric when you're trying to extend or enhance existing functionality. But if you're a PM that doesn't have a designer or need to do that, you can use these tools. So there's, there's kind of those three categories of tools. The hardcore vibe coding tools, where you're really gonna get in the code. There's the lighter weight ones when you're just trying to prototype. And then there's these other ones that focus on this, what I call reverse prototyping use case. You basically suck in a screenshot, and then you can mo- make it really easy to modify and edit it and create, uh, a clickable prototype from that.
- AGAakash Gupta
So if I had to summarize what we talked about in terms of product management in the age of AI, GenAI is changing most of the steps, but the step that's most consequential is the change in prototyping. And PMs need to learn how to use these tools, but they also need to learn within their organization how to collaborate with designers in a way that it doesn't feel like it's stepping on their turf. Is that a fair summary?
- DODan Olsen
I think so. Yeah. I think so. And I think that, that, b- I'm glad you brought up the turf thing. I mean, um, you know, at different companies, sometimes PMs and designers do have minor, you know, some degree of clash or ve- you know, kind of, um, uncertainty about responsibility or overlap and ownership and things like that. So you're right. When you've got tools that can suddenly generate designs, then you wanna be careful about that. And you know, I think good PMs, w-when you do have a, a designer you're working with, and in the interest of time because they're busy or you just wanna get your ideas out there, you would create a wireframe or a mockup or nowadays a vibe coding prototype. It's always with the intent that if it needs to go through a design process, this is just kind of, you know, directional and illustrative. And of course, we would love to get your design expertise to make this even better and add some, you know, some truly unique components to it. So I think that's, that's kinda the, the spirit in which PMs could be approaching it if they're working with designers.
- AGAakash Gupta
I think, yeah, when you're working with designers, a lot of it is also just about being clear and upfront about your intentions, right?
- DODan Olsen
Yes.
- AGAakash Gupta
I'm not trying to say this is pixel for pixel what we should build.
- DODan Olsen
Yeah.
- AGAakash Gupta
But like Dan was saying earlier, to explore the problem space effectively, I wanted to also explore the solution space.
- DODan Olsen
Yep.
- AGAakash Gupta
That was giving me good signals. And so I put together, you know, an AI prototype, but depending on where we see, you know, the impact of this feature and your resource availability, we could go through more design cycles on this, or we could just do a minimal, you know, making sure it conforms to our design system.
- DODan Olsen
Yeah. I think, yeah, exactly. That's the intent. That's the intent of doing it. And it's funny because the way I think about it, too, going back to that lean product process, like, of course you should start off with discovery research. You should talk to customers to understand their needs and preferences and what they're using today and things like that. There's kinda like, that gets you to a certain level, right? There's diminishing returns. After you talk to your 20th customer, you're probably done seeing new patterns, so there's diminishing returns. That's as far as you can go with customer interviews. The next way to kinda really de-risk and increase your confidence is with a clickable interactive prototype, and so that's why they're so valuable. Once you get to a clickable valuable prototype, and then the more, the higher fidelity, more realistic it is, which by definition these vibe coding tools are creating, you know, stuff that's actually in a browser or stuff that's actually in a mobile app, then it, that's, that's when you like... The customer goes, "Oh yeah, I know I asked for X in discovery, but now that I'm going through the workflow, I forgot that you also need to be able to export this to Excel," right? So they can-- So like it really unlocks the next level of customer validation that you can do. And to your point, I think there's value in doing that. Uh, if the PM can drive a lot of that and de-risk it and iterate it without taking design's time, that's great. And but then again, to your point, then it's like, "Hey, we validated these problems with the customers. We validated this high-level flow and functionality set with the customers. Now we'd love to work with you to make it a great UX and a great UI," you know?
- AGAakash Gupta
AI evals are one of the most important skills for PMs, and I know you know they matter. The question is, are you doing them right? Most teams are winging it with basic metrics and hoping for the best. Meanwhile, the teams that actually ship reliable AI, they've cracked the code on systematic evaluation. Today's episode is brought to you by the AI Evals for Engineers and PMs course by Hamel Hussein and Shreya Shankar. This live Maven course will teach you the battle-tested frameworks from Hamel and Shreya, who are the engineers behind GitHub Copilot's evaluation system and 25-plus production AI implementations. Four weeks, live instruction. Next cohort starts July 21st. Start shipping AI that actually works. Enroll at maven.com with my code AG-PRODUCT-GROWTH for over $800 off. That's A-G-D-A-S-H-P-R-O-D-U-C-T-D-A-S-H-G-R-O-W-T-H. Today's episode is brought to you by the AI PM certification on Maven. Run by Miqdad Jaffer, who is a product leader at OpenAI, this is not your typical course. It's eight weeks of live cohort-based learning with a leader at one of the top companies in tech. OpenAI just doesn't stop shipping, and this is your chance to learn how. Run along with product faculty and Mo Ali, the course has a 4.9 rating with 133 reviews. Former students come from companies like OpenAI, Shopify, Stripe, Google, and Meta. The best part? Your company can probably cover the cost. So if you want to get $500 off, use my code AAKASH25 and head to maven.com/product-faculty. That's M-A-V-E-N.com/P-R-O-D-U-C-T-D-A-S-H-F-A-C-U-L-T-Y. And ideally, you're bringing design in early in the process where they can help feedback whether they need to be involved in putting those AI prototypes in front of customers. And if you have a UX research team, whether they can decide. So I think it's almost like before the PM goes off and does all these things, making sure that they have the rec checkpoint to say, "I'm going to be going doing these things. Do you wanna be involved or even lead this process potentially?"
- DODan Olsen
Yeah, that's a great point. I mean, we're kind of, we're k- we're kind of painting a picture where there's, like, limited design bandwidth, and that's why we're kinda going down this path. If there, if that's not a good assumption, if there's adequate design bandwidth, then yeah, it'd be great to partner with UX on this. Um, you know, I think what you'll see is the forward-thinking designers are gonna lean into all these tools and processes. So it might flip back where it's like we're gonna go back to, yeah, you know who's better at creating these prototypes than a PM? A designer who's really good at vibe coding. So [chuckles] you know, like, if they embrace it, right, some people are sticking their head in the sand. Other people, if they embrace it, you know, they could theoretically... And I, I, I, you know, now that we talk about it like this, I am confident in the next year, definitely two years, there will be, like, power AI design tools for designers. You know what I mean? That are gonna have, like, power features and stuff that PMs are gonna be like, "Well, I don't even wanna deal with that. What do you mean? Padding and margin? I don't wanna deal with all that." You know, I don't, you know. So, uh, so I think that, that you might see it bounce back, uh, it, with, with forward-thinking designers where the tools have emerged enough, um, where people, you know, people can, can do that. The funny thing is, you know, we're kinda skipping over... The other thing, another thing I try to talk about is just, like, I've always been a big fan and believer in wireframes because for me, that was the quickest way to kinda get your ideas into a solution UI, like a h- a low fidelity one. One, for understanding with your team and stakeholders, but two, so you could have initial high-level discussion with customers. With tools like Sketch and Figma, because that's the tool and most designers, they go straight to high fidelity, and so we're kinda missing... A lot of times you miss that high-level discussion about the flow and things like that. The funny thing is some of these, these design tools I was talking about, not the lovables and things like that, but the other category of tools like the Visily and the Uizard, some of them actually offer a wireframing mode where they actually, like, deliberately stay low fidelity so that you can kinda work through those issues if you want.
- 36:27 – 42:17
When to stay low-fidelity and how to prompt for better UX outcomes
- AGAakash Gupta
When should people stay deliberately low fidelity?
- DODan Olsen
I think that, you know, it's, it's interesting now 'cause with vibe coding it changes the calculus. It used to be before it was like, hey, the presumption was it would take a lot of effort. And the cool thing is when you do a vibe coding, it's usually a pretty plain vanilla UI. It's not getting crazy with some crazy fonts and colors and stuff that are gonna distract, right? So in a sense, they're kind of low fidelity from a visual design standpoint, but they have to be high fidelity, kinda ha- if you're gonna put a flow in there, they have to be a flow. But what I tell people usually is like, if you're have... Th- the number one situation is, one of the top situations is, this happens a lot. Say your team is trying to figure out the scope of your MVP, and there's one feature that's on the bubble. You know, half your team thinks it needs to be in there for the MVP, this is gonna be a horrible MVP without it. The other half of your team thinks, "Nah, we can get by without it." And usually it's not the team. Usually the team's cool without it. It's some senior stakeholder pounding on the table saying, "I can't believe you're not gonna have this feature." Or sales, "I can't believe you're gonna ship without..." You know, "It's gonna be horrible." So in those situations, that's when I like to, in the interest of time, quickly create a wireframe without the feature, and then say to the stakeholder who's pounding on the table, "Hey, I-- you're so convinced this is a critical feature. I bet you that if we talk to 10 customers, you're gonna s- you're gonna expect that like eight or more are gonna complain about where the heck's this feature," right? And they'll usually say, "Yeah, you're right." I'm like, "Okay, cool. Tell you what. We'll do a quick wireframe. We'll go talk to 10 customers, and if, if like, say, five or more m- complain, we'll be the first ones to say you're right, and we'll put it in. But can you agree that if like, you know, uh, you know, uh, not, uh, like only one or two complain, that we'll be okay without it?" And that's how you set up the rules to try to get progress without taking the hit on that. So wireframing in the past was a great solution to that, and then you prototype it without the feature, and you see how many people go, "Hey, where's this feature?" You know? You don't ask them if they want it. You see if they complain if it's not there. So that's the idea. Theoretically, with vibe coding, you could kinda get to that same point as well. The other use case is if you have like a very complicated flow, like a multi-step workflow with many branches and conditionalities and things like that, um, you might wanna kind of do some UI exercises with wireframes to kind of validate the flow and things like that, right? Um, the interesting thing, as I said that out loud, is if you have a lot of conditionality, a live prototype then might be able to handle that much better than, you know, a clickable prototype. 'Cause a clickable prototype, you've gotta ki- you can't put like an if statement in there, right? You've gotta... You click on this, it's gonna go here. You're gonna click on this, it's gonna go there. But so... So I think that's typical uses of wireframe is... And also just to get the team on board. So I do think that live, live, uh, vibe coding prototypes can, um, replace some of that stuff, um, for teams. But I do think it's important to kinda at least think through. And that's, you know, we didn't get into when you're inputting a vibe coding tool, if you wanna get a good UI or UX out of it, you need to specify those things, right? You need to be like, "Hey, this is gonna be a five-page flow. Page one should have these things. Page two, three, four, five," right? You need to tell it the flow. If it's gonna guess the flow, it's probably not gonna be great. Um.
- AGAakash Gupta
Yeah. So I think that low f- starting low fidelity with your thinking is still important. Still thinking about, okay, well, these are the, the screens that we want. These are the conditional if statements that we want, and potentially using a vibe prototyping tool to exercise those low fidelity. You know, before we had to draw all those boxes on napkins or draw them in Balsamiq. Vibe coding can get that to you pretty quick.
- DODan Olsen
Yeah.
- AGAakash Gupta
So it sounds like s- don't skip the low fidelity step just because we can go high fidelity. Um-
- DODan Olsen
Yeah
- AGAakash Gupta
... and then potentially what I think people miss with the vibe coding tools is always input a screenshot of your product. Always give it your design system so it looks like your product. But you don't need to jump to that right away.
- DODan Olsen
Right. Right. Yeah. I mean, the thing is it's tough. So back to that iceberg. If you tell it to vibe code something, like I did, when I did it in like five minutes for that thing two days ago with that team, it of course put in some like, you know, alert icons, notification icons that were red and this and that, you know. So it, it, it kinda can't help itself from doing some kind of visual design in high fidelity, and then people can fix on it, fixate on that, and say, "Oh, that's a cool feature." And so, you know, another general principle, it, it, it's not so much low fidelity, is when you're designing something, like if your page has different components on it or widgets on it, like what is the goal or point of each of those widgets? Right? That's the key, right? So if you're creating some like dashboard that's gonna have four widgets on it, if you were just to kind of leave that like a Slack variable for the vibe coding, say, "Hey, I want a dashboard with four graphs," it's gonna pick four random graphs, right? But if you say, "Hey, I want a dashboard with four graphs. The graph one should really show user engagement over time. Graph two should show, you know, whatever." You know? So it, it understands the intent. This happens a lot, where people just design things, and that can happen when there's not a good handoff between PM and design, where the designer's just like kinda making stuff up for the design. Um, so it's really important when you do UI design that you understand what problem is this solving and what's the goal of each UI component. What's the goal of this page? What's the goal of this component on this page? You know, things like that. The other thing it can really help with is creating realistic copy. So like that's where it can really save you time as well. Like, you know, back in the old days, designers would put like lorem ipsum, dolorum, like just placeholder text. We... Uh, that, that ship sailed a long time ago. And but, you know, if you're a designer, you gotta put in some data. The d- the, the, you know, those vibe coding tools are really good. Like I had to create a dashboard to track a VC startup portfolio. It created like one AI startup with a cool name. It created a health tech startup. It created, you know, it created like multiple startups with good names in different spaces and, and just kinda auto-generated some data that I didn't even tell it to do, which was pretty cool.
- AGAakash Gupta
Yep. So you can use it for better placeholders as you're going higher fidelity over time.
- DODan Olsen
Yeah.
- 42:17 – 52:56
User testing fundamentals: learning loop, methods, and when to use each
- AGAakash Gupta
So one of the things we, we've basically narrowed in on here is that there are gonna be a lot of PMs or even engineers at startups that are gonna be getting more into the design and user research process, and so I wanna educate themTalk to us a little bit about these timeless principles. How should people be doing user testing?
- DODan Olsen
Definitely. And that's what I'm-- another trend I'm excited about, Aakash, is I think in the last, you know, six, seven years, people have realized the importance of user research. You're seeing more UX research jobs. There's research confer- a lot more research conferences. The value of it, you know, companies like Dovetail, UserTesting is obviously huge. But it's really exciting, and for me, I was, I was very lucky. My first PM job at Intuit, Intuit valued market research so much that we actually had a PhD in market research on our Quicken team. [chuckles] And so I learned from her how to do best practices in qualitative research and survey design and all this jazz. And so it's cool to see a lot of people wanna do it, but to your point, they may not know how to do it. The cool thing I will say at the beginning is it's one of the easiest things to do. It's just talking to other humans. Um, you know, and I'll give you some tips on how to do it in a sec. But why are we doing it? So you can see here, this is my kind of learning or iteration loop that I like to use. Um, a lot of people may be familiar with build-measure-learn from Lean Startup. That was cool 'cause it got people realizing, hey, we need to iterate. We're gonna build, we're gonna learn, and we're gonna iterate. Um, couple things I don't like about it. It starts with build, and at least until vibe coding came along, that was a very expensive step, right? And if you got, if you gotta go into production code, it's still an expensive step. So I like to start with hypothesize. That's what we do. We form hypotheses. We design ways to test those hypotheses. In the product world, they're often prototypes. And then we go out, we test it with customers. From the customer tests, we gain learning, and then from that learning, we revise our hypotheses, and then we go through the loop again. We revise our prototypes, you know, and we go around. And I've seen this work so well. Um, if you do this methodically, you can really, like, iterate your product and, and achieve higher and higher product market fit. And so part of the key component is once you do have the prototype, however you created it, with your designer, with one of these vibe coding tools, that once you have it at the top of the process, it's time to go and ta- close the loop with customers and go talk to customers. And so, you know, when you have your prototype and you wanna test with people, there are fundamentally three high-level different ways of testing it. The first is in-person moderated, where you're actually in person with the person, next to them, you know, with the prototype on their device, their browser or their, uh, you know, their mobile app, mobile phone. Um, that used to be very common. [chuckles] Obviously, with COVID and Zoom and things like that and the convenience, it's, it's much more common to do the second one, which is remote moderated. And the, the-- this is-- remote moderated is totally fine, right? Back in the old days when the screens weren't as good, the tools weren't as good, the bandwidth wasn't as good, the technology could get in the way sometimes. But these days with Zoom, you know, and a, and a, an online prototype, it's pretty easy to have the person share their screen and, and conduct the interview. And then finally, there's remote unmoderated. What does this mean? This means that there's not an actual moderator there. So this is like a recorded session, like usertesting.com, and the way this works is you get your prototype set up, and then you ask people, like, to complete a series of steps or tasks, and you might try to ask them for feedback after each step or things like that. But you kinda really need to put a lot of thought into that script because people may get stuck, they may get off script, they may, you know, they may, may follow it, they may not. So the advantage of remote unmoderated is you can do it quickly, [lips smack] and I think that's, uh, another interesting area, Aakash, where you're seeing AI may come in to try to, like, do that more and try to, like, automate insight gathering. You know, like if, if, "Hey, go take this vibe coding prototype, take this script, go" you know, "Here's a, a recruited set of users. Go do unmoderated with them, have them schedule it, have them run it. Send-- Do text to, you know, do video, you know, speech to text. Do analysis on that and give me the data that..." We are not quite there on that, but that's a future thing I could see happening. But I will say that that is-- the advantage of that is kind of getting a lot of data quickly. The disadvantage is you can't ask why, you can't clarify. And so, you know, the, uh, moderated at the end of the day for me is much more valuable than unmoderated. But unmoderated can be good, especially for usability issues. If you know you've got some usability issue on a certain page but you're not sure why, you can point a bunch of people at that flow and try to figure out why. I like to talk to people in waves of five to eight. You know, there's a lot of old school usability research about that kind of a number. You know, um, even if you talk to 10 or 20, it's still not gonna be, like, statistically significant. But after five or eight, you usually start to get diminishing returns. You've identified the main things that you're gonna learn there. So that's how I like to do it, and then basically, you take your prototype and you test it with people in waves of five to eight.
- AGAakash Gupta
So when should people be thinking about what's the flowchart for if this, then in-person moderated, if this, then remote moderated?
- DODan Olsen
Yeah.
- AGAakash Gupta
How do you think about the pros and cons of these different options?
- DODan Olsen
Yeah. I think that the earlier in your process and the earlier you are in, like, creating this new product, the more you should lean on moderated because you don't even know what questions to ask. You don't even know how to write the script, right? This is a mistake I see people make in user research. It's this allure of, "Hey, instead of talking and in- doing a one-hour interview with 10 users that'll take me 10 hours, why don't I do a survey of 100 users?" Sounds great, right? The math sounds great. But the problem is you're not gonna get the same quality of data. You can't ask follow-up questions. You can't see what's going on, right? So, um, I think in general, when you're trying to achieve product market fit, moderated is much more valuable. In the specific use cases of, okay, we know we have a usability-- we have a product that's live, and we have a usability issue, let's go do remote unmoderated to figure out what, how, where people are getting tripped up. That's one specific use case. Um, [lips smack] you know, I guess you could theoretically, if you were late in the process, say you, like, worked through a lot of stuff and for some reason you just wanted to get a lot more customer feedback quickly, then you could do remote unmoderated on, like, a mature, you know, mature live prototype just to kinda get some, see if there's other feedback, right? If you're worried, maybe if you're worried about browser compatibility, you wanted to test on a range of browsers or a range of devices, that might be something like that. So the general theme there is kinda like-Yeah, testing different conditions to understand what's up. So in general, I'm a big fan of moderated, and, uh, I would only use unmoderated in those kind of specific use cases. Um
- AGAakash Gupta
What do you lose in a remote unmoderated interview versus an in-person?
- DODan Olsen
Well, you, you know, th- that's a great question, Aakash. And they used to, used to lose more because like I said, before, like the tools weren't as good, so sometimes like the s- the, the prototype was bigger than the screen, so they had to scroll up and down. They couldn't see the button. You'd get, you'd run into technical issues like the connection. All those things are gone nowadays. You know, you just have a Zoom and you're good to go. So the last thing you really u- lose is when I am sitting next to you, it's very subtle, but when you get good at user research, when I'm sitting next to you, above and beyond whatever I see on Zoom, I can pick up on little subtle things like you might sigh.
- AGAakash Gupta
Yeah.
- DODan Olsen
Or you might tap your hand, or it might be some other non-verbal body language that's letting me know, oh, you really like this, or oh, there's a problem, or you're concerned or you're confused. I can kind of see where your eyeballs are going, too. It's, it's very subtle, but you know, it's like that extra little 10, 15% of information fidelity. Um, it's not critical. It's not critical by any means, but, uh, if you're a talented researcher, that helps give you an edge and, you know, you can pick up on things a little bit.
- AGAakash Gupta
Yeah. I always go back to some of the best research I've done was when I was working on Fortnite, and we always brought people in to our campus to actually watch them play because-
- DODan Olsen
Yeah
- AGAakash Gupta
... you learn so much more, especially when it comes to a game. Like, are they throwing the controller? Are they rage quitting?
- DODan Olsen
Right.
- AGAakash Gupta
We don't want rage quitting. Or-
- DODan Olsen
Right
- AGAakash Gupta
... are they having so much fun they're at the bridge of their seat, right?
- DODan Olsen
Right.
- AGAakash Gupta
You might be able to understand that over Zoom, but the-
- DODan Olsen
Right
- AGAakash Gupta
... amount of texture you get-
- DODan Olsen
Right
- AGAakash Gupta
... is like, I would say like 50%. Like you get a lot-
- DODan Olsen
Yeah
- AGAakash Gupta
... less texture there.
- DODan Olsen
I, yeah, texture's a good term for it. Two other thoughts I have. One is I forgot to say like, I s- you know, I started my PM career at Intuit, which was like a pioneer in usability testing. We had like these, you know, back in the day, a million-dollar usability lab. It had like a one-way mirror. It had like a camera on the person's face. It had a camera on the screen, you know, back in the day. So you like, you know... But it's funny 'cause that's a little bit like a lab rat thing. So they're coming into your thing. The other thing it made me think about, Aakash, which we also did, which is also important, is are you watching them use the product in the natural environment, like in situ, right?
- AGAakash Gupta
Yeah.
- DODan Olsen
And so the big thing that Intuit did was they would do follow me homes. Like back in the old days, it was like a box shrink-wrapped software you'd buy at the store. So Intuit employees would like camp out at Fry's and Best Buy, and they'd like, you know, when someone checked out, they're like, "Hey, I know this is kind of weird, but can I follow you home and watch you install Quicken and see?" And then you realize, you know, like, oh, their disk drive's over here or their printer's over here and they have internet issues. Like, you know what I mean? So, so that's the other thing you lose if you're, it, there's, it's funny, w- I guess with an in-person moderated, there's in-person moderated in the lab and there's in pros- in-person moderated in the field.
- AGAakash Gupta
Yeah.
- DODan Olsen
And so I do think for certain product types, in the field observation is, is very relevant. I could totally see it for Fortnite, like getting closer to what it's like in the real conditions would make sense.
- 52:56 – 1:08:37
Running great interviews: structure, do’s/don’ts, and note-taking system
- AGAakash Gupta
So a lot of PMs, they'll be new to this. Like UXRs and designers have been doing this. They haven't been doing this. So can you walk us through what is the timeline of a user testing session?
- DODan Olsen
Yeah, I mean, I, and th- and that's, and that's the thing. My book's called a playbook 'cause it's intended to give people like follow these steps. Obviously, the details are different depending on what your product or market area is, but I try to put a lot of practical tips that I, you know, learned along the way. And I mentioned I started, you know, doing user testing at Intuit, where it was like this million-dollar lab. I went straight from Intuit to like a early-stage startup. They didn't have a lab or anything, and so I coined the term ramen user testing 'cause you don't need all the fancy stuff. It's basically like you, the user, and their laptop or their device, and then the prototype, and that's basically it, right? And so just to kind of give people a starter skeleton of what the timeline, like how, how to run this thing, um, this is, this is what I recommend. I, you know, I, I don't like to just jump right into the prototype for a few reasons. One is, um, you know, talking to someone to share your opinions about some new product you may or may not have used, especially if it's a new product concept, it's not an interaction people have every day. It's kind of weird, and especially people that don't like to talk to strangers and stuff. [chuckles] You're like, "Hey, can you..." You know. So y- there is value in, in just getting, warming up the person, letting them know that you're a fellow human who's not gonna be, you know, rude to them, and you're nice, and you're asking about their day, or talking to them about what's going on, just to build some rapport. The more rapport you can build, you're kind of investing in the gas tank for later. You're gonna get more returns out of it, right? And then also, this is where I like to just start asking, "Hey, how do you get this need met today? What are you using? What have you tried before? Why do you use the one you use? What do you like about it, what you don't?"I get hired by clients all the time to do-- to apply my lean product process. I almost always learn about some competitor they didn't know about in this discovery research at the beginning. Like, "Oh, we never heard of those people," right? Or you also might find out that it's not like some other software competitor, but like, "Oh yeah, I just do that with Excel or CSV export or duct tape them." You know, it's like whatever the MVP substitute is. So then once we've done that, um, then I like to just spend, you know, whatever the appropriate amount of time is, you know, usually at least 30 minutes, maybe more if it's a big, robust prototype, where I show them the mock-up or product, right? And then the thing is, [lips smack] uh, this is a mistake I see, you know, some, some PMs do that aren't as familiar with this, is they try to guide the user through the thing. It's like, "Okay, first I want you to register," and then they register. "Okay, now I want you to go try to book a hotel. Now..." You know what I mean? Like they're kind of like directing the user, and the thing that occurs to me, like that is not the real world. The real world is the person's gonna be sitting there with your product in their browser on their phone, and they have to figure it out.
- AGAakash Gupta
Yeah.
- DODan Olsen
So I like to be as real world-- I just shut up. I'm basically like, "Hey, here's our new product." I'll pull up the prototype on the laptop. I'm like, "You know, uh, I'd love to, you know, I'd love to see you, you know, interact with it." And then what I do is I give guidance. I'm like, "Hey, you know, like most people, um," you know, it's, I call it think-aloud protocol. It's on the next slide. But basically, most people are not used to sharing their thoughts out loud with a stranger when they're looking at some weird new product concept, right? So you have to be like, "Hey, share your ideas." And so I'll be sitting there quietly, and sometimes it's very-- it's like this awkward silence and I'm like, "So do you want me to register?" And then I just give my pat line. I'm like, "Do whatever you would do if you're home with your s- uh, with by yourself with this product on your computer."
- AGAakash Gupta
Okay.
- DODan Olsen
And they go, "Okay, so, so I'll register then?" I'm like, "Do whatever you do." And they realize, okay, it's not gonna be a test where I'm gonna guide them through the rat maze. They gotta figure it out, and so they'll keep going. So you may be wondering, "Well, shoot, Dan, you just sit there quiet the whole test?" No. What I do is I wait to observe something noteworthy to pounce on, and that's where being in person, that texture you said, that's where you might pick up some sigh or some little thing where you can tell they're like struggling. Obviously, you can tell if their mouse is looking around. They're looking around, they can't find where to go. So I let them s- you know, I let them flail for a little bit, then I'll be like, "Hey, can you tell me what's going on?" And they'll be like, "Well, you know, I'm trying to find where I can sort by blah, blah, blah." And then I go, "Oh, okay. You know, can you tell me about that?" So I jump in and we talk. So I don't know a priori ahead of time what we're gonna talk about, uh, what's gonna come up in the user session. Then I'll-- we'll share in a minute if we have time here a framework for how to capture this information systematically. But when it comes up, we go deep dive on those things, and then we kind of go. Now if at the end... And so then I don't break character. I wanna be at like, like I'm not even there, right? And I ask questions. Now at the end, the very end, the wrap up, it's important to do a wrap up. You know, they might even be like, "So I was trying to click here. It doesn't work. Is this broken?" You know, I'll be like, "Sorry, I can't answer that." You know, hopefully the engineers are watching it. If it's a live product, you want the engineers there too so they feel the pain. But at the end I'll be like, "Hey, you know, when you were trying to do that thing? Yeah, that's our bad. That's a bug. Sorry about that." And then if for some reason, like there was a key screen or page they didn't get to, I'd be like, "Hey, I wanna just... Let me orient you to this page. Here you go. Can you please give me your feedback," right? Just to make sure you get that. And then at the end, I found it really important to basically ask people, you know, how likely would you be to use this product? Because I learned the hard way that, like, usability feedback, usability-- poor usability can prevent you from having product-market fit, but just because you have good usability does not mean you have product-market fit. They are orthogonal at the end of the day. One can block the other, but they're orthogonal. Because I was working on a product and, you know, I tried to do my best on UX design. We did the V1, we launched it, ran some user tests. There were some UX issues, there were some bugs, there were some, like, we need more instructions, we need examples, some copy needed to be fixed. So after-- And we were a really fast startup. So after about the 15th test, we had fis- fixed all the questions and complaints that had been brought up. So starting about the 15th user test, people were just getting through the new user flow. They were getting to the main page, no issues, no complaints. So I was feeling pretty good. I'm like, "All right, cool. So how likely would you be to use this?" And to my surprise, like about 20% of people said, "I would never use this product," even though they had not complained, issued a single complaint, right? And what it turns out is there were different mental models for how to solve this problem that I didn't know about. And so once I discovered that, then now in my discovery research going forward, I ask people, "Hey, can you tell me how you like to get your news?" And I realized, hey, there's three different ways people like to get their news, and we had kind of like subconsciously designed it for the way me and my co-founder like to do the news, right? [chuckles] And then, uh, we realized it's not gonna be the case. So, so it's a good reminder, you know, that you need to-- the usability if-- You know, what I didn't say about the pyramid is any one of those layers, if it's off, really off, it's gonna prevent you from achieving product-market fit. They don't have to be perfect, but if you have a bad UX, it's gonna do that. So that's the thing.
- AGAakash Gupta
What are the do's and don'ts of these interviews?
- DODan Olsen
Some do's and don'ts. You know, you know, the other thing is most people, uh, remember what their mom told them. If you can't say something nice, don't say anything at all. Like, they're very nice and, and so they're very positive a- about your prototype. Like, "Yeah, this is great. Oh, this is great, great, great." Right? So y- what you need to do is like let them know, and I can do this as a consultant, like, "Hey, the whole reason we're getting feedback is we wanna know, figure out how can we make this product better for you and other customers. So if we get through this whole test and you haven't told us anything we can make better, that actually doesn't help us out. So you're helping us by telling us how we can make this better for you. And by the way, I don't even work at the company. I'm a consultant, so you're not gonna hurt my feelings. Like, please don't worry about hurting our feelings. By telling us constructive criticism, you're actually helping us." And then I mentioned the think-aloud protocol. So most people, I like to prime them at the beginning and say, "Hey, I know this is not typical, but as you're going through the prototype, could you just try to verbalize the thoughts that are going through your mind?" And you usually have to remind somebody 'cause they start just quietly clicking and doing stuff. I'm like, "Hey, can you let me know what's going through your mind as you go?" So that's the idea. Um, and then it's really important to take notes, um, because there's so mu- if you were taking notes and paying attention, there's so much information you can extract even from one user interview, and different people, um, are gonna see different things. And that's why it's also important to do like a review right away. Like you might line up-- Like I like to like line up a power day of like three to fiveUser interviews in a day, we go through them all, and then we have a team meeting at the end, and we debrief and, and different people see different things. Now, if you're, uh, not a super experienced moderator, it might be very overwhelming to try to moderate the test and take your own notes, so it's perfectly fine to have a separate note-taker if that's kind of the situation they're in.
- AGAakash Gupta
So those are the dos. What are the don'ts? What are the big mistakes people are making in user interviews?
- DODan Olsen
One is that they try to help the user get through the user test and have a good experience, right? And I understand if you're the PM and this is your baby, you're kind of secretly rooting for the user. Um, but, you know, again, your product needs to stand on its own, so you wanna be a scientist, an objective realist, and try to detach yourself. Um, I've seen some PMs that go as far as to, like, put their hand on the person's hand on the mouse, move the mouse, and click the button for them. Like, it doesn't count. Like, you know, if, if you're doing it for them, it doesn't count. Um, you obviously don't wanna get defensive or blame the user. That usually doesn't happen, but every once in a while it does. And then the biggest thing is actually how you ask your questions. And so, like, you know, at the end of the day, if you're, if you're a PM or a developer or designer and you're like, "You know, I really wanna do UX design, but I'm just feeling nervous about it or not confident," I encourage you to try. It's one of the easiest things to do in the product world. It really involves active listening, right? You just let the person talk. You can reflect back what you heard to probe more deeply. Ask why, you know, why is that? Things like that. But the biggest thing is how you ask your questions and the difference between leading or closed-ended questions. So if I say, "Hey, that was easy to use, wasn't it?" That's an example of a leading question. It sounds like a question, but it's not really a question. I'm putting the-- I'm really pressuring the person to give a desired answer, which is yes. So you definitely wanna avoid leading questions. The other thing is not as bad, is a closed-ended question. If instead I said, "Did you like that feature?" What are they gonna say? They're probably gonna say yes or no. I'm secretly hoping they-- please say yes, please say yes, you know, 'cause I'm rooting for the thing. But say they say yes, what did I actually learn? I didn't learn why they liked it. Or if they said no, I didn't know why they liked it, right? Now, in normal huber conversation-- human conversation, yes and no questions are great, but you wanna avoid those kind of questions. And basically, it all comes down to how you start your questions. If you start your questions with do you, did you, does that, you're gonna get a yes or no answer. Are you? Was that? Right. So it's just a little Jedi mind trick to catch yourself and start using more open-ended things like how, what, why, things like that. You'll get a lot more richer data from your customer interviews.
- AGAakash Gupta
Brilliant. And how do you capture all these notes? How do you capture all this feedback in a systematic way?
- DODan Olsen
And that's, the, I... That's something that people struggle with, and so what I'm, I'll share here is how I, it literally how I do it. When you do these tests, I like to categorize them in three high-level buckets. You're gonna get feedback on your feature set or functionality, like if they're missing some functionality or this functionality doesn't work the way they want it to, or it's missing some key aspect. You're gonna get feedback on the UX design, that, that iceberg that we talked about. And you're also gonna get feedback on messaging, right? If words don't make sense to people or it means one thing to you but something else to them, you're gonna get feedback. And so I, I start out with a blank slate like this, and I don't know what-- I-in the rows, each row is gonna be kind of a material observation that I have. I don't know ahead of time what it's gonna be. And then at the end, I like to, at the end of the interviews, I like to just ask, "Hey, on a scale of one to 10, how valuable do you find this product? And on a scale of one to 10, how easy to use was it?" And you know, that-- this is what I call semi-quant or quant on qual. We don't have a huge sample size, but we can kind of get some indication, and we can see if there are trends over time. And so I show, I run my first user test with my first user, right? And basically there, uh, this is what I learned. User One, they said, "Hey, feature X that was in your prototype," they thought that was valuable. Awesome. But they complained that we were missing key feature Y. So we note those things down. And by the way, I use a little plus or minus for positive bullets and negative bullets. They, we noticed that they had trouble registering, so we note that as a UX deficiency. And they happened to mention they like the hero image on our homepage, so we put that in the messaging, and we asked them to score value and ease of use. We got seven and five, right? We go to the next user, uh, we move on to the next user, and they bring up the same two feature set things that the first user did, so we're getting signal there. They didn't say anything. They didn't have as much problem with registration, but they didn't see the sign-up link, and they happened to comment that they thought our design looked professional, so that's a positive. In the messaging, they said they didn't understand our tagline. We got a seven and seven on value and ease of use, right? We go through that. We keep going through in this wave. I'll keep it at five to keep it simple. And w- as we talk to more users, we don't get new items coming up. We just get additional, you know, people mentioning or not mentioning the problems that we've seen to date. And then when we get to the end of the wave, we can do this thing I'm talking about, semi-quants, and we can say, "Okay, what percentage of users brought up each issue?" And now we can use this to take a step back and say, "Okay, how do we need to iterate our feature set, our problem space, our UX prototype to address this?" So in this case, obviously, we're gonna have to add feature Y, [chuckles] right? Maybe that was the debate the team had. They're like, "Hey, do we need feature Y or not? Ah, okay, looks like we need it, we need it after all. We need to fix reg. If 60% of people are having trouble with it, we need to fix that. Sign-up link isn't visible, you know, and the tagline." You know, you can prioritize based on this, right? And so that's what we do. So that's-- So we can collapse those five user tests into the feedback from a wave, and then we go back, and we basically say, "Okay, let's go fix all those things." And with vibe coding, it'll be even quicker to have a new prototype that you can now go back. And then now I'm just gonna show you wave by wave. So in each of these waves, we're talking to five to eight users. We're capturing the issues and what percentage of people have it. So this is how-- This is the summary of the problems from wave one. We'd love to see all those go to 0% in wave two. Let's see how we did. We one-- we run wave two, and cool. Feature Y, now we added it. No one's complaining it's missing. That's great. Makes sense. Means we, we nailed that. Check off. Good job, team. However, now we're getting new feedback that, hey, X and Y aren't working together the way they should, but we didn't think about that. We just added Y. We didn't think about how it works with X, and so that's a new issue that we need to address. It looks like we fixed the usability issues with registration. That's awesome. Sign-up linkYou know, we went from 60 to 40. Maybe that's some little progress. Maybe it's just noise. We still haven't really solved that. And now, yes, we did add feature Y, but now we're getting new feedback that it's actually hard to use. It's too hard to use. And it looks like we fixed our tagline issue. We got a slight increase in value and ease of use, seven to eight and five to six. You know, and we go to the next wave, and then cool, we've, we made X and Y work together now. We made progress there. We finally fixed the sign-up link. Looks like we might have made some progress on feature Y usability, but not enough. And, you know, now our value and ease of use score is up nine seven, and then we do one last round, and now we've kind of solved all those issues. That's kind of what that iteration through that loop looks like and how you kind of can systematically capture feedback and, uh, in a way that your team can say, "Cool, let's address these items. Let's see how we did going to the next round," as you're iterating round to round. And if you get, like, all greens, like on wave four, and you're getting high value and ease of use scores, you should be feeling really good that, "Hey, we've really validated product-market fit, and now we can either launch this thing or take it to production code."
- AGAakash Gupta
This is the work that PMs, designers, and user researchers really need to be doing, right? This is how you ensure that your features aren't just bloating up your product with more unnecessary stuff that people don't need. This is how you ensure that you're building a minimum lovable product.
- DODan Olsen
It is. It is. And you know what's interesting is a lot of places, they value prototypes and they value getting user feedback, but the team struggles with s- like, what do we do with this feed? We, we got, we got pages and pages of notes. We've got tons and tons of videos of these sessions. How do we turn the corner? How do we synthesize it and make it actionable? What-- How do we iterate from here? And that's why I kinda like this simple high-level framework for teams to kind of be able to do that.
- 1:08:37 – 1:11:55
PM role realities: ‘Jira jockey’ risk, staffing ratios, and anti-patterns
- AGAakash Gupta
So we've zoomed in on all of these changes that AI has created to product management. I want to reflect on product management a little bit. Have PMs become glorified Jira jockeys? Has the role gone off track?
- DODan Olsen
[sighs] This is a complicated question. The short answer I'll say is mostly no. There are definitely some customer companies where that is the case. There are definitely some companies where that is the case, and PM is basically a waiter taking orders from stakeholders and customers and just turning around and feeding them to dev and Jira. So that's-- they're definitely Jira jockey places like that. Um, but that is, you know, not what you're seeing in the best companies out there. The other thing that you can get into is, like, you spend a lot of time in Scrum, right? You're spending a lot of time writing the stories, refining the stories, um, you know, talking to developers about the stories in all the Scrum ceremonies, right? And you might be on more than one team, so you're going to two sets of ceremonies or three sets of ceremonies, right? Uh, you're working with design. You're probably doing some QA. QA is not sexy, so who does it? PMs, right?
- AGAakash Gupta
Yeah.
- DODan Olsen
Um, status updates, updating stakeholders, all this jazz, making slides, right? So you can get so sucked into that Scrum apparatus that you, you know, you're wholly consumed by develop and you don't have any time for discovery or defining and actually doing your job. So, uh, and I, I hate to say, in the best orgs, though, that doesn't happen, right? So again, if we look at the bell curve, right, the top quarter are not operating that way. The bottom quarter are operating that way. And the main factor if you-- I, I always like to kinda synt-- kind of figure out why, diagnose why, and it's not always just this, but it's usually the staffing ratio. And if you just want to look at one metric, it's like how many, on average, how many devs do you have? How many PMs do you have? Divide the two. Or even on a team. This PM has 10-- I was just at a company where a PM had like 12, 14, 12 or 14 devs. I'm like, "That poor PM. H- they're not gonna have time to do any discovery or do anything but be a Jira jockey, a Scrum jockey," because that's all they have time to do, right?
- AGAakash Gupta
Yes. I think we have been overburdening PMs, where product leaders almost take it as a point of pride. "I've kept my PM org lean. We have a 14 to one developer ratio." But we've made developers so fricking productive with Cursor and Windsurf now. They're turning out stuff so fast, if a PM is supporting 15 developers, they almost inevitably become a Jira jockey.
- DODan Olsen
Yeah, and that's why the ratios are kind of messed up. And this actually, this predates my book. You know, I've been speaking, um, to product managers since 2006, and so this is my original framework. I call it the 4 D's: discover, define, design, develop. Right? And I, I won't go through the questions. You can see the questions that, the key questions you're trying to answer and de-risk in each of those phases. But what we're talking about here is a situation where because of staffing ratios and other things, company culture, not understanding the role of PM, they are spending like over 90% of their time in that develop, that final column. Maybe a little bit working with design. But they, as a result, they are starving themselves and not spending time on discover and define, which at the end of the day is where product managers add the most value, right? Developers are gonna develop. Designers are gonna design. If you're not out there understanding the customer has a problem, then no one else on the team is gonna be. And so it's a problem when PMs don't have time to do that. That's how you end up with a lot of products that are just feature factories that don't solve real customer problem.
- 1:11:55 – 1:19:27
AI trend skepticism + Dan’s business model and where to follow his work
- AGAakash Gupta
What's a product trend right now that's total BS?
- DODan Olsen
Um, [chuckles] I think this happens anytime there's some hot new s- hot new technology. And so right now I call it like, "Hey, sprinkle some AI on your product." I call it that, right? It's like people-- And I think that, you know, a year ago, everyone was like just trying to jam AI in their product, adding co-pilots all over the place just without any rhyme or reason. I think it's starting to settle. There's still some of that going on. It's slowing down a bit. But the, at the end of the day, going back to our discussion about problem space, solution space, AI is a solution, right? Just like, you know, crypto is a solution. Blockchain's a solution. These are all, you know, um, virtual reality, augmented reality, those are solutions. And so the problem that can happen, especially when there's a new big sexy one, is it's like a hammer looking for a nail. It's like, how can we AI this thing? And, uh, and, and, and one, a lot of product managers can't figure out how to AI it because you're trying to force a square peg in a round hole. The other thing is you're not starting with a customer problem. So even if you do manage to sprinkle some dust on there or integrate it somehow, it's not gonna go. And I think we're seeing a lot of failure of those early attempts at trying to just bolt on AI to existing products, um, whereas now you're starting to see people starting from what problems can it really solve. Like a lot of these vibe coding tools are solving real problems, or vibe prototyping tools are solving real problems.
- AGAakash Gupta
So a lot of people will look at you, and a lot of the creators tend to have the highest retention on these podcast, and they'll be wondering, "Well, Dan, he's a really successful book author." But I'm not sure, is that really the business of Dan? When you break down the pie chart of your revenue and how you're making money these days, what does that look like?
- DODan Olsen
Well, it's kind of funny when people think that a book makes you tons of money. So if you publish with a publisher, it's kind of like signing with Hollywood or being Taylor Swift with your records or something like that. Uh, you make some money, but that is not the main, uh, pie slice of my, of my, uh, thing at all. I mainly, um, spend time doing a few things. One is training workshops, right? So as PM has exploded, exploded in the last eight years, um, companies have said, "Okay, we, we understand the importance of PM. We hired a bunch of PMs. We want to train them now." And there aren't a lot of great resour- There are more these days than there used to be, but a lot of times they will hire someone like me to come in, and I'll tailor the training for the team. Sometimes it's just product management. One of my favorites is when it's actually product plus design plus dev, and we go through all these slides and more that I went through with you so they really understand. 'Cause it's kind of funny. I like to use the analogy that product development is a team sport. It's like, you know, the Golden State Warriors. I'm in the Bay Area. They don't just show up at the court and go, "Hey, let's play some basketball. What position are you gonna be?" You know? They know their positions. They know their plays. They practice them. But when it comes to product development, we just hire a bunch of random people, throw 'em in a building or, or Zoom and go, "Hey, go make product," right? We don't talk about positions. We don't talk about plays. We don't practice. So those workshops are very, um, experiential, interactive, a lot of exercises, things like that. I also love speaking at product conferences. And before COVID, you know, I speak at a lot of conferences. I used to write an annual list, and I'd keep track of, like, how, how... Are they growing? How many do we have? Where are the new countries that are having them? And then COVID hit, and I stopped. So this year I was super psyched, uh, on my Substack, danolsen.substack.com, I wrote my 2025 conference guide, and I was d- excited to see there's more this year than there were before COVID, and there's a lot of new conferences. So I just was in San Diego for a first-time product conference. I'm heading to Calgary for a first-time product conference, so a lot of first-time conferences. So workshops and speaking. I also speak at private events if... You know, the other cool thing is m- as products emerge, more and more companies are actually getting, like, their product teams together for a day or a half day or two days for an offsite to talk about the craft of product management. Like, that's the thing. We're so busy that when do you have the time to actually sharpen your ax and sharp... add to your toolkit? There's really kind of... So, so a lot of companies recognize that. So I'll go and speak or give workshops at those events, too. Um, but then the other thing that I do that I love and how this all, a lot of this content developed is hands-on consulting, right? I used to be an interim VP of product for post-Series A startups. So I did that for Box in 2007. I basically was their head of product. I did it for One Medical Group in 2009. I did it for Medallia. I've done it for a lot of companies, and that's... I kind of stumbled into being that interim VP of product back in the day. I did that in 2005, and a lot of people wanted to be fractional CPOs that day. Like, I did it back then. And, uh, that's accelerated my learning, 'cause instead of just being at one company, I saw a lot of other companies. So I still do that today. These days it's more of an advisory or consultant capacity, but I still like working with early-stage startups. In fact, if you're watching this, I'm super excited about AI. So if there is an AI startup out there that doesn't have a head of product, that would be a great fit to kind of play around and help out and help them achieve, apply some of these concepts. Um, and then those are the main ways. And then I also-- I'm just a big believer in best practices, promoting best practices in community. So for over 11 years now, I've been running a monthly meetup here in Silicon Valley where I live, um, called Lean Product Meetup, and I bring in top speakers. Marty Cagan's been there nine times, you know. Uh, I speak there. A lot of other people have spoken. You know, Geoffrey Moore, Crossing the Chasm. A lot of-- All the people that you know that are probably on your podcast and, uh, blogging and have books have been on there. Um, and so during COVID, we had to take it virtual, and when we came back from COVID, we built up this great audience around the world, so I-- we do hybrid events now. So, but that's just, like, a little side project. And there's one other side project I do, which is I run an annual product conference with three other folks, co-organize it with them, called the Product Leader Summit. We've done it nine times. It's a small 120-person invite-only, um, product conference for product leaders 'cause there aren't a lot of, um, events for product leaders. So yeah. And then I try to write on Substack. I try to do stuff on LinkedIn, you know, try to play around with five coding tools, stay current, and, and write thoughts about all that stuff. So it's a, it's a pie chart with a lot of slices, but mainly workshops, speaking, and consulting and advising are the main, main components.
- AGAakash Gupta
How lucrative can that be?
- DODan Olsen
Uh, I mean, it just depends. It can, it can be lucrative. It depends on how much you want to work, right? That's the thing about being a, a solo person. I've been doing it for 20 years on my own. You can decide to dial up your time, dial down your time, you know, have flexibility. So it really-- I don't think there necessarily has to be a limit on that. And you know, the other thing is you can decide, do you wanna, like, scale up and have a team to do some stuff? Do you wanna stay solo? Things like that. Those are some of the decisions that some people get into. And some people that have, like, hired a team, they go, "I don't wanna do that. I wanna be solo." So you know, there's different-- It's interesting to see, you know, in the product world, different people taking, you know, different approaches on how to, you know, create their own kind of product business.
- AGAakash Gupta
If people want to learn more, where should they go?
- DODan Olsen
Yeah. Uh, I have-- My main website is dan-olsen.com, O-L-S-E-N. Um, I'm on Substack, danolsen.substack.com, where I'm actually posting a series about product strategy if people are interested in that. And, uh, and then I have a YouTube channel, just, uh, youtube.com/danolsen. And I'm on LinkedIn. Happy to-- I, I interact mainly with people on LinkedIn. So hopefully also I'll see you at one of these conferences coming up.
- AGAakash Gupta
Awesome. Well, thank you so much for being here, Dan.
- DODan Olsen
Thanks a lot, Aakash. Appreciate it.
- AGAakash Gupta
I really hope you guys enjoyed that episode. It would mean a ton to me and the team if you could please subscribe on YouTube, follow on Apple and Spotify podcasts, and leave a rating and review. Those ratings and reviews really help grow the show and help other people discover the show, and they help fund the production so that we can do bigger and better productions. Can't wait to share the next episode with you. Until then, see you later.
Episode duration: 1:19:37
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode sl7r3wsoVGY
