How this Yelp AI PM works backward from “golden conversations” to create high-quality prototypes

1. 0:00 – 2:54

   Introduction to Priya

   1. CVClaire Vo0:00

      Where do you start when you're thinking about designing and framing out a AI product for what you're working on at work?

   2. PBPriya Badger0:07

      What's different about managing products that are powered by AI is there is the interface of how a user interacts with any product or product feature, and that still really matters, and there's also a lot going on behind the scenes. There's a lot also about how do you drive good quality products, because these technologies produce different results each time you use them. So we start with golden conversations. What's the experience that you're trying to drive? And so this is just a way for me to think about how to write that, role-playing a little bit with AI.

   3. CVClaire Vo0:41

      What you're saying is actually write an example conversation that can represent what a real user might do, and you're working backwards from that example conversation, which I have actually not seen anybody do before. [upbeat music] Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today, we have an AI PM showing us how to AI PM. Priya Mathew Badger is a PM at Yelp and is showing us a completely new way to think about product requirements, prototyping, and how to build effective conversational agents using conversational agents. Let's get to it. This episode is brought to you by GoFundMe Giving Funds, the zero-fee DAF. I wanna tell you about a new product GoFundMe has launched called Giving Funds, a smarter, easier way to give, especially during tax season, which is basically here. GoFundMe Giving Funds is the DAF, or donor-advised fund, from the world's number-one giving platform, trusted by 200 million people. It's basically your own mini foundation without the lawyers or admin costs. You contribute money or appreciated assets, get the tax deduction right away, potentially reduce capital gains, and then decide later where to donate from 1.4 million nonprofits. There are zero admin or asset fees, and while the money sits there, you can invest and grow it tax-free, so you have more to give later, all from one simple hub with one clean tax receipt. Lock in your deduction now and decide where to give later. Perfect for tax season. Join the GoFundMe community of 200 million and start saving money on your tax bill, all while helping the causes you care about the most. Start your giving fund today in just minutes at gofundme.com/howiai. We'll even cover the DAF pay fees if you transfer your existing DAF over. That's gofundme.com/howiai to start

2. 2:54 – 4:33

   The unique challenges of managing AI-powered products

   1. CVClaire Vo2:54

      your giving fund. Priya, welcome to How I AI. I am so excited to have you here, because whenever anybody asks me, and they ask me a lot, "How do I do AI product management?" I have to say, "Wait, are you talking about product managing with AI? 'Cause I have some ideas about that. Or are you talking about product managing AI products?" And what's really great about the conversation we're about to have is you actually do both. So what, in your mind, is really different about product managing products using AI?

   2. PBPriya Badger3:30

      Yeah, I'm really excited to be here. Big fan of the show and have learned a lot about, um, AI, both managing AI products and how to use it in my day-to-day from the podcast, so it's exciting to be here. For me, I think, you know, what's different about managing products that are powered by AI is there is the, you know, interface of how a user interacts with a, with a- any product or product feature, um, and that still really matters with AI products. Um, and I'll show some of the tools that we use, um, to explore that. Then there's also a lot going on behind the scenes that determines the product experience for the consumer. So, um, the system prompts and how that guides the conversation flow is really interesting, and I think kind of a new challenge when you're working on AI-powered products. And there's a lot also about how do you drive good quality products, because these, um, technologies produce different results each time you use them. So there's a lot of, um, interesting challenges

3. 4:33 – 5:53

   Using example conversations as a starting point for design

   1. PBPriya Badger4:33

      there, too.

   2. CVClaire Vo4:34

      Yeah, so I'm really excited to myself learn from your flow, 'cause I'm building an AI-powered product as well, and so let's dive into it. Where do you start when you're thinking about designing and framing out a AI product for what you're working on at work?

   3. PBPriya Badger4:51

      Yeah, absolutely. So I thought a good example would be to talk about building a new feature capability into our Yelp Assistant. So that's a product I work on, and the way it works is a consumer can come in for a service need. So let's say you wanna hire a handyman, a plumber, an electrician, somebody to fix your car, and you can describe the problem in your own words, and then the AI will understand what you're saying, collect some project details, and, um, help you get matched to pros and get quotes. And so that's how the product works, and we recently launched a feature that allowed consumers to upload a photo to help describe their need, and that just makes sense, right? It, it helps for pros sometimes to be able to see a photo along with the description. But one of the things we wanted to do was, because we were doing this in our AI assistant, think about, you know, how can we leverage those AI capabilities? Can the AI understand what's in the photo and customize the conversation from there, um, providing, you know, some recommendations around what the consumer

4. 5:53 – 9:10

   Demo: Prompting Claude to generate sample conversations

   1. PBPriya Badger5:53

      should do next?

   2. CVClaire Vo5:54

      As a, a Yelp user, I can imagine that the variety of services that your pros are providing and, um, you know, with... I don't run consumer businesses, but- [chuckles]

   3. PBPriya Badger6:04

      Mm-hmm

   4. CVClaire Vo6:04

      ... I can imagine the, the variety of things a user puts into these conversational or image upload interfaces could be very diverse. So I'm curious how you-... approach that from a product development perspective.

   5. PBPriya Badger6:17

      Yeah, absolutely. Yeah, we certainly cover a lot of different categories of service needs at Yelp, and one of the challenges is, yeah, making sure that the experiences work across all those different use cases that a consumer might have. Do you wanna jump in, and, uh, I'll, I'll show you my workflow?

   6. CVClaire Vo6:34

      Yeah, let's do that.

   7. PBPriya Badger6:35

      Okay, so I'm gonna just open up Claude, and here we're starting in a totally new window. And, you know, as we talked about, like, I think there's, you know, two pieces to these AI products. There's the behind-the-scenes part, and then there's the interface, uh, user interface that consumers see. Um, and I like to start with thinking about, what is that conversation flow gonna look like when we add this new functionality? And so I'm gonna show you here how you can do that with Claude. Um, and you can also use ChatGPT or any other, um, of these foundational models. So here I'll say, "Write a complete, um, sample conversation between the consumer and the AI assistant," um, where we want consumers to be able to upload their photo, and then just add some scenario requirements, like we want the assistant to analyze the photo, maybe provide some suggested replies, and, uh, continue that back and forth until they have enough info to submit quotes. One thing I'll call out on the prompting is I do like to give a little direction on what the output looks like. So you can see here I'm saying, like, "Use Assistant colon, User colon for labels. Write it as one continuous conversation." I think that really helps make sure that, you know, you get the output that you're looking for, and there's a little less back and forth with the AI.

   8. CVClaire Vo7:48

      So for the folks listening, one of the things I wanna call out that I think is really interesting about this approach is you're sort of using a example conversation as your first-pass wireframe for building a conversational AI. So instead of saying, like, "Show me a chat window, and show me messages that show up, and these buttons," what you're saying is, "Actually write an example conversation, um, that can represent what a real user might do." And, um, you kind of give some, some constraints about what that conversation could look like, and you give it some of the capabilities that might be available during that conversation, and you're working backwards from that example conversation, which I have actually not seen anybody do before. So I think it's a really unique approach that product managers out there working on conversational, um, AI products, including myself, can really take a lot of inspiration from. How did you come to this idea? I mean, uh, was this your... Like, are you just a genius, and you're like, "This is the first thing-"

   9. PBPriya Badger8:47

      [chuckles]

   10. CVClaire Vo8:47

       "... that we need to do?" Or how did you come to this idea?

   11. PBPriya Badger8:50

       No, I mean, I think this is part of, um, our standard LLM-powered playbook at Yelp, where we start with golden conversations. What's the experience that you're trying to drive? Um, and so, you know, I think, uh, this is just a way for me to, like, think about how to write that, um, role-playing a little bit with

5. 9:10 – 9:53

   Prototyping advice

   1. PBPriya Badger9:10

      AI.

   2. CVClaire Vo9:11

      Yeah, and I just wanna call this out. We're gonna take a little side, uh, detour to just some product management ideas, which is I often tell product managers to prototype their product as close to the end product that a consumer is going to consume, including the content. So when I worked in dev tools, um, I would tell a lot of our PMs, "Don't write a PRD. Write a quick start and documentation guide to the product. Write the code snippets, um, and then work backwards into what the product should look like." And so I love this idea of, just from a general product perspective, work with the artifact that's closest to what the con- consumer is actually going to experience, and then you can back into all the requirements once you're kind of inspired by what that

6. 9:53 – 15:03

   Testing with multiple example images and scenarios

   1. CVClaire Vo9:53

      end state is. So what does something like this get you?

   2. PBPriya Badger9:57

      Yeah, absolutely. So let's go through it. So I'm actually gonna upload a real photo of a home service need. So here's, like, a picture with a cracked porch. Um, and-

   3. CVClaire Vo10:07

      I hope that's not your cracked porch.

   4. PBPriya Badger10:08

      [chuckles] It's not, no.

   5. CVClaire Vo10:10

      [chuckles]

   6. PBPriya Badger10:11

      Um, yeah, and then we'll look at what, um, what Claude comes back with. Um, I will say, one of the pictures I'm gonna test that is from my bathroom renovation-

   7. CVClaire Vo10:23

      [chuckles]

   8. PBPriya Badger10:23

      ... so you will see my bathroom. And one thing I'll call out is Claude now shows you a thought process, and you'll see this in a lot of AI tools. I really like to read the thought process, and it's also something to do while you're waiting. [chuckles] Um, but I think it really helps because you can see how it's understanding you, if it doesn't come back with what you want. It also is really good for troubleshooting, so definitely something I recommend doing.

   9. CVClaire Vo10:44

      Yeah, one thing that I'll do while this is loading is call out, I too think that reading the reasoning or the thought process of the AI is interesting for two reasons. One, it can often help you improve your prompts because you understand what the AI is understanding or not understanding about your prompts. As somebody who likes misspelled, no sentence, low-syntax prompts myself-

   10. PBPriya Badger11:06

       Mm-hmm

   11. CVClaire Vo11:06

       ... it's good, good to see where I'm misleading the AI. The other thing is the thought process is often where the AI reveals its personality. I think it is so funny- [chuckles]

   12. PBPriya Badger11:15

       Yeah

   13. CVClaire Vo11:16

       ... to read, like, Gemini 2.5's thought process-

   14. PBPriya Badger11:19

       [chuckles]

   15. CVClaire Vo11:19

       ... versus o3 versus... Claude is very nice.

   16. PBPriya Badger11:21

       Yeah.

   17. CVClaire Vo11:22

       Claude li- Claude practices self-love. Um, Gemini 2.5 does not.

   18. PBPriya Badger11:26

       [chuckles]

   19. CVClaire Vo11:26

       And so I just think it's-

   20. PBPriya Badger11:27

       Love it

   21. CVClaire Vo11:27

       ... uh, it's also interesting from just, like, a, a model understanding perspective. Okay, so we got a, we got a chat here.

   22. PBPriya Badger11:34

       Yeah. So then we can read through the chat, and it's, you know, it's saying, like, "I can see you've uploaded this photo of a frontch- front porch steps with a significant crack running through the concrete," so pretty good recognition of the photo. And then it says, "Let's ask... Let me ask a few questions. H- you know, how urgent is this? You know, are you looking to repair this? Would you prefer to replace the entire steps?" And so I could look through this, you know, and maybe workshop it a little bit, giving it some feedback.... I also find it's helpful to just create some more examples. Um, sometimes, like, when you see a lot of examples, that's when the trends come out, and that's when you see, you know, what you might wanna improve or change. And so I have a bunch of images now. So now that I've tested it with one, and I've seen that, you know, it works pretty well with that one, I'm now gonna test it with a lot more images, and this is the prompt I'm gonna use. So I'm gonna say, "Now create more examples based on these images." And to your point earlier, you know, Yelp covers lots of different, um, types of service needs. So this is where you can kinda test and see how is it gonna do across a lot of different problems. And so here I have, you know, like, a appliance repair issue with an error code. I have a hornet's wa- a wasp nest.

   23. CVClaire Vo12:47

       [chuckles]

   24. PBPriya Badger12:47

       Um, so you can see, you know, a larger variety of things, and just because I know you really wanted to see my bathroom-

   25. CVClaire Vo12:54

       [chuckles]

   26. PBPriya Badger12:54

       ... I will also upload and add a picture of my bathroom renovation in progress. Um, and then I'm gonna say, um, you know, "Label each conversation with a title and a number at, at the top," so just another example of how just that, like, little nudge on the output can really help you get something usable.

   27. CVClaire Vo13:14

       Great, and so we're gonna see here how this AI thinks about potentially framing responses to consumers on a variety of, as a homeowner, total nightmare scenarios.

   28. PBPriya Badger13:26

       Mm-hmm.

   29. CVClaire Vo13:26

       Everything from a wasp to a bathroom renovation, [chuckles] which I am also about to start, um, is just a nightmare to me, whether or not I wanna do it. Um, and so you're getting these example conversations, and what are you looking for? Are you, are you looking for patterns? Are you looking for product inspiration? What's kind of the thing that you're seeking in these examples?

   30. PBPriya Badger13:47

       Yeah, that's a great question, and I think this, like, goes in with, you know, there's the, the... You know, a lot of people talk about, like, evals are the new PRD.
7. 15:03 – 15:59

   Refining conversations based on qualitative assessment

   1. CVClaire Vo15:03

      Okay, so you have these different conversations.

   2. PBPriya Badger15:07

      Yeah.

   3. CVClaire Vo15:07

      What do you, what do you do with them next?

   4. PBPriya Badger15:09

      [chuckles] Yeah, and I'll just show one example of refining these conversations and how I AI's really great for this. So, you know, let's say I say, "I, I think it's good, but I don't think it's being as opinionated as it could be about, like, offering the user a recommendation, and maybe sometimes it's talking about budget, which we think the consumer may not know." So I can ask it to rewrite these conversations based on this feedback, and it will go through and update all those conversations for me, which I think is really nice. And, um, you know, then you can go through and see, you know, do you feel like it's taking that feedback well? Is it actually rewriting it, um, based on that guidance? But definitely, you know, you can see here it's saying, like, "This definitely [chuckles] requires professional pest control. Don't attempt a DIY or removal of this nest," um, which I think is probably good advice. [chuckles]

   5. CVClaire Vo15:58

      [chuckles]

8. 15:59 – 21:22

   Demo: Creating interactive prototypes with Claude Artifacts

   1. PBPriya Badger15:59

      Um, and then to your other point about, like, how do we get, um, an artifact that is closest to the ex- what the consumer will experience, that is the next step that I'm gonna show you and something I think that is pretty unique to Claude. Um, so Claude has a special functionality built in, where it actually can create an artifact that uses the LLM that powers Claude to produce those responses, and that's very unique to Claude. If you did this in another prototyping tool, you would typically have to set up a API key and, um, integration, which just takes a little bit more work, and with Claude, you can do it out of the box. So here, you can see I'm asking it to create an assistant app as an artifact, have a chat interface where the AI responds using the LLM that powers Claude, and then also create system ins- uh, prompt that is based on these example conversations, and then analyze these upload- loaded photos and include a camera, um, icon in the input. And then I'm actually gonna upload some, um, screen grabs of our current Yelp assistant and indicate that it should use these ex- attached screenshots as an example for what the front end should look like, just so that it feels a little bit more real.

   2. CVClaire Vo17:13

      Got it. So you really are using example conversations and just reference designs as your PRD here, and then-

   3. PBPriya Badger17:22

      Mm-hmm

   4. CVClaire Vo17:22

      ... what you called out that's unique about Claude Artifacts is it has fully integrated Claude AI.

   5. PBPriya Badger17:29

      Yeah.

   6. CVClaire Vo17:29

      So you can actually generate artifacts that do make native LLM calls to the Anthropic API. So if you are prototyping a little AI product out there, um, check out Claude, because it just makes it a little simpler, and you don't have to pass it a bunch of API keys.

   7. PBPriya Badger17:45

      Yeah, absolutely, and you can see that it's writing the code here, and at the top, it actually wrote the system instructions. And I think this is also a really good way to learn because you can see that, based on these example conversations, how is Claude translating that into system instructions? Um, so it's, you know, mirroring some of my initial prompting and redirection around providing suggested replies, um, not asking the user about budget, and so I think that's, um, really helpful. And then you can see it gives some examples from my examples as part of how to guide the, um, assistant around photo analysis as well.... All right, and so I'm gonna test it out, and we'll see if it works out of the box. Um, it does sometimes require a little back and forth. Um, so you can see here, I, uh, have uploaded the photo of my issue, and Claude is thinking. Okay, great. Um, so here you can see it worked pretty well. So it said, you know, I can see it's showing F2 in red and the door locked, and this is a common error code relating to the ov- oven lock. You know, typically, you want a repair technician. It's asking about the urgency. So it is, you know, simulating pretty well this conversation, and one of the reasons why I think it's helpful to simulate it in this kind of artifact is you can also get a real feel of how this would be for the user. Like, you can see, like, sometimes a response that looks fine when you have it in a doc feels really long when you see it in-

   8. CVClaire Vo19:16

      Yeah

   9. PBPriya Badger19:16

      ... like, the little chat bubble in the mobile interface. And, you know, that waiting period of, like, the three dots, and then the response comes back when you play out the full conversation can feel very different. So I think this is also a really good step to do.

   10. CVClaire Vo19:31

       And then you can, of course, share this with your team or your designers, your engineers, and they can also start to get a sense of: How does this feel? Can we actually do this? How can we refine it-

   11. PBPriya Badger19:42

       Mm-hmm

   12. CVClaire Vo19:42

       ... or make it even operate better? So I, I just have never thought of this flow. I have to repeat it again for folks. You know, kind of starting inside out with a conversational agent, prototyping example conversations first, getting them, um, refined, getting a good set of example conversations that you can then put into a, um, prototype-generating tool, in this instance, Claude, to then back into the chat experience, including the system prompt, that would best serve those conversations as such. A great flow. I'm so impressed! This episode [upbeat music] is brought to you by Persona, the B2B identity platform helping product, fraud, and trust and safety teams protect what they're building in an AI-first world. In 2024, bot traffic officially surpassed human activity online, and with AI agents projected to drive nearly 90% of all traffic by the end of the decade, it's clear that most of the internet won't be human for much longer. That's why trust and safety matters more than ever. Whether you're building a next-gen AI product or launching a new digital platform, Persona helps ensure it's real humans, not bots or bad actors, accessing your tools. With Persona's building blocks, you can verify users, fight fraud, and meet compliance requirements, all through identity flows tailored to your product and risk needs. You may have already seen Persona in action if you've verified your LinkedIn profile or signed up for an Etsy account. It powers identity for the internet's most trusted platforms, and now it can power yours, too. Visit withpersona.com/howiaai to

9. 21:22 – 25:30

   Using Magic Patterns to design the user interface

   1. CVClaire Vo21:22

      learn more. You know, now what I have to call out is this looks pretty good, but it doesn't look quite like Yelp. So [chuckles] how do you take this-

   2. PBPriya Badger21:31

      Mm-hmm.

   3. CVClaire Vo21:31

      How do you take this to that next step of, you know, really, um, designing out what the real product might look like?

   4. PBPriya Badger21:37

      Yeah, for sure, and I will say, like, I think this is all just a starting point, and it's a part of a conversation with your larger team, right, with the engineers and with the, with designers. Like, I think this is really something that helps me clarify my own thinking and ideas and, like, refine what does that ideal conversation look like, and, and also just, you know, be a better collaborator 'cause I understand system instructions better, um, as, uh, as we're going through features. Um, but yeah, so I think, um, you know, it still goes through our u- our, our usual, like, design and engineering pro, uh, processes once we have a good idea of, you know, where we're headed, and it really has been a collaborative process for us between design, product, and engineering, where we're all writing these conversations together. We're giving each other feedback on them. Um, so now we're gonna... I'm gonna talk about, you know, how do we, how do we think about the exploring ideas on the other side? So we, we went pretty deep on, like, what does that conversation flow look like? How can we use Claude to, um, explore ideas there? And the other piece is, like, how do you use... What does the interface look like? What are the user flows? How does a user get into these assistant experiences? And I have seen that a lot of those little details matter as well. You know, what are the prompts? How do, how does a user understand the capabilities of the assistant? And so here with, uh, I'm gonna show another tool, which is Magic Patterns, and I think Magic Patterns is really great for when you wanna explore something visually and, like, kinda consider what that flow would look like. I know Colin Matthews was on this show earlier, and he showed how you can recreate a, you know, an existing product using component library or screenshots. So I'm not gonna cover that in detail. So here, I've recreated our Yelp assistant, um, with that kind of approach, but I'm gonna show you how you can then move on, um, to actually explore features within, um, Magic Patterns, which I think is a lot of fun. So here, I'm going to actually ask it to add a prompt suggestion at the top for start with a photo, which allows the user to upload a photo. And, you know, you can see here, it's, it's thinking, and it's saying, "I will start, um... Add this prompt suggestion for start with a photo. Um, this will likely require these things. Um, for styling, I'm gonna consider this." So again, like, reading those thinking instructions, I think, is super helpful.

   5. CVClaire Vo23:56

      So what it's doing now, now that it has those instructions, it looks like it's sort of doing this thing that you see in a lot of these prototyping tools, which is it's creating or updating new components, updating components. It's gonna kinda insert those design elements into, into this design for you to give feedback and test with. And I just have to say, you've been a PM for a little bit. I've been a PM for a little bit. Have you ever had access to this kind of, like, on-demand-... design and code? Like, is, has this totally, like, changed the way you think about working through designs, wireframes, stuff like that?

   6. PBPriya Badger24:34

      Yeah, it absolutely has. Yeah, I think my mind was kind of blown, to be honest, [chuckles] the first time I used these, like, natural language prompting prototyping tools, just because, yeah, it's just so magical for you as a PM to be like, "Hey, I can just describe what's in my head and actually have it, you know, come to life, um, in a prototype." So it really has... Ah, you know, I think the core of the, of the PM job and the earliest part of the workflow hasn't really changed, in that you're still trying to understand deeply the user problem, figure out what to prioritize. Um, but I think it really helps in the phase after that, where, as a team, you're exploring the solution space. What can really solve that problem for a user? How do we make them aware of it? How do we make sure it's easy to use? And I feel like it's just really fun to be able to, like, play around in these tools and explore ideas, um, myself visually and, and find better ways where I can communicate something

10. 25:30 – 31:02

    Exploring multiple design variations with Inspiration mode

    1. PBPriya Badger25:30

       that's in my head.

    2. CVClaire Vo25:31

       Amazing. Okay, so now we have a start with a photo.

    3. PBPriya Badger25:33

       Okay. So yeah, we have a start with a photo. So you can see here, it's got this UI where I can start with a photo. Um, so, you know, that's, you know, one option, and, of course, like, you know, we did something simple when you launch this ph- feature, where there's just a camera icon. But I'm showing this example as a way that, you know, you can explore, like, what would other ways be that we could make this experience, um, as you're thinking about iterating? And so here, I'm gonna show you this really cool feature within Magic Patterns, which is called Inspiration mode. Um, and definitely recommend digging into this menu in general. [chuckles] Um, they have, like, a lot of nice little shortcuts, but this Inspiration mode is my favorite because you can quickly explore lots of different options. So here, I can say, "Give me some options on how the start with the photo flow could work to make it feel more guided for the user." And this part of the prompt, I workshopped a little bit [chuckles] but I think works to help have the Inspiration mode come up with different ideas. I say, like, "Think expansively and make each option differentiated, and then explain in, in your response which option, um, what each option is." Um, and so I'm gonna go ahead and submit that, and it will generate for me four different options. And you'll see that, um, once it goes through this process, it will actually have four different boxes on the screen, and as you wanna explore those options, you can click through those boxes, and it'll update what's on the left side. So you can really quickly explore and see the different ideas and, you know, decide what you like. Um, and I like doing this because I think sometimes we come in, and we feel like we need to have a whole PRD before we can start prototyping, and that's definitely one approach and use case for AI prototyping tools. But I've also found that they're helpful even earlier when you, you do understand your, you know, your user problem and what you're trying to solve for, but you may not know really what the solution looks like, and you wanna explore and maybe get some ideas from AI as well.

    4. CVClaire Vo27:34

       Yeah, this just makes me think, I don't know if designers are gonna love this or hate this. I remember this experience when I was a designer, where somebody would give me a PRD or a feature like this, and I would give them back a design like what we see on the left, and they'd be like, "Great, but can we, like, try it over here or try it over there, and move it up there-

    5. PBPriya Badger27:51

       [chuckles]

    6. CVClaire Vo27:51

       ... and make it this button and, like, make it a link?" And that, like, manual iteration, where it wasn't really, um, moving the product forward, it was-

    7. PBPriya Badger28:02

       Mm-hmm

    8. CVClaire Vo28:02

       ... kind of getting our own minds around what the problem space and the solution space could be so that we could move the product forward-

    9. PBPriya Badger28:08

       Mm-hmm

    10. CVClaire Vo28:08

        ... just took a lot of time. And so I think it's really interesting to compress the time for ideation so that you can-

    11. PBPriya Badger28:14

        Mm-hmm

    12. CVClaire Vo28:14

        ... get to the ultimate product a little bit faster.

    13. PBPriya Badger28:16

        Yeah, absolutely, and, like, some of our designers are also using, using Magic Patterns or even other AI prototyping tools, like Figma has a-

    14. CVClaire Vo28:25

        Yeah

    15. PBPriya Badger28:25

        ... Figma Make. And, and so I think it's really just part of the conversation. You know, I'll ping a designer, "Hey, I was thinking about this, and, you know, was thinking maybe we could go in this direction," and send them a link, and they'll be like, "Oh, I was, you know, exploring something similar," and we'll just trade notes. So to me, it's a replacement for what I was doing before, which was really hacky Figma mock-ups and, like, not-so-great wireframes. [chuckles] Um, and so I, I think it's an extension of that, like, wireframing, hacky Figma prototype process, where it just is easier for someone to understand because they can actually click through and see the flow.

    16. CVClaire Vo29:00

        Yeah, it's just more interactive, I think is really-

    17. PBPriya Badger29:02

        Mm-hmm

    18. CVClaire Vo29:02

        ... it might not be higher fidelity, but it's a richer kind of prototype experience than you would get from sort of a flat design. Okay, we at least have three successful-

    19. PBPriya Badger29:12

        Okay

    20. CVClaire Vo29:12

        ... generations we can click through-

    21. PBPriya Badger29:13

        Yeah

    22. CVClaire Vo29:14

        ... quickly.

    23. PBPriya Badger29:14

        With, with, with all AI, you know, sometimes you get errors, but, you know, here it says it's, like, a guided category selection flow, so we'll click through and see what they did. So you can see here, it's, like, y- kind of customizing it a little bit for the category of, um, of the service. So I'm gonna go back and maybe select another category and see how it's different. So it's, like, you know, kind of customizing some of the tips. Um, in this one... Let's see. I might need to actually select a photo to see what it does. Um, so you can see it's, like, going through an analysis. [chuckles]

    24. CVClaire Vo29:49

        Yeah.

    25. PBPriya Badger29:49

        You know, this is not using the LLM, the, the, behind the scenes, so you can see it's not, uh, not making sense. But y- I think the idea here makes sense, where it's like, okay, it's gonna do this, like, kind of real-time detection. Um, and then in this one, it looks like it's, like, multiple photos. You can see here, it's, you know, showing, like, you know, you could, um, prompt the user to maybe take multiple pictures. I will just click on this to show that, you know, this is how AI works, [chuckles] where-

    26. CVClaire Vo30:18

        [chuckles]

    27. PBPriya Badger30:18

        ... sometimes, [chuckles] sometimes you get errors, and you need to fix them. Um, you know, usually there's that, like, shortcut to, like, try to fix it. Um, if it doesn't work, um, there is also, like, a debug command within-

    28. CVClaire Vo30:32

        Yeah

    29. PBPriya Badger30:32

        ... Magic Patterns, which I found pretty useful, which just tells it to, like, look through your code, try to come up with what's wrong to fix it. Um, and-... Let's see if it did fix it.

    30. CVClaire Vo30:43

        For our listeners that are not wa- not, are not watching, I will spare you reading the uncaught React errors about, um, incompatible React versions, but that is what we are looking at [chuckles] right now, which is we are looking at-
11. 31:02 – 33:35

    Quick summary

    1. CVClaire Vo31:02

       All right, so like all good AI demos, this one did not work. But I do wanna say, just stepping back, what I wanted to just call out is you have demoed for us a completely new way of thinking about product management, prototyping, and product requirements in a way that is very different than I think what classic product management has looked at. And so you're starting from a kind of example consumer experience first. You're backing into kind of a rough prototype of what could support that experience. You're using a AI prototyping tool, in this instance, Magic Patterns, to then put that experience in your brand and design guidelines, and then you're using that as a jumping-off point to fork and inspire a couple different versions of what that ultimate user experience could look like. And then I'm presuming you're gonna take one of these, and you're gonna say, "I think we wanna start here for our MVP or our V1," and then that, you know, you get the team together, and then, and then that's where you start. And so I think for the product people listening, what I like about AI is it's not just multimodal in that you can put any sort of, um, file type or data type in, it also allows you to approach problems from the front door, the back door, the side door, the window. Like, you know, you can come at your product problems in a much less linear way, and in fact, you can start at the end, go back to the beginning, come to the middle, fork off, go back to the beginning and re-prototype, and it's not expensive, it's fast, and it's interesting.

    2. PBPriya Badger32:43

       Mm.

    3. CVClaire Vo32:43

       And so I think what you've inspired me to do is actually think a little bit differently about what the starting point of product management could be, not just for AI products, but for product in general. And then, of course, you showed some great ways that AI can help with that.

    4. PBPriya Badger32:56

       Yeah, absolutely. Um, and I will say, yeah, to your point, you know, you can pick which one you like the best, um, which you think fits your, you know, where you are, um, in your, in your product journey and your user needs. Um, you can also... Like, if there's one that feels like, "Hey, this, like, AI-assisted one seems really interesting," or, "This multi-photo one seems really interesting, but maybe not, like, where we're gonna go right away," you can fork this design, and it will create, um, a totally separate window and chat for you, um, of just that variant. And then you can just run off with that, you know, maybe on the side, um, while you're continuing down the original path that you were in.

12. 33:35 – 38:57

    How to apply these AI prototyping techniques to personal projects

    1. PBPriya Badger33:35

       Um-

    2. CVClaire Vo33:35

       I, I love that. So we have seen your AI-powered AI PM process-

    3. PBPriya Badger33:42

       Mm-hmm.

    4. CVClaire Vo33:42

       And usually I would bump us to lightning round, but part of our lightning round is gonna have a couple demos in it. So as my first-

    5. PBPriya Badger33:50

       Yeah

    6. CVClaire Vo33:50

       ... lightning round question, can you do a quick world tour of a couple non-work-related AI use cases that you think our listeners would really get a lot of value from?

    7. PBPriya Badger34:00

       Yeah, absolutely. I can share a, a few personal examples also. Um, so, um, one is, you know, I have started this, um, you know, talk AI channel that was a, a at Yelp, which was actually inspired by a talk AI channel in Lenny's community. And, um, I wanted to create a monthly newsletter that gets sent out that just summarizes all the great discussion and content that was being created there. And so, um, I'm just gonna show an example of how to do that using Lenny's community. Um, and so here I have this, um, set of project instructions that say, you know, "I'm a community manager writing a weekly newsletter. Um, use these Slack conversations and format them just like the Community Wisdom Newsletter." And then I think what's really cool is I can just come in here, and I can say, you know, "I wanna just make a version of this Community Ver- uh Wisdom using this Slack chat," and I can upload the file of all those Slack chats, and I did randomize the names, [chuckles] uh, or, um, replace the names for privacy, also using GPT. H- um, and then you can see here, it's gonna make a version of that Community Wisdom Newsletter just using those Slack chats and, um, reuse that same format. And by using a project, I can, you know, save myself some time on the prompting.

    8. CVClaire Vo35:28

       Great, so you're copying and pasting, um, like, a week's worth of Slack conversations.

    9. PBPriya Badger35:33

       Mm-hmm.

    10. CVClaire Vo35:34

        You're putting it into this Claude project, which you've been given a, um, you've given a template, and then you're having it generate on a weekly basis or whatever, kind of a summary of what's going on in that community and other kind of, like, content that's being shared.

    11. PBPriya Badger35:52

        Yeah, absolutely. And then you can see, it now kind of follows that community wisdom, uh, format and pulls out what the top threads are, and so you might wanna make some edits to this afterwards, but it really, you know, gets a really good first draft that you can then edit.

    12. CVClaire Vo36:06

        Amazing, and you're probably everybody's favorite community member.

    13. PBPriya Badger36:10

        [chuckles] Yeah, it's definitely a lot of fun, um, to, yeah, see what people share. And then I'll show a couple other examples. So, you know, I showed the example of creating the Yelp Assistant, and I actually used the same workflow to create this Parent Pal to explain how artifacts work to my husband, and he was really excited about it. He was like: "Hey, like, let's try it out with, you know, uh, Tommy, where Tommy throws toys down the stairs." So, you know, I did, like, you know, "My two-year-old, um, throws toys down the stairs."... and, uh, it's sim- the same kind of artifact where it's powered by Claude's LLM, and it's gonna ask me some clarifying questions like, "What's the trigger?" And it's like, "Always at dinner time when we are cleaning up." Um, and then you can, you know, see how the AI will provide some parenting guidance. And I think the really fun thing for this [chuckles] is that, you know, you can build something that's just really for your own personal use case. Um, and it's a, a really fun process to do that. I'll show one other one, which is, um, my siblings and I like to play this board game, Settlers of Catan, but the bad thing is it kind of takes a long time, especially if people don't go fast. So I'm working on this Settlers of Catan timer, where, um, I actually have a timer for me and my siblings, and both for the setup and the main gameplay. But this one I actually built in Lovable because my siblings had a lot of feature requests about tracking the feature, uh, you know, who, who's won over time, and having a leaderboard, and handicaps, and all sorts of other ideas. So I definitely think it's a lot of fun to prototype with AI for your personal use cases, and I know some PMs are like, "Hey, I really wanna work on AI products, but I don't have the opportunity right now." I think the fun thing about these prototyping tools is you can build a use case that's just for you or just for you and a family member, um, and learn a lot as you're doing it.

    14. CVClaire Vo38:10

        You just gave me such a good idea, because I don't play a lot of board games, but my kids get, like, 10 to 15 minutes of Minecraft every day, but we only have one-

    15. PBPriya Badger38:21

        Nice

    16. CVClaire Vo38:21

        ... like, a time, time timer. Um, and so [chuckles] so I need an iPad-

    17. PBPriya Badger38:26

        Nice

    18. CVClaire Vo38:26

        ... where they can, like, both click their button and have it, have it count down.

    19. PBPriya Badger38:29

        Nice.

    20. CVClaire Vo38:29

        And then they're also really worried about fairness. So-

    21. PBPriya Badger38:33

        [chuckles]

    22. CVClaire Vo38:33

        ... I will also use a, uh, relational database to store all their time-

    23. PBPriya Badger38:38

        There you go. Mm-hmm

    24. CVClaire Vo38:39

        ... and say, "I promise, every week you are getting an-

    25. PBPriya Badger38:41

        [chuckles]

    26. CVClaire Vo38:41

        ... equal amount of Minecraft time. There is no, no lack of fairness." And then when they fight about it, I'll use your Parent Pal GPT. [chuckles]

    27. PBPriya Badger38:51

        I love it. Yeah, you can just direct them to check the dashboard.

    28. CVClaire Vo38:56

        Amazing.

13. 38:57 – 41:44

    Final thoughts

    1. CVClaire Vo38:57

       Okay, last question, and then I will get you back to all your prototyping and all your AI building. When AI is not listening, other than clicking that debug button in Magic Patterns-

    2. PBPriya Badger39:07

       [chuckles]

    3. CVClaire Vo39:07

       ... what is your tactic? What do you do?

    4. PBPriya Badger39:11

       I, I think that when AI is not working, and you've already tried some of the debug, um, methods, I think it's helpful to actually think about the ways that AI is different than a human. Like, often we just get in this chat, and we're like, "This is just like talking to someone." Um, but when you're hitting the wall, it, it helps to, like, take a step back and be like, "This thing is actually not a human. Like, what could be going wrong?" And think about AI's limitations. And, you know, the ones that I try to keep in mind are it tends to lose context as you go through many different turns, and it has a limited context window. And so when you start having a really long conversation with AI, sometimes it just goes haywire. And so the, um, methods I, um, recommend are if you're doing AI prototyping, you can use that fork or, you know, a remix to start a new chat with the context of that code, and that actually resets the context window. Um, so that's a good idea if you're going really far and deep with a prototype. Um, and the same thing applies to a chat. Like, if it's going haywire, and you've had, like, 100 back-and-forths, you can ask the AI to summarize the chat and the context and start a new chat.

    5. CVClaire Vo40:22

       You gave me such a good idea with your last two answers because I am going to prototype a Parenting Pal for-

    6. PBPriya Badger40:29

       [chuckles]

    7. CVClaire Vo40:29

       ... the relationship between me and my age- [chuckles] my AI. Be like-

    8. PBPriya Badger40:33

       Nice

    9. CVClaire Vo40:33

       ... "AI Parenting Pal, my, my four-second old AI is no longer listening to me. What do, what do I do?" [chuckles]

    10. PBPriya Badger40:42

        I love that.

    11. CVClaire Vo40:42

        Um, that's, that's really great, really great feedback. And, yes, reminder, AI is not human until the AI overlords-

    12. PBPriya Badger40:48

        [chuckles]

    13. CVClaire Vo40:48

        ... take over, and then you can be whatever you want. All right, Priya, this was such a practical, super useful, inspirational conversation. Where can we find you, and how can we be helpful?

    14. PBPriya Badger40:59

        Yeah, you can find me on LinkedIn, and then I also have a Substack called AlmostMagic.Substack, where I share some prototyping tips and other tips about building AI products.

    15. CVClaire Vo41:10

        Amazing. Well, thank you for sharing and joining How I AI.

    16. PBPriya Badger41:14

        Awesome. Thanks so much for having me. [upbeat music]

    17. CVClaire Vo41:17

        Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time.

Episode duration: 41:44