How Amplitude built an internal AI tool that the whole company’s obsessed with (and how you can too)

1. 0:00 – 2:53

   Introduction to Wade Chambers

   1. WCWade Chambers0:00

      I started showing a couple of people internally. It's like, "Oh, this is really cool. You've got to look at this thing," and then a week later, it seemed like the entire company was using it. Moda is that internal tool that we have that unlocks all of the data that we have internally, and then allows us to answer questions, to build artifacts like PRDs.

   2. CVClaire Vo0:20

      What I love about these and other PRD generators is you can go from that little snippet of an idea to something much more robust.

   3. WCWade Chambers0:26

      I got to see it, and I'm all excited about it. I'm like, "When is this going to be pushed so that I can use it?" Monday, he had it pushed live. I started showing a couple of people internally. It's like, "Oh, this is really cool."

   4. CVClaire Vo0:36

      Okay, so this is my challenge to everybody listening: mark the day. A month from now, I want you all to have your own internal tools just like this, or at least a prototype. [upbeat music] Welcome to How I AI. I'm Claire Vo, product leader and AI obsessive, here on a mission to help you build better with these new tools. We've seen a lot of workflows using a lot of tools on this show, but today we have Wade Chambers, Chief Engineering Officer at Amplitude, who's gonna show us the tool they built themselves to do all their enterprise search, answer all their business questions, and I think build all their products. Let's get to it. To celebrate twenty-five thousand YouTube followers on How I AI, we're doing a giveaway. You can win a free year to my favorite AI products, including v0, Replit, Lovable, Bolt, Cursor, and of course, ChatPRD, by leaving a rating and review on your favorite podcast app and subscribing to YouTube. To enter, simply go to howiai pod.com/giveaway, read the rules, and leave us a review and subscribe. Enter by the end of August, and we will announce our winners in September. Thanks for listening. This episode is brought to you by CodeRabbit, the AI code review platform, transforming how engineering teams ship faster with AI without sacrificing code quality. Quality code reviews are critical but time-consuming. CodeRabbit acts as your AI copilot, providing instant code review comments and potential impacts of every pull request, beyond just flagging issues. CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST grep patterns, catching subtle issues that traditional static analysis tools might miss. CodeRabbit brings AI-powered code reviews directly into VS Code, Cursor, and Windsor. CodeRabbit has so far reviewed more than ten million PRs, been installed on one million repositories, and has been used by seventy thousand open source projects. Get CodeRabbit free for an entire year at coderabbit.ai, and use the code HOWIAI.

2. 2:53 – 4:55

   The build vs. buy decision for internal AI tools

   1. CVClaire Vo2:54

      Wade, thanks for being here!

   2. WCWade Chambers2:56

      I am so looking forward to this. Thanks for having me on.

   3. CVClaire Vo3:00

      One of the things that I think is so interesting about how you are approaching AI at Amplitude is you all have decided to build some tools yourself instead of plucking, you know, a bunch of various things off the shelf, and I'm curious, what was the internal thought process around this sort of build vs buy decision, or why did you go down this path of writing a bunch of code?

   4. WCWade Chambers3:22

      Well, l- l- let me start with first, um, it didn't take us as long to do it. So, um, and it's in spare time, people's spare time. They actually put, uh, what we're gonna talk about today, and so it was probably, like, three to four weeks, um, spare time of some pretty talented engineers. One, m- that's a little bit, uh, uh, in the rearview mirror. Two, in talking to a lot of my counterparts in other companies and asking them how they were approaching this and what they were doing, I had it kind of split. About half of them were saying, "Hey, I'm, I'm pulling things off of the shelf and using it," and the other half were kind of like: "No, the way that you're going to want to use this and how you're going to get more leverage, the more that you can do it internally, as long as you don't invest a lot in, you know, overbuilding it and overengineering it, uh, you'd probably be better off doing it yourself." And so once we got into it, I found that we were able to pull a lot of things off the shelf, like Glean APIs and things along those lines, which just allowed us to move really quick, and there's not so much of an investment in it that I, I worry about. If I need to, I will revisit it. I'll throw everything away, do it again. It's really unlocking people and the data that sits in the enterprise that I wanted to do the most.

   5. CVClaire Vo4:38

      That tends to be my whole, um, theme when building things with AI. It's so fast, and it's almost so cheap. I think, "You know what? If in three months I throw the whole thing away, it will have been worth it anyway," so I can definitely understand

3. 4:55 – 7:19

   What Moda is and how it works

   1. CVClaire Vo4:55

      that mindset. Okay, so we're gonna see this internal tool you built, and I'm excited to show it off. Apparently, in the, in the free time [chuckles] of engineers over three, three to four weeks, or at least a little, a little sprint. So tell us what Moda is.

   2. WCWade Chambers5:09

      Like a lot of different companies, I think that right now, um, Amplitude is going through that change of being AI native a- and wanting to, to move really fast. And so, um, I think that in all of those, uh, companies, you need to get access to the data that you have internally. You want as many people as possible to see it. You want to see others being successful with it and say, "Oh, that looks really easy. How could I do that?" Um, and so Moda is that to- that internal tool that we have that unlocks all of the data, um, that we have, um, internally, and then allows us to, to, um, answer questions, to build artifacts like PRDs, things along those lines in- internally, but with the full scope of information that we have access to.

   3. CVClaire Vo5:57

      Great, so you took all of your business data across, I'm sure, like, BI sources and documentation sources, and you exposed it in the tool, and one of the things you were telling me before we started the show is-... one of the approaches you took was this social engineering approach, which is you went into the decision of what platform Moda would be built on. So can you tell me a little bit about how you incepted the organization with the design of this internal product?

   4. WCWade Chambers6:24

      The, the, the problem is, is everybody's starting in a different place, right? And, and so, uh, you use ChatGPT at home, or you use Claude, or you use something else, and some people are really advanced and, and others are, are not. And so how do you create this common language or, or this common fluidity, if, if you will, um, around AI? It felt like if we could do something, and that made it really straightforward, and we took a lot of the complexity and trying to push it down, it would allow a lot of people to, to be able to en- en- engage with it. And so step one, and by the way, um, great minds before me... Well, I wouldn't say I have a great mind, but like great minds that I can leverage, uh, the folks over at Abnormal Security had built a little agent in Slack that I got a, a chance to see, and it allowed all their employees to ask really, uh, good questions. And I'm like: Oh, that's, that's genius

4. 7:19 – 9:17

   The social engineering approach to adoption

   1. WCWade Chambers7:19

      because i- if, if I can see what other people are doing, and it's adjacent to the work that I'm doing, I'll bo- I'll borrow the prompt, I'll borrow the, the question, or if they've already done something that I can leverage, I'll just use the results o- on the other side. And I'm like, "And that is the right way of doing this." The more that I can make this publicly visible and do real solid work on the other side of it, provide great answers on the other side of it, then it should catch like fire. Well, that's the thesis. Uh, the truth is, is that one of our engineers had this working on a Friday afternoon. Uh, I got to see it, and I'm all excited about it. I'm like: "When, when is this going to be pushed so that I can use it?" Monday, we- he had it pushed live. I started showing a couple of people, um, internally. It's like: "Oh, this is really cool. You got to look at this thing." Uh, and then a week later, it seemed like the entire company was using it.

   2. CVClaire Vo8:16

      [chuckles]

   3. WCWade Chambers8:17

      Um, it, it's, it's been pretty in- incredible.

   4. CVClaire Vo8:20

      I don't know. You might be wasting your talent in B2B. It sounds like you got some consumer thinkers there-

   5. WCWade Chambers8:24

      [chuckles]

   6. CVClaire Vo8:24

      ... with how viral this product went. [chuckles]

   7. WCWade Chambers8:28

      It, it, it's, it's helpful, and anything that accelerates your thinking or allows you to get to a deeper truth, all of those things are, are, are just awesome. So if, if this can help you with that, and you can see other people doing it... Uh, true story, right before I, I came on this, I, I was talking to one of our sales execs, and, uh, she hadn't heard of Moda yet, and so I was like: "Ah, well, that'll help you answer that question." And her first thing was, is she opened up the Slack channel, went in there, and she, and she saw a colleague, um, that's using it repeatedly a- as she just scanned down, and was like: "Well, if it's good enough for him, I'm all in."

   8. CVClaire Vo9:05

      [chuckles]

   9. WCWade Chambers9:05

      And so I'm like: Okay, that- that's the, the social engineering aspect of it. If you can see people more credible or equally credible a- as yourself having great effect with this, it's, it's an obvious

5. 9:17 – 10:58

   Demo of Moda in Slack

   1. WCWade Chambers9:17

      thing that I want to use.

   2. CVClaire Vo9:18

      Great. So let's, let's take a look at it.

   3. WCWade Chambers9:20

      Well, as an example, here's today's, um, thing, and you can already see just how many people inside of the company ha- have already been using it today. So let me go through and ask Moda to introduce itself. Um, so if I go through and just, "Hey, Moda, what are you doing?" Well, he's-

   4. CVClaire Vo9:40

      [chuckles]

   5. WCWade Chambers9:40

      ... chomping on data, nom nom. Uh, but it'll come back here quit- pretty quickly and give an answer and say: "Well, I'm Moda internal agent. I search internal knowledge sources. I can also search the public. I always cite my sources, so you can verify the information. You can access me by Slack." There's also a proprietary interface, which we'll get into. Uh, you can even learn more about the implementation and, and things a- along those lines. And so if I wanted to know a little bit more, uh, about Moda, let's just ask it to tell us a little bit more about itself. And again, it's just going to go through, parse it, send it out, gather information, and put it back and just say that it was built. We have our own, uh, little stack that we've built, uh, to be able to process, um, AI request, um, called Langley framework. But on top of that, you can see that it leverages a lot of the things like Glean, uh, to be able to, to access, um, certain parts of this, but it can do external search, and we, we already kind of covered them. Well, what about the datasets that it has access to, um, internally? 'Cause lots of people don't have the exact same things that we have, but let's go there.

6. 10:58 – 12:43

   Data sources Moda has access to

   1. WCWade Chambers10:58

      What datasets do you have access to? Well, Wade, I'm glad you asked. So we can get into Confluence, Jira, Salesforce, Zendesk, Slack, Google Drive, Productboard, Zoom, Outreach for, um, transcribed, uh, meetings, those sorts of things. Even some Gmail, Asana, Dropbox, GitHub, HubSpot. So it does not have access to private, personal, or restricted, uh, datasets. It's generally the public ones, uh, well, enterprise public ones, um, that we have internally. And so if I just was to go through and say: "All right. Well, who is using you?" Completely different way, uh, of looking at it, just how widespread i- is it inside of the company, and, and how many different groups are using this?

   2. CVClaire Vo11:43

      And so you're product managing your own AI product, [chuckles] using your AI, AI product here?

   3. WCWade Chambers11:49

      Well, e- exactly, because if I can understand where we have any friction inside of the, uh, the company, um, and what types of questions are being asked, and, and where are people not having success on the other side, it's just a simple, um, a-... process of like, okay, where did we get the rules wrong? Where did we-- how can we help out with prompting? Uh, what data, uh, sets do we need access to? Do we need to do any grooming of that on, on the output side of it? But you can see product management, engineering, sales, customer support, marketing, our CEO, um, our, our head of sales. Um, I've seen our chief product officer in here. There's, uh... Well, and you can see it just keeps going even a- as we're sitting here, uh, talking, um, about it. So these are all- [chuckles] well, I'm even a reference inside of here, but you can see that, uh, lots of different people are using it for lots of different

7. 12:43 – 17:41

   Analyzing customer feedback themes with Moda

   1. WCWade Chambers12:43

      things, um, internally.

   2. CVClaire Vo12:45

      So could you show us one of the common flows that you might, somebody might use with, with Moda? So it's pretty good at explaining itself, which is great, and it seems like it has access to a lot of data, but... Oh, here we go. And then you can customize it.

   3. WCWade Chambers13:00

      Well, you know, it, it's- I, I wanna make sure that I understand everything that's going on with Moda, so I'm gonna actually give it a, a little bit of attitude. Uh, and let, let's go through there. So I want to go through and actually start... I mean, almost everything that our customers do, they need to understand what's going on, then they wanna make a decision, then they wanna act on it, so we need to do the same thing here. Uh, why don't we actually go use all of these data sets that we've got access to, to find out, um, what's going on? I don't know that we actually need to see every- the top themes and queries, but maybe it's a good place, uh, to start. I'm just looking for an insight that we can kind of chase down, um, inside of.

   4. CVClaire Vo13:42

      No, I think this is great. So I think that thematic analysis and sort of trying to quantify more qualitative or, yeah, more qualitative feedback is a very common kind of product manager use case. And so anything that can have access to a broad set of business data and do that an- analysis for you- [chuckles] oh, my gosh, these, these descriptions are ridiculous. [laughing]

   5. WCWade Chambers14:07

      Isn't this great?

   6. CVClaire Vo14:07

      It really gave you Gen Z slang.

   7. WCWade Chambers14:11

      Bet.

   8. CVClaire Vo14:12

      Bet. It... [chuckles] Bet. [laughing] No cap. All right, so, so this is doing something that I think, uh, would be very popular for a product manager. And so you're saying, "Let's take all this context I have, analyze it, quantify it," and then it actually gives you the scope of its, of its research data. So it says it takes the fifty most recent Slack messages, so that's a good resource. But then you're looking for another one with more external resources. So that last query was really about the Moda bot itself, but this query looks like it's about your actual product, and you're looking for real customer feedback, and this is a very common workflow that a lot of product managers do quite manually.

   9. WCWade Chambers14:56

      Absolutely right. And, and so, you know, go into Productboard, go into Zendesk, where things are going to be logged. Actually go into the transcriptions of conversations that we've had with various, um, customers, and let's get to a point where we believe we know where there's some energy, um, where people want extensions to the product or want, uh, um, to improve the functionality of the product, like connecting session replay to funnel analysis. That seems like a, a pretty good place to dig in. So why don't we just jump in and see what Moda's got to tell us about that?

   10. CVClaire Vo15:35

       Got it. So you're taking this wide funnel of data analysis, trying to figure out what are the subthemes, and then you can use Moda and say, "Okay, I've picked- plucked a subtheme. Give me even more, more data a- around it." And do you think that's a, a useful flow for folks, basically, like, start at a pretty wide data top of the funnel and narrow, narrow in? Is that how you approach things?

   11. WCWade Chambers15:56

       Yeah. Uh, I, I, I think it's useful when you're going through and saying, like, "Okay, what, what could I learn from this? What's something that, that's new?" And so this was me trying to start at the very top and say, "Okay, let's go in and dig into... Well, let's start at the top and say, 'What are people asking for?'" And then if people are asking for, for this, let me make sure that I, uh, believe that, you know, what, what they're actually asking for, there's actual quotes, there's details behind it that I can look at, and we could even ask the opposite, um, of this. And so there's plenty of conversations that come through in this of where you can see, um, specifically, you know, from Zendesk and, uh, from Zendesk, from an outreach call, and so you can see the details of specifically what somebody is asking for and what they're trying to do inside of that, which gives me a, a sense of like, there's some heat here.

   12. CVClaire Vo16:49

       As an AI founder, you're used to sprinting towards product market fit, your next round, or that first enterprise contract. But speed isn't enough for AI startups. Buyers expect security, compliance, and transparency from day one. That's why serious AI startups use Vanta. With deep integrations and automated workflows built for fast-moving AI teams, Vanta gets you audit ready fast and keeps you secure with continuous monitoring as your models, infra, and customers evolve. AI innovators like LangChain, Writer, and Cursor scaled faster and closed bigger deals by getting security right early with Vanta. Listeners can claim a special offer of one thousand dollars off Vanta at vanta dot com slash howIAI.

8. 17:41 – 23:24

   Behind the scenes: how Moda works technically

   1. CVClaire Vo17:41

      For somebody who is building AI products for product managers, I know there's a whole cottage industry that is trying to build SaaS products for what you just, [chuckles] what you just showed on this in- internal tool, and I'm curious if your product managers have, by and large, been quite happy with Moda of their... as their source of insights, or if you feel like there are pieces that this kind of general purpose internal tool is missing, maybe to serve this specific use case better?

   2. WCWade Chambers18:12

      ... I think that, um, a, a lot of product managers that are here take this as a way of getting access to a hundred percent of the data and seeing what AI would generate for it. And if it comes back and it doesn't match their instincts, they'll dig in. Um, and if it does match their instincts, then they'll look for more, more detail inside of it. And so I think that for the most part, we have a lot of product managers using it to great effect, and I'll go into to more detail on how it actually, um, does this. Um, we have a proprietary interface. Actually, let's jump over there real quick.

   3. CVClaire Vo18:47

      Yeah, I would, I, I would love to see that 'cause I think my next question is, like, how does it work behind, behind the scenes for folks that say, "Well, I got a, I got a smart engineer in three weeks. How can I have this, too?" I'm curious how you've approached building this, a little bit more detail.

   4. WCWade Chambers19:03

      Yeah. Then, and the, the, the nice part is that you can even ask Moda, uh, on how it works. We, we have a framework that we've used internally, and we have both a, a, a little bot that sits in Slack that can call out to that framework, but you can see that I can go into a web UI and basically be able to access the, the same thing. In this, um, flow, we've actually got a lot more capabilities that are associated with it, and so we can go in and create, uh, PRDs as an example. You can ask anything. You can do, do, do deep research on it, but you can create a PRD that then allows us to take some piece of this and, uh, go in even deeper. And if I was to go in, you can kind of look at just jumping into GitHub. Uh, right, here's the YAML that kinda defines the, the high level, how we would do a Glean search on ask anything. Uh, we could go back up and even look at the PRD orchestrator on this side, and you can see, you know, how it starts with a prompt and is able to sort of dig in a little bit more based on what you're trying to do. If it needs more details, it's going to ask you for that. It's then going to take all of this and break it down of where it will break things into, "Let's make sure that we do proper problem exploration, solution explo- exploration, detailed requirements that come out the other side." We like to move as quickly as possible to a prototype, and so if it can do prototype generation or at least the prompts, then, then we can plug in... Uh, we can copy and paste into Bolt or you pick your lovable, uh, v0, uh, pick your, your favorite, and be able to move it around, then we can get through that, uh, fairly quickly. And then it will just go through using the Glean API, we'll get access to a lot of the content and be able to do a query, and so it- we use that as, um, part of our RAG to make sure that we go get the right content and then pass it off, and it's able to evaluate all of that and come back with a, a much better answer.

   5. CVClaire Vo21:06

      Yeah. I have a couple technical questions 'cause as a builder, I'm just so curious here. So it sounds like you're using the Glean API for a bunch of your enterprise search, so that simplifies a little bit the data access and controls and all that piece, the RAG piece of it. Then, uh, you've built these two kind of alternate interfaces, which is the Slack bot or, or the web UI, which is nice, and then it seems like you've built also some kinda specialized tools in terms of kind of the general chat, deep research chat, or this PRD flow. And then just looking at that GitHub, if you don't mind pulling it up again, I'm so curious, who got this good at writing prompts? 'Cause this is a [chuckles] this is a well-structured prompt, and I think one of the things that is very mysterious to people right now trying to build something like this is they vaguely know the scent, like a sense of like tool calls, and they vaguely know about instructions, but no less about, like, these sequential instructions or multi-tool or parallel tool calls. And I'm just curious how your team up-leveled or up-skilled on how to build great prompts, how to build great agents. Was that just learn as you go?

   6. WCWade Chambers22:12

      There was part of it that was learn as you go. I mean, we, we've got a couple of talented folks that have a lot of experience with AI, and so I think we were able to use those. But also, AI is a good tool to use for building prompts, and so you can just recursively ask AI to give you better prompts or things that would allow you to focus on very specific things that you want in the result set, and AI will actually generate the prompt for you that you can use. You'll probably have to edit it a little bit, but it does a pretty good job on its own.

   7. CVClaire Vo22:44

      And then how does your team improve, improve this over time? Is it open to anybody to do a pull request on it? Is there a team that owns it? How have you set it up operationally?

   8. WCWade Chambers22:55

      Yeah, we've, we've set it... I mean, it's all checked into GitHub. Um, most of our engineers can, can check it out. Even, um, product managers and designers can probably do the same, uh, to it. Uh, we've had designers that are contributing to this and product managers who are contributing to this. We're currently going through an AI week, uh, this week, of where there's a, a lot of things that are going on. One of my vibe coding ideas is I wanna be able to add NCTs in addition to PRDs, uh, to all of this and see if I can do it totally on

9. 23:24 – 27:30

   Creating a PRD from a single customer insight

   1. WCWade Chambers23:24

      my own.

   2. CVClaire Vo23:24

      Okay. Well, speaking of PRDs, is there any way I could get you to create a PRD with this flow? I'd love to see that.

   3. WCWade Chambers23:30

      Let's... Let, let's do it.

   4. CVClaire Vo23:31

      Okay.

   5. WCWade Chambers23:31

      I was going to go through here and say, uh, let's go through- I'm gonna see if it will take it as is.

   6. CVClaire Vo23:39

      Okay.

   7. WCWade Chambers23:39

      Uh, let's actually go through and create a PRD.

   8. CVClaire Vo23:42

      So for people listening, in our Slack flow, we, we got this insight that people wanted session replay attached to funnel steps, which makes total sense, and so now we're taking that idea and that context and going into the Create PRD Prototype flow in Moda, um, which is asking us what we wanna build.

   9. WCWade Chambers24:01

      So let's just give it the, the prompt that we had, uh, as, um, an answer to a previous question and see if it's able to expand on that.

   10. CVClaire Vo24:08

       Yeah, and again, for folks listening, this is one sentence: "The customers want to see session replays directly linked to funnel steps, so they can watch where users drop off or convert." And what I think is so powerful for product managers is-... They really, uh, convinced themselves for qu- for many years that you needed many more sentences than that [chuckles] to convey what kind of product you need. But what I love about these and other PRD generators is you can go from that little snippet of an idea to something much more, much more robust.

   11. WCWade Chambers24:38

       Yeah, and it, it's gonna go through it- its stages, and it's going to think a little bit, and it's gonna dre- generate things. So I'm just going to very quickly jump over and say, this is kind of what it will produce, and it also produces all of the PRDs in a place that where anybody can go look at them and see the output of it. But it will talk about the problem expl- exploration. It will talk about the actual, uh, solution exploration. It will talk about the detailed requirements that come through on top of it, and then it will go through, and it will generate, um, things that you need to do to generate a prototype on it. So you- it'll give you all of the prompting that you need to do it. In this specific one, they took those prompts and actually fill- fed it to Bolt, to Figma Make, to v0, just to, with the same exact, uh, prompt, see what different systems would actually suggest as a, a great prototype that, that you could interact with.

   12. CVClaire Vo25:35

       And is this the product manager that's taking the output of this automatic PRD generator, which includes prototype instructions, and then they're just copying and pasting that into the prototype tool of their choice, and then comparing the outputs and putting them in a Confluence doc for people to look at?

   13. WCWade Chambers25:52

       They don't even have to put it in a Confluence. Uh, yes, that's exactly what they're doing, um, on, on the other side of that. For everything else, it will generate and, and put it into the Confluence document. It still looks like it's still thinking, uh, about it here, but that's exactly what happens here, and as soon as this comes back, we'll look at some of the results that are associated with it and go create our own prototype, uh, from it.

   14. CVClaire Vo26:15

       What I like about the design of this internal flow is it's clearly multi-step without user interaction, which is quite, [chuckles] quite interesting. And so instead of this sort of iterative, "Do you like this? Do you like this? Do you like this? Do you like this?" Seems like you've built something that you're pretty confident is gonna get you close to what you want with very little human intervention. Of course, you can come and edit it in here, but I'm curious if that was an intentional choice or what drove that sort of kind of decision to say, "Just get it all done at once."

   15. WCWade Chambers26:48

       We, we decided-- I mean, it is a multi, um, step process, but one of the things that you can do is you can go in here and actually say, "Oh, I don't agree with something in here." You can create a comment, um, associated with it, and then you can tell it to go reevaluate things. And so you're not stuck with the answer that you got. Uh, based on your ability to comment, you can actually go and change things pretty rapidly. So if you don't think it got it right, like, the downstream consequences are, are fairly minimal because you can just go to as high up in the stack as possible and say, "Well, the problem isn't right. Let me actually change some things along this." Just regenerate everything that's beneath it, and you can just keep going down the, the stack as

10. 27:30 – 29:09

    How teams actually use AI-generated PRDs

    1. WCWade Chambers27:30

       you need to.

    2. CVClaire Vo27:31

       So I have to ask you about the, um, AI-generated product document elephant in the room, as somebody who's thought about this for a long time.

    3. WCWade Chambers27:38

       You've thought about it a long time.

    4. CVClaire Vo27:39

       I've thought about it a lot. You have created five beautiful, detailed assets in, I don't know, three minutes. Does anybody actually read them, or do they just click right to that prototype and [chuckles] and say, "Yes, this is what we want"?

    5. WCWade Chambers27:54

       We, we do review them, and matter of fact, if you go through and you look at, like, the detailed, um, requirements, almost every document that we produce on, um, the side will have a review segment that we go through. And so we'll have people go through, actually look at the problem statement, look at the, um, solution statement, and we actually go through a review process to make sure that we agree. And honestly, if, if the person who is doing the generating hasn't also done some follow-up queries to say, you know, "What are the cons on this? And is there evidence that suggest other answers would be better?" And even when you get to the prototyping, uh, phase, what are the multiple solutions that you looked at? What did you generate from a prototype perspective so that we can see three different variations, maybe four different variations on the other side, and that will ch- and that will force you to change your prompt as well. And so all of this, you, you... Or at least I don't feel like we've gotten to that place where you can just go YOLO. You're going to have to-- it's, it's going to be an assist, it's actually going to speed up things, and in many cases, it gets it perfect, but you can't assume it's going to. You actually have to apply critical reasoning to see where it

11. 29:09 – 32:37

    Impact on product development velocity

    1. WCWade Chambers29:09

       may have failed you.

    2. CVClaire Vo29:10

       I'm so curious because, again, you've compressed a lot of work. You've compressed user research, quantitative, qualitative research, idea generation, PRD generation, prototype generation into a very small, compressed timeline using all the business data that you have, built by an internal tool. I'm curious what the downstream effects you're seeing in the product and engineering and design organizations when something like this can get done so quickly. Are you, are you building more things? Are you getting more ideas? Are you getting better ideas, worse ideas? I'm curious your point of view of what this is changing.

    3. WCWade Chambers29:45

       It's definitely changing the velocity, number one. Uh, and, and so we see that in, you know, six months ago, eight months ago, um, you know, w- we're, we're an agile shop. We move pretty quickly. Um, you know, we, we employ a lot of Scrum and, and able to sort of iterate through, uh, things pretty quickly. But even then, you, you would say that there was somebody who was out there doing the research and needed to try and put it into a document so that other people could review it. And then when other people would review it, you would, uh, actually then move it into design, and somebody on the design side of things would, uh, use Figma to actually build some mock-ups and, and things along those lines, which then would get handed over f- to engineering for-... inspection and trying to figure out, like, "Okay, how do I turn this design into something that's working code on the other side?" That could take weeks, a- and I, I think the best case [chuckles] is that it took a couple of weeks. Well, maybe, maybe you could get it done in a, in a week. And now we can actually put those three different roles together and actually produce that in a single meeting, where we're going through and using Moda or, um, other tools a- along those lines to actually say, "Let's go find evidence." Do we find customers are actually asking for this? Okay, um, you know, like, what's the right context to provide to Moda to make sure that, that we got it right, or can it provide us that context by searching through all of the enterprise data that we've got? And we'll get to a prototype in a very short period of time. So now, when we do product review sessions, the PRD is a part of that, but oftentimes we're looking, um, to get to the prototype as quickly as possible and work backwards from that.

    4. CVClaire Vo31:32

       Do you feel like you have more ideas than capacity to execute, or are you keeping up speed on the engineering side because you're using all these AI engineering tools? I'm always wondering wh- you know, where there's a misbalance in the force, [chuckles] where it comes from. Um, 'cause you sit on a lot of prototypes, and then I know you have a complex product. I'm just curious how you approach building those.

    5. WCWade Chambers31:51

       It does move around a little bit, a- and that we'll find that, um, if we're really trying to figure out a concept, you know, maybe Moda plus some, uh, prototyping tools can actually get you most of the way there. If it's something that is a product direction or a, an entirely new product, um, you're probably going to need to go do some market research. Um, if it's something that is UI-heavy or, you know, deeply integrated with a part of our product, uh, we'll probably need to slow down a little bit and, and give design the time to actually go through and make sure that they've, they've, like, stitched it all together, and you can make sure that it's complete, coherent thought. So it's a little all over the place, uh, depending on the type of project and the work that

12. 32:37 – 34:38

    Engineers, designers, and PMs swapping roles

    1. WCWade Chambers32:37

       we're looking at.

    2. CVClaire Vo32:37

       And then I can see how... 'Cause I, I love this idea, and I've, I've spoken about it before, that product design and engineering can all be done in a single meeting, in this, you know, single flow. I'm curious, do you see your team swapping roles? Like, do you see engineers going, "All right, the PRD," or PMs being like-

    3. WCWade Chambers32:54

       Absolutely

    4. CVClaire Vo32:54

       ... "Let me, let me put up a PR?" [chuckles]

    5. WCWade Chambers32:56

       Absolutely. A- and, uh, we've actually intentionally done that at times of where we've said, "Okay, you're going to take on a different role." And so once, um... We even had a demo where we had, uh, like, the designer being the engineer, the engineer being the product manager, the product manager being the designer, kind of in the role to just show, like, how you could work through it. It was hilarious, it wa- it was fun, but it was actually very functional. Um, the designer actually got into Cursor and was able to, to extend some things in Cursor. Uh, the engineer was able to co- I mean, very, very good product thinker any- anyhow, uh, but they were able to come up with, like, the right PRD and the right requirements. Uh, even the product manager that, that was there was able to, um, get in and do a better-- multiple iterations on a design until they actually found something that, that hit the sweet spot.

    6. CVClaire Vo33:52

       This is a workflow I have not heard before. So for people listening, I want you to do it. I want you to take a Friday morning, bring your team together, screen share, and do a little role swap, 'cause that's- it's genius just to show it's possible or see where there are struggles, um, see where there's opportunity. I'm also sure it gives empathy between the teammates, saying, "You know what, what you do... When I say just vibe code it, maybe I'm being a little silly, or when I say we can just make the prototype, I understand now why, why we have UX designers." So it's a nice, it's a nice skill development workflow, but I bet it also brings the team kind of closer together in terms of empathy and respect for each other's craft.

    7. WCWade Chambers34:30

       Empathy, respect, and, uh, just, like, fluency in each other's craft and how AI can help with that.

13. 34:38 – 36:00

    Recap of creating Moda

    1. CVClaire Vo34:38

       Okay, so you're setting the- you're very calm, but you've set the bar very high. So just to recap, um, you've told us that you don't have to buy it off the shelf, just pluck a couple engineers and a couple weeks, and build, [chuckles] build this thing by yourself. No big deal. It'll just have all your enterprise data in it that you can query on demand anytime you want. Uh, you built it in Slack and a UI so that your whole team can both access it, as well as see each other's use of it and kind of learn from that. And then you've built these specialized tools, and of course, you know, our audience's favorite is gonna be this PRD to prototyping tool that kind of takes the best of all those workflows and puts them together for a purpose-built, reusable flow that can get your business something that you really need faster. No big deal, all while running an amazing company that tons of product people just, just love, right?

    2. WCWade Chambers35:30

       There, there's a fair amount of true- I mean, uh, I, I, I may have minimized, uh, like, how much work it was, but like, honestly, it was not a bunch of engineers full time working on this for quarters or anything along those lines. It literally was, um, four weeks and part-time with a few engineers.

    3. CVClaire Vo35:50

       Okay, so this is my challenge to everybody listening: mark the date, put a month ahead, and from a month from now, I want you all to have your own internal tools just like this, or at least a prototype.

14. 36:00 – 40:26

    Lightning round and final thoughts

    1. CVClaire Vo36:00

       Okay, Wade, I am going to send you on your way in just a few minutes, but let's wrap with a couple lightning round questions. My first is, we've talked a lot about product and business data, but you're a builder running engineering organizations. What are you excited about on the engineering side? What are you nervous about? Kind of what are your thoughts on all this AI-powered coding?

    2. WCWade Chambers36:21

       Honestly, I, I've never worked at a place [chuckles] where, uh, it felt like we had tech debt under control, and everything was fine, and we didn't have too much surface space. Every company has those challenges. This just gives us a way of being able to deal with those things much more effectively moving forward. There's work that we have to do on our side to make it, uh, more AI-friendly, so that AI can do more work on our side, but this is going to give us the ability to do so much more for our customers.... I'm truly excited. Take the same engineers and multiply their value, uh, based on these tools, and, and just think about what we're going to be capable of doing. I'm genuinely excited by what this means for us.

    3. CVClaire Vo37:03

       Yeah, and I'm glad you called out tech debt and all those challenges that engineering orgs have, because one of my pitches to software engineers is, like, reduce toil, get rid of misery. You know those corners of the app that you hate but you tolerate because you do not have time?

    4. WCWade Chambers37:19

       Exactly.

    5. CVClaire Vo37:19

       Now you have a tool that you didn't have, that you didn't have before on those, so I think that's a great call-out. And then you have built, and your team has built, such a structured, full of personality, [chuckles] internal assistant. But I'm curious, what is your strategy when you have asked Moda to generate a PRD, you know, five times, it's not doing the right thing, it's not listening, or ChatGPT, or whatever your frustrating AI tool of choice is? How... You know, what's your prompting strategy? How do you get it to listen?

    6. WCWade Chambers37:48

       I have a few different strategies. One is I swim upstream. It's like, where did it start to go wrong? And like, let me edit that and actually go through and, and, and see if I can generate a different result on the other side. I always feel like it's- it was an input problem on my side, so if I can figure out where it started to go wrong, let me change that and put it on a better path. Uh, number two is I'll just give it feedback as we're going through it, in a nice way, because, uh, you need to be nice to your AI. Um, but [chuckles] I will go through and, and say, "Hey, I- this didn't quite hit the mark. Um, I was looking for something that felt a little bit more X or Y. I needed more detail here. I want to be able to use this to describe it to my grandmother. I want to, um, have multiple use cases, or I want to hear the customer's voice come through in this a little bit more concretely." Um, I feel like the more that I can give it context, but also tell it what I needed out of it, um, after multiple rounds, you'll either figure out what it needed, um, or it'll figure it out on its own and help you get there.

    7. CVClaire Vo39:04

       Well, I'm gonna give you a compliment, because both of those strategies speak to a very good engineering leader. One, I was like, "Oh, you just go back to the last good commit, and you start over [chuckles] again," and two, how you described giving feedback to an LLM is exactly how people want feedback from their manager. So that came, came through loud and clear. All right, Wade, this has been so fun. It's really interesting to see this, um, behind the curtain, see how a company like Amplitude has built this themselves, some tools that can be really practical for their, their team. How can we find you, and how can we be helpful?

    8. WCWade Chambers39:39

       Uh, LinkedIn is probably the best way to find me personally. Um, and I will say, you know, there's going to be some news coming from Amplitude on, uh, um, some agentic solutions. Stay tuned. Um, it's gonna be a lot of fun.

    9. CVClaire Vo39:53

       Amazing. Well, thank you for being here. I appreciate it.

    10. WCWade Chambers39:56

        Awesome. Thank you.

    11. CVClaire Vo39:57

        [upbeat music] Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time!

Episode duration: 40:26