How this PM streamlines 60k-page FDA submissions with Claude, Streamlit, and clever AI workflows

1. 0:00 – 3:01

   Introduction to Prerna

   1. CVClaire Vo0:00

      So you are working in the life sciences on really health-impacting vaccines and treatments, and with all that amazing investment and scientific work also comes a lot of paperwork.

   2. PKPrerna Kaul0:15

      We had to develop a nearly 60,000-page document. Would have taken about four to six months of effort and nearly 20 specialists, not to mention the millions of dollars spent.

   3. CVClaire Vo0:26

      Where did you start?

   4. PKPrerna Kaul0:27

      I gave Claude the problem statement, the pitch, and a demo. The thinking I had in mind is that Claude is a software engineer, and I'm talking to them and trying to tell them why it matters, like any good PM would, and trying to tell them what end product we want to produce as a result. And not only did it fully understand what I was asking it to develop, it created a little setup instructions markdown file for me, and it gives me all of the setup instructions and all of the tasks that it does and its capabilities, which is super handy.

   5. CVClaire Vo0:57

      Were you really able to shift the cost there? Did you see an impact here?

   6. PKPrerna Kaul1:01

      We did. We did. And any kind of cost savings you can generate have a direct impact to the bottom line. And in addition, if you're saving on time, you're bringing life-saving vaccines in the hands of people who actually need it.

   7. CVClaire Vo1:16

      [upbeat music] Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today, we have Prerna Kaul, who's taken years of machine learning experience and helped develop some of the products you know and rely on every day at companies like Amazon Alexa, Moderna, and Panasonic. She's going to show us how she uses AI to accelerate drug discovery and vaccine approval, something that takes tens of thousands of pages of documents, dozens of people, and months and months and months of work. She'll also show us how you can use Jane Austen, Dale Carnegie, and some AI to manage your stakeholders just a little bit easier. Let's get to it. This episode is brought to you by CodeRabbit, the AI code review platform, transforming how engineering teams ship faster with AI without sacrificing code quality. Quality code reviews are critical but time-consuming. CodeRabbit acts as your AI copilot, providing instant code review comments and potential impacts of every pull request. Beyond just flagging issues, CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST grep patterns, catching subtle issues that traditional static analysis tools might miss. CodeRabbit brings AI-powered code reviews directly into VS Code, Cursor, and Windsurf. CodeRabbit has so far reviewed more than ten million PRs, been installed on one million repositories, and has been used by 70,000 open source projects. Get CodeRabbit free for an entire year at coderabbit.ai, and use the code HOWIAI. Prerna,

2. 3:01 – 5:20

   The FDA submission challenge: 60,000 pages, months of work, millions in costs

   1. CVClaire Vo3:01

      it's so nice to have you. I'm really excited about your workflows.

   2. PKPrerna Kaul3:05

      Thank you so much for having me. Very excited to be here.

   3. CVClaire Vo3:08

      So you are working in the life sciences on really health-impacting vaccines and treatments, and with all that amazing investment and scientific work also comes a lot of paperwork-

   4. PKPrerna Kaul3:24

      [chuckles]

   5. CVClaire Vo3:24

      ... if I am, if I am correct. And so in addition to doing all this amazing research, you also have to document and submit a lot of stuff to the FDA and other regulating industries, and I am sure that is a total pain. So tell me a little bit about the problem that led you to building some of these AI workflows.

   6. PKPrerna Kaul3:45

      Well, yeah. Firstly, thank you so much for having me, and, uh, you're totally right. Uh, with life sciences, I think the challenge across all parts of the organization is, uh, a lot of paperwork, a lot of regulation, and, uh, going back and forth with these regulated industries could take months of time, not to mention all of the people resources involved and delays in getting, in some cases, life-saving drugs and vaccines to market, uh, that people need. For example, when I was working at Moderna, we were developing the INT cancer vaccine, which, uh, is necessary for, uh, folks with skin cancer. It was sort of a new niche concept that had come out, and we had to develop a nearly 60,000-page document, which is known as the... It's, it's, uh, technical jargon, but, uh, biological license agreement, which took around four to six-- it would have taken about four to six months of effort and nearly 20 specialists, not to mention the millions of dollars spent across the organization in getting this document together and providing it to regulated authorities across the world. Very labor-intensive, very time-consuming, and, uh, that's where I kind of came in. I was a GenAI product manager there, and really came in to think through, "How can we reimagine this using generative AI?" Uh, this was back in 2023, where I came up with the solution, so a couple of years back. Uh, and, uh, I kind of evolved it over time to kind of specialize in, uh, certain areas as well, and I'd love to show what we came up with.

   7. CVClaire Vo5:19

      So it

3. 5:20 – 10:13

   Getting started in Claude: from prompt to production-ready prototype

   1. CVClaire Vo5:20

      sounds like you created a really high-impact tool, and 60,000 pages [chuckles] is pretty, pretty intense. Um, what I love about this workflow is you're going to show us actually how you used GenAI to build this GenAI solution. So, like all good product people, you started with requirements. Can you just show us your process from you've, you've identified this problem, which is we've got to create this massive document, and it's going to take millions and millions of dollars and tons of people, to actually having this production thing that, that your team and colleagues could use? So where did you start?

   2. PKPrerna Kaul5:51

      Like every good PM, started with the requirements, and, uh, I actually didn't, uh, hadn't fully formed, uh, all of the ideas initially, because, um, I think that-... with, uh, with the complexity and the domain knowledge required, it took a few iterations, too. But I'd love to showcase sort of where we started with very simple pitch, and then go through how we actually evolved the idea as well, so-

   3. CVClaire Vo6:16

      Great!

   4. PKPrerna Kaul6:17

      So I, I gave Claude, uh, the problem statement, the pitch, and a demo. The thinking I had in mind is that Claude is a, a software engineer, and I'm talking to them and trying to tell them why it matters, uh, like any good PM would, and trying to tell them what end, uh, sort of product we wanna produce as a result. So I've included statements such as, you know, "This is, uh, Stéphane Bancel's, uh, who is Moderna CEO's passion project. He's very excited about it. Here is the cost savings you can get. Here's the- here's why it matters and the impact," and then also spoken about the demo that I wanna showcase, and how we could sort of iterate and build on it. And Claude came back to me with, um, not only did it fully understand what I was asking it to develop, it created, like, a little setup instructions Markdown file for me, and it gives me all of the setup instructions and all of the tasks that it does and its capabilities, which is super handy. Not only did it do that, it also gave me all of the code and all of the different aspects to the, to the code as well. So I kind of started here and started thinking through what, what do I really need to do to make this a production-ready system? And there were two things that were extremely important in the process. The first is because the BLA is a structured document, it is XML-based, it has certain formats, I needed Claude to successfully, in a very strict and structured way, take data and structure it into a certain format. So that was a very strict requirement, and that's not super easy to get with some of these models as well. The second was that because it is clinical data, it's highly regulated, and being able to detect PHI, redacted information, was extremely important, so that it could correctly redact that and then not have that included in the actual BLA that we would be sending to any authority, because it's, uh, patient data, essentially. So the reason why we chose, uh, Anthropic Claude is because of the reputation around safety and alignment with human values as well. And, uh, we formed this partnership with Claude and really focused on some of the PHI and the safety aspects to make sure that, uh, not, not just from a compliance standpoint, but also as we think about alignment, we were taking care of those use cases. So that- this is what it came up with. It did a really good job, and I was able to get, uh, some code running as well, uh, and I'd love to show you what it came up with.

   5. CVClaire Vo8:50

      If we can go back to the requirements really quickly, there's just a couple of things I wanna call out for folks that I think is really interesting. If you go to the step-by-step instructions, what I love about it is, if you go to the top, it not only gives you [chuckles] sort of how to technically write the code, but something I wanted to call out at the bottom is, it actually gives you a pitch narrative to how to run your demo. So what I think is so interesting about this specific prompt is you gave it kind of pretty loose requirements. It's only a couple paragraphs up top, and it gave you a whole set of things, I'm sure many of which you weren't anticipating. I wouldn't have anticipated getting a demo script from, from Claude when I'm asking it to build Python [chuckles] ， but, but it gave you that. I thought that was really interesting. And then if you scroll down in the left side, something else I'd like to call out for folks that I think is really interesting is, it gave you these kind of, like, component names of different technical pieces of, of the product. So, like, coming up with mock data, doing PHI redaction, all that kind of stuff. You know, you as a software engineer and a product person, how long do you think it would've taken you to come up with this full set of just this first one-shot prompt requirements before?

   6. PKPrerna Kaul10:07

      At least a couple of days, [chuckles] at least about a week to really think through everything.

   7. CVClaire Vo10:12

      And then in addition

4. 10:13 – 12:04

   How Claude selected the right models for medical entity recognition

   1. CVClaire Vo10:13

      to knowing you wanted to work with the Anthropic models because of safety and alignment, were there other libraries or other models that you knew you wanted to use? Did it pick the right ones? Like, how close to the technical implementation did it get?

   2. PKPrerna Kaul10:26

      In particular, with the, with the redaction piece, it would've taken a couple of data scientists to figure out the right solution and work with us. Uh, so data scientists as well as software engineers to work with us and figure out what's the right solve for this, how do we implement it, how do we test it against certain synthetic data? So I think where it really excelled is in picking the right models, and I have some amount of machine learning knowledge, but it would've taken me a couple of days of study to figure out, okay, this is the right model for medical named entity recognition, and this is how you go about it. But it did such a great job with very little information upfront on the domain itself. It, it was able to absolutely pick the right models and give me the code, and then implement it and show me the confidence code across that as well.

   3. CVClaire Vo11:15

      Yeah, and if we look at the code, I just have to laugh because, you know, many, many years ago, I think I have lost hours and hours and days and days of my life to writing RegEx. [chuckles] Like, just these-

   4. PKPrerna Kaul11:28

      Oh, oh, yes.

   5. CVClaire Vo11:28

      Just... I, I used to spend all my time writing RegEx. Terrible. That didn't work. It was just awful. And so one of the kind of, uh, miracles of AI I see is it reduces this toil in technical tools you know you need to use that have documented patterns, but they're just so hard to research and get perfectly right out of, out of the gate. So this was my favorite part of seeing, seeing your code up, is seeing all that code just pre-populated for you.

   6. PKPrerna Kaul11:56

      Yeah, that's, uh, such a common and shared pain point. [chuckles] RegEx has been a... Yeah, it's a bane of any software engineer's existence.

   7. CVClaire Vo12:03

      Yeah.

5. 12:04 – 16:04

   Using Streamlit to create accessible UIs for non-technical users

   1. CVClaire Vo12:04

      So, okay, so it created this Python script for you, and you said you scrolled through it, and it looked-... you know, pretty correct. And then you decided to deploy this on Streamlit. Can you tell us a little bit about that choice, why you picked that?

   2. PKPrerna Kaul12:18

      Yeah. So I love using AI to solve for some, and automating my own work, and solve for my own sort of, uh, day-to-day problems, either be it related to work or something I'm trying to work on for, in, in my side projects or personally. But I think the real unlock, at least I've found, is that if I can create something of value for other people, and then, uh, especially as they are non-technical folks, um, making a UI and making something generally available and hosting it locally, has proven to be a really, really big unlock, both career-wise and also, you know, working with different colleagues. They find that to be really valuable because they might have something in mind, and I have all these skills, and I can, and I can sort of offer that. Uh, so that's actually something I've been doing in my current organization as well, where, uh, I recently created a Google add-on to convert PRDs into Jira format, and, uh, that was a lot of fun. And, you know, the product managers especially, were very excited about it, uh, because, uh, they don't necessarily like having to write requirements twice, uh, one in a PRD and then also in Jira. Um, and, uh, we, we found a good, uh, way to kind of circumvent that and do it in a matter of minutes. So I did that, and then the Streamlit idea, I also did in a couple of my previous roles and my current role as well, to give non-technical users access to all of these tools.

   3. CVClaire Vo13:41

      Yeah, and just for folks that don't know, Streamlit is a really easy way to put a pretty simple UI on front of, like, a, a Python kinda file and set of APIs.

   4. PKPrerna Kaul13:52

      Exactly.

   5. SPSpeaker13:52

      Agree.

   6. CVClaire Vo13:52

      So Claude created your PRD. It gave you code. You were pretty happy with the code. How do you get from us looking at this code right now to a product that your team could actually use to create these documents?

   7. PKPrerna Kaul14:04

      I'm gonna run for you, uh, the actual Streamlit app.

   8. CVClaire Vo14:08

      Mm-hmm.

   9. PKPrerna Kaul14:09

      So I'll take you to terminal. I essentially run it from the command line, and it opens up a web app. It takes me here. It has pre-loaded an API key from Claude, which I've provided in the code, so I've hidden some of those properties. And, uh, I'm gonna generate synthetic data for us, and I'll talk through-- and I'll go through the step as well, 'cause it takes a bit of time, and I'll talk through what it's done. So, uh, as I mentioned, uh, there's this format called a Common Technical Document as part of the BLA. There are several modules, about five or six different modules, and it has a structure around it. So there is the, there is a study report that has a tabular listing. It's supposed to have, uh, very specific, uh, points across the data from the clinical trials and then different literature references as well.

   10. CVClaire Vo14:57

       Did Claude know about this format kind of from just general knowledge, or did you have to provide the format to it?

   11. PKPrerna Kaul15:05

       So it searched up, uh, the Common Technical Document, uh, format for me, and then it figured out the right module to apply to my use case.

   12. CVClaire Vo15:15

       Great! That's awesome.

   13. PKPrerna Kaul15:16

       Yeah. It's doing pretty well. So I'm running here what is... Uh, I'm going through the different-- we have different participants here, and, uh, there are some clinical notes per participant there. This is all clinical trial data, very, very, uh, sort of common that you would see in, in any trial that runs. And by the way, the trials itself, they could run in the order of billions. So redoing a clinical trial is, is very, very expensive, and so as, as much as possible, we wanna give very accurate, uh, information to the FDA so they don't ask us to redo anything. And what I've done here is I've de-identified some of this patient data, uh, to remove any sort of names, date of birth, any information that could be associated with the person.

   14. CVClaire Vo16:03

       And one of the

6. 16:04 – 18:44

   Detecting and redacting PHI in unstructured clinical notes

   1. CVClaire Vo16:04

      challenges I'd love to call out for folks that maybe aren't seeing this is, you generated some fake data so you could validate it works. And it's not only this tabular data that has patient ID, ethnicity, whether they were in the control or otherwise, but you also have these free-form clinical notes where there may be embedded PHI written in by a clinician or a, um, study administrator. So it's not as simple as saying, "Just find all the date of birth columns and redact them." You actually have to go through this unstructured data and find potential sources of PHI. Is that part of the challenge?

   2. PKPrerna Kaul16:40

      That's exactly it. It's a, it's a complex problem. Usually, um, you know, machine learning doesn't always do a perfect job there, but, uh, Claude immediately identified the right model to do this.

   3. CVClaire Vo16:52

      Great. So, okay, so you have this clinical data preview, and then you're gonna show us how you de-identify it, so you're not sharing that with folks that don't have access to that health data.

   4. PKPrerna Kaul17:01

      Exactly. So with this synthetic data, I clicked on this button to detect and redact, uh, PHI information, and it scanned through all of the rows, and it found for me all of the PHI. So here's what it came up with. So see in the clinical notes, uh, it, it found that there were some dates, names, uh, associated. There was some birth date information as well, and the remaining, it was able to keep, um, deide-- uh, sort of in a de-identified way so that I can include it in my reports.

   5. CVClaire Vo17:34

      Great, and then how would this have been done in the past? I think you said there are software engineerings or ML, um, you know, engineers that would have built these models to redact the data. Was there any human in the loop at any point in this as well? Like, how, how was this done before?

   6. PKPrerna Kaul17:49

      For the past couple of years, we've been, um... Most pharma companies have been using, uh, machine learning for this, but these, these projects, they end up because, because it is PHI and there's very high bar, they end up taking months of effort, uh, to get right, and especially when clinical trial data, um, might not look the same for every drug.... as a company like Moderna was trying to scale, we were trying to launch nearly fifty drugs, uh, bring fifty drugs to market, um, um, in clinical trials, and then fifteen to market in the next three to five years. Um, that problem was exacerbated. So thinking about coming up with new machine learning, uh, models to solve different problems across each of these vaccines, it's, it's a, it's a matter of scale and time, really, and, uh, we, we really needed to find a better way to do this.

   7. CVClaire Vo18:43

      Great. Okay, so we've got

7. 18:44 – 21:54

   Generating the Common Technical Document (CTD) for FDA submission

   1. CVClaire Vo18:44

      redacted health data. You have all your great, you know, billions of, [chuckles] of patient data points. What's, what's next in this, in this flow?

   2. PKPrerna Kaul18:53

      So the next thing is to generate this common technical document, and I'll, I'll show you a small preview just for the purpose of this demo. So what it did is it took all of that, and it gave me a summary o- of the different data. It was able to give me a synopsis, which is what the FDA looks for: the study participants, the counts, the age, all of the statistical data points, as well as, um, some of the specific medical terminology that they're looking for. So it kind of summarized everything. It gave me this. And for, for context, for folks, uh, this is typically done by a medical writer, and their, their whole... I mean, I've worked very closely with them. It's so time-consuming because they not only- they, they can do a pretty good job of generating a, like, a first draft summary, but they have to go back and forth with their stakeholders for weeks sometimes on a single module [chuckles] to come up with the right set of, uh, criteria. And getting everyone to align, all of their stakeholders to align, is, is very, very time-consuming. Uh, twenty specialists, as I mentioned, are working on it at any given point in time, and as part of what, uh, some of these, uh, the ChatGPTs of the world and, uh, Anthropic can provide, there are also common projects and common environments where all stakeholders can kind of collaborate and work together on these summaries, which is so much easier and so much better than working, let's say, in, um, in a sort of a shared document-

   3. CVClaire Vo20:23

      Mm-hmm

   4. PKPrerna Kaul20:23

      ... uh, and just providing comments.

   5. CVClaire Vo20:25

      Yep, great. Okay, so it generated from that data, this very stri- I mean, it looks official to me. I'm not, [chuckles] I'm not part of the FDA, but, you know, it's a summary of statistics. It's summary of the methodologies. It's summary of the impact. Um, it's pretty long and detailed. It looks like you can download it both to a TXT form, like a DOC form, as well as this, as you mentioned, proprietary XML format that's required. Um, and I'm presuming the Claude Code just gave you two buttons to, to make that happen.

   6. PKPrerna Kaul20:56

      That's exactly it. And I know it's a little bit, uh, proprietary, but I cannot tell you how excited I was [chuckles] to see this button.

   7. CVClaire Vo21:04

      [upbeat music] This episode is brought to you by Lovable. If you've ever had an idea for an app but didn't know where to start, Lovable is for you. Lovable lets you build working apps and websites by simply chatting with AI. Then you can customize it, add automations, and deploy it to a live domain. It's perfect for marketers spinning up tools, product managers prototyping new ideas, or founders launching their next business. Unlike no-code tools, Lovable isn't about static pages. It builds full apps with real functionality, and it's fast. What used to take weeks, months, or even years, you can now do over the weekend. So if you've been sitting on an idea, now's the time to bring it to life. Get started for free at lovable.dev. That's lovable.dev. Okay, so if we could

8. 21:54 – 24:38

   Tracking and displaying AI operation costs for stakeholder buy-in

   1. CVClaire Vo21:54

      just sum- take a halfway point of the podcast, the two things you were most excited about and I was most excited about, I was excited about the RegEx. You were excited about the XML, [chuckles] so this is a, a very nerdy, um, and very specific, uh, How I AI podcast. Okay, so you've generated the doc, and then I love this last part, so let's talk about what's this last piece of, of the product.

   2. PKPrerna Kaul22:14

      What's super interesting is, uh, I- I've worked in a couple of different organizations where as we are trying to scale different AI use cases, cost comes to be a factor because, um, not just token costs, but also, uh, acquiring licenses for these models and then running it. And as the number of consumers or the users of the actual use case scale, the dollar cost also scales, uh, proportionally. So, uh, one of the ways in which we could potentially continue to launch and continue to build products around this is by, by actually being transparent about the, the cost associated with the operations. And, um, you know, generally, the, uh, one of, one of the stakeholder arguments that I've, uh, previously received pushback is, "This is gonna be high cost. Can we find a simpler way that's not, not AI-based?" And, uh, how I've tried to solve this is actually being transparent about the cost associated with it, how long it would take. Um, and as you can see, this trace and cost analysis is super useful because it says cost per operation. It gives a duration against it, and then it actually breaks down by operation, uh, what was the token cost and the total cost, so extremely useful. You can continue to track and monitor this in production as well.

   3. CVClaire Vo23:32

      Yeah, so then you can say, for example, you know, PHI redaction per patient costs fifteen cents, and you can decide as an organization, is that more or less expensive than alternative models? Uh, that's- I mean, I actually haven't heard anybody give this, uh, this suggestion, so I want everybody to hear it, which is if you're getting internal resistance to cost... In fact, I, I feel this even as a, um, even as a solo founder whose vibe coding things, and I see Claude Code give me, like, "This cost you five dollars and thirteen cents." And I send that to people, they're like, "That's so much money." I'm like, "It's, it's five dollars." [chuckles] And so I love this idea of bringing true transparency, and I'm sure these numbers are much smaller than your production numbers on your, you know, million-person studies. But it does bring a true sense of ROI and investment here that I think can get you through internal hurdles.

   4. PKPrerna Kaul24:25

      ... Yeah, it, it really has helped. And, uh, I, I really love Streamlit for this because it's, it's, uh, tracking everything by per operation, so we can even plug and play very easily.

   5. CVClaire Vo24:36

      This is awesome. And so

9. 24:38 – 26:12

   Real-world impact on vaccine development timelines and costs

   1. CVClaire Vo24:39

      you built this and are getting it in the hands of folks that need to submit this. And what do you think the actual... You know, you were telling me millions of dollars, months and months and months, tons of people. Were you really able to shift the, the cost there? Did you see an impact here?

   2. PKPrerna Kaul24:56

      We did. We did. And, uh, we actually applied, uh, some of this, uh, thinking to our, uh, cancer vaccine, the INT vaccine that's coming out, and, uh, our RSV, COVID-RSV combined, uh, vaccine as well. Both are, uh, past their clinical trials, and I think we're looking at commercialization there. So we found that cost savings was definitely one thing, but, um, the stakeholders themselves, um, were super engaged in the process, and they built a system that scales pretty well as the company scales as well. So that was the most useful thing, is we're saving on repeated work, where if one thinks about, uh... I'll just give some numbers out there. So billions of dollars in cost per trial, uh, three clinical trials per vaccine. Want to bring fifteen vaccines to market, uh, that's, that's probably in the order of a couple of hundred billion. I mean, that's, that's a pretty significant number, and usually most companies don't have that sort of cash. And any kind of cost savings you can generate have a direct impact, uh, to the bottom line. And in addition, if you're saving on time, you're bringing life-saving vaccines in the hands of people who actually need it. So, uh, I was really motivated to kind of, kind of help here.

   3. CVClaire Vo26:11

      Well, this

10. 26:12 – 30:22

    Creating an AI communication coach for product managers

    1. CVClaire Vo26:12

       is such a great use case. I love that you showed going from Claude to Streamlit app, and then from Streamlit app to real impact on something that I think, you know, as a mom, RSV and COVID vaccines, I, I thank you. So the sooner you can bring those to market, the happier I'm gonna be. Um, so this is, this is amazing. I know you have one more workflow for us on the product management side, and you teased this, like, PRD to Jira thing. So I know you're thinking a lot about just product management generally and AI. So you want to show us our-- your next workflow?

    2. PKPrerna Kaul26:42

       Absolutely. So the, the problem statement here is that, uh, with, with product managers, we, in our, our day-to-day roles, end up working with a lot of different stakeholders who have, uh, various priorities as when it comes to the product. And I personally, I, I really want to make sure my stakeholders are heard, that we're working very, very collaboratively. Uh, and while we do that, at the same time, I want to make sure we're making the right decisions, and also kind of bringing everyone along and moving things forward. And, uh, with influence and communication, which is a super important skill as a PM, uh, it's, it's, it's sort of custom, where it's, it's hard to say which communication framework to apply when, and what type of communication would land best with a certain stakeholder, because it's so personal, and you really have to empathize and understand folks. So, uh, very important skill to work on. I've been, uh, starting to lean in, uh, on Claude a little bit, uh, and training it, uh, to kind of give me ideas and use it as sort of a, a brainstorming buddy in the process. And this has actually really helped me save time, uh, in meetings, uh, in ahead of meetings in preparation. So I'm gonna go through a prompt that I created. I want to show you this really, really cool, uh, thing that Claude developed. It's called a prompt generator. So if you go to the console, console.anthropic.com, um, you'd be able to create a net new prompt or generate prompts as well, and I created one for influence and communication, where essentially I was-- I, I didn't create this, by the way. It generated it for me. I gave it some instructions, and then it gave me very, very precise information on, uh, what, what the instructions are, and then it gave me structured values that I can include as part of the prompt, and how it would break down the problem, how it would think and reason through it, and then what outputs it would give me in return as well.

    3. CVClaire Vo28:47

       So a couple things I see here is, one, you use this tool that Claude has called Prompt Generator. It generated the prompt. For those that aren't looking at the screen, the other thing I want-- It's... XML is the name, name of the game today. So it has these kind of like code block sections for the situation, your knowledge base, how you do strategic thinking. Like all good prompts, it starts with the, "You are a blank," and gives it a role. And then if we scroll down a little bit, you can see that it also, uh, lets you put in different knowledge, and then it tells you to provide your response in a very specific format that has a combination of analysis, your communication strategy, play-by-play, and then, like, an anticipated question. So this is a very structured prompt. Have you started to reuse this sort of, like, kind of XML-style prompt or, you know, have you, have you played with different things? What's your, what's your approach, or do you just use what Claude gives you?

    4. PKPrerna Kaul29:46

       Yeah. So I, I usually start with, um, generating a, a prompt, and then you see this button here, "Use Claude to optimize your prompt"? This is extremely useful because, uh, what it does, you can give it feedback, and in real time, it will improve your prompt and then add it back in here. So now let's put this prompt in, in the actual, uh, Claude project. So here we go into Projects, and this is our Influence and Communication coach. I'm gonna update this. You can see here that you can set project instructions. So

11. 30:22 – 31:53

    Training Claude on classic literature and persuasion techniques

    1. PKPrerna Kaul30:22

       I'm gonna update this prompt here into Claude.... and save these instructions. And, uh, I've also given it a project knowledge, so I've actually trained it on free books from Project Gutenberg that consists of all sorts of things from literature like Dale Carnegie, and, uh, tactics on persuasion, and how to, um, win friends and influence people. So all of these cool books that, uh, that are all free and in the public domain.

    2. CVClaire Vo30:50

       So I was not expecting Jane Austen to be in your product manager strategic communication, [chuckles] ah, Claude Anthropic project, but I am excited to see it. So you not only gave it's prompt, but you kind of loaded it up with classic, classic literature, classic writing on influence, persuasion, communication. Very, very interesting.

    3. PKPrerna Kaul31:14

       Yeah. Ah, Jane Austen is a favorite of mine. [chuckles] I'm not sure how many, how many folks would use that for persuasion, but I found it very enlightening. So now I'm gonna walk you through a prompt that I w- have been working through, and, uh, this is just a test case, but we can try something live. How about, how about we try that, Claire?

    4. CVClaire Vo31:33

       Let's do that.

    5. PKPrerna Kaul31:35

       Amazing. So do you have a situation in mind, or I can ask it to generate a couple of situations?

    6. CVClaire Vo31:41

       Oh, let's ask it to generate a couple situations. I'm curious what it will come up with.

    7. PKPrerna Kaul31:46

       So I'm using Claude Sonnet 4, which is one of their reasoning models. And as a reminder, the, the

12. 31:53 – 34:40

    Analyzing a complex stakeholder scenario with multiple competing priorities

    1. PKPrerna Kaul31:53

       goal of this particular project is to essentially take a curveball problem at work and then come up with sort of a structured strategy. So here, what it's done, uh, it's made the assumption that I'm Alex Kim. I'm an AI product manager working at a healthcare AI start-up, and these are all of my stakeholders. Very, very, uh, you know, sort of experts in the domain and have very strong opinions about various things. And, uh, it has... It's broken down a problem statement that I'm trying to solve, where two weeks before a major presentation, um, these are all of the concerns that my stakeholders mentioned, so it's actually created that for me. And it has also described the core challenge here: navigating ethics, compliance, competitive pressure, stakeholder expectations. And it's also kind of personalised it, which is cool.

    2. CVClaire Vo32:48

       So for people that aren't seeing what it generated, which is a very realistic, unfortunately, [chuckles] uh, challenge, which is it generated this scenario, which we're just gonna pretend is a scenario we're facing at work, which is: I'm a PM, and my head of clinical data just identified a bunch of issues, both privacy and, um, accuracy data, two weeks before we have a major presentation with a huge, huge prospect. And then it lists out everyone's different opinions. And I think PMs can really empathise with maybe not this particular situation, but a situation like this. So you've created an AI project that says, "Give me a sticky situation, and I will help you navigate how to navigate the situation, given all the s- all the stakeholders." And so it's listed the stakeholders, it's given you the core challenge. So what do we do? What do we get next?

    3. PKPrerna Kaul33:40

       All right, so I'm gonna ask it to use this and proceed next steps.

    4. CVClaire Vo33:46

       And then this is what it's going to use, your strategy framework that you came up with in the prompt, to sort of break down the problem and give you a approach you might be able to use.

    5. PKPrerna Kaul33:57

       Exactly. So it's now sort of analysing, uh, based on the project information that I'd shared with it, and as we can see, it's picked out the most relevant sections from the knowledge as well. So I had given it around ten, fifteen different books and documents, but it found the top two and then picked out the right sections from those. Um, so all of this, it was able to search that information and, and point to that, and analyse, uh, the whole workflow. So it gave me sort of a situation analysis, which, which I found par- particularly helpful because, um, it's sort of validating that it understands the core challenge and the organisational context and some of the cultural factors as well.

13. 34:40 – 35:40

    Getting personalized communication strategies inspired by tech leaders

    1. PKPrerna Kaul34:40

       And here's my favourite part of the prompt, where it actually goes in, and it looks at, uh, some of the influential tech leaders, um, that, that we have all heard and know of, and it sort of inspires the user to say, "Okay, this is how Satya Nadella or Andy Jassy or Tim Cook or Jensen Huang would actually approach the problem."

    2. CVClaire Vo34:59

       This is amazing. And so you're giving really real-life examples or quotes that someone can say, "Okay, if I were to approach it like this other person..." Which I see, I hear is a very common AI prompting technique. Act like this person, or act like you work at this place, um, can really bring a set of unstated principles to a problem that I think is really effective. Okay, and so it's come up with document examples, tech leader examples, communication principles. We all love How to Make Friends and Influence People, so some Dale Carnegie principles, and then it's giving you some strategies you might be able to take here.

    3. PKPrerna Kaul35:39

       Yeah. So

14. 35:40 – 38:26

    Summarizing strategic approaches

    1. PKPrerna Kaul35:40

       out of all of that vast array of information that I gave it, it picked out the three strategies it would apply. So it, it talks about the ethics of it, which I personally agree with quite a bit. So it's, it's talked about, uh, positioning as privacy first as a competitive advantage, and then it's, it's talked about compliance, and then, uh, now it's gonna give me sort of a collaboration strategy as well. So after analysing the situation, it kind of summarised for me, here are the three things that you should do in terms of strategically approaching this situation. First, make sure you're acknowledging and validating each stakeholder's expertise. Second, collaborate to discover an approach, uh, that demonstrates responsible AI leadership, and then talk about the ethics aspect of it. So it's really kind of honed in there. So it's, it's very helpful as, uh... Of, of course, like, a good PM, a product leader in this space would have this info, but it's, it's helpful to see that as a sort of a thought partner as you're brainstorming over this as well. And then it's actually broken down my pre-work-... for me on a day-to-day basis. So what are the individual conversations I can have with each of my stakeholders, and what question to focus on and ask in that meeting? And then it's given me a, a minute-by-minute structure for- [chuckles] ... the final leadership meeting as well. And, and here is my favorite part, not only the play-by-play in the meeting, it has given me back pocket questions that could be thrown as curve balls that I should probably have an answer prepared for.

    2. CVClaire Vo37:17

       My mind is blown right now, because I can just think of so many PMs that stress out so much when they're asked to take on a big problem that has executive, kind of like leadership eyes on it. And so many just don't know how to approach those meetings. Like, "Who do I talk to? Who do I not need to talk to? What do I show up in this meeting and do? And how do I show up to this meeting really, really well-prepared, even when it's a challenging problem?" And you've [chuckles] created this kind of like project that just says, "Here's my problem. Here are all the people," and it's given you a schedule and an agenda to follow in these conversations, which is super helpful and would help PMs not only save a lot of time and prep for these meetings, but actually show up just more prepared and more confident.

    3. PKPrerna Kaul38:08

       I agree completely. [chuckles] It has been extremely helpful. And, uh, yeah, I, I, I'm actually also even happy to share this publicly [chuckles] if it's helpful to folks.

    4. CVClaire Vo38:16

       People are definitely gonna want it. So post in the comments if you would like this publicly shared, so you, too, can prepare for your meeting where all your stakeholders disagree. Well,

15. 38:26 – 45:12

    Conclusion and final thoughts

    1. CVClaire Vo38:26

       this has been so amazing. I love that you both showed us building something very technical that solved a really complex and expensive problem, but also something really practical that brings a lot of like heart, and experience, and strategy to a human problem that feels like it doesn't have a structured solution to it, but in a lot of times it does have a structured solution to it. So we're gonna hit you up with some lightning round questions, and then I will get you back to whatever you are building next or your really tough meetings [chuckles] that you're very prepared for. So I think the first thing I would say is, so many of our listeners are in software and technology, but what you're showing is that the life sciences, biotech, hard sciences, have so much to benefit from embracing AI. So what would you say to your peers in life sciences, um, who have maybe been doing something a, a very specific way for a long time, about how they can a- adopt AI and how it can transform the impact that it can have on the world?

    2. PKPrerna Kaul39:27

       The thing that I have, um, I've been, I've been doing, uh, quite a bit with my stakeholders, and I have been encouraging folks as well, my peers, to work on is, um, f- figuring out where they spend majority of their time, and which things, which tasks do they actually wanna focus on, and which tasks do they actually wanna delegate. So, uh, what, what are energy, uh, sort of maximizing tasks, and what are actually a drain on their energy and time? And, uh, now you have access to AI, and you, you can take a Claude project and create one for any of the use cases and tasks that you don't really [chuckles] want to spend time on. So I think the, the best thing about this is it's not just for individuals, but you can make GPTs and projects available to all of your stakeholders, and all of them can benefit from it and really enjoy the perks of your great thinking. So I would encourage people, uh, in life sciences as well as, um, any, any folks who are interested in, to play around with these tools and not be afraid of actually trying to, um, optimize their own work, because they might find it's a common thing for all of their peers in, in the process.

    3. CVClaire Vo40:40

       So speaking of being a little bit afraid, I mean, you work in a highly regulated industry with, you know, patient data. You have to have the highest ethics. You really- the burden of accuracy is so important. The burden of privacy is so important. How do you think about safety, privacy, alignment? Um, it's not a problem that I think a lot of the kinda like vibe coders or PMs in software companies really have to grapple with in their day-to-day, but you do, and I'm just curious how you approach it. What's your framework? Um, where does it stack-rank in your priorities?

    4. PKPrerna Kaul41:16

       Absolutely. I think with, um, most regulated industries, I've worked in finance before and in healthcare. Um, I also currently work with family wellness, where there are teenagers and kids involved. Compliance and privacy comes, uh, first, uh, but it's n- it's a bit more than privacy and compliance. It's the, the ethics of it all that, uh, one must prioritize. And, um, as I- as that relates to LLMs, I think, um, looking at the benchmark data of which LLMs are performing higher on ethics and what edge cases have been observed in terms of safety is, uh, sort of the priority, priority zero of a product leader in the process. Because, um, of course, technical, technical leaders, engineers can make decisions on which models to use when, but I think the, the product leaders, um, can provide that lens of: Hey, this model has shown, for these kind of populations, has shown, uh, not-so-promising results in terms of ethics and alignment with human values. How might we shift the process, and how might we avoid, uh, using some solutions that might not benefit the end consumer? The other aspect of this is also, um, in particular, with safety and alignment, it's important to identify how you would evaluate your solutions, both offline as you're developing it, but also in production, so the online performance monitoring of it. And that's also an area where product leaders are-... are particularly helpful because they can provide both and share, like, "Here, here are my, like, hundred things I wanna know that the model does well on when it comes to safety. If it's not doing well, let's do a rollback, let's find, you know, a quick solution." And, uh, oftentimes, as, uh, as you know, in tech, things move extremely fast. We're also rolling very quickly, and we're, we're operating one click faster than, than usually we do [chuckles] So I, I think just, like, taking that strategic view is e- e- extremely important, and, uh, those are some practical ways that I would suggest, uh, perhaps folks who are using LLMs could, could apply.

    5. CVClaire Vo43:23

       Well, you're setting the bar, and I'm really glad that you are one of the, the product leaders out there that are building these kinds of products, because it's clear you take it seriously. Okay, last question, have to ask everybody: When AI is not listening, when it gives you terrible strategic communication advice, what's your tactic? How do you get the LLM to listen?

    6. PKPrerna Kaul43:46

       Uh, many emojis. [chuckles] Would-

    7. CVClaire Vo43:49

       [chuckles]

    8. PKPrerna Kaul43:49

       ... it, it does really well with emojis. If you, um... And it, it also seems to adjust when you, when, uh, when you put an emotion. I, I literally tell Claude, "Claude, I'm sad. I have a bug," [chuckles] and it immediately fixes everything.

    9. CVClaire Vo44:04

       I do love the reasoning models that expose their reasoning, because I just love reading, like, in the sidebar of Cursor, like, "User is extremely frustrated and disappointed in my response. I understand why they're so sad." [chuckles] So okay, emojis, emotions, and XML, that's gonna be the tagline of, of this episode. Prerna, it was so great to talk to you. Where can we find you? How can we be helpful?

    10. PKPrerna Kaul44:29

        You can find me on LinkedIn, and, uh, I'm happy to connect with folks. I'm also happy to share some of the tools I've developed and how you can help me. Just stay in touch and, uh, continue to engage and talk about all the cool things you're building.

    11. CVClaire Vo44:42

        Amazing. Well, thank you so much.

    12. PKPrerna Kaul44:43

        Thank you. [upbeat music]

    13. CVClaire Vo44:45

        Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time! [upbeat music]

Episode duration: 45:13