Gemini Gems Masterclass with the Creator at Google: 3 Gems You Must Build

1. 0:00 – 1:49

   Intro

   1. AGAakash Gupta0:00

      You created Gemini Gems, Lisa. What are they? How would you describe-

   2. LHLisa Huang0:04

      So Gemini Gems are custom versions of Gemini that you can create for your specific use case. If you aren't using Gemini Gems, you aren't getting the most out of Gemini.

   3. AGAakash Gupta0:12

      What are the most important gems for product managers to create?

   4. LHLisa Huang0:15

      Well, Aakash, I think there's three must-have gems for every product manager. Let's call them the writing clone, the product strategy advisor, and the user research synthesizer.

   5. AGAakash Gupta0:24

      This is Lisa Huang, the creator of Gemini Gems. She's been an AI PM at Apple, Meta, and Google. Now she's an SVP at the eighteen billion dollar giant, Xero. She also created Gemini Gems. What is the process of creating one? Can you show us?

   6. LHLisa Huang0:38

      Sure. It's actually really simple. So...

   7. AGAakash Gupta0:41

      Wow, these are insanely powerful. So if we were to build a mind map together, what would be the most important projects or gems every PM should be creating?

   8. LHLisa Huang0:49

      Yeah, so when you think about what are the key skill sets of PMs, I might break it down to four big categories. You certainly need to own the overall product strategy, and you'll need to own execution for that with your teams, but also very importantly, you need to communicate that to a lot of different stakeholders. And then informing that product strategy, you've probably done a lot of research on competitors, users, and that those are key data signals to input as well.

   9. AGAakash Gupta1:10

      What are you looking for in an AI PM?

   10. LHLisa Huang1:12

       So I think-

   11. AGAakash Gupta1:18

       Before we go any further, do me a favor and check that you are subscribed on YouTube and following on Apple and Spotify podcasts. And if you want to get access to amazing AI tools, check out my bundle, where if you become an annual subscriber to my newsletter, you get a full year free of the paid plans of Mobbin, Arise, Relayapp, Dovetail, Linear, Magic Patterns, DeepSky, Reforge Build, Descript, and Speechify. So be sure to check that out at bundle.aakashg.com, and now into today's episode.

2. 1:49 – 2:05

   Guest introduction

   1. AGAakash Gupta1:49

      Lisa, welcome to the podcast.

   2. LHLisa Huang1:51

      Thank you. Thanks for having me.

   3. AGAakash Gupta1:53

      As you all know, Google Gemini, since the launch of Gemini 3 Pro, has become one of the top AI models, as good as ChatGPT and Claude. But if you want to get the most out of Gemini, you have to be using Gemini

3. 2:05 – 3:52

   What are Gemini Gems?

   1. AGAakash Gupta2:05

      Gems. You created Gemini Gems, Lisa. What are they? How would you describe Gemini Gems?

   2. LHLisa Huang2:11

      So Gemini Gems are custom versions of Gemini that you can create for your specific use case. Everybody knows that LLMs today are really powerful, but the key problem that they all have is that they lack context. So what ends up happening is that as people use these LLMs repeatedly over and over, they're just entering in the information that's needed to make sure that the LLM gives out the right answer, whether it's your role, your company strategy, your writing style, the history of your product. You're constantly having to type this in again and again, and that's a little bit of a pain in the ass. So you can think of Gemini Gems as the difference between a general contractor versus a master craftsman. Standard Gemini or standard LLMs, they're kind of like your general contractor. They're super powerful, really capable, but you're gonna have to give them detailed instructions every time you hire them, and you have to give them your project's blueprints. Uh, compare this to the Gemini Gem. It's kind of like a specialist or master craftsman. It already knows everything about your house. It understands your preferences, understands where all those little pieces of your furniture are. So when you ask it for something, they can craft the exact thing for you without having to be retold all of the basics. So Gemini Gems, it's a, a little tool that you can create inside Gemini. It holds all of your context. It h- keeps your personalized tone and style, and it's really tailored to your specific use case.

   3. AGAakash Gupta3:34

      I think that's the important thing, is you can create so many different ones. You can create... People have been talking about a personal CTO. You can create a coding gem. You could create a writing style gem. What are the most important gems for product managers to create?

4. 3:52 – 6:05

   The 3 must-have Gems for every PM

   1. LHLisa Huang3:52

      Well, Aakash, I think there's three must-have gems for every product manager. Let's call them the writing clone, the product strategy advisor, and the user research synthesizer. So on the writing clone, one of the key jobs of PMs is to communicate all day, every day to lots and lots of people with different contexts. So what you can do here is create a gem that sounds like you to help you accelerate those communication tasks. What you could do is upload your, uh, key files, like your PRDs, maybe some of your emails, some of your team Slacks, and then you can have a version of Gemini that sounds like you to create those nice first drafts. The second gem, um, is something that can help partner with you to be a strategic partner in, uh, your thinking process. So think of this as your product strategy advisor. You can include things, your, your company's strategy documents, market position, go to market, and, uh, give it some competitor analysis. Then it can be a little thought partner for you as you're thinking through different decisions and helps you think through maybe some angles that you hadn't thought about before. And then lastly, on the user research synthes- synthesizer, obviously as a PM, you have to stay really close to the source. So what are those user needs? What are users saying? But there's quite a lot that's going out there, and you can't always be in all those conversations, all the interviews. So instead, what you could do is upload all of those user interview transcripts, the raw survey data, maybe some customer support tickets, and just ask the Gemini Gem to synthesize all of that for you. You can ask Q&A, ask it for key insights, uh, to help you sort through all of that data.

   2. AGAakash Gupta5:26

      Okay, these are epic. I understand why people should have these. What is the process of creating one? Can you show us?

   3. LHLisa Huang5:33

      Sure. It's actually really simple. So when you go to Gemini, uh, there's a little tab you can click on, which is Gems. Then you click on Create a new Gem. And then three key things you want to do: number one, give it some clear instructions, number two, add knowledge, so upload your key context documents, et cetera, and then thirdly, make sure to test and iterate. It may not be perfect the first time, but as you use it more and you kind of iterate on those instructions and that knowledge, uh, it'll get closer to what you're looking for.

   4. AGAakash Gupta6:01

      Sweet. And do you want to just show us?

   5. LHLisa Huang6:04

      Sure.Let

5. 6:05 – 9:01

   Live demo: building a product strategy Gem

   1. LHLisa Huang6:05

      me switch screens here, and here we go. So here I'm in Gemini, and, uh, what you can go to here on the left, you'll see there's a little link called Gems. And, uh, there are some pre-built gems for you if you wanna try out, if it's kind of your first time and you g- wanna get a feel for what these things do. You can click on New gem, and let's say we wanna go ahead and build that, you know, product strategy buddy, someone to help you think through your product strategies. So, um, someone to bounce product... Oops, bounce product ideas off of. All right. And ins- in instructions, you do want to create very detailed instructions. I'm sure you guys have seen this with how LLMs perform. The more detailed your instructions are, the better output you'll get. So I've prepared some here actually for this. Um, so you'll see, you know, a lot of detailed context, core responsibilities, how do I want them to approach their thinking. And then as you scroll down, um, for those of you who want, depending on your gem, you can select a default tool if there's something that it'll often use. In this case, I'm not gonna select any, um, because it's more of a general kind of brainstorm. And then for knowledge, I'm gonna add some key company files. So here, um, I've selected, you know, four key files I think are most important, some competitive teardowns, our, uh, you know, strategy, and then a recent roadmap. And then what you could do is actually just take it out for a test here on the right. So let's say I say, um, "Give me some product, uh, let's say maybe new, new product ideas for this year." And it'll go off, use all that context, and here-

   2. AGAakash Gupta7:54

      And I like how because you're a public company, you can just put the earnings release in to give it the [laughs]

   3. LHLisa Huang7:59

      Well, actually, this is, this is not real data. [laughs] It's, it's... Yeah. But yeah, so here you go. You can see, um, uh, does a nice job here. You can continue to tweak it. If you don't quite like the output, just change the instructions, change the knowledge here, and just kind of iterate on it until you're happy. And once you're good, you can go ahead and save it. And what you'll see here is, um, you can start a chat with it any time, and it's listed here under your Gems category that you can refer to any time you want. So there you go. Please do check it out.

   4. AGAakash Gupta8:28

      Is this something you can share with your team? So, like, as a product leader, could I create a gem and then say, "Hey, team, here's a gem resource you could use"? Or maybe like-

   5. LHLisa Huang8:36

      Oh, totally. Yeah. So one of the key use cases we do see is, um, for personal productivity, and that extends beyond you yourself to your broader team. So, um, you know, you can, especially for a given function or inside a g- given company, you all have very shared context, and so create them, share them around. Um, you can be that power user in your own company to build it for your teams, and I'm sure your, your colleagues would be really happy to use your gems.

6. 9:01 – 10:32

   The story behind building Gemini Gems

   1. AGAakash Gupta9:01

      So what's the story behind this feature? Because a lot of people, they're being asked to build AI features even if they haven't before. It could be really useful for them to get the AI PM sort of zero to one process of building a big feature on a big AI product like this that is so successful. How did you guys do that?

   2. LHLisa Huang9:22

      Yeah. You know, what happened is, uh, I s- I worked on this back in 2023, which at this point feels like, you know, the age, [laughs] like a long time ago. But, um, at that time, even then we realized LLMs are so powerful. There's so many things they can do. But the average user was having a hard time discovering all of the capabilities that were possible. So we brainstormed as a team, and we're thinking about, "Are there... How do we unlock people to discover more of what can, it can do? Perhaps could we give Gemini different personas, different instructions, um, and maybe let users customize that for themselves so that they can save it in a way that better fits their mental models?" So instead of just, you know, big Gemini for everything you want to ask it, maybe I have my, you know, doctor gem when I want to ask for medical advice, and then maybe I have my, um, you know, writing tutor when it's like I need to write something really nice. So anyway, we want, we want to offer up these tools to help people better discover, um, the different capabilities of Gemini and also create a way for people to share, just like the example you mentioned of maybe sharing with colleagues or others in a similar industry, as a way for people to discover more of the things that Gemini can do.

7. 10:32 – 11:39

   Ads

   1. AGAakash Gupta10:32

      For my money, the most important new skill for every product manager, whether they're an AI product manager or not, is AI prototyping, and Lisa even touches on it in this episode. The PRD used to be a 10-page document. Now we're sending an AI prototype along with a one-page spec aligned with it. The problem with most AI prototyping tools is they're not purpose-built for product managers. They're built for founders or aspiring vibe coders. Realistically, you need a tool purpose-built for product managers, and that is today's sponsor, Reforge Build. Reforge Build is purpose-built by the team that knows product managers better than anyone, Reforge. Brian Balfour himself is going in there and making sure that it takes your design system, it helps you create divergent ideas, and it helps you get the most out of your AI prototyping tool. So use code BUILD to get a discount. Use Aakash's bundle to get a full year free, and click on the link in description below and go check it out. That helps support the podcast, and supporting Reforge Build helps support the Reforge ecosystem.

8. 11:39 – 16:45

   Gemini Gems vs ChatGPT custom GPTs

   1. AGAakash Gupta11:39

      With that, back to today's episode with Lisa. Did this come before or after custom GPTs on ChatGPT?

   2. LHLisa Huang11:48

      Yeah. You know, it's such an interesting story because we started working on this, um, so it kind of came up organically in the team, and then in the middle of it, ChatGPT, you know, OpenAI, they launch custom GPTs. Um, and so suddenly we were, "Okay, I think we have to go faster with our launches, go faster with how we're developing this." Um, yeah, and we're off to the races.

   3. AGAakash Gupta12:10

      How, if at all, did you try to make it different from what they had launched?

   4. LHLisa Huang12:14

      So when OpenAI launched GPTs, the framing of that product launch was really around this can become kind of like the new 3P app ecosystem. So, you know, if you remember, they launched custom GPTs, but then also the GPT store. They talked about there's way-- we will offer tools to monetize over time, so very much the same framework paradigm as an app store. When we looked at the capabilities of and how we thought people might use this, we took a bit of a different approach. Um, I think our perspective at the time was there's not that much proprietary about each of these gems. The instructions are easily copied. Uh, yes, you can give it custom data and knowledge, but at the time, it wasn't clear that you couldn't just easily prompt a gem to kind of spit out what some of that custom knowledge was. So it wasn't clear that this would really foster that burgeoning, um, you know, ecosystem, uh, or, or app ecosystem. So instead, we really focused on how can we, how can we support Gemini Gems as a personal productivity tool, so something you customize for yourself to really amplify your productivity, things you create for your own teams, your colleagues, uh, that can really amplify all of your collective productivity.

   5. AGAakash Gupta13:26

      Mm. And I think that's an important takeaway for folks. Like, a me-too feature doesn't always do as well versus, like, coming up with the first principles of what are we trying to accomplish at our company and then working backwards from that. And it seems like it turned out prescient because not too many people are mos-monetizing custom GPTs. They didn't really pursue that angle. So sometimes you can also really see ahead if you think from first principles.

   6. LHLisa Huang13:53

      Yes, absolutely.

   7. AGAakash Gupta13:53

      So whether people are building a gem or they're building a ChatGPT project or they're using my favorite AI these days, Claude, and building a Claude project, so if we were to build a mind map together, what would be the most important projects or gems every PM should be creating?

   8. LHLisa Huang14:09

      Yeah, so when you think about what are the key skillsets of PMs, uh, I might break it down to four big categories. So you certainly need to own the overall product strategy, um, and you'll need to own execution for that with your teams. But also very importantly, you need to communicate that to a lot of different stakeholders, so that's gonna be key. And then informing that product strategy, you've probably done a lot of research on competitors, on, uh, users, uh, and that those are key data signals to input as well.

   9. AGAakash Gupta14:35

      Okay. Makes sense. So really, if you're using this feature to its best, you probably have something like these 20 Gemini Gems or projects created.

   10. LHLisa Huang14:45

       Yep. Yep, absolutely.

   11. AGAakash Gupta14:47

       What are the biggest mistakes people are making with these?

   12. LHLisa Huang14:50

       Yeah, so I'll flag a few. As I mentioned, you gotta be really clear with your instructions, so not a vague thing like, "Help me write better," but the more detail you can give it, the more examples, the better. Um, so do spend time on those instructions. Related to this, you also wanna make sure you have the key context uploaded. So, um, what is it that makes it personalized to you and your circumstance and not just general instructions? Uh, for example, key company documents, example emails, whatever it is that you're looking for it to help mimic and, and help follow, make sure those are uploaded. I do recommend specialized gems for different jobs, so just like we talked about, uh, in the last exercise. If you, if you break down the different tasks you have, what is the specific gem you want for each of those? And then as with all good products, I'm sure the first version is never gonna be perfect, so make sure that you are iterating, you're testing it, and as you use it, continue to add those tweaks along the way to make it even better over time.

   13. AGAakash Gupta15:45

       Hmm. And does a gem have, like, persistent memory in it, or do you need to make those changes to the system prompt and context files?

   14. LHLisa Huang15:53

       It will, it will read off of what the instructions are there, um, and the context files, so if things change on that over time, then you should update that as well.

   15. AGAakash Gupta16:01

       Okay. So I think that is one distinction maybe with Gemini Gems and Cloud Projects for those of you who have been following my content on Cloud Projects, where they ha- you can kinda train the project. So I think you may want to, maybe even if you're training it, you can probably even ask the gem, "Hey, how would you us- update your system prompt to accommodate the types of edits we made today?" And then it can probably give you a system prompt back.

   16. LHLisa Huang16:24

       Yes. Um, yep.

   17. AGAakash Gupta16:25

       Iteration is key. I've been using these things like crazy, and I have some that I just, like, religiously live with now. So treat it like an AI mini product that you're creating for yourself, and iterate, I think, is the key. So that's our first segment we wanted to cover for you guys. That's Gemini Gems or Projects Masterclass for

9. 16:45 – 23:05

   Career lessons from Apple, Meta and Google

   1. AGAakash Gupta16:45

      you. Now I wanna talk a little bit about the AI PM career, 'cause we have somebody who has crazy amount of experience, right? In fact, Lisa's kind of been collecting these, like, Infinity Stones. She's got McKinsey, Private Equity, Apple, Meta, Google, now Xero. So my question for you is what are the key lessons... You have this vantage point that so many people want. Like, you've seen it all. What are the key lessons you've learned from navigating through all these amazing companies in your career?

   2. LHLisa Huang17:14

      Yeah, you know, it's interesting because every time I get, um, that question, I-- it makes me smile a little bit because I never planned any of this. Um, I'm not someone that knew what I wanted to be when I grew up. Um, I might say my-- I still don't really know what I wanna be when I grow up. And for me, the principle I've always used in my career is really curiosity. Um, what's the thing I'm interested in learning next? How do I wanna grow next? And that's led me to every opportunity. I mean, looking back on, you know, a nice slide like this, it looks like, oh, that was a nice plan. You kind of followed ABC, but in reality, every step of the way, you're kind of figuring out the next step. Um, yeah, and I feel really lucky to have been able to work with some really amazing people on some really mading-- amazing projects, uh, over the time.

   3. AGAakash Gupta17:59

      All right. I think this really resonates with what you've talked about, where you've said, "Make your PM career your own." What do you mean by that? What is the practical takeaways for a more junior PM?

   4. LHLisa Huang18:10

      Yeah, you know, I think what's really interesting about PM is that it's so different.Company to company, role to role, product to product, the responsibilities are quite varied. Um, and so I like to think of there's different archetypes of PM, some that are really good at zero to one, some that have that, like, craftsmanship and, you know, really love consumer, some that really think about platform. So I think the important thing is to figure out what are your strengths, what are the things you enjoy about product, where do you want to play in that product space, and then set out to find those roles, those opportunities, um, the people that will support you to really hone that part of your craft. And then also figure out what are the tool, additional tools you want to add on top of that to round out the skill set.

   5. AGAakash Gupta18:55

      Mm.

   6. LHLisa Huang18:55

      And that's how you can make product, you know, your own.

   7. AGAakash Gupta18:58

      So Apple, Meta, Google, I feel like we-- they seem to have completely different product cultures. From the outside looking in, Apple seems to be a very design-focused culture, where they make design and executive decisions, and then those decisions kind of filter down into how you might build the product. Meta seems to be very growth testing, lots of experimentation type of culture. Google seems to be, you know, putting the user first, being very first principle. That's at least what it seems like from the outside. What can you confirm or deny about-

   8. LHLisa Huang19:31

      [laughs]

   9. AGAakash Gupta19:31

      ... what the product culture is like at Apple versus Meta versus Google?

   10. LHLisa Huang19:35

       Yeah, they're quite different. Um, but I think one thing I really appreciate is what I've taken at, from each place, I take what I like the best, um, of each and kind of take it with me into my future, uh, into my future roles. So Apple, the thing that I really like is product is just supreme there. And what I mean by that is everything bends to the will of having that amazing, amazing product. So we don't care which teams we have to drive crazy for it. We don't care that it's, like, just a sub-millimeter off. We will fix it. We will fix it until it's perfect. Um, and so that, like, love of craft, that, like, detail, that every little detail really matters, um, and holding an extremely high bar on product, that's something I do take with me, um, I've taken with me to different roles. Uh, and then on Meta, yes, it is very experimentation-driven. Um, the thing I love about Meta is very, very data-driven, one of the best data cultures I've been in anywhere. Like, I've been way too, uh, just, uh, spoiled with the kind of data access I had there, um, and the available tools that were there, but also really, really impact-driven, so lots of focus, not just on, like, the perfect strategy, like we need a good strategy, but also, like, we gotta execute fast. We have to compete. So, um, that part I, I take as well, the execution muscle and that, you know, really focusing on metrics and data. And then at Google, Google is the most technical of all the companies I've worked in, and, uh, that applies to PM as well. PMs are expected to be very, very technical. They're expected to partner very, very deeply with their engineering counterparts. And so I grew a ton technically, uh, in my time at Google, and that's something, you know, especially now as you look at AI and the future of where things are going, I think that's just gonna be an expectation of every PM. Like, you really have to get much more technically deep.

   11. AGAakash Gupta21:16

       Mm. So now you're in the unique position as an SVP, I'm sure, of hiring lots of AI PMs. When you are looking at the backgrounds of people, what are the things that stand out or don't stand out? What are you looking for in an AI PM?

   12. LHLisa Huang21:32

       Yeah. So I think there are, maybe I'll call it some core things and then some, um, additions. So on the core, it's your, you know, standard PM 101 skill sets. Do they understand product strategy? Do they have a good vision on things? Um, can they understand metrics? Can they work well with different teams to execute upon that strategy? And then the cross-functional skills. Can they work well with others, influence without authority? These are what I'd call standard. And the things I always look for which are a little bit extra are it is that grit, that growth mindset, the willingness to push beyond, so not just what's expected, but, like, do you care deeply enough about this to solve it, and, like, you'll let nothing stand in your way. Um, some people do have it. You can, you can tell pretty easily as you talk to them about their past projects. Um, and I find that those are folks who, they learn really fast. You're able to put them in any environment. Things are constantly changing really rapidly, and so they're really good folks to have around because, um, they're just able to solve problems and push through things.

   13. AGAakash Gupta22:32

       Mm. Makes sense. So we covered a lot on product development because I just needed to hear about that from you. Now I wanna go deep on one of these experiences we have here, which is Meta. And actually, I wanna put something on the screen for everybody 'cause if you guys don't n-know, like, these Meta glasses, they're really taking over as far as the most successful AR product out there. And you worked on one of the coolest AI features for AR out there. You worked on the first generation Meta AI assistant for Ray-Ban Stories smart glass.

10. 23:05 – 27:43

    Building the AI assistant for Meta RayBan glasses

    1. AGAakash Gupta23:05

       So what was the story behind this feature?

    2. LHLisa Huang23:07

       Yeah, it's, um, as with all things, uh, [laughs] yeah, it's a, it's a really awesome product today. Um, I will say there was definitely a long road in getting here. Um, so I worked on this product back in 2019 through 2021, so a long, long time ago, uh, for the first version, and at the time, I was a part of the Meta assistant, uh, team. Um, and back then, you know, I had a really strong vision that the AI assistant would become the singularly most important feature on these glasses, but it's definitely not true that everybody agreed with me. So there were folks who thought, "Hey, the AI thing is just gonna be a voice input. That's nice, but we also have other input mechanisms," you know. Um, and yeah, I was very convinced that we needed to invest a lot more here and that this would, even though the technology's not fully there at that time, that this would become that, you know, future intelligence and interface layer for this hardware. Anyway, um, long story short, we did manage to get it funded. Um, we worked very closely with the Ray-Ban Stories team. Um, as with all zero to one products, lots of, uh, learnings there. This was the first time that we were working so closely with a major eyewear company like EssilorLuxottica, so lots of executive l- um, involvement on our side, on their side, a lot of joint collaboration, lots and lots of unknown. Um, so from is this gonna have product market fits? What are the real use cases? We put a camera on people's faces. What about privacy? How might bystanders feel about that? What's the user interaction on this thing? And then the engineering complexity was extreme, right? You're talking about lots and lots of technology in a very, very small space, and you still have to make it fashionable, and you have to make it comfortable to wear for a long, many hours. So, uh, yeah, a lot of constraints, a lot of challenges, but, like, tons of fun. It was, uh, just learned a lot through that entire process and, uh, you know, made some, made some good friends along the way as well.

    3. AGAakash Gupta24:59

       So I've talked to a lot of AI PMs. They might be building without those constraints. Like, you are thinking about building an AI into a wearable. Like you said, it can't be too heavy. Like, we already saw the Vision Pro flopped. This hit four million sales in 2025 because it was light enough. How does that impact the type of AI features you can build? What are the ways to get around that? Are you gonna host the AI in the cloud and do the processing there, or how do you build AI for AR?

    4. LHLisa Huang25:27

       Yeah. Um, yes to a lot of today's models are through cloud, but, um, a lot of models are getting much, much better, and I personally believe there's gonna be a wave towards on-device for a lot of reasons, right? Privacy-People don't really want to share their data out there. Once you do make it small enough that it can be on-device, there's a lot more places you can put it. Um, reduces kind of battery and other kind of technical constraints as well. So personally, I think that the future of where AI is gonna go for AR is that it's gonna... a lot more of it, maybe the vast majority of what you want to use it for, will be on-device. And yeah, it'll-- I think it'll make everyone feel more comfortable using it as well, that what's your private device and what it's capturing day to day as you're putting... you know, you have this thing on your face, and you're going through all your different interactions, that that stays local, uh, and secure.

    5. AGAakash Gupta26:14

       So we see kind of at least three big players investing heavily in this space, OpenAI, Apple, and Meta. What would be your advice to PMs if you are one of those PMs working there in those roles? How do you truly build successful features in this space?

    6. LHLisa Huang26:30

       I think it's probably a similar advice for really any, um, product or any AI feature. Uh, don't fall too much in love with the technology, but also deeply, deeply understand it. I, you know, having worked in AI products for about a decade now, I think the best products are created when you mirror, you deeply understand kind of the user need, the product value, but then also deeply understand the technology and find that, like, perfect little intersection in the middle that does both. It leverages the technology in the right ways that are actually really useful for people. So, um, yeah, that's-- I would say that's step one, and then step two is just test and ship and learn, right? Everything is developing so quickly. I don't think it's worth it personally to overthink too much because whatever your assumptions are today, in a month, in two months, it's gonna change. [laughs] So just, just build, go fast, see what people are doing. User behavior is also changing really quickly. User expectations and what they're willing to do is changing rapidly. So, um, yeah, uh, have your hypotheses, stay close to the product and user needs, stay close to technology, and then just build and iterate.

    7. AGAakash Gupta27:33

       We've been following the story of your career. We dove deep into Gemini Gems, AI assistant for Meta RayBan. I want to talk about the latest AI features you have been building.

11. 27:43 – 32:22

    Introducing JAX - Xero's financial super agent

    1. AGAakash Gupta27:43

       This is JAX at Xero. Can you walk us through what this feature is? Some people may not even know what Xero does, so maybe you can give us a teeny bit of background on that so that we can understand how this is a significant feature.

    2. LHLisa Huang27:53

       Sure. So Xero is a finance platform for small businesses. We do accounting, payments, payroll, and other financial services for small businesses globally. Uh, we serve about four million businesses. We are, uh, about eighteen-billion-dollar market cap company. And, uh, yeah, we are, in some sense, for small businesses, finance, this f-financial system is their lifeblood. Um, you know, how they understand what's happening with their business, how they get cash in, how they make payments out. So it is a really, really core part of how they operate, uh, and that's why we are quite deeply embedded, uh, with a lot of these small businesses. So the key product I've been working on, uh, I joined Xero to lead AI as a whole, and, um, the key products I've been building here, it's really around this umbrella of JAX as a financial super agent. And what it means is we map out all of the financial workflows, um, that small businesses use, all of their key jobs to be done, and we look in detail at which parts of those can we automate with agents, with kind of other AI. What can we do to offload a lot of that manual work for small businesses? But on top of that, what can we do then to also understand the deep insights embedded in those businesses? So for example, uh, like, on a platform like Xero, we have transaction-level data of each of these businesses, so every invoice, every bill, every payroll run that happened. We know a lot of detail about what's happening in that business. And this gives us an opportunity to really leverage that data, understand those deep insights, and help our small businesses grow their businesses better, help them survive, thrive, uh, and really achieve what they, what they want to do with their business.

    3. AGAakash Gupta29:29

       So everybody's been trying to build agents into their products. What have been some of the harder lessons learned that you've learned from experience in building an agent? I know sometimes people talk about unreliability of MCP for tool calling. People talk about, you know, customers having higher expectations than what their early versions of the agent could do in terms of tool coverage. People talk about the unreliability and the non-deterministic nature of some of these agents plugging in and then configuring a product wrong, and in accounting, I imagine that's doubly an issue. How have you guys built around some of those key problems, and what have been some of the hard lessons about building an agent into your product?

    4. LHLisa Huang30:07

       Yeah. What's really interesting about working in this space is when you're talking about finance, the decimal, you know, the accuracy down to the decimal really matters. So accuracy is not a nice-to-have, it's a core part of our differentiation. And what we know is that LLMs, out of the box, are not that great at math, accounting, tax, et cetera. Um, so the thing that we bring, it's two things, really, to make sure that our AI is really accurate and really reliable. Number one, the domain knowledge that we apply. So we are in a position to deeply understand what are these workflows that small businesses actually use, what matters to them, what doesn't, every step along the way, what accuracy level is required and what is acceptable, um, at different task and subtask levels. We understand which stakeholders need to be, you know, looking at that data and using it in which ways. And so we do fine-craft our JAX experience to make sure that it is, uh, well suited and, um, something that will plug into users' existing workflows and be really useful for them in what they're trying to achieve. The second thing is what I alluded to before, we have a lot of data on our system, so this definitely gives us a leg up in terms of, um, one, we can personalize to every business much better, and number two, we're able to use that data collectively to understand what are some of the key trends, benchmarks, et cetera, happening across small businesses by, uh, you know, sub-region, sub-industry, et cetera, and really offer much more, um, helpful AI built on top of that. And you mentioned MCP and tool use, et cetera. We do use a hybrid system, so we leverage LLMs, uh, in the...You know, kind of multi-agent workflows in different ways, but then also importantly, we also use, you know, programmatic code and, um, really hybrid structures where we need more control over things to ensure that we have that high quality. I'm pretty blessed to work with some, uh, you know, amazing engineers and AI scientists on the team. We have a bunch of ex-Google, ex-Meta, ex-Amazon folks from, from these places, and, uh, we've taken a lot of care in building out really s- robust quality measurement eval and kind of flywheel systems, as well as a team of expert annotators in, from the finance space that can really make sure that we are, our AI is behaving the way it's supposed to when we're talking about financial data.

    5. AGAakash Gupta32:21

       Mm.

12. 32:22 – 37:39

    How to measure an AI agent

    1. AGAakash Gupta32:22

       So how do you measure success for this feature? And maybe you can talk both about kind of the evals, 'cause that's a really hot topic for AI PMs right now, but also just the basic success metrics, usage, engagement, whatever sort of how did you get over to ARR impact on retention. Would love to understand what is the right framework to measure an, a new agent being launched in your product?

    2. LHLisa Huang32:43

       Yeah. So I, I would break it down to three phases or three, three areas that need to build upon each other. Number one is just baseline quality. Is the AI doing what it's supposed to be doing? Number two is, like, regular product metrics around user engagement, so do people... Are people adopting the product? Are they liking it? And then your third metric, which hopefully all this ladders up to, is the business impact. Are we making money? Are we driving the right revenue? Are we hitting those business metrics? On the first one on quality, we have a pretty robust framework internally. It's something that we've developed but also will continue to evolve over time, um, because for different use cases, you are gonna need different evaluation criteria. They're not always the most straightforward, um, and you need to get clever with how you measure it. We definitely do leverage annotators to help us measure, like, so human evaluators along the way, but of course, that doesn't scale, and so we also build, you know, LLM judges and kind of other eval metrics to help us understand what is the real quality of the product. And that's something we track very regularly across all of our use cases, um, and we, you know, we understand the current quality, what are some of the gaps, uh, and what are the initiatives the, um, investments we're gonna make in different areas in order to improve the quality over time. So that's one. Uh, on the user engagement metrics, very similar to, you know, regular products, so, uh, just usage, MAU. In our... Well, depending on the use case, it could be monthly active, it could be weekly, some of them are daily. Um, yeah, just adoption, usage, retention. And then the qualitative side, so CSAT. Uh, we do talk to our customers very regularly, so we get a lot of anecdotal feedback as well, so that's something we do monitor, uh, you know, on social media channels or as we're having customer conversations. So that, uh, we track as well. And then on monetization, this is all about looking at w- there, every, different companies may have different ways that they attribute revenue. Um, for us, you know, we have a system internally to attribute, um, the, the revenue we're making from AI, and that's something we track as well.

    3. AGAakash Gupta34:38

       So you are at the forefront of building agents into your products. Sequoia had this really interesting thesis that they presented on TVPN, which I wrote about and ended up going pretty viral on Twitter, where they said that you need to start thinking about agent-led growth. I think your product is a perfect example, where it seems like the JAX agent is doing a lot of the work now in the Xero platform for you. And so the i- the thesis of agent-led growth is that agents are gonna be talking to agents. Agents are gonna be picking what tools, what software in the future. And so you need to think about how you build your project, product to be attractive to agents so that maybe the JAX AI actually thinks about using your tool instead of another tool. Is that a valid thesis? What do you think about this concept of agent-led growth?

    4. LHLisa Huang35:26

       I do see it happening in the future, but I think we're pretty far away from it, to be honest. Um, I know that, you know, in Silicon Valley, people do tend to live in the future, but if you take a step out of that, and a lot of our customers are certainly not Silicon Valley, um, you know, homegrown businesses. They're, they're mom and pop stores globally. Uh, you do get a view that people are very excited, and they want to use AI, think they see a lot of potential in it, but the adoption curve is still gonna take time because people have to, you know, trust AI, they have to adopt it over time, um, they need to train their staff on it. So I think before you get to agent-led growth, you first need to, can we... You need to get companies to actually delegate more and more to agents. Right now, even the ag- at least from what I see, the, where there are agents, it is still quite, there's still quite a lot of human monitoring. Um, people are not necessarily... And especially in our space, right, in kind of the finance space, you're not gonna let [laughs] these agents go and pay your bills for you without someone looking at it, et cetera. So, um, I think we'll get there over time, but I think it's still quite a ways away. I think we're still very squarely in the getting people to adopt, um, and really leverage what AI can already do today.

    5. AGAakash Gupta36:39

       Hmm. Okay. So you've done the hard work of building an agent into your product. Should most B2B SaaS product leaders be thinking about building an agent into their product, or what is the framework to understand whether you should?

    6. LHLisa Huang36:54

       Oh, 100%. Like, [laughs] um, in some sense, everybody is already behind. Um, the technology moving so quickly, I think user adoption and company adoption is still behind where the technology actually is. Uh, and so yeah, my advice for every B2B SaaS is, like, you should be deeply integrating agents, AI everywhere, because the future software's not gonna look like it does today, and that's coming, that future's coming very, very fast.

    7. AGAakash Gupta37:19

       Hmm. So we were talking about the future. I wanna talk a little bit about the future of product management, because you have a vantage point that pretty much no one else will have had w- working at all these big companies and now leading product at one of these big companies. What is the future of AI product management?

13. 37:39 – 38:40

    Will AI replace PMs?

    1. LHLisa Huang37:39

       Sure. So let me answer this with two questions. Number one, will AI replace product managers? I don't think so. Um, the reason I don't think so is, yes, AI is great at specific tasks, and 100% the AI role is gonna change. Uh, peop- pr- uh, PMs will be expected to use AI for a lot of things that they still, some people still manually do today. But I don't pay PMs just to write PRDs or just to, you know, create mocks or to, like, you know, manage, um, road maps, et cetera. I pay them for their product judgment, and that's not something I see going to an AI anytime soon because it's not that there... There's not a clear right answer, right? A lot of these things, it's there's a bunch of signals. They're not super clear, and you have to kind of give your product opinion. What should we do? What shouldn't we do? Why? Why is this the right answer? What is the taste and the judgment in all that? So I don't think this is going away anytime soon, and that is a key skill that we're still gonna need PMs for. But they'll just be supercharged by using AI in some of the, the more, um, executional tasks that they do today.

14. 38:40 – 39:45

    Ads

    1. AGAakash Gupta38:40

       You know that feeling when you try to prototype something with AI and it spits out something completely generic? Then you spend hours tweaking colors, fonts, copy, and features just to make it feel like your actual product? Here's the problem: most AI app builders aren't built for product teams. They're built for those starting from scratch. But product teams aren't building from zero. You have an existing product, real customers, design guidelines, a backlog full of ideas you need to explore and validate fast. That's what Reforge Build does, AI prototyping that starts from your product. Add your customer feedback, strategy docs, and product features as context. Create reusable templates using your product design. Explore multiple variants side by side. Collaborate with your team in one place. Reforge Build generates prototypes that reflect your real pricing tiers, real features, real customer language, not generic placeholder. Stop fighting tools built for founders. Start prototyping like a product team. Reforge Build, AI prototyping built for product teams. Try it free at reforge.com/aakash. That's R-E-F-O-R-G-E.com/A-A-K-A-S-H, and use the code BUILD for one month free of premium.

15. 39:45 – 51:15

    Breaking into AI PM

    1. LHLisa Huang39:45

       And then-

    2. AGAakash Gupta39:45

       There's a lot of PMs I talk to who are feeling anxiety about this, you know, because there's the, the layoffs. It seems like the layoffs might be disproportionately hitting the PMs. It seems like, you know, getting a junior PM role is harder than ever. Plus, it seems like the director of PM role might be getting compressed, or group product manager level where VPs are managing a bunch of ICs. What's your take on the future of this role? Is it gonna continue to grow as fast as it has in the past?

    3. LHLisa Huang40:12

       So what I will say is I do think the structure of PM and eng and design is gonna change, and what I mean by that is you don't need as big teams to go do stuff. So the structure is definitely gonna change, and historically we kind of look at PM to eng ratios. That will certainly compress. If you don't have as many engineers, you're not gonna need as many PMs either. Uh, but I think what's interesting is I do think the P- PM role is going to evolve. Uh, PM UX design is gonna become more of a hybrid, so I think it will become the expectation that PMs are also builders, right? They gotta go understand the product, propose it, champion it, but then also create the first designs, create the first prototype. Like, go build it. Go code it. Like, you can go do it with all the tools that are here now. So, um, the role's gonna evolve. There- we're still gonna need products in the future. I think there will still be needy roles. Yes, I think the, the, the change is, um, it can cause anxiety because it's changing so rapidly right now. Um, and it is a more competitive market because there are fewer roles. There was kind of a glut, uh, over the last few years of hiring into this space, so it just means that there's a lot more competition. My advice would be, if this is your passion, it's what you wanna do, just go learn the tools. Reinvent yourself. Now is a time of transformation, and everyone has the availability to go do that. So, uh, yeah, go ahead and, and lean in, and, uh, I, I would still encourage people to, to follow this career path if they're interested.

    4. AGAakash Gupta41:37

       Whenever I give this advice, I hear a bunch of buts, so I wanna role play some of those buts. I think the biggest but is, "But I'm not working on AI features or products at my company, and we don't have access to many AI tools." If that's the case, if you're that PM, how do you lean in? How do you kind of get to the, the part of product management that is growing right now?

    5. LHLisa Huang42:00

       Yeah. I think that's... Personally, I think that's not an acceptable excuse, and the reason I say that is everybody, not even PMs, just, like, people, everyone in the whole world mostly, has access to AI tools now. It's not very expensive. You can get a little, you know, you can get a subscription, do Cloud Code, use OpenAI's tool, use Gemini's tool. A lot of stuff is free, and then it's just a small amount if you wanna pay, you know, for a little bit more of a premium experience. But you have the AI tools with you already. Um, the vast majority of companies are not necessarily fine-tuning models. They're using it off the shelf, and, uh, places like Claude, OpenAI, Gemini, they offer really good ways to access the latest. So you already know what the, these models are capable of just by using those consumer products. So I would 100% encourage people, just go... If your company is not officially giving you a role to go build AI products, why do you need them to build AI products? Just go use the tools yourself. You can go build your own AI products. You can build your own GIFs, right? Just, um, do all that stuff yourself. There's no one stopping you. Uh, yeah, I think it's never been more easy to access these AI tools than it is now. So, uh, for those that are really interested and hustlers, like, please do lean in and, and just go build stuff.

    6. AGAakash Gupta43:09

       And then you get to see the flip side. You get to see how people are marketing themselves having done this. What is the, what is the way as a job seeker to stand out to a hiring manager and showcase my side projects, showcase that I am using AI tools on my own?

    7. LHLisa Huang43:25

       Yeah. So I think here, one is just doing the work. I... A lot of people are still talking about it but not really doing it, so even that [laughs] gives you a bit of a leg up. Um, I would say the second thing is the thing I mentioned before about, like, you're gonna have to check the box in the foundations, right? Product strategy, execution, um, cross-functionalUh, stakeholder management, et cetera, but the extras. Not everybody has those extras. So if you are passionate about the space and you wanna show that you, you know, you're really leaned in, there's lots of ways to demonstrate that. I'll give you an example. In, uh, one of the recent people I hired that I interviewed, um, they, you know, we were assessing them for AI role. They actually didn't have AI experience before. Um, I was not sure that they'd be the right fit for, you know, our particular role, but this person really impressed me with, in our first interview, what they said was, "Okay, looks like you're trying to build, you know, kind of financial tools for small businesses. Well, I went through and I watched three hours of TikTok videos from, uh, these coaches that coach small businesses, and here's a summary of all the things that they said about what these small business needs have and s- advice that they might wanna give them from a finance standpoint." And I was just like, "Whoa." That was... I hadn't met a single other candidate who'd done that, and it just kinda showed the passion, the willingness to, like, roll up your sleeves and go deep, um, and that I, you know, I deeply respected. By the way, this was a very senior person, so the fact that they were willing to just do, like, deep IC work themselves really did speak to me.

    8. AGAakash Gupta44:52

       So you're hiring these AI PMs. What are the vectors you're looking at to see that a AI PM is staying on the bleeding edge? What are the things you're encouraging your AI PMs to do to stay on the bleeding edge of these tools, 'cause they're changing so fast?

    9. LHLisa Huang45:06

       I think the best thing you can do as a PM is just to use them. Um, you can apply them either in your job, or it's actually totally fine to apply them in your personal life. That's typically what I've, what I've tended to do because there are less restrictions on using my personal data [laughs] for a lot of these tools, um, versus, you know, company data. Uh, but yeah, I think, um, you know, really just trying out... Obviously, it's the horizontal LLMs. They provide a lot of new tools and capabilities all the time. They're really at the bleeding edge of a lot of things, so just using the latest that's been launched. Uh, but on top of that, in your day-to-day, you know, are you using design prototyping tools to create your own designs? Um, I have a PM who actually, everything that we talk about, he always pulls up something in our code base, like our actual company [laughs] code base, and he will go prototype the idea he's talking about so it looks like it's inside our actual product, which is pretty impressive. So, um, yeah. Whatever you can do to use AI tools for your product thinking f- um, for research, competitive analysis, user research. Use it for design prototyping. Uh, create your own designs, and then build it yourself. I think the more of that you can do, uh, you'll naturally use these tools. You'll naturally hear about them from people. You'll notice, like, the little details of what's working well and not working well and why. So I think this will c- this is plenty to keep you on the cutting edge. Uh, beyond that, you can always... There's, like, a million news articles about AI these days, so subscribe to your... Just pick three high-quality ones that you kind of follow week to week, and you'll get all the key information you need from that.

    10. AGAakash Gupta46:31

        Mm. So don't overload yourself with unlimited content. Spend more time actually testing out the tools using your personal data so that you aren't restricted by what your company is doing. When it comes to advice that you have, we've gone over so much advice, but-

    11. LHLisa Huang46:47

        Yeah

    12. AGAakash Gupta46:48

        ... fundamentally, the advice that people want the most is, "What is the roadmap for me to break into any one of these, Apple, Meta, Google, Xero, any of these big AI companies?" So if you had to put together a roadmap for somebody, what would be the roadmap to becoming an AI PM?

    13. LHLisa Huang47:04

        I would say if you have the opportunity to do it in your job, that's great, because you're kind of doing it... Um, you have all of... You have to go through all the regular challenges of working with a team to go do it. There may be constraints from a resourcing standpoint, from a how do you go to market, et cetera. So if you d- if you have the opportunity, definitely lean in there. That's always, is always great. Um, if you don't have those opportunities, or even if you do, you could still continue to do things on the side, because, um, yeah, this gives you the maximum flexibility to explore all kinds of things that you may not be doing specifically in your current role. The other thing I would do is, um, it's, it is important to continue investing in building your network. Sometimes that, it can give you leg up into some of these companies, so do continue to meet people, build relationships, keep in touch. Um, and that relationship building, by the way, starts in your current role, so whoever you're working with, they often go off and to do, do great things in the future, so do make sure you're showing up well in your own teams and keeping in touch with people that you've worked with in the past. And then the last thing I would say is if you are tarting one-- targeting one of these big companies, they tend to have fairly standard interview processes. I will say interviews are themselves a skill. [laughs] So you may be a great PM, but if you can't communicate that in an interview, in the structure of that interview with the time constraints and the format that it has, it just may not come across. So when you are getting ready to actually interview, you just gotta practice interviewing. Go do, you know, mock interviews. Go ask your friends who are in those companies to help you. Just practice over and over and over. Just drill it, and, uh, that will set you up for the best possible success in those interviews.

    14. AGAakash Gupta48:41

        That is one of the clearest roadmaps into breaking into FAANG that I've heard in a while. So if I had to summarize what I heard, right? Network with people, get referrals into these. Obviously, do a great job in your job so you have direct AI experiences that you can then talk about in the interview, and be ready for the game that is the interview at these companies. A lot of companies, especially Meta and Google, are not gonna be just asking behavioral questions. They're gonna be asking case interview questions, product execution questions, and product sense being the most common dimensions. And so you really need to prepare for that specific beast that is those interviews. Do you have any tips for people when they're preparing for case interviews at FAANG?

    15. LHLisa Huang49:19

        I think repetition would be my main advice. The more you do it, the more it'll just feel like second nature. Uh, and you'll have seen a breadth of different types of questions. So I would just practice, practice, practice. If you can get folks who work at those companies, um, or who used to work at those companies to, uh, do mock interviews, that's great. And then for additional time, additional practice, you can actually just practice on your own. There's a bunch of sites that offer example questions. Just do them yourself. Uh, you know, do it as if it was a real interview, so speak out loud, write down what you want to, and then you can look at some of the example answers at the end to, to help you as well.

    16. AGAakash Gupta49:52

        Amazing. I tried to pick your brain of all the most important advice. Is there anything I shouldn't, I didn't ask you that I should have?

    17. LHLisa Huang49:59

        About landing a career in PM, uh, I thinkYou know, I think we covered most of it. Maybe the only other thing is, uh, I think passion and grit really shows through. So the more you are truly interested in that role, in that company, in that product, it will show. So do your research. I, I personally think you should always go for something that you're truly interested in, not just because of a name, because of, you know, something on your resume. Um, yeah, the more passion you have for the product, both it'll help you do better in the interview, but, two, once you're in the role, you'll do a better job.

    18. AGAakash Gupta50:36

        Amazing. Lisa, thanks for taking the time to share all these amazing insights with my audience. If they want to find you online, where can they go?

    19. LHLisa Huang50:43

        I'm on LinkedIn. That's probably the... my main thing, although, uh, yeah, you can, you can connect with me on LinkedIn, um, and look forward to connecting.

    20. AGAakash Gupta50:51

        All right, everyone. We got so many different things we learned about today. We talked about how to create great AI features. We did deep dives into Gemini Gems and the AI assistant for Meta RayBan. We talked about how to use Gemini Gems, and we talked about how to become an AI product manager and navigate your career. Hope you enjoyed this. There's more details in the newsletter, and we'll see you in the next episode.

16. 51:15 – 52:01

    Outro

    1. AGAakash Gupta51:15

       I hope you enjoyed that episode. If you could take a moment to double-check that you have followed on Apple and Spotify podcasts, subscribed on YouTube, left a rating or review on Apple or Spotify, and commented on YouTube, all these things will help the algorithm distribute the show to more and more people. As we distribute the show to more people, we can grow the show, improve the quality of the content and the production to get you better insights to stay ahead in your career. Finally, do check out my bundle at bundle.akashg.com to get access to nine AI products for an entire year for free. This includes Dovetail, Mobbin, Linear, Reforge Build, Descript, and many other amazing tools that will help you, as an AI product manager or builder, succeed. I'll see you in the next episode.

Episode duration: 52:10