AI Product Leadership Masterclass with the author of The Making of a Manager

1. 0:00 – 2:30

   Intro

   1. AGAakash Gupta0:00

      Is the product designer role gonna exist in 10 years? Is the product manager role going to exist in 10 years? So what are the right steps people in those roles should be taking?

   2. JZJulie Zhuo0:09

      First, let's not think of ourselves as these predefined roles. Let's just actually think of ourselves as builders. ChatGPT, obviously we all know that it's the fastest-growing consumer app to hit 100 million.

   3. AGAakash Gupta0:20

      Julie Zhuo spent 13 years as a designer at Facebook, where she started as an IC and rose to become a VP of product design. She also wrote the Wall Street Journal bestselling book, The Makings of a Manager, and now she is an AI product leader working as a founder at Sundial and building products for companies like OpenAI.

   4. JZJulie Zhuo0:42

      In some ways, I feel like there's a contraction, because the whole promise of what AI enables is that each individual person can do more than they've ever been able to do before. You have to know-

   5. AGAakash Gupta0:52

      Yeah

   6. JZJulie Zhuo0:52

      ... what the AI is doing and whether it's producing exceptional or average work.

   7. AGAakash Gupta0:57

      How are you day-to-day building your product sense or your product taste?

   8. JZJulie Zhuo1:00

      Go and figure out who are the absolute best, and it's much easier these days. You can go on Twitter-

   9. AGAakash Gupta1:05

      Mm-hmm

   10. JZJulie Zhuo1:05

       ... and you can probably figure out, like, who are the best designers on Twitter.

   11. AGAakash Gupta1:08

       As you're making that transition from IC to new manager, what are the biggest blind spots?

   12. JZJulie Zhuo1:13

       I think the first one is that it's really hard sometimes for people to give up the IC work.

   13. AGAakash Gupta1:20

       Yeah. What makes a really great product leader in the age of AI? Really quickly, I think a crazy stat is that more than 50% of you listening are not subscribed. If you can subscribe on YouTube, follow on Apple or Spotify podcasts, my commitment to you is that we'll continue to make this content better and better. And now on to today's episode. So today, we are gonna break down everything you need to know to become an amazing AI product leader, and specifically how you can work with design, how you can input into design in a way that's really effective. Julie, thanks for being on the podcast.

   14. JZJulie Zhuo2:05

       Thank you for having me, Aakash. I'm super h- happy to be here.

   15. AGAakash Gupta2:08

       Yes, in San Francisco, meeting in person. After reading your content... I think I've been subscribed to your newsletter for, like, three or four years now.

   16. JZJulie Zhuo2:15

       Mm-hmm.

   17. AGAakash Gupta2:15

       It's like a dream come true.

   18. JZJulie Zhuo2:16

       You gotta keep kicking me to write more stuff these days. [laughs]

   19. AGAakash Gupta2:21

       [laughs] Yeah.

   20. JZJulie Zhuo2:21

       It's harder and harder to find the time.

   21. AGAakash Gupta2:22

       Yeah, right? The consistency part is always-

   22. JZJulie Zhuo2:25

       Mm

   23. AGAakash Gupta2:25

       ... the, the challenge there, and that's The Looking Glass, if you guys haven't seen it. So

2. 2:30 – 8:42

   The Death of Product Development

   1. AGAakash Gupta2:31

      you recently wrote in the newsletter-

   2. JZJulie Zhuo2:32

      Mm

   3. AGAakash Gupta2:32

      ... about the death of product development as we know it.

   4. JZJulie Zhuo2:35

      Mm.

   5. AGAakash Gupta2:35

      Can you break this down for us?

   6. JZJulie Zhuo2:37

      Yeah. I was really struck by the fact that with all of the innovation that we're seeing in the AI space, that of course it's changing the way that we interact with products. You know, ChatGPT, obviously we all know that it's the fastest-growing consumer app to hit 100 million. And, you know, I, I lived through the time of Facebook back when that was really fast growth, and Instagram, WhatsApp, and now we're just sort of s- seeing an even steeper trajectory. Um, but yet I didn't feel there was enough people talking about what is changing, not just from the product lens for consumers, but how we work. Like, how we as product builders actually go about creating products. And a lot of what I would call the, the kind of current, uh, bar is this idea of these very well-defined roles. You know, there's product designer, there's engineer, there's product manager, there's researcher, there's data analyst. And essentially at these larger or more mature companies, the goal is to get all of these different... You know, you kind of think about it as like, my analogy in my head was, like, Captain Planet, [laughs] if you ever remember that from, like, the '80s.

   7. AGAakash Gupta3:46

      Oh, yes.

   8. JZJulie Zhuo3:46

      Maybe shows my age.

   9. AGAakash Gupta3:47

      [laughs]

   10. JZJulie Zhuo3:47

       But it's like, you know, you have a representative from each of these different disciplines come together to form a group or a pod, and then this pod goes and creates a product. And so, you know, we've, we've gone and we've invested over these last 20 years in building, you know, a very clear type of career ladder for each of these disciplines. Like, I think there's good standard industry-wide understanding of, like, what is a intro-level designer, what is, like, a senior designer, what is, like, a architect designer. Obviously even more true for engineering, which has been around for a long time. I think that's true for product management as well with even your newsletter or the content of people like you, Lenny, and so forth, right? Trying to spread the knowledge of what makes these disciplines what they are. And what I was observing is that that's actually totally changing because of AI. Um, in some ways, I feel like there's a contraction, because the whole promise of what AI enables is that each individual person can do more than they've ever been able to do before. You know, AI can take care of a lot of the lower-level manual work. It can help a lot with even higher-level ideation. Not to mention, it's getting smarter and smarter, so, you know, every year, every six months, we're seeing new models. And so if things continue at this rate, then it's very clear that we're not gonna have to have, like, all six or five or however many of these roles come together to make anything. It should be the case that one person or two per- two people, [laughs] uh, or three sometimes, can have everything that they need to kind of develop a, a entire end-to-end product.

   11. AGAakash Gupta5:26

       Yeah.

   12. JZJulie Zhuo5:27

       And if that's the case, then we really need to redefine what it means to be each of these different functions.

   13. AGAakash Gupta5:33

       Yeah. I think that's kind of the existential question [laughs] out there for people in each of these roles. When I talk to them, they're kind of worried, like-

   14. JZJulie Zhuo5:41

       Yeah

   15. AGAakash Gupta5:41

       ... is the product designer role gonna exist in 10 years?

   16. JZJulie Zhuo5:44

       Mm-hmm.

   17. AGAakash Gupta5:44

       Is the product manager role going to exist in 10 years? So what are the right steps people in those roles should be taking to prepare themselves for whatever eventual merging that might happen?

   18. JZJulie Zhuo5:55

       I think the first thing is to try and see if we can remove ourselves from the identity of whatever role title. Which is hard, because obviously when we start, you know, we wanna... glom onto something, right? And for me, I know there was, you know, I came in, I actually was trained as an engineer. I joined Facebook, I was on the front-end engineering team.

   19. AGAakash Gupta6:15

       Oh.

   20. JZJulie Zhuo6:15

       Uh, but the front-end engineering, engineering team was kind of also the design team. And I remember, you know, some of my earliest questions are always like, "Well, what does it mean to be a designer? What does it mean to be a good designer? Who are the older designers or more experienced designers who can help me?" And, you know, we all wanna maybe learn a craft or a discipline and feel like we know what the lane is. And I think, though, that what this time calls for is for us to try and separate ourselves from those lanes. Like, I actually-

   21. AGAakash Gupta6:43

       Yeah

   22. JZJulie Zhuo6:43

       ... think it's very, very healthy for us to no longer think of ourselves as, "I'm a product manager," "I'm a designer," "I'm an engineer," but actually to think about these as just different skills and experiences. I mean, we all, you know, even within the realm of product management or design, there's different archetypes. It's not like every single designer is the same. We talk about specific skills. This person is really, really strong at visual aesthetics and craft. This person is a great systems designer. This person is a, a wonderful zero to one type of visionary, right? But, you know, that's true in PM, I think it's true in engineering. I'd rather us talk about the specific skills that people have and also how we can use those skills and the fact that we're good at these particular things to then augment ourselves with tools, AI tools, to kind of help us figure out how we build either, you know, the one or two or three-person team that can come together to build something end to end. So first, let's not think of ourselves as these predefined roles. Let's just actually think of ourselves as builders or product builders.

   23. AGAakash Gupta7:46

       Yeah.

   24. JZJulie Zhuo7:47

       I, I quite like that term 'cause I think it's, it's more general purpose. And at the end of the day, the reason we were in that role to begin with was to contribute some of our skills to the art of building, building something meaningful. So think of ourselves as a builder. Think, "Okay, what are the skills that I come in where I am much better than the average, maybe today much better than the AI?" I'm sure there are still things today that, um, all of us bring to the table where we can likely have better taste than AI, which you and I can talk about in, in just a little bit. Um, figure out how to, uh, you know, use that to direct the AI, but then figure out which skills are we lacking that we can then augment either with, um, another person who can complement us or, you know, by figuring out how to perhaps use the AI in a, in a more sophisticated manner. But that's the way to think about it, right? Not in terms of roles, but in terms of skills.

3. 8:42 – 15:02

   Learn The Craft

   1. AGAakash Gupta8:42

      Yeah. And I feel like one of the important things, if you are a designer or you are an engineer or a product manager, is start to learn the craft of those people in those other roles, right?

   2. JZJulie Zhuo8:54

      Mm-hmm.

   3. AGAakash Gupta8:54

      Ask them questions about, "Well, how are you building this?" What I love to do is, like, shadow people on the job, really empathize-

   4. JZJulie Zhuo8:59

      Mm

   5. AGAakash Gupta8:59

      ... with what they're doing so that you understand what they're specially bringing, right, that's way above average from the AI.

   6. JZJulie Zhuo9:06

      Mm-hmm.

   7. AGAakash Gupta9:06

      I think that that helps the collaboration a lot.

   8. JZJulie Zhuo9:09

      Absolutely. And I think you're hitting on a really important point, which is understanding what great looks like, what the difference between exceptional versus great versus average versus mediocre or le- less than average. The ability for the eye to discern all of these different sub levels is increasingly important. Because, again, if we have this very smart technology, AI, that can do a lot of things, the difference between the work you're gonna do that's great and the work that's gonna be average comes down to you have to know-

   9. AGAakash Gupta9:45

      Yeah

   10. JZJulie Zhuo9:45

       ... what the AI is doing and whether it's producing exceptional or average work.

   11. AGAakash Gupta9:50

       Yep.

   12. JZJulie Zhuo9:51

       And if you don't know because you've never been exposed to that skill or your eye isn't well trained, then we can only expect you to do the average of what the AI can do, which isn't going to stand out.

   13. AGAakash Gupta10:03

       So the critical skill then is product taste or product sense. I think those are the words that people put around it. If you are someone who wants to be that great product leader, how are you day-to-day building your product sense or your product taste?

   14. JZJulie Zhuo10:17

       I think, um, it is a lot of what you mentioned, which is that you wanna first identify who are the best at this.

   15. AGAakash Gupta10:23

       Yeah.

   16. JZJulie Zhuo10:24

       That's always how I start. If I'm trying to learn, you know, for example, when I started Sundial, and a lot of what we do is data analytics, I didn't, I'm not a data analyst by training, so I know that there's a lot I don't know. The first question is always, "What does amazing look like? Who are the people who do it the best?" And I was lucky to work with my co- co-founder, Chandra, who I think is truly, truly world-class. But one of the things that we spent a lot of time on is him actually try to break things down for me.

   17. AGAakash Gupta10:50

       Mm.

   18. JZJulie Zhuo10:50

       Try to help me understand what is the art of the thing, right? Because every discipline or every, uh, creative skill, when you stretch it out to what the very best looks like, it always is much more like an art than it is a science.

   19. AGAakash Gupta11:03

       Yeah. [laughs]

   20. JZJulie Zhuo11:04

       And so the people who are, like, the masters, quote, quote, or the people who are very, very good at that often, you know, learning from them is, is a lot of conversation. It's a lot of also seeing how they will tackle a problem and break it down and just understanding their thought pattern and, and tr- and from that trying to extract, like, okay, what are the principles? How can I build my own mental model and framework from what does, what great looks like?

   21. AGAakash Gupta11:30

       Mm-hmm.

   22. JZJulie Zhuo11:30

       And so I would say, like, f- go and figure out who are the absolute best. And it's much easier these days. You can go on Twitter-

   23. AGAakash Gupta11:36

       Mm-hmm

   24. JZJulie Zhuo11:36

       ... and you can probably figure out, like, who are the best designers on Twitter. And if not, you know, an easy, an easy tactical way is I go to just one person, like any designer, and I'm like, "Who are the 10 designers that you admire the most?" And they give me a list, and then I will go and, you know, recurse it, go to each of those designers, like, "Who are the 10 that you admire the most? Who are the 10 that you..." And eventually you'll see patterns 'cause people start to converge on, like, which people, um, are, is doing really good stuff. And-

   25. AGAakash Gupta12:03

       Yeah

   26. JZJulie Zhuo12:03

       ... this has, you know, worked for me in engineering or even in a lot of other disciplines. But if you just ask people, eventually you do get a good s- list of like, "Oh, these people are really good." And then what I try to do is I try to look at all of their work.

   27. AGAakash Gupta12:16

       Mm.

   28. JZJulie Zhuo12:17

       I try to read everything that they produced, if they blog, if they got, you know, um, I try to, um, get into their heads a little bit. And really what I'm trying to look for is, like, how do they see the world? How do they think? How do they understand the art of what they do?

   29. AGAakash Gupta12:32

       Yeah.

   30. JZJulie Zhuo12:33

       And if I'm lucky enough, if I do work in that domain, then I wanna show it to those people for feedback.
4. 15:02 – 17:00

   Ads

   1. AGAakash Gupta15:02

      job is? Today's episode is brought to you by Jira Product Discovery. If you're like most product managers, you're probably in Jira, tracking tickets and managing the backlog. But what about everything that happens before delivery? Jira Product Discovery helps you move your discovery, prioritization, and even roadmapping work out of spreadsheets and into a purpose-built tool designed for product teams. Capture insights, prioritize what matters, and create roadmaps you can easily tailor for any audience. And because it's built to work with Jira, everything stays connected, from idea to delivery. Used by product teams at Canva, Deliveroo, and even The Economist. Check out why, and try it for free today at atlassian.com/product-discovery. That's A-T-L-A-S-S-I-A-N.com/product-discovery. Jira Product Discovery, build the right thing. Before we dive deeper, let's talk about something every PM faces, getting alignment on product decisions. You know that feeling when you're trying to explain a user flow to engineering or justify a design choice to leadership and you're just describing it with your hands? That's where Mobbin comes in. Mobbin is the world's largest library of real-world mobile and web app designs from industry-leading apps like Airbnb, Uber, and Pinterest. Instead of spending hours taking screenshots or hunting for inspiration, you can instantly find exactly how successful products handle onboarding, paywalls, checkout flows, whatever you're facing. Over 1.7 million product builders use Mobbin to benchmark against best-in-class products and show their teams proven solutions. Whether you need to convince stakeholders there's a better way to handle user activation or research how top apps approach feature discovery, Mobbin gives you the visual proof to back up your product decisions. Check out mobbin.com/aakash. That's M-O-B-B-I-N.com/A-A-K-A-S-H, and get 20% off your first year.

5. 17:00 – 21:12

   Definition of a Managers's Job

   1. JZJulie Zhuo17:00

      To me, what a manager is really trying to do is get better outcomes-

   2. AGAakash Gupta17:04

      Mm

   3. JZJulie Zhuo17:04

      ... from a group of people towards the goal of that group. And so it is very contextual because first of all, you have to understand what is the goal of that group. If this is a group of educators, I'm a superintendent for a school district, my job is to educate kids, like, have basically the highest quality education that I can for the school district and make sure that each child is prepared, right? That's the goal. And so the job of the manager is to figure out how to use the levers, and there's three of them, which we'll talk about, to essentially get this group of people. You know, it's everyone on the school staff. It's, uh, all of the teachers. It's everybody, you know, who is employed by the school district. How do you get them to work together so that you get that outcome, a better outcome-

   4. AGAakash Gupta17:53

      Mm

   5. JZJulie Zhuo17:53

      ... than you would've gotten, let's say, if you weren't there or if, you know, there was another random person who was managing, right? So that's, that's really the, the end thing that we're trying to go after. There has to be something concrete about getting better outcomes from a group of people towards their objective.

   6. AGAakash Gupta18:10

      Mm.

   7. JZJulie Zhuo18:11

      And so now we talk about levers. What are the levers of a manager? And it really comes down to three things. The first is people. Do you have the right people together? Do you have the right teachers? Are y- are you hiring great teachers? [laughs] Are you, uh, hiring the right people to hire those teachers or manage those teachers, or, um, are you hiring the right staff to support those teachers, right? So-People is probably the biggest lever, and the ability to hire people, to set expectations for them, to create role definitions, boundaries, and to be able to performance manage them, including fire them if it's not working, is, is one such lever.

   8. AGAakash Gupta18:50

      Yeah.

   9. JZJulie Zhuo18:50

      So that's probably the most important. The second lever is process. So sure, I might have hired all these amazing people, but how do they work together? Let's say two amazing people disagree. What should happen? How do we make decisions? Are we just gonna argue it out? Are we gonna have three meetings and then make a call? You know, a lot of the, like, how people work together or what are the norms around how w- work happens is what I call process.

   10. AGAakash Gupta19:20

       Process.

   11. JZJulie Zhuo19:20

       Yes. And then the third bucket is just purpose. It's, it's the idea of like, do you have a picture and a vision, and can you get everyone aligned on what we're really trying to do together? Because even if you had great people, and even if they had, you know, all the rules for it, if people are not on the same page about what really matters or what success looks like-

   12. AGAakash Gupta19:42

       Yeah

   13. JZJulie Zhuo19:42

       ... then they're still going to be at odds. So all these three things are really important, and I would say are the levers of what a manager has to be able to get this group to reach their outcome.

   14. AGAakash Gupta19:53

       Hmm. So the original definition, the definition that most people walk in with, is like, I have these discrete tasks.

   15. JZJulie Zhuo20:00

       Mm-hmm.

   16. AGAakash Gupta20:01

       I'm giving feedback to people, I'm meeting with them. But the revised definition is start with the outcome-

   17. JZJulie Zhuo20:06

       Mm-hmm

   18. AGAakash Gupta20:06

       ... and then think about how you're gonna use the three Ps of people, process, and purpose to achieve that outcome.

   19. JZJulie Zhuo20:13

       That's right. Yeah.

   20. AGAakash Gupta20:14

       Okay.

   21. JZJulie Zhuo20:14

       And all of the tasks are in service to being able to drive those outcomes, right? Which is also why, you know, often it's very hard to just judge someone. Like if they seem busy and they're doing all the right things, I always go back to like, well, is the team better off? Are they producing better work, you know? And what, what does a good outcome look like? It, it really only boils down to two things. You're either doing things at higher quality or, or, you know, so quality meaning like, let's say, teachers, going back to our example, right? Quality of education is improved, so like all the students are better off. That's what I mean by, like, the work improves or the quality improves.

   22. AGAakash Gupta20:49

       Yeah.

   23. JZJulie Zhuo20:49

       Or you're doing things faster at the same quality.

   24. AGAakash Gupta20:52

       Mm-hmm.

   25. JZJulie Zhuo20:53

       So you're able to scale, right? So maybe in, you know, same quality, but now, you know, you can do things faster, so everything's more efficient. Um, or perhaps, you know, one teacher can now support 50 students instead of 25 at the same quality. Like that would be another way to think about, you know, we've, we've done better work.

   26. AGAakash Gupta21:11

       Hmm.

6. 21:12 – 28:12

   Julie's Thoughts on AI Agents

   1. AGAakash Gupta21:13

      And this is truly timeless. Is there anything about AI that's changing how you apply those three levers of people, pers- process, and purpose? Is like... Like, I'm just wondering like, you know, AI agents, how do those fit into all of this? Is that another P asset that you need to think about deploying?

   2. JZJulie Zhuo21:29

      Well, it's funny because I've actually thought about agents almost like as another person-

   3. AGAakash Gupta21:34

      Yeah

   4. JZJulie Zhuo21:34

      ... in the workforce, right? So you have to pick the right... You know, we talked about people as a lever, and maybe you can, if you add agents or this type of AI intelligence in there, you're like, "Okay-"

   5. AGAakash Gupta21:43

      Yeah

   6. JZJulie Zhuo21:43

      "... well, do I have the right tools or do I have the, the, the, you know, model with the right skill set or whatever it is for my purpose?" Um, process is still really, really important. I mean, we, you know, prompt engineering and the way that we direct AI-

   7. AGAakash Gupta21:56

      Yeah

   8. JZJulie Zhuo21:56

      ... and we help them understand what does the job look like and what does a good job look, you know, and, and all of the work that goes into trying to, in a highly structured way, help them do their best work.

   9. AGAakash Gupta22:09

      Yeah.

   10. JZJulie Zhuo22:09

       It's not that different in my mind with actually managing humans. You know, for example, if you have somebody who is a very early career type of person on your team, you wanna direct them well, and oftentimes that means break things down for them in a way that they can actually be effective. And I find that that process of being able to break down a complex task into smaller and smaller tasks so that maybe this new grad on your team can do it, is not that different from doing that with, for example, an AI agent.

   11. AGAakash Gupta22:39

       Yeah. I think LLMs are brilliant interns.

   12. JZJulie Zhuo22:41

       Mm-hmm. Yes.

   13. AGAakash Gupta22:42

       Right?

   14. JZJulie Zhuo22:42

       Exactly. Yeah.

   15. AGAakash Gupta22:43

       Yeah.

   16. JZJulie Zhuo22:43

       And, and it, it, it, you know, and they, they kind of struggle sometimes with, like, ambiguity or complexity the same way that you might imagine, you know, an intern or somebody who's just doesn't have a lot of experience yet might struggle.

   17. AGAakash Gupta22:56

       Hmm. So in the AI era, apply that same skill set. And I really like actually that you have purpose as its own, right? Because for the agent, you need to set the right purpose.

   18. JZJulie Zhuo23:05

       That's true. Yeah.

   19. AGAakash Gupta23:06

       It usually needs to be kind of narrow. [laughs]

   20. JZJulie Zhuo23:08

       Yeah.

   21. AGAakash Gupta23:08

       It needs to be well-defined.

   22. JZJulie Zhuo23:08

       And it's like almost the first thing, right? You need to tell the, this is the outcome.

   23. AGAakash Gupta23:12

       Yeah.

   24. JZJulie Zhuo23:12

       Like this is what you're trying to do, and by the way, this is... And then you start with usually the process, like this is all what you should do, like or this is how you should work.

   25. AGAakash Gupta23:19

       Yeah.

   26. JZJulie Zhuo23:20

       Right. And then of course, you have to choose the right one. Like are you gonna use Claude for this one? Are you gonna use OpenAI? Like, you know, some are better than others at, at various tasks. Like which particular submodel? Like that to me is like maybe like the people element.

   27. AGAakash Gupta23:33

       People. Yeah. And some are like 100x more expensive than others too.

   28. JZJulie Zhuo23:36

       Yes. That's right.

   29. AGAakash Gupta23:36

       So you have that same dynamic as we do with people.

   30. JZJulie Zhuo23:39

       Yes. That's right.
7. 28:12 – 30:40

   Blindspots While switching from IC to Manger

   1. JZJulie Zhuo28:12

      awful for th- these reasons," then it's a learning opportunity for me to try and understand what is normal or what is good or what is expected in this place.

   2. AGAakash Gupta28:21

      Mm. And then outside of aligning your expectations, as you're making that transition from IC to new manager, what are the biggest blind spots people have?

   3. JZJulie Zhuo28:33

      IC to new manager. I think the first one is that it's really hard sometimes for people to give up the IC work.

   4. AGAakash Gupta28:40

      Yeah.

   5. JZJulie Zhuo28:41

      Because it was a great point of pride. It's something that they love doing.

   6. AGAakash Gupta28:46

      Mm.

   7. JZJulie Zhuo28:46

      And frankly, they're good at it, and maybe they had the best context about it. And so moving to this mindset where you're not just contributing because of your core skill in this area, but you're also being asked to scale, is challenging. Because what I notice is that, you know, as a new manager, and by the way, I don't think... I think this is a great, um... This is, this is actually a really good thing in certain contexts. But, you know, if you look around and you're like, "Oh my God, this particular piece of code is not as good as it could be or needs these changes," and you're a very good IC, you may have this instinct of, "Let me go in and let me fix it."

   8. AGAakash Gupta29:26

      Yep.

   9. JZJulie Zhuo29:28

      But you can only do that so many times because you only have so many hours in the day. [laughs] Eventually what you really need is, like, hey, everyone's just writing kinda shitty code [laughs] and I'm the one going in trying to fix it. This is definitely not gonna scale. [laughs] And so what the, what the manager needs to elevate their mindset to is, like, these are system problems. You know, it's not just an individual example of bad code which I myself can go, "I need to fix the system problem. Why do we have so many instances of shoddy code-

   10. AGAakash Gupta29:59

       Mm-hmm

   11. JZJulie Zhuo29:59

       ... in the team? How do I fix that?" And usually, again, your lever is, like, somebody else needs to play that role. You know, uh, maybe the reviewers aren't doing their thing and, and I need to hold them accountable. Um, maybe I just need to hire better engineers. [laughs] Or maybe I have a better process, right? We need, we're not doing code reviews and we do code reviews. You know, like, I mean, things like that are how you would want the person to start thinking, but it can be tough because some- it's, like, so... Feels, like, so tempting to just go in and just continue to do the IC thing, which will solve the p- the problem in its small, isolated form. But-May miss solving it in a broader and systematic

8. 30:40 – 35:48

   Ads

   1. JZJulie Zhuo30:40

      scale.

   2. AGAakash Gupta30:40

      Today's episode is brought to you by the AI PM Certification on Maven. Run by Miqdad Jaffer, who is a product leader at OpenAI, this is not your typical course. It's eight weeks of live cohort-based learning with a leader at one of the top companies in tech. OpenAI just doesn't stop shipping, and this is your chance to learn how. Run along with product faculty and Mo Ali, the course has a 4.9 rating with 133 reviews. Former students come from companies like OpenAI, Shopify, Stripe, Google, and Meta. The best part? Your company can probably cover the cost. So if you want to get $500 off, use my code AAKASH25 and head to maven.com/product-faculty. That's M-A-V-E-N.com/P-R-O-D-U-C-T-F-A-C-U-L-T-Y. AI evals are one of the most important skills for PMs, and I know you know they matter. The question is, are you doing them right? Most teams are winging it with basic metrics and hoping for the best. Meanwhile, the teams that actually ship reliable AI, they've cracked the code on systematic evaluation. Today's episode is brought to you by the AI Evals for Engineers and PMs course by Hamel Hussein and Shreya Shankar. This live Maven course will teach you the battle-tested frameworks from Hamel and Shreya, who are the engineers behind GitHub Copilot's evaluation system and 25-plus production AI implementations. Four weeks, live instruction. Next cohort starts July 21st. Start shipping AI that actually works. Enroll at maven.com with my code AG-PRODUCT-GROWTH for over $800 off. That's AG-PRODUCT-GROWTH. I think anyone who's been a manager could empathize with that. So let's move into this third one. Let's say you're now leading a small team. You said that trust is the most important ingredient, and I totally agree, right? Trust is everything. You want people to be able to bring negative issues to you, and I think most of us can understand that, but the harder part is actually making [chuckles] an environment where people feel trust, feel safe. So how do you establish that culture?

   3. JZJulie Zhuo32:56

      I think the first thing that is really important is that you wanna be the kind of team where we can confront reality. And reality, when I say reality, I mean what's really happening on the ground, right?

   4. AGAakash Gupta33:09

      Mm-hmm.

   5. JZJulie Zhuo33:09

      'Cause what might be really happening on the ground is, you know, you have a bunch of customers that don't really love what you're building, or they're, they're unhappy. Or you actually have a team that's not working very well together, and they're not gonna hit their, their, their, uh, deadlines that they set, you know? And so reality is just what it is. This is like what is truly happening on the ground. But what can be tough is that sometimes we don't wanna face reality because it's a problem.

   6. AGAakash Gupta33:35

      Yeah.

   7. JZJulie Zhuo33:35

      You know, we'd rather, of course, all of us would rather be like, "The business is going great. Our customers love us."

   8. AGAakash Gupta33:41

      [laughs]

   9. JZJulie Zhuo33:42

      "We're, we're always on target to hit our..." You know, and, and so I think that if you're the kind of person who maybe almost like holds on a little too tightly to like what is this ideal vision and is not as open to reality, it will be very hard for other people to feel like they can be honest with you.

   10. AGAakash Gupta34:02

       Mm-hmm.

   11. JZJulie Zhuo34:02

       So I would say the first thing, which is how do you get people to trust you, is be the kind of person that can face reality.

   12. AGAakash Gupta34:10

       Yep.

   13. JZJulie Zhuo34:10

       Be the kind of person where it's not, you're not expecting everything to be perfect all the time. You know, you may intend for it. You may wish for it. You may aspire for it, but you don't expect it. And you're the kind of person where given a problem, you're not gonna, you know... 'Cause sometimes, like I think if we get too attached to this vision, "Oh, it should be this way," and then some messenger comes and says something like, "Hey, Julie, we're not gonna go," and, "By the way, you know, this project is really not gonna hit its deadline," I might feel really upset about that because like I have to give away, like I was very happy in my vision that, you know, everything's going to be awesome, and now this is an issue which doesn't match my expect- it doesn't match, like it doesn't match my expectation. And what will happen is sometimes I'll take it out, you know, or it's natural for, for leaders to sometimes take it out on the messenger. [laughs]

   14. AGAakash Gupta34:58

       [laughs]

   15. JZJulie Zhuo34:58

       It's like, "Why did you tell me this?" You know, "Go and fix it," blah, blah, blah, right? But the pro- problem isn't that. The problem is not the messenger. The problem is like this is actually happening, and you just need to be the kind of person that's okay with where reality is and not let it kill your motivation or make you angry, upset, but actually be sturdy. Just be like, "Okay, so that's happening. What do we need to do?" And turn towards action, bias towards action. And I think if you can be that person who, who's collected, who's calm, who even thanks the messenger, like, "I'm really happy that you brought to me an area where my understanding of, of, of how things were going was not calibrated," right?

   16. AGAakash Gupta35:38

       Yeah.

   17. JZJulie Zhuo35:38

       "You helped me see reality. That's a wonderful thing, you know. I appreciate people like you. I hope you do that more often."

   18. AGAakash Gupta35:45

       Yeah.

   19. JZJulie Zhuo35:45

       "Now help me figure out how we're gonna go and resolve this together."

9. 35:48 – 41:20

   The Three Levers That Never Change

   1. AGAakash Gupta35:48

      Yeah. I think you wrote about this link, recently on LinkedIn that like the people that managers value the most bring problems to them.

   2. JZJulie Zhuo35:57

      Mm-hmm.

   3. AGAakash Gupta35:57

      And the hardest part is just creating this environment where you do it. So the tactics you outlined, right? Number one is like control yourself in that conversation [laughs] . Number two is like, as you control yourself, actually thank them for bringing the issue to you. And then number three is like deeply think about solving the system instead of that specific scenario. Did I get that right?

   4. JZJulie Zhuo36:18

      That's right. That's right.

   5. AGAakash Gupta36:19

      Okay.

   6. JZJulie Zhuo36:19

      Yeah. And also ask the person or the rest of the team for help because I go back to like it's not always your job. You know, ultimately, as a manager, as a leader, you own the outcome. But it doesn't mean you have to be the one to come up with every single solution and every single idea. And in fact, if you think that is you, that, that should have been another thing we talked about in terms of like what's another blind spot that ICs often have.

   7. AGAakash Gupta36:45

      Mm-hmm.

   8. JZJulie Zhuo36:45

      You know, as an IC, you were asked to do this, and it is your job. And so if, if, you know, someone said, "Hey, you're, you're gonna do this project," like it's on you to come up with a solution to that project. But as a manager-There's no rule that says the only solutions are the ones that come from your head. It's very, very, um, common that great ideas are gonna be across the organization, and your job is more to curate and find. Like, you know, a lot of people have great ideas. How do you curate the ideas, and how do you invite everyone else to work with you to be the kind- be part of the solution, be members who can go and, and drive the action that solves the problem that an organization needs?

   9. AGAakash Gupta37:28

      Mm-hmm.

   10. JZJulie Zhuo37:28

       So again, it doesn't all have to come from you. It's not like every single problem is ultimately you're the only person who can come up with, with a solve for it. [laughs]

   11. AGAakash Gupta37:36

       Yeah.

   12. JZJulie Zhuo37:36

       Um, there are some problems that only you can solve, but the vast majority are the kinds of things that, that I think great managers, they- they're humble enough to realize that, like, they should ask for help from many other parts of the organization.

   13. AGAakash Gupta37:48

       Mm. And that's probably another blind spot that people have, is you have to be kind of prideful in your craft as an IC, but as you become a manager, you almost need to put the humble hat on, where you're looking for problems, looking for problems in your org instead of just, "We're the best. We're awesome. We're doing great."

   14. JZJulie Zhuo38:05

       Yes. Yeah, it's a, it's a very delicate balance. I think you can go too far on either side. So on one extreme, to your point, if you're-- You wanna have enough conviction, right? Because it goes back to, like, one of the things is purpose, and a lot of times people are looking to you to at least, if not have all the answers, at least have some kind of conviction in the vision and what great looks like and what are we all here to do and why does it matter. And so I think you kind of have to have some amount of conviction. And a lot of the times the teams gain energy from the fact that there's a sturdy person who has some kind of conviction, if not necessarily, like, I know what the answer is, that I have conviction we will find the answer-

   15. AGAakash Gupta38:45

       Mm

   16. JZJulie Zhuo38:45

       ... some way, somehow. So that is important. I think if you show up and you have no clue what to do and you just don't stand for anything, it's very hard to be effective in that role as, as a leader and as a manager. And so I think that conviction, and to your point, there's some amount of, like, belief, trust, um, you know, pride that comes from you having a good eye in things. That's gotta be there. But on the other hand, if, if it's too far that way, right, and you're like, "No, we're perfect. We're great. There's nothing wrong," then you're not even opening yourself up to reality, as we were talking about before, and that also creates issues because then people are afraid to break your bubble. Nobody wants to tell you what's really going on. And then your image of where the organization is starts to deviate more and more from the reality. And then if you don't have a good understanding of reality, all of your judgments and all of your, your guesses as to what to do will be off.

   17. AGAakash Gupta39:42

       Mm. So the north star here is having a really clear-eyed picture of the strengths and weaknesses-

   18. JZJulie Zhuo39:49

       Mm

   19. AGAakash Gupta39:49

       ... of the org that you're managing.

   20. JZJulie Zhuo39:50

       Yeah. And if-- And I also think that there's-- You know, I, I come from design and it's, uh, obviously a very, very ambiguous, right? Same with product management. These are ambiguous problems. It isn't usually the case that someone says, "Hey, we have this big problem to solve. We have to figure out how to make, you know, people not get confused." It's not usually the case that, like, "Oh yeah, I know what the solution is." But what I've learned through being a designer is that even if I don't have conviction in any solution, I can often have a lot of conviction in a process.

   21. AGAakash Gupta40:22

       Mm-hmm.

   22. JZJulie Zhuo40:23

       You know? And the process is go and talk to customers, understand the problem. If we have a very, very deep understanding of a problem, I have very strong conviction that we'll come up with a great answer, right?

   23. AGAakash Gupta40:34

       Mm-hmm.

   24. JZJulie Zhuo40:35

       Another thing that gives me a lot of conviction is this idea of iteration and moving quickly. I have a lot of conviction that, look, if we understand the problem, we'll try the thing. I may not have a lot of conviction that the thing itself is 100% going to work, but I know that if I get it out there and I get it in front of people, I get their feedback, that I will learn something, and the thing that I learn will help me make a better next version and a better next version. And so a lot of just everything I've learned about, you know, what it's like to operate in ambiguity comes from, I think, a much deeper trust of here's the process of what it is. And that gives me a lot of, I would say, optimism-

   25. AGAakash Gupta41:11

       Mm-hmm

   26. JZJulie Zhuo41:11

       ... for things, that, um, as long as I don't just, you know, get lazy or give up or whatnot, that, like, eventually we'll find our way to something really, really great.

   27. AGAakash Gupta41:19

       Mm.

10. 41:20 – 46:43

    What is Feedback

    1. AGAakash Gupta41:21

       Wanna move to this next lesson I have highlighted here, which I believe, yes, is feedback. And you talked a little bit about this, right? Managing, performance managing people. And how do you really deliver feedback that leads to positive action?

    2. JZJulie Zhuo41:39

       So the most important thing is that how we have to talk about what is feedback and why does it matter. And what I think we do for each other when we give feedback is we're kind of holding up a mirror for the other person that reflects back something that we see with the hopes that the other person will then recognize something and it helps them become the kind of person that they're proud to be. So ev- again, go back to, like, feedback is just a, you know, a tactic. What's the end goal? The goal is to help the other person make a change to become a person that they wanna be, so when they look back they're really proud of it. So that's what I mean by, by, um... Well, that's what I think really, really good feedback is. And I, I think both of... You know, if I were to ask you, Aakash, like, what's some of the best feedback you have ever received, it probably has that quality where someone said something, they made you aware of something you weren't aware of, and in becoming aware you change your actions which allowed you to become, um, the person that, that you're happy with or, like, a, a quality that you're happy with, right?

    3. AGAakash Gupta42:45

       Mm-hmm.

    4. JZJulie Zhuo42:46

       So that's what we're trying to do. And so if you think about it, feedback is really a gift. You know, we're trying to do something that helps the other person. So I think the most important thing that helps feedback land well is you genuin- genuinely have to care about the person-

    5. AGAakash Gupta42:59

       Mm-hmm

    6. JZJulie Zhuo42:59

       ... and care about the fact that you're trying to help them.And this sounds really obvious, but I will tell you that even my own experience, there are times, you know, I, with my kids, with my husband, with people at work, where sometimes my motivation for giving feedback is I just wanna be right. [laughs]

    7. AGAakash Gupta43:16

       [laughs]

    8. JZJulie Zhuo43:17

       You know, I just want-- like I, I'm, I feel a little bit peeved. Maybe I feel like there was an injustice, and I wanna tell them how they really pissed me off.

    9. AGAakash Gupta43:26

       Mm-hmm.

    10. JZJulie Zhuo43:26

        In that moment, there's a difference in the feeling, right? The feeling is like I'm trying to get it off my chest 'cause I feel bad and I wanna feel better. I can tell you from a lot of experience that it's unlikely to land [laughs] in the way that I want.

    11. AGAakash Gupta43:42

        Yeah.

    12. JZJulie Zhuo43:42

        But if I can really, you know, give myself a moment, calm down, and think about it, and, you know, maybe the f- the, the substance of the feedback is the same, but now my attitude is, you know, this person really needs to hear it because they may not be aware of the impact that these words have on me, and they'll probably keep saying it to other people, and, and other people will have that type of reaction, and it would help them to be aware of this so that they don't do it again. If that's the i- you, you see the difference is like, you know, the, the substance can be exactly the same, but my mentality makes all the difference.

    13. AGAakash Gupta44:17

        Yeah.

    14. JZJulie Zhuo44:17

        And if I have the second mentality, then the way that I will deliver the feedback, the w- the, the style, the tone, the words I use, my facial expression, my body language is gonna be totally different. It's gonna connote care for the other person. It's gonna connote, or it's gonna, sorry, convey, um, a sense of respect, and it's going to make it far more likely that they're going to hear the message and it will truly change them.

    15. AGAakash Gupta44:45

        Mm-hmm. If you have that caring, that empathy, and you don't bring that emotional baggage of-

    16. JZJulie Zhuo44:51

        Yeah

    17. AGAakash Gupta44:51

        ... just trying to do it for yourself, but you're doing it for them.

    18. JZJulie Zhuo44:53

        Yes. That's the single most important thing, and everything else just naturally, I think, falls into line if that's the mentality that you carry.

    19. AGAakash Gupta45:01

        Mm. So it's less about the tactics of delivering sandwich feedback or those types of things, and it's more about the right mindset going in.

    20. JZJulie Zhuo45:08

        That's the most important. I, I know, I think there are still tactic, tactics. I think a very important tactic is also, you know, I even have like a particular script that I sometimes give people if they're feeling very nervous about giving feedback, and the script goes something like, you know, you start with how I felt. Like, Aakash-- or you start with like a particular fact. When you failed to respond to me after I pinged you four times, I felt, how did I feel? Nervous. I felt, um, rejected maybe. I felt disappointed because it made me think that maybe I was doing something wrong, or maybe you didn't care about me, or whatever it is, right? That's just, that's a general script. So, so therefore, I wanted to bring this up so that I could really understand your perspective and how we can make our relationship work.

    21. AGAakash Gupta46:03

        Mm.

    22. JZJulie Zhuo46:04

        So that is the general feeling of the script. Start with something factual, like this thing happened. How did it make me feel? 'Cause usually this feeling is some kind of negative. I felt worried. I felt rejected, again, disappointed, et cetera, et cetera. Why? Well, I felt that way because I assumed XYZ.

    23. AGAakash Gupta46:21

        Mm-hmm.

    24. JZJulie Zhuo46:21

        And so now my goal is to bring this to the table and just try and see if, you know, my assumptions are right, or maybe they're not, or like, how, how can you help me, essentially? Like, and help and be a part of working this out with me because I really value our relationship, and I wanna make sure that we can grow in our trust of each other and the way that

11. 46:43 – 52:49

    How AI is Changing the Domain

    1. JZJulie Zhuo46:43

       we can work together.

    2. AGAakash Gupta46:44

       So those are four of the timeless nuggets-

    3. JZJulie Zhuo46:46

       Mm

    4. AGAakash Gupta46:46

       ... that I thought we had to cover here because we're trying to cover everything it is people need to know to be a good AI product or design leader. So bringing it back to that, how is AI changing product and design leadership?

    5. JZJulie Zhuo47:01

       I think the first thing is that it is a time of great uncertainty. I think we have to acknowledge that. I think, you know, you started by saying a lot of people are really nervous. Will my role still be around in 10 years? What does it mean for me and my career? You know, we're, you know, you and I are talking about, uh, roles blending, and, like, it's, we're not gonna have these, like, very, very well-defined career ladders as we used to have in the past. Like, what does that mean? And so I think the first thing that leaders have to understand is that there's a lot of uncertainty, and people might rightly be nervous, and they don't know what to expect. And so I think sturdiness is really important.

    6. AGAakash Gupta47:39

       Mm.

    7. JZJulie Zhuo47:39

       Sturdiness being the quality of like-- like I think a lot about like a willow tree, which is the type of tree that is very, very strong, but the limbs are quite flexible. Right? So it's a sturdy tree, but it isn't brittle.

    8. AGAakash Gupta47:53

       Mm.

    9. JZJulie Zhuo47:54

       It's not like you snap it and it breaks. And I think that's kind of the feeling that we need to convey, that, yes, it's a time of change. We're not trying to deny it and that things will change, but, you know, let's go back to like what principles, what are the true principles, um, that keep us together? Uh, you know, and, and I think every company or every organization that's trying to do good work has those principles, which is what are we here to do? What's our outcome? What's the vision? And if we have all of these great tools and this better technology, then, like, might we see it as an exciting time rather than just a purely fearful/negative time? So I think a lot of it about it is like acknowledge where we are, but also use that as an opportunity to create a new narrative-

    10. AGAakash Gupta48:39

        Mm-hmm

    11. JZJulie Zhuo48:39

        ... for like what, again, goes back to like what greatness looks like and how it can help, how all of this change can likely help us achieve or, or, or get to the outcome that we're all here to do. So, um, I think acknowledge the uncertainty, take the time to craft like a new narrative, help people understand like what really matters and what are the things that are timeless even despite this change. And I-- the third is be very experimental.You know, going back to our analogy of the willow tree, this is a time for experimentation. This is not a time when anyone, including me, can sit here and be like, "You know what you need to do? It's like A, B, C, D, that those are it. If you do it, you're gonna succeed." I think it's ... I think all of us have to take this mindset of like, yeah, we can all get great ideas from a lot of places, but we have to try them, and we have to see how it works, and we have to iterate our way. So, um, as an example, you know, we mentioned, uh, just to take this example of like maybe some teams today are constructed where you need one of each of these roles in order to be able to have a team that can go and, and, and operate. Well, is that really true? You know, could it be the case that we might just have two people work on something, and could they ship it end to end? I don't know. It's an experiment. Let's try it. Let's see what we learn. And from the learnings, you know, again, it's probably not binary. We'll learn something. We'll, we'll evolve it. Okay, so, um, how should we piece together the working groups or pods of the future? And I think framing everything as this is an experiment, and we're gonna try it, and we're gonna ... You know, this is how we're gonna set up the experiment, but then this is what we're going to n- notice, or this is how we're gonna debrief on it and take away the learnings. Like, I think that we, we have to kind of embrace experimentation and iteration in this, in this period, you know? And it's not following tried and true playbooks of the past. It's like all of us trying to adopt new things, try this thing, try that thing, and then figure out for ourselves what's gonna happen.

    12. AGAakash Gupta50:37

        Mm-hmm.

    13. JZJulie Zhuo50:37

        And I always ... I felt like the best companies have always kind of carried a little bit of that, right? Not every, um, you know, e- every company who's kind of gone and, and innovated in the way that they work has, has, has kind of had that sense. I just think more companies have to be forced to do that now, um, because we're in a time of greater change.

    14. AGAakash Gupta50:56

        Yeah. Start from the first principles of the talents you have and the things you need to accomplish instead of, okay, these are the traditional titles that I would put against a problem to solve a problem. Be experimental with how you're building your teams.

    15. JZJulie Zhuo51:08

        Mm-hmm. Yeah, and then also I think that leaders have to try and really pay attention to, like, what are the mental models that people are carrying, 'cause they may be older mental models.

    16. AGAakash Gupta51:20

        Yeah.

    17. JZJulie Zhuo51:20

        Right? And if it's ... You know, it goes back to, like, if your mental model is very different from mine, then we're definitely going to, uh, be at odds, uh, on one decision or another somewhere down the line. And so I think we have to be very clear, like, what are people's mental models, and let's just make them explicit and be able to actually have, like, open discussion about, like, okay, no, that was the old mental model. Now we're gonna try a different way, and that does mean we're gonna have to, you know, do things differently. I think when people's expectations are set that like, yeah, okay, we are gonna do things differently or ... You know, um, and one example for me is, like, you know, we ran a hackathon recently for, um, for our team, and I was really blown away by all of the different ideas that came out of the hackathon. And they ... And, and people felt, like, far more empowered, or they were like, you know, one or two people were able to build so much more, especially using all of these AI tools. And it was interesting because it almost felt like people had permission to do a lot of things that maybe they, they didn't feel like they did, they had, um, when it wasn't hackathon time. And that was quite surprising to me because I didn't realize that, like, there's a different mental model. Um, 'cause for me I'm like, oh, aren't we the kind of company where everyone can just have an idea and do it and ... If they wanted to. But I think for everyone else, like, actually the mental model sometimes can very, can be very unconscious in terms of like, well, what does the day job look like versus, like, what does hackathon look like, right? And, um, and how do we, you know, bridge the gap, or how do we become explicit about the fact that that is actually what people think?

    18. AGAakash Gupta52:48

        Yeah. Very

12. 52:49 – 1:00:55

    What Makes Great AI Product Leaders Different

    1. AGAakash Gupta52:49

       ... I think a lot of people feel that way coming out of hackathons. It's a really good tactic probably [laughs] for-

    2. JZJulie Zhuo52:54

       Yeah

    3. AGAakash Gupta52:54

       ... creating a little bit more of that ownership mentality.

    4. JZJulie Zhuo52:57

       Yeah.

    5. AGAakash Gupta52:58

       A little bit more of thinking of the ideas instead of just being handed them. So we talked about these ... Those elements that you just highlighted, I think that's gonna take you to modern product design leadership good stuff. What makes a really great product leader in the age of AI?

    6. JZJulie Zhuo53:18

       I think a really great product leader is somebody who is pretty, you know, is, is very, very solid in, like, things like what is the problem that we're trying to solve, what's the mission, you know, what's ... What are we here for? So all of those principles, those core principles of, like, understanding the problem, the customers, the user, that continues to be true. That was true then. That's still true now. So I'm, you know, saying that-

    7. AGAakash Gupta53:44

       Yeah

    8. JZJulie Zhuo53:44

       ... that still continues to be, um, somebody that or a person needs to kind of have that. I think the second is what I mentioned. Like, I think you have to be very much, um, a learner. So there's a lot changing about the world. I think you kinda have to be at that cutting edge. Like, if you don't understand where technology is today, it's very hard to say that you're a great product leader in the age of AI. You have to understand the tool, and that just means all of us, you know, who are trying to be good product leaders, like, we need to be building things with these tools. We need to be using the tools. We need to be the first people to have switched our habits-

    9. AGAakash Gupta54:22

       Mm-hmm

    10. JZJulie Zhuo54:22

        ... um, over from what they used to be. We're the ones who need to throw away our old systems of working and use them as an opportunity to, like, test what this new generation of tools has to offer, right?

    11. AGAakash Gupta54:36

        Yeah.

    12. JZJulie Zhuo54:36

        I even say that as somebody who, you know, have a very traditional way of, like, writing, 'cause I love writing. And I realize, no, but the world is changing. Like, I need to actively disrupt how I-go about writing or communicating, even though that's something that I felt like I honed and maybe I thought I was, like, getting reasonably good at. But all that is changing because, like, that's, like, people are gonna do it differently and if, like, I'm not there in the trenches, like, actively trying to figure out how these things are changing, like, why would I expect anybody else on my team to, to, to do that as well? So I think, like, leading by example and being, um, at the forefront of using things and disrupting our old habits and, and ways of doing things is also really, really important. Um, I think the third is, as we said, ex- you know, um, the kind of experimental, being able to direct and lead a team through this period of change and, and, and nimbleness, flexibility. I think that's really important. Um-

    13. AGAakash Gupta55:33

        Mm

    14. JZJulie Zhuo55:33

        ... I, I think that all of us just going back to, like, we're all beginners here again, right? Going back to remove our identity from what it was that got us here. You know, like I'm sure many of our listeners are, are very esteemed and experienced product leaders, but even that idea of, like, we're good product leaders almost feel like let's throw that out, out the window.

    15. AGAakash Gupta55:54

        Mm.

    16. JZJulie Zhuo55:55

        Are we actually really good product leaders in this new era? I don't know.

    17. AGAakash Gupta55:58

        Mm-hmm.

    18. JZJulie Zhuo55:58

        I think we have to earn that. You know, we have to almost go back to, like, being a beginner, get back on, um, in the ring, build some things in this new way, and actually earn the stripes again to be considered somebody who is a great... I, I don't know if I'm a great product leader-

    19. AGAakash Gupta56:14

        Mm

    20. JZJulie Zhuo56:14

        ... in the era of AI.

    21. AGAakash Gupta56:15

        [laughs]

    22. JZJulie Zhuo56:16

        I hope I can be. I hope I, you know, I, I wanna go back and, and, and, um, like I, I every day try to show up and, and do these things, but I can't say that I am yet.

    23. AGAakash Gupta56:25

        Mm. With that in mind, staying humble, what are the right tools? Let's say I'm a product leader and I haven't really embraced those AI tools. If I wanted to catch up to the latest in August 2025, what are those AI tools I should be using in my work?

    24. JZJulie Zhuo56:41

        Oh, I mean, [laughs] there's so much.

    25. AGAakash Gupta56:43

        [laughs]

    26. JZJulie Zhuo56:44

        I mean, the default one that, um, I'm sure you know i- is, is already well-known is, like, all you do is just use ChatGPT-

    27. AGAakash Gupta56:51

        Mm-hmm

    28. JZJulie Zhuo56:51

        ... and you just incorporate way more of that through your workflow. To me, it's, like, less about tooling and more about when we use it, right?

    29. AGAakash Gupta56:59

        Okay.

    30. JZJulie Zhuo56:59

        I think there's a lot of things, and I have to catch myself daily, where I have this idea, my first instinct is to just do it the way I would've done it. Like, I'm gonna create a doc. I'm gonna go in and actually create, you know, the Slack post. And now even the idea of, like, wait, can I get feedback first from ChatGPT? [laughs] You know, can I run my high-level thinking and ask it to poke holes at my mental model or my assumptions? [laughs]
13. 1:00:55 – 1:09:15

    Essential AI Tools Every Leader Should Master

    1. JZJulie Zhuo1:00:55

       Yeah.

    2. AGAakash Gupta1:00:55

       Wow. Analyzing your own conversations.

    3. JZJulie Zhuo1:00:58

       Yeah.

    4. AGAakash Gupta1:00:58

       How should product leaders be using AI data tools?

    5. JZJulie Zhuo1:01:01

       How should... I, I think, so my answer to a lot of your questions have had this pattern of like, well, what first, what are some of the principles that are-

    6. AGAakash Gupta1:01:08

       Yeah

    7. JZJulie Zhuo1:01:08

       ... like timeless, right? And then how do we think about the tooling layer on top of that? Um, I think a lot of what I'm seeing right now is that we see a lot of huge organic growth because there's so much market pull for the next generation. I mean, we see that with ChatGPT in terms of its growth. Obviously Cursor, Replit, like, there's these, you know, companies are scaling faster than ever because they've unlocked something really remarkable, which has made it so that, you know, it used to be only people who could code could code, but now lots of people can go and actually create an app. And it used to be, like, only certain people could answer medical questions or, you know, pay for a therapist, and now you've, like, got a tool that can kind of do that, right? And, um, and that's awesome because you, you know, you see, like, a lot of organic growth. But I think what's important with, with, um, any sort of analysis is, like, the whole point of data is to give you a better view of reality and what's going on. And so it's not always just like, oh, if things are going well, then you don't need to look at the data, right? I go back to, like, the best teams that I've ever seen use data are the ones in which people on the ground are, like, absolutely fanatical about trying to understand what is the real phenomenon that's happening, um, and, and in as much texture as possible.

    8. AGAakash Gupta1:02:24

       Mm-hmm.

    9. JZJulie Zhuo1:02:25

       And so the texture can come, you know, from, like, talking to your customers. Like, if you talk to your customers all the time, you're gonna get a lot of texture about, like, what's working, what do they hate, what do they not. That's... I consider that data. But I also think that's true quantitatively as well, which is like, what are people doing? What are they clicking on? You know, how much time did they spend? What are the use cases are... Which use cases are growing for your tools, which aren't? And a lot of times, you know, you, like, I talk to a lot of these startup founders and, like, they don't actually know. They don't have... They know, like, they have maybe good texture if they talk to their customers, but they don't always know exactly, like, what's happening in the product because it isn't instrumented, and even if it is instrumented, it isn't, like, brought together in, um, in an operational model that's easy. Like, I als- I think about everything as like this is just observability. So good analytics is observability for your business, and the more refined your model, meaning the more things you log and track and you understand which metrics impact which things, the easier it is for then if, like, you know, something starts to go off, you catch it early, right? You don't wait until, like, oh my gosh, growth has stopped or stalled. You catch it when the trend is starting early. So that would be, you know, what I would say to everybody who, you know, even if you're growing like a rocket ship and things are going really well, you gotta ask yourself, how much do you really understand about how all of the different users are using the product? And the narrower you can go, where you're like, which kind of user? Which use cases are growing well? Which features are being used? Um, what's the, you know, trend or the impression? What's the conversation in the marketplace about, right? There's a lot of really great tools that can help you now get more and more of that color and that fidelity, but even the best tools won't matter if the person doesn't feel like they want to get exposed to that, those things, if they want to actively be looking for those things. And so it still goes back to, like, you have to be the kind of person who's absolutely curious and maybe fanatical about just understanding reality.

    10. AGAakash Gupta1:04:24

        Hmm. I like how it came full circle back there. So we talked about this a little bit. We talked about the importance of taste, and we talked about, and you previewed, what happens when AI has better taste than you.

    11. JZJulie Zhuo1:04:40

        I think it's coming. [laughs]

    12. AGAakash Gupta1:04:42

        [laughs]

    13. JZJulie Zhuo1:04:42

        So here's... Let's zoom back. This is what I think is the state of the world today, right? AI is, I would say, maybe in the s- 70th or 80th, um, of, of a lot of... So it's, like, probably better than average. It certainly is average in terms of the, the content that's online about a lot of stuff. It's probably a little bit biased towards, like, you know, um, more knowledge because, like, the kinds of people who write the stuff or, or, you know, write books about it is, like, maybe, um, uh, uh, ha-has structured it or kind of has, like, a little bit more, um, uh, depth of understanding. But it isn't usually great at, like, the cutting edge today, right? And that's true with, um, design. I think that's true with, like, uh, you know, the engineering. It's true with even writing. Like, it's not as good as a Hemingway or a C.S. Lewis or, or any of that. Um, and so it, you know, we can say that it's, like, slightly better than average on a lot of stuff. And so, um, so, so then if we look at, okay, like, well, let's say I'm an individual. There's certain things that I think I've, um, you know, invested a lot of my time and energy into, and so maybe I can do better than AI, but there's, like, a whole lot of stuff that I'm worse than where the bar is. And so if I'm worse where the bar is, then AI's actually doing better than me, and I should just kind of trust it more and use it more, right? So examples for me include things like, um, uh, anything that's related to go-to-market. Like, I'm kind of a noob there, and so all of its ideas that AI has for me is, like, clearly better than mine, so I should probably follow more of that [laughs] and not just rely on my own instincts.

    14. AGAakash Gupta1:06:17

        Hmm.

    15. JZJulie Zhuo1:06:18

        Um, similarly, you know, maybe drawing or illustration. Like, I think I draw fine, but clearly, like, it's doing much better than me, um, already in terms of, like, what Midjourney and other things can create. So like, cool, I'm gonna use that, and I'm just gonna go with what its recommendations are, and maybe I can continue to develop my eye, but I know it can certainly, um... There are areas where, like, look, I can just kind of trust it a little bit more. Um, and if I don't trust it enough, I'll try and, like, go back to, like, find the expert who can, who's still a human, and, and have them help me out.

    16. AGAakash Gupta1:06:48

        Mm-hmm.

    17. JZJulie Zhuo1:06:49

        Now, in areas where I think I'm still better than AI, I think a couple of things. The first is I should assume that with the rate of change, one day AI will have better taste than me. [laughs]

    18. AGAakash Gupta1:06:59

        Mm-hmm.

    19. JZJulie Zhuo1:06:59

        Right? And so-What I can continue to try and do in all respects is continue to train my eye, 'cause the eye is easier to train usually than the hand. Um, so we, this also brings us back to what we were talking about in the initial parts of our conversation today, which is, like, how can I learn from the best so that I have, maybe for as long as possible, have an advantage in terms of at least knowing or recognizing what great is. And so then for, I will have more of an ability to direct the AI towards, um, um... So I can try and do that as long as possible, okay? But let's assume that one day AI gets better than me, then even at that, because we're getting just better and better intelligence. I think then, okay, what does that world look like? I don't know. I think there's, like, a lot of speculation. I think, like, lots of things will break or be better. Um, but I will say that one thing that I think AI can never really take away from us is the journey and the process, um, of learning and growing and doing things that we're proud of, right? And so for me, I look at this example of chess. So it's been a while now since computers have bested humans at the game of chess. But it isn't the case that somehow it, we, we all gave up and nobody plays chess anymore because what's the point, because AI is better than humans. I think actually chess has become way more popular since we've had AI come to the forefront and all those news articles about how it, you know, bested the best chess player. And I think a lot of the reason why is it's still an inherently human activity to wanna choose to do this thing, to compete against someone, to feel like you're getting better in this particular skill. Even if you're not gonna be the best in the world at it because AI is better than you, there is still something very human, very joyful about doing these things.

    20. AGAakash Gupta1:08:47

        Yeah.

    21. JZJulie Zhuo1:08:48

        Right? And that is very human, you know? So even if AI can do all of the skills better than me, I don't think it should take away from the fact that if I derive joy, pleasure, and meaning from getting better at certain skills, let me continue to do that. Let everyone continue to do that. Let us all enjoy chess for the sheer joy of playing the game.

    22. AGAakash Gupta1:09:08

        Mm-hmm.

    23. JZJulie Zhuo1:09:09

        You know? And that might, that might be a fine world for us to be in.

14. 1:09:15 – 1:15:55

    Lessons from OpenAI's Product Team

    1. AGAakash Gupta1:09:16

       Speaking of learning from the best, you get to work with some of the best product leaders in the world because you build data analytics products for the product leaders at OpenAI. What have you learned from them? What do they do differently?

    2. JZJulie Zhuo1:09:29

       I think that they are, um... So sp- okay, so first, OpenAI, wonderful company, really amazing, super, super high talent density. Um, I will say is they work really hard. [laughs]

    3. AGAakash Gupta1:09:41

       [laughs]

    4. JZJulie Zhuo1:09:42

       Um, and I think that that's probably already well known. Uh, but it's, it's such a competitive time right now, and it's-

    5. AGAakash Gupta1:09:47

       It's, like, seven days a week is what I hear

    6. JZJulie Zhuo1:09:48

       ... Yeah, and, and, and so I can tell you, yeah, like, people are looking at data and dashboards and stuff, like, every day of the week. [laughs] So that's data point number one. Um, I think data point number two is what I was, uh, alludes to what I was, uh, speaking of earlier, which is that having a very, very, very detailed and refined understanding of what is happening on the ground. So, like, every single day, you know, I see analysts and data practitioners from OpenAI come in, and they are just, you know, they're, they're like, their usage is extremely deep. They spend a lot of time, and that's because anytime something changes, you know, this amount grew. Like, you know, this country grew by this amount, or this country sort of stalled. They want to understand why.

    7. AGAakash Gupta1:10:36

       Mm.

    8. JZJulie Zhuo1:10:36

       Why did it happen? What does it tell us? You know, what, what can we learn? Is there a pattern? Is there something that we can do to change that trend if we don't like it? But even not, do we start to understand a little bit more about human nature and the nature of our customers, or what works for them, or what they want, or how they behave? So it's, I think, this extremely obsessive need to, like, get at the deeper explanation of understanding what is really going on. And I think the other thing is just, like, good hygiene, right? What I see, like, really top companies h- have is this hygiene of, like, we have a metrics review every week. Why? A lot of the purpose of a metrics review, okay, is, you know, metrics review is essentially like, you know, you get a group of leaders there, um, and everyone looks at the numbers, the KPIs, and the dashboards, and everyone talks about, like, how they're trending. And I think that that's really important because it does a couple things. So first is it creates alignment, right? E- everyone now understands, okay, we're on the same page about the story. You know, if we didn't do that, you may have a week where you think, "Oh, our company's doing really awesome. We had a great month." And I might think, "No, we did really poorly," and it's 'cause I have one anecdote about how, like, we lost two clients.

    9. AGAakash Gupta1:11:46

       Mm.

    10. JZJulie Zhuo1:11:46

        But you're, like, looking at how many features we shipped. So it's like we have a different frame. You know, we have different, um, mental model of, like, what success looks like. I think when you go back and you have, like, a really great observability layer and you've kind of agreed that this is the right observability layer for, like, truly, you know, measuring reality, and therefore you also have a forum in which every week we can get out in there and we're like, "Well," you know, I can say something like, "Well, we had a great week." And you're like, "Wait, why do you s- why do you think that? We had a shitty week." Because, you know, then we're on the same page, right? That is a forum for us to be able to align as a leadership team or as a, as a, as a group on that. So that's, I think, the hygiene of something like our metrics review. I think the second one is just, you know, uh, like, if you're representing a particular country and that went down, it's an accountability where we're like, "Hey, Aakash, why did it go down?" And you should be expected to, like, know because you've done your research, 'cause you're curious, and if not, it sort of builds that discipline that, like, we should know. You know? It's your responsibility to go and figure it out. It's my responsibility for, like, the areas that I own. So I think the metric review provides that kind of rigor, um, so that we can be accountable to understanding what is actually happening and why, so that we can be more in control of it.

    11. AGAakash Gupta1:12:57

        Mm-hmm.

    12. JZJulie Zhuo1:12:58

        I think the third thing that it does well is that it, uh, allows us to have, like, a, a shared common language which we can then take back to the rest of our teams.

    13. AGAakash Gupta1:13:08

        Yeah

    14. JZJulie Zhuo1:13:09

        ... about what, what, uh, so I do see, like, some of the best companies have that kind of rigor when it comes to understanding their analytics, understanding, uh, which in my mind is just rigor about, like, understanding operationally how things work-

    15. AGAakash Gupta1:13:24

        Mm-hmm

    16. JZJulie Zhuo1:13:24

        ... and what all the levers are.

    17. AGAakash Gupta1:13:26

        So today we covered, I think, all 360 degrees of being a great AI product leader. The reason you've become tech's favorite management guru is this book. So I'm curious, for people who are creators, who are, who wanna learn, like, how did you create a Wall Street Journal bestselling book on management?

    18. JZJulie Zhuo1:13:48

        I, it's hard to answer that because for me, I've always, I liked writing, and writing works for me in, on honesty because I always felt like I was a better writer, I was a clearer communicator in writing than in person.

    19. AGAakash Gupta1:14:03

        Hmm.

    20. JZJulie Zhuo1:14:03

        And it's taken me a lot of time to develop, but I used to get a lot of feedback that I rambled a lot.

    21. AGAakash Gupta1:14:08

        [laughs]

    22. JZJulie Zhuo1:14:09

        I didn't r- quite get to the point very quickly. I, it was hard for me to think on my feet. And so writing for me is, like, a way of actually getting my thoughts more clearly articulated. And so I started the practice when I was at Facebook. It, you know, I would go into a meeting and everybody would be talking, and I would, like, struggle to figure out what to say. And my manager actually gave me a lot of feedback that, like, "Hey, you, you need to, like, speak up more."

    23. AGAakash Gupta1:14:33

        Yeah.

    24. JZJulie Zhuo1:14:33

        "Like, we don't know what you're thinking." And so my first instinct is, okay, so maybe after the meeting I'll just write what I was thinking. That's how it started. And then I realized it was actually almost very therapeutic for me. [laughs] And I thought about my writing as letters to myself. And a lot of this book is essentially that. The, one of the primary reasons that I decided to even take on writing this book is I thought if I wrote a book about management, it's like writing this giant letter to myself [laughs] about what it is that I want to be, and how to, um, get the values that I have on paper in a way that will help me become a better manager. So I still am on that journey. I still am trying every day to be a better manager. I think there's so much for me to learn and grow. You know, I think as with a lot of things, sometimes the theory is easier to explain and the practice is still hard. [laughs]

    25. AGAakash Gupta1:15:22

        Yeah.

    26. JZJulie Zhuo1:15:22

        So much of life is like that, right? Like, we might understand intellectually that this is what it is. You know, like even very basic stuff like talk to your customers, but the actual reality of doing it day in and day out is a practice. And I think about management or leadership or any of-

    27. AGAakash Gupta1:15:37

        Yeah

    28. JZJulie Zhuo1:15:37

        ... the stuff that we do, design, like product, these are practices.

    29. AGAakash Gupta1:15:40

        Mm-hmm.

    30. JZJulie Zhuo1:15:40

        And the only way... There's no perfection. The only thing we can do is just show up, do the practice, and through the art of doing the practice, hopefully derive a lot of joy and meaning, and watch ourselves become better day every day than where we were yesterday.

15. 1:15:55 – 1:16:50

    Outro

    1. AGAakash Gupta1:15:55

       Wow. What a way to end the podcast. After literally years of reading your content, this conversation over-delivered. Thank you so, so much.

    2. JZJulie Zhuo1:16:05

       Thank you. This was really fun. And I mean, you, you do amazing work too, so I'm a big fan of your podcast and, um, your newsletter, and I love that you, you try so hard to, uh, to, to make this knowledge accessible and make all of us better.

    3. AGAakash Gupta1:16:20

       Appreciate that. All right, everybody, we'll see you later. I really hope you guys enjoyed that episode. It would mean a ton to me and the team if you could please subscribe on YouTube, follow on Apple and Spotify podcasts, and leave a rating and review. Those ratings and reviews really help grow the show and help other people discover the show, and they help fund the production so that we can do bigger and better productions. Can't wait to share the next episode with you. Until then, see you later.

Episode duration: 1:16:53