Nick Turley: How a hackathon shipped ChatGPT to 700M users

1. 0:00 – 4:52

   Introduction to Nick Turley

   1. LRLenny Rachitsky0:00

      You were a product leader at Dropbox, then Instacart. Now you're the PM of the most consequential product in history.

   2. NTNick Turley0:05

      I didn't know what I would do here because it was a research lab. My first task was to fix the blinds or something like that.

   3. LRLenny Rachitsky0:11

      When someone offers you a rocket ship, don't ask which seat.

   4. NTNick Turley0:13

      We set out to build a super assistant. It was supposed to be a hackathon codebase.

   5. LRLenny Rachitsky0:16

      What was it called before?

   6. NTNick Turley0:17

      It was going to be Chat with GPT-3.5. Because we really didn't think it was gonna be a successful product.

   7. LRLenny Rachitsky0:20

      And then Sam Altman's just like, "Hey, let me tweet about it."

   8. NTNick Turley0:23

      This is a pattern with AI. You won't know what to polish until after you ship. My dream is that we ship daily.

   9. LRLenny Rachitsky0:27

      By the time people hear this, they're gonna have their hands on GPT-5.

   10. NTNick Turley0:30

       About 10% of the world population uses it every week. With scale comes responsibility. It just feels a little more alive, a bit more human. This model has taste.

   11. LRLenny Rachitsky0:38

       Kevin Wheal, your CPO, said to ask you about this principle of, is it maximally accelerated?

   12. NTNick Turley0:43

       I just really want to jump to the punchline, why can't we do this now? I always felt like part of my role here is to set the pace and the resting heartbeat.

   13. LRLenny Rachitsky0:49

       Everyone's always wondering, is chat the future of all of this stuff?

   14. NTNick Turley0:52

       Chat was the simplest way to ship at the time. I'm baffled by how much it took off. I'm even more baffled by how many people have copied.

   15. LRLenny Rachitsky0:58

       ChatGPT is now driving more traffic to my newsletter than Twitter.

   16. NTNick Turley1:01

       That is a type of capability that has been incredibly retentive. I've been really excited about what we've been doing in search.

   17. LRLenny Rachitsky1:06

       Can you give us a peek into where this goes long term?

   18. NTNick Turley1:08

       ChatGPT feels a little bit like MS-DOS. We haven't built Windows yet, and it will be obvious once we do.

   19. LRLenny Rachitsky1:15

       Today, my guest is Nick Turley. Nick is head of ChatGPT at OpenAI. He joined the company three years ago when it was still primarily a research lab. He helped come up with the idea of ChatGPT and took it from zero to over 700 million weekly active users, billions in revenue, and arguably the most successful and impactful consumer software product in human history. Nick is incredible. He's been very much under the radar. This is the first major podcast interview that he has ever done, and you are in for a treat. We talk about all the things, including the just-launched GPT-5. A huge thank you to Kevin Wheal, Claire Vo, George O'Brien, Joanne Zhang, and Peter Deng for suggesting topics for this conversation. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. And if you become an annual subscriber of my newsletter, you get a year free of a bunch of incredible products, including Lovable, Replit, Bolt, n8n, Linear, Superhuman, Descript, Whisperflow, Gamma, Perplexity, Warp, Granola, Magic Patterns, Raycast, ChatPIR, Dee, and Mobbin. Check it out at lennysnewsletter.com and click bundle. With that, I bring you Nick Turley. This episode is brought to you by Orkes, the company behind open source Conductor, the orchestration platform powering modern enterprise apps and agentic workflows. Legacy automation tools can't keep pace. Siloed low-code platforms, outdated process management, and disconnected API tooling fall short in today's event-driven, AI-powered agentic landscape. Orkes changes this. With Orkes Conductor, you gain an agentic orchestration layer that seamlessly connects humans, AI agents, APIs, microservices and data pipelines in real time at enterprise scale. Visual and code-first development, built-in compliance, observability, and rock-solid reliability ensure workflows evolve dynamically with your needs. It's not just about automating tasks. It's orchestrating autonomous agents and complex workflows to deliver smarter outcomes faster. Whether modernizing legacy systems or scaling next-gen AI-driven apps, Orkes accelerates your journey from idea to production. Learn more and start building at orkes.io/lenny. That's O-R-K-E-S.io/lenny. This episode is brought to you by Vanta, and I am very excited to have Christina Cacioppo, CEO and co-founder of Vanta, joining me for this very short conversation.

   20. CCChristina Cacioppo3:31

       Great to be here. Big fan of the podcast and the newsletter.

   21. LRLenny Rachitsky3:34

       Vanta is a longtime sponsor of the show, but for some of our newer listeners, what does Vanta do and who is it for?

   22. CCChristina Cacioppo3:41

       Sure. So we started Vanta in 2018, focused on founders, helping them start to build out their security programs and get credit for all of that hard security work with compliance certifications like SOC 2 or ISO 27001. Today, we currently help over 9,000 companies, including some startup household names like Atlassian, Ramp, and LangChain, start and scale their security programs and ultimately build trust by automating compliance, centralizing GRC, and accelerating security reviews.

   23. LRLenny Rachitsky4:12

       That is awesome. I know from experience that these things take a lot of time and a lot of resources, and nobody wants to spend time doing this.

   24. CCChristina Cacioppo4:20

       That is very much our experience, both before the company and to some extent during it. But the idea is with automation, with AI, with software, we are helping customers build trust with prospects and customers in an efficient way. And, you know, our joke, we started this compliance company so you don't have to.

   25. LRLenny Rachitsky4:36

       We appreciate you for doing that. And you have a special discount for listeners. They can get a thousand dollars off Vanta at vanta.com/lenny. That's V-A-N-T-a.com/lenny for $1,000 off Vanta. Thanks for that, Christina.

   26. CCChristina Cacioppo4:50

       Thank you.

2. 4:52 – 9:13

   GPT-5 launch

   1. CCChristina Cacioppo4:53

   2. LRLenny Rachitsky4:55

      Nick, thank you so much for joining me and welcome to the podcast.

   3. NTNick Turley4:59

      Thanks for having me, Lenny.

   4. LRLenny Rachitsky5:00

      I already had a billion questions I wanted to ask you, and then you guys decided to launch GPT-5 the week that we're recording this, so now I have at least two billion questions for you.

   5. NTNick Turley5:08

      (laughs)

   6. LRLenny Rachitsky5:08

      I hope you have... I hope you have a lot of time. First of all, just congrats on the launch. It's coming tomorrow, the day after recording this. Just, uh, congrats. How are you feeling? I imagine this is an ungodly amount of work and stress. How are you doing?

   7. NTNick Turley5:22

      It's a busy week, but you know, we... we've been working on this for a while, so it also feels really good to get it out.

   8. LRLenny Rachitsky5:27

      So by the time people hear this, they're gonna have their hands on GPT-5 and the newest ChatGPT. What's the simplest way to just understand what this is, what it unlocks, what people can do with it? Give us kind of the- the pitch.

   9. NTNick Turley5:39

      I'm so excited about GPT-5. It, uh... I think for most people it's going to feel like a- a real step change. If you're the average ChatGPT user, and we have, you know, 700 million of them, um, this week, we, uh... You know, you've probably been on GPT-4o for, you know, a while. You probably don't even think about the model that powers the product. And GPT-5 is- is... it just feels categorically different. I'll talk about a lot of specifics, but, you know, at the end of the day, the vibes are good. At least we feel that way. We hope that users feel the same. Um, and increasingly, that is the thing that I think most people notice, right? Um, they don't look at the academic benchmarks, they don't look at evaluations. They try the model and, and see what it feels like. And just on that dimension alone, I'm so excited. I've been using it for a while. But it is also, you know, the smartest, um, most useful and, um, fastest frontier model, um, that we've ever launched. Uh, you know, on, on pure smarts, one way to look at that is academic benchmarks. On many of the standard ones, um, whether or not it's math or reasoning or, you know, just raw intelligence, this model is state-of-the-art. I'm especially excited about its performance on coding, um, whether or not that's SWE-Bench, which is a common benchmark, or actually front end coding is really, really good, um, as well. And, um, that's an area where I, I feel like there's, there's a true step change improvement in, in, in GPT-5. But really, no matter how you sort of measure the smarts, it's, it's, it's quite remarkable, and I think people are gonna feel the upgrade, especially if they weren't using 03 already. And, you know, the, the second thing, um, beyond smarts is i- it's just really useful. Coding is one access of utility, whether or not you have coding questions or you're vibe coding an app, um, but it's also a really good writer. I write for a living, uh, internally, externally. I just wrote a big blog post, um, that we published Monday. And, you know, this thing is, like such an incredible editor, um, and, and, you know, compared to some of the, the, the, the older models, it just got, it's got taste, which I think is really exciting. And, um, to me that's like something that is truly useful, um, in, in, in my day-to-day. And, um, there's other... a bunch of other areas, like it's, it's state-of-the-art on health, which is useful when you need it. But again, the, the sort of the thing you can't really express in use cases or even... yeah, in use cases or, or data, is sort of the vibe of the model, and it just feels a little more alive, a bit more human in a way that is kind of hard to articulate until you try it. So, feel good about that. And yeah, as mentioned, it's faster. Um, it, uh, it thinks too, just like 03 did, but you don't have to manually, you know, tell it to do that. It'll just dynamically decide to think when it needs to, um, and when it doesn't need to think, it just responds instantly, and that ends up feeling quite a bit faster than using 03 did. And then, you know, maybe the thing that's most exciting is that we're making it available for free, and that's like one of those things that I feel like we can uniquely do at OpenAI, because, you know, many companies, I think if they have a subscription model like us, they would gate it behind their paid plan, and for us, you know, if we can scale it, we will, and that just feels awesome. We did that with 40 as well. So, everyone's gonna be able to try GPT-5, uh, tomorrow hopefully.

   10. LRLenny Rachitsky8:46

       How long does something like this take? Like, I don't know if there's a simple answer to this, but just how long have you guys been working on GPT-5?

   11. NTNick Turley8:51

       We've been working on it for a while. Um, you know, you can kind of view GPT-5 as a culmination of a bunch of different efforts. You know, we had, uh, reasoning tech, we had a more c- classic post-screening, um, methodologies, um, and, uh, therefore it's really hard to put a beginning on it. But, but, you know, um, it, it really is kind of the end point of a bunch of different techniques that we've been

3. 9:13 – 13:52

   The vision for ChatGPT and AI assistants

   1. NTNick Turley9:13

      working on for a while.

   2. LRLenny Rachitsky9:13

      Can you give us a peek into the vision for where ChatGPT is going, GPT in general is going? Like, if you look at it on the surface, it's just, it's been kind of the same idea with a much smarter brain for a long time. I'm curious where this goes long term.

   3. NTNick Turley9:27

      So, to, to maybe back up a bit, um, now you think of ChatGPT as this kind of ubiquitous product, um, again, about 10% of the world population uses it every week.

   4. LRLenny Rachitsky9:37

      (laughs) Holy shit.

   5. NTNick Turley9:37

      Um, uh, you know, f- I think we have like five million business customers now. Um, it's like a, you know, an established category in its own right, but really when we started, we set out to build a super assistant. That's what we... that's how we talked about it at the time. In fact, the code base that we use is, is called SA Server. Um, (laughs) it was, it was supposed to be a hackathon code base, um, but, you know, things, things always turn out a little bit differently. And, uh, uh, so, so yeah, in some ways that is still the vision. The reason I don't talk about it more than I, you know, do, is because I think "assistant" is a bit limiting in terms of the mental model we're trying to create. You think of this like very personified human thing, maybe utilitarian, maybe, uh, you know... and, and frankly, uh, you know, having an assistant is not particularly relatable to most people unless they're like in Silicon Valley and they're a manager or something like that. So, it's imperfect, but like really what, you know, we envision is, is this entity that can help you with any task, whether or not that's at home or at work or at school, um, really any context, and, uh, it's an entity that you know, knows what you're trying to achieve. So, you know, unlike ChatGPT today, you, uh, uh, don't have to describe your problem in, in, in minute of detail 'cause it already understands your overarching goals and has context on your life, et cetera. Um, so, you know, that's one thing that we're really excited about. Um, the, the sort of inverse of giving it more inputs on your life is giving it more action space, so we're really excited to allow it to do, um, over time what a smart, empathetic human with a computer could do for you. Um, and I think, you know, the limit of the, the types of problems that you can solve for people once you give it access to, to tools like that, um, is, is very, very different than what you might be able to do in a chatbot today. So, you know, that's more outputs, and I often think, okay, you know, I'm a general intelligence. If I... what, what happened if I, you know, became Lenny's, uh, intern or something, um, and, you know, I wouldn't be particularly effective despite, you know, having both of those attributes that I just mentioned. Um, and it's because, you know, um, I think this idea of building a relationship with this technology is also incredibly important. So, that's maybe the third piece that I'm excited about, is building a product that can truly get to know you over time, and you saw us launch some of those things, you know, with, uh, improved memory earlier this year, and that's just the beginning of what we're hoping to do, so that it really feels like it's your AI. So, I don't know if super assistant is still the right, um, exact analogy, but I think people will just think of it as their AI, um, and I think we can put one in everyone's pocket and, uh, um, help them solve real problems, whether or not that's becoming healthy, whether or not that's, you know, um, starting a business, whether or not that's, you know, just having a second opinion on anything.Um, there's so many different problems that you can help with people in their, in their daily life, and that's what motivates me.

   6. LRLenny Rachitsky12:16

      So, an interesting, uh, kind of between the lines that I'm reading here is, the vision is for it to be an assistant for people, not to replace people. It feels like a really important, um, piece of the puzzle. Maybe just talk about that.

   7. NTNick Turley12:28

      AI's really scary to people, um, and I understand, you know, there's decades of movies on AI that have a certain mental model kind of baked in. And even if you just look at the technology today, once... Everyone I think has this moment where the AI does something that was really deeply personal to them and you're like... Kind of thought, "Hey, the, uh, AI can never do that." You know, for me, it was like, like weird music theory things where I was like, "Wow, this thing actually, like, understands music better than I do," and that's like something I'm passionate about. And, uh, you know, so, so it, it's naturally scary and I think the thing that's been really important to us, um, for a long time is to build something that feels like it, it's helpful to you but you're in the driver's seat, and that's in- even more important as this stuff becomes agentic, right? Um, like the feeling of being in control. And that can be small things like, you know, we built this way of sort of watching what the AI is doing when it's in agent mode. Um, and it's not that like you actually are gonna watch it the whole time, but it gives you a mental model and makes you feel in control in the same way that when you're in a Waymo you, you get that screen for those of you who've tried Waymo. You know, you can see the other cars. It's not like you're gonna actually watch, but it gives you the sense that you know how this thing works and what's happening. Or we, you know, we always check with you to confirm things. It's a little bit annoying, but it puts you in the driver's seat which is, which is, um, important. And for that reason, you know, we always view technology and the technology that we build as something that amplifies what you're capable of rather than replacing it, and, uh, that becomes important as the

4. 13:52 – 17:14

   The early days of ChatGPT

   1. NTNick Turley13:52

      tech gets more powerful.

   2. LRLenny Rachitsky13:53

      Okay. So, you mentioned the beginnings of ChatGPT. I was reading in a different interview. So, you joined OpenAI. ChatGPT was kind of just this internal experimental project that was basically a way to test GPT-3.5 and then Sam Altman's just like, "Hey, let me tweet about it, maybe see if people find this interesting," yada, yada, yada. It's the most, uh, successful consumer product in history, I think both in growth rate and users and revenue and just absurd. Can you give us a glimpse into that early period before it became something everyone's obsessed with?

   3. NTNick Turley14:23

      Yeah. Um, so we had decided that we wanted to do something consumer-facing, I think, you know, right around the time that GPT-4 finished training, and it was actually, um, mainly for a couple reasons. You know, we already had a product out there which is our developer product. That's actually what I came in, um, to help with initially. And, uh, you know, that has been amazing for the mission. In fact, it's grown up and now it's the OpenAI platform with, I don't know, four million developers, I think. But, you know, at the time, it was, you know, early stage and, and we were running into- running into some constraints with it because, um, we... There was two problems. One, you couldn't iterate very quickly because every time you would change the model, you'd break everyone's app. So, it was really hard to try things. And then the other thing, um, was that it was really hard to learn because e- the feedback we would get was like the feedback from the end user to the developer to us. So, it was very disintermediated and we were excited to make fast progress toward- towards AGI, and I just felt like we needed a more direct relationship with, with consumers. So, we were trying to figure out where to start and, you know, in classic OpenAI fashion, especially back then, um, we put together a hackathon of enthusiasts of just hacking on GPT-4 to kind of see what awesome stuff we could create and maybe ship to users. And, um, everyone's idea had- was, was some flavor of a super assistant. Like, they were more specific ideas. Like, we had a meeting bot that would call into, uh, meetings and, you know, the vision was, you know, maybe we would like help- h- h- it will help you run the meeting over time. We had a coding tool which, you know, um, full circle now, probably ahead of its time. Um, and, you know, the, the challenge was that we, we tested those things, but every time we tested these more bespoke ideas people wanted to use it for all this other stuff because it's just a very, very generically powerful technology. So, after a couple of months of prototyping, we took that same kind of crew of volunteers and it was truly a volunteer group, right? We had like someone from the super computing team who had built an iOS team- uh, iOS app before. We had, um, someone, you know, on the research team who had written some backend code in their life. They, they, they were all part of this initial ChatGPT team, and we decided to ship something open-ended because we just wanted a real use case distribution. Um, and this is a pattern with AI I think, where you know, you really have to ship to understand what is even possible and what people want, um, rather than being able to reason about that a priori. So, ChatGPT came together at the end because we just wanted the learnings as soon as we could and, um, we shipped it be- right before the holiday thinking we would sort of come back and get the data and then wind it down. And obviously that part turned out super differently because, um, um, people really liked the product as is. Um, so I remember sort of going through the motions of like, "Oh man, dashboard's broken. Oh wait, people are liking it. I'm sure it's just, you know, going viral and, and stuff is gonna die down," to like, "Oh wow, people are retaining but I don't understand why." Um, and then eventually we kind of like, you know, fell into product development mode but it was a little bit by accident.

5. 17:14 – 20:44

   The success and impact of ChatGPT

   1. NTNick Turley17:14

   2. LRLenny Rachitsky17:14

      Wow, I did not know that, uh, ChatGPT emerged out of a hackathon project. Definitely the most successful hackathon (laughs) project in history.

   3. NTNick Turley17:21

      I like to tell the story when we- when we talk about- uh, when we- when we do our, our hackathons because I really do want people to feel like they can ship their idea and it's certainly been true in the past and we'll continue to make it true.

   4. LRLenny Rachitsky17:31

      (laughs) Maybe you don't wanna share these things, but I wonder who that team was.

   5. NTNick Turley17:34

      The team's, um, largely still around. Some of the researchers working on GPT-5 actually, you know, they were always part of the, the ChatGPT team. Um, engineers are still around, um, designer- um, designers are still around. I'm still here I guess. (laughs) So yeah, uh, you've got the team, um, still running things but obviously we've grown up tremendously and we've had to because, you know, with scale comes responsibility and, um, you know, um, we're gonna hit a billion users soon and you, you kind of have to begin acting in a way that is appropriate, um, um, to that scale.

   6. LRLenny Rachitsky18:06

      Okay, so let me spend a little time there. So, I don't know if this is 100% true but I believe it is that ChatGPT is the fastest growing, most successful consumer product in history, also the most impactful on people's lives. It feels like it's just part of the ether of society now. It's just, my wife talks to it, like at every...Question I have, I go to it. Voice mode. My wife's just like, "Let me check with, check with ChatGPT." It's just such a part of our life now, and, and I think it's still early. So many people don't even know what the hell is going on. Just as someone leading this, how does just... Do you ever just take a moment to reflect and think about just like, "Holy shit?"

   7. NTNick Turley18:45

      I have to. It's quite humbling to get to run a product like that. And, um, I have to pinch myself very frequently. And I also have to sometimes sit back and let, you know, just think, which is really hard when things are moving so quickly, you know. And I love setting a fast pace, um, at, at the company, but in order to do that with confidence, I, you know, I need at least one day every week that I'm like entirely unplugged, and I'm just thinking about, you know, what, what to do and process the week, et cetera. Um, and, uh, the other thing is I've never ever worked on a product that is so empirical in its nature, where if you don't stop and watch and listen to what people are doing, you're gonna miss so much. Like, both on the utility and on the risks actually, because normally, you know, by the time you ship a product, you, you, you, uh, know what it's going to do. You don't know if people are gonna like it, that's al- always empirical, but you know what it can do. And with AI, because I think so much of it is emergent, you actually really need to stop and listen after you launch something, and then, you know, iterate on, on, on the things people are trying to do and on, and, and on, on the things that aren't, aren't quite working yet. So, for that reason alone, I think it's very important to take a break and, and just watch what's going on.

   8. LRLenny Rachitsky20:03

      Okay, so you take a day off every week. Not off (laughs) , okay, that's not the right way to put it. You take a day of, of thinking time, deep work every week?

   9. NTNick Turley20:09

      I, I need it. Yeah, yeah, yeah.

   10. LRLenny Rachitsky20:11

       Mm-hmm.

   11. NTNick Turley20:12

       And, and, um, and I need to hard unplug, you know, on a Saturday or something like that, obviously more often.

   12. LRLenny Rachitsky20:15

       Not on a Saturday. I like that. (laughs)

   13. NTNick Turley20:17

       (laughs) But, uh, uh, you know, it, it's just not possible otherwise. It's, this has been a giant marathon for three years now. Um, and-

   14. LRLenny Rachitsky20:25

       Like a sprint marathon.

   15. NTNick Turley20:26

       Sprint marathon, that's right, or interval training or something. I, I don't know how to exactly describe the OpenAI launch cadence, but, you know, uh, you gotta, you gotta, you know, set yourself up in a way that is sustainable. Even, even at, if even if this wasn't AI, and it didn't have the interesting attributes that I just mentioned, I think you, you would need to do that, but, um, especially with AI, it's important

6. 20:44 – 23:11

   Product development and iteration

   1. NTNick Turley20:44

      to go watch.

   2. LRLenny Rachitsky20:45

      So, on, along those lines, I talked to a bunch of people that work with you, that work at OpenAI. Uh, Joanne specifically said that, uh, urgency and pace are a big part of how you operate, that that's just, uh, something you find really important to create urgency within the team constantly, even when you are the fastest growing product in history, growing like crazy. Talk about just your philosophy on the importance of pace and urgency on teams.

   3. NTNick Turley21:08

      Well, it's nice of her to say that. Um, you know, I, I spent a lot of... two things. You know, with ChatGPT, I, you know, the... when we decided to do it, you know, we had been prototyping for so long, and I was just like, you know, "In 10 days we're gonna ship this thing," and, you know, we did. So, that was like maybe a moment in time thing where I just really wanted to make sure that we go learn something. Um, but for... you know, ever since then, I, I just spend so much time thinking about why ChatGPT became successful in the first place. And I think there was some element of just doing things, where, you know, there was many other companies that had, um, technology in the LLM space that just never got shipped. And I just felt like, you know, of all the things we could optimize for, learning as fast as possible is incredibly important. So, I just started rallying people around that. And that took different forms. Like, for a while when we were of that size, I just ran this like, you know, daily release sync, and it had everyone who was required to make a decision in it, and we would just talk about what to do and pivot from yesterday, et cetera. Obviously, at some point that doesn't scale, but I always felt like part of my role here obviously was like to think about, you know, the direction of the product, but also to just set the pace and the resting heartbeat, um, for our teams. And again, this is important anywhere, but it's especially important when, you know, the only way to find out what people like and, um, and, and what's valuable is to bring it into the external world. Um, so for that reason, I think it's become a superpower of OpenAI, and I'm glad that Joanne thinks I had some part in that, but it, it really has taken a village.

   4. LRLenny Rachitsky22:38

      I love this phrase, "The resting heart rate of your team."

   5. NTNick Turley22:41

      Yeah. (laughs)

   6. LRLenny Rachitsky22:41

      That's such a perfect metaphor of just the pace, uh, being equivalent to your resting heart rate.

   7. NTNick Turley22:46

      I actually learned that, uh, at, at Instacart when I, when I showed up there, because we were in the pandemic, and it was, um, kind of all hands on deck for a while. There was this like, you know, I think there was a companywide standup, um, because we disbanded all teams, we were just trying to keep the site up. And for me, you know, I, I had been used to kind of taking my sweet time and just thinking really hard about things, and that's important, but I really learned to hustle over there, and, um, uh, I think that's come in handy, um,

7. 23:11 – 26:17

   Maximally accelerated: the OpenAI approach

   1. NTNick Turley23:11

      at OpenAI.

   2. LRLenny Rachitsky23:11

      Okay, so along these same lines, I asked Kevin Weil, your CPO, what to ask you, and he said to ask you about, uh, this principle of, is it maximally accelerated?

   3. NTNick Turley23:21

      (laughs)

   4. LRLenny Rachitsky23:21

      Talk about that.

   5. NTNick Turley23:22

      Uh, that's funny. There, we have a Slack emoji apparently for this there, 'cause I used to say that. Now, now I try to like paraphrase. Um, sometimes I just really want to jump to the, you know, to the punchline of like, "Okay, why can't we do this now?" Or, "Why can't we do it tomorrow?" Um, and I think that, you know, it, it's a good way to cut through (laughs) a huge number of blockers, uh, with the team and just instill... especially if you come from a larger company. You know, at some point we started hiring people from, from, you know, larger tech companies. I think they're used to, you know, "Let's check, check in on this in a week," or, "Let's, you know, um, circle back next quarter to see if we can go on the, on, on, on the plan." And I just, kind of as a thought exercise, always like people asking like, "Okay, if like this was the most important thing, and you wanted to truly maximally accelerate it, what would you do?" That doesn't mean that you go do that, but it's really a good forcing function for understanding what's critical path versus what, you know, can happen later. And I've just always felt like, you know, execution is incredibly important. Like, these ideas are, they're everywhere. Everyone's talking about, you know, a personal AI, you know, you might have seen news on that, you know, and, and, and you know, I, I really think that execution is, is, is-... one of the most important things in this space, and this is the tool. So, um, it's funny that that became a meme. Um, it's like a little pink Slack emoji that people just put on, um, whatever they're trying to, to force the question.

   6. LRLenny Rachitsky24:45

      I was gonna ask if you So it's a little pink. W- is there something in there, like "Maximally accelerated?"

   7. NTNick Turley24:48

      It's a Comic Sans emoji that says, "Is this maximally accelerated?"

   8. LRLenny Rachitsky24:51

      (laughs) Okay. (laughs) And so, the kind of the culture there is when someone is working on something, the ques-

   9. NTNick Turley24:56

      Yeah.

   10. LRLenny Rachitsky24:56

       ... the push is, "Is this maximally accelerated? Is there a way we can do this faster? Is there anything we can unblock?"

   11. NTNick Turley25:02

       Yeah. And, uh, you know, we use that sparingly, right? Uh, because I- it has to need to be appropriate to the context. Um, there, there's some things where you don't want to accelerate, um, as, as, as quickly as possible, um, because you, you kind of want process. And we're very, very deliberate on that, where your process is a tool. And one of the areas where we have an immense amount of process is safety, uh, because, you know, A, the stakes are already really high, um, especially with these models, you know, GPT-5, which is a frontier in so many different ways. But B, you kind of, if you believe in the exponential, which I do, and, you know, most people who work on this stuff do, you have to play practice for a time where, you know, you really, really need the process for sure, sure, sure. And that's why I think it's been really important to separate out, you know, the product development velocity, which has to be super high, from, okay, for things like frontier models, there actually needs to be a, a, a rigorous process, where you red team, you work on the system card, you get external input, um, and then you put things out with, with confidence that it's gone through, you know, the right safeguards. So again, it's a nuanced concept, but I found it very, very useful when we need it, um, and for everything product development, you're, uh, dead on arrival, so it's, it's important to get stuff out.

   12. LRLenny Rachitsky26:11

       We got to open source this meme so that other teams can build on this, uh- (laughs)

   13. NTNick Turley26:15

       (laughs)

   14. LRLenny Rachitsky26:15

       ... approach.

   15. NTNick Turley26:16

       Absolutely.

8. 26:17 – 33:42

   Retention and user engagement

   1. NTNick Turley26:17

   2. LRLenny Rachitsky26:17

      So interestingly, with ChatGPT, and it's not a surprise, but not only is it the fastest growing, most successful consumer product ever, retention is also incredibly high. People have shared these stats that one month retention is something like 90%. Six month retention is something like 80%. First of all, are these numbers accurate? What can you share there?

   3. NTNick Turley26:39

      I'm obviously limited on what exactly I-

   4. LRLenny Rachitsky26:41

      Okay.

   5. NTNick Turley26:41

      ... can share. Uh-

   6. LRLenny Rachitsky26:42

      Okay.

   7. NTNick Turley26:42

      ... but it is true that our retention numbers are really exciting. And that is actually the thing we, we look at. You know, we, we don't care at all how much time you spend on the product. Um, you know, in fact, our incentive is just to solve your problem and, you know, if you really like the product, you'll subscribe. But, you know, there's no incentive to keep you in the product, um, for long. But we are obviously really, really happy if, you know, over the long run, you know, three-month period, et cetera, you're still using this thing. And for me, this was always the elephant in the room early on. It's like, hey, this may be a really cool product, but, you know, is this really the type of thing that you come back to? And it's been incredible to not just see strong retention numbers, but to see, you know, an, an, an improvement in retention over time, um, even as our cohorts become, you know, um, less of a early adopter and more, you know, the, the average person. So, um-

   8. LRLenny Rachitsky27:29

      Yeah. So tha- well, like, that note is something that I don't think people truly understand how rare this is.

   9. NTNick Turley27:34

      Yeah.

   10. LRLenny Rachitsky27:34

       When a product, the cohort of users comes, tries it out, and then retention over time goes down, and then it comes back up. People come back to it a few months later and use it more. And that's a- it's called a smiling curve or smile curve, and that's extremely rare.

   11. NTNick Turley27:48

       Yeah. Yeah, yeah. No, there's, there's some smiling going on, um, uh, not just on the team. And, um, the, you know, I feel like I have to acknowledge that some of it is, is not the product. I think people are actually just getting used to this technology in, like, a really interesting way, where I find, and this is why the product needs to evolve too, that this idea of delegating to an AI, it's not natural to most people. It's not like you're going through your life and figuring out, "What can I delegate?" Like, certain sphere of Silicon Valley does that, you know, because they're in, like, a self-optimization mode, and they're trying to delegate everything they can. But I think for most people in the world, it's actually quite unnatural, and you really have to learn, "Okay, what, what are my goals actually, and what could a- another intelligence help me with?" And I think that just takes time. And people do figure it out once they've had enough time with the product. But then, of course, there's been tons of things that we've done in the product too, whether or not it's making the core models better, whether or not it's, you know, new capabilities like search and personalization, um, and, and all that, uh, kind of stuff, or, you know, um, just standard growth work too, which we're starting to do. You know, th- that stuff matters too, of course.

   12. LRLenny Rachitsky28:49

       So, uh, you might have, you might be answering this question already, but let me just ask it directly. People may look at this and be like, "Okay, they're building this kind of layer on top of this godlike intelligence. Uh, of course it will grow incredibly fast and retention will be incredible." What the heck does ... What are you guys actually doing that sits on top of the model that makes it s- grow so fast and retain so much? Is there something that has worked incredibly well that has moved metrics significantly that you can share?

   13. NTNick Turley29:18

       I mean, one thing we've learned, um ... I'll answer that question in a minute, but, you know, the, the ... One thing we've learned with ChatGPT is that there really is no distinction between the model and the product. Like, the model is the product. Um, and therefore you need to iterate on it like a product. Um, by that I mean, is like, you know, if there's ... You obviously, you typically start by shipping something very open-ended, um, at least if you're OpenAI. Um, that blends ... That's kind of a playbook. Uh, but then you really have to look at, what are people trying to do? Okay, they're trying to write, they're trying to code, they're trying to get advice, they're trying to get recommendations. And you need to systematically improve on those use cases. And that is pretty similar to product development work. Obviously, the methodology is a bit different, but the discovery is, is, is the same. You got to talk to people. You got to do data science, and you got to try stuff and, and get feedback. Um, so that's, like, one chunk of work that we've been very consciously doing, um, is improving the model on the use cases people care about. And there's also such thing as vibes, as, uh, because I'm sure you, you know, and that's one of the things that I'm excited about in GPT-5, is that the, the vibes are really good. So that too is, you know, we have a, a model behavior team, and they really focus on, you know, what is the personality of this model and how, you know, how does it speak and talk. So there's that kind of work. I would say that's maybe, you know, a third of the, you know, retention, uh, improvements that we see or so just roughly. And then I think another third is, is, is what I would call...... sort of product research capabilities. Um, they, they're research-driven for sure. They have a research component, but they're really new product features or capabilities. And, like, search is one example of that where, you know, if you remember in the olden days, aka, like, you know, maybe 20 months ago or something, you would talk to ChatGPT and it'd be like, you know, "As of my knowledge cutoff," uh, or, you know, "I can't answer that because that happened too recently," or something like that. And, you- you know, that is the type of capability that has been incredibly retentive, um, and, um, for- for good reason. It just allows you to do more with the product. Personalization, like this idea of advanced memory where things can really get to know you over time is another example of a capability like that. You know, I think that's another good chunk. And then, you know, the third stuff is the stuff you would do in any product, and those things exist too, where, you know, um... Like, not having to log in was a huge hit, um, because it removed a ton of the friction, you know? Um, um, I think we, uh, we had this intuition from the beginning, but we n- never got to it 'cause we didn't have enough GPU or, you know (laughs) , uh, other- other constraint to really- really- really go do that. So, you know, there's the, like, kind of traditional product work too. So, I often think about it sort of as roughly a third, a third, a third, but really, you know, we're still learning and, um, we're planning to evolve the product a ton, which is why I'm sure there's gonna be new levers.

   14. LRLenny Rachitsky31:51

       Uh, you mentioned something that I wanna come back to real quick. You said that the... It was something like 10 days from hackathon to Sam tweeting about ChatGPT being live?

   15. NTNick Turley32:01

       Th- you know, the hackathon happened much earlier and we were prototyping for a long time, but at some point-

   16. LRLenny Rachitsky32:04

       Mm-hmm.

   17. NTNick Turley32:04

       ... we basically ran out of patience on, you know, on trying to, you know, build something more bespoke. And again, that was mostly because people always wanted to do all this other stuff, um, whenever we tested it. So, it was 10 days from- from when we decided we were gonna ship to when we shipped. Um, and, um, you know, the- the research we'd been testing for a long time, it was kind of an evolution of what we called instruction following, uh, which was the idea that, you know, instead of just completing the sentence, these models could actually follow your instructions. So, if you said, "Summarize this," it would actually do so. And the research had evolved from that into a chat format where we could do it multi-turn, so that research took way longer than 10 days and that kind of baking in the background. But the, you know, the productization of this thing, um, was very, very fast, um, and, you know, s-... Lots of things didn't make it in, like, remember we didn't have history, which of course was, like, the, you know, first user feedback we got. The model had a bunch of, you know, shortcomings, and it was so cool to be able to iterate on the model. Like, the thing I just talked about, like, treating the model as a product was not a thing before ChatGPT because we would ship it more like hardware, where, you know, there would, there'd be a- a release like GPT-3 and then we would start working on GPT-4, and these weird, giant, big spend R&D projects that would take a really long time and you kind of... The spec was whatever the spec was, and then you'd have to wait another year. And ChatGPT really broke that down because we were able to make- make, uh, iterative improvements to it, just like software. And really, my dream is that it would be amazing if we could just ship daily or even hourly, like in software land, because you could just fix stuff, et cetera. But there's, of course, all kinds of challenges in how you do that while, you know, keeping the personality intact, while, like, not regressing other capabilities, so it's an open- open research field to get there.

9. 33:42 – 36:31

   The future of chat interfaces

   1. NTNick Turley33:42

   2. LRLenny Rachitsky33:42

      That's such a good example of, is it maximally accelerated? Okay, we're gonna ship ChatGPT-

   3. NTNick Turley33:46

      (laughs) I think that's true.

   4. LRLenny Rachitsky33:46

      ... okay, 10 days.

   5. NTNick Turley33:47

      Totally agree. (laughs)

   6. LRLenny Rachitsky33:48

      Holy moly. We've been talking about ChatGPT. Clearly, it's a, kind of a chat interface. Everyone's always wondering, is chat the future of all of this stuff? Interestingly, Kevin Wheel had made this really profound point that has always stuck with me when he was on the podcast, that chat is a- actually a genius interface for building on a superintelligence, because it's how we interact with humans of all variety of intelligence. It scales from someone at the lower end to the- to a super- super smart person. And so it's really valuable as a way to kind of scale the spectrum. Uh, maybe just talk about that and just, is chat the long-term interface for ChatGPT? I guess it's called ChatGPT.

   7. NTNick Turley34:27

      I feel like we should either drop the chat or drop the G- GPT at some point because it is a mouthful, uh, we're stuck with the name, um, but yeah, no matter what we do with that, you know, it- it, uh, um... The product will evolve. I- I think that... I agree that there's something profound about, um, natural language. Like, it just really is the most natural w- form of communicating, um, to humans, and therefore, it feels important that you should be communicating with your software in natural language. I think that's different from chat though. You know, I think chat was the simplest way to put something, uh, to, you know, to ship at the time. I'm baffled by how much it took off, um, as- as a concept. I'm even more baffled by how many people have copied the paradigm rather than, you know, trying out a different way of interacting with AI. I'm still hoping that will happen. So, I think natural language is here to stay, but this idea that it has to be a turn-by-turn chat interaction, I think, um, is really limiting. Um, and this is one of the reasons I don't love the super assistant analogy even though we, uh, you know, used to always use it, is because if you think that way, then you kind of feel like you're talking to a person. But, you know... And GPT-5 is amazing at- at, um, making great front end applications, so I- I don't see a reason why you wouldn't have, you know, AIs that, you know, can- can render their own UI in some way. And you obviously want to make that re- predictable and feel good, but it feels limiting to me to think of the end-all be-all interface as- as a chatbot. It- it actually kind of feels dystopian almost, where I, like, I don't want to use all my software through the proxy of some interface. Like, I love being in Figma. I love being in, you know, uh, Google Docs. Those are all great products to me and they're not chatbots. So, um, yes on natural language, but no on chat is- is where I would describe my- my point of view. Um, and I'm just hoping in general that we see more sort of consumer innovation on how people interact with AI, because there's so many possibilities and you've just got to try stuff. That's why chat stuck is like, you know, we just did it and peop- peep- people liked it, so I'm- I'm hoping that, um, we see- we see more there and we'll- we'll try to do our part.

10. 36:31 – 38:52

    The evolution of ChatGPT

    1. LRLenny Rachitsky36:31

       So, you mentioned that you kind of, like, got stuck with this name ChatGPT. Uh, maybe this is part of the answer, but I'm curious just, are there any accidental decisions you guys made early on that have stuck and have essentially become history changing?

    2. NTNick Turley36:45

       There- there- there's so many, and it's- it's funny 'cause you...... had, like, no time to think about them and then they end up being super consequential. You know, in the name of what, you know? From chat with GPT-3.5 to ChatGPT the night before, slightly better, but still really bad.

    3. LRLenny Rachitsky36:58

       What was it called before?

    4. NTNick Turley36:59

       It was gonna be Chat with GPT-3.5.

    5. LRLenny Rachitsky37:01

       (laughs) Breaker.

    6. NTNick Turley37:01

       We really didn't think it was going to be a successful product. Like, we were trying to actually be as nerdy as we could about it, because that's really what it was. It was like, you know, a research demo, not, not a product. So, we didn't think that was bad, but, um... You know, I think that in the original release, you know, making it free was a big deal. I, I don't think we appreciate that, because the, uh, GPT-3.5 model was in our API for, you know, at least six months prior to that. I think anyone could have built something like this. Might not have been quite as good on the modeling side, but I think it would have taken off. So, making it free and putting a nice UI on it, very consequential in the way that you take for granted now. And this is why I think that A, distribution, and B, the, you know, the interface are continued- continuously important, even in, in 2025. The paid business, which now is, it's, it's, it's a, it's a giant business, um, both in, you know, the consumer space and in the enterprise space. The birth of that was just to turn away demand, originally. Like, it was not like, you know, we brainstormed, "Oh, what is the best monetization model for AI?" It was really, "What is, what monetization model ha- Or what, what mechanism would allow us to turn away people who are, like, you know, less serious than the people who are really trying to use it?" And subscriptions just happened to have that property and it, you know, grew into a large business. You know, I think shipping really kind of funky capabilities before they were polished is another thing, where, you know, that feels like a tactical decision, but it became a playbook because we would learn so much. Like, remember when we shipped Code Interpreter? We learned so much after, um, we shipped it. You know, now it's known as, I think, data analysis in ChatGPT or something like that. Just because we actually got real world use cases back that we could then optimize. So, I think there's been, like, a lot of decisions over, over time that, um, proved pretty consequential, but, you know, we made them very, very quickly as, as, as we have to, so,

11. 38:52 – 42:10

    Subscription model and pricing strategies

    1. NTNick Turley38:52

       um...

    2. LRLenny Rachitsky38:52

       The $20 a month feels like an important part of this. Feels like everybody's-

    3. NTNick Turley38:56

       Yeah. (laughs)

    4. LRLenny Rachitsky38:56

       ... just doing that now, and...

    5. NTNick Turley38:57

       Oh, that one actually-

    6. LRLenny Rachitsky38:58

       ... I think that's right.

    7. NTNick Turley38:58

       ... I remember I had this, like, kind of panic attack because we, we really needed to launch subscriptions because at the time we, we were, we were taking the product down every time. Um, it was like... I don't know if you remember, we had this, like, fail whale that was like a little-

    8. LRLenny Rachitsky39:08

       Mm-hmm.

    9. NTNick Turley39:09

       ... GPT-3.5 generated poem on it. So, we were like, we had to get this out and I remember calling up, um, someone I greatly respect who's like, you know, incredible at pricing. Um, and, and you know, it was like, "What should I do?" And like, we talked a bunch, and I just ran out of time to, to incorporate most of that feedback. So, what I did do is ship a Google Form to Discord with like, I think the four questions you're supposed to ask on how to price something.

    10. LRLenny Rachitsky39:31

        The Van, Van Wonders, so say?

    11. NTNick Turley39:32

        Yeah, exactly. It literally had those four questions and I remember distinctly, A, you know, I got a price back, um, and that's kind of how we got to $20. But B, uh, (laughs) the next morning there was like a press article on like, "You won't believe the, like, four genius questions the ChatGPT team asked to price their..." It was like, if only you knew. So, there's like, something about building in this extreme public where people interpret so much more intentionality into what you're doing than, you know, might have actually existed at the time. But we got with the 20, we were debating, you know, something slightly higher at the time. I often wonder what would have happened, because so many other companies ended up to- copying the $20 price point. So, I'm like, "Did we, like, erase a bunch of market cap by pricing it this way?" But ultimately, I don't care because the, the more accessible we can make this stuff, the better, and I think this is the price point that, in western countries, has been, um, reasonable to a lot of people in terms of the value that they get back. And, um, most importantly, we were able to push things down to the free tier, um, semi-regularly, and we always do that when we can, um, including with GPT-5.

    12. LRLenny Rachitsky40:35

        So, the survey, just to give your official name, the Van Westenraad Survey, uh, is how you guys ended up pricing ChatGPT?

    13. NTNick Turley40:42

        It was the top Google result, this was before ChatGPT had realtime information, otherwise it could have maybe priced itself. But, uh, it was Discord plus Google Form plus a blog post on that methodology that, um, got us there, so, so...

    14. LRLenny Rachitsky40:54

        That is incredible. What a fun story.

    15. NTNick Turley40:56

        (laughs)

    16. LRLenny Rachitsky40:56

        This is the survey that Rahul Vohra at Superhuman kind of popularized in his first run article several months ago.

    17. NTNick Turley41:00

        Yeah, yeah, yeah, that's right. That's right. Um, yeah, definitely don't bring me on here as a pricing expert.

    18. LRLenny Rachitsky41:05

        (laughs)

    19. NTNick Turley41:05

        I think you, you, you have got better people for that. (laughs)

    20. LRLenny Rachitsky41:08

        Whether it was right or wrong, it is now the fastest growing (laughs) , um, insane revenue generating business in the world. So-

    21. NTNick Turley41:14

        (laughs)

    22. LRLenny Rachitsky41:14

        ... uh, I wouldn't feel too bad.

    23. NTNick Turley41:16

        No, it worked out, yeah.

    24. LRLenny Rachitsky41:17

        (laughs) It worked out. Uh, and by the way, I'm on the $200, uh, a month tier, so there's clearly, uh, room-

    25. NTNick Turley41:22

        Thank you.

    26. LRLenny Rachitsky41:23

        ... and op-

    27. NTNick Turley41:23

        Thank you.

    28. LRLenny Rachitsky41:25

        ... opportunity.

    29. NTNick Turley41:25

        You know, that, that, the story of that one is, is interesting too, because, you know, originally the purpose of the Plus plan was to be able to ship, first up time, and then be able to ship capabilities that we couldn't scale to everyone. And at some point, we got so many people in the Plus tier that it just lost that property. Um, so the rea- main reason we came up with the $200 tier is just, we had so much incredible research that's actually really, really powerful, um, like, you know, O3 Pro or, you know, tomorrow GPT-5 Pro, um, and just having a vehicle of shipping that to people who really, really care is exciting, even though it kind of violates the standard way a SaaS page should look. Um, it's like a little jarring to see the, see the 10X jump. So, um, um, thank you for being a subscriber on that, and thank you everyone else who's watching who's subscribed to any tier. Um, it's, it's great.

12. 42:10 – 44:10

    Enterprise adoption and challenges

    1. NTNick Turley42:10

       (laughs)

    2. LRLenny Rachitsky42:10

       I'm just gonna throw a, a fishing line into this pond of, are there any other stories like this? You shared this incredible story of Chat with GPT-3.5 being the original name, how you came up with pricing. Is there anything else?

    3. NTNick Turley42:22

       Enterprise is an interesting one too, because we've been s- seeing so much, um, incredible adoption in the enterprise. And it's sort of objectively crazy to try to take on building a developer business and a consumer business and a devel- and, and, and an enterprise business, and, and, and, all at once. But, you know, the story there is, you know, in like month one or, or two, I, it was like very clear that most of the usage was like kind of work-y usage. Actually much more than today where you've got so many like kind of consumers, uh, on the product and, you know, it's kinda-... sort of transcended into pop culture. But at the time, it was like, you know, writing, coding, analysis, that kind of stuff. And, uh, we were pretty quickly in, you know, organically in, like, 98% of Fortune 500 companies, in a way that I had seen maybe at Dropbox back when I, you know, that was my two jobs ago, where we had kind of had a similar story. And since then, there's been more PLG companies. But the real reason we did enterprise, I remember we were debating, "Should we do enterprise or should we launch an iOS app?" Because that's how small the team was. Eh, the reason we did, yeah, did this, we were starting to get banned in companies because they all, you know, felt, you know, rightfully or wrongfully that, you know, the, the privacy and deployment story, et cetera, wasn't there. So I was just like, "Man, we have to do something. We're gonna miss out on a generational opportunity to build a, a, a, a, a work product." And, you know, we've literally defined AGI as, you know, outperforming most humans at economically valuable work or... I probably butchered that, but you know, I think, um, I think that's the way we put it. And, um, um, so it, I feel like we had to be present there. And it was a fairly, you know, quick decision at the time, but it's grown into an immense, uh, business. We just hit five million, um, business subscribers, up from three, I think, um, a month or two ago. So, it is kind of the spinoff that's taking a life of its own that I'm really, really excited about, um, um, for,

13. 44:10 – 52:13

    Balancing multiple product lines

    1. NTNick Turley44:10

       for obvious reasons.

    2. LRLenny Rachitsky44:11

       That is a lot to be handling. Uh, the platform, essentially the API, the consumer product, the fastest growing, most successful product in history, and also the B2B side, which is, uh, clearly a massive business. Uh, do you have any kind of heuristics for how to make these trade-offs, do all this at once and stay sane and be successful?

    3. NTNick Turley44:30

       Uh, it's a good question, and you know, um, first off, I don't run the developer stuff anymore. We found someone way more competent, uh, with EV to do that, um, and he's amazing. So, I still look after the, you know, various forms of, of, of, of chat, but you know, I luckily don't have to make, make that trade-off. OpenAI does, and I mean, I can get into that too, but um, it keeps me a little bit more sane. I will say that there, you kind of have to practice in two different ways when you're, when you're building on this AI stuff. One is sort of working backwards from the model capabilities, and that is much more art than science, where I think you really need to look at what tech do we have available, and what is like the most awesome way to prod- productize it? And if you applied to, some sort of PM framework to that, I think you would do something horribly wrong because if you have tech that's, you know... Um, for example, GPT-5 is, is really, really good at front-end coding now. Like, I think we, that means you've got to reprioritize, and you've got to like actually bring that capability to life. Maybe that's, you know, uh, making, making ChatGPT better at, at live coding and rendering, you know, applications. Maybe that's more like, you know, leveraging the taste of the model to make the, the UI more expressive. There's like a number of things we could do, right? But you kind of have to re-plan and reprioritize, and that, you know, is more important than any particular audience segmentation. It's really just looking at, you know, what is the magic thing we have and how do you make it shine? Voice is a similar thing. It wasn't like our customers need voice, they're begging for it or something like that. It's like, well, we figure out a way how to, you know, to make these things anything in and anything out, what is like a creative, awesome way to productize that? And then we can see what, what people do. So I think that's one chunk of it. But then the other chunk of it really is more like classic product management, where you need to listen to customers. And then when your customers are really different, that can be confusing. Because, uh, you know, ChatGPT is a very general-purpose product, we see, when you look at end-users, there's actually an immense amount of overlap in terms of what they want. Like primitives like projects or, um, you know, history s- um, search or, um, sharing, um, and co- collaboration. All, all those kind of things, they are actually very, very present, whether or not you're talking to people at work or you're talking to people at home, at school. They're slightly different mechanics sometimes, um, but they're, they're largely similar investments that I think we can get a lot of mileage out of. And then there's enterprise-specific work that we just have to do. Like you've got to do HIPAA, you've got to do SOC 2, you gotta do all those things if you wanna be a serious player. And those are just non-negotiable. So, it's complex, as you correctly identified, um, but it's kind of the, the curse of working on a very open-ended and powerful, um, technology. Uh, one analogy that, that, um, someone at OpenAI who I really respect sometimes uses is like, we're kind of like Disney, where, uh, Disney has this like, one kind of creative IP, um, which is like their, their, their content, then they have cruises and they have, um, uh, you know, uh, theme parks, they have comics, they have all these different things. And I think we have amazing models, but there's all these different ways that you could productize them and we kind of just have to maximize the impact in, um, in all these different ways.

    4. LRLenny Rachitsky47:38

       As you were talking, I was thinking about how usually, uh, horizontal platforms that are just so general and can do so much take a long time to take off because people don't know what to do with them, they're not amazing at anything. And this is an amazing counter-example where it took off immediately and everyone figured it out, and then over time they figured it out more and more.

    5. NTNick Turley47:54

       But I, I think the reason why is because it just went live. Talk about another consequential decision, actually. You know, we were debating waitlist, no waitlist, because we just really knew we couldn't scale the engineering systems and, you know, the fact that there was no waitlist, which no OpenAI release had worked like that before. You know, it ended up being consequential because like you were able to watch what everyone else was doing, live. So I think when you launch these things all at once for everyone, there really is a special moment where you can see what other people are doing and learn from that. And a lot of that is actually out of product. There's these crazy TikTok posts that go viral and they have like 2,000 use cases in the comments. And I go through those in detail because it's, it's not like I knew about those use cases either. Like they're, they're very, very emergent and I just go through the comments and, you know, process because there's so much to learn. And for that reason, I think we get to escape the empty box problem a little bit because, you know, the, so much learning is happening out of product, um, as people are watching each other either in IRL or, uh, or online.

    6. LRLenny Rachitsky48:55

       That is so interesting 'cause you, you think about Airtable, you think about Notion, all these companies, they took like years to just build and craft and think and go deep on what it could be.

    7. NTNick Turley49:04

       It's like, but if I compare Airtable which like, you know, they, they had to do templates, they had to do, um, like all these kind of things of taking the horizontal product and making it like, use case driven.I mean, compared to the, like the Instant Pot, um, which, you know, there's recipes being shared on- everywhere online. Like, there's a kind of... There's a whole ecosystem around it. I think we are really lucky with ChatGPT that that happened, where there's just users sharing use cases with other users everywhere. Um, and, and therefore, I, I think, you know, we, we, we, we, we kind of got very lucky by, by, by, you know, jumping, jumping ahead, um, on, on that journey, right?

    8. LRLenny Rachitsky49:40

       And it feels like a quarter there, Sam had a big following and everyone would pay attention to something he launched. So, that's a really interesting new strategy for launching a horizontal product with a huge distribution channel. Just launch it and see what, see what comes up.

    9. NTNick Turley49:51

       Yeah. And of course, I'm, I'm actually really excited to take some of that into the product. Like, I think there's, there's... uh, we shouldn't, you, you know, rest on the fact that there's so much out-of-product discovery happening. Like, I actually think for the average consumer, it would be amazing if the product did a little bit more work on really exposing to you what is possible. I, I still feel like ChatGPT feels a little bit like MS-DOS. Uh, like, we haven't built Windows yet, and it will be obvious once we do, but, you know, there, there, there's something that feels a little bit like... Like, imagine if MS-DOS had gone viral and you were just trying to, like, hack, like, little conversation starters onto it. That might have missed sort of the big picture in terms of how to really communicate performances and value to people. And so, I, I think there's actually a ton more product work to do in addition to, um, you know, just seeing use cases spread.

    10. LRLenny Rachitsky50:32

        Are you able to share just what you think that might look like, this Windows version of ChatGPT, or is that-

    11. NTNick Turley50:37

        I'll let you know when we figure it out.

    12. LRLenny Rachitsky50:39

        Okay. (laughs)

    13. NTNick Turley50:39

        Um, we're hiring. Um, you know, I, I think there's so many interesting product problems here.

    14. LRLenny Rachitsky50:42

        Okay, got it. Uh, by the way, I also love that TikTok was like (laughs) your feedback, uh, channel.

    15. NTNick Turley50:48

        Those comment threads are... They're, they're just so wild. And, and, and also the love that people have for it, like, the excitement with which they're sharing their product, I, I, I, I, I kind of feel like it's, it, it's special that people are so excited about to share what they're doing with your product. And, um, I don't take that for granted either.

    16. LRLenny Rachitsky51:06

        This episode is brought to you by Posthog, the product platform your engineers actually want to use. Posthog has all the tools that founders, developers, and product teams need, like product analytics, web analytics, session replays, heat maps, experimentation, surveys, LLM observability, error tracking, and more. Everything Posthog offers comes with a generous free tier that resets every month. More than 90% of customers use Posthog for free. You are gonna love working with a team this transparent and technical. You'll see engineers landing pull requests for your issues, and their support team provides code level assistance when things get tricky. Posthog lets you have all your data in one place. Beyond analytics events, their data warehouse enables you to sync data from your PostgreSQL database, Stripe, HubSpot, S3, and many more sources. Finally, their new AI product analyst, MaxAI, helps you get further faster. Get help building complex queries and setting up your account with an expert who's always standing by. Sign up today for free at posthog.com/lenny, and make sure to tell them Lenny sent you. That's PostH-O-G.com/lenny.

14. 52:13 – 1:02:15

    Emergent use cases and user feedback

    1. LRLenny Rachitsky52:13

       How do you find emergent use cases these days? I imagine the volume is very high. Do you have kind of a trick for figuring out, oh, here's a new thing we should really think about?

    2. NTNick Turley52:22

       Before I built the product team, I actually built the data science team, um, because I, I was getting frustrated. I was talking to as many users as I could. In my calendar, you know, the weeks after ChatGPT was just 15-minute user interview the whole week through. And it was... Usually, I stop doing interviews when I, like, can predict what the next person's gonna say. That's how I know I've talked to enough users, but it just wasn't happening. Like, I just kept getting new stuff. So, data is one way out, where I think you... You know, we, we have conversation classifiers that without, you know, us having to look at the conversations, allow us to kind of figure out what are people talking about, what use cases are taking off, et cetera. And I think that's very, very helpful. The qualitative stuff is important for empathy. Even though you're never gonna get a wrap on, like, all the use cases people have, um, I still spend a huge amount of my time do- doing that. And then, yeah, things like those TikToks, um, collections of threads. I think they're really, really useful. And, uh, uh, it's just fun to watch people talk to each other about the various use cases that they have.

    3. LRLenny Rachitsky53:22

       Is there kind of a new emergent use case that you're excited about? Or is there, like, a really unusual use of ChatGPT that you think about that would be fun to share?

    4. NTNick Turley53:30

       I mentioned this earlier, but I had always conceptualized ChatGPT as a work-y product, whether or not you're at home or you're at work, like, you... I feel like, you know, helping, getting help with your taxes is very similar to, you know, um, the types of things you do at work or, you know, planning a trip is actually very similar to, you know, planning an event for work. So, I always felt like, okay, this thing is going to kind of be a productivity tool. And I think something has happened over the last, you know, few months where that has begun to change, and I really do think the fact that you have consumers turning to this thing for day-to-day advice, helping them, like, have better relationships, like, seeing... Like, you know, people talk about how this thing, like, you know, saved their marriage is, like, really exciting to me because, like, they, you know, proc- use it to process their own emotions, get feedback on their communication style. They just have a buddy to talk to about, like, really difficult things. And that comes with a ton of responsibility and work that we have to do to make those things like life advice great. But it also is really, really important to me because you can't run awa- away from those use cases. You have to run towards them and make them awesome. And, um, that's part of what we're trying to do. So, that emergent behavior is really, really cool. And more broadly, I am so excited about education. I'm so excited about, um, health. Like, I, I think it would really be a waste if we didn't take the opportunity of using ChatGPT to really, really help people, and I think we've just begun to scratch the surface, um, on, on that. So, um, there's many aspirational use cases that I want to make happen.

    5. LRLenny Rachitsky55:05

       Along those lines, an interesting use case I've recently had, I feel like it's gonna be really helpful for, uh, couples that are disagreeing about something when they need, like, a third opinion. I just had this recently where my wife's like-

    6. NTNick Turley55:15

       (laughs)

    7. LRLenny Rachitsky55:16

       ... "You can't heat a whole thing that you're gonna only eat part of in the microwave and then put it back in the fridge." It's like, "What's the problem? I'll heat it up. I'll put it back in the fridge." And she's like, "No, that's really dangerous." I'm like, "Let's ask ChatGPT." And the fact that she-... so trusts ChatGPT now and relies on it throughout the day. It's such a valuable third independent party that we can go to.

    8. NTNick Turley55:35

       Yeah. Yeah. Totally. And, and, and, you know, there, a lot of those micro interactions talk about, like, interesting product work, right? Those are micro interactions that are important, right? Did it, like, definitively weigh in or did it help you guys think through, you know, that, that, that disagreement and, you know, um, solve it on your own? I think those details actually matter a lot, and it's where we're spending a bunch of time.

    9. LRLenny Rachitsky55:54

       Along those lines, there was this whole launch of the very sycophantic version of ChatGPT where it was just, "You are the best person in the world. Everything you tell me is amazingly correct." Uh, are you able to tell us just what happened there?

    10. NTNick Turley56:08

        Yeah. We- You know, we, we have all kinds of collateral, um, online because we really felt like we should over communicate on how we discovered it, what we did about it, et cetera. So I encourage people to check that out. Um, we have a whole retro, um, o- on, on that model release. But basically what happened is that we pushed out an update that, you know, made the model more likely to, you know, tell you things that sound good in the moment and, um, you- like, "You're, you're totally right." You know? "You, you, you know, should break up with your boyfriend," or something like that. And, you know, that's just really dangerous, and it's in, in, we, we took it more seriously than you even might expect because again, at current technology levels, you can kind of laugh about it maybe as like, "Ah, this thing's always complimenting me. I thought it was just me. I saw all those comments online." But, you know, it actually is, is really important to make sure that, um, these models are optimized for the right things. And we have an immense, I think, luxury to have a mission that affords us to really help people, a business model that does not incentivize, you know, maximizing engagement, um, um, you know, um, or time spent i- in the product, right? So it's really important to us that you feel like this product is helping you with your goals, whether or not that's your current goals or even your long-term goals. And oftentimes, you know, uh, being extremely complimentary with the user isn't actually in, in service of that. So we instilled new measurement techniques like, you know, whenever we put these models in contact with reality and we, you know, learn about a problem, we actually go back and make sure we have good metrics for this stuff. So, you know, we measure sycophancy now with every release to make sure we don't regress and can actually improve on that metric. Um, GPT-5 is an improvement, which is really exciting for me, but we have more work from there. Um, and more broadly, it caused us to articulate our point of view, actually spent a bunch of time on a blog post that we just published on Monday on what we're optimizing ChatGPT for, and it really is for your, you know, to, to, to help you thrive, um, um, and achieve your goals, not to, you know, keep you in the product. And, um, um, so there was a bunch of good outcomes from, from, from that incident. It's a good example of how contact with reality is not just important for the use cases, but also for learning what to avoid, because you would have never discovered this issue purely in a lab unless you actually heard it from users.

    11. LRLenny Rachitsky58:26

        I am excited to read that blog post, and I was gonna ask you this, just like-

    12. NTNick Turley58:29

        Yeah, I love your feedback on it. Yeah.

    13. LRLenny Rachitsky58:30

        And yeah, I guess, is there anything more there just like how you... 'cause this tension is so difficult, like, you know, helping people feel supported, but not just letting them believe everything they want to believe. Is there anything more you can share there, just trying to find that middle ground?

    14. NTNick Turley58:43

        Incentives are important. Uh, there's a famous saying, "Show me the incentive and I'll show you the outcome."

    15. LRLenny Rachitsky58:48

        Charlie Munger, maybe?

    16. NTNick Turley58:49

        Um, yeah. (laughs) Uh, I think that's where it came from, right?

    17. LRLenny Rachitsky58:52

        Yeah.

    18. NTNick Turley58:52

        And I think that's very, very important. So I would take a good look at, you know, our mission, our business model, the type of product we're trying to build. And, you know, I, I, I really think that, you know, ChatGPT is a very special product because it, I think, in vast majority of cases, makes you, you leave it feeling better, not worse, and you f- like, you know, feeling like you're achieving something you're trying to, trying to do. And so I think that those incentives really matter because it helps you reason about, okay, when there isn't behavior in the wild that's not good, was that a bug or was that by design? You know, and with, with sycophancy, I can very much say that to us that's a bug. And then on, you know, the, the forward-looking work, there's so many, you know, kind of challenging scenarios to get right, and you could easily run away from, from, from, from these use cases, like, you know, the like, you know, you and your wife go and do this thing, um, for, you know, input on a relationship, um, um, a question or like a, a dispute. You could very easily run away if you were totally risk-avoidant and say, "Sorry, I can't help you with that." I think that's what most tech companies do when they hit a certain scale. They run away from these use cases, and I think it's a lost opportunity to help people. So we want to run towards these use cases by making the model behavior really, really great. Um, that can mean connecting you with external resources when you're struggling. That can mean not directly answering your question, but instead giving you a helpful framework, you know, in the case of like, "Should I break up with my boyfriend?" ChatGPT should probably not answer that question for you, but it should help you think through that question in the way that a thoughtful companion would. So I think it's really important to do the work be- because I think the upside is immense.

    19. LRLenny Rachitsky1:00:37

        That is a really profound point you're making there, that if most companies, if their, if their users want to ask them something risky like getting medical advice, or, "Should I break up with my partner?" or, "What should I do with this big problem I have?"

    20. NTNick Turley1:00:51

        I feel like we would have immense regret if you had a model that was state of the art on HealthBench, which is, you know, a, um, um, GPT-5 is state of the art on, you know, a bunch of these medical benchmarks, right, and you didn't use that to help people. Like, if you just disabled that use case because you wanted to, like, avoid all possible downside. I, I think the duty is to make it awesome, um, and to do the work, talk to experts, figure out how good it really is, where it breaks down, communicate that, and, um, you know, I, I think this, this technology is too important and has too much potential positive impact on people to, to run away from, from, um, these high-stakes uses.

    21. LRLenny Rachitsky1:01:25

        Mm. And, uh, fast forward to today, it's saving lives regularly. It's probably saving relationships regularly, such a consequential decision, which I imagine was made early on.

    22. NTNick Turley1:01:36

        You know, we're, we're just at the beginning of, of watching how this peop- th- this, this, this stuff can transform people. Um, it's incredibly democratizing. If you compare, you know, the rollout of this with the rollout of the personal computer, right? You know, computers were, like, so scarce when they first came out, and this stuff is ubiquitous in a way, where I, uh, you, you have access to a second opinion on, on medical stuff. You have access to, you know, um, um, a, a relationship buddy. You have access to a personal tutor on literally any topic that, uh, makes you curious. Uh, it's really, really special that, that, that we get to do that, so, um, um, unique point in t- in

15. 1:02:15 – 1:05:07

    OpenAI’s unique product development approach

    1. NTNick Turley1:02:15

       history.

    2. LRLenny Rachitsky1:02:15

       Let me zoom out a bit and talk about OpenAI, and just product in general. So you've worked at traditional, let's say traditional product companies, Dropbox, Instacart. Now you're at OpenAI. What's, what's maybe the most counterintuitive lesson you've learned about building products from your time at OpenAI?

    3. NTNick Turley1:02:33

       Each time, like, I always tried to pick the most different, maximally different job whenever I made a job change, you know? So, you know, after Dropbox I was, like, craving a real world product, because it was just so different than working on SaaS, et cetera. Uh, and after Instacart, I was craving on, working on something that intellectually was interesting, um, and had, you know, this kind of, like, sort of invoked the nerd in me. And, you know, so I've always looked for things that are really different. And then, once I showed up at these places, I tried to understand what makes that place successful, like what is truly the thing that they cracked, and how we can lean in that- into that even more. And I think I spent a lot of time thinking about this with OpenAI, um, especially after ChatGPT. Before that, you know, it was kind of a moot point, because we didn't really have much revenue, or products, or anything that, you know, like that. And there's a f- you know, a few things, um, that, that, that, that come to mind that have driven many decisions. Um, one is the empiricists, and we talked about that a bit, the fact that you can only find out by shipping, um, which is why I've maximally leaned into that, and that's, you know, huge part of why, uh, we ship so much. Um, one of them is that, you know, amazing ideas come from anywhere. Um, the re- the thing about running a research lab is, you really don't tell people what to research. Um, that's not what you do, and we inherited that culture, even as we become a research and product company. So just letting people do things who have amazing ideas, rather than sort of being the, the gatekeeper or prioritizer of everything, or something like that, um, has t- been proven, you know, immensely valuable to us, and that's where much of the innovation comes from, is empowered smart people in any function, really. Um, so that was a good inheritance from what I think made OpenAI successful, and makes this s-successful. The interdisciplinari-ness of really making sure that you put research and engineering and design and product together, rather than treating them as silos, I think that's the thing that has made us successful, and that you see come through in every product we ship. Like if, you know, if we're shipping a feature and it doesn't get 2X better as the model gets 2X smarter, it's probably not a feature we should be shipping. Um, you know, not always true, you know, SOC 2 doesn't get better with, uh, you know, uh, smarter models. But, you know, I think for many of the core capabilities, that's a good litmus test. So I've always found you really have to lean into, why is this place successful? And then maximally (laughs) accelerate that, um, so to speak, because, um, uh, it's, it's what allows you to turn something that feels like an accident into something that is a repeatable, uh, playbook.

16. 1:05:07 – 1:08:50

    The importance of team composition

    1. NTNick Turley1:05:07

    2. LRLenny Rachitsky1:05:07

       So you talked about this kind of collaboration between researchers and product people, and you've been at the beginning of ChatGPT from day one to today, from zero to 700 million weekly active users. Not just registered users, weekly active users. How have you approached building out that team over time?

    3. NTNick Turley1:05:24

       One of the other inheritances of, um, being in a research lab is that you take recruiting really seriously. That's something that, you know, AI labs know, every person matters. But many tech companies that go through hyper growth, and they kind of lose their identity, they lose th- you know, their, their talent bars, they, they, they just kind of have chaos. Um, so we've always had this tendency to run relatively lean, so it is a small team that is running ChatGPT. Um, I, I take inspiration from WhatsApp, where like, you know, it was this very small team running a very global scope product. Um, and then more importantly, I, y- I, you know, you have to treat hiring a little bit more like executive recruiting, and less like just pure pipelined recruiting, where you really need to understand, what is the gap you're trying to fill on each team? What is the specific skill set, and how do you fill it? Um, to give you an example, you know, I'm a product person at heart, but sometimes a team doesn't need a product person, because, like, there's already someone doing that role, like, like, you know, in many cases, we have a really talented engineering leader who has amazing product sense, or we have a researcher who has product ideas. And then, in my mind, they can play that role, and maybe we have something else missing, um, instead. Like, maybe we need, like, a little bit more front end, um, or something like that. In other cases, um, maybe what you're missing is an incredible data scientist. So I really like to go through every single team and figure out, what is the skillsets that that team needs, and how do you put it together from principles, rather than just assuming, hey, we're gonna do, like, you know, a bunch of pipeline recruiting for all these different roles, and then, you know, people will find a team later. So, so I think that's always felt really important to me, um, and it's the way that you keep your team really small, yet super high throughput. Also allows you to hire people who, I think Keith, Keith Rabois calls this, like, like, barrels, I think. Um, b- barrels and amm- ammunition, where he thinks, I, I think, I think this comes from him, but I, um, the idea being that sort of the throughput of your depends on how many barrels you have, um, which is like, people who can make stuff happen, and I think you can hire, um... And then you can add, um, ammunition around them, um, which is, like, people helping those people. And, uh, you know, I, I think that's been really true for our recruiting too. We try to maximize sort of the number of empowered people who can ship, because that's how you have a small team and still get a ton done.So, there, there's a couple of things. Um, and, uh, I spent a lot of time on like, vibes too, with like, each team. Because I think one of the things that is challenging when you try to do research and product together is that the cultures are different, people have different backgrounds. And, um, I think to make that go super well, you need to spend time team building and making sure that people have a huge amount of trust for each other's skill sets, um, feel like they can think ac- across their boundaries. Um, like, you know, um, I really believe that product is everyone's job, for example. And, and, and for that reason, the recruiting sort of doesn't stop when the people are in the door, it actually starts because you have to, you know, start making the teams awesome.

    4. LRLenny Rachitsky1:08:24

       Is there something you do in team building that would be fun to share? Just like, something you do to create a good culture?

    5. NTNick Turley1:08:28

       I just love whiteboarding with teams. Like, I just like, like love getting into a generative mindset. It breaks down everything. So that's, that's the thing that I, I, I try. It, it's not particularly creative, but I found it to be, um, a universal tool, where the minute you can get people to stop thinking about, you know, what's my job versus the other person's job and more like, you know, we're all in a room like, trying to crack something together, that

17. 1:08:50 – 1:14:23

    Balancing speed and quality in AI development

    1. NTNick Turley1:08:50

       is incredible.

Episode duration: 1:35:37