OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil
Kevin Weil (guest), Lenny Rachitsky (host), Narrator, Narrator, Narrator, Narrator
In this episode of Lenny's Podcast, featuring Kevin Weil and Lenny Rachitsky, OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil explores openAI’s CPO on building products atop rapidly evolving AI foundations Kevin Weil, Chief Product Officer at OpenAI, explains how building on AI is fundamentally different from past tech shifts because the underlying capabilities improve dramatically every few months. This forces product teams to plan loosely, ship quickly, and design around fuzzy, probabilistic model behavior instead of deterministic software. He highlights the rising importance of evals, fine-tuning, and ensembles of models, and argues that every serious product team will eventually embed ML researchers as core members. Weil also reflects on missed opportunities like Facebook’s Libra, the transformative potential of AI tutoring, and the skills he’s encouraging his kids (and future builders) to develop in an AI-first world.
OpenAI’s CPO on building products atop rapidly evolving AI foundations
Kevin Weil, Chief Product Officer at OpenAI, explains how building on AI is fundamentally different from past tech shifts because the underlying capabilities improve dramatically every few months. This forces product teams to plan loosely, ship quickly, and design around fuzzy, probabilistic model behavior instead of deterministic software. He highlights the rising importance of evals, fine-tuning, and ensembles of models, and argues that every serious product team will eventually embed ML researchers as core members. Weil also reflects on missed opportunities like Facebook’s Libra, the transformative potential of AI tutoring, and the skills he’s encouraging his kids (and future builders) to develop in an AI-first world.
Key Takeaways
Treat today’s AI as the worst you’ll ever use—build for where capabilities are going, not where they are.
Weil stresses a “model maximalist” mindset: if your product idea is barely feasible with current models, you’re probably in the right spot, because models are improving so quickly that what barely works now will feel magical in a few months.
Get the full analysis with uListen AI
Mastering evals is becoming a core product-building skill.
Because models are fuzzy and probabilistic, you must measure how well they perform on specific tasks (e. ...
Get the full analysis with uListen AI
Future products will rely on fine-tuned, task-specific ensembles of models, not just a single general model call.
OpenAI internally uses multiple models of different sizes, prompts, and fine-tunes in concert (an ensemble) to solve problems like support, demonstrating that breaking workflows into specialized sub-tasks yields better results and should become standard practice.
Get the full analysis with uListen AI
Org design matters: small, high-agency, PM-light teams move fastest in AI.
OpenAI keeps relatively few PMs, leans heavily on product-minded engineers and researchers, and uses lightweight quarterly planning, empowering teams to ship without waiting on top-down approvals, then iterate in public.
Get the full analysis with uListen AI
Think about AI like people: human analogies often guide better product and UX decisions.
Weil repeatedly reasons about models as if they were humans or teams (e. ...
Get the full analysis with uListen AI
Chat and natural language interfaces are more powerful and durable than many assume.
Because humans already communicate flexibly through language across huge IQ and domain ranges, chat is a uniquely general interface for interacting with increasingly capable models, even if more constrained UIs will coexist for narrow, high-volume tasks.
Get the full analysis with uListen AI
There’s vast startup whitespace around verticalized AI, not just foundation models.
Weil emphasizes that no foundation-model company can cover all domains or access all proprietary data; the big opportunity is building domain-specific products that combine general models with private data, fine-tuning, and bespoke evals in each industry.
Get the full analysis with uListen AI
Notable Quotes
“The AI models that you’re using today is the worst AI model you will ever use for the rest of your life.”
— Kevin Weil
“Every two months, computers can do something they’ve never been able to do before and you need to completely think differently about what you’re doing.”
— Kevin Weil
“If you’re building and the product you’re building is right on the edge of the capabilities of the models, keep going because you’re doing something right.”
— Kevin Weil
“Sometimes it’s not any one thing, it’s just good work consistently over a long period of time.”
— Mark Zuckerberg (as quoted by Kevin Weil)
“Libra is probably the biggest disappointment of my career… the world would be a better place if we’d been able to ship that product.”
— Kevin Weil
Questions Answered in This Episode
How should a startup decide when a problem is ‘just barely’ within current model capabilities versus still too early, and how much risk is acceptable?
Kevin Weil, Chief Product Officer at OpenAI, explains how building on AI is fundamentally different from past tech shifts because the underlying capabilities improve dramatically every few months. ...
Get the full analysis with uListen AI
What are concrete first steps for a non–AI-native product team to start building and maintaining meaningful evals and fine-tuned models?
Get the full analysis with uListen AI
Given the rapid improvement and commoditization of foundation models, what kinds of moats can AI application companies realistically build?
Get the full analysis with uListen AI
How can education systems and policymakers practically prepare workers for the transition Weil describes, beyond generic calls for ‘reskilling’?
Get the full analysis with uListen AI
If Weil could redesign Libra today with current AI, crypto, and regulatory realities, what would he do differently—and what might that unlock globally?
Get the full analysis with uListen AI
Transcript Preview
The AI models that you're using today is the worst AI model you will ever use for the rest of your life. And when you actually get that in your head, it's kind of wild. Everywhere I've ever worked before this, you kind of know what technology you're building on. But that's not true at all with AI. Every two months, computers can do something they've never been able to do before and you need to completely think differently about what you're doing.
You're Chief Product Officer of maybe the most important company in the world right now. I want to chat about what it's just like to be inside the center of the storm.
Our general mindset is, in two months there's going to be a better model and it's going to blow away whatever the current set of limitations are. And we say this to developers too. If you're building and the product that you're building is kind of right on the edge of the capabilities of the models, keep going because you're doing something right. Give it another couple months and the models are going to be great. And suddenly the product that you have that just barely worked is really gonna sing.
Famously, you led this project at Facebook called Libra.
Libra is probably the biggest disappointment of my career. It fundamentally disappoints me that this doesn't exist in the world today because the world would be a better place if we'd been able to ship that product. We tried to launch a new blockchain. It was a basket of currencies originally. It was integration into WhatsApp and Messenger. I would be able to send you 50 cents in WhatsApp for free. It should exist. To be honest, the current administration is super friendly to crypto. Facebook's reputation is in a very different place. Maybe they should go build it now.
Today my guest is Kevin Wheel. Kevin is Chief Product Officer at OpenAI, which is maybe the most important and most impactful company in the world right now being at the forefront of AI and AGI and maybe someday super intelligence. He was previously head of product at Instagram and Twitter. He was co-creator of the Libra cryptocurrency at Facebook, which we chat about. He's also on the boards of Planet and Strava and the Black Product Managers Network and the Nature Conservancy. He's also just a really good guy and he has so much wisdom to share. We chat about how OpenAI operates, implications of AI and how we will all work and build product, which markets within the AI ecosystem companies like OpenAI won't likely go after and thus are good places for startups to own, also why learning the craft of writing evals is quickly becoming a core skill for product builders, what skills will matter most in an AI era and what he's teaching his kids to focus on, and so much more. This is a very special episode and I'm so excited to bring it to you. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. If you become an annual subscriber of my newsletter, you get a year free of Perplexity Pro, Linear, Notion, Superhuman and Granola. Check it out at lennysnewsletter.com and click bundle. With that, I bring you Kevin Wheel. This episode is brought to you by Eppo. Eppo is a next generation AB testing and feature management platform built by alums of Airbnb and Snowflake for modern growth teams. Companies like Twitch, Miro, ClickUp and DraftKings rely on Eppo to power their experiments. Experimentation is increasingly essential for driving growth and for understanding the performance of new features. And Eppo helps you increase experimentation velocity while unlocking rigorous deep analysis in a way that no other commercial tool does. When I was at Airbnb, one of the things that I loved most was our experimentation platform where I could set up experiments easily, troubleshoot issues and analyze performance all on my own. Eppo does all that and more with advanced statistical methods that can help you shave weeks off experiment time, an accessible UI for diving deeper into performance and out of the box reporting that helps you avoid annoying prolonged analytic cycles. Eppo also makes it easy for you to share experiment insights with your team, sparking new ideas for the AB testing flywheel. Eppo powers experimentation across every use case including product, growth, machine learning, monetization and email marketing. Check out Eppo at geteppo.com/lenny and 10X your experiment velocity. That's geteppo.com/lenny. This episode is brought to you by Persona, the adaptable identity platform that helps businesses fight fraud, meet compliance requirements and build trust. While you're listening to this right now, how do you know that you're really listening to me, Lenny? These days it's easier than ever for fraudsters to steal PII, faces and identities. That's where Persona comes in. Persona helps leading companies like LinkedIn, Etsy and Twilio securely verify individuals and businesses across the world. What sets Persona apart is its configurability. Every company has different needs depending on its industry, use cases, risk tolerance and user demographics. That's why Persona offers flexible building blocks that allow you to build tailored collection and verification flows that maximize conversion while minimizing risk. Plus, Persona's orchestration tools automate your identity process so that you can fight rapidly shifting fraud and meet new waves of regulation. Whether you're a startup or an enterprise business, Persona has a plan for you. Learn more at withpersona.com/lenny. Again, that's with P-E-R-S-O-N-A dot com slash lenny. Kevin, thank you so much for being here and welcome to the podcast.
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome