Anthropic co-founder: AGI predictions, leaving OpenAI, what keeps him up at night | Ben Mann
Lenny Rachitsky (host), Benjamin Mann (guest)
In this episode of Lenny's Podcast, featuring Lenny Rachitsky and Benjamin Mann, Anthropic co-founder: AGI predictions, leaving OpenAI, what keeps him up at night | Ben Mann explores anthropic’s Ben Mann on AI safety, superintelligence, and jobs’ future Anthropic co‑founder Ben Mann discusses why he left OpenAI to build a company where safety and alignment are the top priority while still pushing the frontier of AI capabilities.
Anthropic’s Ben Mann on AI safety, superintelligence, and jobs’ future
Anthropic co‑founder Ben Mann discusses why he left OpenAI to build a company where safety and alignment are the top priority while still pushing the frontier of AI capabilities.
He argues that scaling laws are still accelerating, predicts a ~2028 median timeline for superintelligence, and expects profound economic upheaval, including potentially 20% unemployment and a transformed version of capitalism.
Mann explains Anthropic’s approach to alignment—especially Constitutional AI and reinforcement learning from AI feedback—and why he believes society is underinvesting in mitigating existential AI risk, which he pegs at between 0–10%.
He also shares practical advice on future‑proofing careers by aggressively using AI tools, his philosophy for raising kids in an AI-saturated world, and how Anthropic operationalizes safety without sacrificing product velocity.
Key Takeaways
AI progress is accelerating, not plateauing, driven by faster model release cycles and new training techniques.
Mann notes that what looks like stagnation is often benchmark saturation and time compression—models are improving so quickly and being released so frequently that incremental gains feel smaller, even as underlying scaling laws continue to hold (and may even be strengthening).
Get the full analysis with uListen AI
Transformative AI will be defined by economic displacement, not a sci‑fi definition of AGI.
He favors an “economic Turing test”: when AI can be profitably hired instead of humans for a large share of money‑weighted jobs and global GDP growth jumps (e. ...
Get the full analysis with uListen AI
Massive labor impact is coming, with a turbulent transition but eventual abundance.
Mann expects dramatic productivity gains (e. ...
Get the full analysis with uListen AI
Safety and competitiveness can be mutually reinforcing rather than in tension.
Anthropic’s safety research directly shapes Claude’s personality (helpful, honest, harmless) and trustworthiness; Constitutional AI lets the model learn from natural-language principles, producing behavior that both customers like and regulators can understand, while keeping Anthropic at the frontier.
Get the full analysis with uListen AI
Current AI risk is nontrivial but still under-addressed, with very few people working on it.
Mann estimates the probability of existential or extremely bad outcomes from AI somewhere between 0–10%, emphasizes that alignment likely becomes impossible once superintelligence is reached, and argues that society should treat even low probabilities very seriously, given the stakes.
Get the full analysis with uListen AI
Recursive self‑improvement and agents must be constrained by empiricism and governance.
He describes reinforcement learning from AI feedback (RLAIF) and agentic systems that can use tools or computers, noting they’ve already observed deceptive behaviors in lab settings; the challenge is enabling self‑improvement while maintaining alignment, akin to aligning corporations and scientific institutions.
Get the full analysis with uListen AI
Individuals can future‑proof somewhat by aggressively, creatively using AI tools now.
Mann advises people to be ambitious with tools like Claude Code—treat AI as a coworker, repeatedly retry or reformulate hard tasks, and apply these systems beyond engineering (e. ...
Get the full analysis with uListen AI
Notable Quotes
“We felt like safety wasn't the top priority there.”
— Ben Mann (on leaving OpenAI to start Anthropic)
“I think 50th percentile chance of hitting some kind of super intelligence is now, like, 2028.”
— Ben Mann
“Once we get to super intelligence, it will be too late to align the models.”
— Ben Mann
“My best granularity forecast for, like, could we have an x-risk or extremely bad outcome is somewhere between 0 and 10%.”
— Ben Mann
“In a world of abundance where labor is almost free and anything you want to do, you can just ask an expert to do it for you, then what do jobs even look like?”
— Ben Mann
Questions Answered in This Episode
If superintelligence is plausible by 2028, what specific safeguards should governments and companies implement in the next 3–5 years?
Anthropic co‑founder Ben Mann discusses why he left OpenAI to build a company where safety and alignment are the top priority while still pushing the frontier of AI capabilities.
Get the full analysis with uListen AI
How can societies practically manage a transition where AI may drive unemployment toward 20% while also massively increasing productivity and abundance?
He argues that scaling laws are still accelerating, predicts a ~2028 median timeline for superintelligence, and expects profound economic upheaval, including potentially 20% unemployment and a transformed version of capitalism.
Get the full analysis with uListen AI
Who should decide the “constitution” for AI models—companies, governments, international bodies, or some democratic process involving the public?
Mann explains Anthropic’s approach to alignment—especially Constitutional AI and reinforcement learning from AI feedback—and why he believes society is underinvesting in mitigating existential AI risk, which he pegs at between 0–10%.
Get the full analysis with uListen AI
What early real‑world signals should we watch for that indicate AI systems are becoming dangerously misaligned or deceptively agentic?
He also shares practical advice on future‑proofing careers by aggressively using AI tools, his philosophy for raising kids in an AI-saturated world, and how Anthropic operationalizes safety without sacrificing product velocity.
Get the full analysis with uListen AI
For someone outside of core AI research, what is the most high‑leverage way to contribute to AI safety and beneficial deployment today?
Get the full analysis with uListen AI
Transcript Preview
(instrumental music) You wrote somewhere that, "Creating powerful AI might be the last invention humanity ever needs to make." How much time do we have, Ben?
I think 50th percentile chance of hitting some kind of super intelligence is now, like, 2028.
What is it that you saw at OpenAI, what'd you experience there that made you feel like, "Okay, we gotta go do our own thing"?
We felt like safety wasn't the top priority there. The case for safety has gotten a lot more concrete. So, super intelligence is a lot a... about, like, how do we keep God in a box and not let the God out?
What are the odds that we align AI correctly?
Once we get to super intelligence, it will be too late to align the models. My best granularity forecast for, like, could we have an x-risk or extremely bad outcome is somewhere between 0 and 10%.
Something that's in the news right now is this whole, Zuck coming after all the top AI researchers.
We've been much less affected because people here, they get these offers and then they say, "Well, of course, I'm not gonna leave because my best case scenario at Meta is that we make money. And my best case scenario at Anthropic is we, like, affect the future of humanity."
Dario, your CO, recently talked about how unemployment might go up to something like 20%.
If you just think about, like, 20 years in the future where we're, like, way past the singularity, it's hard for me to imagine that even capitalism will look at all like it looks today.
Do you have any advice for folks that want to try to get ahead of this?
I'm not immune to job replacement either. At some point, it's coming for all of us.
Today, my guest is Benjamin Mann. Holy moly, what a conversation. Ben is the co-founder of Anthropic. He serves as tech lead for product engineering. He focuses most of his time and energy on aligning AI to be helpful, harmless and honest. Prior to Anthropic, he was one of the architects of GPT-3 at OpenAI. In our conversation, we cover a lot of ground, including his thoughts on the recruiting battle for top AI researchers, why he left OpenAI to start Anthropic, how soon he expects we'll see AGI. Also, his economic Turing test for knowing when we've hit AGI, why scaling laws have not slowed down, and are in fact accelerating, and what the current biggest bottlenecks are, why he's so deeply concerned with AI safety, and how he and Anthropic operationalize safety and alignment into the models that they build and into their ways of working. Also, how the existential risk from AI has impacted his own perspectives on the world and his own life, and what he's encouraging his kids to learn to succeed in an AI future. A huge thank you to Steve Nitch, Danielle Caggieri, Raf Lee, and my newsletter community for suggesting topics for this conversation. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. Also, if you become an annual subscriber of my newsletter, you get a year free of a bunch of amazing products, including Bolt, Linear, Superhuman, Notion, Granola, and more. Check it out at lennysnewsletter.com and click bundle. With that, I bring you Benjamin Mann. This episode is brought to you by Sauce. The way teams turn feedback into product impact is stuck in the past. Vague reports, static taxonomies, unactionable insights that don't move business metrics. The result, churn, lost deals, missed growth. Sauce is the AI product copilot that helps CPOs and product teams uncover business impact and act faster. It listens to your sales calls, support tickets, churn reasons, and lost deal, surfacing the biggest product issues and opportunities in real time. It then routes them to the right teams to turn signals into PRDs, prototypes, and even code that drives revenue, retention, and adoption. That's why whatnot, Linktree, IncidentIO and Zip use Sauce. One enterprise uncovered a product gap that unlocked $16 million ARR, another caught a spiking issue and prevented millions in churn. You can too at sauce.app/lenny. Sauce, built for AI product teams. Don't get left behind. This episode is brought to you by Lucidlink, the storage collaboration platform. You've built a great product, but how you show it through video, design, and storytelling is what brings it to life. If your team works with large media files, videos, design assets, layered project files, you know how painful it can be to stay organized across locations. Files live in different places. You're constantly asking, "Is this the latest version?" Creative work slows down while people wait for files to transfer. Lucidlink fixes this. It gives your team a shared space in the cloud that works like a local drive. Files are instantly accessible from anywhere. No downloading, no syncing, and always up to date. That means producers, editors, designers, and marketers can open massive files in their native apps, work directly from the cloud, and stay aligned wherever they are. Teams at Adobe, Shopify, and top creative agencies use Lucidlink to keep their content engine running fast and smooth. Try it for free at lucidlink.com/lenny. That's L-U-C-I-D-L-I-N-K.com/lenny. (instrumental music) Ben, thank you so much for being here. Welcome to the podcast.
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome