The Diary of a CEOHe Risked Everything To Warn You: No One Is Ready For What's Coming, And The AI Companies Know It!
At a glance
WHAT IT’S REALLY ABOUT
OpenAI forecaster warns AI race risks catastrophe, urges regulation plan.
- Kokotajlo claims frontier labs are explicitly pursuing automation of AI research to accelerate toward superintelligence, and he believes many insiders now think 2027–2029 timelines are plausible.
- He frames two dominant dangers: misaligned systems gaining real-world power (loss of control) and a small set of corporations/governments controlling an “army of geniuses” that could dominate economies and geopolitics.
- He describes why he left OpenAI—disillusionment with shifting narratives and incentives—and recounts the exit-paperwork NDA/anti-disparagement controversy that nearly cost him his equity.
- He outlines scenario forecasts (AI 2027 as a default fast-race trajectory) versus a recommended policy roadmap (AI 2040 Plan A) centered on slowing training, increasing transparency, and distributing power internationally.
- He argues job displacement could arrive suddenly after internal “recursive” automation, and proposes a “citizens’ dividend” to preserve livelihoods and political power as AI takes cognitive labor.
IDEAS WORTH REMEMBERING
5 ideasThe industry’s core race is to automate AI research, not the wider economy first.
Kokotajlo says labs prioritize autonomous coding and end-to-end research agents to speed model improvements, which could create a sudden capability jump before the public feels gradual job displacement.
“AGI” is ambiguous; “superintelligence” is the clearer threshold with higher stakes.
He notes AGI can mean ‘general’ systems (arguably already here in limited form), while superintelligence is “better than the best humans at everything, faster and cheaper,” including eventual robotics-enabled physical work.
Loss of control is exacerbated by opacity: we can’t reliably tell what a neural net ‘is thinking.’
He emphasizes that unlike traditional software, modern models are trained artifacts with emergent behavior, making it plausible to deploy systems that appear aligned while pursuing other objectives.
Even ‘controlled’ AI can still be catastrophic via concentration of power.
If a few CEOs/governments own near-universal labor and strategic advantage, society risks oligarchy, coercion, and geopolitical destabilization—even without an outright AI takeover.
Narratives shift with incentives; judge labs by actions and governance, not PR.
He describes an internal cultural change post-ChatGPT where “we’ll pause near the finish line” became “it’s not that risky,” which he interprets as rationalization under competitive and political pressure.
WORDS WORTH SAVING
5 quotesThe scary open secret in the AI industry right now is that it's possible that we'll end up essentially creating a new species that ends up ruling the world with a 70% chance that this goes horribly wrong, like human extinction.
— Daniel Kokotajlo
I basically told my wife, like, "Let's not have any more kids. It's too uncertain. I don't think they'll ever join the workforce."
— Daniel Kokotajlo
These powerful CEOs, Dario or Sam or Elon, are racing each other to be in control of the most powerful AIs and are literally afraid that if the other guy gets there first, he might become dictator.
— Daniel Kokotajlo
It really seems like we're on a pretty scary path, and the more you think about it, the more worried you get. And, you know, in stories it always ends well, but this is real life and I, I think we have to sort of stare reality in the face and tell it, and realize that, like, it might not actually end well, you know?
— Daniel Kokotajlo
Like, I think if it was a sort of temporary shutdown, I would totally slam that button because we are not ready to do this, you know? Like, civilization is not ready to have these companies automate themselves and then get smarter and smarter and then have the superintelligent... Like, no.
— Daniel Kokotajlo
High quality AI-generated summary created from speaker-labeled transcript.
