By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
The Twenty Minute VCJev Creator on Will US Open Models Catch Chinese | The Cost of Intelligence & Tokens | Diogo Almeida
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Great conversations run long. Their best ideas don't have to.
uListen is free on iPhone, iPad, Android, Chrome, and Claude.
See howAt a glance
WHAT IT’S REALLY ABOUT
Diogo Almeida on reliability-first AI, data over compute, and benchmarks
- Diogo Almeida argues that the central bottleneck in AI automation is reliability of intelligence—models can be “Einstein” most of the time and “Mr. Bean” the rest, which makes high-stakes automation uneconomical.
- He claims the field overemphasized compute scaling and underemphasized choosing the right task and generating the right data, and he positions Jev/TypeSafe as optimizing for decision reliability rather than human-pleasing or benchmark performance.
- He says the public model-evaluation ecosystem is distorted because benchmarks are easy to game and don’t reflect real usability, creating misinformation that harms people who can’t test models directly.
- He is skeptical that most companies need to “own their own intelligence” via training proprietary models, arguing instead that durable differentiation is more likely to come from software/engineering that locks in real automation.
- He expects large future demand for background automation (“mega Jevons”) but also believes AI has been in a bubble and that demand could drop if FOMO-driven expectations collide with what current systems can actually do.
IDEAS WORTH REMEMBERING
5 ideasReliability of intelligence—not availability—is the main blocker to real automation.
He argues models can appear brilliant on hard problems yet fail at simpler, high-stakes work because their output quality is high-variance; a small failure rate breaks expected value for automation. His framing is that solving this requires “nines of intelligent reliability,” especially in judgment/decision-making rather than surface-level conversational consistency.
Choosing the right training objective matters more than scaling compute.
He lays out an ordering: doing the “right task” first, then data, then compute, then algorithms. In his view, RLHF (“please humans”) and RLVR (often treated as “score high on verifiable rewards/benchmarks”) don’t directly optimize for background automation reliability, so they produce systems that look impressive but aren’t trustworthy for decisions with stakes.
Scaling compute alone stopped being a straightforward path to better, useful models.
He says the field’s earlier “scale to the moon” narrative broke after GPT-4.5, which he describes as very expensive per token yet not commercially successful. He attributes the plateau not to compute being useless, but to compute without enough appropriately-shaped data (and the right task) failing to pay off.
Valuable training data is task-defined and hard to bootstrap because it often doesn’t exist yet.
He describes “good data” as data that measurably improves cutting-edge models on the intended task, which requires being at the right difficulty and often being “on-policy” with current model capabilities. He is skeptical that raw enterprise data lakes are directly the right shape without significant task-driven reshaping.
Synthetic data is neither a silver bullet nor useless; it’s execution- and task-dependent.
He calls synthetic data a tool whose value depends on how it’s used; he notes a common research conclusion that it helps smaller models catch up but doesn’t improve frontier models, while also saying his team uses a lot of synthetic data in a different, task-focused way. The key claim is that “synthetic vs. real” is less important than using it with laser focus on reliability of intelligence.
WORDS WORTH SAVING
5 quotesAlmost everyone who's paying attention to AI knows reliability is the absolute largest bottleneck. You get Einstein maybe 95% of the time and Mr. Bean the other 5%.
— Diogo Almeida
I would argue that there's a bitter lesson, which is that, um, data matters a lot more than compute.
— Diogo Almeida
You have no fu- you have no fucking clue how little they care.
— Diogo Almeida
Benchmarks are total bullshit. All benchmarks are bad.
— Diogo Almeida
Everyone has just been in this, like, mono model, like, horse blinders for so fucking long. And, like, I, I just reject that premise originally.
— Diogo Almeida
Ask ChatGPT about this episode
Get its key ideas, plus a summary of each chapter linked to that point in the video, right in your chat.
Add uListen AI to ChatGPT.
Install uListen AI in ChatGPT (opens in a new tab)Needs a ChatGPT account.Paste this prompt into a new chat.
Use uListen AI to give me this episode's key ideas and a chapter-by-chapter summary, with timestamps: https://www.youtube.com/watch?v=BE2JqjsN4aY
SPEAKERS
Diogo Almeida
guestFounder of TypeSafe and creator of Jev, working on AI reliability and model evaluation.
Harry Stebbings
hostHost of 20VC (The Twenty Minute VC), interviewing founders and investors in tech and venture capital.
High quality AI-generated summary created from speaker-labeled transcript.