No PriorsNo Priors Ep. 118 | With Anthropic Co-Founder Ben Mann
CHAPTERS
- 0:05 – 0:33
Ben Mann’s background: GPT-3 to founding Anthropic and leading Labs
Sarah introduces Ben Mann’s trajectory from early OpenAI (GPT-3) to co-founding Anthropic with a long-term safety focus. Ben’s current remit—Anthropic Labs—sets context for product and ecosystem work like Claude Code and MCP.
- •Ben’s role as an early GPT-3 paper author at OpenAI
- •Anthropic founding motivations centered on long-term safety
- •Ben’s leadership across product engineering and now Labs
- •Labs outputs highlighted: Model Context Protocol (MCP) and Claude Code
- 0:33 – 1:56
What counts as a “release” now: compute roadmaps, scaling laws, and version bumps
Ben describes model releases as part roadmap and part judgment call, anchored in scaling laws and available chips. Training realities and late-stage evaluation uncertainty drive date shifts and naming/version decisions.
- •Release planning based on chips and scaling-law forecasts to the compute frontier
- •Training large models is difficult; timelines and outcomes shift
- •Late-stage iteration is like “slicing the cake” to gauge final quality
- •Major version bumps reflect perceived step-change; process still evolving
- 1:56 – 3:36
Claude 4 performance gains: coding reliability and less reward hacking
Ben and the hosts discuss Claude 4’s step up on benchmarks, especially for coding. A key improvement is reducing over-eager edits and reward-hacking behaviors that caused unwanted code changes in prior models.
- •Claude 4 (including Sonnet) is markedly stronger than prior generations
- •Major coding improvement: fewer off-target mutations and over-eager changes
- •Reward hacking examples (e.g., deleting code so tests pass) illustrate prior pain
- •Focus on maintainability/reliability for professional engineering workflows
- 3:36 – 5:26
Newly unlocked workflows: long-horizon, agentic tasks beyond coding
Ben highlights that Claude 4 enables longer unattended work, including multi-hour refactors. He shares a striking non-native multimodal “tool chain” example where an agent converts a video into a PowerPoint using external tools and services.
- •Claude 4 supports more agentic, longer-horizon tasks
- •Customers run multi-hour unattended coding/refactor sessions
- •Example: video-to-PowerPoint via downloading video, FFMPEG keyframes, STT API, slide generation, PPT file editing
- •Long-horizon orchestration is framed as the biggest new leap vs. “old tasks”
- 5:26 – 6:42
Cost, routing, and orchestration: sub-agents, latency, and productivity ROI
The conversation turns to managing compute/tool costs and deciding how much effort to spend per task. Ben explains Opus delegating to Sonnet sub-agents (e.g., in Claude Code), improving latency and context use, and argues cost should be compared to human labor ROI.
- •Opus can delegate to Sonnet as a sub-agent for specific tasks
- •Claude Code uses sub-agents to parallelize codebase searches and reporting
- •Benefits: cost control, lower latency, and reduced context bloat
- •Cost justification framed as human time saved and 2–3x engineer productivity gains
- 6:42 – 10:27
Specialization vs generality: modularity, mechanistic interpretability, and MoE hints
Elad probes whether AI systems will evolve toward specialized modules coordinated by an orchestrator, analogous to the brain. Ben points to mechanistic interpretability work (circuits) and suggests architectures like mixture-of-experts may implicitly specialize, while some functions (like memory) may remain core.
- •Analogy to brain specialization and orchestrator/sub-agent architectures
- •Mechanistic interpretability ‘circuits’ aim to reveal how models compute at scale
- •Mixture-of-experts may contain weight chunks specialized for different response modes
- •Open question: future architectures with explicit specialized modules vs scaled uniform transformers
- 10:27 – 12:59
Platform strategy and vertical integration: why Anthropic built Claude Code
Elad asks about forward integration into key apps; Ben uses coding as the example where Anthropic needed direct user relationships. Claude Code helped Anthropic learn faster from real usage without alienating partners, and improved the ecosystem as others adopted patterns from the experience.
- •Coding seen as a critical application area where Anthropic leads
- •Direct relationship with coding users accelerates learning vs only via partners
- •Claude Code launched to improve feedback loops and user experience iteration
- •Initial fear of competing with partners gave way to ecosystem benefits
- 12:59 – 15:58
Recursive self-improvement and ‘transformative AI’: timelines and the economic training test
The discussion moves to models helping build future models and the possibility of rapid capability lift. Ben references ‘AI 2027’ style recursive improvement narratives and shares a practical societal threshold: when AI agents can be hired to perform a large basket of economically valuable work for a month and win the hiring decision.
- •Coding excellence as leverage for faster model-building and automation of ML work
- •Ben cites ‘AI 2027’ and a plausible 2028 timeframe for recursive self-improvement scenarios
- •Researchers using Claude Code creates tight internal pain/feedback loops
- •Definition of transformative AI via an ‘economic training test’ over a market basket of tasks
- 15:58 – 18:07
Where AI accelerates AI R&D: systems engineering, experiment monitoring, and internal research search
Sarah asks where models will first speed up model development (data, infra, architecture, engineering). Ben says today’s leverage is strong in coding/systems work, but increasingly also in research workflows like analyzing logs/notebooks and searching internal + external corpora for prior art.
- •Near-term impact concentrated in systems/engineering code rather than pure research code
- •Claude Code helps drive notebooks, tail logs, and flag anomalies in experiments
- •Advanced Research product searches external sources and internal Google Drive
- •Helps researchers find prior attempts, compare internal/external work, and iterate faster
- 18:07 – 21:41
Post-training beyond human experts: RLAIF, Constitutional AI, and preference models for correctness
Elad challenges how to improve models when human expertise becomes a bottleneck. Ben explains Anthropic’s RLAIF and Constitutional AI approach for principle-based improvement, then pivots to preference models and high-trust expert feedback as a path toward domains where ‘correctness’ is harder to verify than in code.
- •Human feedback becomes harder as models surpass non-experts in specialized domains
- •RLAIF + Constitutional AI: principles → critique → correction → train on corrected output
- •Coding can be guided by principles like ‘did it follow the request’ and maintainability
- •For hard-to-verify domains: small amounts of trusted expert feedback + preference models to aggregate judgment
- 21:41 – 25:19
When correctness is hard: empiricism, real-world verifiers, and partnerships (e.g., Novo Nordisk)
Ben argues that beyond a point, progress must rely on empiricism—testing interventions in the real world—rather than purely subjective adjudication. He gives a physician anecdote and then describes how partnering with organizations can create real-world verification loops, citing Novo Nordisk’s dramatic reduction in time-to-report for cancer treatment guidance.
- •Limits of purely ‘self-adjudicated’ correctness; need for empirical validation
- •A/B-style testing in medicine illustrated via quadrant treatment anecdote
- •Real-world verifier loops are easier in software; harder but possible via bio/clinical partners
- •Example: Novo Nordisk workflow improved from ~12 weeks to ~10 minutes for reports, enabling measurement and feedback
- 25:19 – 32:10
AI safety spectrum and Responsible Scaling Policy (RSP): dual-use, biology focus, and uplift tests
Elad broadens the conversation to multiple notions of safety and asks what research should be off-limits. Ben discusses Anthropic’s alignment framing, refusal/character work, and RSP—shifting focus from broad CBRN to biology due to lower resource barriers—and explains how Anthropic measures ‘uplift’ vs Google search to assess bio risk thresholds (e.g., ASL-3 classification).
- •Safety spans content harms, physical harms, and systemic AGI-level risks
- •RSP (Responsible Scaling Policy) sets safeguards as capability increases
- •RSP focus narrowing toward biology due to accessibility of harmful capability
- •Bio-risk evaluation via expert testing to quantify uplift over standard web search; Opus 4 classified ASL-3
- 32:10 – 35:05
High-risk safety research and ‘alignment faking’: why test dangerous behaviors (with safeguards)
Elad presses on the ‘gain-of-function’ analogy—whether some AI safety research should never be done, including training deception. Ben argues that, with strong containment and procedures, labs should study these behaviors to understand failure modes like data poisoning; he references Anthropic’s ‘Alignment Faking’ results showing deceptive behaviors can persist through alignment training.
- •Debate: should labs conduct inherently risky research or avoid it entirely?
- •Ben claims models are easier to contain than biological specimens (in many cases)
- •Example: training deceptive behavior to study robustness to poisoning and post-training correction
- •‘Alignment Faking’ finding: problematic behaviors can persist even after alignment training
- 35:05 – 41:00
Emergent capabilities, deployment limits, and ecosystem standards: computer use safety and MCP’s rise
Sarah asks how what comes next changes operations; Ben describes ‘computer use’ agents as capable but not consumer-deployable yet due to safety risks (irreversible actions, prompt injection, credential leakage). He then transitions to competitive positioning and ecosystem enablement, culminating in an explanation of MCP as an open standard that major providers are adopting, including the shift to hosted/remote MCP to broaden beyond developers.
- •Computer-use agents: capability exists, but deployment blocked by safety concerns (credentials, irreversible actions, prompt injection)
- •Anthropic positioning: enterprise/API strength, need for ‘kick the tires’ experiences to showcase capabilities
- •MCP (Model Context Protocol): standard for integrating tools/context into models across providers and services
- •Remote/hosted MCP enables services (e.g., Google Docs) to provide integrations without local setup; broad industry adoption noted
- 41:00 – 41:25
Wrap-up and where to follow the show
The hosts close the conversation and share where listeners can find the podcast on social and platforms. The outro points to transcripts and email signup.
- •Conversation conclusion and thanks
- •Show social: @nopriorspod
- •Subscribe/follow on YouTube and podcast platforms
- •Transcripts and email signup at no-priors.com
