No PriorsWhy Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon
CHAPTERS
- 0:05 – 2:59
Stefano Ermon’s path into generative modeling and “world model” motivation
Stefano recounts starting as a Stanford professor in 2014 when generative modeling was niche and hard to publish. He explains his early conviction that generative modeling is the right way to learn structure from unlabeled data, motivated by the idea of “world models” for prediction and decision-making.
- •Generative modeling wasn’t fashionable in 2014; results were limited (e.g., MNIST)
- •Early justification: generative models as representation learning for downstream supervised tasks
- •Personal motivation: world-model style imagination and model predictive control
- •Unexpected pace: capabilities evolved far faster than he anticipated
- 2:59 – 3:42
From autoregressive and GAN pain points to score-based diffusion (2019)
He walks through the evolution from slow, blurry autoregressive image models to VAEs and GANs, and why GAN instability pushed his lab toward more principled approaches. This culminated in score-based/denoising methods that became the foundation of modern diffusion models.
- •Autoregressive image generation was slow and low quality; VAEs/GANs followed
- •GANs produced strong results but were unstable and hard to reproduce
- •Key idea: learn denoisers that capture image structure
- •2019 lab work (with Yang Song) helped establish modern diffusion modeling
- 3:42 – 4:27
How diffusion generates—and why it took over images/video/music/proteins
Stefano describes diffusion as starting from noise and iteratively refining to a clean sample, contrasting it with left-to-right generation. He notes diffusion’s dominance across continuous modalities and its broad adoption in image/video generation and beyond.
- •Diffusion: iterative denoising from noise to data sample
- •Contrast with autoregressive: sequential token/pixel generation
- •Diffusion now leads in image and increasingly video generation
- •Extensions appearing in music and protein-related modeling
- 4:27 – 7:04
Breakthrough: diffusion for text/code and the origin of Inception
He explains the technical leap required to make diffusion competitive on discrete data, and highlights a 2024 paper demonstrating parity with GPT-2-scale autoregressive models. The speed advantage (parallel token generation) motivated starting Inception to scale diffusion-based language models commercially.
- •Goal: move beyond autoregressive text generation to parallel, error-correcting generation
- •2024 result: diffusion transformer matched autoregressive perplexity at GPT-2 scale
- •Achieved ~10× faster generation by emitting many tokens in parallel
- •Inception founded to scale and commercialize diffusion-based LLMs
- 7:04 – 11:11
Core thesis: diffusion wins inference-time scaling vs. sequential autoregressive
Stefano argues the decisive battleground will be inference-time efficiency and scaling, not just training. He draws an analogy to the RNN→Transformer shift (parallelism), claiming diffusion is the analogous leap for inference because autoregressive decoding remains fundamentally sequential and memory-bound.
- •Autoregressive decoding cannot parallelize across future tokens
- •Inference is memory-bound on GPUs due to weight movement vs. arithmetic
- •Diffusion inference resembles training-style parallel workloads (GPU-friendly)
- •Inference-time scaling matters for economics (intelligence per watt/dollar) and reasoning/test-time compute
- 11:11 – 13:20
Discrete vs. continuous: why text diffusion is hard and what Inception claims now
They discuss the conceptual mismatch between diffusion’s continuous interpolation intuition and discrete language tokens. Stefano claims Inception has bridged that gap and now serves diffusion LLMs (Mercury) with benchmark quality comparable to fast “mini/flash” offerings, plus meaningful speed gains—enabled by bespoke infrastructure.
- •Discrete spaces lack natural interpolation (unlike pixels/colors)
- •Required new research to adapt diffusion to text/code
- •Claimed performance: Mercury comparable to fast frontier-lab “small” models on benchmarks
- •Productionization required building a custom serving engine (not vLLM/SGLang compatible)
- 13:20 – 16:32
Inception today: team size, R&D focus, and why speed is the wedge
Stefano outlines the company’s current scale (~50 people) and ongoing emphasis on R&D across training recipes and fast sampling. He frames speed as the current differentiator while leaving open the possibility of future capability advantages as the paradigm matures.
- •~2 years old, ~50 employees; still R&D heavy
- •Diffusion allows multiple compute–quality tradeoffs at inference (sampling/distillation/ODE solvers)
- •Reuse of transformer components (attention, datasets, evals) for scrappiness
- •Speed is current focus; future intelligence gains are uncertain but plausible
- 16:32 – 18:51
Where speed matters most: latency-sensitive products and voice agents
Sarah and Stefano discuss the market pull for faster models, especially once users adapt to low latency. Stefano gives a concrete example in voice agents, where response time is a product requirement and diffusion LLMs can deliver high speed on commodity GPUs.
- •Latency-sensitive applications prioritize faster generation at similar quality
- •User expectations shift: once fast, hard to return to slower experiences
- •Voice pipeline: ASR → LLM/tool calls → TTS; LLM speed is critical
- •Example: OpenCall switched from custom-chip serving to Mercury on NVIDIA GPUs for speed/availability/cost
- 18:51 – 19:35
Software vs. hardware acceleration—and why they’re multiplicative
They explore how diffusion’s parallelism interacts with the hardware landscape, including custom chips optimized for memory. Stefano argues software-side parallelism can unlock big gains on existing GPUs, and could stack with future hardware tailored to diffusion workloads.
- •Hardware acceleration is valuable, but software can exploit existing GPU fleets
- •Diffusion parallelism provides software-level speedups without custom chips
- •Software and hardware gains can be multiplicative
- •Potential future: hardware designed specifically around diffusion inference patterns
- 19:35 – 21:42
Competing with big labs: moats via IP, serving stack, and customer feedback loops
Sarah raises the risk that large labs can absorb architectural advances quickly. Stefano argues Inception’s moat comes from end-to-end execution: proprietary training know-how plus the production stack (e.g., serving engine) and iterative learning from real customer usage.
- •Risk: big players can copy ideas once proven at scale
- •Moat components: trade secrets, training techniques, and systems know-how
- •Serving engine is a key differentiator and barrier to replication
- •Customer deployments generate feedback, evals, and sometimes data that compounds advantage
- 21:42 – 24:47
Data structure, compression, and why “messy” code/text still works
They unpack an intuition question: diffusion thrives on structured physical signals, but text/code can be noisy. Stefano reframes generative modeling as learning compression; if perplexity improves, structure is being captured—making it ultimately an empirical question of which inductive biases work best.
- •Training generative models ≈ learning an efficient compression scheme
- •Perplexity measures how much structure the model captures
- •Empirical parity with autoregressive suggests comparable structure discovery in text
- •Key open question: which modeling objective/inductive bias (next-token vs. denoising) is best
- 24:47 – 27:26
Controllability and steering: diffusion as a more “course-to-fine” control surface
They discuss whether diffusion LLMs can fit into existing alignment and tooling. Stefano says Inception kept interfaces backward compatible (OpenAI-style APIs), while also highlighting diffusion’s potential advantage: steering generation early via intermediate states and external reward/constraints.
- •Backwards-compatible text-in/text-out APIs; works with existing product stacks
- •Instruction following and structured outputs (e.g., JSON) supported in practice
- •Diffusion can be easier to steer because generation is iterative and inspectable
- •Potential for new product interfaces enabling constraints/reward-guided decoding not feasible in autoregressive
- 27:26 – 29:04
Emergent capabilities at scale: data efficiency as a possible advantage
Stefano emphasizes uncertainty about what new capabilities may emerge as diffusion LLMs scale. He points to academic evidence that diffusion training can be more data-efficient due to denoising-based “augmentation,” which could matter as high-quality data becomes scarce.
- •Beyond speed, future advantages are unknown and potentially emergent
- •Hypothesis: diffusion may be more data-efficient due to multiple noisy views per sample
- •If true, could help in low-data domains or as data becomes bottleneck
- •Scaling will reveal whether these benefits hold for language models
- 29:04 – 34:05
Workload split outlook, adoption frictions, and building the missing ecosystem
Stefano estimates a meaningful share of workloads are latency constrained and thus candidates for diffusion LLMs even without frontier intelligence parity. He also outlines adoption hurdles: lack of mature open tooling, need to build post-training stacks in-house, and strategic tradeoffs around open-sourcing.
- •Estimate: ~20–30% of workloads are strongly latency-sensitive
- •Diffusion LLM adoption limited by ecosystem gaps (kernels, serving, tooling)
- •Inception built in-house SFT/RLHF/RL pipelines for this architecture
- •Tradeoff: keeping IP closed vs. community contributions and easier on-prem adoption
- 34:05 – 38:13
Team structure, recursive self-improvement, and academia’s continued impact
They close by discussing how a small team can execute (agents, talent, compute as bottleneck) and how work is organized (serving current models vs. building next-gen). Stefano argues humans still matter for idea selection, and defends academia’s role by citing diffusion, FlashAttention, and DPO as impactful academic-origin innovations.
- •Org split: customer/platform team serving current model + research team building next model
- •Productivity: agents help; compute often the limiting factor more than headcount
- •Recursive self-improvement not fully here yet; human judgment still crucial
- •Academia can take contrarian bets; examples: diffusion origins, FlashAttention, DPO
