By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
The Twenty Minute VCModal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
At a glance
WHAT IT’S REALLY ABOUT
Inference becomes the new cloud as platforms replace commodity tokens
- Erik Bernhardsson frames inference as “the new cloud” and predicts the market will become enormous while splitting into many specialized product categories.
- He argues that selling inference alone becomes a commodity, so the winning strategy is a platform that bundles inference with sandboxes, post-training, reinforcement learning, and other adjacent tools.
- He describes GPU procurement as chaotic and says Modal’s value is aggregating supply across many vendors while pooling customer demand with custom infrastructure to deliver rapid scaling without customer capacity planning.
- He believes classic 90% software margins are structurally unlikely in today’s AI stack, but still expects meaningful long-term margin expansion for strong platforms.
- He anticipates an eventual 20–30% market pullback but remains highly bullish on long-term compute demand and says both open and frontier model usage will grow significantly.
IDEAS WORTH REMEMBERING
5 ideasPure inference commoditizes; the defensible play is bundling inference into a broader platform.
He argues that token-for-dollar inference is a race to the bottom with low switching costs, so providers need a broader suite—sandboxes, post-training, batch/low-latency modes, agentic workflows, and reinforcement learning—to create differentiation and capture margin.
Inference will be huge and will fragment into specialized products.
He says the market is becoming “the new cloud,” and because it’s grown far larger than he expected, it will split into many specialized product needs (e.g., low-latency vs batch, fine-tuning/post-training) even as a single platform can host multiple specialized offerings.
Modal’s operational edge is aggregating compute supply and smoothing demand across many customers.
He describes GPU procurement as chaotic—like an “open-air drug market”—and positions Modal’s value as doing the capacity deals and planning so customers don’t have to, supported by Modal’s custom-built scheduler/runtime/filesystem that treats compute as a pooled resource.
Compute planning is driven by predictable aggregate growth, not perfect foresight.
He says Modal forecasts demand using aggregate customer growth, citing ~34% month-on-month demand growth over the last 12 months and projecting 3–4 months ahead due to lead times; he also disputes the idea that capacity is simply unavailable if you’re willing to pay market-clearing prices.
AI infrastructure margins won’t look like classic SaaS, but there’s still significant profit opportunity.
He contends that 90%+ “software margins” are largely gone because revenue dollars now flow through many stack layers (apps → models → infra → cloud → Nvidia), but believes there’s still meaningful long-term margin potential (he mentions a long-run opportunity for ~40–60% margins for Modal).
WORDS WORTH SAVING
5 quotesDude, I, I hate that stuff. It's like, I, I don't get it. Like, you're eating glass, like, every day. Like, why are you doing it then?
— Erik Bernhardsson
I don't wanna sell this company, 'cause I would just go and start Modal2.
— Erik Bernhardsson
I would describe that as like an open-air drug market, but GPUs instead of, uh, drugs.
— Erik Bernhardsson
I think basically almost no company today has software margins a-anymore.
— Erik Bernhardsson
There's gonna be a pullback. I'm okay with that, 'cause I think the long-term opportunity is so fucking big. It's, like, so large.
— Erik Bernhardsson
High quality AI-generated summary created from speaker-labeled transcript.
