By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
The Twenty Minute VCModal CEO: Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
CHAPTERS
- 0:00 – 1:25
Modal’s scale and Erik’s long-term ambition
Erik frames the size of the inference opportunity and signals he’s comfortable with volatility because he believes the long-term market is enormous. Harry introduces Erik and positions the conversation around compute, inference, and what others miss.
- •Erik says there’s an opportunity to create a $100B company with specialized models in inference
- •Erik anticipates a pullback and is okay with it given the long-term upside
- •Harry introduces Erik as Modal’s CEO and sets up the episode’s focus on inference and compute
- 1:25 – 3:43
Early love of programming and the confidence gap before founding
Erik describes learning to code at eight and how programming became his core hobby. He explains why he didn’t start a company earlier, pointing to confidence and life experience as key factors.
- •Erik started coding as a child and built games and competed in programming later as a teen
- •He says he lacked confidence earlier in life, which delayed founding
- •He frames founding Modal as driven by seeing an “obvious” market opportunity
- •He argues founders in their 20s can have too much confidence, and experience matters
- 3:43 – 5:33
Rejecting the “eating glass” narrative and not wanting to sell
Erik pushes back on the idea that building a company must be constant suffering. He says he enjoys the work and would rather keep building than sell, even if there were a major liquidity event.
- •Erik says he hates the ‘eating glass’ framing and doesn’t relate to it
- •He describes enjoying day-to-day company building despite hard moments
- •Erik says he doesn’t want to sell because he’d just start another company
- •Harry shares a similar perspective about not selling his own business
- 5:33 – 8:03
Fundraising philosophy: relationships over decks, and valuation isn’t the goal
Erik explains he dislikes fundraising and has never made a pitch deck, emphasizing long-term relationships instead. He also argues that optimizing for the highest valuation can be counterproductive early on.
- •Erik says he hates fundraising, even though it can be enjoyable with the right people
- •He says he has never made a pitch deck; rounds happened after long relationship-building
- •He argues early-stage companies shouldn’t always maximize valuation or take more money than needed
- •He says raising less can keep investors “hungry” for the next round
- 8:03 – 11:43
What moats really are in early-stage startups—and why brand is rising in importance
Erik argues startups don’t truly have moats early; they win by moving faster. He and Harry discuss why brand and “vibes” matter more now, especially in AI infrastructure, and how Modal thinks about community and identity.
- •Erik defines moats as something that typically comes much later, not in the early years
- •He cites the idea of winning by speed rather than defensibility
- •He says brand value has increased and ties it to developer experience and consistent identity
- •Erik contrasts “cloneable” products with building a personality and movement around a company
- 11:43 – 13:32
Inference becomes ‘the new cloud’: why pure inference commoditizes and how platforms win
Erik says inference will be enormous, but selling tokens alone becomes a race to the bottom. He argues the winning approach is bundling inference with a broad set of adjacent products that create switching costs and differentiation.
- •Erik calls inference ‘the new cloud’ and expects a very large market
- •He says pure inference becomes commodity-like with minimal switching costs
- •He argues providers need breadth: sandboxes, post-training, agentic workflows, and different inference modes
- •He says the hard part is the infrastructure required to deliver this product suite
- 13:32 – 15:00
Where margins come from: the stack reality and why ‘software margins’ are fading
Erik explains why 90% gross margin software businesses are less common due to multiple layers splitting value. He still sees meaningful margin opportunity in inference by attaching higher-value products around the core service.
- •Erik says gross margin upside comes from building products around inference (workflows, sandboxes, post-training)
- •He argues margin is constrained by a multi-layer vendor stack taking a cut at each layer
- •He believes very high ‘classic software’ margins are largely gone
- •He targets a longer-term margin profile below old-school software margins
- 15:00 – 16:28
Fragmentation into specialized inference products—and Modal’s bet to host them under one roof
Erik predicts the inference market will fragment into many specialized products as it grows. He says Modal’s opportunity is to build and host many of those specialized products within a rebuilt, AI-native version of cloud.
- •Erik predicts specialization across latency, throughput, fine-tuning, post-training, and more
- •He clarifies he expects fragmentation by product, not necessarily by separate vendors for everything
- •He says Modal already supports more than LLM inference and wants to expand the product set
- •He frames the strategy as deconstructing and rebuilding cloud for the AI era
- 16:28 – 19:06
Inside the GPU scramble: ‘open-air drug market’ dynamics and Modal’s compute aggregation thesis
Erik describes the chaotic process of sourcing GPUs and how Modal shields customers from it. He explains Modal’s long-term goal of aggregating supply and demand across many providers into one efficiently managed compute pool.
- •Erik describes Nvidia’s conference environment as an ‘open-air drug market’ for GPUs
- •He says Modal works with many compute providers so customers don’t have to do GPU deal-making
- •He outlines a vision of controlling more of the world’s compute and managing it as pooled capacity
- •He says Modal’s systems smooth demand across many customers, reducing the need for customer capacity planning
- 19:06 – 21:14
Forecasting demand, lead times, and what happens to GPU prices over time
Erik explains how Modal forecasts GPU needs using aggregated customer demand and lead times of several months. He says capacity can be purchased if you’re willing to pay, and expects hardware prices to fall in the long run even if shortages persist near-term.
- •Erik says GPU lead times are about three to four months
- •He describes projecting demand forward based on aggregated growth rates and diversified customer usage
- •He says the market isn’t ‘out of capacity’ if you’re willing to pay more
- •He expects hardware prices to go down in the very long run, while noting uncertainty in timing
- 21:14 – 24:39
How early the compute boom is, plus the hardware landscape and Nvidia’s real advantage
Erik and Harry discuss how early AI adoption is and why demand may stay strong. Erik says Nvidia will keep growing while specialized chips also grow, and argues CUDA itself isn’t the moat—ecosystem support and ‘Nvidia-first’ tooling is.
- •Erik says AI is still very early and there’s significant latent demand
- •He notes constraints like data center build speed and broader capital conditions could matter
- •He expects both Nvidia and specialized chip companies to grow, not a strict winner-take-all
- •He argues CUDA is overstated as a moat; the broader ecosystem and tooling support is the advantage
- 24:39 – 26:47
Neo-clouds as partners and rivals: why moving up the stack is hard
Erik describes the relationship with neo-clouds as both cooperative and competitive. He’s skeptical that many can move effectively into inference because Modal has spent years building complex infrastructure, while neo-cloud scaling often hinges on financial execution.
- •Erik calls other startups ‘a friend and a foe’ simultaneously
- •He’s skeptical neo-clouds can move up the stack while staying excellent at their core business
- •He emphasizes Modal’s deep infra build: scheduler, runtime, filesystem, and large-scale workload placement
- •He says neo-cloud success often depends heavily on contracting, financing, and capacity procurement execution
- 26:47 – 29:52
Open models vs frontier models: control, licensing shifts, and geographic dynamics
Erik argues open-source models will remain important for teams that want control, fine-tuning, and predictable behavior. He discusses Europe’s challenges with aggregating talent and capital, and says Nvidia is smart to invest in building an open-model ecosystem.
- •Erik says open source will persist because many users want control, fine-tuning, and performance tuning
- •He notes open models are shifting toward licensing to capture value and fund better training
- •He questions whether Europe can aggregate capital and talent into single efforts at the needed scale
- •He says Nvidia is strategically building ecosystem strength, including supporting open-source model efforts
- 29:52 – 32:57
Specialized model businesses, Jev’s brand power, and why founders must ‘become a personality’
Erik and Harry discuss the unbundling into specialized models and why that can create large businesses. Erik makes a strong case that brand, community, and movement-building are major assets, and reflects on founder-led marketing as a growth lever.
- •Erik connects specialized models to market expansion and the creation of sub-markets
- •He says there’s $100B potential in specialized inference models when paired with strong distribution/brand
- •Erik praises Jev’s brand/community as a major asset and differentiator
- •He says founder-led marketing and direct engagement were important to Modal’s early growth
- 32:57 – 39:11
Becoming a platform: Sandboxes explosion, product timing, and admitting underinvestment in inference
Erik outlines Modal’s plan to expand from a product into a multi-product platform and explains how hard that prioritization is. He describes Sandboxes taking time to find its breakout moment, and says Modal initially underinvested in open-model inference before accelerating after market shifts.
- •Erik says improving customer experience requires more products and stronger go-to-market support
- •He describes Sandboxes as taking years to become significant, then accelerating sharply later
- •He frames infrastructure product-building as creating strong primitives before killer use cases arrive
- •He says Modal underinvested early in open-model inference and later ramped quickly to compete
- 39:11 – 48:47
Growth trajectory, post-training, why agents won’t kill engineers, and the data economy
Erik discusses Modal’s rapid growth and where new revenue lines could emerge, including training and post-training. He argues software engineering demand will persist as tools improve, shares Modal’s spend on coding agents, and calls training data a lucrative market.
- •Erik describes sustained rapid growth and says he expects major continued upside
- •He highlights training and post-training as potentially large future revenue lines, plus many adjacent infrastructure categories
- •He argues agents will increase software output and demand rather than eliminate engineers
- •He shares Modal spends about half a million per month on coding agents and expects usage to grow with new use cases
- •He says selling data (especially code environments) to labs is currently very lucrative
- 48:47 – 56:58
Owning ‘your own intelligence,’ model routing defensibility, pullback expectations, and the talent war
Erik takes a nuanced stance on whether companies should own their own models, emphasizing tradeoffs like stability, cost, and latency. He discusses model routing as a viable business but likely stronger as part of a platform, predicts a potential market pullback, and explains why employee secondaries matter amid intense competition for talent.
- •Erik says not every company needs to own its own intelligence; it depends on the use case
- •He argues fine-tuned open models provide consistency and control when underlying frontier models may change
- •He says model routing can be a standalone business today but may need to integrate into broader platforms long-term
- •Erik predicts a potential 20–30% pullback and says he’s comfortable with it
- •He supports employee secondaries with guardrails and says top new grads can receive extremely large offers from major labs
- 56:58 – 1:02:11
Quick-fire: TurboPuffer, investment caution, parenting, Scandinavian culture, and AI drug discovery
Erik answers rapid questions on what he’d bet on, what he’s changed his mind about, and what he’d avoid investing in. He closes with personal reflections on parenting and cultural attitudes, and shares excitement about AI-aided drug discovery—while acknowledging regulation will take time.
- •Erik names TurboPuffer as a favorite and imagines many adjacency expansions around databases
- •He says the sandbox space magnitude surprised him and continues to grow beyond expectations
- •He says he’s wary of companies dependent on continual equity fundraising without a path to commercialization
- •He discusses parenting as finding the right balance of pushing kids without overdoing it
- •He says drug discovery is one of the most exciting AI applications, though approvals will take time
