CHAPTERS
- 0:05 – 0:38
Listener Q&A kickoff and framing the GPU crunch question
Sarah and Elad open the episode in a Q&A format and tee up a common founder concern: access to compute for training and serving modern AI models. They introduce the episode’s first big topic—the ongoing GPU bottleneck—and why it matters for companies of all sizes.
- •Episode format: answering listener questions across tech and AI
- •Compute access is a recurring constraint for startups experimenting with models
- •Sets up the central question: what’s happening with the GPU crunch
- 0:38 – 2:23
Why AI demand collides with a fragile GPU supply chain
Sarah explains the structural reasons GPU supply can’t quickly respond to explosive AI demand. She walks through concentrated chip production, dependency on a few foundries, and how physical manufacturing limits create slow capacity expansion.
- •GPU supply is concentrated (NVIDIA, AMD) with NVIDIA leading at the high end
- •Pandemic-era supply disruptions still reverberate
- •TSMC/foundry dependency makes scaling slow and capital intensive
- •Semiconductor manufacturing yields are complex and sensitive to many variables
- 2:23 – 3:18
How severe is the shortage—and what does near-term availability look like?
Elad presses on the magnitude of the gap (2x vs 10x), and Sarah explains why it’s hard to measure in a normal market sense. They discuss lead times, sell-outs at major cloud providers, and large buyers sourcing capacity wherever they can.
- •No clear way to measure “true” demand due to constrained supply and pricing dynamics
- •Delivery timelines: small quantities sooner, larger batches later
- •Major cloud providers sold out for meaningful capacity well into next year
- •Large buyers (e.g., big cloud/lab players) source GPUs from alternative providers
- 3:18 – 4:30
Will the bottleneck persist? Tooling and manufacturing blockers
Sarah argues the bottleneck may not resolve quickly because demand keeps rising and manufacturing has additional hidden constraints. She highlights specialized equipment in the GPU assembly pipeline as another limiting factor beyond chip design and foundry capacity.
- •Core question: will demand growth continue to outpace physical capacity expansion?
- •Specialized manufacturing/assembly tooling can become the true bottleneck
- •Large labs’ plans to scale model training by orders of magnitude sustain demand pressure
- •Inference growth can rival or exceed training demand over time
- 4:30 – 6:16
Second-order effects: new GPU clouds and alternative AI silicon gain pull
Elad outlines how scarcity reshapes the startup landscape: new intermediaries aggregate and rent GPU capacity, and non-NVIDIA silicon becomes more attractive. He points to dedicated GPU cloud players and AI chip startups seeing increased adoption due to urgency.
- •GPU scarcity enables new “GPU-as-a-service”/aggregation businesses (e.g., CoreWeave-style models)
- •Crypto-era GPU capacity may shift toward AI workloads when economics favor it
- •AI chip startups (e.g., Cerebras/Groq-style) see stronger demand as substitutes
- •Shortages create market openings for new infrastructure providers
- 6:16 – 8:11
Compute scarcity pushes efficiency: routing, distillation, and smarter training mixes
Sarah describes how blocked scaling forces the ecosystem to prioritize efficiency techniques that were previously undervalued. They discuss approaches like model routing, distillation, and better data/training strategies to reduce compute while maintaining quality.
- •Scarcity incentivizes efficiency rather than pure scaling along one dominant recipe
- •Dynamic routing to smaller/cheaper models (e.g., “frugal” approaches)
- •Distillation and model compression regain importance
- •Smarter data and training mixtures can reduce compute for comparable quality
- 8:11 – 10:48
AI adoption is still early: enterprise products likely 1–2 years behind
Elad argues that despite the hype, real enterprise-scale AI product deployment is barely underway. He explains enterprise planning/prototyping cycles and why a larger demand ramp (including for compute) may still be ahead.
- •Transformers created new capabilities; the adoption wave is still young
- •Most adoption so far is AI-native firms + early startup and founder-led incumbent experiments
- •Enterprise planning cycles are long; real at-scale deployments lag by 1–2 years
- •Future adoption implies continued infrastructure/semiconductor demand growth
- 10:48 – 12:58
What ‘agents’ mean: from chat to autonomous planning and tool use
The conversation shifts to AI agents, prompted by hackathons and current founder interest. Sarah defines agents as systems that plan and take actions—often using tools/APIs or code—to complete multi-step tasks, spanning consumer and enterprise use cases.
- •Agents go beyond chat: planning + autonomous action toward a goal
- •Tool use (APIs, external systems) is central to agent utility
- •Consumer examples: personalized assistants and web agents
- •Enterprise examples: multi-step workflows like legal and analytics work
- 12:58 – 15:12
Vertical vs broad agents: why focus usually wins early
Elad compares agent-building to prior tech waves and argues that successful products often start narrow and expand later. Broad “do everything” agents struggle to delight users initially, while targeted agents can nail a specific workflow and then generalize.
- •Pattern: vertical products often win first, then broaden into platforms
- •Founders frequently pitch overly broad assistants without a crisp initial use case
- •Product strategy: pick 1–2 tasks and do them exceptionally well
- •Principle: delight a small group vs. indifference from a large group
- 15:12 – 18:24
Research vs product vs tooling: three paths to winning in agents
Sarah highlights tension between general-purpose research agendas and shipping task-completing products, using bug-fixing/code generation as an example of tractable progress. Elad adds a third path: building enabling infrastructure/tooling for others, and explains when platform approaches work.
- •General tech can still benefit from task focus to drive measurable progress
- •Task completion enables techniques like testing/search/verification loops
- •Third path: build agent infrastructure/tooling rather than the end application
- •Platform timing matters: some platforms work early (payments), others after scale (auth)
- 18:24 – 20:49
2024–2025 tech market outlook: ‘four markets,’ AI strength, and non-AI shakeout
They close with predictions for tech markets, fundraising, and valuations. Elad forecasts a strong AI market but significant carnage among 2021-vintage non-AI unicorns, with ripple effects across hiring, real estate, and venture capital over the next few years.
- •AI remains its own hot market; “expensive now, cheap in hindsight” dynamic
- •Non-AI 2021-era unicorns split: some fail, some stagnate, some grow through
- •Knock-on effects: easier hiring, commercial real estate pressure, VC portfolio hits
- •Time-delayed impact likely plays out across 2024–2025+ cycles
- 20:49 – 23:24
Founder advice on valuations, burn, and choosing what’s worth your time
Sarah advises companies to mentally detach from peak-era valuations to avoid contorted financing decisions. Elad argues valuation resets are survivable, but high burn with weak revenue traction is dangerous—and founders should treat time as their most precious asset.
- •Avoid anchoring to 2021 valuations; down rounds/valuation resets are common
- •The bigger risk: burning large amounts of capital without business results
- •Adjust cost structures early rather than waiting until options disappear
- •Founders should evaluate opportunity cost—time is the scarce resource
- 23:24 – 23:57
Wrap-up and where to follow the show
They end by thanking listeners for questions and sharing where to find the podcast, video episodes, and transcripts. The closing reinforces the show’s ongoing Q&A engagement loop with the audience.
- •Thanks to listeners who submitted questions
- •Social and distribution: Twitter/X, YouTube, Apple Podcasts, Spotify
- •Transcripts and email signup available on the show website
