The Twenty Minute VCDavid Luan: Why Nvidia Will Enter the Model Space & Models Will Enter the Chip Space | E1169
CHAPTERS
- 0:00 – 0:57
From “papers” to “Apollo projects”: the next phase after Transformers
David opens with a thesis that OpenAI (and DeepMind) recognized early: the post-Transformer era would be driven less by publishing and more by attacking large, concrete scientific/engineering problems. He also previews his view that compute demand will keep rising because new training paradigms will soak up more resources.
- •Shift from academic paper progress to focused “solve the big problem” programs
- •Compute won’t hit diminishing returns in practice because new approaches will consume it
- •Tier-1 cloud providers have existential incentives to win the model layer
- 0:57 – 3:40
Google Brain as the “Bell Labs” era: what bottom-up research looks like
Harry asks about David’s Google Brain takeaways and how they shaped Adept. David describes Brain’s 2012–2018 dominance, talent density, and the mechanics of curiosity-driven, bottom-up research that produced foundational breakthroughs.
- •2012–2018 as a golden period of rapid deep learning breakthroughs
- •Brain’s concentration of researchers behind Transformers, diffusion, optimization methods
- •Bottom-up research: minimal near-term accountability, curiosity-driven exploration
- •How this research culture differs from product-driven engineering
- 3:40 – 5:22
Why Transformers changed everything—and why that shifted the playbook
David frames Transformers as the first broadly general architecture that replaced a zoo of task-specific model families. Once the “universal model” emerged, the frontier shifted from inventing new basic architectures to scaling and applying Transformers to bigger problems.
- •Transformers were invented at Google, not OpenAI
- •Pre-Transformer world: CNNs for vision, RNNs for text, RL/tree search for games
- •Transformers as a universal substrate across tasks
- •After 2017, progress becomes less about low-level modeling novelty and more about scaling/application
- 5:22 – 6:49
Why ChatGPT arrived years after Transformers: capability thresholds + packaging
Harry probes the gap between the 2017 breakthrough and mass consumer adoption. David argues it took both (1) models crossing a “minimum viable smartness” threshold and (2) consumer-friendly product packaging—where GPT-3 existed earlier but lacked a viral, consumer interface.
- •Incremental improvements 2017→2022 created a “boiling frog” effect
- •GPT-2 as an early “aha” for general text generation
- •Minimum viable smartness: capability must cross a compelling-experience threshold
- •Packaging matters: GPT-3 API was developer-only; ChatGPT made it consumer-playable
- 6:49 – 8:30
OpenAI lessons: mission-driven teams that tackle specific hard problems
David contrasts Google Brain’s federated research with OpenAI’s problem-centric execution. He describes OpenAI’s “choose a major unsolved problem and solve it” approach and how that informs Adept’s culture and structure.
- •OpenAI’s cultural shift: fewer isolated papers, more integrated “big team” initiatives
- •Examples: robot hand control, game-playing, scaling GPT to general reasoning/chat
- •“Apollo project” framing vs organic self-organization
- •How this structure influences how Adept builds and prioritizes
- 8:30 – 14:53
Compute, data, and the next scaling frontier: self-improvement loops & synthetic environments
Harry challenges the idea of diminishing returns from compute and asks whether data is the real bottleneck. David argues scaling curves remain predictable at the right lens (e.g., doubling compute), and that a second wave—models generating/collecting their own training signal via tools, simulations, and RL loops—will drive further gains and consume massive compute.
- •Diminishing returns per GPU vs predictable returns per compute doubling (log-like scaling)
- •Analogy to Moore’s Law: progress continues via scale-up + scale-out systems
- •Next frontier: models learning by interacting with environments (tool use, reflection, retries)
- •Example: math reasoning via theorem provers/Jupyter + iterative self-critique
- •Why this shift is happening now: base-model training costs reaching multi-billion-dollar levels
- 14:53 – 18:18
Agents vs chatbots: reliability, hallucinations, and emergent capability thresholds
The conversation turns from model improvement to what agents require to be useful in the real world. David explains why hallucinations can be helpful for creative chat but unacceptable for agents, then introduces “minimum viable capabilities” where new skills appear suddenly once scale crosses a threshold.
- •Agents and chatbots are “different species” with different requirements
- •Hallucinations: feature for creativity vs bug for operational workflows
- •Minimum viable capability: sudden emergent skills at certain scales (GPT-2 arithmetic example)
- •Why agents are hard: key work knowledge isn’t neatly captured in datasets
- 18:18 – 23:21
Reasoning and memory: what must be solved at the model layer vs the product layer
Harry asks about breakthroughs in reasoning and why memory remains challenging. David argues reasoning likely requires training regimes with environments, trials, and feedback (not just internet-scale next-token prediction), while long-term memory is often better handled by end applications as part of a broader system rather than as an intrinsic LLM feature.
- •Reasoning definition: composing known thoughts to produce new ones
- •Pure scaling alone won’t solve reasoning; needs environment interaction + feedback loops
- •Reasoning improvements must happen at model-provider level (model must change)
- •Memory split: short-term context window progress vs long-term preference/task memory
- •LLMs aren’t products; products are full systems that incorporate LLMs (e.g., travel preference memory)
- 23:21 – 31:46
Who wins the foundation-model layer: clouds, NVIDIA, Apple, and vertical integration pressure
David predicts a small set of long-term frontier providers due to cost and incentives. He explains why cloud providers “must” own strong model offerings, why NVIDIA faces pressure from in-house chips, and why chips and models will converge through vertical integration; he also discusses Apple’s edge advantage and a future where model providers become hot-swappable behind the interface.
- •Steady state: ~5–7 max-scale LLM providers due to economics
- •Model layer becomes a core computing primitive; controlling it controls downstream compute value
- •NVIDIA vs in-house silicon: TPU as proof that challengers can succeed with strong will
- •Vertical integration: chipmakers move up to models; model providers move down to chips
- •Apple’s leverage: running capable models at the edge for privacy/latency/cost
- •Interface ownership enables commoditized, swappable “big brain” providers behind Apple
- 31:46 – 35:53
What happens to independent model companies—and why Adept isn’t “just” a model seller
Harry asks about the end state for foundational model startups that may not afford ongoing frontier training. David predicts tier-1 clouds will sustain their own efforts, while independents must either attach to clouds or build huge economic flywheels; he positions Adept as an enterprise agent company building a vertically integrated slice from UX to model adaptations.
- •Clouds will fund/ensure their own frontier efforts because it’s existential
- •Independent model sellers face a narrow window before commoditization
- •Survival path: massive cash-flow flywheel (often via strong distribution/product)
- •Adept’s stance: not selling foundation models to developers; selling enterprise agents
- •Why agents need vertical integration: reliability, UX leverage, and model-layer tailoring
- •Enterprise workflows as edge cases (e.g., every Salesforce instance configured differently)
- 35:53 – 42:46
RPA vs agents: why the shift is disruptive, and how pricing and orgs may change
Harry pushes on whether agents are simply “RPA 2.0.” David distinguishes RPA as deterministic automation for repetitive, uniform tasks, versus agents as planning, re-evaluating systems akin to “full self-driving,” then explains why incumbents struggle due to business-model disruption and why copilots/teammates reshape pricing and team structure.
- •RPA analogy: robots following a painted line for consistent, high-volume tasks
- •Agents analogy: full self-driving—planning, adapting, re-evaluating at every step
- •Why incumbents may be disadvantaged: current sales/implementation model is slow and services-heavy
- •Adept’s approach: end users teach capabilities via SOPs, demonstrations, corrections; aim for self-serve over time
- •Pricing debate: “price per work” fits commoditized repetition; knowledge work value looks more like teammate augmentation
- •Org impact: “collapsing the talent stack”—humans become broader generalists supervising AI specialists
- 42:46 – 48:20
Enterprise adoption reality: experimental budgets, services growth, and productization waves
Harry asks whether enterprises are moving from experimentation to core budgets, and whether AI services firms will outgrow model providers. David argues most enterprise spend remains experimental (and he avoids low-quality experimental revenue), adoption will be long and uneven, and today’s services-led implementations will be productized into repeatable software companies that capture major value.
- •Enterprise AI adoption is mostly experimental; core-budget penetration is still early
- •Enterprises lag even on “mature” tech like cloud due to on-prem/mainframe inertia
- •Pragmatic revenue quality: avoid deals funded only by experimentation budgets
- •Services fill the “gulf” between base models and enterprise needs early on
- •Over time, successful services patterns get productized into scalable companies
- 48:20 – 54:18
Regulation, open vs closed, and why human-computer interaction is the missing ingredient
David warns more about regulatory capture than over-regulation, arguing incumbents may “pull up the ladder” by shaping rules. He defends open source as a way for the broader field to keep up, then claims the critical frontier is better human-computer interaction—richer than prompting—so humans can supervise, correct, and align increasingly capable systems.
- •Primary regulatory risk: capture that entrenches a few dominant players
- •Consequence: harder to build on open source; higher barriers for new AI entrants
- •Open vs closed: misuse risks are real, but open typically lags closed due to resources
- •AGI discussions can be “infinity reasoning”; focus on near-term path dependence
- •HCI as key: chat/prompting is too thin vs human collaboration on shared canvases/tools
- •Need better supervision/alignment interfaces to shape training data and system design
- 54:18 – 58:17
Quick-fire: speciation of products, misconceptions about automation, and the agent endgame
In the closing rapid-fire, David reiterates that agents and chatbots will diverge into distinct products and challenges the narrative that AI simply replaces human work step-by-step. He paints a five-year vision where agents become a higher-level interface to computing—and flags walled-garden lock-in as a major failure mode.
- •Changed mind: agents vs chatbots will split into separate product categories
- •Misconception: AI will neatly automate and replace each human capability
- •Vision: agents as a “GUI→agents” leap, enabling higher-level goal-based interaction
- •Failure mode: incumbents’ walled gardens prevent cross-domain agents from working
- •Market size: agents address far more work than traditional RPA
