The Twenty Minute VC

Aidan Gomez: What No One Understands About Foundation Models | E1191

Aidan Gomez is the Co-founder & CEO at Cohere, the leading AI platform for enterprise, having raised over $1BN from some of the best with their last round pricing the company at a whopping $5.5BN. Prior to Cohere, Aidan co-authored the paper “Attention is All You Need,” which introduced the groundbreaking Transformer architecture. He also collaborated with a number of AI luminaries, including Geoffrey Hinton and Jeff Dean, during his time at Google Brain, where the team focused their efforts on large-scale machine learning. ----------------------------------------------- Timestamps: (00:00) Intro (00:45) Childhood & Backround (04:29) Is More Compute the Only Path to Better Performance? (08:07 ) Can Anyone Afford to Stay in the AI Race Besides Tech Giants? (13:44) Is AI Heading Toward a Race to the Bottom? (16:55) Will Companies Keep Building Their Own Chips? (18:30) Is Model Progression Outpacing Compute Advancement? (19:41) Early Challenges in Accessing Compute Chips (23:48) Are We Underestimating the Short-Term Impact of AI Advancements? (27:06) Is It Too Late for Startups to Enter the AI Model Space? (27:55) AI Development: The Exponential Rise in Costs (30:40) Will Cloud Giants Continue Acquiring Smaller AI Model Providers? (35:10) Is OpenAI Prioritizing AGI Over Practical Products? (48:29) What's the Biggest Overlooked Factor in AI's Future? (50:09) Concerns About a Future Where AI Replaces Human Interaction (54:20) What Will AI Do in Three Years That It Doesn't Do Today? (55:48) Quick-Fire Round ----------------------------------------------- In Today’s Episode with Aidan Gomez We Discuss: 1. Compute vs Data: What is the Bottleneck: Does Aidan believe that more compute will result in an equal increase in performance? How much longer do we have before it becomes a case of diminishing returns? What does Aidan mean when he says “he has changed his mind massively on the role of data”? What did he believe? How has it changed? 2. The Value of the Model: Given the demand for chips, the consumer need for applications, how does Aidan think about the inherent value of models today? Will any value accrue at the model layer? How does Aidan analyze the price dumping that OpenAI are doing? Is it a race to the bottom on price? Why does Aidan believe that “there is no value in last year’s model”? Given all of this, is it possible to be an independent model provider without being owned by an incumbent who has a cloud business that acts as a cash cow for the model business? 3. Enterprise AI: It is Changing So Fast: What are the biggest concerns for the world’s largest enterprises on adopting AI? Are we still in the experimental budget phase for enterprises? What is causing them to move from experimental budget to core budget today? Are we going to see a mass transition back from Cloud to On Prem with the largest enterprises not willing to let independent companies train with their data in the cloud? What does AI not do today that will be a gamechanger for the enterprise in 3-5 years? 4. The Wider World: Remote Work, Downfall of Europe and Relationships: Given humans spending more and more time talking to models, how does Aidan reflect on the idea of his children spending more time with models than people? Does he want that world? Why does Aidan believe that Europe is challenged immensely? How does the UK differ to Europe? Why does Aidan believe that remote work is just not nearly as productive as in person? ----------------------------------------------- Subscribe on Spotify: https://open.spotify.com/show/3j2KMcZTtgTNBKwtZBMHvl?si=85bc9196860e4466 Subscribe on Apple Podcasts: https://podcasts.apple.com/us/podcast/the-twenty-minute-vc-20vc-venture-capital-startup/id958230465 Follow Harry Stebbings on Twitter: https://twitter.com/HarryStebbings Follow Aidan Gomez on Twitter: https://twitter.com/aidangomez Follow 20VC on Instagram: https://www.instagram.com/20vchq Follow 20VC on TikTok: https://www.tiktok.com/@20vc_tok Visit our Website: https://www.20vc.com Subscribe to our Newsletter: https://www.thetwentyminutevc.com/contact ----------------------------------------------- #20vc #harrystebbings #aidangomez #cohere #openai #venturecapital #founder #computing

Aidan GomezguestHarry Stebbingshost

Aug 18, 20241h 3mWatch on YouTube ↗

WHAT IT’S REALLY ABOUT

Aidan Gomez Explains Foundation Models, Data, and AI’s Real Future

Aidan Gomez, cofounder and CEO of Cohere and coauthor of the Transformer paper, discusses the economics, technical progress, and product landscape of large AI foundation models.
He argues that while scaling compute reliably improves models, the real frontier is data quality, new methods for reasoning, and efficient smaller models tailored to enterprises.
Gomez predicts a world of multiple horizontal and vertical models, falling inference costs, tight margins at the model layer, and major value capture at both the chip and application layers.
He is optimistic about AI’s role in productivity, agents, robotics, and copilots for workers, while dismissing doomsday scenarios and emphasizing trust, privacy, and deployment models for enterprises.

IDEAS WORTH REMEMBERING

5 ideas

Scaling models with more compute works, but it’s inefficient and economically constrained.

Bigger models almost always perform better, but each incremental gain requires exponentially more compute and cost; this favors tech giants unless startups differentiate via data, algorithms, and efficiency.

High-quality and synthetic data are now the primary drivers of model improvement.

Open-source gains largely come from better data filtering, weighting, and synthetic generation; models are extremely sensitive to data quality, making curation and task-specific datasets a competitive edge.

We’re heading toward a multi-model world combining large general models and small specialized ones.

Teams prototype with powerful general models, then distill or fine-tune down to smaller, cheaper models optimized for specific tasks or verticals, creating an ecosystem rather than a single-model monopoly.

The model API layer is becoming commoditized, with margins squeezed by price cuts and open source.

With OpenAI price dumping and Meta releasing strong open models for free, selling “just models” will be a low-margin business; durable value is more likely at the chip and application/product layers.

Enterprise adoption hinges on trust, privacy, and deployment flexibility—not just raw capability.

Large customers resist training on their data and fear IP leakage, so vendors must support private deployments (e.g., in-VPC, on-prem, multi-cloud) and strong guarantees that customer data isn’t used for training.

WORDS WORTH SAVING

5 quotes

There’s no market for last year’s model.

— Aidan Gomez

It’s definitely true that if you throw more compute at the model, if you make the model bigger, it’ll get better. It’s also the dumbest way to improve models.

— Aidan Gomez

Pretty much all of the major gains that we’ve seen in the open source space have come from data improvements.

— Aidan Gomez

If you’re only selling models, it’s going to be a really tricky game… it’s going to be like a zero-margin business.

— Aidan Gomez

You might want your children to be speaking to an extremely empathetic, extraordinarily intelligent and knowledgeable, safe intelligence that can teach them things and doesn’t get tired of them.

— Aidan Gomez

Impact of gaming, early life, and learning on founder mindsetScaling laws, compute costs, and limits of “just make models bigger”Data quality, synthetic data, and method/algorithmic innovationsMarket structure: horizontal vs vertical models, pricing, and commoditizationCloud, chips, and the risk of becoming a “subsidiary” of hyperscalersEnterprise AI adoption: trust, privacy, hallucinations, and RAGFuture directions: agents, robotics, work displacement, and productivity growth

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.