Cohere's Chief AI Officer, Joelle Pineau: Why Scaling Laws Will Continue & Future of Synthetic Data
Joelle Pineau (guest), Harry Stebbings (host)
In this episode of The Twenty Minute VC, featuring Joelle Pineau and Harry Stebbings, Cohere's Chief AI Officer, Joelle Pineau: Why Scaling Laws Will Continue & Future of Synthetic Data explores joelle Pineau on scaling laws, RL, enterprise AI and risk realism Joelle Pineau, Chief Scientist at Cohere and longtime AI researcher, discusses why scaling laws continue to hold, why reinforcement learning (RL) remains fundamental yet inefficient, and how algorithmic breakthroughs drive non‑linear progress in AI.
Joelle Pineau on scaling laws, RL, enterprise AI and risk realism
Joelle Pineau, Chief Scientist at Cohere and longtime AI researcher, discusses why scaling laws continue to hold, why reinforcement learning (RL) remains fundamental yet inefficient, and how algorithmic breakthroughs drive non‑linear progress in AI.
She emphasizes that AI’s real value will come from enterprise integration, efficiency and human–AI complementarity, not from near‑term AGI or extreme existential risk scenarios.
Pineau highlights the growing importance and cost of specialized and synthetic data, the security and impersonation risks of AI agents, and the need for efficient, on‑prem models that respect data confidentiality.
Throughout, she argues for open research, diverse global development of models, and a pragmatic focus on productivity gains, scientific discovery, and realistic regulation over speculative doomsday narratives.
Key Takeaways
Scaling laws are still reliable, but need algorithms to unlock real leaps.
Throwing more compute and data generally improves models in a roughly linear way, but step‑change progress comes from algorithmic innovations like transformers, Adam, and structured reasoning; betting against scaling laws has mostly been wrong so far.
Get the full analysis with uListen AI
Reinforcement learning remains conceptually essential but is highly inefficient today.
RL’s sequential decision‑making compounds errors and requires interactive environments or simulators rather than static data, making it expensive and sample‑inefficient; it works well where reward functions are precise (games, math), but we’re far from using RL to shape social behavior or reach AGI.
Get the full analysis with uListen AI
Enterprise value comes more from 10x productivity than outright job replacement.
Pineau argues AI will most usefully amplify most employees’ output (e. ...
Get the full analysis with uListen AI
Data—not just compute—is becoming a major cost and strategic lever.
Simple labeling tasks are largely solved; high‑value data now requires domain experts, domain‑specific business logic, and synthetic environments for agents, all of which are expensive to design, curate, and integrate into training pipelines.
Get the full analysis with uListen AI
Synthetic data can be powerful, but careless use causes distribution collapse.
When models train predominantly on their own outputs in open‑ended domains like language or images, diversity erodes and quality degrades; in more structured domains (Go, chess, or carefully diversified code), synthetic data can scale with far less degradation.
Get the full analysis with uListen AI
AI agents introduce new security risks, especially impersonation.
Beyond LLM jailbreaking and prompt injection, agentic systems create attack surfaces where malicious agents can impersonate legitimate entities and take unauthorized actions; rigorous standards, red‑teaming, and sometimes offline operation will be needed to manage these risks.
Get the full analysis with uListen AI
Team composition and openness matter more than just ‘buying Galacticos’.
A few world‑class experts are valuable, but high‑performing AI teams also require strong executors and social “glue” roles, clear focus, and open circulation of ideas; over‑concentration on superstar hires or closed research ecosystems can actually reduce innovation.
Get the full analysis with uListen AI
Notable Quotes
“The scaling laws have been remarkably robust. I wouldn’t bet against them.”
— Joelle Pineau
“Where we’re maybe getting a little bit ahead is thinking that just RL out of the box is going to give us AGI.”
— Joelle Pineau
“Can most of your employees do 10X the amount of work with AI versus on their own? That, to me, is actually a better barometer.”
— Joelle Pineau
“I don’t have a lot of patience as a scientist for people who are predicting the extremist scenarios, the catastrophic risks of AI.”
— Joelle Pineau
“This thought that you can just like close this down is absolutely false… It’s a mistake from a point of view of fostering innovation.”
— Joelle Pineau
Questions Answered in This Episode
How can we systematically discover the next ‘transformer‑level’ algorithmic breakthroughs rather than relying on serendipity?
Joelle Pineau, Chief Scientist at Cohere and longtime AI researcher, discusses why scaling laws continue to hold, why reinforcement learning (RL) remains fundamental yet inefficient, and how algorithmic breakthroughs drive non‑linear progress in AI.
Get the full analysis with uListen AI
What concrete methods could improve the sample efficiency of reinforcement learning enough for complex, real‑world tasks?
She emphasizes that AI’s real value will come from enterprise integration, efficiency and human–AI complementarity, not from near‑term AGI or extreme existential risk scenarios.
Get the full analysis with uListen AI
How should enterprises design workflows and roles when most employees can theoretically be 10x more productive with AI tools?
Pineau highlights the growing importance and cost of specialized and synthetic data, the security and impersonation risks of AI agents, and the need for efficient, on‑prem models that respect data confidentiality.
Get the full analysis with uListen AI
What technical and governance mechanisms could reliably verify the identity and legitimacy of AI agents operating across the web?
Throughout, she argues for open research, diverse global development of models, and a pragmatic focus on productivity gains, scientific discovery, and realistic regulation over speculative doomsday narratives.
Get the full analysis with uListen AI
Where is the line between productive synthetic data generation and harmful distribution collapse, and how can we monitor that in large‑scale training pipelines?
Get the full analysis with uListen AI
Transcript Preview
The scaling laws have been remarkably robust. There's a lot we don't know yet in terms of the vulnerability of these systems.
Today, we have one of the leading minds in AI, Joelle Pino. Joelle is the chief scientist at Cohere. If you don't need to buy the Galacticos, why do you have, like, an Andrew Tulloch, a Daniel Gross, an Alex Huang, and the Galacticos assembling?
I used to be quite skeptical that neural networks were necessarily the ultimate solution to machine learning. I seem to be quite wrong on this one.
Knowing what you know, what do you not let your children do?
Ah. (laughs) Eat too much sugar. I don't have a lot of patience as a scientist for people who are predicting the extremist scenarios, the catastrophic risks of AI. You know, AI becomes our overlord kind of scenario.
If I gave you $10 billion, what would you spend it on first? Ready to go? (instrumental music plays) Joelle, it is so great to have you in the studio. I've heard many great things from Nick, Aidan, Schreppe. So, thank you so much for joining me.
Thank you. Happy to be here.
Now, you spent over six years at Meta, and I want to start there because it's a very transformative time and place. What are the biggest takeaways for you from that time? And how did that shape your mindset to how you think today?
Well, I was there from t- 2017 to 2025. And you have to see just how much AI changed over that period of time. And what we were really focused on is fundamental AI research. Um, and what, you know, one thing that I've learned is just sometimes how long it takes to prove out a hypothesis. We feel like AI is moving at the speed of lightning. But in fact, there's some things that it just takes a few years to mature, to get the right optimizer, the right compute, the right data for that to really make a difference.
I look at where we are today, and everyone kind of goes, "It's here, it's here, it's here."
Yeah.
And then you actually look at what a lot of the leaders have been saying recently, where it's like, actually, you know, um, Andrej was saying, "It's not the year of the agents, it's the decade of agents."
(laughs)
Sam's kind of pulling back too.
Yeah.
Have, have we got over our skis and we're actually kind of all pulling back, realizing that time is the factor we need to rely on?
Well, I'll give you an example. You know, I've been in research for, for a couple decades now. I've been working on reinforcement learning for over 20 years. And suddenly, everyone's talking about reinforcement learning, you know? (laughs) Since the advent of reasoning models, agents and so on. So, you know, sometimes you have to be a little patient with these ideas. And the right algorithmic tweak, the right context, the right problem domains just opens up the magic.
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome