Vladimir Vapnik: Statistical Learning | Lex Fridman Podcast #5
Lex Fridman (host), Vladimir Vapnik (guest), Narrator
In this episode of Lex Fridman Podcast, featuring Lex Fridman and Vladimir Vapnik, Vladimir Vapnik: Statistical Learning | Lex Fridman Podcast #5 explores vladimir Vapnik on learning, intelligence, and the limits of deep learning Vladimir Vapnik discusses the philosophical and mathematical foundations of statistical learning, contrasting instrumentalism (prediction) with realism (understanding "God's laws"). He argues that modern machine learning overemphasizes brute-force prediction and deep learning, while neglecting conditional probabilities, invariants, and the role of a "teacher" in providing powerful predicates. Vapnik introduces his view that there are two mechanisms of learning—strong and weak convergence—with weak convergence relying on high‑level predicates like “swims like a duck” that dramatically reduce data requirements. He sees the central open problem as understanding intelligence: how good teachers generate such predicates, and how to formalize that process to achieve learning with far fewer examples.
Vladimir Vapnik on learning, intelligence, and the limits of deep learning
Vladimir Vapnik discusses the philosophical and mathematical foundations of statistical learning, contrasting instrumentalism (prediction) with realism (understanding "God's laws"). He argues that modern machine learning overemphasizes brute-force prediction and deep learning, while neglecting conditional probabilities, invariants, and the role of a "teacher" in providing powerful predicates. Vapnik introduces his view that there are two mechanisms of learning—strong and weak convergence—with weak convergence relying on high‑level predicates like “swims like a duck” that dramatically reduce data requirements. He sees the central open problem as understanding intelligence: how good teachers generate such predicates, and how to formalize that process to achieve learning with far fewer examples.
Key Takeaways
Distinguish prediction from understanding in machine learning.
Vapnik argues that most current ML is instrumentalist—focused on finding rules that predict well—whereas deeper understanding requires modeling conditional probabilities and “how God plays dice,” not just learning classifiers.
Get the full analysis with uListen AI
Use mathematics to uncover structures humans cannot intuit.
He maintains that the strongest human intuition resides in well‑chosen axioms; once those are set, following equations often reveals simple, beautiful principles that intuition alone would likely miss.
Get the full analysis with uListen AI
Incorporate predicates and invariants to reduce data needs dramatically.
Vapnik’s weak convergence mechanism uses high-level predicates (e. ...
Get the full analysis with uListen AI
Focus on constructing good admissible sets of functions, not just fitting models.
Statistical learning theory assumes a given hypothesis space, but Vapnik emphasizes that the real hard problem is building an admissible set: small VC dimension yet rich enough to contain a good solution, guided by invariants from data.
Get the full analysis with uListen AI
Reconsider the role and necessity of deep learning architectures.
He criticizes deep learning as largely interpretive “fantasy,” noting that mathematics doesn’t need neurons per se and that representer theorems point to shallow networks as sufficient optima in many learning formulations.
Get the full analysis with uListen AI
Study teachers to understand intelligence.
Vapnik sees the core of intelligence in what great teachers do—producing insightful predicates like “play like a butterfly” that instantly reshape a learner’s behavior—yet notes that we have almost no formal theory of this process.
Get the full analysis with uListen AI
Formulate concrete challenges to probe intelligence and efficiency.
He proposes benchmarks such as matching deep learning’s digit recognition performance using 100× fewer examples by introducing well-chosen invariants, framing this as a litmus test for progress on the intelligence problem.
Get the full analysis with uListen AI
Notable Quotes
“The goal of machine learning is to find the rule for classification. That is true, but it is an instrument for prediction. For understanding, I need conditional probability.”
— Vladimir Vapnik
“The best human intuition, it is putting in axioms, and then it is technical where to see where the axioms take you.”
— Vladimir Vapnik
“In weak convergence mechanism you can use predicates—that’s what ‘play like a butterfly’ is—and it will immediately affect your playing.”
— Vladimir Vapnik
“The most difficult problem is to create the admissible set of functions… This was out of consideration.”
— Vladimir Vapnik
“I think that this [deep learning] is fantasy. Everything which… like deep learning, like features… they are not really want of the problem.”
— Vladimir Vapnik
Questions Answered in This Episode
How could we formally model the process by which a great teacher invents powerful predicates like “play like a butterfly” or “swims like a duck”?
Vladimir Vapnik discusses the philosophical and mathematical foundations of statistical learning, contrasting instrumentalism (prediction) with realism (understanding "God's laws"). ...
Get the full analysis with uListen AI
What practical steps can current ML researchers take to integrate invariants and weak convergence mechanisms into mainstream learning algorithms?
Get the full analysis with uListen AI
Are there concrete examples where shallow, theoretically motivated models can match or outperform deep networks with far less data, and what do they teach us?
Get the full analysis with uListen AI
How might a theory of intelligence that goes beyond imitation (à la Turing) incorporate Vapnik’s ideas about predicates, teachers, and shared ‘ground truths’?
Get the full analysis with uListen AI
What kinds of empirical experiments could distinguish between problems that truly require massive data and those that mainly need better invariants and hypothesis spaces?
Get the full analysis with uListen AI
Transcript Preview
The following is a conversation with Vladimir Vapnik. He's the co-inventor of support vector machines, support vector clustering, VC theory, and many foundational ideas in statistical learning. He was born in the Soviet Union and worked at the Institute of Control Sciences in Moscow. Then, in the United States, he worked at AT&T, NEC labs, Facebook Research, and now is a professor at Columbia University. His work has been cited over 170,000 times. He has some very interesting ideas about artificial intelligence and the nature of learning, especially on limits of our current approaches and the open problems in the field. This conversation is part of MIT course on Artificial General Intelligence and the Artificial Intelligence podcast. If you enjoy it, please subscribe on YouTube or rate it on iTunes or your podcast provider of choice, or simply connect with me on Twitter or other social networks @lexfridman, spelled F-R-I-D. And now, here's my conversation with Vladimir Vapnik. Einstein famously said that God doesn't play dice.
Yeah.
You have studied the world through the eyes of statistics, so let me ask you in terms of the nature of reality, fundamental nature of reality, does God play dice?
We don't know some factors, and because we don't know some factors which could be important, it looks like God play dice, but we only should describe. In philosophy, they distinguish between two positions, positions of instrumentalism, where you're creating theory for prediction, and position of realism, where you're trying to understand what God did.
Can you describe instrumentalism and realism a little bit?
For example, if you have some mechanical laws, what is that? Is it law which true always and everywhere or it is law which allow you to predict position of moving element? The, uh, what, what you believe, you believe that it is God's law, that God created the world which adhere to this physical law-
Yeah.
... or it is just law for predictions?
And which one is instrumentalism?
For predictions.
Just predict.
If you believe that this is law of God-
Mm-hmm.
... and it's always true everywhere, that means that you're realist.
So you-
You're trying to re- to really, uh, understood, understand the God's thought.
So the way you see the world is, is an instrumentalist?
You know-
Absolutely.
... I'm working for some models, model of, uh, machine learning. So in this model, we can see, um, setting and we try to solve, resolve the setting to solve the problem. And you can do it in two different way. From the point of view of instrumentalist, and that's what everybody does now, because, uh, they say the goal of machine learning is to, uh, find the rule for classification.
Mm-hmm.
That is true, but it is instrument for prediction. But I can say the goal of, uh, machine learning is to, to learn about conditional probability, so how God play deuce, and he... if he play, what is probability for one, what is probability for another given situation. But for prediction, I don't need this. I need the rule.
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome