Yann LeCun: Dark Matter of Intelligence and Self-Supervised Learning | Lex Fridman Podcast #258

The following is a conversation with Yann LeCun, his second time on the podcast. He is the chief AI scientist at Meta, formerly Facebook, professor at NYU, Turing Award winner, one of the seminal figures in the history of machine learning and artificial intelligence, and someone who is brilliant and opinionated in the best kind of way, and so is always fun to talk to. This is a Lex Fridman podcast. To support it, please check out our sponsors in the description, and now here's my conversation with Yann LeCun. You co-wrote the article "Self-Supervised Learning: The Dark Matter of Intelligence," great title by the way, with Ishaan Misra. So let me ask, what is self-supervised learning and why is it the dark matter of intelligence?

I'll start by the dark matter part.

(laughs)

Uh, there is obviously a kind of learning that humans and animals are, uh, are doing that we currently are not reproducing properly with machines or with AI, right? So the most popular approaches to machine learning today are, or paradigms I should say, are supervised learning and reinforcement learning and they're extremely inefficient. Supervised learning requires many samples for learning anything, and reinforcement learning requires a ridiculously large number of trial and errors to, for, you know, a system to learn anything. Um, and that's why we don't have self-driving cars.

(laughs) That was a big leap from one to the other. Okay. So that to solve difficult problems, you have to have a lot of, uh, human annotation for supervised learning to work, and to solve those difficult problems with reinforcement learning, you have to have some way to maybe simulate that problem such that you can do that large scale kind of learning that reinforcement learning requires.

Right. So how is it that, you know, most teenagers can learn to drive a car in about 20 hours of, uh, practice, whereas, uh, even with millions of hours of simulated practice-

Mm-hmm.

... a self-driving car can't actually learn to drive itself properly? Um, and so obviously we're missing something, right? And, and it's quite obvious for a lot of people that, you know, the immediate response you get from many people is, "Well, you know, humans use their background knowledge to learn faster." And they're right. Now, how was that background knowledge acquired? And that's the big question. So now you have to ask, you know, how do babies in their first few months of life learn how the world works? Mostly by observation because they can hardly act in the world. Um, and they learn an enormous amount of background knowledge about the world that may be the, the basis of what we call common sense. Uh, this type of learning is not learning a task, it's not being reinforced for anything, it's just observing the world and figuring out how it, how it works. Building world models, learning world models. Um, how do we do this and how do we reproduce this in, in machines? So self-supervised learning is, you know, one instance or one attempt at trying to reproduce this kind of learning.