Lex Fridman PodcastTomaso Poggio: Brains, Minds, and Machines | Lex Fridman Podcast #13
EVERY SPOKEN WORD
115 min read · 22,643 words- 0:00 – 15:00
The following is a…
- LFLex Fridman
The following is a conversation with Tomaso Poggio. He's a professor at MIT, and is a director of the Center for Brains, Minds, and Machines. Cited over 100,000 times, his work has had a profound impact on our understanding of the nature of intelligence in both biological and artificial neural networks. He has been an advisor to many highly impactful researchers and entrepreneurs in AI, including Demis Hassabis of DeepMind, Amnon Shashua of Mobileye, and Christof Koch of the Allen Institute for Brain Science. This conversation is part of the MIT course on artificial general intelligence, and the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter, @LexFridman, spelled F-R-I-D. And now, here's my conversation with Tomaso Poggio. You've mentioned that in your childhood, you've developed a fascination with physics, especially the theory of relativity, and that Einstein was also a childhood hero to you. What aspect of Einstein's genius, the nature of his genius, do you think was essential for discovering the theory of relativity?
- TPTomaso Poggio
You know, Einstein was, uh, a hero to me, and I'm sure to many people, because he was able to make, uh, uh, of course, a major, major contribution to physics with, simplifying a bit, just a gedanken experiment, a thought experiment.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
You know, imagining, uh, communication with lights between a stationary observer and somebody on a train.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
And, uh, I thought that, um, you know, the, the, the, the fact that just with the force of y- of his thought, of his thinking, of his mind, it could get to some- something so deep in term of physical reality, how time depend on space and speed, is, was something absolutely fascinating. It was the power of intelligence, the power of the mind.
- LFLex Fridman
Do you think the ability to imagine, to visualize as he did, as a lot of great physicists do, do you think that's in all of us human beings? Or is there something special to that one particular human being?
- TPTomaso Poggio
I think, uh, you know, a- all of us can learn and have, uh, in principle similar b- breakthroughs. Uh, there is lesson to be learned from Einstein. Uh, he was one of five PhD students at ETH, uh, the Eidgenössische Technische Hochschule in, uh, Zurich, in physics, and he was the worst of the five. The only one who d- did not get an academic position when, uh, he, he graduated, when he finished his PhD, and he went to work, as everybody knows, for the patent office. And so it's not so much that he worked for the patent office, but the fact that obviously he was smart, but he was not a top student. O- obviously, he was the anticonformist, he was not thinking in the traditional way that probably his teachers and the other students were doing. So there is a lot to be said about, uh, you know, trying to be... to do the opposite or something quite different from what other people are doing. That's certainly true for the stock market. Never...
- LFLex Fridman
(laughs)
- TPTomaso Poggio
Never buy if everybody's buying. (laughs)
- LFLex Fridman
And also true for science.
- TPTomaso Poggio
Yes.
- LFLex Fridman
So you've also mentioned, staying on the theme of physics, that you were excited at a young age by the mysteries of the universe that, uh, physics could uncover. Such, as I saw mentioned, the possibility of time travel.
- TPTomaso Poggio
(laughs)
- LFLex Fridman
So the most out-of-the-box question I think I'll get to ask today, do you think time travel is possible?
- TPTomaso Poggio
Well, it would be nice if it were possible right now. Uh, you know, in science, you never say no, um...
- LFLex Fridman
But your understanding of the nature of time.
- TPTomaso Poggio
Yeah, it's very likely that it's not possible to travel in time. Um, you may be able to travel forward in time if we can, for instance, freeze ourselves or, uh, you know, go on some spacecraft traveling close to the speed of light. But in terms of actively traveling, for instance, back in time, I find probably very unlikely.
- LFLex Fridman
So do you still hold the, the underlying dream of the engineering intelligence that we'll build systems that are able to do such huge leaps, like discovering the kind of mechanism that would be required to travel through time? Do you still hold that dream or is- or echoes of it from your childhood?
- TPTomaso Poggio
Yeah, I, uh, you know, I don't think whether... Uh, there are certain problems that probably cannot be solved depending what, uh, what you believe about the physical reality, like, uh, you know, maybe totally impossible to create energy from nothing or to travel back in time. But, uh, um, about making machines that can, uh, think as well as w- we do or better, or more likely, especially in the short and mid-term, help us think better, which is in a sense is happening already with the computers we have, and it will happen more and more. Well, that I certainly believe, and I don't see in principle why computers at some point, uh, could not become more intelligent than we are. Although the word intelligence...... is a tricky one and one we should discuss- (laughs)
- LFLex Fridman
Yeah, for sure.
- TPTomaso Poggio
... what I mean with that. (laughs)
- LFLex Fridman
Uh, in- intelligence, consciousness-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... words like love, is, all these are very, uh-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... need to be disentangled. So, you've mentioned also that you believe the problem of intelligence is the greatest problem in science, greater than the origin of life and the origin of the universe. You've also, uh, in the talk I've listened to, uh, said that you're open to arguments against, uh, against you. So, uh, what do you think is the most captivating aspect of this problem of understanding the nature of intelligence? Why does it captivate you as it does?
- TPTomaso Poggio
Well, originally, I think one of the motivation that I had as a, I guess, a teenager, when I was infatuated with theory of relativity, was really that I- I found that there was, uh, the problem of time and space and general relativity, but there were so many other problems of the same level of difficulty and importance that I could ... even if I were Einstein, it was difficult to hope to solve all of them. So, what about solving a problem whose solution en- allowed me to solve all the problems? And this was (laughs) what if we could find the key to an intelligence, you know, 10 times better or faster than Einstein?
- LFLex Fridman
So, that's sort of seeing artificial intelligence as a- as a tool to expand our capabilities, but is there just an inherent curiosity in you in just understanding what it is in our- in- in here that makes it all- all work?
- TPTomaso Poggio
Yes, absolutely. You are right. So, I was starting- I started saying this was the motivation when I was a teenager-
- 15:00 – 30:00
So one of the…
- LFLex Fridman
- TPTomaso Poggio
So one of the main differences, and, um, you know, problems, in terms of deep learning today, and it's not only deep learning, and the brain, is the need for deep learning techniques to have, uh, a lot of labeled examples. You know, for instance, for ImageNet, you have like a training set which is one million images, each one labeled by some human-
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
... in terms of which object is there, and, um, it's, it's clear that in biology, a baby, uh, may be able to see million of images in the first years of life, but will not have million of labels given to him or her by parents or take, take, uh, caretakers. So, uh, how do you solve that? You know, I think that there is this interesting challenge that today, uh, deep learning and related techniques are all about big data, big data meaning a lot of examples labeled by humans-
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
... um, whereas in, uh, nature you have, uh ... So the, the, th- this big data is n going to infinity, that's the best, you know, n meaning labeled data. But I think the biological world is more n going to one.
- LFLex Fridman
(laughs)
- TPTomaso Poggio
A- a child can learn-
- LFLex Fridman
It's a beautiful way to put it.
- TPTomaso Poggio
... from very small number of, you know, labeled examples. Like you tell a child, "This is a car." You don't need to say, like in ImageNet, you know, "This is a car, this is a car, this is not a car, this is not a car" o- one million times. (laughs)
- LFLex Fridman
So... And of course, with AlphaGo and, or at least the-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... AlphaZero variance, there's ... Because of the, because the world of Go is so simplistic, that you can actually learn by yourself through self-play, you could play against each other.
- TPTomaso Poggio
Yep.
- LFLex Fridman
And the real world, I mean, the visual system that you've studied extensively, is a lot more complicated than the game of Go. So-
- TPTomaso Poggio
Right.
- LFLex Fridman
... uh, on the comment about children, which are fascinatingly good at learning new stuff, how much of it do you think is hardware and how much of it is software?
- TPTomaso Poggio
Yeah, that's a, a good, a deep question, is, in a sense, is the old question of nurture and nature.
- LFLex Fridman
Yeah.
- TPTomaso Poggio
How much is in, in the gene and how much is, uh, in the experience of an individual. Obviously, it's both that play a role, and, uh, I believe that the way evolution gives, put prior information, so to speak, hardwired, it's not really hardwired, but, um, uh, that's-Essentially an hypothesis. I think what's going on is that w- evolution, as, um, you know, almost necessarily if you believe in Darwin, is very opportunistic, and, and think about, uh, uh, our DNA and the DNA of Drosophila.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
Uh, our DNA does not have many more genes than Drosophila. Oh, now I'm-
- LFLex Fridman
The fly.
- TPTomaso Poggio
The fly.
- LFLex Fridman
Yeah.
- TPTomaso Poggio
The fruit fly.
- LFLex Fridman
Yeah.
- TPTomaso Poggio
Now, we know that the fruit fly does not learn very much during its individual existence. It looks like one of these machinery that it's really mostly, not 100%, but, you know, 95%, hardcoded by the genes. But since we don't have many more genes than Drosophila is, evolution could encode in us a kind of general learning machinery, and then had to give very weak priors.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
Um, like, for instance, let me take, give a, uh, a specific example which is a recent work by a member of our Center for Brains, Minds, and Machines. We know because of work of other people in our group and other groups that there are cells in a part of our brain, neurons-
- 30:00 – 45:00
Right.…
- TPTomaso Poggio
I could say, "Well, I understand how to use PowerPoint."
- LFLex Fridman
Right.
- TPTomaso Poggio
... that's my level of understanding a computer. It's, it tends reasonable, you know, it give me some power to produce slides, and beautiful slides, and ... Now, you can ask somebody else he says, "Well, I, I know how the transistor work that are inside the computer. I can write the equations for, you know, transistor, and diodes, and circuits, uh, logical circuits." And I can ask this guy, "Do you know how to operate PowerPoint?" "No idea." Right?
- LFLex Fridman
Yeah. So, do you think if we discovered computers walking amongst us, full of these transistors that are also, uh, operating under Windows and have PowerPoint, do you think it's ... digging in a little bit more, how useful is it to understand the transistor in order to be able to understand PowerPoint and these higher level-
- TPTomaso Poggio
Very good. Yes.
- LFLex Fridman
... intelligent processes.
- TPTomaso Poggio
So, I think in the case of computers, because they were made by engineers, by us-
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
... this different level of understanding are rather separate on purpose.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
You know, you, they are separate modules so that the engineer that designed the circuit for the chips does not need to know what, uh, power, is inside PowerPoint.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
And somebody can write to the, the software translating from one to the end, uh, to the other end. So, um, in that case, I don't think, uh, uh, understanding the transistor help you understand PowerPoint, or very little.
- LFLex Fridman
Right.
- TPTomaso Poggio
Um, if you want to s- understand the computer, there is question, you know, I would say you have to understanding at different levels if you really-
- LFLex Fridman
Yeah.
- TPTomaso Poggio
... want to s- to build it, one, right? (laughs) But, uh, but for the brain, I think these levels of understanding, so the algorithms, which kind of computation, you know, the equivalent of PowerPoint, and the circuits, you know, the transistors, I think they are more, much more intertwined with each other. There is not, you know, innately a level of the software separate from the hardware. And so, that's wha- why I think, in the case of the brain, the problem is more difficult and more than for computers, requires the interaction, the collaboration between different types of expertise.
- LFLex Fridman
So, it's a big, the brain is a big hierarchical mess-
- TPTomaso Poggio
Mm-hmm.
- LFLex Fridman
... that you can't just un- disentangle, uh, l- levels.
- TPTomaso Poggio
Uh, I think you can, but is, is much more difficult, and it's not, uh, you know, it's not completely obvious. And, as I said, I think is one of the, personally I think is the greatest problem in science.
- LFLex Fridman
(laughs) Yeah.
- TPTomaso Poggio
So, you know, it, I think is, is fair that it's difficult. (laughs)
- LFLex Fridman
(laughs) That's a difficult one. That said, you do talk about compositionality-
- TPTomaso Poggio
Mm-hmm.
- LFLex Fridman
... and why it might be useful, and when you discuss wha- why these neural networks, in artificial or biological sense, learn anything, you talk about c- uh, compositionality.
- TPTomaso Poggio
Yeah.
- LFLex Fridman
So, you, there's a sense that nature can be disentangled, or perpe- uh, well, all aspects of our cognition could be disentangled-
- TPTomaso Poggio
Mm-hmm.
- LFLex Fridman
... a little, to some degree. So, why do you think, what, first of all, how do you see compositionality, and why do you think it exists at all in nature?
- 45:00 – 1:00:00
No, they, this one,…
- LFLex Fridman
with just a finite number of neurons in a single hidden layer. Do you, do you find this theorem, one, surprising? Do you find it useful, interesting, inspiring?
- TPTomaso Poggio
No, they, this one, uh, you know, I never found it very surprising. It's, uh, was known since the '80s, since I entered the field, because it's basically the same as Weierstrass theorem, which says that I can approximate any continuous function with a polynomial of sufficiently, with a sufficient number of terms, monomials.
- LFLex Fridman
Mm-hmm, yeah.
- TPTomaso Poggio
So, basically the same, and the proofs are very similar.
- LFLex Fridman
So y- your intuition was, there was never any doubt that neural-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... networks, in theory, could-
- TPTomaso Poggio
Right.
- LFLex Fridman
... could be very strong approximators even?
- TPTomaso Poggio
Right. The, the, the question, the interesting question is that if this theorem, uh, says you can approximate, fine, but when you ask how many neurons, for instance, or in the case of polynomial, how many monomials I need to get a good approximation, then it turns out that that depends on the dimensionality of your function, how many variables you have. But it depends on the dimensionality of your function in a bad way. It's, for instance, suppose you want an error which is, uh, no worse than 10% in your approximation. You come up with a network that approximate your function within 10%. Then turns out that the number of units you need are in the order of 10 to the dimensionality, D.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
How many variables. So if you have, you know, two variables is these two, and you have 100 units, and okay. But if you have, say, 200 by 200 pixel images, now this is, you know, tw- 40,000, whatever, and that-
- LFLex Fridman
We again go to the size of the universe pretty quickly.
- TPTomaso Poggio
Yeah, ra- exact. 10 to the 40,000 or something, and (laughs) -
- LFLex Fridman
Yeah. (laughs)
- TPTomaso Poggio
And so, uh, this is called the curse of dimensionality.
- LFLex Fridman
Yeah.
- TPTomaso Poggio
Not, you know, quite appropriately. (laughs)
- LFLex Fridman
(laughs) And the hope is with the extra layers, you can, uh, uh, remove the curse.
- TPTomaso Poggio
What we proved is that if you have deep layers or hierarchical architecture of the, with the local connectivity of the type of convolutional deep learning, and if you're dealing with a function that has this kind of, um, hierarchical architecture, then you avoid completely the curse.
- LFLex Fridman
(laughs) You've spoken a lot about supervised deep learning.
- TPTomaso Poggio
Yeah.
- LFLex Fridman
Uh, what are your thoughts, hopes, views on the challenges of unsupervised learning, uh, with, uh, with GANs, with, uh, generative adversarial networks? Uh, do you see those as distinct... The, the power of GANs, do you see those as distinct from supervised methods in neural networks, or are they really all in the same representation ballpark?
- TPTomaso Poggio
GANs is, uh, one way to get, um-... estimation of, uh, uh, probability densities, which is a somewhat new way that people have not done before. I, I don't know whether, um, this will really play an important role in, uh, you know, in intelligence or, um, it's, it's interesting. I'm, I'm less enthusiastic about it than many people in the field.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
I have the feeling that many people in the field are, um, really impressed by the ability to p- of producing realistic-looking images in a, in this generative way.
- LFLex Fridman
Which describes the popularity of the methods, but you're saying that while that's exciting and cool to look at, it may not be the tool that's useful for-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... for... So, you described it kind of beautifully. Uh, current supervised methods go N to infinity in terms of the number of labeled points, and we really have to figure out how to go to N to one.
- TPTomaso Poggio
Yeah.
- 1:00:00 – 1:15:00
Right. …
- TPTomaso Poggio
of the simple, simpler kernel machines or linear classify, we really don't understand the individual units-
- LFLex Fridman
Right.
- TPTomaso Poggio
... also. We, but we understand, you know, what the computation and the limitations and the properties of it are. Uh, it's similar to many things, you know. We-What does it mean to understand how a fusion bomb works? How many of us ... you know, many of us understand the basic principle, and some of us may understand deeper details.
- LFLex Fridman
In that sense, understanding is, as a community, as a civilization, can we build another copy of it?
- TPTomaso Poggio
Okay.
- LFLex Fridman
And in that sense, do you think there'll be, there'll need to be some evolutionary component where it runs away from our understanding? Or, do you think it could be engineered from the ground up, the same way you go from the transistor to PowerPoint?
- TPTomaso Poggio
Right. All right. So, many years ago, this was actually, let me see, 40, 41 years ago, (laughs) I wrote a paper with, uh, David Marr, who was, um, one of the founding father of computer vision, computational vision. I wrote a paper about levels of understanding, which is related to the question we discussed earlier about, uh, understanding PowerPoint, understanding transistors, and so on. And, uh, uh, you know, in that kind of fr- framework, we are at the level of the hardware-
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
... and the top level of the algorithms. We did not have learning. Recently, I updated, adding levels, and one level I added to those three was learning. So, and you can imagine, you could have a good understanding of how you construct learning machine, like we do, but being unable to describe in detail what the learning machines will discover.
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
Right? Now, that would be still a powerful understanding, if I can build a learning machine, even if I don't understand in detail every time it, it learns something.
- LFLex Fridman
Just like our children, if they, if they start listening to a certain type of music, I don't know, Miley Cyrus or something, you don't understand why they came-
- TPTomaso Poggio
Yep.
- LFLex Fridman
... to that particular preference, but you understand the learning process.
- TPTomaso Poggio
Right.
- LFLex Fridman
That's very interesting.
- TPTomaso Poggio
Yep, yep.
- LFLex Fridman
So, uh, unlearning, for systems to be part of our world, it has a certain ... One of the challenging things that you've spoken about is learning ethics, learning-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... morals. And what, w- how hard do you think is the problem of, first of all, humans understanding our ethics? What is the origin on a neural and low level of ethics? What is it at the higher level? Is it something that's learnable for machines, in your intuition?
- TPTomaso Poggio
I think, uh, yeah, ethics is learnable, very likely. Um, I, I think I, it's one of these problems where ... I think understanding the neuroscience of ethics. You know, people discuss, there is an ethics of neuroscience.
- LFLex Fridman
(laughs)
- TPTomaso Poggio
(laughs)
- LFLex Fridman
Yeah, yes.
- TPTomaso Poggio
You know, how a neuroscientist should or should not behave. Uh-
- LFLex Fridman
Yeah.
- TPTomaso Poggio
Can you think of a neurosurgeon and the ethics that he r- he really has to obey, or she has to obey. But I'm more interested on the, on the-
- LFLex Fridman
(laughs)
- TPTomaso Poggio
... neuroscience of ethics.
- LFLex Fridman
You're blowing my mind right now. The neuroscience of ethics is very meta.
- 1:15:00 – 1:20:26
So, if you see…
- TPTomaso Poggio
you know, discovering something is more fun if it's together with other intelligent and curious and fun people.
- LFLex Fridman
So, if you see the fun in that process, ult- the side effect of that process will be that you'll actually end up discovering some interesting things.
- TPTomaso Poggio
Yeah, yes.
- LFLex Fridman
So, as, uh, you've led, uh, uh, uh, many incredible efforts here, what's the secret to being a good advisor, mentor, leader in a research setting?
- TPTomaso Poggio
Oh, th-
- LFLex Fridman
Is it a similar spirit? Or, yeah, what's, what, what advice could you give to people, young faculty and so on?
- TPTomaso Poggio
It's partly repeating what I said about-
- LFLex Fridman
Mm-hmm.
- TPTomaso Poggio
... an environment that should be friendly and fun and, uh, ambitious. And, uh, you know, I, I think I learned a lot from some of my advisors and friends, and some were physicists, and, uh, there was, for instance, this, um, um, behavior that was encourage of when somebody comes with a new idea in the group, you are, unless it's really stupid, but you are always enthusiastic. And then, and you're enthusiastic for a few minutes, for a few hours. Then you start, you know, asking critically a few questions-
- LFLex Fridman
Yeah.
- TPTomaso Poggio
... to in- te- like, testing this. But, you know, this is a process that is, I think is very, very good, this. You have to be enthusiast- Sometime people are very critical from the beginning. That's, that's, that's not-
- LFLex Fridman
You have to, you have to give it a chance for that-
- TPTomaso Poggio
Yes.
- LFLex Fridman
... for that seed to grow.
- TPTomaso Poggio
Yeah.
- LFLex Fridman
That said, s- with some of your ideas, which are quite revolutionary, so there's... I've, I witnessed, especially in the human vision side and neuroscience side, there could be some pretty heated arguments. Um, do you enjoy these? Is that a part of science and acad-
- TPTomaso Poggio
Yeah.
- LFLex Fridman
... eca- academic pursuits that you enjoy?
- TPTomaso Poggio
Yeah.
- LFLex Fridman
Is it... (laughs) Is that something that happens in your group as well? Uh-
- TPTomaso Poggio
Yeah, absolutely. I also spent some time in Germany. Again, there is this tradition in which people are more, uh, forthright, less kind than here.
- LFLex Fridman
Yeah.
- TPTomaso Poggio
So, you know, in the US, y- you, when you write a bad letter, you still say, "This guy is nice." You know? (laughs)
- LFLex Fridman
Yes, yes.
- TPTomaso Poggio
(laughs) And so, I th-
- LFLex Fridman
Yeah, here in America, it's degrees of nice. Uh-
- TPTomaso Poggio
Yes.
- LFLex Fridman
... (laughs) it's, it's all just degrees of nice, yeah.
- TPTomaso Poggio
Right, right. So, as long as this does not become personal, and it's really like, you know, a football game with its rules, that's great. (laughs)
- LFLex Fridman
(laughs) And it's fun. So, so if you somehow found yourself in a position to ask one question of an oracle, like a genie maybe, a god, wh- and you're guaranteed to get a clear answer, what kinda question would you ask? What, what would be the question you would ask?
Episode duration: 1:20:20
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode aSyZvBrPAyk
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome