Skip to content
Lex Fridman PodcastLex Fridman Podcast

Tomaso Poggio: Brains, Minds, and Machines | Lex Fridman Podcast #13

Lex Fridman and Tomaso Poggio on tomaso Poggio on Intelligence, Brains, and the Limits of AI.

Lex FridmanhostTomaso Poggioguest
Jan 19, 20191h 20mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:0015:00

    The following is a…

    1. LF

      The following is a conversation with Tomaso Poggio. He's a professor at MIT, and is a director of the Center for Brains, Minds, and Machines. Cited over 100,000 times, his work has had a profound impact on our understanding of the nature of intelligence in both biological and artificial neural networks. He has been an advisor to many highly impactful researchers and entrepreneurs in AI, including Demis Hassabis of DeepMind, Amnon Shashua of Mobileye, and Christof Koch of the Allen Institute for Brain Science. This conversation is part of the MIT course on artificial general intelligence, and the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter, @LexFridman, spelled F-R-I-D. And now, here's my conversation with Tomaso Poggio. You've mentioned that in your childhood, you've developed a fascination with physics, especially the theory of relativity, and that Einstein was also a childhood hero to you. What aspect of Einstein's genius, the nature of his genius, do you think was essential for discovering the theory of relativity?

    2. TP

      You know, Einstein was, uh, a hero to me, and I'm sure to many people, because he was able to make, uh, uh, of course, a major, major contribution to physics with, simplifying a bit, just a gedanken experiment, a thought experiment.

    3. LF

      Mm-hmm.

    4. TP

      You know, imagining, uh, communication with lights between a stationary observer and somebody on a train.

    5. LF

      Mm-hmm.

    6. TP

      And, uh, I thought that, um, you know, the, the, the, the fact that just with the force of y- of his thought, of his thinking, of his mind, it could get to some- something so deep in term of physical reality, how time depend on space and speed, is, was something absolutely fascinating. It was the power of intelligence, the power of the mind.

    7. LF

      Do you think the ability to imagine, to visualize as he did, as a lot of great physicists do, do you think that's in all of us human beings? Or is there something special to that one particular human being?

    8. TP

      I think, uh, you know, a- all of us can learn and have, uh, in principle similar b- breakthroughs. Uh, there is lesson to be learned from Einstein. Uh, he was one of five PhD students at ETH, uh, the Eidgenössische Technische Hochschule in, uh, Zurich, in physics, and he was the worst of the five. The only one who d- did not get an academic position when, uh, he, he graduated, when he finished his PhD, and he went to work, as everybody knows, for the patent office. And so it's not so much that he worked for the patent office, but the fact that obviously he was smart, but he was not a top student. O- obviously, he was the anticonformist, he was not thinking in the traditional way that probably his teachers and the other students were doing. So there is a lot to be said about, uh, you know, trying to be... to do the opposite or something quite different from what other people are doing. That's certainly true for the stock market. Never...

    9. LF

      (laughs)

    10. TP

      Never buy if everybody's buying. (laughs)

    11. LF

      And also true for science.

    12. TP

      Yes.

    13. LF

      So you've also mentioned, staying on the theme of physics, that you were excited at a young age by the mysteries of the universe that, uh, physics could uncover. Such, as I saw mentioned, the possibility of time travel.

    14. TP

      (laughs)

    15. LF

      So the most out-of-the-box question I think I'll get to ask today, do you think time travel is possible?

    16. TP

      Well, it would be nice if it were possible right now. Uh, you know, in science, you never say no, um...

    17. LF

      But your understanding of the nature of time.

    18. TP

      Yeah, it's very likely that it's not possible to travel in time. Um, you may be able to travel forward in time if we can, for instance, freeze ourselves or, uh, you know, go on some spacecraft traveling close to the speed of light. But in terms of actively traveling, for instance, back in time, I find probably very unlikely.

    19. LF

      So do you still hold the, the underlying dream of the engineering intelligence that we'll build systems that are able to do such huge leaps, like discovering the kind of mechanism that would be required to travel through time? Do you still hold that dream or is- or echoes of it from your childhood?

    20. TP

      Yeah, I, uh, you know, I don't think whether... Uh, there are certain problems that probably cannot be solved depending what, uh, what you believe about the physical reality, like, uh, you know, maybe totally impossible to create energy from nothing or to travel back in time. But, uh, um, about making machines that can, uh, think as well as w- we do or better, or more likely, especially in the short and mid-term, help us think better, which is in a sense is happening already with the computers we have, and it will happen more and more. Well, that I certainly believe, and I don't see in principle why computers at some point, uh, could not become more intelligent than we are. Although the word intelligence...... is a tricky one and one we should discuss- (laughs)

    21. LF

      Yeah, for sure.

    22. TP

      ... what I mean with that. (laughs)

    23. LF

      Uh, in- intelligence, consciousness-

    24. TP

      Yeah.

    25. LF

      ... words like love, is, all these are very, uh-

    26. TP

      Yeah.

    27. LF

      ... need to be disentangled. So, you've mentioned also that you believe the problem of intelligence is the greatest problem in science, greater than the origin of life and the origin of the universe. You've also, uh, in the talk I've listened to, uh, said that you're open to arguments against, uh, against you. So, uh, what do you think is the most captivating aspect of this problem of understanding the nature of intelligence? Why does it captivate you as it does?

    28. TP

      Well, originally, I think one of the motivation that I had as a, I guess, a teenager, when I was infatuated with theory of relativity, was really that I- I found that there was, uh, the problem of time and space and general relativity, but there were so many other problems of the same level of difficulty and importance that I could ... even if I were Einstein, it was difficult to hope to solve all of them. So, what about solving a problem whose solution en- allowed me to solve all the problems? And this was (laughs) what if we could find the key to an intelligence, you know, 10 times better or faster than Einstein?

    29. LF

      So, that's sort of seeing artificial intelligence as a- as a tool to expand our capabilities, but is there just an inherent curiosity in you in just understanding what it is in our- in- in here that makes it all- all work?

    30. TP

      Yes, absolutely. You are right. So, I was starting- I started saying this was the motivation when I was a teenager-

  2. 15:0030:00

    So one of the…

    1. LF

    2. TP

      So one of the main differences, and, um, you know, problems, in terms of deep learning today, and it's not only deep learning, and the brain, is the need for deep learning techniques to have, uh, a lot of labeled examples. You know, for instance, for ImageNet, you have like a training set which is one million images, each one labeled by some human-

    3. LF

      Mm-hmm.

    4. TP

      ... in terms of which object is there, and, um, it's, it's clear that in biology, a baby, uh, may be able to see million of images in the first years of life, but will not have million of labels given to him or her by parents or take, take, uh, caretakers. So, uh, how do you solve that? You know, I think that there is this interesting challenge that today, uh, deep learning and related techniques are all about big data, big data meaning a lot of examples labeled by humans-

    5. LF

      Mm-hmm.

    6. TP

      ... um, whereas in, uh, nature you have, uh ... So the, the, th- this big data is n going to infinity, that's the best, you know, n meaning labeled data. But I think the biological world is more n going to one.

    7. LF

      (laughs)

    8. TP

      A- a child can learn-

    9. LF

      It's a beautiful way to put it.

    10. TP

      ... from very small number of, you know, labeled examples. Like you tell a child, "This is a car." You don't need to say, like in ImageNet, you know, "This is a car, this is a car, this is not a car, this is not a car" o- one million times. (laughs)

    11. LF

      So... And of course, with AlphaGo and, or at least the-

    12. TP

      Yeah.

    13. LF

      ... AlphaZero variance, there's ... Because of the, because the world of Go is so simplistic, that you can actually learn by yourself through self-play, you could play against each other.

    14. TP

      Yep.

    15. LF

      And the real world, I mean, the visual system that you've studied extensively, is a lot more complicated than the game of Go. So-

    16. TP

      Right.

    17. LF

      ... uh, on the comment about children, which are fascinatingly good at learning new stuff, how much of it do you think is hardware and how much of it is software?

    18. TP

      Yeah, that's a, a good, a deep question, is, in a sense, is the old question of nurture and nature.

    19. LF

      Yeah.

    20. TP

      How much is in, in the gene and how much is, uh, in the experience of an individual. Obviously, it's both that play a role, and, uh, I believe that the way evolution gives, put prior information, so to speak, hardwired, it's not really hardwired, but, um, uh, that's-Essentially an hypothesis. I think what's going on is that w- evolution, as, um, you know, almost necessarily if you believe in Darwin, is very opportunistic, and, and think about, uh, uh, our DNA and the DNA of Drosophila.

    21. LF

      Mm-hmm.

    22. TP

      Uh, our DNA does not have many more genes than Drosophila. Oh, now I'm-

    23. LF

      The fly.

    24. TP

      The fly.

    25. LF

      Yeah.

    26. TP

      The fruit fly.

    27. LF

      Yeah.

    28. TP

      Now, we know that the fruit fly does not learn very much during its individual existence. It looks like one of these machinery that it's really mostly, not 100%, but, you know, 95%, hardcoded by the genes. But since we don't have many more genes than Drosophila is, evolution could encode in us a kind of general learning machinery, and then had to give very weak priors.

    29. LF

      Mm-hmm.

    30. TP

      Um, like, for instance, let me take, give a, uh, a specific example which is a recent work by a member of our Center for Brains, Minds, and Machines. We know because of work of other people in our group and other groups that there are cells in a part of our brain, neurons-

  3. 30:0045:00

    Right.…

    1. TP

      I could say, "Well, I understand how to use PowerPoint."

    2. LF

      Right.

    3. TP

      ... that's my level of understanding a computer. It's, it tends reasonable, you know, it give me some power to produce slides, and beautiful slides, and ... Now, you can ask somebody else he says, "Well, I, I know how the transistor work that are inside the computer. I can write the equations for, you know, transistor, and diodes, and circuits, uh, logical circuits." And I can ask this guy, "Do you know how to operate PowerPoint?" "No idea." Right?

    4. LF

      Yeah. So, do you think if we discovered computers walking amongst us, full of these transistors that are also, uh, operating under Windows and have PowerPoint, do you think it's ... digging in a little bit more, how useful is it to understand the transistor in order to be able to understand PowerPoint and these higher level-

    5. TP

      Very good. Yes.

    6. LF

      ... intelligent processes.

    7. TP

      So, I think in the case of computers, because they were made by engineers, by us-

    8. LF

      Mm-hmm.

    9. TP

      ... this different level of understanding are rather separate on purpose.

    10. LF

      Mm-hmm.

    11. TP

      You know, you, they are separate modules so that the engineer that designed the circuit for the chips does not need to know what, uh, power, is inside PowerPoint.

    12. LF

      Mm-hmm.

    13. TP

      And somebody can write to the, the software translating from one to the end, uh, to the other end. So, um, in that case, I don't think, uh, uh, understanding the transistor help you understand PowerPoint, or very little.

    14. LF

      Right.

    15. TP

      Um, if you want to s- understand the computer, there is question, you know, I would say you have to understanding at different levels if you really-

    16. LF

      Yeah.

    17. TP

      ... want to s- to build it, one, right? (laughs) But, uh, but for the brain, I think these levels of understanding, so the algorithms, which kind of computation, you know, the equivalent of PowerPoint, and the circuits, you know, the transistors, I think they are more, much more intertwined with each other. There is not, you know, innately a level of the software separate from the hardware. And so, that's wha- why I think, in the case of the brain, the problem is more difficult and more than for computers, requires the interaction, the collaboration between different types of expertise.

    18. LF

      So, it's a big, the brain is a big hierarchical mess-

    19. TP

      Mm-hmm.

    20. LF

      ... that you can't just un- disentangle, uh, l- levels.

    21. TP

      Uh, I think you can, but is, is much more difficult, and it's not, uh, you know, it's not completely obvious. And, as I said, I think is one of the, personally I think is the greatest problem in science.

    22. LF

      (laughs) Yeah.

    23. TP

      So, you know, it, I think is, is fair that it's difficult. (laughs)

    24. LF

      (laughs) That's a difficult one. That said, you do talk about compositionality-

    25. TP

      Mm-hmm.

    26. LF

      ... and why it might be useful, and when you discuss wha- why these neural networks, in artificial or biological sense, learn anything, you talk about c- uh, compositionality.

    27. TP

      Yeah.

    28. LF

      So, you, there's a sense that nature can be disentangled, or perpe- uh, well, all aspects of our cognition could be disentangled-

    29. TP

      Mm-hmm.

    30. LF

      ... a little, to some degree. So, why do you think, what, first of all, how do you see compositionality, and why do you think it exists at all in nature?

  4. 45:001:00:00

    No, they, this one,…

    1. LF

      with just a finite number of neurons in a single hidden layer. Do you, do you find this theorem, one, surprising? Do you find it useful, interesting, inspiring?

    2. TP

      No, they, this one, uh, you know, I never found it very surprising. It's, uh, was known since the '80s, since I entered the field, because it's basically the same as Weierstrass theorem, which says that I can approximate any continuous function with a polynomial of sufficiently, with a sufficient number of terms, monomials.

    3. LF

      Mm-hmm, yeah.

    4. TP

      So, basically the same, and the proofs are very similar.

    5. LF

      So y- your intuition was, there was never any doubt that neural-

    6. TP

      Yeah.

    7. LF

      ... networks, in theory, could-

    8. TP

      Right.

    9. LF

      ... could be very strong approximators even?

    10. TP

      Right. The, the, the question, the interesting question is that if this theorem, uh, says you can approximate, fine, but when you ask how many neurons, for instance, or in the case of polynomial, how many monomials I need to get a good approximation, then it turns out that that depends on the dimensionality of your function, how many variables you have. But it depends on the dimensionality of your function in a bad way. It's, for instance, suppose you want an error which is, uh, no worse than 10% in your approximation. You come up with a network that approximate your function within 10%. Then turns out that the number of units you need are in the order of 10 to the dimensionality, D.

    11. LF

      Mm-hmm.

    12. TP

      How many variables. So if you have, you know, two variables is these two, and you have 100 units, and okay. But if you have, say, 200 by 200 pixel images, now this is, you know, tw- 40,000, whatever, and that-

    13. LF

      We again go to the size of the universe pretty quickly.

    14. TP

      Yeah, ra- exact. 10 to the 40,000 or something, and (laughs) -

    15. LF

      Yeah. (laughs)

    16. TP

      And so, uh, this is called the curse of dimensionality.

    17. LF

      Yeah.

    18. TP

      Not, you know, quite appropriately. (laughs)

    19. LF

      (laughs) And the hope is with the extra layers, you can, uh, uh, remove the curse.

    20. TP

      What we proved is that if you have deep layers or hierarchical architecture of the, with the local connectivity of the type of convolutional deep learning, and if you're dealing with a function that has this kind of, um, hierarchical architecture, then you avoid completely the curse.

    21. LF

      (laughs) You've spoken a lot about supervised deep learning.

    22. TP

      Yeah.

    23. LF

      Uh, what are your thoughts, hopes, views on the challenges of unsupervised learning, uh, with, uh, with GANs, with, uh, generative adversarial networks? Uh, do you see those as distinct... The, the power of GANs, do you see those as distinct from supervised methods in neural networks, or are they really all in the same representation ballpark?

    24. TP

      GANs is, uh, one way to get, um-... estimation of, uh, uh, probability densities, which is a somewhat new way that people have not done before. I, I don't know whether, um, this will really play an important role in, uh, you know, in intelligence or, um, it's, it's interesting. I'm, I'm less enthusiastic about it than many people in the field.

    25. LF

      Mm-hmm.

    26. TP

      I have the feeling that many people in the field are, um, really impressed by the ability to p- of producing realistic-looking images in a, in this generative way.

    27. LF

      Which describes the popularity of the methods, but you're saying that while that's exciting and cool to look at, it may not be the tool that's useful for-

    28. TP

      Yeah.

    29. LF

      ... for... So, you described it kind of beautifully. Uh, current supervised methods go N to infinity in terms of the number of labeled points, and we really have to figure out how to go to N to one.

    30. TP

      Yeah.

  5. 1:00:001:15:00

    Right. …

    1. TP

      of the simple, simpler kernel machines or linear classify, we really don't understand the individual units-

    2. LF

      Right.

    3. TP

      ... also. We, but we understand, you know, what the computation and the limitations and the properties of it are. Uh, it's similar to many things, you know. We-What does it mean to understand how a fusion bomb works? How many of us ... you know, many of us understand the basic principle, and some of us may understand deeper details.

    4. LF

      In that sense, understanding is, as a community, as a civilization, can we build another copy of it?

    5. TP

      Okay.

    6. LF

      And in that sense, do you think there'll be, there'll need to be some evolutionary component where it runs away from our understanding? Or, do you think it could be engineered from the ground up, the same way you go from the transistor to PowerPoint?

    7. TP

      Right. All right. So, many years ago, this was actually, let me see, 40, 41 years ago, (laughs) I wrote a paper with, uh, David Marr, who was, um, one of the founding father of computer vision, computational vision. I wrote a paper about levels of understanding, which is related to the question we discussed earlier about, uh, understanding PowerPoint, understanding transistors, and so on. And, uh, uh, you know, in that kind of fr- framework, we are at the level of the hardware-

    8. LF

      Mm-hmm.

    9. TP

      ... and the top level of the algorithms. We did not have learning. Recently, I updated, adding levels, and one level I added to those three was learning. So, and you can imagine, you could have a good understanding of how you construct learning machine, like we do, but being unable to describe in detail what the learning machines will discover.

    10. LF

      Mm-hmm.

    11. TP

      Right? Now, that would be still a powerful understanding, if I can build a learning machine, even if I don't understand in detail every time it, it learns something.

    12. LF

      Just like our children, if they, if they start listening to a certain type of music, I don't know, Miley Cyrus or something, you don't understand why they came-

    13. TP

      Yep.

    14. LF

      ... to that particular preference, but you understand the learning process.

    15. TP

      Right.

    16. LF

      That's very interesting.

    17. TP

      Yep, yep.

    18. LF

      So, uh, unlearning, for systems to be part of our world, it has a certain ... One of the challenging things that you've spoken about is learning ethics, learning-

    19. TP

      Yeah.

    20. LF

      ... morals. And what, w- how hard do you think is the problem of, first of all, humans understanding our ethics? What is the origin on a neural and low level of ethics? What is it at the higher level? Is it something that's learnable for machines, in your intuition?

    21. TP

      I think, uh, yeah, ethics is learnable, very likely. Um, I, I think I, it's one of these problems where ... I think understanding the neuroscience of ethics. You know, people discuss, there is an ethics of neuroscience.

    22. LF

      (laughs)

    23. TP

      (laughs)

    24. LF

      Yeah, yes.

    25. TP

      You know, how a neuroscientist should or should not behave. Uh-

    26. LF

      Yeah.

    27. TP

      Can you think of a neurosurgeon and the ethics that he r- he really has to obey, or she has to obey. But I'm more interested on the, on the-

    28. LF

      (laughs)

    29. TP

      ... neuroscience of ethics.

    30. LF

      You're blowing my mind right now. The neuroscience of ethics is very meta.

  6. 1:15:001:20:26

    So, if you see…

    1. TP

      you know, discovering something is more fun if it's together with other intelligent and curious and fun people.

    2. LF

      So, if you see the fun in that process, ult- the side effect of that process will be that you'll actually end up discovering some interesting things.

    3. TP

      Yeah, yes.

    4. LF

      So, as, uh, you've led, uh, uh, uh, many incredible efforts here, what's the secret to being a good advisor, mentor, leader in a research setting?

    5. TP

      Oh, th-

    6. LF

      Is it a similar spirit? Or, yeah, what's, what, what advice could you give to people, young faculty and so on?

    7. TP

      It's partly repeating what I said about-

    8. LF

      Mm-hmm.

    9. TP

      ... an environment that should be friendly and fun and, uh, ambitious. And, uh, you know, I, I think I learned a lot from some of my advisors and friends, and some were physicists, and, uh, there was, for instance, this, um, um, behavior that was encourage of when somebody comes with a new idea in the group, you are, unless it's really stupid, but you are always enthusiastic. And then, and you're enthusiastic for a few minutes, for a few hours. Then you start, you know, asking critically a few questions-

    10. LF

      Yeah.

    11. TP

      ... to in- te- like, testing this. But, you know, this is a process that is, I think is very, very good, this. You have to be enthusiast- Sometime people are very critical from the beginning. That's, that's, that's not-

    12. LF

      You have to, you have to give it a chance for that-

    13. TP

      Yes.

    14. LF

      ... for that seed to grow.

    15. TP

      Yeah.

    16. LF

      That said, s- with some of your ideas, which are quite revolutionary, so there's... I've, I witnessed, especially in the human vision side and neuroscience side, there could be some pretty heated arguments. Um, do you enjoy these? Is that a part of science and acad-

    17. TP

      Yeah.

    18. LF

      ... eca- academic pursuits that you enjoy?

    19. TP

      Yeah.

    20. LF

      Is it... (laughs) Is that something that happens in your group as well? Uh-

    21. TP

      Yeah, absolutely. I also spent some time in Germany. Again, there is this tradition in which people are more, uh, forthright, less kind than here.

    22. LF

      Yeah.

    23. TP

      So, you know, in the US, y- you, when you write a bad letter, you still say, "This guy is nice." You know? (laughs)

    24. LF

      Yes, yes.

    25. TP

      (laughs) And so, I th-

    26. LF

      Yeah, here in America, it's degrees of nice. Uh-

    27. TP

      Yes.

    28. LF

      ... (laughs) it's, it's all just degrees of nice, yeah.

    29. TP

      Right, right. So, as long as this does not become personal, and it's really like, you know, a football game with its rules, that's great. (laughs)

    30. LF

      (laughs) And it's fun. So, so if you somehow found yourself in a position to ask one question of an oracle, like a genie maybe, a god, wh- and you're guaranteed to get a clear answer, what kinda question would you ask? What, what would be the question you would ask?

Episode duration: 1:20:20

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode aSyZvBrPAyk

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome