Jay McClelland: Neural Networks and the Emergence of Cognition | Lex Fridman Podcast #222

The following is a conversation with Jay McClelland, a cognitive scientist at Stanford and one of the seminal figures in the history of artificial intelligence, and specifically neural networks, having written the Parallel Distributed Processing book with David Rumelhart, who co-authored the Backpropagation paper with Geoff Hinton. In their collaborations, they've paved the way for many of the ideas at the center of the neural network-based machine learning revolution of the past 15 years. To support this podcast, please check out our sponsors in the description. This is the Lex Fridman podcast, and here is my conversation with Jay McClelland. You are one of the seminal figures in the history of neural networks at the intersection of, uh, cognitive psychology and computer science. What to you has, over the decades, emerged as the most beautiful aspect about neural networks, both artificial and biological?

The fundamental thing I think about with neural networks is how they allow us to link biology with the mysteries of thought. And, um, you know, in the. When I was first entering the field myself in the late '60s, early '70s, cogni- cognitive psychology had just become a field. There was a book published in '67 called Cognitive Psychology. Um, and the author said that, you know, the study of the nervous system was only of peripheral interest. It wasn't going to tell us anything about the mind, and I didn't agree with that. I, I always felt, "Oh, look, I'm, I'm a physical being." I... From dust to dust, you know, ashes to ashes, and somehow I emerged from that. Um-

So, so that's really interesting. So there was a sense with cognitive psychology that in understanding the sort of neuronal structure of things, you're not going to be able to understand the mind? And then your sense is if we study these neural networks, we might be able to get at least very close to understanding the fundamentals of the human mind.

Yeah. I used to think, um, or I used to talk about the idea of awakening from the Cartesian dream.

(laughs)

So Descartes, um, you know, thought about these things, right? He, he was walking in the gardens of Versailles one day, and he stepped on a stone and a statue moved. And he walked a little further, he stepped on another stone and another statue moved, and he, like, "Why did the statue move when I stepped on the stone?" And he went and talked to the gardeners, and he found out that they had a hydraulic system that allowed the physical contact with the stone to cause water to flow in various directions, which caused water to flow under the statue and moved the statue. And he used this as the beginnings of a theory about how animals act, and he had this notion that these little fibers that people had identified that weren't carrying the blood, you know, were these little hydraulic tubes that if you touch something, there would be pressure and it would send a signal of pressure to the other parts of the system and that would cause action. So he had a mechanistic theory of animal behavior, and he thought that the human had this animal body, but that some divine something else had to have come down and been placed in him to give him the ability to think, right? So the physical world includes the body in action, but it doesn't include thought according to Descartes, right?