Ben Goertzel: Artificial General Intelligence | Lex Fridman Podcast #103

Lex Fridman 0:00 The following is a conversation with Ben Goertzel, one of the most interesting minds in the artificial intelligence community. He's the founder of SingularityNET, designer of OpenCog AI framework, formerly a director of research at the Machine Intelligence Research Institute, and chief scientist of Hanson Robotics, the company that created the Sophia robot. He has been a central figure in the AGI community for many years, including in his organizing and contributing to the Conference on Artificial General Intelligence, the 2020 version of which is actually happening this week, Wednesday, Thursday, and Friday. It's virtual and free. I encourage you to check out the talks, including by Joscha Bach, uh, from episode 101 of this podcast. Quick summary of the ads. Two sponsors, The Jordan Harbinger Show and Masterclass. Please consider supporting this podcast by going to jordanharbinger.com/lex and signing up on masterclass.com/lex. Click the links, buy all the stuff. It's the best way to support this podcast and the journey I'm on in my research and startup. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcasts, support it on Patreon, or connect with me on Twitter @LexFridman, spelled without the E, just F-R-I-D-M-A-N. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation. This episode is supported by The Jordan Harbinger Show. Go to jordanharbinger.com/lex. It's how he knows I sent you. On that page, there's links to subscribe to it on Apple Podcasts, Spotify, and everywhere else. I've been binging on his podcast. Jordan is great. He gets the best out of his guests, dives deep, calls them out when it's needed, and makes the whole thing fun to listen to. He's interviewed Kobe Bryant, Mark Cuban, Neil deGrasse Tyson, Garry Kasparov, and many more. His conversation with Kobe is a reminder of how much focus and hard work is required for greatness in sport, business, and life. I highly recommend the episode if you want to be inspired. Again, go to jordanharbinger.com/lex. It's how Jordan knows I sent you. This show, sponsored by Masterclass. Sign up at masterclass.com/lex to get a discount and to support this podcast. When I first heard about Masterclass, I thought it was too good to be true. For 180 bucks a year, you get an all-access pass to watch courses from, to list some of my favorites, Chris Hadfield on space exploration, Neil deGrasse Tyson on scientific thinking and communication, Will Wright, creator of the greatest city-building game ever, SimCity and Sims, on game design, Carlos Santana on guitar, Garry Kasparov, the greatest chess player ever, on chess, Daniel Negreanu on poker, and many more. Chris Hadfield explaining how rockets work and the experience of being launched into space alone is worth the money. Once again, sign up on masterclass.com/lex to get a discount and to support this podcast. And now, here's my conversation with Ben Goertzel.

Lex Fridman 3:20 What books, authors, ideas had a lot of impact on you, um, in your life in the early days?

Ben Goertzel 3:27 You know, what got me into AI and science fiction and such in the first place wasn't a book, but the original Star Trek TV show, which my dad watched with me, like, in its first run. It would have been 19-1968, '69 or something, and that, that was incredible, 'cause every, every show, they visited a different, a different, uh, alien civilization with a different culture and weird mechanisms. But that, that got me into science fiction, and there wasn't that much science fiction to watch on TV at that stage, so that got me into reading the whole, the whole literature of science fiction, you know, from, from the beginning of the previous century un- un- until that time, and there, I mean, there was so many science fiction writers who were in- inspirational to me. I'd say if I had to pick two, it would have been, uh, Stanislaw Lem, the, the Polish writer-

Lex Fridman 4:18 Solaris. Yeah.

Ben Goertzel 4:19 Yeah. Well, S- Solaris, and then he had, he had a bunch of more obscure writings on, on superhuman AIs that were engineered. Solaris was sort of a superhuman, naturally occurring in- in- intelligence. Then Philip K. Dick, who-

Lex Fridman 4:33 Ah.

Ben Goertzel 4:34 ... you know, u- ultimately my fandom for Philip K. Dick is one of the things that brought me together with David Hanson, my collaborator on, on, on robotics projects. So, you know, Stanislaw Lem was, was very much an intellectual, right? So he, he had a very broad view of intelligence going beyond the human and into what I would call, you know, open-ended superintelligence. The, the Solaris superintelligent ocean was intelligent, in some ways more generally intelligent than people, but in a complex and confusing way so that human beings could never quite connect to it. But it wa- but it was still probably very, very smart. And then the, the Golem-4 supercomputer in one of, one of Lem's, Lem's books, this was engineered by people, but eventually it became very intelligent in a different direction than humans and decided that humans were kind of trivial and not that interesting. So it, it put some impenetrable shield around itself, shut itself off from humanity, and then issued some philosophical screed about the pathetic and hopeless nature of, of humanity and, and all human thought, and th- and then disappeared. Now, Philip K. Dick, he was a bit different. He was human-focused, right? Hi- his main thing was...... you know, human compassion and the human heart and soul are going to be the constant that will keep us going through whatever aliens, aliens we discover, or telepathy machines, or, or, or super AIs, or, or whatever it might be. So, he didn't believe in reality, like the reality that we see may be a simulation, or, or, or a dream, or something else we can't even comprehend. But he believed in love and compassion as something persistent through the various simulated realities. So those, those two science fiction writers had a, had a huge impact on me. Then, a little older than that, I got into Dostoevsky and, uh, Friedrich Nietzsche and, uh, Rimbaud, then a bunch of, uh, more, more literary-type writing.

Lex Fridman 6:37 Can we talk about some

Lex Fridman 6:38 of those things? So, on the Solaris side, Stanislaw Lem, uh, this kind of idea of there being intelligences out there that are different than our own, do you think there are intelligences maybe all around us that we're not able to even detect? So, the, this kind of idea of, uh... Maybe you can comment also on, uh, Stephen Wolfram thinking that there's computations all around us, and we're just not smart enough to kinda detect their, their intelligence or appreciate their intelligence.

Ben Goertzel 7:10 Yeah. So my friend, Hugo de Garis, who I've been talking to about these things for, for, for many decades, since the early '90s, he had an idea he called SIPI, the Search for Intra-Particulate Intelligence.

Lex Fridman 7:24 (laughs)

Ben Goertzel 7:25 So, the concept there was as AIs get smarter and smarter and smarter, you know, assuming the laws of physics as we know them now are still, are still what these super-intelligences perceive to hold and are bound by, as they get smarter and smarter, they're gonna shrink themselves littler and littler, because of special relativity, make, makes it so they can't communicate between two spatially distant points. So, they're gonna get smaller and smaller. But then ultimately, what does that mean? The minds of the super, super, super-intelligences, they're gonna be packed into the, the interaction of, of elementary particles or, or quarks, or the partons inside quarks, or whatever it is. So, what we perceive as random fluctuations on the quantum or sub-quantum level may actually be the thoughts of the micro, micro, micro-miniaturized super-intelligences. 'Cause there's no way we can tell random from structured, but with an algorithmic information more complex than our brains, right? We can't tell the difference. So, what we think is random could be the thought processes of some really tiny super minds. And if so, there's not a damn thing we can do about it, except, you know, try to upgrade our intelligences and expand our minds so that we can, we can perceive more of what's around us.

Lex Fridman 8:41 But if the, if those random fluctuations, like even if we go to, like, quantum mechanics, if that, if that's actually, uh, super-intelligent systems, aren't we then part of the soup of super-intelligence? So, we're... Aren't we just like, like a finger (laughs) of the entirety of the body of the super-intelligent system? Okay.

Ben Goertzel 9:01 We could be... I mean, a finger is a, is a strange metaphor.

Lex Fridman 9:06 (laughs)

Ben Goertzel 9:06 I mean, we, we, we... (laughs)

Lex Fridman 9:08 Well, a finger is dumb is what I mean. Is, uh, is...

Ben Goertzel 9:10 Well, the finger is also useful and is controlled-

Lex Fridman 9:13 Right.

Ben Goertzel 9:13 ... with, with intent by, by the brain. Whereas we may be much less than that, right? I mean, I mean, yeah, we may be just some random epiphenomenon that, that they don't care about too much. Like, think, think about the, the shape of the crowd emanating from a sports stadium or something, right?

Lex Fridman 9:28 Right.

Ben Goertzel 9:28 There, there's some emergent shape to the crowd. It's there. You could take a picture of it. It's kind of cool. It's irrelevant to the main point of the sports event or where the people are going, or, or, or what's on the minds of the people making that shape in the crowd, right? So, we, we may just be some semi-arbitrary, higher-level pattern popping out of, of a lower level, hyper-intelligent self-organization. And I mean, so, so be it, right?

Lex Fridman 9:55 (laughs)

Ben Goertzel 9:55 I mean, that, that's one thing that-

Lex Fridman 9:57 Still a fun ride.

Ben Goertzel 9:58 Yeah. I mean, the older I've gotten, the more respect I've achieved for our fundamental ignorance. I mean, my, mine and, and everybody else's. I mean, I look at my, my two dogs, two beautiful little toy poodles, and you know, they watch me sitting at the computer typing. They just think I'm sitting there wiggling my fingers to exercise them maybe, or guarding the monitor on the desk, that they have no idea that I'm communicating with other people halfway around the world, let, let alone, you know, creating complex algorithms running in, in RAM on some computer server in Saint Petersburg or something, right? That, although they're right there, they're right there in the room with me. So, what things are there right around us that we're just too stupid or close-minded to comprehend?

Lex Fridman 10:40 Well, pro-

Ben Goertzel 10:40 Probably, probably quite a lot.

Lex Fridman 10:42 Your very, your very poodle could be, uh, could also be communicating across multiple dimensions with, with other, with other beings, and you're too, you're too unintelligent to understand the kind of communication mechanism they're going through.

Ben Goertzel 10:55 There, there, there have been various, uh, TV shows and science fiction novels positing cats, dolphins, uh, mice and whatnot are actually super-intelligences h- here to observe that. I wou- I would, I would guess as one or the other quantum physics founders said, those theories are not crazy enough to be true.

Lex Fridman 11:15 (laughs)

Ben Goertzel 11:15 The reality's probably crazier than that.

Lex Fridman 11:17 Beautifully put. So, on the human side, uh, with, uh, Philip K. Dick and, and, uh, in general, where do you fall on this idea that, uh, love and just the basic spirit of human nature persists throughout these multiple realities? Um, are you on the side... Like, the thing that inspires you about artificial intelligence, is it the human side of somehow persisting through all of the different systems we engineer? Or is it, or does AI inspire you to create something that's greater than human, that's beyond human, that's almost non-human?

Ben Goertzel 11:59 I would say my motivation to create AGI comes from, from both of those directions, actually. So when I, when I first became passionate about AGI when I was, it would have been two or three years old after watching robots on, on Star Trek. I mean, then, it was really a combination of intellectual curiosity, like can a machine really think? How, how would you do that? And yeah, just ambition to create something much better than all the clearly limited and fundamentally defective humans I saw around me. Then as I got older and got more enmeshed in, in the human world, and you know, got married, had, had children, saw my parents begin to age, I started to realize, well, not only will AGI let you go far beyond the limitations of the human, but it could also, like stop us from dying and, and suffering and, and feeling pain and, and tormenting ourselves mentally. So you can see AGI has amazing capability to do good for humans, as humans a- alongside with its capability to go far, far beyond the human level. So I mean both, both aspects are, are there which makes it, uh, even more exciting and important.

Lex Fridman 13:13 So you mentioned Dostoevsky and Nietzsche. What did you pick up from those guys? I mean...

Ben Goertzel 13:17 Ah.

Lex Fridman 13:18 (laughs)

Ben Goertzel 13:18 That would probably go beyond the...

Lex Fridman 13:20 (laughs)

Ben Goertzel 13:20 ...beyond the scope of a brief interview, certainly.

Lex Fridman 13:24 (laughs) Sure.

Ben Goertzel 13:24 But both of those are amazing thinkers who one will necessarily have a complex relationship with, right? So I mean, Dostoevsky on the minus side, he's kind of a religious fanatic and he sort of helped squash the Russian nihilist movement, which was very interesting because what nihilism meant originally in that period of the mid-late 1800s in Russia was not taking anything fully 100% for granted. It was really more like what we'd call Bayesianism now where you don't want to adopt anything as a dogmatic certitude and always leave your mind open. And how Dostoevsky parodied nihilism was wha- wa- wa- was a bit different, right? He parodied it as people who believe absolutely nothing so they must assign an equal probability weight to every proposition which, which, which doesn't really work. So on the one hand I didn't, I didn't really agree with Dostoevsky on his sort of religious point of view. On, on, on the other hand, if you look at his understanding of human nature and sort of the human mind and, and, and heart and soul it's, it's really unparalleled and he had an amazing view of how human beings, you know, construct a world for themselves based on their own understanding and their own mental predisposition. And I think if you look in The Brothers Karamazov in particular, the Russian literary theorist Mikhail Bakhtin wrote about this as a polyphonic mode of fiction which means it's not third-person but it's not first-person from any one person really. There are many different characters in the novel and each of them is sort of telling part of the story from their own point of view so the reality of the whole story is an intersection like synergetically of the many different characters' world views and that really it's a beautiful metaphor and even a reflection I think of how all of us socially create our reality. Like e- e- each of us sees the world in a certain way, each of us in a sense is making the world as we see it based on our own minds and understanding but it's polyphony like, like in music where multiple instruments are coming toge- coming together to create the sound. The ultimate reality that's created comes out of each of our subjective understandings you know intersecting with each other and that, that was one of the many beautiful things in Dostoevsky.

Lex Fridman 15:55 So

Lex Fridman 15:56 maybe a little bit to mention you have a connection to Russia and the Soviet culture. I mean I'm not sure exactly what the nature of the connection is but there at least the spirit of your thinking is in there.

Ben Goertzel 16:07 Ah yeah well my, my, my ancestry is three-quarters Eastern European Jewish.

Lex Fridman 16:13 Oh shit. Oh.

Ben Goertzel 16:13 So I mean my... Three of my great-grandparents emigrated to New York from Lithuania and sort of border regions of Poland which were in and out of Poland in around the ti- around the time of World War I and they were, they were socialists and communists as well as Jews, mostly Menshevik not Bolshevik and they sort of they fled at just the right time to the US for their own personal reasons and then almost all or maybe all of my extended family that remained in Eastern Europe was killed either by Hitler's or Stalin's minions at some point so the branch of the family that emigrated to the US was pretty much the only one right

Lex Fridman 16:56 So how much...

Ben Goertzel 16:57 ...that survived.

Lex Fridman 16:57 ...of the spirit of the people is in your blood still? Like do you when you look in the mirror, do you see uh... What do you see? (laughs)

Ben Goertzel 17:05 Meat. I see a bag of meat that I want to transcend by uploading into some sort of superior reality but...

Lex Fridman 17:12 (laughs)

Ben Goertzel 17:13 Very...

Lex Fridman 17:14 Yeah.

Ben Goertzel 17:15 I mean yeah very clearly...

Lex Fridman 17:17 Well put.

Ben Goertzel 17:18 I mean I'm not religious in a traditional sense but clearly the Eastern European Jewish tradition was what I was raised in. I mean there was... My grandfather Leo Zwell was a physical chemist who worked with Linus Pauling and a bunch of the other early greats in quantum mechanics. I mean he was into X-ray diffraction. He was on the material science side, an experimentalist rather than a theorist. His sister was also a physicist and my father's father Victor Goertzel was a PhD in psychology who had the unenviable job of giving psychotherapy to the Japanese in internment camps in the US in World War II like to counsel them why they shouldn't kill themselves e- e- even though they'd had all their stuff taken away and been imprisoned for no good reason. So I mean there yeah there was a lot of uh-... Eastern European Jewishness in my, in my background. One of my great uncles was, I guess, conductor of San Francisco Orchestra. So there, there's a lot of, uh, Mickey Salk and bunch of m- music, music in there also. And clearly, this culture was all about learning and, and understanding, understanding the world and also not quite taking yourself too seriously while you do it, right?

Lex Fridman 18:39 Yeah.

Ben Goertzel 18:39 There's a lot of Yiddish, Yiddish humor in there. So I, I do appreciate that, that culture. Although, the whole idea that, like, the Jews are the chosen people of God never resonated with me too much.

Lex Fridman 18:51 The graph of the Gödel family, I mean, just the people I've encountered just doing some research and just knowing your work through, through the decades, uh, is kind of fascinating. Um, just the, the number of PhDs. (laughs)

Ben Goertzel 19:06 Yeah, yeah, I mean, my, yeah, my, my-

Lex Fridman 19:07 It's kind of fascinating.

Ben Goertzel 19:08 My dad is a sociology professor-

Lex Fridman 19:10 Yeah.

Ben Goertzel 19:10 ... who recently retired from, from Rutgers University. But that, clearly, that gave me a head start in life. I mean, my, my grandfather gave me all his quantum mechanics books when I was like seven or eight years old, and I, I remember going through them and it was all the old quantum mechanics, like R- R- Rutherford atoms and stuff. So I got to the part of wave functions which I didn't understand, although I was a very bright kid. And I realized he, he didn't quite understand it either. But at least... Like, he pointed me to some professor he knew at, at UPenn nearby who, who understood these things, right? So that's, that, that's an unusual opportunity for a kid to have, right? And my, my dad, he was programming Fortran when I was 10 or 11 years old on like HP3000, uh, mainframes at Rutgers University.

Lex Fridman 19:57 Mm-hmm.

Ben Goertzel 19:57 So I got to do linear regression in Fortran on, on punch cards, uh, when, when I was in, in, in middle school, right? 'Cause he, he was doing, I guess, analysis of demographic and, and, and sociology data. So yes, certainly, certainly that gave me a head start and a push towards science be- beyond what would have been the case with m- many, many different

Ben Goertzel 20:19 situations.

Lex Fridman 20:19 When did you first fall in love with AI? Is it the, is it the programming side of Fortran? Is it the, maybe the sociology, psychology that you picked up from your dad or is it the quantum mechanics?

Ben Goertzel 20:28 I fell in love with AI when I was probably three years old when I saw a robot on Star Trek.

Lex Fridman 20:32 (laughs)

Ben Goertzel 20:32 It was turning around in a circle going, "Error, error, error, error," because Spock and Kirk had tricked it into a mechanical breakdown by presenting it with a logical paradox. And I was just like, "Well, this makes no sense. This AI is very, very smart. It's been traveling all around the universe. But these people could trick it with a simple logical paradox." Like why... If, you know, if the human brain can get beyond that paradox, wh- wh- wh- why, why can't, why can't this AI? So I, I, I felt the, the screenwriters of Star Trek had misunderstood the nature of intelligence. And I complained to my dad about it and he... (laughs)

Lex Fridman 21:08 (laughs)

Ben Goertzel 21:09 He, he, he wasn't, he wasn't gonna say anything one way or the other.

Lex Fridman 21:12 Yeah.

Ben Goertzel 21:12 But, you know, I... And before I was born, when my dad was at Antioch College in the, in, uh, the middle of the US, he, he led a, he led a protest movement called SLAM, Student League Against Mortality. They were protesting against death wh- wh- wh- wha- wa- wandering-

Lex Fridman 21:30 Brilliant.

Ben Goertzel 21:30 ... across the campus. So he, he, he was into some futuristic things even back then. But whether AI could confront logical paradoxes or not, he didn't, he didn't know. But that... You know, when I... 10 years after that is something I discovered Douglas Hofstadter's book, Gödel, Escher, Bach, and that was sort of to the same point of AI and paradox and logic, right? 'Cause he was over with, over and over with Gödel's incompleteness theorem, and can an AI really fully model itself reflexively or does that lead you into some paradox? Can the human mind truly model itself reflexively or does that lead you into some paradox? So when... I think that book, Gödel, Escher, Bach, which I think I read when it first came out, I would have been 12 years old or something. I remember it was like 16-hour day. I read it cover to cover and then re-read it.

Lex Fridman 22:18 Oh, really? Wow.

Ben Goertzel 22:19 Re-read it af- I re-read it after that 'cause there was a lot of weird things with little formal systems in there that were hard for me at the time. But that was the first book I read that gave me a feeling for AI as like a practical academic or engineering discipline that, that people were working in. 'Cause before I read Gödel, Escher, Bach, I was into AI from the point of view of a, of a science fiction fan. And I, I had the idea, "Well, it may be a long time before we can achieve immortality and superhuman AGI. So, I should figure out how to build a spacecraft traveling close to the speed of light, go far away, then come back to the Earth in a million years when technology is more advanced and we can build these things." Reading Gödel, Escher, Bach, while it didn't all ring true to me, a lot of it did, and, but I could see, like, there are smart people right now at various universities around me who are actually trying to work on building what I would now call AGI, although Ho- Hofstadter didn't, didn't call it that. So really, it was when I read that book, which would have been probably middle school, that then I started to think, "Well, this, this is something that I could, I could practically work on." Yeah.

Lex Fridman 23:29 As opposed to flying away and waiting it out, you can actually be the, one of the people that actually, uh, builds the systems.

Ben Goertzel 23:34 Yeah, exactly. And if you think about, um, I was interested in what we'd now call nanotechnology and in, uh, human immortality and time travel, all the same cool things as every other, like, science fiction-loving kid. But AI seemed like, if Hofstadter was right, you just figure out the right program, sit there and type it. Like, you, you don't, you don't need to, you don't need to spin stars into weird configurations or get government approval to cut people up and fiddle with their DNA or something, right?

Lex Fridman 24:04 Yeah.

Ben Goertzel 24:05 It's just programming. And then of course, that can achieve anything else. The... There's another book from back then which was by Oda...... Feinbaum, Gerald Feinbaum, who was a, who was a physicist at, at, at Princeton. And that was the Prometheus project. Ah. And this book was written in the late 1960s, though I encountered it in the mid-'70s. But what this book said is in the next few decades, humanity is going to create superhuman thinking machines, molecular nanotechnology, and human immortality. And then the challenge we'll have is what to do with it. Do we use it to expand human consciousness in a positive direction or, or do we use it just to further vapid, uh, consu- consumerism? And what he proposed was that the UN should do a survey on this, and the UN should send people out to every little village in, in remotest Africa or South America and explain to everyone what technology was going to bring the next few decades and the choice that we had about how to use it. (laughs) And let everyone on the whole planet vote about whether we should develop, you know, super AI nanotechnology and, and, and immortality for expanded consciousness or for rampant, rampant consumerism. And needless to say, that didn't quite happen.

Lex Fridman 25:22 Right. (laughs)

Ben Goertzel 25:22 And I think this guy died in the mid-'80s, so he didn't even see his ideas start to become, become more mainstream. But it's interesting, many of the themes I'm engaged with now, from AGI and immortality even to trying to democratize technology as I've been pushing for with singularity

Lex Fridman 25:37 (laughs)

Ben Goertzel 25:38 ... in my work in the blockchain world, many of these themes were there in, you know, Feinbaum's book in the, in the late '60s even. And of course, Valentin Churchin, uh, a Russian writer who, who I... and a great Russian physicist, who I got to know when we both lived in New York in the late '90s and early aughts. I mean, he, he had a book in the late '60s in, in Russia which was The Phenomenon of Science which laid out, laid out all these, all these same things a- a- as well. And Val died in, I don't remember, 2004 or '05 or something of, of Parkinson's disease. So yeah, it's easy, easy for people to lose track now of the fact that the, the futurist and singularitarian advanced technology ideas that are now almost mainstream and are on TV all the time, I mean, these, these are not that new, right? They're sort of new in the history of, of the human species. But I mean, these were all around in fairly mature form in, in the middle of the last century, were written about quite articulately by fairly mainstream people who were professors at, at top universities. It's just until the enabling technologies got to a c- a certain point, then you, you couldn't make it real. So and, and even in the '70s, I was sort of seeing that and, and living, living through it, right? From Star Trek to Douglas Hofstadter, things were getting very, very practical from the late '60s to the late '70s. And, you know, the first computer I bought, you could only program with hexadecimal machine code, and you had to solder it together.

Lex Fridman 27:19 (laughs) Yeah.

Ben Goertzel 27:19 And then, then like a few years later, there's punch cards. Then a few years later, you could get like Atari 400 and Commodore VIC-20 and you could, you could type on the keyboard and program in higher level languages along- alongside the assembly language. So these ideas have been building up a while, and I guess my generation got to feel them-

Lex Fridman 27:41 Mm-hmm.

Ben Goertzel 27:42 ... build up, which is different than people coming into the field-

Lex Fridman 27:45 Now, yeah.

Ben Goertzel 27:46 ... now for whom these things have just been part of the ambiance of, of culture for their whole career or even their, or even their, even their whole life.

Lex Fridman 27:54 Well, it's fascinating to think about, you know, there being all of these ideas kind of swimming, you know, almost with the noise all around the world, all the different generations and, and some kind of non-linear thing happens where they percolate up and, and, uh, capture the imagination of the mainstream.

Ben Goertzel 28:12 Yeah.

Lex Fridman 28:12 And that seems to be what's happening with AI now.

Ben Goertzel 28:14 I mean, Nietzsche, who you mentioned-

Lex Fridman 28:16 Yeah.

Ben Goertzel 28:16 ... had the idea of the superman, right?

Lex Fridman 28:18 Yeah.

Ben Goertzel 28:18 But he, he didn't understand enough about technology to think you could physically engineer a superman by piecing together molec- molecules in, in a certain way. He wa- he was a bit vague about how, how the, how the superman would appear. But he was quite deep in thinking about what the state of consciousness and the mode of cognition of, of a superman would be. He, he was a very astute analyst of, you know, how the human mind constructs the illusion of a self, how it constructs the illusion of free will, h- how, how it constructs values like, like good and evil out of its own, you know, desire to maintain and advance its own organism. He understood a lot about how human minds work, then he understood a lot about how post-human minds would work. I mean, this superman was supposed to be a mind that would basically have complete root access to its own brain and consciousness and be able to architect its, its own, its own value system and inspect and fine-tune all, all of its own, its own biases. So that's a lot of powerful thinking there, which then fed in and, and sort of seeded all of post-modern continental philosophy and all, all, all sorts of, of things that have been very valuable in development of culture and indirectly even, even of technology. But of course, without the technology there, it was all some quite abstract thinking. So now, now we're at a time in history when a lot of these ideas can be, can be made real, which is-

Lex Fridman 29:52 Yeah.

Ben Goertzel 29:52 ... amazing, amazing and scary, right?

Lex Fridman 29:54 It's kind of interesting to think, what do you think Nietzsche would if he was born a century later or transported through time? What, what do you think he would say about AI? I mean-

Ben Goertzel 30:03 Well, those are quite different. If he's born a century later or transported through time, you think that- (laughs)

Lex Fridman 30:07 Well, he'd be, he'd be on like TikTok and Instagram and he would never write the great works he's written. So let's transport him through time.

Ben Goertzel 30:13 Well, yeah. I mean, maybe, maybe also Sprach Zarathustra would be a music video, right?

Lex Fridman 30:16 Yeah. (laughs)

Ben Goertzel 30:17 I mean, I mean, I mean, (laughs) who, who, who knows?

Lex Fridman 30:19 Yeah, but if he was transported through time, do you think, uh...And that, that'd be interesting actually to go back, uh, you, you just made me realize that it's possible to go back and read Nietzsche with an eye of, "Is there some thinking about artificial beings?" I'm sure there, he has ink- he had inklings. I mean, with Frankenstein before him, I'm sure he had inklings of artificial beings somewhere in the text. It'd be interesting to see, to try to read his work to see if he had an... if, if, uh, uh, Superman was actually an AGI system, like if he had inklings of that kind of thinking.

Ben Goertzel 30:57 He didn't.

Lex Fridman 30:58 He didn't.

Ben Goertzel 30:59 No, I, I, I would say not. I mean, uh, he had, he had a lot of inklings of modern cognitive science-

Lex Fridman 31:06 Right.

Ben Goertzel 31:06 ... which are very interesting. If you look in, like, the third part of, of the collection that's been titled The Will to Power, I mean, in, in book three there, there's, there's very deep analysis of thinking processes. But he, he wasn't so much of a physical tinkerer-type, uh, type guy, right? He, he, he was very abstract and...

Lex Fridman 31:29 Do you think,

Lex Fridman 31:30 uh... what do you think about the will to power? Do you think human... what do you think drives humans? Is it, is it, um...

Ben Goertzel 31:37 Oh, an unholy mix of things. I, I, I don't think there's one pure, simple, and elegant objective function dri- driving humans by, by, by any means.

Lex Fridman 31:47 Well, do you think, um, if we look at... I know it's hard to look at humans in aggregate, but do you think overall humans are good or, uh, do we have both good and evil within us that, uh, depending on the circumstances, depending on the whatever-

Ben Goertzel 32:05 Oh.

Lex Fridman 32:05 ... can, can, can, uh-

Ben Goertzel 32:06 So I think-

Lex Fridman 32:06 ... percolate to the top?

Ben Goertzel 32:08 Good and evil are very ambiguous, complicated, and in some ways silly concepts. But if we, we could dig into your question from a couple directions. So I think if you look in evolution, humanity is shaped both by individual selection and what biologists would call group selection, like tribe-level selection, right? So, individual selection has driven us in a selfish DNA sort of way. S- so that each of us does to a certain approximation what will help us propagate our, our DNA to future generations. I mean, that, that, that, that's why I've got, uh, four kids so far, and, uh, and, uh, probably that's not the last one. (laughs)

Lex Fridman 32:53 Yeah. (laughs)

Ben Goertzel 32:53 On the other hand-

Lex Fridman 32:55 I like the ambition. (laughs)

Ben Goertzel 32:56 ... tribal, like group selection means humans, in a way, will do what, what will advocate for the persistence of the DNA of their whole, their whole tribe or, or, or their, their social group. And in biology, you, you have both of these, right? Like a... and you can see, say, an ant colony or beehive, there's a lot of group selection in, in, in the evolution of those social animals. On the other hand, say a, a big cat or some very solitary animal, it's a lot more biased toward, toward individual selection. Humans are, are an interesting balance, and I think this reflects itself in what we would view as selfishness versus altruism to, to, to some extent. So, we just have both of those objective functions contributing to the, the makeup of, of our brains. And then as Nietzsche analyzed in his own way, and others have analyzed in different ways, I mean, we abstract this as, well, we have both good, good and, and, and evil with- within us, right? 'Cause a lot of what we view as evil is really just selfishness, and a lot of what we view as good is altruism, which means doing, doing what's good for the, for, for the tribe. And on that level, we have both of those just baked, baked into us, and that's how it is. Of course, there are psychopaths and sociopaths and people who, you know, get gratified by the suffering of others, and that's, that, that, that's, that's a different thing.

Lex Fridman 34:25 Yeah, those are exceptions-

Ben Goertzel 34:26 But I, I-

Lex Fridman 34:26 ... but on the whole...

Ben Goertzel 34:27 ... yeah. Well, I, I think at, at core we're not purely selfish, we're not purely altruistic, we, we are a mix, and that's, that's the nature of it. And we also have a complex constellation of values that are just very specific to our, our evo- evolutionary history. Like we, you know, we, we love waterways and, and, and mountains, and the, the ideal place to put a house is in the mountain overlooking the water, right? And-

Lex Fridman 34:56 (laughs)

Ben Goertzel 34:57 ... you know, we, we, we care a lot about our, our kids, and we care a little less about our cousins, and even less about our fifth cousins.

Lex Fridman 35:04 Yeah.

Ben Goertzel 35:04 I mean, there are many particularities to human values-

Lex Fridman 35:09 Yeah.

Ben Goertzel 35:09 ... which whether they're good or evil depends on your, (laughs) on, on, on, on your perspective really.

Lex Fridman 35:15 It-

Ben Goertzel 35:15 See, I, I, I spent a lot of time in Ethiopia in Addis Ababa where we have one of our AI development offices for my SingularityNET project. And when I walk through the streets in Addis, you know, there's so... there's people lying by the side of the road, like just living there by the side of the road, dying probably of curable diseases without enough food or medicine. And when I walk by them, you know, I feel terrible, I give them money. When I come back home to the developed world, they're not on my mind that much. I, I do donate some, but, I mean, I, I also spend some of the limited money I have enjoying myself in frivolous ways rather than donating it to those people who are right now, like, starving, dying, and suffering on, on the roadside. So, does that make me evil? I mean, it makes me somewhat selfish and somewhat altruistic, and we each, we each balance that in, in, in our own way, right? So that's... that, that... whether that will be true of all possible AGIs is a, is a, is a subtler question.

Lex Fridman 36:19 So you, you have-

Ben Goertzel 36:19 But that's how humans are.

Lex Fridman 36:21 So, you have a sense, you kinda mentioned that there's a selfish... um, I'm not gonna bring up the whole Ayn Rand idea of, uh, selfishness being the core virtue, that's then a whole interesting kinda-... tangent that I think we'll just-

Ben Goertzel 36:34 (laughs)

Lex Fridman 36:35 ... distract ourselves on it.

Ben Goertzel 36:36 I, I, I, I have to make one amusing comment.

Lex Fridman 36:38 Sure. (laughs)

Ben Goertzel 36:38 Or a comment that has amused me, anyway.

Lex Fridman 36:41 Yeah.

Ben Goertzel 36:41 So the, the... Yeah. I, I, I have extraordinary negative respect for, for Ayn Rand.

Lex Fridman 36:47 Negative. What, what's a negative respect? (laughs)

Ben Goertzel 36:50 (laughs) But when, when I worked with a company called Genescient-

Lex Fridman 36:54 Uh-huh.

Ben Goertzel 36:54 ... which was evolving flies to have extraordinary long lives in, in, in Southern California.

Lex Fridman 37:01 That's fascinating.

Ben Goertzel 37:01 So, we had, we had flies that were evolved by artificial selection to have five times the lifespan of normal fruit flies.

Lex Fridman 37:07 Wow.

Ben Goertzel 37:07 But the population of super long-lived flies was physically sitting in a spare room at an Ayn Rand elementary school in Southern California.

Lex Fridman 37:18 (laughs)

Ben Goertzel 37:18 So, that was just like... Well, if, if I saw this in a movie, I, I wouldn't believe it, right? (laughs)

Lex Fridman 37:22 (laughs) Well, yeah, the universe has a sense of humor in that kind of way. That fits in there. Humor fits in somehow into this whole absurd existence. But you, you mentioned the, the balance between selfishness and altruism as kind of being innate. Do you think it's possible that's kind of an emergent phenomena, those peculiarities of our value system? How much of it is innate? How much of it is something we collectively, kind of like a Dostoevsky novel, bring to life together as a civilization?

Ben Goertzel 37:54 I mean, the, the answer to nature versus nurture is usually both. And, of course-

Lex Fridman 37:59 Right, so-

Ben Goertzel 37:59 ... it's nature versus nurture versus self-organization, as, as, as you mentioned. So, clearly, there are evolutionary roots to individual and group selection, leading to a mix of selfishness and altruism. On the other hand, different cultures manifest that in, in, in, in different ways, while we, we all have basically the same biology. And if you look, if you, if you look at sort of pre-civilized cultures, you have tribes like the Yanomamo in Venezuela, which, which their, their culture is focused on, on killing other, killing other tribes. And you, you have other Stone Age tribes that are, are mostly peaceful and have big taboos against violence. So, you, you can certainly have a big difference in, in how culture manifests these innate biological characteristics. But still, you know, there's probably limits 'cause they're given by our biology. I used to argue this with my great-grandparents, who were Marxists, actually. Because they, they believed in the withering away of the state. Like that, they, they believed that, you know, as you move from capitalism to socialism to communism, people would just become more social-minded, so that a state would be unnecessary and people would just give you, everyone would give everyone else what, what they needed. Now, setting aside that that's not what the various Marxist experiments on the planet seem to be heading toward in, in practice, just as a theoretical point, I was very dubious that human nature could go there. Like at that time, when my great-grandparents were alive, I was just like, you know, "I'm a cynical teenager. I think humans are, humans are just jerks. The state is not going to wither away. If you don't have some structure keeping people from screwing each other over, they're going to do it." So, now I actually don't quite see things that way. I mean, I think the... My feeling now, subjectively, is the culture aspect is more significant than I thought it was when I- when I was a teenager. And I think you could have a human society that was dialed dramatically further toward, you know, self-awareness, other awareness, compassion and sharing than our current society. And, of course, greater material abundance helps. But to some extent, material abundance is a subjective perception also. Because many Stone Age cultures-

Lex Fridman 40:27 Right.

Ben Goertzel 40:27 ... perceived themselves as living in great material abundance, that they had all the food and water they wanted, they lived in a beautiful place, they had sex lives, they, they had children. I mean, they- they- they had abundance without any factories, right? So, I think humanity probably would be capable of fundamentally more positive and joy-filled mode of social existence than what we have now. Clearly, Marx didn't quite have the right idea about how to get there. I mean, he missed, he missed a number of key aspects of human society and its evolution. And if we look at where we are in society now, how to get there is a quite different question, because there are very powerful forces pushing people in different directions than a positive, joyous, compassionate existence, right?

Lex Fridman 41:26 So, if we were to try to, uh... You know, Elon Musk is, uh, dreams of colonizing Mars at the moment. So we, maybe he'll have a chance to start a new civilization with a new governmental system. And certainly there's quite a bit of chaos. We're sitting now... I don't know what the date is, but this is June. There's quite a bit of chaos in all different forms going on in the United States and all over the world. So, there's a hunger for new types of governments, new types of leadership, new types of systems. What... And so what are the forces at play and how do we move forward?

Ben Goertzel 42:04 Yeah, I mean, colonizing Mars, first of all, it's a- it's a super cool thing to do. We- we should be doing it.

Lex Fridman 42:10 So you're- you love the idea?

Ben Goertzel 42:11 Yeah, I mean, it's more important, it's more important than making chocolatey chocolates and-

Lex Fridman 42:16 (laughs)

Ben Goertzel 42:17 ... and sexier lingerie and- and many of the things that we spend a lot more resources on as a species, right?

Lex Fridman 42:24 Yeah.

Ben Goertzel 42:24 So, I mean, we should- certainly should do it. I think the possible futures in which a Mars colony makes a critical difference for humanity are- are- are- are very few. I mean, I- I- I think...... I mean, assuming we make a Mars colony and people go live there in a couple of decades, I mean, their supplies are going to come from Earth. The money to make the colony came from Earth. And whatever powers are supplying the, the, the goods there from, from Earth are going to, in effect, be in, in control of that, of that Mars colony. Of course, there are outlier situations where, you know, Earth gets nuked into oblivion and somehow Mars has been made self-sustaining by that point, and, and then Mars is what allows humanity to persist. But I think the... Those are very, very, very un- un- un- unlikely possibilities.

Lex Fridman 43:19 You don't think it could be a first step on a long journey? Uh, let's-

Ben Goertzel 43:22 Of course, it's a first step on a long journey, which, which is, which is awesome. I'm guessing the colonization of the rest of the physical universe will probably be done by AGIs that are better designed to live in space than by, by the meat machines that, that, that we are. But I mean, who knows? We may cryopreserve ourselves in some superior way to what we know now and, like, shoot ourselves out to Alpha Centauri and beyond. I mean, that's all cool, it's very interesting, and it's much more valuable than most things that humanity is spending its resources on. On the other hand, with AGI, we can get to a singularity before the Mars colony becomes sustaining, for sure, possibly before it's even operational. And so-

Lex Fridman 44:10 Interesting. So your intuition is that's- that's the problem if we really invest resources and we can get to faster than a legitimate, full, like, self-sustaining colonization of Mars?

Ben Goertzel 44:19 Yeah, and it's almo- it's very clear that we will, to me, because there's so much economic value in getting from narrow AI toward, toward AGI, whereas the Mars colony, there's less economic value until you get quite far- far- far out in- into the, into the future. So, I think that's very interesting. I just think it's- it's somewhat, somewhat off to the side. I mean, ju- just as a, I think, say, you know, art and music are- are very, very interesting and I want to see resources go into amazing art and music being, being created. And I'd rather see that than a lot of the garbage that society (laughs) spends their money on. On- on- on the other hand, I don't think Mars colonization or inventing amazing new genres of music is- is not one of the things that is most likely to make a critical difference in the evolution of human or non-human life in- in- in- in this part of the universe o- o- over the next decade.

Lex Fridman 45:19 Do you think AGI is really...

Ben Goertzel 45:21 AGI is- is by far the most important thing that's on the horizon. And then technologies that have direct ability to enable AGI or to accelerate AGI are also very important. For example, say, quan- quantum computing. I don't think that's critical to achieve AGI, but certainly you could see how the right quantum computing architecture could massively accelerate AGI. Similar other- other types of- of nanotechnology, right? Now, the quest to cure aging and end disease, while not in the big picture as important as- as- as AGI, of course it's important to- to all of us as- as- as individual humans. And if someone made a super longevity pill and distributed it tomorrow, I mean, that would be huge and a much larger impact than a Mars colony i- is- is gonna have for quite some time.

Lex Fridman 46:20 But perhaps not as much as an AGI system? I mean, these are...

Ben Goertzel 46:23 No, because if you get... If you can make a benevolent AGI, then all the other problems are solved. I mean, the... If... Then the AGI can be... Once it's as generally intelligent as humans, it can rapidly become massively more generally intelligent than humans. And- and then that- that AGI should be able to solve science and engineering problems much- much better than- than- than human beings, as long as it is, in fact, motivated to do so. That's why I said a- a benevolent AGI. There could

Ben Goertzel 46:53 be other kinds.

Lex Fridman 46:54 Maybe it's good to step back a little bit. I mean, we've been using the term AGI. People often cite you as the creator, or at least the popularizer of the term AGI, artificial general intelligence. Can you tell the origin story of the term?

Ben Goertzel 47:08 Sure, sure.

Lex Fridman 47:09 Or maybe...

Ben Goertzel 47:09 So yeah, I would say I- I launched the term AGI upon the world for- for- for what- what it's worth, without ever fully being in- in- in love with the term.

Lex Fridman 47:20 Right.

Ben Goertzel 47:21 What happened is, I was editing a book, and this process started around 2001 or '02. I think the book came out 2005, funnily. I was editing a book which I provisionally was titling Real AI.

Lex Fridman 47:35 Mm-hmm.

Ben Goertzel 47:35 And I mean, the goal was to gather together fairly serious academic-ish papers on the topic of making thinking machines that could really think in the sense like people can or- or- or even more broadly than people can, right? So then I was reaching out to other folks that I had encountered here or there who were in- interested in- in- in that, which included some- some other folks out of the... Who I knew from the transhumanist and singularitarian world, like Peter Voss, who has a company, AGI Incorporated still in- in California, and included, uh, Shane Legg, who had worked for me at my company WebMind in New York in the late '90s, who by now has become rich and famous.

Lex Fridman 48:20 (laughs)

Ben Goertzel 48:20 He was one of the co-founders of- of Google DeepMind.

Lex Fridman 48:22 Yeah.

Ben Goertzel 48:23 But at that... At that time, Shane was, uh... I think he may have been... Have just started doing his PhD with, uh, Marcus Hutter, who at that time hadn't yet published his book Universal AI, which sort of gives a mathematical foundation for artificial general intelligence. So I reached out to Shane and Marcus and Peter Voss and Pei Wang, who was another former employee of mine who had been Douglas Hofstadter's PhD student, who had his own approach to AGI, and a bunch of... Some Russian folks.... reached out to these guys and they contributed papers for the book. But that was my provisional title, but I never loved it because in the end, you know, I was doing some, what we would now call narrow AI, uh, as well, like applying machine learning to genomics data or chat data for sentiment analysis. And I mean, that work is real. And then in a sense, in a sense, it's, it's really AI. It's just a different kind of, kind of AI. Ray Kurzweil wrote about narrow AI versus strong AI. But that seemed weird to me because first of all, narrow and strong are not antonyms. (laughs)

Lex Fridman 49:37 (laughs) That's right.

Ben Goertzel 49:38 I mean, but secondly, strong AI was used in the cognitive science literature to mean the hypothesis that digital computer AIs could have true consciousness like-

Lex Fridman 49:48 Right.

Ben Goertzel 49:48 ... like human beings. So there was already a meaning to strong AI, which was complexly different, but related, right? So we were tossing around on an, an email list whether, what title it, title it should be, and so we, we talked about narrow AI, broad AI, wide AI, narrow AI, general AI, and I think it, it was either Shane Legg or Peter Vohs on the private email discussion we had, he said, "Well, why don't we go with AGI, artificial general intelligence?" And Pei Wang wanted to do GAI, general artificial intelligence 'cause in Chinese it goes in that order.

Lex Fridman 50:27 (laughs) Right.

Ben Goertzel 50:27 But we figured gay wouldn't work in, in-

Lex Fridman 50:30 (laughs)

Ben Goertzel 50:30 ... in US culture at that time, right?

Lex Fridman 50:32 Yeah. Yeah. Yeah. (laughs)

Ben Goertzel 50:34 So, so we, we went with the-

Lex Fridman 50:36 AGI.

Ben Goertzel 50:36 ... AGI. We used it for the, for the title of that book. And part of Peter and Shane's reasoning was you have the G factor in psychology, which is IQ, general intelligence-

Lex Fridman 50:47 That's right.

Ben Goertzel 50:47 ... right? So you have a meaning of GI, general intelligence-

Lex Fridman 50:51 It's interesting.

Ben Goertzel 50:51 ... in psychology, so then you're looking like artificial GI. So then, then we-

Lex Fridman 50:57 Oh, that makes a lot of sense, I think.

Ben Goertzel 50:57 Yeah. We used that for the, we used that for the title of the book. And so I think, and maybe both Shane and Peter think they invented the term, but, but then later after the book was published, this guy Mark Goodbread came up to me and he's like, "Well, I, I published an essay with the term AGI in, in like 1997 or something." And so I'm just waiting for some Russian to come out and say-

Lex Fridman 51:20 (laughs)

Ben Goertzel 51:20 ... they published that in 1953, right?

Lex Fridman 51:23 (laughs)

Ben Goertzel 51:23 I mean, that term-

Lex Fridman 51:24 For sure.

Ben Goertzel 51:25 That term is not dramatically innovative or anything.

Lex Fridman 51:28 Yeah. No.

Ben Goertzel 51:28 It's one of these obvious in hindsight things, which is also annoying in a way because, you know, J- J- Joshua Bach, who you, you interviewed, he's a close friend of mine, he likes the term synthetic intelligence, which I like much better, but it hasn't, it hasn't actually caught on, right? Because I mean, artificial is a bit off to me 'cause artif- artifice is like a tool or something. But, but not all AGIs are going to be tools. I mean, they may be now, but we're aiming toward making them agents rather than, than tools. And in a way, I don't like the distinction between artificial and natural because I mean, we're, we're part of nature also, and machines are part of, are, are part of nature. I mean, you can look at evolved versus engineered, but that, that's a different, that's a different distinction. Then it should be engineered general intelligence, right? And then general, well, if you look at Marcus Hutter's book Universal AI, what he argues there is, you know, within the domain of computation theory, which is limited but interesting. So if you assume computable environments and computable reward functions, then he articulates what would be a truly general intelligence-

Lex Fridman 52:39 Mm-hmm.

Ben Goertzel 52:40 ... a system called AIXI-

Lex Fridman 52:42 Mm-hmm.

Ben Goertzel 52:42 ... which is quite beautiful.

Lex Fridman 52:43 AIXI.

Ben Goertzel 52:43 AIXI. And that's, that's the middle name of m- of my, uh, latest, uh, child actually, is, uh, uh-

Lex Fridman 52:49 What's the first name?

Ben Goertzel 52:50 First name is Quarxi, Q-U-A-R-X-I-

Lex Fridman 52:52 (laughs)

Ben Goertzel 52:52 ... which my, my wife came up with, but that, that's an acronym for quantum organized rational expanding intelligence. And his-

Lex Fridman 52:59 AIXI is the middle name?

Ben Goertzel 53:01 His middle name is, his middle name is X- Xiphines actually, which is a, mean- means the, the former principle underlying AIXI.

Lex Fridman 53:08 (laughs)

Ben Goertzel 53:08 But in any case-

Lex Fridman 53:09 You're giving Elon Musk's-

Ben Goertzel 53:10 (laughs)

Lex Fridman 53:11 ... new child a run for his money.

Ben Goertzel 53:12 Well, I, I, I, I did it first. He, he, he copied-

Lex Fridman 53:14 You did? (laughs)

Ben Goertzel 53:14 He copied me with this, this new freakish name. But now if I have another baby, I'm going to have to out- outdo him.

Lex Fridman 53:19 Outdo him. It's-

Ben Goertzel 53:20 It's become an, uh, an arms race of weird geeky baby names. Yeah. We'll see what the babies think about it, right? (laughs)

Lex Fridman 53:26 Yeah. (laughs)

Ben Goertzel 53:26 But I mean, my oldest son Zarathustra loves his name and, uh, my daughter-

Lex Fridman 53:31 Lovely name.

Ben Goertzel 53:32 ... Sharazad loves her name. So, so far, basically if you give your kids weird names and you-

Lex Fridman 53:36 They live up to it? (laughs)

Ben Goertzel 53:37 Well, you're obliged to make the kids weird enough that they like the names, right? (laughs)

Lex Fridman 53:41 Yeah.

Ben Goertzel 53:42 It directs their upbringing in a certain way. But yeah, anyway, I mean, what Marcus showed in that book is that a truly general intelligence theoretically is possible, but would take infinite computing power. So then the artificial is a little off. The general is not really achievable within physics as, as, as we know it. And I mean, physics as we know it may be limited, but that's what we have to work with now. Intelligence-

Lex Fridman 54:06 Infinitely general, you mean? Like information-

Ben Goertzel 54:07 Yeah.

Lex Fridman 54:08 Yeah, information processing perspective. Yeah. Sure.

Ben Goertzel 54:10 Yeah. Inte- intelligence is not very well defined either, right? (laughs) I mean, what, what, what, what does it mean? I mean, in AI now it's fashionable to look at it as maximizing an expected reward over the future, but that's, that sort of definition is pathological in various ways. And my, my friend David Weinbaum, AKA Weaver, he had a beautiful PhD thesis on open-ended intelligence, trying to conceive intelligence in a-

Lex Fridman 54:36 Without a reward, without

54:38 (...)

Ben Goertzel 54:38 Yeah. He's just looking at it differently. He's looking at complex self-organizing systems-

Lex Fridman 54:42 Right.

Ben Goertzel 54:42 ... and looking at an intelligent system as being one that, you know, revises and grows and improves itself in conjunction with, with its, with its environment without necessarily there being one objective function it's trying to maximize. Although-... over certain intervals of time. It may act as if it's optimizing a certain objective function.

Lex Fridman 55:01 Right.

Ben Goertzel 55:01 Very much Solaris from Stanisław Lem's novels, right? So the, uh, the point is, artificial general unintelligence-

Lex Fridman 55:07 Don't work. (laughs)

Ben Goertzel 55:08 ... they're all bad.

Lex Fridman 55:09 (laughs)

Ben Goertzel 55:09 On the other hand, everyone knows what AI is.

Lex Fridman 55:11 Yeah.

Ben Goertzel 55:12 And AGI seems immediately comprehensible-

Lex Fridman 55:15 Yeah.

Ben Goertzel 55:16 ... to people with, with a technical background. So, I think that the term has served-

Lex Fridman 55:19 It's stuck.

Ben Goertzel 55:19 ... a sociological function. And now, now, now it's out there everywhere, which, which-

Lex Fridman 55:23 It's stuck.

Ben Goertzel 55:24 ... which baffles me.

Lex Fridman 55:24 It's like KFC.

Ben Goertzel 55:25 Right?

Lex Fridman 55:25 I mean, that's it. You're, we're stuck with AGI, probably for a very long time, until AGI systems take over and rename themselves.

Ben Goertzel 55:33 Yeah. And that, I mean, that's-

Lex Fridman 55:35 And then we'll be biological-

Ben Goertzel 55:36 We're stuck, we're stuck with GPUs too, which mostly have nothing to do with graphics anymore, right? So.

Lex Fridman 55:40 I wonder what the AGI system will call us humans. Uh, maybe-

Ben Goertzel 55:44 Grandpa. Yeah. (laughs)

Lex Fridman 55:46 (laughs) GPs? (laughs)

Ben Goertzel 55:47 Yeah. Grandpa Processing Unit, yeah.

Lex Fridman 55:50 (laughs) Biological Grandpa Processing Units.

Ben Goertzel 55:53 (laughs) Yeah.

Lex Fridman 55:53 Uh, (laughs) okay. So, um,

Lex Fridman 55:56 maybe also just a comment on AGI representing before even the term existed, representing a kind of community. Now, you've talked about this in the past, sort of AI has come in waves. But there's alwa- always been this community of people who dream about creating, uh, general human level super intelligence systems. Uh, can you maybe give your sense of the history of this community as it exists today, as it existed before this deep learning revolution, all, all throughout the winters and the summers of AI?

Ben Goertzel 56:29 Sure. Uh, first I would say, as a side point, the winters and summers of AI are greatly exaggerated-

Lex Fridman 56:37 Yeah.

Ben Goertzel 56:38 ... by, by Americans.

Lex Fridman 56:39 Yeah.

Ben Goertzel 56:40 And, in that if you look at the publication record of the artificial intelligence community since, say, the 1950s, you would find a pretty steady growth and advance of ideas and, and, and papers. And what's thought of as an AI winter or summer was sort of how much money is the US military pumping into AI.

Lex Fridman 57:01 (laughs)

Ben Goertzel 57:02 Which was, was meaningful. On the other hand, there was AI going on in Germany, UK, and, and Japan and, and Russia all, all, all over the place while US military got more and less, less en- en- enthused about AI. So, what, uh, I mean-

Lex Fridman 57:17 That happened to be, just for people who don't know-

Ben Goertzel 57:20 Yeah.

Lex Fridman 57:20 ... the US military happened to be the main source of funding for AI research. So, another way to phrase that is it's up and down of, uh, funding for artificial intelligence research.

Ben Goertzel 57:31 It's true.

Lex Fridman 57:31 Yeah.

Ben Goertzel 57:31 And I would say the correlation between funding and intellectual events was not 100%, right? Because, I mean, in, in Russia as an example, or in Germany, there was less dollar funding than in the US, but many foundational ideas were, were laid out, but it was more theory tha- than implementation, right? And US really excelled at sort of breaking through from theoretical papers to working implementations, which, which did go up and down somewhat with US military funding. But still, I mean, you can look, in the 1980s, Dietrich Dörner in Germany had self-driving cars on the Autobahn, right? And, uh, I mean, this, it was a little early with regard to the car industry, so it didn't catch on such as has, has happened now. But, I mean, that whole advancement of self-driving car technology in Germany was pretty much independent of AI military summers and, and, and winters in the US. So there, there's been more going on in AI globally than not only most people on the planet realize, but than most new AI PhDs realize, because they've come up within a certain sub, sub-field of, of AI and haven't had to look so much, so much beyond that. But I, I would say when I got, when I got my PhD in 1989 in, in mathematics, I was interested in AI already.

Lex Fridman 58:56 In Philadelphia, by the way.

Ben Goertzel 58:56 Yeah. At Tem- I started at NYU, then I transferred to, to Philadelphia, to Temple University.

Lex Fridman 59:02 Yeah.

Ben Goertzel 59:02 Good old North Philly, yeah.

Lex Fridman 59:04 North Philly, yeah.

Ben Goertzel 59:04 Yeah, yeah, yeah. The pearl of, pearl of the US, right?

Lex Fridman 59:08 (laughs)

Ben Goertzel 59:08 Yeah. You never stopped at a red light then 'cause you were afraid-

Lex Fridman 59:11 (laughs)

Ben Goertzel 59:11 ... if you stopped at a red light, someone will carjack you.

Lex Fridman 59:13 (laughs)

Ben Goertzel 59:13 So you drive through every red light, yeah.

Lex Fridman 59:16 Yeah. (laughs)

Ben Goertzel 59:16 It's, it's a, it's a... Ev- every, everyday driving or bicycling to Temple from my house was a, is like a new, new adventure, right? But yeah, when I... The reason I didn't do a PhD in AI was what people were doing in the academic AI field then was just astoundingly-

Lex Fridman 59:32 Uh-huh.

Ben Goertzel 59:33 ... boring and seemed wrongheaded to me. It was really, like, rule-based expert systems and production systems. And I, actually, I loved mathematical logic. I had nothing against logic as the cognitive engine for an AI. But the idea that you could type in the knowledge that AI would need to think seemed just completely stupid and, and, and, and wrongheaded to me. I mean, you can use logic if you want, but somehow the system has got to be-

Lex Fridman 1:00:00 Automated somehow.

Ben Goertzel 1:00:00 ... learning, right?

Lex Fridman 1:00:01 Yeah.

Ben Goertzel 1:00:01 It should be learning from experience. And the AI field then was not interested in learning from experience. I mean, some researchers certainly were. I mean, I, I remember in mid-'80s, I discovered a book by a John Andreas-

Lex Fridman 1:00:17 Mm-hmm.

Ben Goertzel 1:00:17 ... which was, it was about a reinforcement learning system called PURR-PUSS, P-U-R-R dash P-U-S-S, which was an acronym that I can't even remember what it was for, but PUSS anyway. But he, I mean, that was a system that was supposed to be an AGI. And basically, by some sort of fancy, like, Markov decision process learning, it, it was supposed to learn everything just from the bits coming into it and learn to maximize it, its, its reward and, and become, become intelligent, right? So, that was there in academia back then, but it was, like, isolated, scattered, weird people. But all these isolated, scattered, weird people in that period, I mean, they...... they laid the intellectual grounds for what happened later. So you look at John Andreas at University of Canterbury, with his perpo-reinforcement learning Markov system. He was the PhD supervisor for John Cleary in, in, in New Zealand. Now, John Cleary worked with me when I was at Waikato University in, in 1993 in, in New Zealand, and he worked with Ian Witten there, and they launched Weka, which was the first open source machine learning toolkit.

Lex Fridman 1:01:29 Mm-hmm.

Ben Goertzel 1:01:29 Which was launched in, I guess, '93 or '94 when I was at Waikato University.

Lex Fridman 1:01:35 Written in Java, unfortunately.

Ben Goertzel 1:01:36 Written in Java-

Lex Fridman 1:01:37 (laughs)

Ben Goertzel 1:01:37 ... which was a cool language back then, though, right? And-

Lex Fridman 1:01:39 Yeah, I guess it's still, well, it's not cool anymore but it's (laughs) it's powerful.

Ben Goertzel 1:01:43 I, I, I find, like most programmers now, I find Java unnecessarily bloated. But back then it was-

Lex Fridman 1:01:50 Yeah.

Ben Goertzel 1:01:50 ... like Java or C++ basically, and Java was-

Lex Fridman 1:01:52 Object-oriented, so it's nice.

Ben Goertzel 1:01:53 Java was easier for students.

Lex Fridman 1:01:55 Yeah.

Ben Goertzel 1:01:55 Amusingly, a lot of the work on Weka when we were in New Zealand was funded by a U.S., sorry, a New Zealand government grant to use machine learning to predict the menstrual cycles of cows.

Lex Fridman 1:02:07 (laughs)

Ben Goertzel 1:02:08 So in the U.S. all the grant funding for AI was about how to-

Lex Fridman 1:02:11 Military.

Ben Goertzel 1:02:11 ... kill, how to kill people or spy on people.

Lex Fridman 1:02:13 (laughs)

Ben Goertzel 1:02:13 In New Zealand, it's all about cows or kiwifruits, right? (laughs)

Lex Fridman 1:02:16 Yeah.

Ben Goertzel 1:02:16 So, so yeah, anyway, I mean, Andre- John Andreas had his probability theory based reinforcement learning, proto-AGI. John Cleary was trying to do much more am- ambitious probabilistic AGI systems. Now, John Cleary helped do Weka, which is the first open source machine learning toolkit, so the predecessor for TensorFlow and Torch and, and all these things. Also, Shane Legg w- was at, at Waikato working with, working with, with, with John Cleary and Ian Witten and this whole group and then working with my own companies, my company WebMind, an AI company I had in the late '90s with a team there at Waikato University which is how Shane got his head full of AGI, which led him to go on and with Demis Hassabis found DeepMind. So what you can see through that lineage is, you know, in the '80s and '70s, John Andreas was trying to build probabilistic reinforcement learning AGI systems. The technology, the computers just weren't there to support it. His ideas were very similar to what people are doing now. But, you know, although he's long since passed away and didn't become that famous outside of Canterbury, I mean, the lineage of ideas passed on from him to his students to their students. You can go trace directly from there to me and to DeepMind, right? So the... There was a lot going on in AGI that did ultimately lay the groundwork for what we have today. But there wa- there wasn't a community, right? And so wh- when I s- when I started trying to pull together an AGI community, it was in the, I guess, the early aughts when I was living in Washington DC and making a living doing AI consulting for variou- various US government agencies. And I organized the first AGI workshop in 2006, and I mean, it wasn't, it wasn't like it was literally in my basement or something. I mean, it was in the conference room at a Marriott in, in Bethesda. It's n- not that, not that edgy or underground unfortunately, but still-

Lex Fridman 1:04:24 How many people attended?

Ben Goertzel 1:04:25 About 60 or something.

Lex Fridman 1:04:27 That's not bad.

Ben Goertzel 1:04:28 I mean, DC has a lot of AI going on. Probably until the last five or 10 years, much more than Silicon Valley, although it's just quiet because of the nature of wh- what hap- what happens in, in, in DC. Their business isn't driven by PR. Mostly when something starts to work really well, it's taken black and becomes even, even more quiet, right?

Lex Fridman 1:04:48 (laughs) Yeah.

Ben Goertzel 1:04:49 But yeah, the thing is that really had the feeling of a group of starry-eyed mavericks-

Lex Fridman 1:04:55 Mm-hmm.

Ben Goertzel 1:04:55 ... like huddled-

Lex Fridman 1:04:57 Yeah.

Ben Goertzel 1:04:57 ... in a basement, like plotting how to overthrow the narrow AI establishment. And you know, for the first time in some cases, coming together with others who shared their passion for AGI and the technical seriousness about working on it, right? And th- I mean, that's very, very different th- than what we have today. I mean, now, now it's a little bit different. We have AGI conference every year and there's several hundred people rather th- than 50. N- now it's more like this is the main gathering of people who want to achieve AGI and think that, uh, large-scale non-linear regression i- is not the golden path to AGI.

Lex Fridman 1:05:42 Yeah.

Ben Goertzel 1:05:42 So I mean it's-

Lex Fridman 1:05:43 AKA neural networks.

Ben Goertzel 1:05:44 Yeah, yeah, yeah. (laughs)

Lex Fridman 1:05:44 Okay.

Ben Goertzel 1:05:45 Well, certain architectures for-

Lex Fridman 1:05:48 Sure.

Ben Goertzel 1:05:49 ... learning u- using neural networks. So yeah, the AGI conferences are sort of now the main concentration of people not obsessed with deep neural nets-

Lex Fridman 1:05:58 The rebels.

Ben Goertzel 1:05:59 ... and deep reinforcement learning, but s- but still interested in AGI. Not, not the only ones. I mean, th- there's other little conferences and groupings interested in, uh, hum- human-level AI and cognitive ar- architectures and so forth. But yeah, it's been a big shift. Like back then, you couldn't really... It would be very, very edgy then to give a university department seminar that mentioned AGI or human-level AI.

Lex Fridman 1:06:28 Yeah.

Ben Goertzel 1:06:28 It was more like you had to talk about something more short-term and immediately practical than, you know, in the bar after the seminar, you could bullshit about AGI in the same breath as, uh, as time travel or the simulation hypothesis or something, right? Whereas now, now AGI is not only in the academic seminar room. Like you have Vladimir Putin knows what AGI is.

Lex Fridman 1:06:51 (laughs)

Ben Goertzel 1:06:51 And he's like, "Russia needs to become the leader in AGI," right? So national leaders and CEOs of large corporations, I mean-... the CTO of Intel, Justin Rattner. This was years ago, a Singularity Summit conference, 2008 or something. He's like, "We believe Ray Kurzweil, the singularity will happen in 2045, and it will have Intel inside."

Lex Fridman 1:07:13 (laughs)

Ben Goertzel 1:07:14 So, I mean, so-

Lex Fridman 1:07:15 Yeah.

Ben Goertzel 1:07:15 ... it's gone, it's gone from being something which is the pursuit of, like, crazed mavericks, crackpots, and, and science fiction fanatics to being, you know, a, a, a marketing term for large, large corporations and, and national leaders, right? Which i- is a astounding transition, but yeah, in the, in the course of this transition, I think a bunch of sub-communities have formed. And the community around the AGI conference series is certainly one of them. It, it hasn't grown as big as I might have liked it to. On, on the other hand, you know, sometimes a, a modest-sized community can be better for making intellectual progress-

Lex Fridman 1:07:58 Yeah.

Ben Goertzel 1:07:58 ... also. Like, you go to a Society for Neuroscience conference, you have 35,000 or 40,000 neuroscientists. On the one hand, it's, it's amazing. On, on the other hand, you're not going to talk to the leaders of the, of the, of, of the field there if, if, if you're an outsider.

Lex Fridman 1:08:14 Yeah, in the same s- in the same sense, the triple AI, the artificial intelligence, uh, the main kind of generic artificial intelligence (laughs) commu- uh, conference is too big. It's, uh, too amorphous. Like, it, it doesn't make s- uh, uh, it doesn't, it doesn't-

Ben Goertzel 1:08:30 Well, yeah, but, and, and NIPS has become a, a, a company, uh, advertising outlet now, right? So, I mean-

Lex Fridman 1:08:36 (laughs) So, yeah. So, I mean, to, to, to comment on, uh, the role of AGI in the research community, I'd still ... If you look at NeurIPS, if you look at CVPR, if you look at these, uh, ICLR, you know, um, AGI is still seen as the outcast. I would still, I would say in these main machine learning and these main artificial intelligence c- uh, conferences amongst the researchers, I don't know if it's an accepted term yet. I, what I've seen bravely, you mentioned Shane Legg, is DeepMind and then OpenAI are the two places that are, I would say, unapologetically, so far, I think it's actually changing unfortunately, but so far they've been pushing the idea that the goal is to create an AGI.

Ben Goertzel 1:09:22 Well, they have billions of dollars behind them. So, I mean, th- the, the, in the public mind, that, that certainly carries some oomph, right? I mean, I mean, m- m- money always trumps-

Lex Fridman 1:09:30 Yeah, but they also have really strong researchers, right? They-

Ben Goertzel 1:09:33 They do. They're great teams. I mean, DeepMind in particular, b- yeah.

Lex Fridman 1:09:36 And they have, I mean, DeepMind has Marcus Hutter, uh, walking around. I mean-

Ben Goertzel 1:09:40 Yeah.

Lex Fridman 1:09:40 ... there's all these folks who basically their full-time position involves-

Ben Goertzel 1:09:45 I mean-

Lex Fridman 1:09:46 ... dreaming about (laughs) creating AGI.

Ben Goertzel 1:09:47 Yeah. I mean, Google Brain has a lot of amazing A- AGI-oriented people also. I mean, and, and, I mean ... So, I'd say from a public marketing view, DeepMind and OpenAI are the two large well-funded organizations that have put the term and concept AGI out there, sort of, a- a- as part of their public image. But, I mean, they're, they're certainly not ... There, there are other groups that are doing research that seems just as, as, as AGI-ish to me. I mean, including a bunch of groups in, in Google's main, main Mountain View office. So, yeah, uh, it's true. AGI is somewhat away from the mainstream now. But if you compare it to where it was-

Lex Fridman 1:10:35 Right.

Ben Goertzel 1:10:36 ... you know, 15 years ago, there's n- there's, there's, there's been an amazing mainstreaming. You could say the same thing about super longevity research, which is one of my, my application areas that I'm excited about. I mean, I've been talking about this since the '90s, but working on this since 2001. And back then, really to say you're trying to create therapies to allow people to live hundreds or thousands of years, you, you were way, way, way, way out of the, out of the industry academic mainstream. But n- now, you know, Google had, had Project Calico, Craig Venter at Human Longevity Incorporated, and then once, once the suits come marching in, right?

Lex Fridman 1:11:17 (laughs)

Ben Goertzel 1:11:17 I mean, once, once there's big money in it, then people are forced to take it, take it seriously, 'cause th- that's the way mo- modern society works.

Lex Fridman 1:11:24 Yeah.

Ben Goertzel 1:11:24 So, it's still not as mainstream as cancer research, just as AGI is, is not as mainstream as automated driving or something. But the degree of mainstreaming that's happened in the last, uh, you know, 10 to 15 years is, is, is astounding to those of us who've been at it for a while.

Lex Fridman 1:11:42 Yeah, but there's a marketing aspect to the term, but in terms of actual full-force research that's going on under the header of AGI, it's currently, I would say, dominated, maybe you can disagree, dominated by neural networks research, that, uh, the nonlinear regression as you mentioned. Um, like, what's your sense with, with OpenCog, with your work, uh, and in, in general? I was, uh, logic-based systems and expert systems, for me, always seemed to capture a deep element of intelligence that needs to be there. Uh, like you said, it needs to learn. It needs to be automated somehow, but that seems to be missing from a lot of the, a lot of research currently. Um, so, what's your sense ... I guess, one way to ask this question,

Lex Fridman 1:12:36 what's your sense of what kind of things will an AGI system need to have?

Ben Goertzel 1:12:43 Yeah. That, that's a very interesting topic that I've thought about for, for a long time. And, uh, I, I, I think there are many, many different approaches that can work for getting to, to human level AI. So, I, I, I don't, I don't think there's, like, one golden algorithm or one, one golden design that, that, that can, that can work. And I mean, flying machines ...... is the, the much-worn analogy here, right? Like, I mean, you have airplanes, you have helicopters, you, you, you have balloons, you have stealth bombers that don't look like regular airplanes. You, you, you've got all s- blimps.

Lex Fridman 1:13:21 Birds too. (laughs)

Ben Goertzel 1:13:21 Birds, yeah, and, and bugs, right?

Lex Fridman 1:13:24 (laughs) Yeah.

Ben Goertzel 1:13:25 And, and, uh, you, I mean, and there are certainly many kinds of flying machines that-

Lex Fridman 1:13:29 And there's a catapult that you can just launch. (laughs)

Ben Goertzel 1:13:32 And there's bicycle-powered, like, uh, flying machines, right?

Lex Fridman 1:13:36 Nice, yeah.

Ben Goertzel 1:13:36 Yeah, so now these are all analogizable by basic theory of, of aerodynamics, right? Now, so one issue with AGI is we don't yet have the analog of the theory of aerodynamics and that, that's what Marcus Hutter was trying to make with the AXE and his general theory of general intelligence. But that theory in its most clearly articulated parts really only works for either infinitely powerful machines or almost or insanely-

Lex Fridman 1:14:09 (laughs) Yeah.

Ben Goertzel 1:14:09 ... impractically powerful machines. So, I mean, if, if you were going to take a theory-based approach to AGI, what you would do is say, "Well, let's, let's take what's called, say, AXE-TL, which is a, which is Hutter's AXE machine that can work on merely insanely much processing power, rather than infinitely much processing power."

Lex Fridman 1:14:30 What does TL stand for?

Ben Goertzel 1:14:31 Uh, ti- time and length.

Lex Fridman 1:14:33 Okay.

Ben Goertzel 1:14:33 So you're... Basically, how, how, uh-

Lex Fridman 1:14:35 Like constrained somehow.

Ben Goertzel 1:14:36 Yeah, yeah, yeah. So how AXE works basically is each-

Lex Fridman 1:14:40 (laughs)

Ben Goertzel 1:14:40 ... each, each action that it wants to take, before taking that action, it looks at all its history.

Lex Fridman 1:14:46 Yeah.

Ben Goertzel 1:14:47 And then it looks at all possible programs that it could use to make a decision.

Lex Fridman 1:14:51 Yeah.

Ben Goertzel 1:14:51 And it decides like which decision program would have let it make the best decisions according to its reward function over its history, and it uses that decision program to take, to make the next decision, right?

Lex Fridman 1:15:02 Yeah. It's not afraid of infinite resources and (laughs) -

Ben Goertzel 1:15:04 It's searching through the space of all possible computer programs-

Lex Fridman 1:15:08 Yeah.

Ben Goertzel 1:15:08 ... in between each action and its next action.

Lex Fridman 1:15:10 Yeah.

Ben Goertzel 1:15:11 Now, AXE-TL searches through all possible computer programs that have runtime less than T and length less than L.

Lex Fridman 1:15:17 Mm-hmm.

Ben Goertzel 1:15:18 So it's... Which is still an impracticably humongous-

Lex Fridman 1:15:22 Yeah.

Ben Goertzel 1:15:22 ... space, right? So what, what you would like to do to make an AGI and what will probably be done 50 years from now to make an AGI is say, "Okay, well, we, we have some constraints." We have these processing power constraints, and, you know, we have the space and time constraints on, on, on, on, on the, on the program. We have energy utilization constraints, and we have this particular class environments, class of environments that we care about, which may be, say, you know, manipulating physical objects on, on the surface of the Earth, communicating in, in, in human language. I mean, whatever our particular... N- not, not annihilating humanity or what-

Lex Fridman 1:16:02 Yeah.

Ben Goertzel 1:16:02 ... whatever our particular requirements happen to be. If you formalize those requirements in some formal specification language, you should then be able to run a automated programs specializer on AXE-TL, specialize it to the, the computing resource constraints and the particular environment and goal.

Lex Fridman 1:16:23 Yeah.

Ben Goertzel 1:16:23 And then it will spit out, like the, the specialized version of AXE-TL to your resource restrictions and your environment, which will be your AGI, right? And th- that, that, that I think is how our super AGI will create new AGI systems, right?

Lex Fridman 1:16:38 (laughs)

Ben Goertzel 1:16:38 But, but... And that's a very Ru-

Lex Fridman 1:16:40 That just seems really inefficient though.

Ben Goertzel 1:16:41 That's a very Russian approach, by the way.

Lex Fridman 1:16:43 (laughs)

Ben Goertzel 1:16:43 Like the whole field of program specialization ca- ca- came out of Russia.

Lex Fridman 1:16:47 Can you backtrack? So what is program specialization? So it's basically...

Ben Goertzel 1:16:51 Well, take, take, take-

Lex Fridman 1:16:52 Uh...

Ben Goertzel 1:16:52 ... take sorting, for example. You can have a generic program for sorting lists, but what if all your lists you care about are length 10,000 or less?

Lex Fridman 1:17:00 Got it.

Ben Goertzel 1:17:00 You can run an automated program specializer on your sorting algorithm, and it will come up with the algorithm that's optimal for sorting lists of length 1,000 or le- or 10,000 or less, right?

Lex Fridman 1:17:09 That's kind of like... Isn't that what the kind of the process of evolution is, uh, is a program specializer (laughs) to the environment?

Ben Goertzel 1:17:17 Yeah. Yeah, yeah, yeah.

Lex Fridman 1:17:17 So you're, you're kind of evolving human beings or living creatures so that-

Ben Goertzel 1:17:20 Well, exactly. I mean, your, your Russian heritage is, is showing there a bit.

Lex Fridman 1:17:24 (laughs)

Ben Goertzel 1:17:24 So with Alexander Vityev, I mean, uh, and, uh, Peter Anokhin and so on.

Lex Fridman 1:17:28 Yeah.

Ben Goertzel 1:17:28 I mean, there's a... Yeah, there, there's a long history of, of thinking about e- evolution, evolution that way, that way also, right? So... Well, my, my, my point is that what we're thinking of as a human-level general intelligence, you know, if you start from narrow AIs like are being used in the commercial AI field now, then you're thinking, "Okay, how do we make it more and more general?" On the other hand, if you start from AXE or Schmidhuber's Godel machine-

Lex Fridman 1:17:58 Yeah.

Ben Goertzel 1:17:58 ... or these infinite, infinitely powerful, but practically infeasible AIs, then getting to a human level AGI is a matter of specialization.

Lex Fridman 1:18:08 Interesting.

Ben Goertzel 1:18:08 It's like how do you, how do you take these maximally general learning processes and how do you, how do you specialize them so that they can operate within the resource constraints that you have but will achieve the particular things that, that you care about? Because we, we are not... We humans are not maximally general intelligences, right?

Lex Fridman 1:18:28 Right.

Ben Goertzel 1:18:28 If I ask you to run a maze in 750 dimensions, you'll probably be very slow. Whereas in two dimensions, you're, you're probably, you're probably way better, right? So, I mean, we're special b- because our, our hippocampus has a two-dimensional map in it, right?

Lex Fridman 1:18:43 Mm-hmm.

Ben Goertzel 1:18:43 And it does not have a 750-dimensional map in it. So, I mean, we- we're, you know, a peculiar mix of generality and, and, and specialization, right?

Lex Fridman 1:18:55 We probably start quite general at birth. Uh, not obviously...

Ben Goertzel 1:19:00 Well, no, I mean-

Lex Fridman 1:19:00 It's still narrow, but like more general than we are at age, uh, 20 and 30 and 40 and 50 and 60.

Ben Goertzel 1:19:07 I don't think that... I, I, I think it's more complex than that because... I mean, the yo- uh, in, in some sense, a young child...... is less biased and the brain has yet to sort of crystallize into appropriate structures for processing aspects of the physical and social world. On, on, on the other hand, the young child is very tied to their sensorium whereas we can-

Lex Fridman 1:19:30 Right.

Ben Goertzel 1:19:31 ... we can deal with abstract mathematics, like 750 dimensions, and the young child cannot because they, they haven't, they haven't grown what Piaget called the, the formal capabilities. The, the, they haven't learned to abstract yet, right? So, and the ability to abstract gives you a different kind of generality than, than, than what a baby has. So there, there's both more-

Lex Fridman 1:19:53 Right.

Ben Goertzel 1:19:53 ... specialization (laughs) and more generalization that comes with, with the development process, a- actually.

Lex Fridman 1:19:59 I mean, I guess just the, the trajectories of the specialization are most controllable at the young age, I guess is, uh, is one way to put it.

Ben Goertzel 1:20:09 Do you have, do you have kids?

Lex Fridman 1:20:10 No.

Ben Goertzel 1:20:11 All right. (laughs) .

Lex Fridman 1:20:11 (laughs)

Ben Goertzel 1:20:11 They're not as controllable as you think. (laughs)

Lex Fridman 1:20:14 (laughs) To compensate. Okay, cool.

Ben Goertzel 1:20:56 But, yeah. Go- going back to your, your, your original question. So, the, the w- the way I look at human-level AGI is, yeah, how do you specialize, you know, unrealistically inefficient, superhuman brute force learning processes to the specific goals that humans need to achieve, and the specific resources that, that, that we have? And both of these, the goals and the resources and the environments, I mean, all, all this is i- important. And on the, on the resources side, it's important that the hardware resources we're bringing to bear are very different than the human brain. So the way, the way I would want to implement AGI on a, a bunch of neurons in a vat that I could rewire arbitrarily is quite different than the way I would want to create AGI on say a modern server farm of CPUs and GPUs, which in turn may be quite different than the way I would want to implement AGI on, you know, whatever quantum computer we'll, we'll have in, in 10 years, supposing someone makes a robust quantum Turing machine or so- or something, right? So I, I, I think, you know, there, there's been co-evolution of the, the patterns of organization in the human brain and, and the physiological particulars of, of, of the human brain o- o- over time. And when you look at neural networks, that is one powerful class of learning algorithms. But it's also a class of learning algorithms that evolve to exploit the particulars of the human brain as, as a computational substrate. If you're looking at the computational substrate of a modern server farm, you won't necessarily want the same algorithms that, that you want, that you want on a, on a human brain. And, you know, from the right level of abstraction, you, you, you could look at maybe the best algorithms on the brain and the best algorithms on a modern computer network as implementing the same abstract learning and representation processes. But, you know, finding that level of abstraction is its own AGI research-

Lex Fridman 1:23:03 (laughs) . Right.

Ben Goertzel 1:23:03 ... pr- project then, right?

Lex Fridman 1:23:04 Yeah.

Ben Goertzel 1:23:05 So, that's about the hardware side and, and the software side whi- which follows from that. Then regarding what are the requirements, I, I wrote the paper years ago on what I called the embodied communication prior, which was quite similar in intent to Yoshua Bengio's recent paper on the consciousness prior, except I, I didn't want to wrap up consciousness in it because to me, the qualia problem and subjective experience is a very interesting issue also, which, which, which we can chat about.

Lex Fridman 1:23:37 Yeah.

Ben Goertzel 1:23:37 But I would rather keep that philosophical debate distinct from the debate of what kind of biases do you want to put in a general intelligence to give it human-like general int- intelligence.

Lex Fridman 1:23:49 And I'm not sure Yoshua Bengio is, is really addressing that kind of consciousness.

Ben Goertzel 1:23:54 No, I, I, I-

Lex Fridman 1:23:55 He's just using the term.

Ben Goertzel 1:23:56 I love Yoshua to, to pieces.

Lex Fridman 1:23:58 Yeah.

Ben Goertzel 1:23:58 Like he, he's by far my favorite of the, the lions of, of deep learning.

Lex Fridman 1:24:02 (laughs) Yeah.

Ben Goertzel 1:24:03 He's su- such a, such a good-hearted guy and a great thinker.

Lex Fridman 1:24:05 He's a good human being. Yeah, for sure.

Ben Goertzel 1:24:07 I am not, I am not sure he has plumbed to the depths of the philosophy of, of, of consciousness.

Lex Fridman 1:24:13 No, he's using it as a sexy term.

Ben Goertzel 1:24:15 Yeah, yeah, yeah.

Lex Fridman 1:24:15 (laughs)

Ben Goertzel 1:24:16 So, I, I... What I called it was the embodied communication prior.

Lex Fridman 1:24:21 Can you maybe explain it a little bit?

Ben Goertzel 1:24:22 Yeah, yeah. What, what I meant was, you know, what are we humans evolved for? You can say being human, but that's, that's very abstract, right? I mean, we... Our minds control individual bodies which are autonomous agents moving around in a world that's composed largely of solid objects, right? And we have also evolved to communicate via language with other, you know, solid object agents that are going around doing things collectively with us in a, in a world of solid objects. And these things are very obvious, but if you compare them to the scope of all possible intelligences or even all possible intelligences that are physically realizable, that actually constrains things a lot. So, if you start to look at, you know, how would you realize some specialized or constrained version of universal general intelligence in a system that has, you know, limited memory and limited speed of processing-... but whose general intelligence will be biased toward controlling a solid object agent, which is mobile in a solid object world for manipulating solid objects and communicating via language with other similar agents in that same world, right? Then starting from that, you're starting to get a requirements analysis for, for, for human, human-level general intelligence and then that, that leads you into cognitive science. And you can look at... Say, what are the different types of memory that the human mind and, and brain has? And this, this has matured over the last decades. And I got into this a lot in... So, after getting my PhD in math, I was an academic for eight years. I was in departments of Mathematics, Computer Science, and Psychology. When I was in the Psychology department at University of Western Australia, I was focused on cognitive science of, of memory and, and perception. Actually, I was, I was teaching neural nets and deep neural nets and it was multi-layer perceptrons, right? Most-

Lex Fridman 1:26:23 In psychology? (laughs)

Ben Goertzel 1:26:25 (laughs) Yeah. Cognitive science-

Lex Fridman 1:26:26 I love it.

Ben Goertzel 1:26:26 ... it was cro- cross-disciplinary among engineering, math, psychology-

Lex Fridman 1:26:30 Beautiful.

Ben Goertzel 1:26:30 ... philosophy, linguistics, computer, computer science. But yeah, we, we were teaching psychology students to try to model the data from human cognition experiments using multi-layer perceptrons, which was the, the early version of, of a deep neural network. Very, very... Ba- yeah, recurrent backprop was very, very slow to train back then, right? (laughs)

Lex Fridman 1:26:51 So, this is the study of these constrained systems-

Ben Goertzel 1:26:53 Yeah.

Lex Fridman 1:26:53 ... that are supposed to deal with-

Ben Goertzel 1:26:55 Right, right. So-

Lex Fridman 1:26:55 ... physical objects.

Ben Goertzel 1:26:55 ... so if you look, if you look at, if you look at cognitive psychology, you can see there's multiple types of memory which are, to some extent, represented by different subsystems in the human brain. So, we have episodic memory, which takes into account our life history and everything that's happened to us. We have declarative or semantic memory, which is like facts and beliefs abstracted from the particular situations as they occurred in. There's sensory memory, which to some extent, is sense modality-specific and, and to some extent is, is unified acro- across, across sense modalities. There's procedural memory, memory of how to do stuff, like how, how to swing the tennis racket, right? Which is... There's motor memory, but it's also a little more ab- more abstract than, than motor