Wojciech Zaremba: OpenAI Codex, GPT-3, Robotics, and the Future of AI | Lex Fridman Podcast #215

1. 0:00 – 1:18

   Introduction

   1. LFLex Fridman0:00

      The following is a conversation with Wojciech Zaremba, co-founder of OpenAI, which is one of the top organizations in the world doing artificial intelligence research and development. Wojciech is the head of language and co-generation teams, building and doing research on GitHub Copilot, OpenAI Codex, and GPT-3 and who knows, 4, 5, 6, and, and, n+1, and he also previously led OpenAI's robotic efforts. These are incredibly exciting projects to me that deeply challenge and expand our understanding of the structure and nature of intelligence. The 21st century, I think, may very well be remembered for a handful of revolutionary AI systems and their implementations. GPT, Codex, and applications of language models and transformers in general to the language and visual domains may very well be at the core of these AI systems. To support this podcast, please check out our sponsors. They're listed in the description. This is the Lex Fridman Podcast, and here is my conversation with Wojciech Zaremba.

2. 1:18 – 8:20

   The Fermi paradox

   1. LFLex Fridman1:18

      You mentioned that Sam Altman asked about the Fermi paradox, and, uh, the people at OpenAI had really sophisticated, interesting answers, so that's when you knew this is the right team to be working with. So let me ask you about the Fermi paradox about aliens. Why have we not found overwhelming evidence for aliens visiting Earth?

   2. WZWojciech Zaremba1:41

      I don't have a conviction in the answer, but rather kind of probabilistic perspective on what might be a, let's say, possible answers. It's also interesting that the question itself even can touch on the, you know, your typical question of what's the meaning of life because like-

   3. LFLex Fridman1:56

      Yes.

   4. WZWojciech Zaremba1:56

      ... if you assume that like we don't see aliens because they destroy themselves, that kind of upweights the, um, focus on making sure that we won't destroy ourselves.

   5. LFLex Fridman2:06

      Yeah.

   6. WZWojciech Zaremba2:06

      Um, but at the moment, the place where I am actually with my belief, and these things also change over the time, is I think that, uh, we might be alone in the universe, which actually makes life more or let's say consciousness life more kind of valuable, and that means that we should more appreciate it.

   7. LFLex Fridman2:26

      Have we always been alone? So y- what's your intuition about our galaxy, our universe? Is it just sprinkled with graveyards of intelligent civilizations, or are we truly... Is, is life, intelligent life truly unique?

   8. WZWojciech Zaremba2:40

      At the moment, my belief that it, it is unique, but I would say I could also, uh, you know... There was like some footage, uh, released, uh, with UFO objects-

   9. LFLex Fridman2:49

      Mm-hmm.

   10. WZWojciech Zaremba2:49

       ... which makes me actually doubt my own belief.

   11. LFLex Fridman2:51

       Yes.

   12. WZWojciech Zaremba2:52

       Uh, yeah, I can tell you one crazy answer that I have heard.

   13. LFLex Fridman2:56

       Yes.

   14. WZWojciech Zaremba2:56

       Um, so, um, apparently when you look actually at the limits of computation, you can compute more if the temperature of the universe were to drop down. So, one of the things that aliens might want to do if they are truly optimizing to maximize amount of compute which, you know, maybe can lead to more let's say simulations or so, it's instead of wasting current entropy of the universe, because you know, we by living we are actually somewhat wasting entropy.

   15. LFLex Fridman3:28

       Mm-hmm.

   16. WZWojciech Zaremba3:29

       Then you can wait for the universe to cool down, such that you have more computation. Now that's kind of a funny answer, I'm not sure if I believe in it, but that would be one of the reasons why you don't see aliens. It's also possible, see some people say that maybe there is n- not that much point in actually going to other galaxies, uh, if you can go inwards. So, there is no limits of what could be an experience if we could, uh, you know, connect machines to our brains. Uh, while there are still some limits if we want to explore universe.

   17. LFLex Fridman4:00

       Yeah, there could be a lot of ways to go inwards too. Once you figure out some aspect of physics we haven't figured out yet, maybe you can travel to different dimensions. I mean, travel in three-dimensional space may not be the most fun kind of travel. There may be like just a huge amount of different ways to travel and doesn't require a spaceship going slowly in 3D space or spacetime.

   18. WZWojciech Zaremba4:28

       It, it also feels, you know, one of the problems is that speed of light is low and universe is vast.

   19. LFLex Fridman4:34

       Yeah.

   20. WZWojciech Zaremba4:35

       And, um, it seems that actually most likely if we want to travel very far, then, uh, then we would instead of actually sending spaceships with humans that weight a lot, we would send something similar to what Yuri Milner is working on. These are like a huge, uh, sail which is at first powered... Or there, there is a shot of laser from an air, and it-

   21. LFLex Fridman4:58

       Yeah.

   22. WZWojciech Zaremba4:58

       ... can propel it to quarter of speed of light.

   23. LFLex Fridman5:01

       Mm-hmm.

   24. WZWojciech Zaremba5:01

       And, uh, sail itself contains a few grams of equipment, and that might be the way to actually transport matter through universe. But then when you think what would it mean for humans, it means that w- we would need to actually put their 3D printer, and you know, 3D print a human on other planet. I don't know, play them YouTube or let's say, or like a pre- 3D print like huge human right away or maybe a womb or so. Um, yeah.

   25. LFLex Fridman5:29

       With our current techniques of archeology if, if, if a civilization was born and died, uh, long, long enough ago on Earth, we wouldn't be able to tell. And so, that makes me really sad, and so I think about Earth in that same way. How can we leave some remnants if we do destroy ourselves? How can we leave remnants for aliens in the future to discover? Like, here's some nice stuff we've done. Like Wikipedia and YouTube, do we have it like in a satellite orbiting Earth?... with a hard drive. Like, how, how do we say, how do we back up human civilization, uh, for the good parts? Or all of it is good parts, so that, uh, it can be preserved longer than our bodies can. That's, uh, uh, that's kind of a, it's a difficult question. It also requires the difficult acceptance of the fact that we may die, and if we die, we may die suddenly as a civilization.

   26. WZWojciech Zaremba6:29

       So, let's see. I think it kind of depends on the cataclysm. We have observed in other parts of universe that births of gamma rays, uh, these are, uh, high energy, uh, rays of light that actually can apparently kill entire galaxy. So, there might be actually nothing even to, nothing to protect us from it. Um, also when I'm looking actually at the past civilizations, so, it's like Aztecs or so, they disappear from the surface of the Earth, and one can ask, "Why is it the case?" And, um, the way I'm thinking about this, you know, that definitely they had some problem that they couldn't solve, and maybe there was a flood and all of a sudden they couldn't drink, uh, there was no potable water and they all died. And, um, I think that, uh, so far, the best solution to such, uh, problems is, I guess, technology. So, I mean, if they would know that you can just boil water and then drink it after-

   27. LFLex Fridman7:31

       Mm-hmm.

   28. WZWojciech Zaremba7:32

       ... then that would save their civilization. And even now when we look actually at the current pandemic, it seems that, uh, once again, actually science comes to rescue. And somehow science increases size of the action space, and, um, I think that's a good thing.

   29. LFLex Fridman7:47

       Yeah, but nature has a vastly larger action space.

   30. WZWojciech Zaremba7:52

       Yeah, but still it might be a good thing for us to keep on increasing action space. (laughs)
3. 8:20 – 10:57

   Systems of government

   1. WZWojciech Zaremba8:20

      I was wondering, um, why actually even, uh, we have negative impact on the, uh, globe. Because, you know, if you ask every single individual, they would like to have clean air, they would like healthy planet, but somehow it actually is not the case that a- as a collective, we are not going in this direction. I think that there exists very powerful system to describe what we value, that's capitalism. You know, science actually monetarize values, uh, to various activities. At the moment, the problem in the current system is that there's some things which we value, there is no cost assigned to it.

   2. LFLex Fridman8:56

      Mm-hmm.

   3. WZWojciech Zaremba8:56

      So, even though we value clean air or maybe we also, uh, value lack of distraction on, let's say, internet or so, at the moment, these quantities, you know, companies, corporations can pollute them, uh, for free. So, in some sense, I wish or like, and that's, I guess, purpose of politics to, to align the incentive systems. And we are kind of maybe even moving in z- this direction. The first issue is even to be able to measure the things that we value. Then we can actually assign the monetary value to, to them.

   4. LFLex Fridman9:34

      Yeah, and that's, so it's getting the data and also probably through technology enabling people to vote and to move money around in a way that is aligned with their values, and that's very much a technology question. So, like, having one president and congress and voting that happens every four years or something like that, that's a very outdated idea. There could be some technological improvements to that kind of idea.

   5. WZWojciech Zaremba10:03

      So, I'm thinking from time to time about these topics, but it also feels to me that it's, it's a little bit like, uh, it's hard for me to actually make correct predictions what is the appropriate thing to do. I extremely trust, uh, Sam Altman, our CEO o- on these topics. He, um, okay, I'm more on the side of being, I guess, naïve hippie that, uh, yeah. (laughs)

   6. LFLex Fridman10:29

      (laughs) That, that's your life philosophy. Um, well, like, I think self-doubt and, uh, I, I think hippie implies optimism. Those, those two things are pretty, pretty good way to operate.

   7. WZWojciech Zaremba10:43

      I mean, still it is, uh, hard for me to actually understand how the politics works, or like, uh, how this, like, uh, exactly how the things would play out. And Sam is, uh, really excellent with

4. 10:57 – 18:10

   Life, intelligence, and consciousness

   1. WZWojciech Zaremba10:57

      it.

   2. LFLex Fridman10:57

      What do you think is rarest in the universe? You said we might be alone. What's hardest to build is another engineering way to ask that. Life, intelligence or consciousness? So, like, you said that we might be alone. Which is the thing that's hardest to get to? Is it just the origin of life? Is it the origin of intelligence? Is it the origin of consciousness?

   3. WZWojciech Zaremba11:22

      So, um, let me at first explain you my kind of mental model what I think is needed for life to appear.

   4. LFLex Fridman11:29

      Mm-hmm.

   5. WZWojciech Zaremba11:30

      Um, so, I imagine that at some point there was this primordial, uh, soup of, uh, amino acids and maybe some proteins in the ocean, and, uh, you know, some proteins were turning into some other proteins through reaction. And, uh, you can almost think about this, uh, cycle of what, uh, turns into what as there is a graph essentially describing which substance turns into some other substance. And essentially life means that all the sudden in the graph has been created a cycle such that the same thing keeps on happening over and over again. That's what is needed for life to happen. And in some sense you can think almost that you have this gigantic graph and it needs, like, a sufficient number of edges for the cycle to appear.

   6. LFLex Fridman12:15

      Mm-hmm.

   7. WZWojciech Zaremba12:16

      Um-... then, um, from perspective of intelligence and consciousness, uh, my current intuition is that they might be quite intertwined. First of all, it might not be that it's like a binary thing, that you have intelligence or consciousness. It, it seems to be a, a, a more a c- c- continuous component. Let, let's see. If we look for instance on the event networks, uh, recognizing images, uh, people are able to show that the activations of n- these networks correlate, uh, very strongly, uh, with activations in visual cortex-

   8. LFLex Fridman12:51

      Mm-hmm.

   9. WZWojciech Zaremba12:51

      ... uh, of some monkeys. The same seems to be true about language models. Um, also if you, for instance, um, look, um, if you train agent in, um, 3D world, um, at first, you know, it, it, it, it barely recognizes what is going on. Over the time, it kind of recognizes foreground from background. Over the time, it kind of knows where there is a food, uh, and it just follows it. Um, over the time, it actually starts having a 3D perception. So, it is possible for instance to look inside of the head of an agent and ask what would it see if it looks to the right, and the crazy thing is, you know, initially when the agents are barely trained, the- these predictions are pretty bad. Over the time, they, they become better and better. You can still see that if you ask what happens when the head is turned by 360 degrees, for some time, they think that a different thing appears, and then at some stage, they understand actually that the same thing's supposed to appear-

   10. LFLex Fridman13:52

       Wow.

   11. WZWojciech Zaremba13:52

       ... so they get like a understanding of 3D structure. It's also, you know, very likely that they have inside some level of un- uh, of like a symbolic reasoning, like a, there are particular, these symbols for other agents. So, when you look at DOTA, uh, agents, they collaborate together and, uh, and, uh, you know, they, they, th- they have some anticipation of, uh, if, if they would win battle. They have some, some expectations with respect to o- other agents. I might be, you know, too much anthropomorphizing, um, the, the, the how the things look, l- look for me, but, um, the fact that they have a symbol for other agents, uh, makes me believe that, uh, at some stage as the, uh, you know, as they are optimizing for skills, they would have also symbol to describe themselves.

   12. LFLex Fridman14:42

       Mm-hmm.

   13. WZWojciech Zaremba14:43

       Uh, this is like a very useful symbol to have, and this particularly I would call it like a self-consciousness or self-awareness.

   14. LFLex Fridman14:51

       Mm-hmm.

   15. WZWojciech Zaremba14:51

       Uh, and, uh, still it might be different from the consciousness. So, I guess the, the way how I'm understanding the word consciousness, let's say the experience of drinking a coffee or let's say experience of being a bat.

   16. LFLex Fridman15:04

       Mm-hmm.

   17. WZWojciech Zaremba15:04

       Uh, that's the meaning of the word consciousness. It doesn't mean to be awake. Uh, yeah. It feels n- it might be also somewhat related to memory and recurrent connections. So, um, it's kind of like a... if you look at anesthetic drugs, they might be, uh, prev- like a... they, they essentially, they, they disturb, uh, uh, brain waves-

   18. LFLex Fridman15:28

       Mm-hmm.

   19. WZWojciech Zaremba15:28

       ... uh, such that, um, mm, maybe memory is not, not formed.

   20. LFLex Fridman15:33

       Mm-hmm. And so there's a lessening of consciousness when you do that.

   21. WZWojciech Zaremba15:37

       Correct.

   22. LFLex Fridman15:37

       And so that's one way to intuit what is consciousness. The- there's also kind of another element here. It could be that it's, you know, this kind of self-awareness module that you described.

   23. WZWojciech Zaremba15:51

       Mm-hmm.

   24. LFLex Fridman15:52

       Plus the actual subjective experience is a storytelling module that tells us a story about, uh, what we're experiencing.

   25. WZWojciech Zaremba16:05

       The crazy thing... So let's say, I mean, in meditation they teach people not to speak story inside of the head.

   26. LFLex Fridman16:12

       Mm-hmm.

   27. WZWojciech Zaremba16:12

       And there is also some fraction of population who doesn't have actually narrator. I know people who don't have a nar- nara- narrator and, you know, they have to use external people in order to, uh, kind of solve tasks that require internal nar- nar- narr- narrator. Uh, so it seems that it's possible to have the experience without the talk.

   28. LFLex Fridman16:37

       What are we talking about wh- when we talk about the internal narrator?

   29. WZWojciech Zaremba16:40

       So-

   30. LFLex Fridman16:41

       Is, is that the voice when you're like reading a book?
5. 18:10 – 20:23

   GPT language model

   1. WZWojciech Zaremba18:10

      uh, thinking about that even... so, you know, obviously I'm, I, I like thinking about consciousness. Uh, I like thinking about AI as well, and I'm trying to see analogies of these things in AI, what would it correspond to. So, uh-Um, you know, OpenAI trained, uh, uh, a model called GPT, uh, which, uh, can generate, uh, 3:00 PM using text on arbitrary topic.

   2. LFLex Fridman18:40

      Mm-hmm.

   3. WZWojciech Zaremba18:41

      And, um, um, and one way to control GPT is, uh, by putting into prefix at the beginning of the text some information, what would be the story about.

   4. LFLex Fridman18:53

      Mm-hmm.

   5. WZWojciech Zaremba18:53

      Uh, you can have even chat with, uh, um, you know, with GPT by saying that the chat is with Lex or Elon Musk or so.

   6. LFLex Fridman19:01

      Mm-hmm.

   7. WZWojciech Zaremba19:01

      And GPT would just pretend to be you or Elon Musk or so. And, um, uh, it almost feels that this, uh, story that we give ourselves to describe our life, it's almost like things that you put into context of GPT-

   8. LFLex Fridman19:19

      Yeah.

   9. WZWojciech Zaremba19:19

      ... to prime-

   10. LFLex Fridman19:19

       And it generates the ... And so but the, the context we provide the GPT is, uh, is multimodal.

   11. WZWojciech Zaremba19:27

       It's mul- So GPT itself is multimodal. GPT itself, uh, hasn't learned actually from experience of single human but from the experience of humanity. It's a chameleon. You can turn it into anything and in some sense by providing context, uh, it, it, it, you know, behaves as the thing that you wanted it to be.

   12. LFLex Fridman19:47

       Mm-hmm.

   13. WZWojciech Zaremba19:47

       It's interesting that, uh, you know, people have stories of who they are and, uh, as you said, these stories, they help them to operate in the world. Uh, but it's also, you know, interesting, because various people find it out through meditation or so that, uh, there might be some patterns that you have learned when you were a kid that actually are not serving you anymore, and you also might be thinking that that's who you are and that's actually just a story.

   14. LFLex Fridman20:15

       Mm-hmm. Yeah, so it's a useful hack but sometimes it gets us into trouble. It's a local optima.

   15. WZWojciech Zaremba20:21

       It's a local optima.

6. 20:23 – 24:40

   Engineering consciousness

   1. WZWojciech Zaremba20:23

   2. LFLex Fridman20:23

      You wrote that Stephen Hawking ... You tweeted, "Stephen Hawking asked what breathes fire into equations, which meant what makes given mathematical equations realize the physics of a universe? Similarly, I wonder what breathes fire into computation. What makes given computation conscious?" Okay, so how do we engineer consciousness? How do you breathe fire and magic into the machine?

   3. WZWojciech Zaremba20:52

      So, um, it seems clear to me that not every con- uh, computation is conscious. I mean, you can, let's say, just keep on multiplying one matrix over and over again and it might be gigantic matrix, you can put a lot of computation, I don't think it would be conscious. So in some sense the question is, uh, what are the computations which could be conscious? Uh, I mean, so, so, uh, one assumption is that it has to do purely with computation that you can abstract the way matter. Uh, other possibility is there ... it's very important what's the realization of computation that it has to do with some, uh, uh, force fields or so and they bring consciousness.

   4. LFLex Fridman21:30

      Mm-hmm.

   5. WZWojciech Zaremba21:30

      At the moment my intuition is that it can be fully abstracted away. So in case of compu- computation you can ask yourself, what are the mathematical objects or so that could bring such a properties? So for instance, if we think about the models, uh, AI models, the ... what they truly try to do, uh, or like models like GPT is, uh, uh, you know, they try to predict next word or so.

   6. LFLex Fridman22:00

      Mm-hmm.

   7. WZWojciech Zaremba22:00

      And this turns out to be equivalent to, uh, compressing, uh, text. Um, and, uh, because in some sense compression means that, uh, you learn the model of reality and you have just to, uh, remember what are your mistakes, the better you are in predicting the ... And, and, and in some sense when we look at our experience also when you look, for instance, at the car driving you know in which direction it will go, you are good, like, uh, in prediction and, um, you know, it might be the case that the consciousness is intertwined with, uh, compression. It might be also the case that self-consciousness, uh, has to do with compressor trying to compress itself.

   8. LFLex Fridman22:42

      Mm-hmm.

   9. WZWojciech Zaremba22:42

      So, um, okay, I was just wondering what are the objects in, you know, mathematics or computer science which are mysterious-

   10. LFLex Fridman22:51

       Mm-hmm.

   11. WZWojciech Zaremba22:51

       ... that could, uh, that, that, that could have to do with consciousness and th- uh, then I thought, um, you know, you, you see in, uh, mathematics there is something called Gödel theorem.

   12. LFLex Fridman23:01

       Mm-hmm.

   13. WZWojciech Zaremba23:02

       Uh, which means, okay, you have ... if you have sufficiently complicated mathematical system it is possible to point the mathematical system back on itself.

   14. LFLex Fridman23:11

       Mm-hmm.

   15. WZWojciech Zaremba23:11

       In computer science there is, uh, something called halting problem.

   16. LFLex Fridman23:14

       Mm-hmm.

   17. WZWojciech Zaremba23:14

       It's, it's somewhat similar construction. So I thought that, you know, if we believe that, uh, that, uh, that under assumption that consciousness has to do with, uh, with compression, uh, then you could imagine that the... that the ... As they keep on compressing things then at some point it actually makes sense for the compressor to compress itself.

   18. LFLex Fridman23:37

       Meta-compression.

   19. WZWojciech Zaremba23:38

       Yeah.

   20. LFLex Fridman23:39

       Consciousness is meta-compression. (laughs)

   21. WZWojciech Zaremba23:41

       That's, uh, uh, that's an i- an, an, an idea.

   22. LFLex Fridman23:45

       (laughs)

   23. WZWojciech Zaremba23:45

       And in some sense, you know, the crazy-

   24. LFLex Fridman23:46

       I love it.

   25. WZWojciech Zaremba23:47

       Th- um, thank you.

   26. LFLex Fridman23:48

       So, uh, but do you think if we think of a Turing machine, a universal Turing machine, can that achieve consciousness? So is there something beyond our traditional definition of computation that's required?

   27. WZWojciech Zaremba24:03

       S- so it's a specific computation and I said this computation has to do with compression.

   28. LFLex Fridman24:07

       Mm-hmm.

   29. WZWojciech Zaremba24:08

       And, uh, the compression itself ... Uh, maybe other way of putting it is like you're internally creating the model of reality in order ... Like a, like a you try inside to simplify reality in order to predict what gonna happen.

   30. LFLex Fridman24:21

       Mm-hmm.
7. 24:40 – 32:02

   Is there an algorithm for intelligence?

   1. LFLex Fridman24:40

      Are you by any chance, uh, aware of, uh, the Hutter Prize? Marcus Hutter? He, uh, he made this prize for compression of Wikipedia pages and, uh, there's a few qualities to it. One I think has to be perfect compression, which makes ... I think that little quirk makes it much less, um, applicable to the general task of intelligence 'cause, uh, it feels like intelligence is always going to be messy. Uh, i- like, perfect compression is ... feels like it's not the right goal but it's nevertheless a very interesting goal. So for him, intelligence equals compression, and so the smaller you make the file given a large Wikipedia page, the more intelligent the system has to be.

   2. WZWojciech Zaremba25:31

      Yeah, that makes sense. So you can make perfect compression if you store errors.

   3. LFLex Fridman25:34

      Mm-hmm.

   4. WZWojciech Zaremba25:35

      And I think that actually what he meant is you have algorithm plus errors.

   5. LFLex Fridman25:38

      Yeah.

   6. WZWojciech Zaremba25:38

      Uh, by the way, Hutter h- Hutter is a ... He was PA- uh, uh, PhD advisor of Yann LeCun who is a DeepMind, uh, uh, DeepMind co-founder.

   7. LFLex Fridman25:48

      Yeah. Yeah, so there's an interesting, uh, and now he's at DeepMind. There's an interesting, uh, network of people. He's one of the people that I think seriously took on the task of, what would an AGI system look like? I think for the longest time, the, the question of AGI was not taken seriously or rather rigorously and he, he did just that. Like, mathematically speaking, what would the model look like? If you remove the constraints of it having to be, uh, um ... having to have a reasonable amount of memory, reasonable bo- amount of, uh, running time complexity, uh, c- computation time, what would it look like? And essentially it's, it's a half math, half philosophical discussion of, uh, how would a, like, a reinforcement learning type of, uh, framework look like for an AGI.

   8. WZWojciech Zaremba26:47

      Yeah, so he developed a framework even to describe what's optimal with respect to reinforcement learning, like, there is a theoretical framework which is, as you said, under assumption there is infinite amount of memory and compute.

   9. LFLex Fridman26:59

      Mm-hmm.

   10. WZWojciech Zaremba26:59

       Um, there was actually one person before. His name is Solomonoff. Hutter extended, uh, Solomonoff work to reinforcement learning but there exists a, a theoretical algorithm which is optimal algorithm to build intelligence and I can actually explain you the algorithm.

   11. LFLex Fridman27:17

       Yes.

   12. WZWojciech Zaremba27:18

       (laughs)

   13. LFLex Fridman27:18

       Let's go. Let's go. Let's go.

   14. WZWojciech Zaremba27:20

       So the task itself you can-

   15. LFLex Fridman27:22

       C- Can I just pause how absurd it is for a brain in a skull trying to explain the algorithm for intelligence? Just go ahead.

   16. WZWojciech Zaremba27:31

       It is pretty crazy. It is pretty crazy that, you know, the brain itself is actually so small and it can't f-

   17. LFLex Fridman27:36

       (laughs)

   18. WZWojciech Zaremba27:36

       ... ponder, uh ...

   19. LFLex Fridman27:39

       How to design algorithms that optimally solve the problem of intelligence? Okay. All right. So what's the algorithm?

   20. WZWojciech Zaremba27:45

       So let's see. So first of all, the task itself is, uh, described as, uh, you have infinite sequence of zeros and ones, okay? You read, uh, n bits and you are about to predict n plus one bit.

   21. LFLex Fridman27:58

       Mm-hmm.

   22. WZWojciech Zaremba27:59

       So that's the task and you could imagine that every task could be casted as such a task. So if for instance you have images and labels, you can just turn every image into sequence of zeros and ones, then label ... You concatenate labels and you ... and that, that's actually the, the ... and you could, you could start by having training data first, and then afterwards you have test data. So theoretically any problem could be casted as a problem of predicting zeros and ones on this, uh, infinite type.

   23. LFLex Fridman28:29

       Mm-hmm.

   24. WZWojciech Zaremba28:30

       So, um, so let's say you read already n bits and you want to predict n plus one bit. Then I will ask you to write every possible program that generates these n bits. Okay? So, um, and you can have ... you, you, you choose programming language. It can be in Python or C++. And the difference between programming languages, uh, might be there is a difference by constant. Asymptotically, your predictions will be equivalent.

   25. LFLex Fridman28:58

       Mm-hmm.

   26. WZWojciech Zaremba29:00

       So you, you read n bits, you enumerate all the programs that produce these n, n bits in their output.

   27. LFLex Fridman29:07

       Mm-hmm.

   28. WZWojciech Zaremba29:07

       And then in order to predict n plus one bit, you actually weight the programs according to their length, uh, there is-

   29. LFLex Fridman29:17

       Hm.

   30. WZWojciech Zaremba29:17

       ... like some specific formula how you weight them and then the n plus, uh, one bit prediction is the prediction, uh, from each of this program according to that weight.
8. 32:02 – 44:53

   Neural networks and deep learning

   1. WZWojciech Zaremba32:02

   2. LFLex Fridman32:02

      Let me ask you the high-level basic question that's not so basic. What is deep learning?

   3. WZWojciech Zaremba32:09

      Hmm.

   4. LFLex Fridman32:10

      Is there a way you'd like to think of it that is different than like a generic textbook definition?

   5. WZWojciech Zaremba32:16

      The thing that I hinted just a second ago is maybe the, uh, closest to how I'm thinking these days about, um, deep learning. So, uh, you know, the statement is, uh, neural networks can represent some programs. Uh, it seems that various modules that we are actually adding up to- or like a... You know, we, we want networks to be deep because we, we want multiple steps of the computation.

   6. LFLex Fridman32:42

      Mm-hmm.

   7. WZWojciech Zaremba32:45

      And, uh, uh, and deep learning provides the way to represent space of programs which is searchable, and it's searchable with, uh, stochastic gradient descent. So, we have an algorithm to search over a humongous number of programs and gradient descent kind of bubbles up the things that are, uh, tend to give correct answers.

   8. LFLex Fridman33:04

      So, a neural network with a, with fixed weights that's optimized. Do you think of that as a single program?

   9. WZWojciech Zaremba33:14

      Um, so there is a, uh, work by Christopher Olah where he, uh... So, he works on interpretability of neural networks, and he was able to, uh, to identify inside of the neural network, for instance, a detector of a wheel for a car or the detector of a mask for a car-

   10. LFLex Fridman33:33

       Mm-hmm.

   11. WZWojciech Zaremba33:34

       ... and then he was able to separate them out and assemble them, uh, together using a simple program, uh, for the detector, for a car detector.

   12. LFLex Fridman33:43

       That's like a... I- if you think of traditionally defined programs, that's like a function within a program that this particular neural network was able to find, and you can tear that out just like you can copy and paste from Stack Overflow. That... So, uh, a- any program is a composition of smaller programs?

   13. WZWojciech Zaremba34:03

       Yeah. I mean, the nice thing about the neural networks is that it allows the things to be more fuzzy than in case of programs. Uh, in case of programs, you have this like a branching this way or that way, and the neural networks, they, they have an easier way to, to be somewhere in between or to share things.

   14. LFLex Fridman34:22

       What to you is the most beautiful or surprising idea in deep learning-

   15. WZWojciech Zaremba34:26

       Mm.

   16. LFLex Fridman34:26

       ... and the utilization of these neural networks? Which, by the way, for people who are not familiar, neural networks is a bunch of, um... What would you say? It's inspired by the human brain. There's neurons. There's connection between those neurons. There's inputs and there's outputs, and there's millions or billions of those neurons, and the learning happens, uh, by adjusting the weights on the edges that connect these neurons.

   17. WZWojciech Zaremba34:54

       Thank you for giving definition. (laughs) I'm supposed to do it, but I guess you have enough empathy to-

   18. LFLex Fridman34:59

       No, you-

   19. WZWojciech Zaremba34:59

       ... listeners to actually know that the, that that might be useful.

   20. LFLex Fridman35:02

       No, that's like... So I'm asking Plato of like, "What is the meaning of life?" He's not gonna answer str- you're, you're being philosophical and deep and quite profound talking about the space of programs, which is, which is very interesting, but also (laughs) for people who's just not familiar what the hell we're talking about when we talk about deep learning. Anyway, sorry. What is the most beautiful or surprising idea to you in, in, um, in all the time you've worked at deep learning? And you worked on a lot of fascinating projects, applications of neural networks. It doesn't have to be big and profound. It could be a cool trick.

   21. WZWojciech Zaremba35:38

       Yeah. I mean, I'm thinking about the trick, but like, uh, it's still a- amusing to me that it works at all.

   22. LFLex Fridman35:44

       Yeah.

   23. WZWojciech Zaremba35:44

       That, let's say that the extremely simple algorithm stochastic gradient descent, which is something that I would be able, you know, to derive on the piece of paper to high school student, uh, when put at the ins- at the scale of, you know, thousands of machines actually, uh, can create the behaviors we, which we call kind of human-like behaviors.

   24. LFLex Fridman36:08

       So, in general any application of stochastic gradient descent to neural networks is, is amazing to you? So, the... Or is there a particular application in natural language, reinforcement learning? Uh, and also, would you attribute that success to... is it just scale? What profound insight can we take from the fact that (laughs) the thing works for gigantic, uh, sets of variables?

   25. WZWojciech Zaremba36:40

       I mean, the interesting thing is these algorithms, they were, uh, invented, uh, decades ago and, uh, people actually, uh, gave up on the idea.

   26. LFLex Fridman36:51

       Yeah.

   27. WZWojciech Zaremba36:51

       And, um...... you know, back then, they thought that we need profoundly different algorithms and they spent a lot of cycles on very different algorithms. And I believe that the, you know, we have seen that various al- uh, various innovations, let's say, like transformer or, uh, or dropout or so they can, uh, you know, vastly help. But it's also remarkable to me that this algorithm from '60s or so, uh, are, I, I mean, you can even say that the gradient descent was invented by Leibniz in, I guess, 18th century or so, that actually is the core of learning. In the past, people are, uh, it's almost like a, out of the maybe an ego, people are saying that it cannot be the case that such a simple algorithm-

   28. LFLex Fridman37:39

       Mm-hmm.

   29. WZWojciech Zaremba37:39

       ... is the, you know, uh, could solve complicated problems. So they were, uh, in search for the other algorithms. And as I'm saying, like, I believe that actually we are in the game where there is, there are actually, frankly, three levers. There is compute, there are algorithms, and there is data. And, uh, if we want to build intelligent systems, we have to pull, uh, all three levers and they are actually multiplicative. Um, it's also interesting, so you ask, is it only compute? Uh, people internally, they did the studies to determine how much gains they were coming from different levers. And so far, we have seen that more gains came from compute than algorithms. But also we are in the world that in case of compute, there is a kind of, you know, exponential increase in funding and, at some point, it's impossible to, uh, invest more. It's impossible to, you know, invest $10 trillion 'cause we are speaking about the, let's say, all taxes in US, uh-

   30. LFLex Fridman38:38

       But you're talking about money, there could be innovation in the compute.
9. 44:53 – 49:47

   Human reward functions

   1. WZWojciech Zaremba44:54

   2. LFLex Fridman44:54

      And, uh, you know, therapist is, is a funny word 'cause I see friendship and love as therapy. I mean, so therapy is broadly defined as just friendship, as a friend. So like therapist is, has a very kinda clinical sense to it. But what is human connection? You're like, uh, (laughs) not to get all Camus and Dostoevsky on you, but, you know, life is suffering and we draw ... we seek connection with other humans as we, uh, desperately try to make sense of this world in the deep overwhelming loneliness that we feel inside.

   3. WZWojciech Zaremba45:34

      So I think connection has to do with understanding. And I think that almost like a lack of understanding causes suffering. If you speak with someone and you ... you feel ignored, that actually causes pain. If you are feeling deeply, uh, understood, that actually c- ... that they, they might not even tell you what to do in life-

   4. LFLex Fridman45:53

      Yeah.

   5. WZWojciech Zaremba45:54

      ... but like a pure understanding.

   6. LFLex Fridman45:56

      Or just being heard. Understanding is a kinda ... it's a lot, you know. Just being heard, feel like you're being heard. Like, somehow that's, uh, alleviation temporarily of the loneliness, that if somebody f- ... knows you're here, with their body language, with the way they are, with the way they look at you, with the way they talk, you feel less alone for a brief moment.

   7. WZWojciech Zaremba46:23

      Yeah. Ver- very much agree. So I thought in the past about, uh, somewhat similar question to yours which is, what is love? Uh, rather what is connection.

   8. LFLex Fridman46:33

      Yes.

   9. WZWojciech Zaremba46:33

      And, um, and obviously I think about these things from AI perspective, what would it mean. Um, so I said that the, you know, intelligence has to do with some compression which is more or less like you can say almost understanding of what is going around. It seems to me that, uh, other aspect is there seem to be reward functions and you can have a, a, you know, reward for, uh, food, for maybe human connection, for, uh, let's say warmth, uh, sex, and so on. And, um, and it turns out that, uh, various people might be optimizing slightly different, uh, reward functions. They essentially might care about different things. And, um, uh, in case of, uh, love, at least the love between two people, you can say that the, um, you know, boundary between people dissolves to such extent that, uh, they end up optimizing each other reward functions.

   10. LFLex Fridman47:30

       Mm-hmm.

   11. WZWojciech Zaremba47:31

       Uh ... (laughs)

   12. LFLex Fridman47:33

       (laughs) Yeah. Oh, th- that's interesting. Um, celebrate the success of each other.

   13. WZWojciech Zaremba47:39

       Yeah. Or in some sense, I would say love means, uh, helping others to optimize their, uh, reward functions, not your reward functions, not the things that you think are important, but the things that the person cares about, you'll try to help them to, uh, optimize it.

   14. LFLex Fridman47:55

       So love is, uh, if you think of two reward functions, you just ... it's like addition.

   15. WZWojciech Zaremba48:00

       Yeah.

   16. LFLex Fridman48:00

       You combine them together.

   17. WZWojciech Zaremba48:01

       Yeah. Pretty much.

   18. LFLex Fridman48:02

       Maybe like with a weight and depends like the dynamic of the relationship.

   19. WZWojciech Zaremba48:06

       Yeah. I mean, you could imagine that if you are fully, uh, optimizing someone's reward function without yours, then-

   20. LFLex Fridman48:11

       Yeah.

   21. WZWojciech Zaremba48:11

       ... then maybe you are creating codependency or something like that.

   22. LFLex Fridman48:14

       Yeah. (laughs)

   23. WZWojciech Zaremba48:14

       The ... I, I'm not sure what's the appropriate weight. But the interesting thing is I even, I even think that the, uh, individual person, uh, uh, we ourselves we are actually less of a unified insight. So for instance, if you look at, at the donut, on the one level, you might think, "Oh, this like, uh, looks tasty. I would like to eat it." On other level, you might tell yourself, "I shouldn't be doing it because, uh, I want to gain muscles." So a- and, you know, you might do it regardless, kind of against yourself. So it seems that even within ourselves, they're almost like a kind of intertwined personas.

   24. LFLex Fridman48:51

       Mm-hmm.

   25. WZWojciech Zaremba48:51

       And, um, I believe that the self-love means that, uh, the love between all these personas, which also means being able to love...... love, uh, yourself when you are angry or stressed or so.

   26. LFLex Fridman49:06

       Combining all those reward functions of the different selves you have.

   27. WZWojciech Zaremba49:09

       Yeah, and accepting that they are there. Like, you know, often people, they have a negative self-talk or they say, "I don't like when I'm angry."

   28. LFLex Fridman49:17

       Yeah.

   29. WZWojciech Zaremba49:17

       And like, try to imagine, try to imagine if there would be like a small baby, Alex, like five years old, and-

   30. LFLex Fridman49:26

       Who's angry.
10. 49:47 – 52:14

    Love is part of the human condition

    1. LFLex Fridman49:47

       But still that doesn't explain the why of love. Like, why is love part of the human condition? Why is it useful to combine the reward functions? It seems like that doesn't... I mean, I don't think reinforcement learning frameworks can give us answers to why. Even e- even the hotter framework has a objective function that's static.

    2. WZWojciech Zaremba50:09

       So, we came to existence as a consequence of evolutionary process.

    3. LFLex Fridman50:14

       Mm-hmm.

    4. WZWojciech Zaremba50:14

       And, in some sense, the purpose of evolution is survival.

    5. LFLex Fridman50:17

       Yeah.

    6. WZWojciech Zaremba50:17

       And then the, this complicated optimization objective baked into us, let's say, compression, which might help us, uh, operate in the real world and be baked into us by these reward functions.

    7. LFLex Fridman50:30

       Yeah.

    8. WZWojciech Zaremba50:31

       Uh, then, to be clear, at the moment, we are operating in the regime which is somewhat out of distribution where the even evolution optimized us.

    9. LFLex Fridman50:38

       It's almost like love is a consequence of, uh, cooperation that we've discovered is useful.

    10. WZWojciech Zaremba50:44

        Correct. In some way, it's even the case, if you-

    11. LFLex Fridman50:47

        I just love the idea that love is, like, the out of distribution... (laughs)

    12. WZWojciech Zaremba50:51

        Oh, it's not out of distribution. It's like Assi said, it evolved for co- cooperation.

    13. LFLex Fridman50:55

        Yes.

    14. WZWojciech Zaremba50:56

        And I believe that the co-op- like, uh, in some sense, cooperation ends up helping each of us individually. So, it makes sense evolutionary. And there is a, in some sense... And, you know, love means there is this dissolution of boundaries that you have a shared reward function.

    15. LFLex Fridman51:09

        Mm-hmm.

    16. WZWojciech Zaremba51:10

        And we evolved to actually identify ourselves with larger groups. So, we, we can identify ourselves, you know, with our family. We can identify ourselves with a country to such extent that people are willing to give away their life for a country.

    17. LFLex Fridman51:24

        Mm-hmm.

    18. WZWojciech Zaremba51:25

        Uh, so there is... We are wired actually even, uh, uh, for love. And, at the moment, I guess the, hmm, maybe it would be somewhat more beneficial if you w- if we would identify ourselves with all the humanity as a whole. So, so you can clearly see when people travel around the world when they run into person from the same country, they say, "Oh, we see PR," and all this... Like, all the sudden, they f- find all these similarities. They, they, they find some... They, they befriend th- those folks earlier than others. Uh, so there is like a sense, some sense of the belonging. And I would say I think it would be overall good thing to the world, uh, for people to move towards, uh, I think it's even called open individualism. Uh, move toward the, uh, mindset of a larger and larger groups.

11. 52:14 – 56:19

    Expanding our circle of empathy

    1. LFLex Fridman52:14

       So, the challenge there, a- that's a beautiful vision and I share it, to expand that circle of empathy, that circle of love towards the entirety of humanity. But then you start to ask, well, where do you draw the line? Because why not expand it to other conscious beings? And then at the finally, for our discussion, something I think about is why not expand it to AI systems? Like, we, we start respecting each other when the oth- the person, the entity on the other side has the capacity to suffer, 'cause then we develop a capacity to sort of empathize. And so I could see AI systems that are interacting with humans more and more having conscious-like displays. So like, they display consciousness through language and through other means. And so then the question is like, well, is that consciousness? Because they're acting conscious. And so, you know, the reason we don't like torturing animals is because they look like they're suffering when they're tortured. And if AI looks like it's suffering when it's tortured, how is that not requiring of the same kind of empathy from us, and respect, and rights that animals do and other humans do?

    2. WZWojciech Zaremba53:39

       I think it requires empathy as well. I mean, I would like, guess, us or humanity or so make a progress in understanding what consciousness is, because I don't want just to be speaking about the, the philosophy but rather actually make a scientific... Uh, to have a, like a, you know, there was a time that people thought that, uh, there is a force of life. And, uh, the things that have this force, they are alive.

    3. LFLex Fridman54:05

       Mm-hmm.

    4. WZWojciech Zaremba54:07

       And, um, I think that there is actually a path to understand exactly what consciousness is. And, uh, um, in some sense, it might require essentially putting probes inside of a human brain-

    5. LFLex Fridman54:21

       Mm-hmm.

    6. WZWojciech Zaremba54:21

       ... uh, what Neuralink, uh, does.

    7. LFLex Fridman54:23

       So the goal there... I mean, there's several things with consciousness that make it a real discipline, which is one, is rigorous measurement of consciousness-

    8. WZWojciech Zaremba54:31

       Mm-hmm.

    9. LFLex Fridman54:32

       ... and then the other is the engineering of consciousness, which may or may not be related. I mean, you: could also run into trouble, like for example, in the United States for, for the department, DOT, Department of Transportation, you know, a lot of different places put a value on human life. I think DOT's, uh, value is $9 million per person. Sort of in that same way, you can get into trouble if you put a number on how conscious a being is, 'cause then you can start making policy. If a cow is, uh, 0.1...... or, like, um, 10% as conscious as a human, then you can start making calculations and it might get you into trouble. But then again, that might be a very good way to do it.

    10. WZWojciech Zaremba55:19

        I would like, uh, to move to that place that actually we have scientific understanding of what consciousness is.

    11. LFLex Fridman55:24

        Yeah.

    12. WZWojciech Zaremba55:25

        And then we'll be able to actually assign value. And I believe that there is even the path for the experimentation in, uh, in it. So, uh, you know, we, we said that, you know, you could put the probes inside of the brain. There is actually few other things that you could do with devices like Neuralink. So, you could imagine that the way even to measure if AI system is conscious is by literally just plugging into the brain.

    13. LFLex Fridman55:52

        Mm-hmm.

    14. WZWojciech Zaremba55:53

        Um, I mean, that assumes that it's kind of easy.

    15. LFLex Fridman55:55

        (laughs)

    16. WZWojciech Zaremba55:55

        But they're plugging into the brain and asking person if they feel that their consciousness expanded. Um, this direction, of course, has some issues. You can say, you know, if someone takes a psychedelic drug, they might feel that their consciousness expanded, even though that drug itself is not conscious.

    17. LFLex Fridman56:11

        Right. So, like, you can't fully trust the self-report of a person saying their, their consciousness is, is expanded or not. (sighs)

12. 56:19 – 1:07:46

    Psychedelics and meditation

    1. LFLex Fridman56:19

       Uh, let me ask you a little bit about psychedelics 'cause, uh, there have been a lot of excellent research on, uh, different psychedelics, psilocybin, MDMA, uh, even DMT, drugs in general, marijuana too. Uh, what do you think psychedelics do to the human mind? It seems they take the human mind to some interesting places. Is that just a little, uh, hack, a visual hack? Or is there some profound expansion of the mind?

    2. WZWojciech Zaremba56:49

       So, let's see. I, I don't believe in magic. I believe in, uh, I believe in, uh, in science, in, in causality. Um, still, let's say, and then as I said, like, I think that the brain, that the, our subjective experience of reality is, uh, we, we live in the simulation run by our brain.

    3. LFLex Fridman57:12

       Mm-hmm.

    4. WZWojciech Zaremba57:12

       And the simulation that our brain runs, they can be very pleasant or very hellish. Drugs, they are changing some hyperparameters of the simulation. It is possible, thanks to change of these hyperparameters, to actually look back on your experience and even see that the given things that we took for granted, they are changeable. So, they allow to have a, a amazing perspective. There is also, for instance, the fact that after DMT, people can see the full movie inside of their head.

    5. LFLex Fridman57:46

       Mm-hmm.

    6. WZWojciech Zaremba57:47

       Gives me further belief that the brain can generate that full movie, that the brain is actually learning the model of reality to such extent that it tries to predict what gonna happen next.

    7. LFLex Fridman58:00

       Yeah, very high resolution, so it can replay realities, I'm sure.

    8. WZWojciech Zaremba58:03

       E- extremely high resolution. Yeah, it's also kind of interesting to me that somehow there seems to be some similarity between these, uh, drugs and meditation itself. And I actually started even these days to think about meditation as a psychedelic.

    9. LFLex Fridman58:21

       Mm.

    10. WZWojciech Zaremba58:22

        Um-

    11. LFLex Fridman58:22

        Do you practice meditation?

    12. WZWojciech Zaremba58:24

        I, I, I practice meditation. I mean, I went few times on the re- retreats. And it feels after, like, after second or third day of meditation, uh, th- th- there is a, there is almost like a sense of, you know, tripping.

    13. LFLex Fridman58:41

        What, what does a meditation retreat entail?

    14. WZWojciech Zaremba58:44

        So, when you w- you wake up, uh, early in the morning and you meditate for extended period of time, uh, and th-

    15. LFLex Fridman58:52

        Alone? Just, just so I do- did

    16. WZWojciech Zaremba58:54

        Uh, as- yeah.

    17. LFLex Fridman58:54

        ... keep in mind.

    18. WZWojciech Zaremba58:54

        So, it's optimized. Even though there are other people, it's, uh, optimized for isolation. So, you don't speak with anyone. You don't actually look into other people eyes.

    19. LFLex Fridman59:03

        Mm-hmm.

    20. WZWojciech Zaremba59:04

        And, uh, you know, you sit on that chair. Um, say, the Vipassana meditation tells you, uh, to focus on the breath. So, you try to put, uh, all the all- attention into breathing and, uh, breathing in and breathing out. And, um, crazy thing is that as you focus attention like that, uh, after some time, there's some s- starts coming back, like some memories that you've completely forgotten.

    21. LFLex Fridman59:34

        Mm-hmm.

    22. WZWojciech Zaremba59:34

        It almost feels like, um, that you have a mailbox and then you, you know, you are just like, uh, archiving email one by one.

    23. LFLex Fridman59:42

        Mm-hmm.

    24. WZWojciech Zaremba59:43

        And at some point, at some point, there is this, like, a amazing feeling of getting to mailbox zero.

    25. LFLex Fridman59:50

        (laughs)

    26. WZWojciech Zaremba59:50

        Zero emails. And, uh, it's very pleasant. It's, it's kind of... it's, it's, it's crazy to me that, um, that once you resolve these, uh, inner sto- stories or, like, uh, inner traumas-

    27. LFLex Fridman1:00:07

        Mm-hmm.

    28. WZWojciech Zaremba1:00:08

        ... then once there is nothing, uh, left, the default, uh, state of human mind is extremely peaceful and happy. Extreme... like, uh, some sense, it, it feels that the... it feels, at least to me, the way how when I was a child that I can look at any object and it's very beautiful. I have a lot of curiosity about the simple things.

    29. LFLex Fridman1:00:34

        Mm-hmm.

    30. WZWojciech Zaremba1:00:34

        And that's where usually meditation takes me.
13. 1:07:46 – 1:15:03

    Ilya Sutskever

    1. LFLex Fridman1:07:46

       You mentioned Ilya, Ilya Sutskever.

    2. WZWojciech Zaremba1:07:49

       He's very, he's our chief scientist, but also he's a very close friend of mine.

    3. LFLex Fridman1:07:53

       He co-founded OpenAI with you. I've, uh, spoken with him a few times. He's brilliant, I really enjoy talking to him. Uh, (laughs) his mind, just like yours, works in fascinating ways. Uh, both of you are not able to define deep learning simply. (laughs)

    4. WZWojciech Zaremba1:08:10

       (laughs)

    5. LFLex Fridman1:08:12

       Uh, what's it like having him as somebody you have technical discussions with on, in the space of machine learning, deep learning, AI, but also life? What's it like when these two, um, agents s- get into a self-play situation in, in a room? What's it like collaborating with him?

    6. WZWojciech Zaremba1:08:33

       So, I believe that we have, uh, extreme, uh, respect to each other. So, um, in, I love Ilya's insight, both like, uh, m- I guess about consciousness, uh, life, AI.

    7. LFLex Fridman1:08:49

       But, uh, in terms of the ... It's interesting to me 'cause y- y- you're a, a brilliant, uh, thinker in the space of machine learning, like intuition, like digging deep in what works, what doesn't, why it works, why it doesn't, and so is Ilya. I w- I'm wondering if there's interesting deep discussions you've had with him in the past, or disagreements that were very productive.

    8. WZWojciech Zaremba1:09:15

       So, I can say I also understood over the time where are m- my strengths. So obviously, we have plenty of AI discussions.

    9. LFLex Fridman1:09:23

       Mm-hmm.

    10. WZWojciech Zaremba1:09:23

        And, uh, um, and, you know, I myself have plenty of ideas, but like I consider Ilya, uh, one of the most prolific, uh, AI scientists in the entire world.

    11. LFLex Fridman1:09:35

        Mm-hmm.

    12. WZWojciech Zaremba1:09:35

        And, uh, I think that, um, I realized that maybe my super skill, uh, is, uh, being able to bring people to collaborate together, that I have some level of, uh, empathy that is unique in AI world. And that might come, you know, from either meditation, psychedelics, or let's say I read just hundreds of books on this topic. So, and I also went through a journey of, you know, I develop all sorts of, uh, algorithms. So, I think that maybe I can ... That's my s- superhuman skill. Uh, Ilya is, uh, one of the best AI scientists, but then I'm pretty good in assembling teams.

    13. LFLex Fridman1:10:15

        Mm-hmm.

    14. WZWojciech Zaremba1:10:15

        And I'm also not holding to people. Like, I'm growing people and then people become managers at OpenAI.

    15. LFLex Fridman1:10:20

        Gotcha.

    16. WZWojciech Zaremba1:10:21

        I grew many of them, like research managers.

    17. LFLex Fridman1:10:23

        So you, you find, you find places where you're excellent and, and he finds, like, his, his, his deep scientific insights is where he is, and y- you find ways you can d- the puzzle pieces fit together.

    18. WZWojciech Zaremba1:10:36

        Correct. Like, uh, you know, ultimately, for instance, let's say, Ilya, he doesn't manage people. Uh, that's not h- what he likes or so. Um, I, I like, I like hanging out with people. By default, I'm an extrovert and I care about people.

    19. LFLex Fridman1:10:51

        Oh, interesting. Okay. All right, okay. Cool. So, that, that fits perfectly together, right? But I, I mean, uh, I also just like your intuition about various problems in machine learning. He, he's definitely one I really enjoy. I remember talking to him about something I was struggling with, which is coming up with a good model for pedestrians, for human beings that cross the street, in the context of autonomous vehicles (laughs) . And he immediately started to, like, uh, formulate a framework within which you can evolve a model for pedestrians, like through self-play, all that kind of mechanisms. The depth of thought on a particular problem, especially problems he doesn't know anything about, is- is fascinating to watch. It's, makes you realize like, um, yeah, the- the- the limits of the ... that the human intellect might be limitless. It was just impressive to see a descendant of ape come up with clever ideas.

    20. WZWojciech Zaremba1:11:52

        Yeah, I mean, so even in the space of deep learning when you look at various people, there are people, you know, who invented, uh, some breakthroughs once, but there are very few people who did it multiple times.

    21. LFLex Fridman1:12:06

        Mm-hmm.

    22. WZWojciech Zaremba1:12:06

        And you can think, if someone invented it once, that might be just the sheer luck, and if someone invented it multiple times, you know, if a probability of inventing it once is one over million-

    23. LFLex Fridman1:12:17

        Yeah.

    24. WZWojciech Zaremba1:12:17

        ... then probability of inventing it twice or three times would be one over million squared, or, or to the power of three, uh, which- which would be just impossible. So, it literally means that it's- it's given that, uh, it's not the luck.

    25. LFLex Fridman1:12:30

        Yeah.

    26. WZWojciech Zaremba1:12:31

        Uh, and Ilya is one of these few people who, uh, uh, who have, uh, a lot of these inventions in his arsenal. It also feels that, um, you know, for instance, if you think about folks like Gauss or Euler, uh, you know, at first they read a lot of books, and then they did thinking, and then they figure out math.

    27. LFLex Fridman1:12:53

        Mm-hmm.

    28. WZWojciech Zaremba1:12:53

        And that's how it feels with Ilya.

    29. LFLex Fridman1:12:56

        Yeah.

    30. WZWojciech Zaremba1:12:56

        That, you know, at first he read stuff and then, like, he spent his thinking cycles and ...
14. 1:15:03 – 1:24:56

    How does GPT work?

    1. WZWojciech Zaremba1:15:03

    2. LFLex Fridman1:15:03

       Before I go onto that, uh, thread, let me return to our friend GPT, and let me ask you another ridiculously big question. Can you give an overview of what GPT-3 is or, like you say in your Twitter bio, GPT n+1, how it works and why it works?

    3. WZWojciech Zaremba1:15:24

       So, um, GPT-3 is a humongous neural network. Um, let's assume that we know what is neural network, okay, by the definition. And, uh, it is trained on the entire internet just to predict next word. So let's say it sees part of the, uh, article and it... the only task that it has at hand, it is to say what would be the next word, uh, what would be the next word? And, uh, it becomes, uh, uh, really exceptional at the task of figuring out what's the next word. So you might ask, "Why would, uh, this be an important, uh, task? Why would it be important to predict what's the next word?" And it turns out that a lot of problems, uh, can be formulated, uh, as a text completion problem. So GPT is purely, uh, learning to complete the text. As you could imagine, for instance, if you are asking a question, uh, who is the president of United States, then GPT can give you an answer to it. Uh, it turns out that many more things can be formulated this way. You can format text in the way that you have sentence in English. You make it even look like, uh, some content of a website, uh, elsewhere which would be teaching people how to translate things between languages. So it would be EN: uh, text in English, FR:, and then you'll, uh, uh, and then you'll ask peop- uh, and then you'll ask model to, to continue. And it turns out that the... such a model is predicting translation from English to French. The crazy thing is that this model, uh, can be used for way more sophisticated tasks. So you can format text such that it looks like a conversation between two people, and that might be a conversation between you and Elon Musk and because the model read all the texts about Elon Musk, it will be able to predict Elon Musk words as it would be Elon Musk. It will speak about colonization of, uh, Mars, about s- sustainable future, and so on. And, um, it's also possible to, to even give arbitrary personality to the model. You can say, "Here is a conversation with a friendly AI bot."

    4. LFLex Fridman1:17:36

       Mm-hmm.

    5. WZWojciech Zaremba1:17:38

       And the model, uh, will complete the text as a friendly AI bot.

    6. LFLex Fridman1:17:42

       So, I mean, (sighs) how do I express how amazing this is? So just to clarify, uh, uh, a conversation, generating a conversation between me and Elon Musk, it wouldn't just generate good examples of what Elon would say, it would get the syntax all correct. So, like, interview style, it would say, like, Elon: and Lex:, like, it, it's not just like, uh, inklings of, um, semantic correctness. It's, like, the whole thing, grammatical, syntactic, semantic. It's just really, really impressive, uh, generalization.

    7. WZWojciech Zaremba1:18:30

       Yeah, I mean, I also want to, you know, provide some caveats though. It can generate few paragraphs of coherent text, but as you go to, uh, longer pieces, it, uh, it actually goes off the rails. If you would, uh, try to write a book, it won't work out, uh, this way. And-

    8. LFLex Fridman1:18:45

       What way does it go off the rails by the way? Is there interesting ways in which it goes off the rails? Like, (laughs) wha- wha- what falls apart first?

    9. WZWojciech Zaremba1:18:54

       So the model is trained on the... all the existing data, uh, that is out there, which means that it is not trained on its own mistakes. So for instance, if it would make a mistake, then, uh, I ke- so to give y- give you an example.

    10. LFLex Fridman1:19:08

        Sure.

    11. WZWojciech Zaremba1:19:08

        So let's say I have a conversation with a, a model pretending that is Elon Musk.

    12. LFLex Fridman1:19:14

        Mm-hmm.

    13. WZWojciech Zaremba1:19:14

        And then I start putting some, uh... I'm start actually making up things which are not factual.

    14. LFLex Fridman1:19:20

        Mm-hmm.

    15. WZWojciech Zaremba1:19:21

        Um, I would say-

    16. LFLex Fridman1:19:22

        Sounds like Twitter.

    17. WZWojciech Zaremba1:19:23

        (laughs)

    18. LFLex Fridman1:19:23

        (laughs) But I gotcha, sorry. Yeah.

    19. WZWojciech Zaremba1:19:27

        Um, like, uh, the... I don't know, I would say that Elon is my wife.... and the model will just keep on carrying, uh, it on and, uh-

    20. LFLex Fridman1:19:35

        As if it's true.

    21. WZWojciech Zaremba1:19:37

        Uh, yes. And in some sense, if you would have a normal conversation with Elon, he would be, "What the fuck?"

    22. LFLex Fridman1:19:43

        Yeah. There would be some feedback between... So the, the model is trained on things that humans have written but through the generation process, there's no human in the loop feedback.

    23. WZWojciech Zaremba1:19:54

        Correct.

    24. LFLex Fridman1:19:55

        That's fascinating.

    25. WZWojciech Zaremba1:19:56

        Ma- makes sense?

    26. LFLex Fridman1:19:57

        So it's magnified, it's like the errors get magnified and magnified.

    27. WZWojciech Zaremba1:20:00

        Correct.

    28. LFLex Fridman1:20:01

        And, and, and it's as, it's also interesting... I mean, first of all, humans have the same problem, it's just that we, uh, we make f- fewer errors and magnify the errors slower.

    29. WZWojciech Zaremba1:20:13

        Yeah, I think that the, actually what happens with humans is if you have a wrong belief about the world as a kid then very quickly you will learn that it's not correct because you are grounded in reality and you are learning from your new experience.

    30. LFLex Fridman1:20:26

        Yes. But do you think the model can correct itself too? Won't it, through the power of the representation and so the absence of (laughs) Elon Musk being your wife, information on the internet, won't it correct itself?
15. 1:24:56 – 1:31:42

    AI safety

    1. WZWojciech Zaremba1:24:56

    2. LFLex Fridman1:24:56

       I asked this question of Ilya, let me ask you of, uh, you this question. I've been reading a lot about Stalin and power. (laughs) If you're in possession of a system that's like AGI, that's exceptionally powerful, do you think your character and integrity might become corrupted? Like famously power corrupts and absolute power corrupts absolutely.

    3. WZWojciech Zaremba1:25:25

       So I believe that the, h- he want at some point to work toward distributing the power.... I think that, uh, you want to be in the situation that actually AGI is not controlled by a small number of people, uh, but, uh, essentially, uh, by a larger collective.

    4. LFLex Fridman1:25:45

       So, the thing is, that requires a George Washington style move. In the ascent to power there's always a- a moment when somebody gets a lot of power and they have to have the integrity and, uh, the moral compass to give away that power. That humans have been good and bad throughout history at this particular step, and I wonder... I wonder, we, like, blind ourselves in a, for example, between nations, a race, uh, towards, um, yeah, AI race between nations. We might blind ourselves and justify to ourselves the development of AI without distributing the power because we want to defend ourselves against China, against Russia, that kind of, that kind of logic. And, um, I wonder how we, um, how we design governance mechanisms that, uh, prevent us from becoming power hungry and in the process destroying ourselves.

    5. WZWojciech Zaremba1:26:48

       So, let's see. I have been thinking about this topic quite a bit, but I also wanna admit that, uh, once again I actually want to rely way more on Sam Altman on it.

    6. LFLex Fridman1:26:57

       Yes.

    7. WZWojciech Zaremba1:26:58

       He wrote an, uh, he wrote an, uh, excellent blog on how even to distribute wealth.

    8. LFLex Fridman1:27:03

       Mm-hmm.

    9. WZWojciech Zaremba1:27:04

       Um, and his propo- he proposed in his blog, uh, to tax, uh, equity of the companies rather than profit and to distribute it.

    10. LFLex Fridman1:27:13

        Interesting.

    11. WZWojciech Zaremba1:27:13

        And this is, this is an example of a Washington move.

    12. LFLex Fridman1:27:18

        Mm-hmm.

    13. WZWojciech Zaremba1:27:20

        I guess I personally have insane trust in Sam. He already spent plenty of money running, uh, universal basic income, uh, project.

    14. LFLex Fridman1:27:31

        Yeah.

    15. WZWojciech Zaremba1:27:31

        That like, uh, gives me, I guess, maybe some level of trust to him, but I also, I guess love him as a friend.

    16. LFLex Fridman1:27:41

        Yeah. I wonder, because we're sort of summoning a new set of technologies, I wonder if we'll be, um, cognizant... Like you're describing the processes at OpenAI, but it could also be at other places like in the US government, right? Uh, both China and the US are now full steam ahead on autonomous weapons systems development and that's really worrying to me because in the framework of something being a national security danger or a military danger you can do a lot of pretty dark things that, uh, blind our moral compass, and I think AI will be one of those things. Um, in some sense, the- the mission and the work you're doing at OpenAI is like the counterbalance to that, so you want to have more OpenAI and less autonomous weapons systems.

    17. WZWojciech Zaremba1:28:38

        Uh, uh, uh, I- I- I like these statements. Like, uh, to be clear, like, uh, this interesting, and I'm thinking about it myself, but, uh, this a place that I- I, like, uh, I put my trust actually in Sam's hands because it's extremely hard for me to reason about it.

    18. LFLex Fridman1:28:53

        Yeah. I mean, one important statement to make is, um, it's good to think about this.

    19. WZWojciech Zaremba1:28:59

        Uh, yeah, no question about it.

    20. LFLex Fridman1:29:00

        Right.

    21. WZWojciech Zaremba1:29:00

        No question about it.

    22. LFLex Fridman1:29:01

        So, even, like, low-level quote unquote "engineer," like there's such a... I remember I- I programmed a car, a- a- a RC car (laughs) that went really fast, like 30, 40 miles an hour, and I remember I was like sleep-deprived so I programmed it pretty crappily and it like, uh, the- the code froze. So, it's- it's doing some basic computer vision and it's going around on track, but it's going full speed (laughs) and, uh, there was a bug in the code that, uh, the car just went s- s- it didn't turn, it went straight full speed and smashed into the wall. And I remember thinking, the seriousness with which you need to approach the design of artificial intelligence systems and the programming of artificial intelligence systems is high 'cause the consequences are high. Like that- that little car smashing into the wall, for some reason I immediately thought of, like, an algorithm that controls nuclear weapons having the same kind of bug. And so like, the lowest level engineer and the CEO of a company all need to have the seriousness, uh, in approaching this problem and thinking about the worst case consequences.

    23. WZWojciech Zaremba1:30:17

        So, I think that is true. I mean, the... What I also recognize in myself and others even asking this question is that it evokes a lot of fear.

    24. LFLex Fridman1:30:27

        Mm-hmm.

    25. WZWojciech Zaremba1:30:28

        And, uh, fear itself ends up being actually quite dehabilitating. The place where I arrived at the moment might sound cheesy or so, but it's almost to build things out of love rather than fear.

    26. LFLex Fridman1:30:44

        Yeah.

    27. WZWojciech Zaremba1:30:45

        Uh, like a focus on how, uh, I can, you know, maximize the value, how the systems that I'm building might be, uh, useful.

    28. LFLex Fridman1:30:56

        Mm-hmm.

    29. WZWojciech Zaremba1:30:56

        Uh, I'm not saying that the fear doesn't exist out there, and like, uh, it totally makes sense to minimize it, but I don't want to be working because, uh, I'm scared.

    30. LFLex Fridman1:31:06

        Yeah.
16. 1:31:42 – 1:45:15

    OpenAI Codex

    1. LFLex Fridman1:31:42

       Can we just take a step back to something else super cool which is, uh, OpenAI Codex?

    2. WZWojciech Zaremba1:31:47

       Mm-hmm.

    3. LFLex Fridman1:31:48

       Can you give an overview of what OpenAI Codex and GitHub Co-pilot is, how it works, and why the hell it works so well?

Episode duration: 2:51:32