Joe Rogan Experience #1350 - Nick Bostrom

1. 0:00 – 1:50

   AI as humanity’s biggest hope—and biggest fear

   1. JRJoe Rogan0:00

      And here we go. All right, Nick. This is, uh, one of the things that scares people more than anything, is the idea that we're creating something, or someone's gonna create something, that's gonna be smarter than us, that's gonna replace us. Is that something we should really be concerned about?

   2. NBNick Bostrom0:21

      I presume you're referring to babies.

   3. JRJoe Rogan0:23

      (laughs)

   4. NANarrator0:24

      (laughs)

   5. JRJoe Rogan0:24

      I'm referring to artificial intelligence.

   6. NBNick Bostrom0:26

      Ah, yes.

   7. JRJoe Rogan0:27

      Ugh.

   8. NBNick Bostrom0:29

      Well, it's the, the big fear and the big hope, I think.

   9. JRJoe Rogan0:32

      Both?

   10. NBNick Bostrom0:33

       At the same time, yeah.

   11. JRJoe Rogan0:34

       How is it the big hope?

   12. NBNick Bostrom0:36

       Well, there are a lot of things wrong with the world as it is now.

   13. JRJoe Rogan0:39

       I'm trying to pull this up to your face, if you would.

   14. NBNick Bostrom0:41

       Um, all, all the problems we have, uh, most of them could be solved if we were smarter or if we had somebody on our side who are a lot smarter with better technology and so forth. Um, also, I think if we wanna imagine some really grand future where humanity or our descendants one day go out and colonize the universe, I think that's likely to happen, if it's gonna happen at all, after we have superintelligence that then develops the technology to make that possible.

   15. JRJoe Rogan1:19

       The real question is whether or not we would be able to harness this intelligence, or whether it would dominate.

   16. NBNick Bostrom1:24

       Yeah, that certainly is one question. Um, not the only. You could imagine that we harness it, but then use it for bad purposes as we have a lot of other technologies through history.

   17. JRJoe Rogan1:36

       Yeah.

   18. NBNick Bostrom1:37

       So I think there are really two challenges we need to meet. One, one is to make sure we can align it with human values, and then make sure that we, together, do something better with it than fighting wars or oppressing one another.

2. 1:50 – 3:12

   Obsolescence, evolution, and why “different” isn’t automatically better

   1. JRJoe Rogan1:50

      I think... Well, what I'm worried about more than anything is that human beings are gonna become obsolete, that we're going to invent something that's the next stage of evolution. I'm, I'm really concerned with that. I'm really concerned with if we look back on ancient hominids, uh, Australopithecus, just think of some primitive ancestor of man, we don't wanna go back to that. Like, that, that's a terrible way to live. I'm worried that what we're creating is the next thing.

   2. NBNick Bostrom2:18

      I think we don't necessarily want, or at least I wouldn't be totally thrilled with, with a future where humanity as it is now was, was the last and final word, the pa- like, ultimate version beyond.

   3. JRJoe Rogan2:35

      Right.

   4. NBNick Bostrom2:35

      I, I think there's a lot of room for improvement.

   5. JRJoe Rogan2:37

      Sure.

   6. NBNick Bostrom2:37

      But not anything that is different is an improvement.

   7. JRJoe Rogan2:40

      Right.

   8. NBNick Bostrom2:40

      So, so the key would be, I think, to find some path forward where the best in us, uh, can continue to exist and develop, uh, to even greater levels. And maybe at the end of that path, it looks nothing like we do now. Maybe it's not two-legged, two-armed creatures running around with three pounds of thinking matter, right? It might be something quite different. But as long as it... what, what we value is, is present there, and ideally in a much higher degree than in the current world, then that could count as a success.

3. 3:12 – 7:01

   How fast is progress really? Innovation pace, uncertainty, and ‘the gate’ metaphor

   1. JRJoe Rogan3:12

      Yeah, the idea that we're in a state of evolution, that we are... just like we look at ancient hominids, that we are eventually going to become something more advanced or at least more complicated than we are now. But what I'm worried is that biological life itself has so many limitations. When we look at the evolution of technology, if you look at Moore's law or if you just look at new cellphones, like, they just released a new iPhone yesterday and they talked about all these incremental increases in the ability to take photographs and wide-angle lenses and night mode and a new chip that works even faster, these things, uh... there's not... the, the word evolution's incorrect, but the innovation of technology is so much more rapid than anything we could ever even imagine, biologically. Like, if we had a thing that we'd create, if we created, um... instead of artificial intelligence in terms of, like, some- something in a chip or a computer, if we created a life form, a biological life form, but this biological life form was improving radically every year, like, it didn't even exist, like a... the iPhone existed in 2007, that's when it was invented. If we had something that was 12 years old, but all of a sudden was infinitely faster and better and smarter and wiser than it was 12 years ago and the newest version of it, version X1, we would, we would start going, "Whoa, whoa, whoa! P- hit the brakes on this thing, man." How, how many more generations before this thing's way smarter than us? How many more generations before this thing thinks that human beings are obsolete?

   2. NBNick Bostrom4:41

      Yeah, it's coming, coming at us fast it feels like.

   3. JRJoe Rogan4:43

      Yes.

   4. NBNick Bostrom4:43

      I mean, uh... and, but some, some people think, "Oh, it's, uh, slowing down now." Um...

   5. JRJoe Rogan4:48

      Who thinks it's slowing down?

   6. NBNick Bostrom4:50

      Well, uh, don't they have a, like... Tyler Cowen and e- even, uh, um, Peter Thiel sometimes goes on about the, the, the pace of innovation not really being what it needs to be.

   7. JRJoe Rogan5:01

      Mm.

   8. NBNick Bostrom5:02

      Um, I mean, maybe it was faster in, like, 1890s or... but, but still, compared to almost all of human history, it seems like a period of unprecedented rapid progress right now.

   9. JRJoe Rogan5:17

      Unprecedented?

   10. NBNick Bostrom5:18

       I would say so. Yeah, I mean, except for maybe a couple of decades, uh, 100 years ago when there was a lot of, you know, electricity, the whole thing.

   11. JRJoe Rogan5:25

       Yeah. No, I agree. Um, I just... I'm... I don't think it's a concern because, um, it's more of a curiosity to me. I mean, I am concerned, but the more I look at it and go, "Well, this is see-..." It seems inevitable that we're, we're going to run into artificial intelligence. But the questions are so open-ended. We really don't know when, we don't, really don't know what form it's gonna take, and we really don't know what it's going to do to us.

   12. NBNick Bostrom5:54

       Yeah. So I, I see it as not something that, um, should be avoided, uh...... neither something that we should just be completely gung-ho about, but more like a kind of gate through which we will have to pass at some point. All paths that are both plausible and lead to really great futures, I think at some point involve the development of greater than human intelligence, machine intelligence. And so, that our focus should be on getting our act together as much as we can in whatever period of time we have before that-

   13. JRJoe Rogan6:24

       (laughs)

   14. NBNick Bostrom6:25

       ... occurs.

   15. JRJoe Rogan6:25

       Prepare ourselves.

   16. NBNick Bostrom6:27

       Well, I mean, that might involve doing some, some research into various technical questions as how you build these systems so that we actually understand what they are doing, and they have some, you know, intended impact on the world. It might also, if, if we are able to get our act together a little bit at, at, on, on the kind of global political scene, um, a little bit more peace and love in the world would be good, I think.

   17. JRJoe Rogan6:48

       Sure, that'd be nice.

   18. NBNick Bostrom6:50

       Um, so, uh, and, and then, like, refraining from destroying ourselves through some other mean before we even get a chance to try to, uh, needle our way through this, uh, this gate.

4. 7:01 – 8:26

   Where AI is today: deep learning’s leap in perception

   1. JRJoe Rogan7:01

      Well, that's certainly possible. We're, we're certainly capable of screwing it all up. Where is the current state of technology now in regards to artificial intelligence, and how far away do you think we are from AGI?

   2. NBNick Bostrom7:15

      Well, dif- different people have different views on that. I think the, uh, the truth of the matter is that it's very hard to, to have accurate views about the timelines for these things, that, that's, you know, are still involve kind of big new breakthroughs that have to happen. Um, uh, certainly, I mean, over the last eight or 10 years, there has been a lot of excitement with the deep learning revolution. Um, things that ... I mean, it used to be that people thought of AI as this kind of autistic savant, really good at logic and counting and memorizing facts, but with, with no, no intuition. And there's this deep learning evolution when you begin to do these deep neural networks, you kind of solved perception in some sense.

   3. JRJoe Rogan8:01

      Mm-hmm.

   4. NBNick Bostrom8:02

      Um, you, you now have computers that can see, that, that can hear, and that have visual intuition. Um, so, so, so that, that has enabled the whole wide suite of applications, uh, which makes it commercially valuable, which then drives a lot of investment in it, which, you know, there's a ... so there's now quite a lot of momentum, um, uh, in, in machine learning and, and trying to kind of stay ahead of that.

5. 8:26 – 10:39

   Agency, goals, and why motivation is the real danger

   1. JRJoe Rogan8:26

      It's interesting that when we think about artificial intelligence and it, whatever potential form that it's gonna take, if you look at films like, uh, 2001, like HAL, like, "Open the door, HAL," you know? Like-

   2. NBNick Bostrom8:38

      Uh-huh.

   3. JRJoe Rogan8:39

      ... we think of something that's communicating to us like, like a person would, and maybe is a little bit colder and doesn't, doesn't share our values and has a more pragmatic view of life and death and, and things. When we think of intelligence, though, I think intelligence in our mind is almost inexorably connected to all the things that make us human, like emotions and, and, and ambition, and all these things, like the reason why we innovate. Like we ... it's not really clear, like why ... we innovate because we enjoy innovation and because we want to make the world a better place and because we want to fix some problems that we've created and we want to solve some limitations of the human body and the environment that we live in, but we sort of assume that intelligence that we create will also have some motivations.

   4. NBNick Bostrom9:28

      Well, there is a fairly large class of possible structures you could do. If you want to do anything that has any kind of cognitive or intellectual capacity at all, a large class of those would be what we might call agent.

   5. JRJoe Rogan9:41

      Agents?

   6. NBNick Bostrom9:42

      So this would be, yeah, systems that interact with the world, um, in pursuit of some goal. Um, and if they are a sophisticated class of agents, they, they can plan ahead the sequence of actions. Like, more primitive agents might just have reflexes. Um, but, but the sophisticated agent might have a model of the world where it can kind of think ahead before it starts doing stuff. It can-

   7. JRJoe Rogan10:06

      Hm.

   8. NBNick Bostrom10:06

      ... kind of think, "What, what would I need to do in order to reach this desired state?" And then reason backwards from that. So I think it's a fairly natural ... it's not the only possible cognitive system you could build, but it's also not this weird, bizarre special case that, you know ... it, it's a fairly natural thing to, to aim for. If you're able to specify the goal, something you want to achieve, but you don't know how to achieve it, the natural way of trying to go about that is by building the system that has this goal and is an agent and then moves around and tries different things and eventually perhaps learn to, to solve that task.

6. 10:39 – 13:12

   Human-like AI vs alien AI: brain emulation and synthetic routes

   1. JRJoe Rogan10:39

      Do you anticipate different types of artificial intelligence, like artificial intelligence that mimics the, the human emotions? Like, do you th- do you think that people will construct something that's very similar to us in a way that we can interact with it in, in common terms, or do you think it will be almost like communicating with, uh, an alien?

   2. NBNick Bostrom11:04

      So there are different scenarios here.

   3. JRJoe Rogan11:06

      Yeah.

   4. NBNick Bostrom11:06

      I mean, I, I'd guess, my guess is that the first thing that actually achieves superintelligence would not be very human-like. Um, there are different possible ways you could try to get to this level of technology. One, one would be by trying to reverse engineer the human brain.

   5. JRJoe Rogan11:23

      Hm.

   6. NBNick Bostrom11:24

      Like, we have in existence. In, in, in the limit- limiting case, imagine if you just made an exact duplicate in, in silicon of the human brain, like every neuron had some counterpart. Um, so that, that, that seems technologically very difficult to do, but it wouldn't require a big theoretical breakthrough to do it. You could just through, if you had sufficiently good microscopy and large enough computers and enough elbow grease, you could-

   7. JRJoe Rogan11:48

      Hm.

   8. NBNick Bostrom11:48

      ... kind of ... but it seems to me plausible that what will work before we are able to do it that way will be some more synthetic approach work, um, that would only be a very rough resemblance-... uh, maybe with the, the neocortex.

   9. JRJoe Rogan12:01

      Yeah, that's one of the big questions, right? Whether or not we can replicate all the functions of the human brain in the way it functions and, and like mimic it exactly, or whether we could have some sort of superior method that achieves the same results that the human brain does in terms of its ability to calculate and reason and, and do multiple tasks at the same time.

   10. NBNick Bostrom12:21

       Yeah. And I also think that maybe once you have a sufficiently high level of this general form of intelligence, then you could use that maybe to emulate or mimic things that we do differently. So may- maybe we... O- Our cortex is quite limited, so we rely a lot on earlier neurological structures that we have. We have to, we have to be guided by emotion because we can't just calculate everything out.

   11. JRJoe Rogan12:46

       Mm-hmm.

   12. NBNick Bostrom12:47

       Um, and, and instinct and, and m- may... and may... And if we lost all of that, we would be helpless. But maybe some system that had a sufficiently high level of this more abstract reasoning capability could maybe use that to substitute for things that weren't built in, in the same way that we do.

   13. JRJoe Rogan13:03

       Have you ever talked to Sam Harris about this?

   14. NBNick Bostrom13:05

       Yeah, a little bit.

   15. JRJoe Rogan13:07

       Have you ever had a podcast with him?

   16. NBNick Bostrom13:09

       I... Yeah. A- Actually, he had him on his podcast, uh, half a year ago or something like that.

7. 13:12 – 15:00

   Why push forward anyway? Incentives, arms races, and overdetermined progress

   1. JRJoe Rogan13:12

      I'll have to, I'll have to listen to it because he has the worst view of the, the, the future in terms of artificial intelligence. He's terrified of it. And when I talk to him, he terrifies me. And, uh, Elon Musk is right up there.

   2. NBNick Bostrom13:26

      Uh-huh.

   3. JRJoe Rogan13:26

      He also has a terrifying view-

   4. NBNick Bostrom13:28

      Uh-huh.

   5. JRJoe Rogan13:28

      ... of what our artificial intelligence could potentially be. What do you say to those guys?

   6. NBNick Bostrom13:34

      Well, I mean, I, I, I do think that there are these significant risks that will be associated with this transition to the machine intelligence era, um, incl- including existential risks, threats to the very survival of humanity or what we care about.

   7. JRJoe Rogan13:50

      So why are we doing this? (laughs)

   8. NBNick Bostrom13:52

      Ah, well, um, there are a lot of things we are, we are doing that maybe globally it would be better if we didn't do. I mean, just-

   9. JRJoe Rogan13:58

      Sure.

   10. NBNick Bostrom13:58

       ... why, why do we build thousands of nuclear weapons, right?

   11. JRJoe Rogan14:01

       Right.

   12. NBNick Bostrom14:01

       Uh, uh-

   13. JRJoe Rogan14:01

       Why do we overfish the oceans?

   14. NBNick Bostrom14:02

       Yeah. So, um... Now, I mean, if one actually ask why do different individuals work on AI research or why do different companies and governments fund it, I mean, there are a lot of explanation. It's like a great scientific, uh, endeavor. If, if you can make the Google search engine 1% better, that's gotta be worth like a billion dollars right off the bat. Um, it's become a kind of prestige thing now where, where nations want to have some sort of strategy 'cause it's seen as this new frontier. Um, just like when you had, um, you know, steam engines and industrialization a few hundred years ago and electricity. Like, it's gonna just open up a lot of economic opportunities. You want to be in there where it's hap- you, you wanna be this kind of, "We- we are gonna do subsistence agriculture while the w- rest of the world is moving on."

   15. JRJoe Rogan14:49

       Right.

   16. NBNick Bostrom14:50

       Um, so, so there's, there's a, uh, a lot of... It's kind of overdetermined. Like, you could, you could remove some of these reasons, and there would still be enough reasons for why people would be pushing forward with this.

8. 15:00 – 27:25

   Intelligence explosion: hardware limits, speedups, and scaling uncertainty

   1. JRJoe Rogan15:00

      One of the things that scares me the most is the idea that if we do create artificial intelligence, then it will improve upon our design and create far more sophisticated versions of itself, and that it'll continue to do that until it's unrecognizable, until it reaches literally a godlike potential.

   2. NBNick Bostrom15:17

      Mm-hmm.

   3. JRJoe Rogan15:17

      That su- I mean, I forget what the real numbers were, maybe you could tell us, but someone had calculated, some reputable source had calculated the amount of improvement that sentient artificial intelligence would be able to create inside of a small window of time. Like, if it was allowed to innovate and then make better versions of itself, and those better versions of itself were allowed to innovate and make better versions of itself, you're talking about not an exponential increase of intelligence, but an explosion.

   4. NBNick Bostrom15:45

      Well, well, we don't know. So it, it's hard not to forecast the pace at which we will make advances in AI, b- because we just don't know how hard the problems are that we haven't yet solved.

   5. JRJoe Rogan15:58

      Right.

   6. NBNick Bostrom15:58

      And, you know, once you get to human level or a little bit above, I mean, who, who knows? It could be that there is some level where to get further you would need, like, to put in a lot of thinking time to kind of get there. Now, what is easier to, to estimate is if, if you just look at the speed, 'cause that's just a function of the hardware that you're running it on, right? So, so there we know that there is a lot of room in principle. If, if you look at the physics of computation and you look at what would an optimally arranged physical system be that was optimized for computation, that would be like way many, many orders above what, what we can do now. Um, and that then you could have arbitrarily large systems like that. So, um, from, from that point of view, we, we know that that could be things that would be like a million times faster than the human brain and, and with a lot more memory and stuff like that.

   7. JRJoe Rogan16:48

      That... And then something... If it did have a million times more power than the human brain, it could create something with a million times more comput- (clears throat) computational power than itself.

   8. NBNick Bostrom17:00

      Well-

   9. JRJoe Rogan17:00

      It could make better versions. It could continue to innovate. Like, if we-

   10. NBNick Bostrom17:04

       Let me-

   11. JRJoe Rogan17:04

       ... create something that we, and we say, "You are..." I mean, "It is sentient. It is artificial intelligence. Now, please go innovate. Please go follow the same directive and improve upon your design."

   12. NBNick Bostrom17:19

       Yeah. Well, but we don't know how, how long that would take then-

   13. JRJoe Rogan17:23

       Right.

   14. NBNick Bostrom17:23

       ... to get to... So, I mean, we already have sort of millions of times more thinking capacity than a human has. I mean, we have millions of humans.

   15. JRJoe Rogan17:31

       Right.

   16. NBNick Bostrom17:32

       Um, so if you, if you kind of break it down, you think there's like one milestone when you have maybe an AI that could do what one human can do, but then that might still be quite a lot of orders of magnitude, uh, you know, until it would be equivalent of the whole human species. Um, and maybe during that time other things happen. Maybe we upgrade, you know, our, our own abilities in some way. So there, there are some scenarios where it's so hard to get even to one human baseline that, that we kind of use this massive amount of resources just to barely create kind of, you know, a village idiot.

   17. JRJoe Rogan18:07

       Yes.

   18. NBNick Bostrom18:07

       Uh, using billions of dollars of compute, right?

   19. JRJoe Rogan18:10

       (laughs)

   20. NBNick Bostrom18:10

       So if, if that's the way we get there, then, I mean, it might take quite a while.... because you can't easily scale something that you've already spent billions of dollars building.

   21. JRJoe Rogan18:18

       Yeah, some people think the whole thing is blown out of proportion, that we're so far away from creating artificial general intelligence that resembles human beings, that it's all just vaporware.

   22. NBNick Bostrom18:28

       Mm-hmm.

   23. JRJoe Rogan18:28

       What do you say to those people?

   24. NBNick Bostrom18:30

       Uh, well, I mean, uh, uh, well, one, one would be that I would wanna be more precise about just how far away does it have to be in order for us, uh, to be rational to ignore it.

   25. JRJoe Rogan18:41

       Right.

   26. NBNick Bostrom18:41

       And it, it might be that if something is sufficiently important and high stakes, then even if it's not gonna happen in the next five, 10, 20, 30 years, it might still be wise for, you know, our pool of seven billion plus people to have some people actually thinking about this ahead of time. Um-

   27. JRJoe Rogan18:59

       Yeah, for sure.

   28. NBNick Bostrom18:59

       So, so, so some of these disagreements, I guess this is my point, are, are more apparent than real. Like, there's some people say it's gonna happen soon, and some other people say, "No, it's not gonna happen for a long time." And then, (laughs) you know, one, one person means by soon five years, and another person means by a long time five years. And, uh, you know, it's more of different attitudes rather than different specific beliefs. So, so I would first want to make sure that there actually is a disagreement.

   29. JRJoe Rogan19:23

       Mm-hmm.

   30. NBNick Bostrom19:24

       Um, now if there is, if somebody is very confident that it's not gonna happen in hundreds and hundreds of years, then, then I, I guess I would wanna know their reasons for, for that level of confidence. What, what's, what's the evidence they're looking at? Uh, you know, do they have some ground for, for being very sure about this? Th- certainly, the history of technology prediction is not that great. Um, you, you can find a lot of other examples where even very eminent technologist and scientists were, were quite sure it's not, not gonna happen in our lifetime. And then, like-
9. 27:25 – 31:47

   Human enhancement as a parallel track: Neuralink skepticism and embryo selection

   1. JRJoe Rogan27:25

      Yeah. One of the technologies, or one of the things that's been discussed to sort of mitigate the dangers of artificial intelligence is a potential merge. Um, some sort of symbiotic relationship with technology that you, you see, you hear discussed. Like, um, I don't know exactly how Elon's Neuralink works, but it seems like a step in that direction. There's some sort of a brain implant that in- that interacts with an external device and this, all of this increases the bandwidth for available intelligence and knowledge.

   2. NBNick Bostrom28:01

      Yeah. I'm, I'm sort of skeptical that that will work. I mean, good that somebody tries it, you know. Uh, but, um, I think it's quite technically hard to improve a normal healthy human being's, uh, say cognitive capacity or other capacities by implanting things in them, um, and get benefits that you couldn't equally well get by having the gadget outside of the body.

   3. JRJoe Rogan28:26

      Mm-hmm.

   4. NBNick Bostrom28:27

      So, um, I, I don't need to have an implant to be able to use Google, right?

   5. JRJoe Rogan28:32

      Right.

   6. NBNick Bostrom28:33

      I, uh... And there are a lot of advantages to, to having it external. You, you can kind of upgrade it very easily.

   7. JRJoe Rogan28:38

      You can shut it off.

   8. NBNick Bostrom28:39

      You can sh- well, hopefully you could do that even with implant.

   9. JRJoe Rogan28:42

      (laughs) .

   10. NBNick Bostrom28:43

       Um, and once you start to look into the details, there's sort of these kind of demos, but then if you, if you actually look at the papers often you find, well, and then there were these side effects and the person had headaches, or they had some deficit and the speech didn't... like-

   11. JRJoe Rogan28:56

       Yes.

   12. NBNick Bostrom28:56

       ... infections. Like it is just... biology is messy.

   13. JRJoe Rogan28:58

       Yes.

   14. NBNick Bostrom28:59

       Um, so, um, maybe it will work better than, than I expect. That, that, that could be good. But oth- otherwise, I think, um, that the, the place where it will first become possible to enhance, you know, hu- human biological capacities, uh, uh, would, would be through, um, genetic selection, um, which is technologically, uh, something very near.

   15. JRJoe Rogan29:27

       Do you mean like CRISPR type?

   16. NBNick Bostrom29:28

       Well, so that would be editing, right? When you're-

   17. JRJoe Rogan29:30

       Mm-hmm.

   18. NBNick Bostrom29:30

       ... actually going and change things. That, that also is moving quickly.

   19. JRJoe Rogan29:33

       What do you mean by selection?

   20. NBNick Bostrom29:34

       Well, so this would just be in the context of say, in vitro fertilization. You have, um, usually some half dozen or a dozen embryos created during this fertility procedure, which is standardly used. So rather than just the doctor kind of looking at these embryos and saying, "Well, that one looks healthy. I'm gonna implant that," uh, you, you could run some genetic test and then use that as a predictor and select the one you think has the, the most, the most desirable attributes.

   21. JRJoe Rogan30:01

       And so this could be a trend in terms of how human beings reproduce, that we instead of just randomly having sex, woman gets pregnant, gives birth to a child, we don't know what it's gonna be, what's, what's gonna happen, we just hope that it's a good kid. Instead of that, you start looking at the... all the various components that we can measure-

   22. NBNick Bostrom30:24

       Yeah. Uh, and so, I mean, in, to some extent, we already do this. There are a lot of, um, testing done, um, um, for various chromosomal ab- abnormalities that you can already check for. But, but our ability to, uh, to look beyond clear, stark diseases that is one gene is wrong-

   23. JRJoe Rogan30:44

       Right.

   24. NBNick Bostrom30:44

       ... like the, like the, to look at more complex trait is, is, is, is increasing rapidly. Um, so obviously, there are a lot of ethical issues and-

   25. JRJoe Rogan30:51

       Yeah.

   26. NBNick Bostrom30:51

       ... different things that come into that.

   27. JRJoe Rogan30:53

       That's what I was gonna get to.

   28. NBNick Bostrom30:53

       But if I, if we're just talking what is technologically feasible-

   29. JRJoe Rogan30:56

       Mm-hmm.

   30. NBNick Bostrom30:56

       ... I, I think that, that ... I mean, already you could do a very limited amount of that today, and maybe you'd get, you know, two or three IQ points in expectation more if you selected using current technology based on 10 embryos, let us say, so very small. But, but as genomics, uh, gets better at deciphering the genetic architecture of complex traits, like whether it's intelligence or, or personality attributes, then, then you, you would have more selection power and you could do more. A- and then there is a number of other technologies we don't yet have, but which if you did, would then kind of stack with that and, and enable much more powerful forms of, of enhancement. Um, so, so, so there, uh, yeah, I don't think there are any major technological hurdles really in, in the way, just some small amount of incremental further improvement.

10. 31:47 – 45:10

    Ethics, wisdom gaps, and coordination failures (nukes as a lesson)

    1. JRJoe Rogan31:47

       That's wh- when you talk about doing something with genetics and human beings and selecting, selecting for the superior versions, and then if everybody starts doing that, the ethical concerns when you start discussing that, people get very nervous-

    2. NBNick Bostrom32:05

       Mm-hmm.

    3. JRJoe Rogan32:05

       ... 'cause they start to look at their own genetic defects and they go, "Oh my God, what if I didn't make the cut?"

    4. NBNick Bostrom32:09

       Yeah.

    5. JRJoe Rogan32:09

       Like, "I wouldn't be here."

    6. NBNick Bostrom32:10

       Yeah.

    7. JRJoe Rogan32:10

       And then you start thinking about all the imperfect people that have actually contributed in some pretty spectacular ways-

    8. NBNick Bostrom32:15

       Yeah.

    9. JRJoe Rogan32:16

       ... to what our culture is. And like, w- what if everybody has perfect genes, would all these things even take place? Like, what are we doing really if we're bypassing nature and we're choosing to select for the traits and the attributes that we find to be the most positive and attractive? Like, what are ... like, that gets slippery.

    10. NBNick Bostrom32:35

        I, uh, and, and you, you think what, what would happen if, say, at some earlier age had had this ab- ability to kind of lock in their-

    11. JRJoe Rogan32:44

        Yes.

    12. NBNick Bostrom32:44

        ... you know, their, their, their prejudices or, um, if the Victorians had had this.

    13. JRJoe Rogan32:50

        Sure.

    14. NBNick Bostrom32:50

        Maybe we would all be, uh, whatever, pious and patriotic now or something.

    15. JRJoe Rogan32:55

        Yeah. Who knows? The Nazis.

    16. NBNick Bostrom32:56

        Uh, or any other, yeah. So, um, so, so in general, with all of these powerful technologies we, we are developing, there, there is ... I- I think the ideal course would be that we would first gain a bit more wisdom, and then we would get all of these powerful tools. Um, but it looks like we're getting the powerful tools before we have really a- achieved a very high level of wisdom.

    17. JRJoe Rogan33:21

        Yeah.

    18. NBNick Bostrom33:22

        And so-

    19. JRJoe Rogan33:22

        But we haven't earned them. The people that are using them are sort of, uh, we're, we haven't ... Like, think about the th- the technology that all of us use. How many, how many pieces of technology do you use in a day and how much do you actually understand any of those? Most people have very little understanding of how any of the things they use work. They put no effort at all into creating those things, but yet they've inherited the responsibility of the power that those things possess.

    20. NBNick Bostrom33:51

        Yeah. I mean, that, that's the only way we, we can do it. Uh, it is just way too complex for any person.

    21. JRJoe Rogan33:55

        Yeah.

    22. NBNick Bostrom33:56

        If, if you had to sort of learn how to build everything, every tool you use, like we wouldn't get very far.

    23. JRJoe Rogan34:01

        Isn't that fascinating though, when you think about human beings and all the different things we do? We have very little understanding of the mechanisms behind most of what we need for day-to-day life, yet we just use them because there's so many of us, and so many people are understanding various parts of all these different things, that together, collectively, we can utilize the intelligence of all these millions of people that have innovated and-

    24. NBNick Bostrom34:25

        Yeah.

    25. JRJoe Rogan34:25

        ... we with no work whatsoever just go into the Verizon store and pick up the new phone.

    26. NBNick Bostrom34:30

        Yeah. I mean, and not just technology, but worldviews and political ideas as well. It's not as if most people sit down, uh, with an empty table and try to think from the basic principles of what would be the ideal configuration of the state or something like that.

    27. JRJoe Rogan34:47

        Yeah.

    28. NBNick Bostrom34:47

        You just kind of absorb it and go with the ... You float in the stream of culture.

    29. JRJoe Rogan34:51

        Yeah.

    30. NBNick Bostrom34:51

        And, uh, um, and, uh, it's amazing just how little of that actually, at any point, channels through your sort of conscious attention where you make some rational or otherwise, but, like, deliberate decision. M- most, you just get carried away with. Um, so, but, but that, that again, I mean, if, if we have, if, if this is what we have to work with, then-
11. 45:10 – 1:02:08

    Autonomous weapons and ‘robot wars’: bans, tradeoffs, and unintended consequences

    1. JRJoe Rogan45:10

       Now, you pay attention to, like, Boston Dynamics and all these, uh, all these different robotic creations that they've made?

    2. NBNick Bostrom45:17

       Well, they seem to have a penchant for doing really sinister-looking, um, bots.

    3. JRJoe Rogan45:22

       (laughs) I think all robots that are... uh, you know, anything that looks autonomous is kind of sinister-looking, if it could do backflips.

    4. NBNick Bostrom45:28

       Well, I mean, you see the Japan... Yeah, I mean, th- like, the Japanese have these, like, big eyes, sort of rounded.

    5. JRJoe Rogan45:33

       Yeah.

    6. NBNick Bostrom45:33

       So it's a different type.

    7. JRJoe Rogan45:34

       They're trying to trick us.

    8. NBNick Bostrom45:34

       Boston Dynamics is-

    9. JRJoe Rogan45:35

       Yes.

    10. NBNick Bostrom45:35

        ... I guess, they want the Pentagon to, uh, give them funding or something.

    11. JRJoe Rogan45:38

        Right, DARPA. Yeah, they, they look like they're developing terminators.

    12. NBNick Bostrom45:42

        Yeah.

    13. JRJoe Rogan45:43

        Yeah. But what, what I was thinking is, if we do eventually come to a time where those things are going to war for us instead of us, like, if we get involved in robot wars, our robots versus their robots-

    14. NBNick Bostrom46:01

        Yeah.

    15. JRJoe Rogan46:01

        ... and this becomes the next motivation for increased technological innovation to try to deal with superior robots by the Soviet Union or by China, like these, these are more things that could be threats that could push people to some crazy level of technological innovation.

    16. NBNick Bostrom46:18

        Yeah, it, it, it could. I mean, I think there are other drivers for technological innovation as well, um, that, that seems, uh, plenty, um, strong, um-

    17. JRJoe Rogan46:29

        Sure.

    18. NBNick Bostrom46:30

        ... like com- commercial drivers, let us say, um, that we wouldn't have to rely on, on war or the, the threat of war to, to kind of stay innovative. Um, and I mean, there has been this effort to try to see if it would be possible to, uh, have some kind of ban on lethal autonomous weapons.

    19. JRJoe Rogan46:50

        Mm-hmm.

    20. NBNick Bostrom46:51

        Um, just as... I mean, there are, there are a few-

    21. JRJoe Rogan46:53

        Drone.

    22. NBNick Bostrom46:53

        There are a few te- technologies that we have, like there is, has been a relatively successful ban on, on chemical and biological weapons, um, which have, by and large, been, you know, uh, honored and upheld. Um, there, there are kind of treaties on, on nuclear weapons, which has limited proliferation. Yes, there are now maybe, I don't know, a, a dozen. I don't know the exact number. But it, it's certainly a lot better than 50 or 100 countries.

    23. JRJoe Rogan47:20

        Yes.

    24. NBNick Bostrom47:21

        Um, and some other weapons as well, uh, uh, blinding lasers, um, landmines, cluster munitions.

    25. JRJoe Rogan47:29

        Mm-hmm.

    26. NBNick Bostrom47:29

        And so, so, so some people think may- maybe we could do something like this with, um, lethal autonomous weapons, killer bots that-

    27. JRJoe Rogan47:35

        Yeah.

    28. NBNick Bostrom47:35

        ... you know, do we... is that really what humanity needs m- most now? Like, another arms race to develop, like, killer bots? It seems arguably the answer to that is no. Um, I've, I've, I've kind of... there's a lot of my friends who are, who are supportive. I, I kind of stood a little bit on the sidelines on that particular campaign, being a little unsure, um, exactly what it is that... Well, I mean, certainly, I think it'd be better if we refrain from having some arms race to develop these than not. But if, if you start to look in more detail, what, what precisely is the thing that you're hoping to ban? So if the idea is the autonomous bit, like, the robot should not be able to make its own firing decision, well-

    29. JRJoe Rogan48:17

        Right.

    30. NBNick Bostrom48:18

        ... if the alternative to that is, um, there are some 19-year-old guys sitting in some, uh, office building and his job is whenever the screen flashes "Fire now," he has to press a red button.
12. 1:02:08 – 1:08:54

    AI, nanotech, and space colonization: ‘technological maturity’ changes everything

    1. JRJoe Rogan1:02:08

       Well, when the iconic image of aliens from another world is this, these little gray things with, uh, no sexual organs and large heads and black eyes, this is the iconic thing that we, uh, uh, imagine when we think about things from another planet. I've often wondered if what we think of w- in terms of, like, artificial life from another plan- or life from another planet is that, it's like an artificial creation. Like, in our ideas that we understand that the biological limitations of the body when it comes to traveling through space, the dealing with radiation, the death, need for food, things along those lines, that what we would do is create some artificial thing to travel for us, like we've-

    2. NBNick Bostrom1:02:55

       Yeah.

    3. JRJoe Rogan1:02:55

       ... already done on Mars.

    4. NBNick Bostrom1:02:56

       Sure.

    5. JRJoe Rogan1:02:56

       Right? We have-

    6. NBNick Bostrom1:02:57

       Yeah.

    7. JRJoe Rogan1:02:57

       ... the rover that roams around Mars. The next step would be an artificial autonomous intelligent creature that has no biological limitations like we do in terms of, like, its ability to absorb radiation from space. And we create one of those little guys, just like that, with an enormous head, no sex organs, doesn't need sex organs, you know, and we have this thing pilot these ships that can defy our own physical limitations in terms of what would happen to us if we had to deal with, you know, one million G-force because it's moving at some preposterous rate through space. Like, we, when we think of these things coming from another planet, if we think of life on another planet, if they can innovate in a similar fashion the way we do, we would imagine they would create an artificial creature to do all their dirty work. Like, why would they wanna, like, risk their body?

    8. NBNick Bostrom1:03:52

       Right. Yeah, I mean, except I think creature might conjure up stuff that... I mean, I, I...

    9. JRJoe Rogan1:03:57

       Okay, things.

    10. NBNick Bostrom1:03:57

        If you have the spaceship, I mean, you don't-

    11. JRJoe Rogan1:03:58

        Right.

    12. NBNick Bostrom1:03:58

        ... have to have, like, build a little thing that sits and turns the steering wheel, right?

    13. JRJoe Rogan1:04:02

        Right. Right.

    14. NBNick Bostrom1:04:02

        I mean, this could be all automated.

    15. JRJoe Rogan1:04:04

        Sure.

    16. NBNick Bostrom1:04:04

        And you'd imagine a technology, um, that is space-faring in a serious way would have nanotechnology, so they would have basically the ability to arbitrarily configure, uh, matter in whatever structure they wanted. They would have, like, nanoscale-

    17. JRJoe Rogan1:04:20

        Right.

    18. NBNick Bostrom1:04:20

        ... probes and things that could shape-shift. And it, it, it's, it would not be that there would be this person sitting in a seat behind the steering wheel. Like-

    19. JRJoe Rogan1:04:27

        Sure.

    20. NBNick Bostrom1:04:28

        ... it, it would just, if they wanted to, they could be invisible toss (?) I think, like, nano-scale things hiding in a rock somewhere.

    21. JRJoe Rogan1:04:35

        Yes.

    22. NBNick Bostrom1:04:36

        Um, then just connecting with an information link up to some planetary-sized computer somewhere-

    23. JRJoe Rogan1:04:41

        Sure.

    24. NBNick Bostrom1:04:41

        ... far away, uh, which would be doing the... So, yeah, I think that's the way that space is most likely to get colonized. It's not, not gonna be like with meat sacks kind of driving spaceships around and having Star Trek adventures. It's gonna be some spherical frontier, um, emanating from whatever the home planet was, moving at some significant fraction of the speed of light and converting everything in its path into infrastructure, um, of whatever type is maximally valuable for that civilization, maybe computers and, uh, launchers to launch more of these space probes so that the whole wavefront can continue to propagate.

    25. JRJoe Rogan1:05:24

        But we are, I mean, one of the things you brought up earlier is that if human beings are going to continue and we, we're gonna propagate through the universe, we're going to try to go to other places, we're going, we're going to try to populate other planets, and are we gonna do that with just robots? Are we gonna try to do that biologically? We're probably gonna try to do it biologically. And one of the things you were saying earlier is one of the things that artificial intelligence could possibly do is accelerate our ability to travel to other lands or other planets.

    26. NBNick Bostrom1:05:53

        Yeah. I mean, we're gonna try. I mean, in fact, some people are, right, trying to-

    27. JRJoe Rogan1:05:57

        Yeah.

    28. NBNick Bostrom1:05:57

        ... biological... I just think that's gonna not, um, lead to anything important, uh, until those efforts becomes obsoleted by some radical new technology wave, probably, uh, triggered by machine superintelligence that then rapidly leads to something approximating technological maturity. Once, once innovation happens at digital timescales rather than human timescales, then all these things that you could imagine we're doing, if we had 40,000 years to work on it, we would have space colonies and cures for aging and-

    29. JRJoe Rogan1:06:31

        Yeah.

    30. NBNick Bostrom1:06:31

        ... all of these things, right? But, um, if, if that thinking time happens in a digital space, then that long future gets telescoped and I think you fairly quickly reach a condition where you have close to optimal technology. Um, and then you can colonize the space cost-effectively. You just need to send out one little probe that then can land on some resource and set up a production facility to make more probes and then it spreads exponentially everywhere. And then if you want to, you could then, like... After that initial infrastructuring has happened, you could transport biological human beings-
13. 1:08:54 – 1:17:19

    Mind uploading and the appeal of our ‘cosmically consequential’ era

    1. NBNick Bostrom1:08:54

       Yes. Now, I also think that at this, this, this is a very radical context, technological maturity-

    2. JRJoe Rogan1:09:00

       Yeah.

    3. NBNick Bostrom1:09:00

       ... because we already... Maybe there are additional technologies we can't even think of yet, but even just what we already know about physics, et cetera, we can sort of see possible technologies that we don't yet... or that we're not yet able to build, but we can see that they would be consistent with physics, that they would be stable structures.

    4. JRJoe Rogan1:09:16

       Yeah.

    5. NBNick Bostrom1:09:16

       And, uh, and already that, um, creates a vast space of things you could do. And so, for example, I think it would be possible at technological maturity to upload human minds into, to, to computers, for example.

    6. JRJoe Rogan1:09:31

       Yeah.

    7. NBNick Bostrom1:09:32

       And so that-

    8. JRJoe Rogan1:09:32

       You think that's gonna happen, like Ray Kurzweil style?

    9. NBNick Bostrom1:09:34

       Well, I think, again, it would be technologically possible at technological maturity to do it.

    10. JRJoe Rogan1:09:39

        Mm-hmm.

    11. NBNick Bostrom1:09:40

        Um, now, whether it's actually gonna happen then depends, A, do we reach technological maturity, and B, do we... are we interested in using our technology for that purpose at that time? But, uh, both of those seem kind of reasonably possible.

    12. JRJoe Rogan1:09:59

        Yeah, reasonably possible in-

    13. NBNick Bostrom1:10:01

        Possible, yeah.

    14. JRJoe Rogan1:10:01

        ... when, especially in comparison to what we've already achieved. If, if I had a time machine, and it could jump you 1,000 years from now into the future, would you do it? Would you jump in?

    15. NBNick Bostrom1:10:15

        Uh, I mean, I think just-

    16. JRJoe Rogan1:10:16

        Do you like how it is right now?

    17. NBNick Bostrom1:10:17

        ... just going on like a, a long jet flight is kind of already stretching my-

    18. JRJoe Rogan1:10:23

        What if it was instantaneous? What if it was an instantaneous trip-

    19. NBNick Bostrom1:10:24

        What? Could, could I come back?

    20. JRJoe Rogan1:10:25

        ... to 1,000 years? No.

    21. NBNick Bostrom1:10:26

        Well, uh, I probably wouldn't.

    22. JRJoe Rogan1:10:33

        Would you not now?

    23. NBNick Bostrom1:10:33

        I, I don't know. I mean, I'm, I'm kind of a, a, uh, uh, bit cautious with these things.

    24. JRJoe Rogan1:10:39

        (laughs)

    25. NBNick Bostrom1:10:39

        But at least, at the very least, I've gotta think about it for a long time before-

    26. JRJoe Rogan1:10:42

        It would be so hard.

    27. NBNick Bostrom1:10:42

        I mean, also I have, I mean, attachments. I mean, there are people I care about here.

    28. JRJoe Rogan1:10:46

        Sure, sure.

    29. NBNick Bostrom1:10:46

        And projects, and maybe even opportunities to try to make some difference. If, if we actually are in this weird time right now, different from all of earlier human history where nothing really much was happening, and we're not yet where it's all out of our hands and the super intelligence is running the show-

    30. JRJoe Rogan1:11:02

        Right.
14. 1:17:19 – 1:31:58

    From ‘too interesting’ to Simulation Argument: three options and substrate independence

    1. NBNick Bostrom1:17:19

       And you wonder, hmm, it's a little bit too much of a coincidence. I mean, it might be the case, but yeah, it, it does put some strain on it.

    2. JRJoe Rogan1:17:28

       When you say a little too much of a coincidence, how so?

    3. NBNick Bostrom1:17:30

       Well, so, um, I mean, I guess the intuitive way of thinking about it, like what way, like what, what are the chances that-

    4. JRJoe Rogan1:17:36

       Right.

    5. NBNick Bostrom1:17:36

       ... just by chance you would happen to be, uh, living in the most interesting time in history-

    6. JRJoe Rogan1:17:42

       Yeah.

    7. NBNick Bostrom1:17:42

       ... being like a celebrity, like whatever, like what, well, like that's pretty low prior probability. Like most people-

    8. JRJoe Rogan1:17:49

       Like you mean like for me?

    9. NBNick Bostrom1:17:49

       Well, for you. Or I mean, for, for ... But for all of us, really.

    10. JRJoe Rogan1:17:54

        For all of us.

    11. NBNick Bostrom1:17:54

        Um, um, and so, that, that, that could just be. I mean, uh, uh, if there's a lottery, somebody's got to have the ticket, right?

    12. JRJoe Rogan1:18:02

        Yeah. But, um-

    13. NBNick Bostrom1:18:04

        Or.

    14. JRJoe Rogan1:18:05

        Or, yeah, or, or, or we are wrong about this whole picture and, uh, there is some very different structure in place.

Episode duration: 2:32:58