Modern WisdomA History Of Existential Risk - Thomas Moynihan | Modern Wisdom Podcast 306
EVERY SPOKEN WORD
145 min read · 28,954 words- 0:00 – 15:00
99.9% of all species…
- TMThomas Moynihan
99.9% of all species that have ever existed are now extinct. So it's, extinction is the rule. Survival's the exception. That's an important thing to know is that, you know, potentially we have come close before. This isn't something that is completely, uh, unprecedented.
- CWChris Williamson
If we're worried about existential risks annihilating our future, why spend any time studying the past?
- TMThomas Moynihan
(laughs) That's a good question. Uh, so I, I, I hope as we talk through this, uh, that the true, uh, significance of what I'm about to say will, um, be elaborated further. But, um, I think that it's so easy to focus on, uh, the risks coming towards us, coming down the track. Um, and it's slightly harder to take stock and look backwards and see just how far we've come. One of the things I mean by that is that, uh, the very ability for us to even be able to see those risks ahead, uh, the risks on the horizon, uh, that's a massive achievement, uh, for, for humanity, um, for, for our knowledge of the world, for our knowledge of what is best to do within the world. Um, that's a massive achievement. And again, I hope that, you know, as we speak through this, uh, you know, um, the truth of this might, uh, hopefully kind of, uh, unfurl. Um, but, you know, some of our biggest achievements are almost in- invisible to us. Um, some of, you know, uh, some of the most profound, um, uh, breakthroughs of human knowledge are often invisible to us. Um, so, you know, I often point towards the fact, um, of, uh, well, take slavery, for example. For most, for the majority of human history, uh, people presumed that it was just part of the natural order of things. Uh, you know, um, it wasn't questioned. Uh, all of us these days kind of take it for granted that, uh, that's, you know, inherently wrong. Um, a- another example I like to use, uh, is, um, perspective, right? So, you know, think back to being a kid in school. Um, you'd learn to draw your first cube or your first, uh, uh, prism or, you know, triangle. Uh, pyramid, sorry. Um, it's so easy. It comes to you so naturally. Uh, rewind, you know, uh, six centuries, seven centuries, uh, it wouldn't have come naturally at all. Uh, you know, I was drawing cubes, I don't know what age, but pretty young. And that's not because I'm a genius or a prodigy or some da Vinci tier, uh, you know, mega genius. It's because of, uh, cultural osmosis, because the ideas that we just take for granted, we inherit. But someone had to come up with them and... Well, often lots of people had to come up with them. And it takes, uh, you know, centuries, decades of effort, of hard work and error correction, uh, of finding out the ways in which we are so severely wrong about the world. Um, and yeah, so to tie up my point, um, thinking, being able to even notice these risks, uh, the risks facing humanity or just the fact of how bad, uh, human extinction would be, um, those are really, uh, huge achievements. And they're quite modern ones as well. Uh, so yeah, I, I would say, I would put it like this. It's, it's a, it's a, uh, it's a cure for despondency, um, because like I said, it's easy to see the risks ahead, uh, harder to see how far we've come. So it's easy to be despondent. It's easy to despair. Um, but it's deceptively easy, because we have that kind of bias where it's easier to look straight ahead rather than look to the past.
- CWChris Williamson
Are you trying to say that a book about the annihilation of humanity is somehow hopeful?
- TMThomas Moynihan
(laughs) Yes. (laughs) So, I, I, I often get this when, uh, people, uh, read my book. They, they, uh, are surprised that, um, it has this... Well, it attemps, it attempts to have this, this hopeful message. Um, but yeah, nonetheless, it does. It does. Uh, you know, and, and funnily enough, I came into all this way less hopeful. And, um, I'm not sure if there's, it would be the perfect, the right word, but, um, more fatalistic. Um, and it was through actually like tinkering through, tracing through, uh, the contours of like discovery and intellectual progress and, you know, just how far we've come in terms of, you know, even if we know that we're not true or we have 100% certainty of anything right now, uh, there's just the abil- the ability to realize that we're wrong, right? To realize that we're wrong, and therefore realize that we can know better, uh, we can correct ourselves. I find that fascinating. So yeah, um, yeah, I think it's a hopeful book. Uh, I hope it's a hopeful book. (laughs)
- CWChris Williamson
I think so. You mentioned there I think quite an important point. You say the, the ability to grasp the prospect of our own extinction is a significant intellectual achievement. It separates us from other animals. Are you also trying to say that that's something that we should be thankful for, this species-wide denial of death that Ernest Becker, like turned up to a million, would be proud of?
- TMThomas Moynihan
Yeah, I mean, so there's, uh, there's a... Jonathan Schell, he was a guy that wrote a book, The Face of the Earth, in the 1980s. Um, and it was one of the first books, it's relevant to this discussion because it was one of the first books to, uh, really, like crisply state how bad human extinction would be. So this was in the context of the Cold War, um, you know, thermonuclear proliferation. Uh, and he, amongst other people, and we can, we can talk about this later if we want, but amongst other people pointed out this kind of asymmetry and how bad extinction is compared to lots of other, you know, kind of disasters is that it's the foreclosure of the whole future.Um, the way- one of the ways in which, uh, Schell expressed this was that there are these two deaths. So the first death is the death that we're all kind of familiar with. Uh, our own death, our individual death. And so that's the kind of Ernst Becker denial of death. You know, a lot of culture, um, is in a sense, kind of, you know, seems to be this, that humans have this unique awareness of our own mortality. Um, I, I'm not f- I'm not too familiar with the Becker thesis but like, you know, say the, say- that was clearly one of o- one of the kind of foundations of culture, right? Humans, since they started using language became behaviorally modern, uh, have probably been kind of aware of mortality in that sense. Um, but yeah, then Schell makes this point that there's this second death, uh, which is the death of the whole species and the loss of its entire future. Um, and that's the more modern achievement. So, uh, yeah, we've been denying death since day one, I guess. But, uh, being able to think about this second death, this, you know, ultimate fate, the loss of the entire future, um, yeah, that's a lot more recent. Uh, and so I'm hoping that we can level up our denial of death to that kind of civilization scale. (laughs)
- CWChris Williamson
(laughs) That's so funny. Can you give, for the uninitiated, for the people that haven't taken the existential risk red pill, what is the most compelling hammer blow that you can give them about why there is an importance to existential risk? I'm already one of the initiates, right? I pray at the altar, I wear the weird mask with the long crow nose on it. Like, you don't need to convince me but what to you is the hit in the existential soul example that you can give people?
- TMThomas Moynihan
Hmm. Yeah. Yeah. So, so, so for me the kind of penny drop, um, uh ... It, the best place I've seen it argued, uh, is, um, uh, philosopher Derek Parfit, who, um, actually was Nick Bostrom's, uh, uh one of his teachers, supervisors. Um, so, and this is around s- the same time as that Jonathan Schell book that I just mentioned, so in the, in the '80s. Uh, he wrote this, uh, book called Reasons and Persons. Uh, and it's, you know, this kind of voluminous, uh, meticulous tome of, uh, you know, kind of, um, uh, very detailed ethical philosophy. Um, and it's, it's a masterwork, uh, you know, um, but it's, you know, kind of deeply philosophical, deeply complex. But in the last couple of pages he makes this argument, um, about that asymmetry that I just pointed to. Um, and he, so he makes, the, the, I think this is the best place this argument's been made so I'll try and rehearse it. So he says, "Think of three scenarios." Uh, the first one is peace. The second one is a, uh, a nuclear exchange wherein, uh, 95 to 99% of humans, uh, are killed. Uh, the third one is, uh, is some kind of exchange where 100% of people are killed. Uh, and then he says, "Where is the biggest difference? Is it between one and two?" So between peace and 95 to 99%, or is it between the 95 to 99% and the 100%? Now intuitively, and, you know, our moral intuition's often wrong, uh, intuitively you might kind of think instantly, uh, "Well obviously it, it's the difference between the first, you know, the first and the second. It's the difference between peace and, you know, the, the 95 to 99% fatality." Uh, Parfit makes, uh, the argument that that's absolutely not the case. The larger distinction, uh, the larger difference in severity is between two and three. Uh, and that's because, again, it's the loss of the whole future. Uh, so he makes, uh, these points that, you know, the earth is likely to remain habitable for another billion years or so. Uh, within that time, uh, there will be vastly more generations of humans than there already have been. Uh, so, you know, civilization itself has only existed for, like, something like 10,000 years. Um, so if we don't screw things up, uh, there's a lot of future ahead of us. Uh, a whole lot of future. And this is just constraining it to the earth, right? Uh, there are other places we can go and other places where we can have even more future.
- CWChris Williamson
(laughs) .
- TMThomas Moynihan
Uh, so basically we're kind of in the daybreak of, uh, civilization, of the human story. Um, and so since then, you know, uh, other people have, uh, extended, uh, Parfit's argument. And it's so funny to me that Parfit kind of, it's like a throwaway in the last couple of pages of this, this huge book. Um, but, you know, namely it's, uh, Nick Bostrom, for example, he, uh, kind of formalized the concept of existential risk. Uh, you know, the first paper where he talks about it, uh, is from, uh, 2002 or 2003. Um, and he makes this further point, uh, coming from, uh, this kind of transhumanist lens where, um, it's not just kind of the duration of the future, it's also how much more quality there could be in it. Because, you know, uh, should humans use their technology in the right way, uh, there's this whole kind of possibility space of other experiences, uh, above the human condition. Um, so, you know, I think a, a, a way that people often put it is that, you know, uh, mice probably aren't very good at experiencing symphonies. Uh, but we are, uh, so there's presumably kind of or- there's presumably headroom above us, right? You know, there's kind of orders of magnitude potentially. Uh, so you add this extra transhumanist kind of, uh, you know, um, uh, spin on it, um, and yeah, I think all the pieces are, are together there. Um, you know, uh, Toby Ord's now, uh, kind of given an even more, uh, even simpler, uh, and even more effective possibly definition of the whole thing. Uh, but yeah, for me it's those arguments that really, um, really drive it home, uh, you know, um, particularly the Parfit one. Uh, you know, it's, it's-It's easy to, uh, miss that fact, uh, that, you know, it's not just, uh, seven billion lives at stake. Uh, it's not just, you know, the current population of the world. It's our whole future and our whole potential as well.
- CWChris Williamson
Didn't Bostrom come up with a number of 10 with 100 zeros after it? Isn't that in your book?
- TMThomas Moynihan
(laughs) .
- CWChris Williamson
If we do a, an okay amount of space colonization, this is the potential number of lives that we could have ahead of us.
- TMThomas Moynihan
Yeah, yeah. So there are, there are, there are huge numbers out there. So, um, uh, Bostrom has this paper, uh, uh, it's called, uh, Astronomical Waste, uh, where he makes this argument that there's like a kind of opportunity cost, uh, for, you know, um, delaying, um, space colonization. Uh, given that there's, you know, there is actually finite, uh, resource, um, within the, the, the kind of accessible universe. Uh, so-
- CWChris Williamson
It's depreciatingly finite as well, right? For every second that we don't, that's another bit of expansion that we can never anymore access.
- TMThomas Moynihan
Mm. Yeah, yeah. So he makes this argument that, you know, uh, we could expand out and, uh, you know, create so many value structures and yeah, that's his word for kind of people or, you know, uh, functional equivalence of people living happy, uh, quality filled lives. Uh, and there are, yeah, there are these computations of the, the kind of upper bound of amount of, uh, you know, souls that we could spread throughout the universe. Um, uh, one of Bostrom's colleagues, uh, Milan Cerkovec, uh, he, uh, came up with a figure before then. I, I think it's some... It's, it's like a, it's a kind of crazy, uh, number. It's like a quadralecto-ili-illion or something. I g- I can't even pronounce it. It's humongous. But yeah, you know, go, g- go to those papers for the precise, the precise, uh, uh, uh, figures, but, um, yeah, I mean, so, so, uh, you know, there's, there's a depreciating amount of this potential as well. But, um, more recently there's been this kind of, uh... So Ord calculates that, um, you know, the, the kind of opportunity cost of delaying is potentially m- you know, not too awful. And so we should be patient and, uh, shouldn't, you know, kind of rush ahead if that, uh, can, you know, um, foreclose our potential if we are too hasty.
- CWChris Williamson
I'm gonna jump ahead to something that I've been thinking about for ages.
- TMThomas Moynihan
Sure.
- CWChris Williamson
And you sadly, Thomas, have the job of being, um, the recipient of all of my bent, pent up x-risk ideas, um, because I don't get to talk about it to everyone. S- for some reason, not everyone wants to talk about the extinction of... Permanent and ultimate extinction of the human race. So it's you and now thousands of people that are listening. As far as I can see, there seems to be three main factors at play, right? When we're talking about
- 15:00 – 30:00
Hmm. Yeah, yeah. I…
- CWChris Williamson
existential risk and how we should potentially move forward. The first one being the danger of technological progress. As Bostrom calls it, the putting the hand into the urn and pulling out a technology. This technology could be good and improve human life but every so often you pull out one which is either gray or black, and if you pull out a misaligned super intelligence, then you, you, you're dead. Game over. And if you pull out, uh, uh, nanotechnology turns us all into gray goo, then you're dead. And if you pull out an engineered pandemic, then you're dead. Um, so that's first bit. You need... There is a danger that's associated with technological progress. Secondly, there is a requirement of technological protection because there is a nonzero amount of natural risk. There are volcanoes and there are asteroids and there is the inevitable heat death of the sun. And we're gonna... You know, we need to continue to technologically progress or else we are sure that we're going to have a limited future because we know that there is a nonzero amount of ex- uh, existential risk that occurs naturally. So there's a balance between those two. And then the final part is the opportunity cost of delaying space colonization. Is that an okay framework to kind of view what we should be doing moving forward? That there is an opportunity cost, there is a requirement for us to not move too quickly as to pull out black ball, and that by moving a little bit more slowly that we reduce the risk of pulling out black ball. But also we can't not move forward at all because there is the natural risk.
- TMThomas Moynihan
Hmm. Yeah, yeah. I think, I think that's a good way of, um, uh, like taxonomizing the major, uh, parts of the, the, the argument. Um, yeah, it's, it's, it's, uh, precisely as you say. It's this, you know, um, mature acknowledgement of the risks, uh, of technology. But conjoint with a mature acknowledgement of just how good, uh, technology could create the f- you know, could, could make the future. Uh, but then also, yes, that also the other acknowledgement that, um, without technology, uh, kind of background natural risks will, you know, the probability will accumulate over time. And, you know, it's a death sentence, uh, it's just a delayed one, right? Um, you know, so yeah, yeah. I mean, and, and that's what's novel about it, um, uh, historically speaking. Um, and so, you know, we can go into the long, the long run history of this stuff 'cause, you know, that's, that's what I love. But, um, you know, it's people have been, uh, talking about human extinction like as a, uh, you know, as a natural possibility for, um, I would say, you know, kind of, uh, three centuries, two centuries, somewhere in that ballpark. Um, you know, uh, then it becomes, uh... Y- during that time with, you know, we're talking kind of scientists during the Enlightenment, uh, you know, they're kind of playing around with it as this, uh, you know, interesting philosophical, uh, natural philosophical possibility. Um, but it remains very kind of distant, very far off. Um, and then, you know, it's really around World War II, uh, and, uh, the production of nuclear weapons, um, and then, you know, uh...... these humongous, uh, nuclear weapons like, uh, the Tsar Bomb and, you know, these kind of really significantly powerful ones, um, in the 1950s. Um, that's the idea of, you know, uh, the idea of human extinction, which had previously just been possible, becomes slightly more probable, plausible, uh, and therefore a policy issue. So people have been talking about, you know, this worst case scenario for, for, for, for decades but often it was constrained to, uh, one technology, that being, uh, nuclear weapons. Um, and, uh, often there were kind of like t- kind of these quite, um, distinct polls of thinking where, you know, um, it's often this kind of, um, idea that, you know, uh, we need to just rush towards technology, have it all. Uh, that's, that's what will save us or, "Oh no, technology's bad, we should, uh, you know, kind of, um, be careful, uh, with what we do with it." This is what's innovative around, um, you know, Bostrom's work and people in that kind of area is this, you know, mature acknowledgement that it's, you know, the poison and the cure and therefore requires lots of care, um, and, uh, lots of careful thinking. And, um, you know, i- what's really great is it's kind of given a shot in the arm to philosophy because it's, you know, there are these philosophical questions that we need to figure out before we have the, you know, the, the, the, the technologies, uh, to wield th- you know, wield the power on the world. So, uh, so yeah, that's, that's how I see it and that's why I say it's important. Yeah.
- CWChris Williamson
As far as I can see, man, it is the most important conversation to have. It is... It blows my mind that we have Greta Thunberg rolling around, going on a, a pedalo from fucking South America back to Europe or whatever to try and reduce her carbon footprint, to talk about a problem which is going to affect us on an existential risk scale in millennia. And we don't have any... I mean, t- in the nicest possible way, like, Nick, N- Nick spends a lot of time working. He's, he's not fantastic with media obligations. I think he probably had to save up about seven, seven months worth of his allowance to go on the Joe Rogan Experience and also the last hour of that podcast was the most painful hour of any podcast in history. Um, for, for anyone that doesn't know what I'm talking about, just listen to the first hour and a half and then please do not delve into the end of it. Um, so I wanna get into kind of my thoughts around culturally the problems with existential risk but as you say, we've got this, this sort of wonderful research that you've done to do with the history of x-risk. You actually start with this kind of cool timeline thing that's all drawn out of the, the, um, the landmarks of the, the, the life of existential risk. You mentioned that existential risk in pre-history before 1600 BC was framed differently to how it is now. Why?
- TMThomas Moynihan
Hmm. Yeah, yeah. So I would say that the, the, the concept simply didn't exist, right? Uh, people couldn't think about it. Um, and again, yeah, this goes back to what I was saying earlier, is you know, one of the things that really, you know, drives me... and so yeah, I mean, you know, I'm just a historian. I, I'm interested in, you know, this, this amazing work, this super important work, uh, in existential risk but, you know, the expert's in that are not me. I'm, I go through and I try and, you know, tell the story of how we got here which I, again, like I said, I think that's important, uh, moving forwards because people so often fail to, um, notice the potential of humanity and part of that potential is looking backwards, as we were talking about. But anyway, so, you know, there are times when there are new ideas, uh, you know, um, ideas that haven't occurred to anyone before. Um, so the, the obvious ones, uh, the, the, I think, you know, most people are familiar with, uh, things like, you know, Darwin's theory of natural selection. Um, you can go back and you can find, uh, you know, you can pick through the ancient Greeks and you can find someone here and there saying, "Oh, maybe, uh, maybe humans used to be fish." Um, that's not, that's not a theory of natural selection, you know? Um, so often the way that like, uh, history is done when it comes to the history of ideas, uh, it's a case of people kind of picking back through the past and going, "Oh, here's something that looks like this new idea." Um, and so that's, that's absolutely the opposite of what, what I wanted to do. Um, I wanted to go, no, this thinking is new and that's why it's important. Um, so you can go back and you can find, uh, these kind of, uh, you know, the greats of, uh, ancient Greek, uh, philosophy, uh, talking about things that deceptively look a bit like extinction events, right? Um, so, you know, humans have always loved massive catastrophes. Uh, we've always loved to, to, to narrativize, to talk about huge disasters, calamities, you know, pyrotechnic volcanic explosions. You know, you name it. Um, you can find Plato. Uh, he, he... So the myth of Atlantis, right? Uh, that's, that's one of Plato's kind of, um... You know, it's, it's a thing that he talks about and he talks about these cataclysms that have wiped, uh, civilization from the Earth, uh, in the past. Um, but then he says, as, as humanity will be wiped from the Earth in the future. So you can start to notice that there's a cycle occurring here. Uh, so, uh, Plato, Aristotle, a lot of these, um, kind of ancient authors, they would talk of these massive, uh, catastrophes but the important thing is, uh, they were presuming that after ca- the catastrophe, uh, humanity would return or recover, uh, civilization would kind of just happen again. Um, so, you know, it's, it's... That's not an existential risk 'cause the important thing in existential risk is the irreversibility.... it's the fact that, uh, our potential is lost forever, um, the human species is gone forever, uh, therefore, you know, it won't ever realize that potential. Um, the very idea that a species could disappear and never return, uh, is a really modern idea as well. Like, all of these ancient philosophers, uh, spent most of their time thinking that, uh, you know, if a species disappears, um, it doesn't matter 'cause it will kind of, uh, it will, uh, continue to exist elsewhere or it will just return at another time. Um, so you can find, you can find them saying stuff like this. There's one in particular, uh, Lucretius, a Roman philosopher, um, and he talks about, um, he talks about, you know, uh, uh, the earth is aging, right? Uh, it's, it's kind of falling apart. It's, uh, losing its, uh, you know, kind of, um, it's losing its, uh, life force is kind of the way he talks about it. Um, and so again, you might think this looks like, oh, he's talking about something like entropy or, you know, but he's not because he says nothing in creation is the only one. Uh, nothing in nature ever d- is ever destroyed because, um, if it's destroyed here, it will reemerge at some other point in the vastness of the cosmic infinity. Um, and so this is a really important idea, uh, that took a long time to dismantle, is this, uh, this confidence that, uh, nothing can ever really be lost from nature. Uh, so be that species, be that the Dodo or be that basically value. Uh, so the recognition, the acknowledgement that, uh, value and the potential to create value, uh, can be irreversibly lost, um, yeah, it's a really modern one. Uh, and like I said, you know, that's super important, uh, because, you know, um, you know, we often talk about these huge revolutions in the way we think. Uh, you know, um, Darwin's theory of natural selection, it, it, it completely changes how we relate to ourselves in this universe. Uh, y- you know, it completely changes how we think about what we are and what we can do. This is another one that hasn't really been written about or noticed yet, is, uh, you know, this recognition that yeah, if, um, you know, if, if, if humanity is lost, if, uh, we lose our potential, if we destroy it, uh, through our own folly or, uh, you know, through insignificant precau- uh, you know, insufficient precaution, um, that's it forever. That's a really important idea. And it's, yeah, it's a really new one as well.
- CWChris Williamson
What was the first existential risk that humans faced as a species? Was there something in, in Paleolithic ancestry where we got down to a population of 12 or something like that?
- TMThomas Moynihan
Yeah. So there's this theory, and it's, uh, I stress theory, uh, there's been, um, there's been some work more recently that's questioned, put, put this into question, but there is this, uh, this, this idea that, uh, at some point, um, I think it's, uh, 75,000 years ago, uh, there was this, uh, super eruption, uh, is the Toba, uh, supervolcano.
- CWChris Williamson
Where's that?
- TMThomas Moynihan
Um, Toba, I think it's in, in Indonesia. Um, so this... and we're talking ginormous, like, you know, uh, there's a cool graph, uh, if you Google Toba, um, you'll be able to see it. It's, it's, it's, uh, it's a graph of like the size of the volcanic eruptions ranked and some of the, the, m- you know, the ones that, uh, are kind of more memorable like Krakatoa, they're just like absolute- they're like pimples compared to Toba. It was huge. Um, so yeah, this absolutely ginormous, um, volcanic eruption, uh, the theory is that it created a, um, a population, um, bottleneck, uh, because it, uh, the climatic format- fallout, um, seems to have, you know, um, this is, we're talking kind of early, um, uh, you know, kind of early behaviorally modern humans so that's like, you know, when we were kind of, you know, talking, doing culture, um, there's evidence that, you know, the pop- the population, like, really narrows down, uh, um, through genetics. There's, there's kind of evidence for this, um, a- at this point and it kind of, you know, the dates line up nicely with Toba. Um, so yeah, again, like I said, it's been put into question more recently, but there is, you know, there is some evidence to show that humanity has come close to the knife edge before. Um, and, you know, that's not surprising because 99.9% of all species that have ever existed, uh, are now extinct. So it's, extinction is the rule, uh, you know, uh, survival is the exception. So, um, yeah, like, you know, I think that's, that's, that's an important, that's an important thing to know is that, you know, potentially we have come close before. Uh, you know, this isn't, um, you know, this isn't, uh, you know, something that is completely, uh, unprecedented.
- CWChris Williamson
Indonesia would have been a real hotspot as well 'cause that's where an awful lot of humans developed from, right? In, I think it was only 12,000 years ago that you had a different homo species that was still existing. You had this sort of pygmy-sized miniature human species that was still existing. So 75,000 years ago, you'd have probably still had Neanderthals, you would've had homo sapiens, you would've had a few others. So potentially it would have caused this bottleneck for all of them, and then maybe we didn't make it out the other side. And I think
- 30:00 – 45:00
(laughs) Um, so no.…
- CWChris Williamson
that that probably highlights one of the real, um, importances with regards to space, space exploration and just setting up a colony somewhere else. You know, we can look at Elon going to Mars and say, "Well, it's spending a lot of money and it's taking up... we could be spending this on inequality or on world hungers and stuff like that." But from a civilization, God's eye view-This is super important. We need to get ourselves off Indonesia because if the volcano decides to erupt, and that can occur in any one of a number of different ways, we also need to make sure that there is no internet connection between us and Mars. (laughs) Uh, that maligned artificial general intelligence is not allowed to get up there as well. But my point is like, that, that really sort of quite nicely I think demonstrates how precarious... 'cause we look at that and we think, "Oh yeah, but, you know, they didn't have technology and there was only a small number." You're maybe talking, I don't know, 50,000 humans at most. I don't know. It could be... It's not gonna be more than a few million humans. Definitely not. Such a small number. How could that occur? And you realize that is just, uh, an inability to judge scale correctly. That for every size that you go up, there is an equivalent catastrophe that could then completely annihilate it. So what about the first record of a human thinking about our own extinction? Was it, was there some philosopher in the, in ancient Greece that you found? Nostrabostardamus?
- TMThomas Moynihan
(laughs) Um, so no. So, so, uh, all of the people from, uh, from, uh, so kind of, you know, ancient, um, classical, so we're talking about like ancient Greeks, Romans, um, yeah. They're always, when they're talking about these big cataclysms, it's always the case, um, as far as, as far as, you know, in my opinion, that, uh, these are kind of false friends. So this is a, this is an idea that I take from linguistics is, uh, um, there are words where, you know, in one language, uh, they... Well, they sound the same in two languages, but mean very different things, right? Um, so I think a, a good example is das gift in German means poison. So... (laughs)
- CWChris Williamson
Not a gift.
- TMThomas Moynihan
(laughs) Not a gift. Yeah. Um, so I think you get false friends in, uh, in, in concepts as well. So, um, you know, uh, like I was saying earlier, Plato talking about these huge, the word he uses is conflagration. So this is like, you know, fire burning up the world. He says the whole surface of the world has, uh, suffered these conflagrations that have, uh, wiped out humanity. Um, but, you know, then he goes, "Oh, as it will happen to humanity again in the future." So it's just part of this cycle that I was mentioning. That's a false friend because on the surface if you just read that sentence, it's like, "Oh, Plato had this kind of, you know, nascent, uh, you know, uh, theory, this incorrect theory of, um, conflagration as a, as a risk. Uh, you know, a civilizational risk." But it's not because again-
- CWChris Williamson
It's not irreversible. Yeah.
- TMThomas Moynihan
Exactly. Exactly. Exactly. So, um, yeah. All of these premodern, uh, thinkers, uh, it's, yeah, there, there's always this strange sense where they talk about these, you know, huge disasters, these global, um, you know, um, uh, burnings or freezings. Uh, those are the things that seem to attract our intuitions. We love fire and ice. Um, uh, but it's always, yeah, within this confidence of, um, you know, everything lost will later return. Uh-
- CWChris Williamson
Who gets a hat then? Who's the, who's the first person that does it properly?
- TMThomas Moynihan
Mm-mm. So there's this kind of, um, it's, it's very gradual. Uh, so we're talking, um, you know, the scientific revolution, that kind of swings around, like, uh, 1500, 1600. Uh, people start to think scientifically. Um, you might think that that would kind of instantly knock all of this kind of naïve thinking, um, out of the way, but it really doesn't. You know, there's this thing, when you work with like the history of ideas, uh, there's this really persistent, um, thing that I call conceptual inertia where, you know, old ways of thinking persist into new frameworks and do so very stubbornly often. Uh, and so I'm sure we're suffering from a lot of it now that hopefully our forebears, if we make it out of the precipice, uh, will look back on us and see how, uh, you know, see how, uh, naive we were. But, um, yeah. So this conceptual inertia in the scientific revolution, uh, is, you know, you have this big shift in worldview, uh, to do with people realizing that we're not the center of the universe. Um, so this is the Copernican revolution. Um, the medievals thought that, you know, uh, that the sun goes around the earth. Uh, Kepler, Copernicus, these scientists, they completely changed that. So you might think people are thinking differently now. They might start to think that, you know, if humanity disappears, that would actually be it. That would be really bad because we're this one planet in this, you know, cosmic, uh, cosmic void, this massive, uh, expanse. They didn't. Uh, so, you know, you get, um, people, uh... So there's some of the original, the first scientists, uh, so Edmund Halley, uh, uh, Robert Hooke, these are the, these kind of pioneers of science. Um, they started to think, um, geologically about the history of the planet. Uh, they started to say, "Oh, there are these huge catastrophes in the past, like massive earthquakes that probably completely reorganized the surface of the earth." Um, but, uh, (laughs) well, actually Halley's very interesting because he says, um, "You know, every time this would have wiped out a civilization, it would have reemerged. Um, it's just such a shame that we've lost all of the achievements of that previous civilization." They might have, uh, had this learned age where they reached peaks of knowledge way higher than us. Uh, it's just a shame that, you know, um, we, we'll have to kind of catch up to them. Um, so even though they're thinking geologically, scientifically, naturalistically, uh, there's still this obstruction here. And the other way that this is expressed, and this is a really important one, is, um, and this has to do with this, this Copernican revolution, this revelation about how huge the universe is. Um, so people would look at these other planets, uh, well, not actually look at them. That happened a lot later, but they would theorize about, uh, how, you know, all these other stars, these, uh, you know, pricks of light in the sky, uh, they're other stars and they must have planets like our own revolving around them.... uh, and they thought, oh, it'd be an awful waste of space, uh, if they weren't populated. Um, so, yeah, they probably have, uh, all of them probably have aliens on them. And people also presumed that these aliens were actually basically humanoid, uh, or, um, you know, were interested in values that were like ours. Uh, so you have this really high confidence, uh, that, you know, humans pretty much exist everywhere, or if not humans, the values that matter to us. Uh, so you know, there wasn't, there wasn't really a sense of there being any kind of, um, uh, possibility for, you know, wasted opportunity for values or kind of vacuums where there is no value. Uh, value is thought to exist throughout the whole universe, which basically, like, fills it up, you know? Um, this is this idea called the principle of plenitude. Um, so, uh, you had to dismantle that before you get this person who gets the hat of being the first person to go, "Oh, maybe this will really matter." And yeah, it's not as simple as that unfortunately. You know, there's no one person who goes, "Oh, uh, if humanity's gone, that's it forever." Um, you know, uh, you find people kind of at the beginning of the 1700s, uh, particularly, um, towards the end of the 1700s starting to play around with the idea of, uh, human extinction, um, and this idea that, you know, we might disappear, the exact same thing might not reemerge. Um, so there's, you know... And this is when people started to dig up bones of, you know, uh, prehistoric beasts, um, and started to realize that actually there are animals that, you know, have disappeared and are gone forever. So again, this irreversibility starts to kind of, you know, trickle into the picture. Um, but still, I mean, h- a really good example is from D- uh, Denis Diderot, this, like, you know, kind of massive, uh, mind of the French Enlightenment. Um, and, you know, he was at this, uh, dinner party, uh, with his other friends, you know, they were probably talking about regicide and guillotining the king. Uh, but during one of these conversations, uh, one of them asks Diderot, uh, uh, who, you know, had these kind of, um, uh, you know, quite, um, iconoclastic, uh, m- materialist theories. Uh, so materialist there means that, you know, he was just being quite mature and saying, "Maybe spirits and supernatural things don't exist." So they asked him at this dinner party, they said, um, "Can humanity go extinct?" Uh, and Diderot said, "Yes, it can." Um, but, you know, uh, it would just re-evolve again in, you know, uh, however many millions of years. Uh, so yeah, you know, it's, it's, it's, it's this kind of gradual process. I mean, funnily enough, um... And let's, actually, I'm gonna give you a definitive answer here. Uh, the person who gets the hat is one of Diderot's friends, um, he was another French philosopher called, uh, Baron d'Holbach. And he, uh, actually has said, uh, he said, "We cannot be sure that all these other planets, uh, contain humans, and that humans are therefore the natural end of all kind of, uh, all evolution, uh, all natural history." Obviously evolution not in the Darwinian sense back then. Uh, but he said that and then said, you know, "Thus, therefore, if our planet was, you know, knocked off its course, that could be it for humanity." So yeah, I think, you know, the- the... Uh, let's give him the hat. He's the first person to, uh, you know, say, "Yeah, uh, A, ahead of evidence we can't just assume that humans are everywhere and values, human values are everywhere. Uh, B, we can't assume that they're the, the end of everything, the purpose of this whole cosmos that we live in. Uh, and C, therefore we can't be sure, again, ahead of good evidence that, uh, if we screw it up, um, you know, something else will just re-evolve like us."
- CWChris Williamson
I think that that really explains nicely the answer to the first question that I gave you about (clears throat) why is it important for us to look back. And given that we're in the scientific, post-scientific world, utilitarian rationalists, I can Scott Alexander my way to an Eliezer Yudkowsky blog and I understand my cognitive biases. And, like, we, we believe that we have reality in our grasp, but the intellectual inertia, cognitive inertia that you just mentioned there, the principle of plenitude as well, this presumption that everything will be okay, when you combine that with, is it scope insensitivity?
- TMThomas Moynihan
Scope neglect. Yeah.
- CWChris Williamson
Scope neglect, that's it. Sorry.
- TMThomas Moynihan
Yeah.
- CWChris Williamson
Um, that big things are really, really hard for us to, to work out and as you scale up-
- TMThomas Moynihan
Mm.
- CWChris Williamson
... small things to big things, the death of a person, uh, the death of two people doesn't feel as bad as the death of one person. And the death of a million people doesn't feel a million times worse than the death of one person. So, when you combine all of that together, I think it, it starts to get us to a place where those of us who want to force the existential red pill down everybody's throats actually start to understand why it might be a little bit of a, a big medication to swallow. Um, wasn't it Thomas Jefferson was famous for believing that some animals, or that no animals ever went extinct? And you think Thomas, Thomas Jefferson's like, he's modern history, you know? There's like drawings of him and paintings and stuff like that, and he was part of a country that's still around now.
- TMThomas Moynihan
Yeah. Yeah. Definitely. Uh, uh, so I mean, if there's anything that my daily work is, it's, uh, just cata- cataloging the vast library of how often we're wrong. Uh, and, and, and, and not just, you know, um, you know, uh, normal people but these huge, uh, these huge great minds. So yeah, like, you know, Thomas Jefferson. Yeah. He's a, he's a great example. So, um-You know, and this goes back to, um, what I was just talking about with, you know, in the 1700s people started unearthing these, these huge bones, uh, of, you know, fossilized bones of unknown beasts. And, you know, prior to this, and I think this is an interesting context actually, prior to this, scientists, um, and natural philosophers, uh, presumed that fossils were, um, they didn't think that they were, um, the kind of impressions of prehistoric animals. Um, that was... Again, I think this is a great example of what we were talking about earlier of, uh, you know, it seems so obvious now, uh, that fossils are, you know, kind of the evidence of prehistoric animals. Like, you know, when you're a kid and you watch Jurassic Park, like you understand that. Uh, but this took centuries for people to figure out. Uh, so the medieval theory, so during the Middle Ages, uh, what people thought was that, um, these strange animalistic imprints in rocks, uh, were actually nature playing jokes on us. Uh, so they had this idea of the kind of scale of being, the great chain of being. So it's everything is in this kind of ordered, uh, hierarchy, uh, from, you know, rocks to plants, to oysters, to monkeys to man. Uh, and they thought that fossils were evidence of where rocks had like, uh, basically become upstart and wanted to, you know, jump above their, um, their, their, their office basically. Uh, so, you know, this is, this is, this is how people thought. And, you know, it's, it's... I, I, I find it really charming and I love it. But, you know, it's, it's, yeah, again, people can be so wrong for them, you know. Yeah. We're, we're often very wrong about a lot of things. So this is how people thought for a long time about these fossils. Da Vinci was one of the first people to go, "Hang on, maybe these are impressions of, uh, of, of animals from, you know, the deep past." Um, and yeah, so, you know, people used to kind of study, it was mainly shells, like small, uh, you know, small, um, small fossils that was very easy to say, you know, this animal, uh, we might not see it in our
- 45:00 – 1:00:00
Every age has probably…
- TMThomas Moynihan
kind of vicinity, uh, alive. We might only have fossil evidence of it. But it probably exists somewhere else in the world. Um, and that was how people, uh, uh, you know, for a long time got around the fact that we had evidence of fossils. Uh, but we don't want to accept species extinctions or the possibility that nature could irreversibly, again, irreversibly lose, uh, any part of it. Uh, and so this was this kind of, you know, tricky maneuver that all these very clever scientists took to deny the possibility of, of extinction, of anything. Um, you know. Yeah. Then people started finding mammoth bones, mastodon bones, uh, and people started to accept, scientists started to accept that perhaps species extinction was a thing that has happened and will happen again. Um, and this was in the late 1700s. Uh, Thomas Jefferson, yeah, he still thought this even up until the point when, uh, you know, scientific consensus was reached. So it's like in the 1780s he was still writing letters to people very confidently claiming that mammoths, uh, still exist in the, uh, kind of un- unexplored regions of the Americas so we just need to go and find a mammoth and then, you know, we don't need to worry about extinction anymore. Um, so, so yeah, yeah, no you're right. You're absolutely right to link this up into, uh, you know, the kind of prevalence of bias and wrong think, uh, yeah. It's, you know, I want to give this hopeful message of, you know, how far we've come and how important all that is, uh, but at the same time, you know, you only reach that by seeing how, how, you know, vastly wrong we often are. Yeah.
- CWChris Williamson
Every age has probably had its Cassandras, right? The, the people that were certain that the end was on its way and you highlight some differences between extinction and apocalypse, and between prophecy and prediction. Can you lay out how all of this works for us?
- TMThomas Moynihan
Mm-hmm. Yeah. So, um, I guess listeners might have already th- thought, you know, "What's this guy talking about? Uh, people have been thinking about the end of the world forever." Uh, have you, I often get-
- CWChris Williamson
Have you not read the Bible? (laughs)
- TMThomas Moynihan
Exactly. Exactly. I often get that comment. It's, uh, yeah, it's, it's one of the more common, uh, comments is, uh, "Has this guy not read-"
- CWChris Williamson
"What is judgment day?" Yeah, exactly.
- TMThomas Moynihan
Yeah. "Has this guy not read the Book of Revelations?" Um, so yeah, the, the claim I make is that apocalypse is distinct from extinction. Uh, and what I mean by that is that, you know, you look at, um, the end of the world as it's presented in religious traditions and mythological traditions, uh, and often, uh, it's seen as the consummation or the fulfillment of the moral order. Uh, so what I mean by that is think of, um, think of judgment day, the Christian version. It's, it's just in the words "judgment day". It's the, the revelation of how God thinks everything should be. So it's the consummation of morality. It's not like, you know, it's not anything bad, even though we might be, it might be very inscrutable to us mere mortals, uh, we might not be able to understand it fully, you know, God's decree, that tribunal is ultimately the right decree. So, you know, another image that, um, pops up is this idea of sorting the good from the bad. That's what the end of the world judgment day is, kind of this point where, uh, the good from the bad is all fully sorted, everything's in its right place, and that's the end. And, you know, then the curtains can close. Um, that's not actually a bad thing. It might be the end of time, but it's, you know, it's actually really great. Like, uh, like I said, morality is fulfilled in this, uh, instance. So in the modern, naturalistic scientific idea of extinction, it's completely different. It's not just, you know, uh, a new version of that old idea of apocalypse. It's actually a contradictory concept because instead of the, um, f- the ultimate, uh, fulfillment of morality, it's the irreversible frustration of it. Um, at least, you know, kind of human morality and, and going back to this point about aliens and other planets, uh, as far as we know, we're the only, uh, animal that follows ethical argumentation, that's able to, you know, kind of think about, um, moral reasons and what should be and what shouldn't be. Um, so you know, if we're gone, all of that is frustrated, potentially irreversibly. Um, and that's really important. You know, those are completely contradictory ways of looking at this. Uh, so there's a pithy way that I like to put it is that, uh, you know, apocalypse supplies a sense of an ending, uh, whereas extinction, uh, anticipates the ending of sense. Um, so it's this idea that, uh, you know, meaning and purpose are irreversibly frustrated within this vast physical cosmos, this vast silent physical cosmos that continues quasi purposely, uh, without us. And yeah, you know, as a final point, and that's also just a really simple, uh, another point to make, a really simple one is that, um, often in religious apocalypses, uh, the physical cosmos doesn't go on without us, it ends with us. Um, or it's again nested in these cycles. So a lot of, uh, eastern kind of, uh, ideas. So the Buddhist, uh, apocalypse, it's, you know, this cyclical, you know, the world ends, it gets reborn, it burns, it gets recreated. Um, so yeah, you know, nothing's at stake in apocalypse, whereas in extinction everything is at stake.
- CWChris Williamson
It's like an egotistic metaphysical, uh, Copernican view of ourselves, right? That the earth's the center of the universe but not only that, we are the center of the universe. And not only are we the center of the universe, we are the bookend to the universe because without us, what is the point of the universe? But oddly that line of reasoning, "Without us, what is the point of the universe?" Is still what we're continuing forward now. The two potential answers that we have to the Fermi paradox of where are all the aliens, one is there aren't any, it's just us. The other is they're out there. Both of them are fascinating, both of them are terrifying, but they do have slightly presuming that the aliens that we're talking about also have morality and the ability to step into their own ethics and stuff like that. The differences are quite profound because it is the difference between being every other semi sentient being, bottlenose dolphins and, and, and bonobos and stuff like that, isn't crew aboard spaceship earth. They're cargo. We are crew. We can affect the direction of what is going to happen. We can save the other animals. We can a- a- allow them to exist at greater and greater levels of comfort and of bliss and of happiness, right? And we can also do that to ourselves. And then we can scale that across the universe. Um, it's so interesting to see this apocalyptic approach. The, it is, it very much is a, a, a very ego-driven sort of Copernican centric, this, this human-centric view of everything. And why wouldn't it be that way? When you're told that God built the world and the universe in six days and the most important thing that he built was on the sixth day and it was the humans, et cetera, et cetera, all of your culture, all of your stories, all the narratives that you've been given are telling you this is why everything is here. You are why everything is here. This is how special you are. And I think this might also highlight why I love existential risk so much, that it's the same as looking at the night sky. It provides an equal amount of awe and dread, and it reminds me that the universe very, very much is indifferent whether or not we continue going on. And it is our stone to roll up the hill if we want to do it. Um, and yet upon realizing that, I, I think that also probably highlights why the denial of death thing from Ernest Becker kind of gets macro aggregated across people with this. It's a very uncomfortable topic to think about because it reminds you that no one's coming to save you. No one gives a shit. Nothing cares about whether or not we continue to exist or not, except for us, which means it's all on us. There is risk, there is responsibility, and the buck stops with us.
- TMThomas Moynihan
Yeah, yeah. I couldn't, I couldn't agree more. Um, if there's one, if there's one major theme of, uh, what I've written on this history, it's the, it's breaking the spell on that kind of wishful thinking, um, where we allow what we want the world to be, to contaminate our theories of what the world actually is. So really broad brush generalization and I'm being unfair to lots of very clever people that came before us and whose shoulders that we stand upon. But the pre-modern worldview often, you know, isn't often doesn't really even think, uh, that there's a distinction between ethics and physics. Uh, what I mean by that is, you know, take for example the, the, the medieval cosmos. Um, you know, this is the pre-Copernican one. The idea of earth in the center, there are these, uh, concentric nested spheres, uh, and not at the edge. There's this primum mobile, it's the outer sphere. That's where God lives. That's where all the value, the best stuff is. Um-... all of those spheres is populated with hierarchies of angels. Uh, you know, the whole thing, there's value suffusing the whole thing. Um, but the very structure of the cosmos, right? The whole, the whole structure is the moral order. So that's what I mean by there's no distinction between ethics and physics there. I perceive, you know, part of the, uh, what kickstarted science and, um, you know, not just science as this, you know, disinterested, objective endeavor of finding out, you know, uh, the- how things hang together, how the facts hang together in the broadest possible sense, but this newer, uh, idea of how we can then, knowing those facts, get our values to fit together with this picture in the broadest possible sense. So I guess, I mean, what is to be done? You know, we've learnt a lot about the cosmos, uh, the objective, physical, you know, uninterested, unresponsive cosmos in independence of us. We've learnt a lot about that. But now we're asking this question of how do our values, how could our values, um, uh, fit into this? And yeah, I see this as this is all part of this, um, this picture. So yeah, you have to kind of, um, basically shake off that wishful thinking of thinking that, you know, in independence of what we do, the universe is just a great place and, uh, you know, kind of all- aligns with our values no matter what. Um, which I think is, uh, the default way of, um, of human thinking. So, you know, there's this idea, um, of, uh, in the philosophy of science of folk psychology, um, where... Or another word for it is the manifest image, um, as opposed to the scientific image. Uh, the manifest image is, you know, our picture of the world as filled with colors, intentions, emotions, uh, all the things that we're used to on our daily, daily lives. And the scientific image is this really barren alien place that's made up of atoms, electrons, uh, you know, subatomic forces. Um, you know, you have to realize that distinction and the fact that the way the world actually is, is not the way we want it to be or it should be or it ought to be, uh, to then reintegrate and think of, "Well, okay, we've realized that. We've woken up, we've broken the spell. Uh, how- what do we do next? Um, how do we make this world, uh, that, you know, just is the way it is independently of us, how do we make it into the one, uh, that we want? Or not just that we want, but that would be worthy, uh, in some, uh, you know, meatier moral sense?" And yeah, this wide scale history that I try and tell of us waking up to the possibility of extinction, it's this, you know, uh, it's this kind of landmark event in that. It's realizing that, you know, um, everything rests on us, uh, not because we're the center of the universe but because the universe simply doesn't care about us. Um, but strangely enough, that also kind of recentralizes everything upon us, um, you know, until we find evidence to the contrary, uh, and I hope we do. I, I, I'm very hopeful. You know, I, I want SETI to be, uh, you know, a successful endeavor. Um, I hope there are wiser beings out there than us. Uh, but we can't just act as if there are, um, ahead of that evidence. So yeah, it's this, you know, it's this strange historical dance between disillusion and, um, you know, that mature, uh, that mature recognition of, "Yeah, I feel alienated by the possibility of extinction, uh, in a way that, you know, the version of me that lived 500 years ago just simply couldn't be. Yeah, I'm alienated by that. It terrifies me." But this is the thing I often say is, you know, if, if I'm hurtling towards a cliff edge, if I'm driving, you know, 60 miles per hour towards a cliff edge, I want to know where that cliff edge is, uh, rather than just wishfully thinking, "Oh yeah, I'll, you know, the car will be fine. I'll be fine. There's no cliff edge, whatever." So I see, yeah. Um, another way you could put this is, you know, waking up to extinction is kind of a, a, a... Santa Claus is in real moment for the human race. Um, and yeah, you know, it might be disillusioning, it might be, uh, upsetting, it might be alienating, but it was something that we needed to do, uh, if we were to have a future. Um, so yeah, I think it's, again, like I say, it's a massive achievement. And, you know, we need to keep that in view, uh, with the future, because I feel that, you know, it's so easy to be dis- to be disillusioned with human potential, uh, to focus on, uh, to focus on the kind of, the atrocities of the recent past, um,
- 1:00:00 – 1:15:00
I don't know what…
- TMThomas Moynihan
and think that that, um, think that that... there's an inevitability that that will color the whole future. Um, so, you know, we need to, um, I'm not saying that we don't, I'm not saying that we should forget, you know, these things, but, um, yeah, I think that there's often, there's often... E- especially, particularly when we talk about the space colonization discussion, there's often a kind of, uh, I refer to as a geocentrism about history. This idea that we will repeat, just repeat everything. Um, you know, all these kind of mistakes of the past, that's just what spreading outwards will do. CS Lewis has this great quote where he's, you know, says, "I feel sorry for the aliens," is basically his sentiment, because, you know, us horrible sinners are going to go out and, you know, ruin the galaxy. Um, you know, that, that's a form of geocentrism, just like thinking that the earth is the center of the universe, thinking that our, um, history colors our future. And of course it does, in an important sense, but the work is, you know, keeping an eye on the places where real progress has happened and seeing the capacity, um, because...And this goes back to the point of, you know, uh, dolphins and, uh, you know, marmosets, and, you know, all these other very incredibly intelligent animals that we share this planet with. Um, is one way of talking about, you know, humans are the only animal that responds to ethical argumentation. And that can sound really abstract and really, um, kind of up in the clouds. I think a, a good way of making that concrete is we're the only animal that's ever corrected itself. Uh, it's ever thought, "Oh, what I previously thought was wrong about what I should do or what I think about the world." Um, so yeah, we've been really wrong but we have that capacity to correct ourselves, and that's the capacity to make the world a better place. So, um, you know, yeah, I really like your, um, the, the cargo and, uh, and crew. Uh, I think it's... Toby- this is a way that Toby Ord puts it, but, um, is that, you know, animals are like moral patients. We're moral patients, but also moral agents. So, you know, I, I see there's a, a, you know, the crew and cargo is another way of putting that. So yeah, I, you know, um, history can be hopeful. Yeah.
- CWChris Williamson
I don't know what it is about this time at the moment. Everybody that's listening and yourself knows, and you have very cleverly sort of managed to evade the, uh, social justice precipice of saying that there aren't any problems right now and that we don't have anything to fix in your little sort of last monologue there. And it's because at the moment, there are... To me, it would appear that there is an obsession with injustices and many of them are indeed injustices that need to be fixed, but I think it discounts so much of how far we've come. It discounts this view that you've given us of just how backward our views were when only Thomas Jefferson was on the planet, not long ago. And yet, when we have these times of crises, when we have times of real, serious bloodshed and apocalypse and concern and catastrophe, people have to center their values. In the time of a real crisis, we focus our values. In the time of no crisis, we create our own. And I think that the problem right now is that people think there is no crisis. I don't think... Because the existential risk, the potential for human catastrophe and the permanent and irreversible stop of the human pursuit isn't there at the forefront of our minds. If we found out that there was a meteor that was heading toward Earth and it was going to hit us within the next 1,000 years, and we had it with certainty and the news stories said the same, I think everybody would live life in a very different sort of way. Some ways better, some ways worse, but I think it would bind us together as a civilization. I think it would stop us from focusing on things that, in the grand scheme of human civilization, don't matter. As Toby Ord says in The Precipice, we are shuffling currently along this cliff edge. It's real precarious. Um, we only need to do this for a little bit of time. I'm not saying that we need to completely dispense with trying to fix the wrongs that are going on in the world. What I'm saying is that right now, there is a very, very important job that needs to be done by all of human civilization, and it is a game of whether or not we get past it or we don't. And if we don't, it doesn't matter how much social injustice you've fixed. What I think that's led to, and everybody will know this sentiment at the moment, is this oddly homo-deprecating view that the world has where we talk about... listen to people talk about ecology and the environment like, as if we go out of our... as if dropping a can on the floor is because you hate the world. As if because the fact that you don't drive an electric car or you don't get the bus to work or you don't get the, the, uh, cycle to work is because you actively hate Mother Nature and she's rebelling against us. And we just start personifying and adding narratives like it's the fucking 2000 BC. Do you know what I mean? Like, we're just layering all of this real primitive thinking that we've got about the sacred and the profane. Meanwhile, Cardi B is on stage singing about a WAP and no one cares. So, all of this bizarre human hating that's going on both within culture and within certain sub-groups within people, I think that a lot of that would be fixed if we understood just how close we are to complete annihilation of everything that you care about, of everything that your genetics down the line can ever care about, and of the ability to save the opportunity to step into programming. I think that's what you said there. The opportunity and the ability for humans to step into their own programming, to redirect the direction in which they are going, is what matters. And, um, man, I really, really hope that the work of the guys at the Future Humanities Institute continues to blow up. We need, we need a Greta Thunberg of existential risk. And I mean that in, in a very, very real way. Like, we need a front of Time Magazine, social media savvy, great speaker that gets the, the, the world behind them because it is the biggest question that we all have. One thing that I haven't heard you talk about, and I'd love to know your opinion on this, given the fact that you've done such a broad view look over this subject over the past of history, just how important do you think this period of the last 30 years with Parfit and Bostrom and Ord, how much, if you were drawing a graph in terms of progress and understanding, how much of a hockey stick has that been? Is it linear or has that really been a marked jump in our understanding around existential risk?
- TMThomas Moynihan
Yeah, I think, um, if you wanted to, uh... Yeah, if you, if you were, if you had, had to, you know, depict it pictorially, I think it has been. Yeah, there's a step change. Um, and-Again, you know, this is one of my senses from studying the history of ideas and the history of intellectual progress is, um, that, that progress is often lumpy and, you know, kind of uneven. And there's, you know, it was kind of punctuated, right? Um, and that's often because these ideas require lots of smaller sub-ideas to come together. So, you know, you have it bubbling away, these sub-ideas, but they're not, you know, converging and then bam. And I think we've just gone through, um, we've just gone through a point like that. So I mean, (sighs) I'm hopeful about, um, I'm not despondent about the fact that m- you know, not enough people are talking about existential risk. Uh, I mean, there's one thing to be sensitive of is like, you know, thinking about the long-term future and how all of our potential vastly outweighs, you know, uh, y- you know, the present and all that stuff. There's this issue of fanaticism that the community, er, around this is very aware of. So it's this idea, um, that, you know, uh, basically when you're thinking of these, you know, huge potential ahead of us, uh, it can outweigh, you know, any sensible decision at the moment because we'd want to vouchsafe that regardless of anything. So, you know, this is, this, this, this is something that the c- you know, community is very self-conc- self-conscious of and it's an issue within, you know, this tech- very technical ethical, uh, discussion. Um, so that's, yeah, that's one thing to, to be, uh, aware of. Um, y- yeah, another thing is I'm hopeful because, um, uh, it's, uh, it's, it's, i- i- it's, it's new. That's why there aren't many people that are talking about it, is because it's new. Uh, these ideas haven't been around for long, and it often takes a while for ideas to really catch on and to start to trickle through the, you know, relevant, uh, streams, the, you know, the relevant institutions. Um, so yeah, I think it, it's, it's new. It's, it's, it's new and that's why, um, and, um, so I am hopeful that, that the discussion will continue. And I, I don't think that, um, uh, you know, people talking about other things like, you know, Greta Thunberg, she's not, like, taking resources from people to talking about existential risk. So, and, you know, climate change is actually a r- like, it might not be an existential risk in the sense that it will, uh, you know, destroy homo sapiens, all of them, but it could actually irreversibly make our future poorer and less well off. Uh, so, you know, it's, I think it has to, you have to, you have to balance all these things. Yeah.
- CWChris Williamson
I agree. Thinking about something that you brought up, which I'd read in a book ages ago, and I'd totally forgotten about and is absolutely fascinating. Can you explain what the Doomsday Argument is, please?
- TMThomas Moynihan
(laughs) Yeah, I'll, I'll give it a try. So, um, there's, there's, there's different versions of it, uh, and interestingly, uh, appear around the same time in- independently. So this is a- another thing in the history of science, I'm gonna nerd out about the history of science for a second, is, uh, often really great ideas, um, people independently arrive upon them around the same time. Uh-
- CWChris Williamson
Why do you think that is?
- TMThomas Moynihan
... I think it's, again, similar to what I was talking about earlier, is, uh, there are all these subcomponent ideas that require, uh, that are required to, uh, then reach, um, the big theory. Uh, so natural selection requires an awareness of species extinctions in the past, it requires an awareness of population dynamics, blah, blah, blah, blah, blah. Uh, so Darwin came up with it, but, uh, Alfred Russel Wallace, uh, almost at the same time completely independently. Anyway, Doomsday Argument, um, basically, uh, you should, applying the Copernican principle, the principle of mediocrity, um, we should assume that we're in a, uh, in a, um, unexceptionable, uh, unexceptional, uh, place. Uh, and that applies, that principle can be applied very broadly. And so when you apply it to our position in human history, uh, so imagine a reference class of all humans ever born, uh, ordered from, um, ordered temporally. So obviously it's arbitrary, the first human ever is arbitrary in a sense because, you know, evolution. Uh, but imagine first human ever, last human humans ever, um, we should p- we should assume that we're in an unexceptionable, unexceptional, uh, placement within that broad, broad reference class, right? Um, so given, uh, given one of the versions makes a reference to population, uh, and how population has expanded, and, uh, if humanity survives into the long term will expand a lot more, uh, it's far more likely that we are later on where there are more people, uh, so imagine this is a curve, than we are very early on, uh, where there are barely any. So the, the analogy that's often used is, um, uh, imagine that, uh, imagine that you have an, a, a bag of, of balls, um, and you, someone's, someone's told you that this bag eith- either has five balls in it or 100 balls in it, um, and you pick the number four. Uh, you would rationally presume that you've got the bag with the five balls, right? Um, so applying that reasoning to that, you know, long scale human race, uh, you ha- you know, you have to, yeah, the, the, the, the conclusion is that we're probably living later on towards the end. Uh, now this gets into all kinds of thorny issues. Um, i- uh, you know, super complicated, uh, kind of reasoning and it's, it's very interesting. Um-Uh, and it's very controversial as well so, um, yeah. I think the best place to go to find out more about it is actually the Wikipedia page. Um, I think Vox did quite a good explainer on it as well. But it's, it's a really fascinating argument, um, really technical, really complicated but, uh, and also mind-bending-
- CWChris Williamson
There'll be a Reddit thread. There'll be a Reddit thread somewhere. Reddit's explained everything. Reddit's becoming, like the new Wikipedia, I think. Thinking about sort of looking forward, what lessons can we learn from looking at our extinction that inform the enlightenment mission moving forward? How do we carry this into the future?
- TMThomas Moynihan
Hmm. Hmm. So I think that, um ... I think ... And this goes back to, uh, what I was saying about disillusion, is, um, we had this enlightenment, uh, where some really brilliant ideas were invented, some pretty bad ones were implemented as well. Um, we had that, you know, uh, 300 years ago, began 300 years ago. Um, and then afterwards we'd entered into this, um, period afterwards where, uh, certain other cultural forces have start- you know, come into the picture, started to compete with that enlightenment idea of progress, of human potential, of, um, the capacity for reason to correct itself to, um, uh, to basically supply reasons to everything. Uh, so, you know, uh, remove the arbitrary,
- 1:15:00 – 1:24:48
But I think we're…
- TMThomas Moynihan
the unjust, the irrational from, uh, not just our picture of our w- the world but our conduct within it. Um, we've had this kind of counteren- the Counter-Enlightenment was actually a historical moment but I think that we're still, uh, there's still, um ... it's still a prominent cultural strain, are these Counter-Enlightenment, uh, you know, it can be pessimistic, it can be romantic, um, there are these cultural strains, um, around that. You know, the, the, the big philosophers like Nietzsche, uh, uh, you know, um, uh, some of the, you know, 20th century philosophers as well, like from Sartre, uh, down to Derrida, they're all kind of playing with these anti-enlightenment ideas. Now I don't want to make predictions about, uh, culture, 'cause there are so many degrees of freedom that you'd be an idiot to do so. But I do feel that, um, that that disillusion with, with enlightenment and its capacity, so not just what it was but what it could be, taking the good bits out of it, uh, and critiquing the bad bits. Um, you know, it's kind of growing up, and this is what the enlightenment, you know, the, the big dogs of the enlightenment said themselves. Kant said that it w- it is just, uh, you know, um, humanity using its own reason to exit its infancy. Um, this is where the metaphor comes from actually, where you see Sagan, uh, y- you know, Carl Sagan said, uh, that we're, we're in this period of technological adolescence, um, defined by this mis- you know, this misfit, this disjunct between our might and our wisdom. Um that's where the metaphor comes from. It actually comes from, uh, Kant and his, his enlightenment forebears. So anyway, what I'm saying is, uh, you know, we have this period of enlightenment then we had the kind of, uh, critique of it, which is itself part of the enlightenment, is criticizing, uh, you know, critiquing, um, and unveiling the biases, unveiling the unj- the injustices. Um, and we're still kind of living through this point and I think it's, it is in a sense, to use Sagan's term, it is a kind of adolescent phase where it's like we've realized our capacities, we've realized the sheer damage we can do to ourselves and this world that we live in, uh, and lots of people react to that with this, this continuing disillusion of, "This is awful. Humans are awful." Um, we can be, right? And it's, you know, again these, these metaphors can become cheesy and they can become very broad brush and very over-generalized but I think we are going through that, you know, not just this technological ado- adolescence but also this kind of adolescence in our image of ourselves and what we can do within the world and what, uh, you know, what we should do, uh, what we ought to do. Um, yeah, where, you know, there's a lot of disillusion, um, and taking this really broad, broad long term view of human history and culture and ideas, uh, that I like to do, I see that as just a necessary step in, you know, waking up and, uh, growing up and ... You know, when you're a teenager there are times when you, you know, do something really awful and, you know, uh, screw up. Then you feel really bad about yourself. Uh, but then you learn that that's just part of this process. And so yeah, uh, you know, real broad brush, long term, uh, talking, I think that, you know, as a, uh, civilization, um, and obviously, you know, again that's a big abstraction. There are lots of parts and there are lots of, uh, parts below that. (laughs) .
- CWChris Williamson
But I think we're going through that phase right now, um, and, yeah, so I'm hopeful as to what's come- what's, what's, what's to come next. Uh, you know, we've realized how bad we can be, uh, but now we can focus on stopping that badness from, uh, from, you know, um, coming to the fore. And we can focus on what we can do, uh, and our potential. We are gods but for the wisdom.
- TMThomas Moynihan
Who's that?
- CWChris Williamson
Uh, that is Eric Weinstein actually.
- TMThomas Moynihan
Oh, right.
- CWChris Williamson
And Daniel Schmactenberger said, "We're gods. We're just shitty gods." Uh, which is-
- TMThomas Moynihan
(laughs) .
- CWChris Williamson
... the same thing said in a different way. One thing that's just popped into my head there, mate, do you think that we're further ahead in terms of ethics or technology?
- TMThomas Moynihan
Um, I mean they're kind of, uh, uh, uncomparable, uh-
- CWChris Williamson
I, I, I know that that's a-
- TMThomas Moynihan
(laughs)
- CWChris Williamson
... an apples and oranges comparison but-
- TMThomas Moynihan
(laughs)
- CWChris Williamson
... if you were to think about... Personally to me, it, it seems like we're further ahead in technology. It seems like our, our technological power outstrips our wisdom. We've made-
- TMThomas Moynihan
Mm-hmm.
- CWChris Williamson
... a lot of progress. We made a lot of progress with ethics, with understanding what good is and how to act in the world, um, but it seems like we're able to scale technological progress much more quickly than that.
- TMThomas Moynihan
Hmm. Yeah, yeah, yeah. So caveat aside of, you know, incomparability, um, broadly, yeah, we are definitely ahead in, uh, technological, uh, progress. Um, this is actually a s- this is actually a point that Parfit makes, you know, to look back to the beginning, this is a point that Parfit makes, um, in this argument about how bad extinction would be. Um, is he points out that applied ethics, so, um, you know, not just thinking about what value is and arguing about metaethics, which is, you know, what value is and how we can define it, but arguing how we can actually implement our ideas of value into the world. So like, oh, you know, ef- making ethics effective, um, is actually really young. Um, and so as part of his wider argument that human extinction would be so awful because we have this vast future ahead of us if we get things right, um, he says, you know, you can be, uh, kind of aggregative about it and say, you know, "There will be so many, zero, zero, zero, zero, uh, future human, uh, lives." But he says there are also these atri- intrinsic goods, these intrinsic values, um, uh, so like art, um, uh, science, right? Knowing about the world. Uh, but then he points to this other one is, um, ethics, and he says this is the most immature field of them all, uh, is it's only, uh, since the Enlightenment that, uh, people have, um, been doing secular ethics, so thinking about morality without God breathing down the back of your neck, um, and it's only even more recently than that pe- that people like Parfit himself have been dedicating, uh, their entire lives to the pursuit of this, uh, and trying to, you know, um, cre- you know, get that low-hanging fruit when it comes to, uh, you know, moral progress. So, um, yeah-
- CWChris Williamson
That, that was the words that I had in my head. The low-
- TMThomas Moynihan
Yeah, right, so-
- CWChris Williamson
... the, the low-hanging fruit.
- TMThomas Moynihan
So I think, um, I think, uh, yes, we are in this stage where our m- you know, our might kind of outstrips our, um, our wisdom. Uh, we are in our technological adolescence, um, or the precipice, you know, however you want to describe it. Um, but that doesn't mean that there's... You know, uh, th- again, you know, I think that there's big potential for, for moral progress and, uh, you know, people are working on these things. Like applied ethics is booming. There's this effective altruist movement where people are, you know, thinking how best can we use our resources. Uh, there's, uh, y- you know, a, a kind of offshoot of that, uh, more recently, um, long-termism is this, uh, you know, um, idea. And I, I, the Precipice, Toby Ord's The Precipice, is kind of the, the, the, um, I guess the founding, uh, text of this. But, um, and there's a lot- lots of work, uh, coming out of the Global Priorities Institute, which is kind of the sister institute of the Future of Humanity Institute, on this long-termism. Yeah, and it's about, you know, how can we, how can we do applied ethics and, uh ... So yeah, I, I think, you know, our technology is outstripping our, our wisdom right now but, um, there's, there's, there's low-hanging fruit to be picked.
- CWChris Williamson
There's some good players about to run out onto the pitch for the applied ethics team as well. I hope so, man. I really do. I think, um, I think it's very timely for you to, to release a book about the, the history of existential risk. I really do. Um, The Precipice by Toby Ord is a fucking magnum opus. I, I absolutely adore that book. Um, Superintelligence, fantastic. Uh, uh, what is it? Human Compatible by Stuart Russell which came out last year, that's also fantastic. But this, Thomas Moynihan, X-Risk: How Humanity Discovered Its Own Extinction. Um, dude, you've done a really, really good job with this. The number of footnotes is absolutely terrifying. Um, it's, (laughs) it's completely ridiculous and, uh, for anybody that's enjoyed this conversation today, it will be linked in the show notes below. I've also added this to my Amazon reading list because that's how much I enjoyed it. So go and check this out if you want. Uh, anywhere else that you wanna send people?
- TMThomas Moynihan
Um, my website is thomasmoynihan.xyz. Uh, I keep that updated with, uh, like short essays that I do. Um, so yeah, I think that's it.
- CWChris Williamson
Perfect, man. The end of civilization hasn't occurred during this podcast, so we have managed to make it through. Thank you for coming on.
- TMThomas Moynihan
Cheers. Thanks a lot for having me.
- NANarrator
(music)
Episode duration: 1:24:49
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 3lfIBgRaOWw
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome