Modern WisdomA History Of Existential Risk - Thomas Moynihan | Modern Wisdom Podcast 306
CHAPTERS
- 0:00 – 1:57
Extinction is the rule: why studying x-risk history can be hopeful
Moynihan opens with the sobering baseline that extinction is normal in nature, not exceptional. He argues that looking backward reveals how hard-won our ability to even notice existential risks is—and that this perspective can counter despair.
- •99.9% of species are extinct; survival is the exception
- •Studying the past shows how far human understanding has progressed
- •Recognizing risks on the horizon is itself an achievement
- •Many breakthroughs become ‘invisible’ once culturally absorbed
- •Historical perspective as a cure for despondency
- 1:57 – 5:24
Invisible moral and intellectual progress (slavery, perspective, and inherited ideas)
To illustrate ‘invisible’ progress, Moynihan compares modern moral certainty about slavery and modern ease with drawing perspective to earlier eras. He emphasizes cultural inheritance: what feels obvious now often took centuries of collective error-correction to discover.
- •Moral progress can become taken for granted (e.g., slavery)
- •Cognitive tools like perspective drawing are historically recent
- •We inherit concepts through cultural osmosis
- •Progress depends on long cycles of trial, error, and correction
- •Noticing extinction’s badness is a modern intellectual development
- 5:24 – 7:44
The ‘second death’: why species extinction is different from individual mortality
Drawing on Jonathan Schell’s Cold War writing, Moynihan distinguishes personal death from the ‘second death’—the death of the species and the foreclosure of the entire future. He frames this as a more recent conceptual achievement than ordinary mortality awareness.
- •Schell’s ‘Face of the Earth’ helped articulate extinction’s stakes
- •Extinction uniquely forecloses all future generations
- •Humans long grappled with individual mortality; species-level mortality is newer
- •Civilization-scale ‘denial of death’ as a needed upgrade
- •Cold War nuclear context made extinction feel policy-relevant
- 7:44 – 12:26
Parfit’s hammer blow: the real moral gap is 99% vs 100%
Moynihan presents Derek Parfit’s thought experiment contrasting peace, near-total nuclear catastrophe, and total extinction. The key claim: the largest moral difference is between near-total death and total extinction because the latter destroys all future value.
- •Parfit’s three-scenario comparison (peace, 95–99%, 100%)
- •Intuition misleads: biggest difference is between 99% and 100%
- •Earth remains habitable ~a billion years—enormous future at stake
- •Civilization is early (‘daybreak’) relative to potential timeline
- •Extinction risk is about the whole future, not just present lives
- 12:26 – 14:32
Astronomical value and the space-colonization opportunity cost debate
The conversation expands to Bostrom’s ‘Astronomical Waste’ and the enormous upper bounds on potential future lives across space. They also contrast ‘rush’ vs ‘wait’ views, noting Ord’s caution that moving too fast could increase catastrophic risk.
- •Bostrom’s ‘Astronomical Waste’ and opportunity-cost framing
- •Universe expansion implies diminishing accessible resources over time
- •Orders-of-magnitude estimates of potential future ‘value structures’
- •Transhumanist ‘headroom’ above current human experience
- •Ord’s argument: patience may reduce risk; don’t be hasty
- 14:32 – 21:15
A practical framework: tech creates risk, but tech also provides protection
Chris proposes a three-part framework: (1) technology can generate black-ball risks, (2) technology is required to mitigate natural risks, and (3) delaying expansion has opportunity costs. Moynihan agrees and highlights the novelty of holding these truths together in one mature view.
- •‘Urn’ model: some technologies are catastrophic black balls
- •Natural risks accumulate without technological defenses
- •Space expansion introduces opportunity-cost considerations
- •Modern x-risk thinking integrates ‘poison and cure’ together
- •Shift from single-risk focus (nukes) to general existential-risk taxonomy
- 21:15 – 26:59
Why premodern ‘end-of-the-world’ thinking wasn’t existential risk (cycles & reversibility)
Moynihan argues that ancient catastrophism (e.g., Plato, Lucretius) often assumed recovery or recurrence—civilization returns after cataclysm. Existential risk requires irreversibility: the idea that a species (and value) can be lost forever is historically recent.
- •Ancient stories emphasize catastrophe but within repeating cycles
- •Key missing ingredient: irreversibility of permanent loss
- •Premodern confidence that ‘nothing is ever really lost’
- •‘False friends’: texts that sound like extinction talk but mean something else
- •Extinction as a modern conceptual breakthrough akin to Darwinian revolution
- 26:59 – 33:42
Did humanity ever come close? The Toba bottleneck and natural near-misses
In response to early-species x-risk, Moynihan discusses the (contested) Toba supervolcano bottleneck hypothesis. The broader point: humanity may have approached the knife edge before, reinforcing that extinction threats are not purely theoretical.
- •Toba supereruption (~75,000 years ago) as a possible bottleneck
- •Genetic evidence suggests a population narrowing (debated)
- •Scale comparison: famous eruptions vs ‘pimple’ next to Toba
- •Natural events can plausibly threaten early human survival
- •Extinction as the default outcome for species in general
- 33:42 – 42:17
Who ‘gets the hat’? From plenitude and inertia to early modern extinction realism
Moynihan traces how scientific revolution thinkers still carried ‘conceptual inertia,’ including the principle of plenitude (the universe must be filled with life/value). He identifies Baron d’Holbach as an early figure who rejected easy assumptions about ubiquitous humanoid life—and thus allowed that humanity could end for good.
- •Conceptual inertia: old frameworks persist inside new science
- •Principle of plenitude: ‘waste of space’ implies life everywhere
- •Early scientists imagined alien-filled, value-filled cosmos
- •Late 1700s: gradual acceptance that ‘reappearance’ is not guaranteed
- •Baron d’Holbach: argues we can’t assume humans/values exist elsewhere
- 42:17 – 46:40
Fossils, Jefferson, and the long resistance to extinction as a concept
The discussion shows how even brilliant modern-era minds resisted extinction: fossils were misunderstood for centuries, and Jefferson speculated mammoths still lived in unexplored regions. Moynihan uses this to underscore how frequently humans—and elites—are confidently wrong.
- •Medieval fossil theories: ‘nature playing jokes’ and the chain of being
- •Da Vinci as an early voice recognizing fossils as past life
- •Workarounds: assume species exist elsewhere rather than extinct
- •Mammoth/mastodon discoveries forced extinction into consensus
- •Jefferson’s letters denying extinction, imagining living mammoths
- 46:40 – 53:57
Apocalypse vs extinction: prophecy vs prediction and the loss of moral meaning
Moynihan distinguishes religious apocalypse (moral consummation, sorting of good and bad, cosmic closure) from scientific extinction (the irreversible frustration of meaning and morality). Extinction implies the universe continues indifferent, potentially without moral agents at all.
- •Apocalypse as fulfillment/consummation of moral order
- •Extinction as permanent frustration of human moral projects
- •Religious end-times often end the cosmos or reset cycles
- •‘Apocalypse supplies an ending; extinction anticipates the ending of sense’
- •Humanity may be the only known ethical reasoner; its loss is uniquely grave
- 53:57 – 1:07:05
Breaking wishful thinking: ethics vs physics, responsibility in an indifferent universe
Moynihan argues premodern worldviews merged physics and ethics (cosmos structured as moral order), while modernity separates them, forcing humans to shoulder responsibility. The ‘Santa isn’t real’ moment is recognizing no cosmic guarantee protects value—so safeguarding the future becomes a deliberate task.
- •Premodern cosmos infused with value; no clean ethics/physics split
- •Science reveals an indifferent universe; values don’t come for free
- •‘Manifest image’ vs ‘scientific image’ as competing pictures of reality
- •Recognizing extinction risk is disillusioning but necessary for survival
- •Anti-geocentrism about history: progress is possible; self-correction is real
- 1:07:05 – 1:09:52
A hockey-stick in x-risk thinking (Parfit → Bostrom → Ord) and the risks of fanaticism
Asked about recent progress, Moynihan describes a punctuated ‘step change’ in understanding over the last few decades. He also notes internal concerns about fanaticism in longtermist reasoning and argues that uptake takes time because the ideas are genuinely new.
- •Intellectual progress is ‘lumpy’: sub-ideas converge then breakthroughs happen
- •Recent decades represent a marked jump in x-risk clarity and framing
- •Community worry: long-term stakes can encourage ethical ‘fanaticism’
- •X-risk discourse is new; diffusion through institutions is slow
- •Climate change may not be literal extinction but can irreversibly worsen futures
- 1:09:52 – 1:13:56
The Doomsday Argument: statistical reasoning about our place in human history
Moynihan outlines the Doomsday Argument’s basic intuition: applying a ‘Copernican’ mediocrity principle to birth rank suggests we shouldn’t expect to be extremely early in humanity’s total timeline. He stresses it’s technical, controversial, and presented in multiple formulations.
- •Copernican/mediocrity principle applied to temporal placement
- •Reference class of all humans ever born and inferred ‘birth rank’
- •Ball-in-a-bag analogy: drawing ‘4’ suggests the small-bag scenario
- •Conclusion (controversial): we may be closer to the end than assumed
- •Argument has multiple independent origins and disputed assumptions
- 1:13:56 – 1:24:49
Carrying the Enlightenment forward: technological adolescence and catching ethics up
In closing, Moynihan frames the current era as an adolescent phase: our power outpaces our wisdom, but moral progress is possible through self-correction and applied ethics. He highlights Parfit’s view that ethics is among the youngest ‘intrinsic goods’ and points to effective altruism and longtermism as emerging attempts to operationalize moral reasoning.
- •Enlightenment ideals vs ongoing Counter-Enlightenment disillusion
- •Sagan/Kant metaphor: exiting infancy into ‘technological adolescence’
- •We’re ahead in technology; applied ethics is comparatively immature
- •Parfit: ethics is a late-developing intrinsic good; much ‘low-hanging fruit’ remains
- •Effective altruism and longtermism as efforts to implement ethics at scale
