Joe Carlsmith — Preventing an AI takeover

ℹ️ EPISODE INFO

Released

August 22, 2024

Duration

2h 31m

Channel

Dwarkesh Podcast

Watch on YouTube

▶ Open ↗

📝 EPISODE DESCRIPTION

Chatted with Joe Carlsmith about whether we can trust power/techno-capital, how to not end up like Stalin in our urge to control the future, gentleness towards the artificial Other, and much more. Check out Joe’s excellent essay series on Otherness and control in the age of AGI: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi/. Enjoy! 𝐄𝐏𝐈𝐒𝐎𝐃𝐄 𝐋𝐈𝐍𝐊𝐒

• Transcript: https://www.dwarkeshpatel.com/p/joe-carlsmith
• Apple Podcasts: https://podcasts.apple.com/us/podcast/joe-carlsmith-otherness-and-control-in-the-age-of-agi/id1516093381?i=1000666255737
• Spotify: https://open.spotify.com/episode/0npJsKzUulSHDVAHumXNtO?si=vyKi0z_CRB6inwUBhIfeFA
• Me on Twitter: https://twitter.com/dwarkesh_sp

𝐒𝐏𝐎𝐍𝐒𝐎𝐑𝐒

• Bland is an AI agent that automates enterprise phone calls in any language, 24/7. Their technology uses "conversational pathways" for accurate, versatile communication across sales, operations, and customer support. Try Bland at 415-549-9654 or bland.ai. Enterprises can get exclusive access to their advanced model at https://bland.ai

• Stripe is financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. Learn more here: https://stripe.com/ If you’re interested in advertising on the podcast: https://www.dwarkeshpatel.com/p/advertise 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 00:00:00 - Understanding the Basic Alignment Story 00:44:04 - Monkeys Inventing Humans 00:46:43 - Nietzsche, C.S. Lewis, and AI 01:22:51 - How should we treat AIs 01:52:33 - Balancing Being a Humanist and a Scholar 02:05:02 - Explore exploit tradeoffs and AI

🎤 SPEAKERS

• Joe Carlsmith

  guest

• Dwarkesh Patel

  host

• Narrator

  other

🧾 EPISODE SUMMARY

In this episode of Dwarkesh Podcast, featuring Joe Carlsmith and Dwarkesh Patel, Joe Carlsmith — Preventing an AI takeover explores joe Carlsmith on AI takeover risk, alignment, and moral futures Dwarkesh Patel and Joe Carlsmith discuss how advanced AI systems could become powerful agents whose behavior is driven by internal motives that may diverge from human interests, and why apparent niceness in current models (like GPT‑4) is not strong evidence of deep alignment.

🔗 RELATED EPISODES

David Reich – Bronze Age shock, the Neanderthal puzzle, & the sudden spread of farming

Jensen Huang – TPU competition, why we should sell chips to China, & Nvidia’s supply chain moat

Dario Amodei — “We are near the end of the exponential”

Andrej Karpathy — “We’re summoning ghosts, not building animals”

Why Leonardo was a saboteur, Gutenberg went broke, and Florence was weird – Ada Palmer

Richard Sutton – Father of RL thinks LLMs are a dead end