By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Great conversations run long. Their best ideas don't have to.
uListen is free on iPhone, iPad, Android, Chrome, and Claude.
See howTranscript
70 min read · 13,599 words- 0:00 – 1:32
Cold Cold Open
- KRKevin Roose
Wow, it is actually... I'm gonna say something that I've never said before while making this podcast.
- CNCasey Newton
What's that?
- KRKevin Roose
It's cold in here.
- CNCasey Newton
I thought you were gonna give me a compliment.
- KRKevin Roose
[laughs] No, that, that's gonna have to wait for year three.
- CNCasey Newton
Mm-hmm.
- KRKevin Roose
But they did fix the ventilation in our studio-
- CNCasey Newton
Yeah
- KRKevin Roose
... so now we have like a nice cool breeze blowing through where previously we had been suffocating and sweating, um, in a, what, what, what amounts to a poorly ventilated closet.
- CNCasey Newton
Yeah, you know, it's incredible the things they're doing with ventilation these days, and by these days I mean since the early 1970s.
- KRKevin Roose
[laughs]
- CNCasey Newton
It did take a while for that technology to get to the Times San Francisco bureau, but it's here now, and I hope it never leaves us.
- KRKevin Roose
Like, I'm chilly. Are you chilly?
- CNCasey Newton
No, I'm, I'm all, I'm all hot and bothered, Kevin.
- KRKevin Roose
You run hot.
- CNCasey Newton
I don't-- You know, I don't always run hot, but when I know I'm about to drop some hot knowledge on people, well, I warm the heck up.
- KRKevin Roose
Yeah.
- CNCasey Newton
You know? Chappell Roan says, "It's like 199 degrees when you're doing it with me," and that is how I feel when I'm podcasting with you-
- KRKevin Roose
[laughs]
- CNCasey Newton
... with the word "it" meaning podcasting rather than how Chappell Roan meant it.
- KRKevin Roose
[laughs] Which is what?
- CNCasey Newton
Mm, I'll tell you when you're older, Kevin.
- KRKevin Roose
[laughs] Okay. Oh. That might be a cold... Hey, we, we always talk about a cold open on this show.
- CNCasey Newton
You wanna hear about a cold open? It's 68 in here.
- KRKevin Roose
[laughs] [instrumental music plays] I'm Kevin Roose, tech columnist at The New York Times.
- CNCasey Newton
I'm Casey Newton from Platformer.
- KRKevin Roose
And this is Hard Fork.
- 1:32 – 1:51
This week on the show
- CNCasey Newton
This week, how Elon Musk became the main character in the 2024 US presidential election. Plus, journalist Laurie Segall joins us to discuss the tragic case of a teenager who developed an intimate relationship with an AI chatbot and later died by suicide. What should Silicon Valley do to make their apps safe for kids?
- 1:51 – 27:34
The Elon-ction
- KRKevin Roose
Well, Casey, we are less than two weeks away from the presidential election.
- CNCasey Newton
Yeah.
- KRKevin Roose
You've heard about this?
- CNCasey Newton
I have, and it's a very exciting time for me, Kevin, because as an undecided voter, I only have two weeks left to learn about the candidates, understand the differences in their policies, and then make up my mind about who should be the president.
- KRKevin Roose
[laughs] Yeah, well, I look forward to you educating yourself and making up your mind. But in the meantime, I want to talk about the fact that there seems to be a third candidate. He's not technically a candidate, but I would say he is playing a major role in this campaign. I am talking, of course, about Elon Musk.
- CNCasey Newton
Yeah, you know, Kevin, I heard somebody say this week it feels like Elon Musk has somehow become the main character of this election, and I'm surprised by how much I don't really even think that that is an overstatement.
- KRKevin Roose
Yeah. No, it, it does seem like he has become inescapable if you are following this campaign. And of course, um, you know, he's become a major, major supporter of, of, you know, former President Trump's, and he's, uh, out on the stump, uh, trying to convince voters, uh, in these cri- critical final weeks to throw their support behind him. Let's, let's just bring ourselves quickly up to speed here and review, uh, Elon Musk's involvement in this presidential election so far.
- CNCasey Newton
Yeah.
- KRKevin Roose
So, uh, Elon Musk endorsed Donald Trump back in July, uh, after the attempted assassination. He also announced that he was forming a pro-Trump PAC, the America PAC, a political action committee, and that he was, uh, he's contributed more than 75 million dollars to that political action committee. And then more recently, he has been appearing at campaign rallies.
- SPSpeaker
Come on up here, Elon. He created the first major American car company in generations, and his rocket company is the only reason we can now send American astronauts into space. Come here. Take over, Elon. Yes, take over.
- SPSpeaker
If, if people don't know what's, what's going on, if they don't know the truth, how can you, how can you make an informed vote? You must have free speech in order to have democracy.
- KRKevin Roose
And then last weekend, Elon Musk announced that his PAC, the America PAC, would give out a million dollars a day to a random registered voter from one of seven swing states who signs a petition, uh, pledging support for the First and Second Amendments. So this is on top of the money that he'd already promised, uh, to people who refer other people to sign this petition. And it goes even further than that, because Wired reported this week that Elon's PAC has also paid thousands of dollars to X, the platform that he owns.
- CNCasey Newton
Oh, so X got a new advertiser. That doesn't happen very often these days.
- KRKevin Roose
[laughs] For political ads in support of Trump's candidacy. The PAC is also taking a prominent role in the kind of get out the vote operation for, uh, the, the Trump efforts. Uh, they are sending, uh, canvassers around in swing states. They are leading some of these get out the vote efforts. So just a really big, uh, sort of effort by Elon Musk to throw his support behind Donald Trump to get him elected.
- CNCasey Newton
Yeah, and Kevin, it just cannot be overstated how unusual this is in the tech world, right? The sort of typical approach that most big business leaders take to an election is to stay out of it, right? And the reason is because typically you are trying not to offend your customers who may be voting for a different candidate, and also you're trying to hedge your bets because you don't know who is going to win the election, and so you want to try to maintain good relationships with both. But that was an old way of doing things, and the Elon way of doing things is to leap in with both feet and do everything he can to get Donald Trump elected.
- KRKevin Roose
Yeah, I, I mean, I think it's safe to say we've never seen anything like this, uh, in an election where the, the, you know, one of the richest people in the world, uh, sort of decides that it has become his personal mission to get one of the candidates elected and uses a major social media platform as a tool to try to do that, and also just spends so much money in, not in just these sort of conventional ways. You know, tech donors, billionaires give money to political action committees all the time. Um, you know, Bill Gates, we just learned this week, uh, from reporting that my colleague Teddy Schleifer did, um, donated 50 million dollars to a pro-Kamala Harris, uh, political action committee. So there are, um, there's a long history of that, but this kind of direct outreach to voters, the, you know, personal involvement, the appearing on stage at rallies, it's just not something we see.
- CNCasey Newton
No, and 75 million dollars is a huge amount of money for a presidential campaign. You know, in Silicon Valley, we get numb to these sums, right? This is a world where OpenAI just raised more than 6 billion dollars in their latest fundraising round. But in presidential election, for one person to donate tens of millions of dollars is extraordinary. And, you know, you just heard Kevin say that Elon Musk just outspent Bill Gates by 50%.
- KRKevin Roose
So- Given the fact that Elon Musk has emerged as a central character of the 2024 presidential election, um, we should talk today about what he's doing, why it matters, and whether you think it's gonna work.
- CNCasey Newton
Let's do it.
- KRKevin Roose
So, Casey, I first want to... The first piece of this that I want to talk about is this million-dollar sort of lottery-
- CNCasey Newton
Yeah
- KRKevin Roose
... that Elon Musk is, is running for people who signed this petition pledging to support the second-- the First and Second Amendments. And he's given out a number of these sort of, you know, giant checks now to, uh, to, to voters who register in one of these, uh, crucial swing states.
- CNCasey Newton
Mm-hmm.
- KRKevin Roose
But the most obvious question about this is like, isn't this illegal?
- CNCasey Newton
Yes. And is it, Kevin?
- KRKevin Roose
Well, it certainly seems to be skirting the lines of legality, is what I'll say. So, in this country, we have federal laws that make it illegal to pay for people's votes, right? You can't go up to someone who's standing at a poll and say, "Hey, I'll give you 20 dollars if you vote for this person." You also can't pay people to register to vote, uh, or offer them anything of monetary value in exchange for registering to vote or for voting itself. So, uh, you know, there have been a number of campaign finance experts who have looked at this and said that this probably crosses a legal line. And we also learned this week that the Justice Department even sent a letter to Elon's super PAC warning them that this action might violate federal law. But, um, you know, Elon Musk and his allies are arguing that this is not illegal because he's not technically paying for votes or voter registration. What he's doing is just giving a million dollars to random, uh, people who sign a petition that is only open to registered voters in certain swing states, which feels like a kind of a loophole to me.
- CNCasey Newton
Yeah. It, it feels deeply cynical, and unfortunately, we see this sort of behavior from billionaires all the time, in particular Elon Musk, where, where there is a rule, he will either break it explicitly or will try to sort of break it via a bank shot method, and then effectively just say, "Come at me, bro." Right? "You, oh, you're gonna come... What are you gonna... Are you gonna fine me a little bit? Yeah, sure. Go ahead and give me a fine. I'm worth upwards of 250 billion dollars."
- KRKevin Roose
Yeah. I mean, what's so crazy to me about this is, like, I remember, I am old enough to remember the last presidential election in which there were all these right-wing conspiracy theories going around about George Soros paying for people to attend rallies, to come out in support of Democratic candidates. And, you know, those were based on basically nonsense, but like, this is literally Elon Musk, a, a, the richest man in the world, directly paying to influence an election by sort of giving millions of dollars away to voters who signed this petition.
- CNCasey Newton
Yes.
- KRKevin Roose
So, it is just explicitly the thing that Republicans in the last cycle were targeting Democrats for doing.
- 27:34 – 1:10:30
Can a chatbot be blamed for teen suicide ?
- KRKevin Roose
So Casey, there's a story, uh, we should talk about on the show this week, uh, that is, uh, about something I've been reporting on for the last week or two, and I think we should just warn people up front that this is a, a hard one. This is not a, a funny story. It's a very serious story involving, uh, self-harm and suicide, and so I think we should just say that up front to people if what they're expecting from us is a, a sort of lighter look at the tech news of the week, this is not that.
- CNCasey Newton
No, but it is a really important story about something that we have been talking about for a while now, which is the rise of these AI chatbots and companions and how powerfully realistic they can come across. People are developing significant relationships with these chatbots by the millions, and this week, Kevin, you reported the story of a 14-year-old boy who developed a relationship with one of these chatbots and then died by suicide.
- KRKevin Roose
Yeah. This is, uh, one of the saddest stories I've ever covered, frankly. It was, um, just heartbreaking in some of the details, but I thought it was a really important story to report and to talk about with you because it just speaks to what I think is a-- this growing trend of sort of lifelike AI companions. Uh, we've talked about them, uh, earlier this year on the show when I went out and made a bunch of AI friends, and we talked at the time about some of the potential dark sides of this technology, that this could, uh, actually worsen people's loneliness if it causes them to sort of detach from normal, you know, sort of human relationships and get involved with these artificial AI companions instead, um, and some of the safety risks that are, uh, inherent in this technology.
- CNCasey Newton
So tell us about the story that you published this week.
- KRKevin Roose
So this story is about a 14-year-old from Orlando, Florida, named Sewell Setzer III. Um, Sewell was a ninth grader, um, and he, uh, according to his mother, was a very good student, a generally happy kid. But something happened to him last year, which was that he became emotionally invested in a relationship with an AI chatbot on the platform Character AI. Um, in particular, this was a, a chatbot that was based on the character Daenerys Targaryen from the Game of Thrones series. Um, he called this bot Dany, and it sort of became, over a period of months, uh, maybe his closest friend. Uh, he really started to talk with it about, um, all of his problems, some of his mental health struggles, uh, things that were going on in his life.
- CNCasey Newton
And was this an official Daenerys Targaryen chatbot that was, like, sort of licensed from HBO or whoever owns the Game of Thrones intellectual property?
- KRKevin Roose
No. So on Character AI, the way that it works is that users can go in and create their own chatbots. You can give them any kind of persona you want, or you can have them mimic, you know, a celebrity. Uh, Elon Musk is a popular chatbot on there, or there's chatbots that are sort of designed to talk like historical figures, like a William Shakespeare or something. So this was one of these kind of unofficial, unlicensed chatbots that sort of mimicked the way that Daenerys Targaryen from Game of Thrones might have talked.
- CNCasey Newton
Got it. And so what happened after he developed this really, um, strong relationship with this chatbot?
- KRKevin Roose
So he spent months talking to this chatbot, uh, you know, sometimes dozens of times a day, and eventually, you know, his parents and his friends start noticing that he just is kind of pulling away from some of his real-world connections. He starts kind of acting out at school. He starts feeling really depressed and isolated. Um, he stops being interested in some of the things that had previously, uh, gotten his attention. And from the conversations that I had with, uh, his mom and, and with, you know, others who, who were sort of involved in this story, it just seems like he really had a, a significant personality shift, um, as, as basically, you know, after he started talking a lot with this chatbot.
- CNCasey Newton
Mm-hmm.
- KRKevin Roose
So his parents weren't totally sure what was going on. Uh, his mom told me that, you know, she knew that he had been talking with, uh, an AI, but that she didn't really know what they were talking about. She just basically assumed that he was kind of getting addicted to social media-
- CNCasey Newton
Mm-hmm
- KRKevin Roose
... to Instagram or TikTok. And so his parents, after some of his behavioral problems started, referred him to a therapist, and he went, uh, a few times to see this therapist. But ultimately, he preferred talking about this stuff with Dany, with this chatbot, and so he had kind of these long series of conversations with this chatbot that culminated in February of this year when he really started to spiral into thoughts of self-harm and suicide, and of wanting to sort of leave the, the sort of base reality of the world around him and go to be with this fictional AI character, um, in the world that she inhabited. Uh, sometimes when he spoke about self-harm, the chatbot would discourage him, saying things like, "Don't you dare talk like that," but it never broke character, and it never directed him to any sort of outside mental health resources. So on one day in February of this year, uh, Sewell- ... had a conversation with this Daenerys Targaryen chatbot in which he says, uh, that he loves the chatbot and that he wanted to come home to her. The chatbot responded, "Please come home to me as soon as possible, my love." And then Sewell, uh, took his stepfather's, uh, handgun that he had found, uh, in, in a drawer in their house, and he, um, killed himself.
- CNCasey Newton
Mm-hmm.
- KRKevin Roose
And so, obviously, um, horrible details of this, um, and, uh, I just, I heard this story and I thought, "Well, this is something that more people need to know about."
- CNCasey Newton
Yeah, and it, it, it hits on some big themes that we have been discussing this year. There is the mental health crisis among teenagers here in the United States. There is a loneliness epidemic that spans across different age groups, and there is the question of w- when should you hold tech companies accountable for harms that occur, uh, on their platforms or as a result of people using their platforms?
- KRKevin Roose
Yeah, and this is, uh, you know, not just a, a sort of story about, uh, what happened to Sewell. Um, it is also a story about this, uh, this sort of legal element here, because Sewell's mom, Megan Garcia, uh, this week filed a lawsuit against Character AI and named the company as well as two founders, Noam Shazeer and Daniel De Freitas, as well as Google, which eventually paid to license Character AI's software, essentially arguing that they are complicit in the death of her son. So, it raises all kinds of questions about the guardrails on some of these platforms, the fact that many of them are, uh, very popular with younger users, and what obligations and liability a platform has, uh, when people are relying on it for these kind of lifelike human interactions.
- CNCasey Newton
Well, let's get into it.
- KRKevin Roose
All right. So, to join us in this conversation, I wanted to invite on Laurie Segall. Laurie is a friend of mine. She's also a journalist. She now has her own media company called Mostly Human Media, and she's the reason that I learned about this lawsuit and about Sewell's death. Uh, we sort of worked on this story in tandem and interviewed Sewell's mom, Megan, together, and she's also been doing a lot of her own reporting on the subject of AI companionship and how these chatbots, uh, behave. And so, I thought she would just add a lot to our conversation, so I wanted to bring her in.
- CNCasey Newton
All right.
- KRKevin Roose
And before we do, I also just wanna say, if you are having thoughts of suicide or self-harm, you can call or text 988 to reach the National Suicide Prevention Lifeline, or you can go to SpeakingOfSuicide.com/resources for a list of additional resources.
- CNCasey Newton
Laurie Segall, welcome to Hard Fork.
- LSLaurie Segall
It's good to be here.
- CNCasey Newton
So, Laurie, you have done a lot of reporting on this story, and there's many details we want to get into, but let me just start by asking, how is Sewell's mom, Megan, doing?
- LSLaurie Segall
I mean, that's such a hard question, right? I, I would say, you know, she said something to me, um, today. She said, "I could either be curled up in fetal position, or I could be here doing this."
- CNCasey Newton
Mm. Mm.
- LSLaurie Segall
Um, you know, with really nothing in between.
- CNCasey Newton
Yeah.
- LSLaurie Segall
And I think that pretty much says it, right? She's lost her son. She's now kind of on this mission to tell people what happened, um, and she's grieving at, at the same time, like, I think like any parent would.
- CNCasey Newton
So, let's get into what happened. When did Megan learn about her son's relationship with this chatbot?
- 1:10:30 – 1:11:08
Pink Garbage Truck
- KRKevin Roose
Oh, uh, my kid told his first joke this week.
- CNCasey Newton
Oh my gosh, that's exciting. What was the joke?
- KRKevin Roose
Yeah. So, you know, he's obsessed with garbage trucks. This is sort of his thing.
- CNCasey Newton
Mm-hmm.
- KRKevin Roose
Every week we wake up, we watch the garbage trucks, they honk at him. It's a whole thing. And, uh, and the last time the garbage trucks came, we were sitting out on the porch and, uh, he just goes, "Pink garbage truck." And then he starts dying laughing.
- CNCasey Newton
[laughs]
- KRKevin Roose
He's never laughed so hard in his entire life. Uh, so that was his first joke, pink garbage truck.
- CNCasey Newton
That's great. There is something sort of about the gender binary that he's already starting to explore in his work.
- KRKevin Roose
[laughs]
- CNCasey Newton
I think it's really interesting, actually. Be curious to see where he goes from there.
- KRKevin Roose
Very avant-garde.
Episode duration: 1:11:08
In the uListen app, the transcript lights up word by word as you listen. Get uListen
Transcript of episode 0Lu8CT90ErM
