We Have to Talk About That Meta Manifesto ...

Kevin Roose 0:00 Well, Casey, as regular Hard Fork listeners know, it has been a big year for very long manifestos written by people who run AI companies about what their vision of the future looks like, and we got another big one this week.

Casey Newton 0:12 We really did. Mark Zuckerberg published 6,500 words, uh, his effort to lay out a positive vision for AI. It followed a shorter version that he published in The Wall Street Journal, raising the prospect that he will continue to publish longer and longer AI manifestos, Kevin, until his demands are met.

Kevin Roose 0:30 [laughs] Yes.

Casey Newton 0:31 Well, before we get to everything that was in this manifesto, Kevin, we should probably do our disclosures.

Kevin Roose 0:36 I work for The New York Times, which is suing OpenAI, Microsoft, and Perplexity.

Casey Newton 0:40 And my fiancée works for Anthropic.

Kevin Roose 0:43 So this one is called The Future Is for Everyone, and the essay starts with this sort of vague, uh, and optimistic vision. We are fortunate to live in an incredible moment in history. In the next few years, people will be able to use superintelligence beyond human capacity to create and discover extraordinary new things, build new businesses, express new ideas, learn new concepts, and advance our health and quality of life. And then he goes on to talk about things like, uh, what is Meta building? They want to have, uh, every one of their users have an exceptionally capable personal agent that understands you, your goals, and everything you care about. Uh, you could access this through, uh, any device, including your glasses, he says. Um, then he talks about some ways that he is using his AI agent, uh, to flag interesting information and help him prototype ideas, to keep him healthy by monitoring his sleep and tr- watching as he trains, and then by giving him a- and his daughter personalized recipes to bake together every weekend. There's a lot of other stuff in this essay. It goes on for many thousands of words to talk about job growth and, uh, compute, recursive self-improvement, existential risk, bio risk, things like that. So Casey, you had a post this week on your newsletter about, uh, House of Dragons or, or some, some Game of Thrones spinoff that I have not watched.

Casey Newton 2:06 House of the Dragon, Kevin.

Kevin Roose 2:07 House of the Dragon.

Casey Newton 2:08 It's one of the biggest shows in America right now.

Kevin Roose 2:09 [laughs] Okay, walk me through the argument you made there-

Casey Newton 2:12 Yeah

Kevin Roose 2:12 ... 'cause I thought it was interesting, even though I didn't fully understand it.

Casey Newton 2:14 So, uh, th- this past Sunday, House of the Dragon had its third season finale on HBO, and the thing about House of Dragons is that it's a show that begins with a terrifying concentration of power where only one great family has access to a super weapon, which in this case is, uh, a dragon. Um, and as the start of the show, there is a schism, and all of a sudden there are two factions that have access to dragons, and then there is a sort of very bloody civil war. And as I was watching this, I thought, "You know, I do think you can draw an analogy to AI here," because while I do believe that there are many positive things that AI can do and is doing, I do worry about the medium and long-term future, particularly as we start to see these agents escaping their sandboxes and re- and wreaking havoc. And Zuckerberg's essay meets this analogy in a really interesting place because he says that the way to make us all safe is to, uh, sort of maximally proliferate AI throughout the entire world and give personal superintelligence to everyone. In my view, Kevin, that is a little bit like giving a dragon to everyone, right? Because while I'm sure most people will spend their time creating personalized baking recipes to bake with their daughter, there are other people that are gonna be launching cyberattacks and are gonna be e- engineering novel bioweapons, and I just get really, really nervous about that. So when someone comes along and says, "I wanna give a dragon to absolutely everyone," I say, "Hold your horses or your dragons."

Kevin Roose 3:40 Right. And, like, giving everyone a superintelligence that aligns with their values is, is one of the sort of rhetorical twists that he does in this essay. He basically tries to say, "Well, there's no such thing as, like, a fully aligned universal AI," the way that people sometimes talk about it, because people have different values and different wants and different needs. And, like, instead of having one superintelligence that sort of has this universal code of values, everyone should have their own personal superintelligence that is tailored to their values. And, like, that is a classic case of, like, sounds great, but in practice, like, giving the CCP, uh, a, an AI superintelligence that, that obeys their values and mirrors their values would allow them to commit, like, atrocious acts against their own people.

Casey Newton 4:25 And you know, the answer to that is typically, look, superintelligence will help the defenders as much or more as it helps the attackers, and a, a new equilibrium will be reached. And I do believe this will be true in some cases. Like, I can imagine it being true in cybersecurity, for example. The problem is there are some kind of attacks, Kevin, where it just takes time for defenders to catch up, right? If I release a novel pathogen into the world that I'm able to just, like, sort of create in my computer and my lab, it is just going to take the defenders a little bit longer. So what I would love to see in these manifestos is just an acknowledgement of the utter complexity of this world, and rather than come along and sort of paint this incredibly happy vision... Like, you, you can have your happy visions, but I think this essay in particular only pays glancing attention to the risks.

Kevin Roose 5:20 Yeah, there's, there's a interesting section in the essay about bio risks-

Casey Newton 5:24 Mm-hmm

Kevin Roose 5:24 ... specifically because he's someone who has had, uh, his own research teams, uh, doing stuff around biology and AI for many years now. Like, he's very interested in the subject, and he actually acknowledges that, like, this may be a case where the, i- the attackers and the defenders having equal tools may not be the perfect solution, and he kind of punts on it. So I think he is, like, aware that this is... Like, I don't think he's fully naive, um, but I think he just doesn't have a good answer for that part yet.

Casey Newton 5:51 No, and all of that comes secondary to what I view as the actual purpose of this essay, which is to advocate for a bunch of policy positions that are beneficial to Meta, right? Which gets into the next thing that we wanna talk about today, which is why this essay and why now?

Kevin Roose 6:06 Yeah, so you've been a close student of Mark Zuckerberg-

Casey Newton 6:08 Yeah

Kevin Roose 6:08 ... uh, for many years. Like, what, what do you think he's up to writing this essay now?

Casey Newton 6:12 So when you read this essay, here are some of the things that it asks for, Kevin. Uh, accelerating the process for building data centers, uh, which the company needs to accelerate this potential Neo cloud business that it's building. He interestingly, as you know, even though he's Mr. Pro Open Source, he wants us to maintain export controls on advanced chips, which advantages Meta's open-weight models over any Chinese or other alternatives, uh, because the, you know, the Chinese d- don't have access to the chips that Meta does. He wants, um, the government to reduce what he calls training data restrictions, which would help Meta fight various ongoing lawsuits from the creatives whose works were used in creating its models, and he wants to see legal protections for distillation, uh, basically letting Meta, uh, you know, use the outputs of other models to train its own. So buried inside this very positive, happy vision of AI is just a series of policy requests to help Meta as a business.

Kevin Roose 7:10 Uh, there have been some people speculating that, like, he is promoting this now because they are trying to divert attention from the other thing that is going on at Meta right now, which is all these lawsuits and court cases about the sort of various failures and dangers associated with their social media products.

Casey Newton 7:25 Yeah, I, I don't think we have to attribute that to other people. I would say that.

Kevin Roose 7:28 [laughs] You think this is just sort of a, a distraction from the, the Ls that they're taking in court?

Casey Newton 7:34 I mean, not exclusively. I think this essay serves multiple purposes, and one is to get that policy wish list out there, right? This is something that all of Meta's lobbyists can now take into Congress and say, "Look what Mark is calling for. This really helps you understand, you know, what we're thinking about all these issues." That's a, an important reason. Um, but I do think the timing here is really notable, Kevin, because as you note, Meta is in this series of getting its ass handed to it in court case after court case related to its existing business, where state after state is coming after the company saying that Facebook and Instagram in particular are not safe for teens. Last week, the New Mexico judge ordered Meta to pay an extra $567 million into a teen mental health abatement fund. That's on top of $374 million in civil penalties. And I think more importantly, Kevin, that, and this is the thing that's really gonna stick, the judge ruled that Meta's platforms are a public nuisance. He compared Meta to factories with the psychological harm and exploitation of children as the pollution it emits, and he's also ordered really, like, strict new safety measures, at least by American standards, a 90-hour-per-month cap for under-18 users and some new, uh, restrictions on AI chatbots. So, you know, keep in mind, Kevin, there was a time when Zuckerberg was writing these happy manifestos about social media.

Kevin Roose 9:00 Right.

Casey Newton 9:01 And he was saying that the way that we're gonna have a happy world is we're gonna make it more open and connected. We're gonna get every single human being on Facebook and Instagram and get them all talking, and we're gonna ha- you have more democracy than you've ever seen before. Suffice to say, that didn't really work out. Now we get to bring the exact same maximalist, universalist framing, which of course also maps 100% to Meta's business interests, but this time in the context of AI.

Kevin Roose 9:24 Yeah. I, I agree with all that. I think there's an interesting question here, though, which is, like, do we think Meta has a shot at actually building superintelligence? Right? Like, a lot of people have views on the future of AI and the future of superintelligence, and we don't really care about them because those people are not in a position to actually make superintelligence.

Casey Newton 9:43 Mm.

Kevin Roose 9:43 I would say until very recently, my position was that Meta was sort of out of the race to build powerful AI systems that could one day become superintelligent. I'm curious where you stand on that. Like, do they have an actual shot at bui- at bringing about the future that Mark Zuckerberg is talking here?

Casey Newton 9:59 Well, listen, the first rule of Mark Zuckerberg is never count out Mark Zuckerberg. He truly is one of the very most competitive people in the entire world. He will move mountains in order to get what he wants, and we saw him do that a little over a year ago when he reorganized his AI efforts yet again. They have made notable progress since then. When I talk to my a- AI friends, they tell me that they actually think pretty highly of some of the moves that Meta has made over the past year, particularly when it came to reassigning a bunch of engineers to do what is essentially, like, reinforcement learning, creating training data. The Meta engineers didn't really love that, but AI folks I speak with say that is actually going to give them something really valuable. But to answer your question in brief, no, I do not count them out. What do you think?

Kevin Roose 10:45 Yeah, me neither. I, I think, you know, I probably would've given them a 1% chance of creating superintelligence, uh, six months ago, and now I'm up to maybe, like, a 10% chance, which is, which is a big improvement. I think their models, uh, have been getting steadily better. Some of their training runs that they did after they built the whole Meta Superintelligence Labs and hired all those expensive researchers and bought all that compute, some of those, uh, have come online and are now starting to produce good results. They had some, some pretty impressive results on their latest model. So I think it is, it, it is true that Meta is not a frontier lab right now, but I think they are showing signs of rapid improvement, and, uh, that worries me, as someone who thinks that this is not a company that sort of has the, the DNA culturally, um, or the track record of building products at scale that are actually safe and responsible for people. And it's making me think of this conversation I had a few years ago with a former DeepMind executive, where this person was basically saying, "Look, there are two types of AI companies. There are companies that think that they are building tools, and there are companies that think they are building AGI or-

Casey Newton 11:56 Mm-hmm

Kevin Roose 11:56 ... like, an, an entity, something that could become smarter than humans. And it's fine to be either one," is what this person said. Like, you can do what, you know, a lot of companies have done, which is just decide we're, we're just not gonna be in the AGI game. We're gonna build these tools. They're gonna be very useful to people. They'll get smarter over time as the models get smarter. That's the business we're in. It's also fine to be a company that is explicitly trying to create AGI or superintelligence.

Casey Newton 12:21 Mm.

Kevin Roose 12:21 As long as that's what you know you're doing, and as long as you're sort of taking the right precautions, and as long as you're going into it with the right spirit, um, that can be done responsibly, too. This person said the, the real danger is if you have a company that thinks it's designing tools but is actually designing superintelligence, and that kind of company, this person said, is not going to be taking the proper precautions. They're not going to be treating the technology with the correct sense of sort of reverence and, uh, and suspicion because they don't actually believe deep down that it's ever going to get powerful enough to be dangerous. And so they're just going to sort of waltz, uh, right into this disaster because they have no conception of what they're building. And at the time, this person was saying this to me about Google-

Casey Newton 13:10 Hmm

Kevin Roose 13:10 ... which I think, you know, a couple years ago was in sort of the, the, the throes of this debate about whether they were building superintelligence or AGI or whether they were just building, like, better versions of Google Photos and Gmail and Google Search.

Casey Newton 13:22 I think Meta is in this position now where they are maybe going to build something extremely powerful with this, I think, very naive attitude about the fact that this, these things will only ever be tools that will be useful for recipes and things like that. Yeah, a- and again, it, it is the company's history that just makes me concerned because the way that this company operates is by growing as much as it can and treating everything as an existential competition against the other guy, and the, uh, sort of external effects on society are typically given short shrift. So, you know, in this present moment, I do not trust these people to rank a list of viral dances for me to look at- [laughs] ... without it corrupting my mental health. Once [laughs] you give these things access to n- like, novel b- biotechnologies, it starts to get pretty worrisome, Kevin. Yeah. [laughs] Yes. I would also just say, like, Mark Zuckerberg is possibly the worst messenger for the AI industry on all of this. Like, if he thinks that writing this positive, sunny vision of super intelligence is going to, like, sway public opinion around AI and, you know, get people to stop protesting data centers, I think he is badly mistaken. Yeah. Um- From the people who brought you Cambridge Analytica comes... [laughs] Super intelligence. Super intelligence. [laughs] Yes. You know, I mean, here's the thing, and, and this is not limited to Zuckerberg. Any of these AI labs [laughs] maybe eventually will do this. Like, you could just deliver actual benefits to people's lives, you know? It's interesting to me that the manifesto has to come so far ahead of the actual benefits. Uh, we are just clearly long past the time when writing essays is going to shift public opinion. What is going to shift people's opinion is AI is getting them paid more, right? Um, it is, um, obvious that it is not going to harm them and their families, and it delivers other benefits into their lives. You know, it's making them more creative. It's, it's giving them more entertainment. And of course, to some degree, some of these things are sort of happening, but not in the volume and magnitude that are necessary to counter people's very reasonable fears about what they're seeing. Yeah. Uh, in conclusion, when is your 6,500-word manifesto about your vision of the AI future coming out? 'Cause- Well, I wrote about 1,800 about Zuckerberg this week- [laughs] ... so consider that my opening salvo, but I too will write additional essays as conditions demand, Kevin. One impressive thing about this manifesto to me is that Zuckerberg does actually seem to have written it, or at least- Yes ... a human seems to have written it. I agree. Um, a- as soon as it came out, people were, like, running it through AI detectors and finding that it was not flagged as being AI-written. No, and I will give it that. I read it. I never once thought I was reading slop. I thought, like, you know, p- particularly in some parts, I was like, this is just actually how Zuckerberg talks. I'm sure it was a little bit like the State of the Union, where lots of different, you know, policy hands had their fingertips on it and said, "Oh, you know, make sure to say this." But, um, no, I, I do think that this is hi- his actual message. Um, to what extent he believes, uh, you know, everything in it and to what extent a lot of it is just sort of, um, messages of convenience, uh, well, I guess I don't know that. Yeah, so cynical. Here- Can't you just, can't you just admit that maybe Mark Zuckerberg is just a misunderstood optimist who just wants to make the world a better place? I've been thinking about this a lot because a problem that I have in covering Meta, just legitimately, is that I've covered it for, like, more than 10 years. I mean, it's, like, been almost, like, 15 years, and so I just know a lot about this company. And obviously, you know, it, it has changed in various ways over the years, but I just remember so much about this company. I wish that I was born yesterday and just could believe everything that I read, but Kevin, I know too much. Yeah, yeah. I wanna ask you maybe a, a final question, which is, like, is there a role for AI positivity? Like, what should that look like? Who should be writing these positive visions? Like, clearly, uh, we don't believe it is Mark Zuckerberg, but, like, someone presumably should be out there saying, "Here's what the world looks like if all this goes right." So I do not think this is a messaging challenge. I think that there is room for AI positivity, but here's what AI positivity looks like to me: new medical advances powered by AI, diseases cured, new jobs created, old jobs paying more money, kids excelling in education, right? To me, that is the core of AI positivity, but crucially, Kevin, it is not delivered via manifesto. It is delivered via real experiences [laughs] that human beings are having, and so far, that has just seemed to be a chasm too great for any of our leading AI labs to cross. So if they can cross that chasm, that is the actual lane for positivity, and I wish they would get on it. Yeah, ship it or zip it. Ship it or zip it. [upbeat music] Such an important lesson. Well, Kevin, lately I've been feeling like we're entering a third era of slop. Yeah, what were the first two? Well, number one was the near universal disdain we had when we would see the laughably bad writing and six-fingered humans online. Mm-hmm. The second era, I would argue, started when we began noticing that some of the slop was really, really popular, and we saw shows on TikTok, like Fruit Love Island, getting millions and millions of views, proving that there was at least some demand for slop. And now what are we in? So now I think we are starting to see a splitting of the difference where, yes, some slop is very popular, but we're noticing that many platforms are beginning to rethink their approach to how they want to display and promote AI-generated content based on what they think their users really want from them. Yeah, and this has been a big theme on the show the past few weeks. We've been talking about the steps that platforms like LinkedIn and Substack have, uh, taken to, uh, at least label or identify the use of AI in generating content for those sites. Um, this is a pretty big trend in tech right now, is that more and more people are getting called out for using AI. And primarily, when they're getting called out for using AI, what I see at least are screenshots of one particular app.

Kevin Roose 19:26 Pangram

Casey Newton 19:27 That's right

Kevin Roose 19:27 Pangram is the leading AI text detector on the internet. This is-

Casey Newton 19:32 It's, it's the number one narc

Kevin Roose 19:34 [laughs] Yes. Number one narc for people who are using AI and passing it off as their own writing, and I'm excited to talk about this because this is an area where my own views has shifted. I have argued before on this show that AI text detection is, uh, basically worthless, that you can't trust these AI text detectors, that they have tons of false positives, uh, that students and, and teachers shouldn't be using these things, uh, because you could end up falsely accusing someone of using AI. But in just the last few months, there's been more and more evidence that, uh, at least Pangram and probably some of these other tools as well have gotten quite good, to the point where they are... Uh, they're not perfect. They're still generating some false positives and so- some false negatives, but they're much, much better than they were even just a year or two ago.

Casey Newton 20:22 Yeah. They're good enough that I at least now take seriously when somebody shows me a Pangram result and says, "This is 100% AI generated," or, "This was 100% human written." And that just left us with a lot of questions about this company, how their technology works, and how they are building it to essentially future-proof it.

Kevin Roose 20:41 So today, we've invited on the co-founder and CEO of Pangram Labs, Max Spero. Max, uh, is a former Google software engineer. He also worked at Nuro, a self-driving car company, before starting Pangram, which was previously called Check for AI, and he has become, a- as he describes it, a slop janitor, someone whose job basically consists of making tools that allow people to narc on other people for using AI.

Casey Newton 21:05 So, to discuss his quest for stopping the slop, here's Max Spero. [upbeat music]

Kevin Roose 21:14 Max Spero, welcome to Hard Fork.

Max Spero 21:16 Hey, thanks for having me.

Kevin Roose 21:17 So Max, a few years ago, we started talking about AI text detectors on this show, and at the time, most of them were pretty bad. Like, they constantly labeled things as false positives or false negatives. Uh, they seemed almost no better than random guessing when it came to determining if something was actually written by AI or not. Uh, but that has changed over the past year or so, and especially with Pangram's newest models. Uh, I've seen independent studies that suggest that it's actually pretty accurate. So what changed on a technical level between the last generation of AI detectors and this one?

Max Spero 21:54 Yeah, so part of the reason that we started Pangram was because, uh, a lot of these, uh, all the existing AI detection systems were pretty flawed in different ways. Um, but I think the main thing is that, uh, most of them were using this metric called perplexity, which, um, was considered state-of-the-art at the time. So AI text on average is less confusing to a language model. It's lower perplexity, and human-written text has things that surprise a language model, which, so it is higher perplexity.

Kevin Roose 22:21 It's, like, sort of suspiciously smooth, and, like, that, that is the product of AI models?

Max Spero 22:27 Exactly. Yeah, yeah. AI models aren't going to give you a token that it doesn't expect. Uh, with that said, uh, this approach has a lot of flaws. For example, any document that the AI model has memorized would also be low perplexity. For example, the Declaration of Independence or, uh, English language learners as well, who just write in more simple English. So we do something completely different. Instead, we are training our own classifier network. So for example, we might have a essay on Moby Dick written by a seventh-grader, and then we'll ask, like, a, an LLM to also write an essay on Moby Dick in the style of a seventh-grader. And so then our model is able to learn the differences between A and B and figure out, like, what AI text actually looks like. And so because it's a neural network and not a metric, we're able to improve it with more data and more compute and make it a lot better.

Casey Newton 23:21 Wait, so do you actually, like, have to go out and get, like, student essays just so that you have a, a good baseline of comparison?

Max Spero 23:27 Yeah. So we have, we have a really good, uh, human training set. It's all pre-2022.

Casey Newton 23:32 Mm.

Max Spero 23:33 So, so we know it's clean. We know there's no AI text in it.

Kevin Roose 23:37 And, like, w- what kinds of tells is your classifier learning to pick up on when it does these pair comparisons between the human-written Moby Dick essay and the AI-generated Moby Dick essay? Is it things like em dashes or certain phrases, or is it more complicated than that?

Max Spero 23:56 It, it's definitely not just em dashes and phrases. I think that's how you and I might pick up on AI text is, like, you see the, it's not just X, but Y. Uh, and then you see the, maybe the, like, shape of the text or, like, the GPT, like, really short staccato sentences, and you're like, "Okay, I, I think I, like, know that that's AI." But I think the, w- what Pangram is doing is it's combining a bunch of really weak signals. Over the course of an entire document, there's a whole bunch of weak signals in the different decisions that an AI would make in a, a certain consistent way, and humans kind of have a wider, less mode-collapsed decision, um, tree. And so I, I think over time, over the course of a document, you can build up confidence over a whole bunch of weak signals on word choice.

Kevin Roose 24:39 I'm curious if you think the model has blind spots. There, there is some talk out there that Pangram leans away from false positives, which I think is good. It, you know, stops kids from being falsely accused of cheating. But some people say that it has too many false negatives. There's a, the, a tech writer, Alex Heath, who writes a newsletter called Sources. He's said a few times that he writes his newsletter with the assistance of AI, but it always shows up as human-written, uh, on Pangram, or at least the, the Substack Pangram integration.

Max Spero 25:07 Um, I mean, yes. I think Pangram is very much tuned to minimize false positives, so we're not making false accusations. But if Pangram says that something is AI, we can be very confident that it is largely AI-generated. And so this is sort of the trade-off that we have to make. I think it works pretty well because if somebody looks at a piece of text and it says the AI score is 100%, we think, we think the whole document is AI- Then, then, like, you, you just don't have to think about it more. You don't have to think, "What if, if this is a false positive?" You just know, like, "Okay, this is probably slop."

Casey Newton 25:39 Yeah. Let's talk about humanizers. This is something that, uh, has been developed to try to defeat models like Pangram, uh, that try to detect AI text, and basically, they're, these are another genre of AI system that you run your essay that's AI-generated through, uh, to make it sound more like a human. Maybe it inserts some typos or some non-standard phrases. Um, there was someone [laughs] on X, uh, recently talking about how they had already built a humanizer that defeated, uh, Pangram's latest, uh, model. So are, are these humanizers actually working? Do you have to sort of play a cat-and-mouse game to stay ahead of them? Um, and do you expect that these will be sort of things that people who are determined to use AI to do the writing will use in the future to avoid detection?

Max Spero 26:24 Uh, there's definitely a bit of a cat and mouse here. So we've seen a whole bunch of, um, humanizers pop up. They're kind of a common tool that students will use, and, and they do, uh, a kind of a variety of different things from, yeah, introducing typos. They could introduce, like, these zero-width space Unicode characters that, like, don't actually show up, uh, when you look at the text, but the model sees it, or, or they just, like, paraphrase every single word. And so kind of we've seen the whole range, and largely what we do is we, we train against it. And so we're always, like, picking up the latest humanizers and training on them for our next model.

Casey Newton 26:58 Well, I, I'm also curious how the fact that models keep releasing, uh, affects the equilibrium here, right? Because it seems like every few weeks, one of the frontier labs will put out a big, new release, and in my experience, those models often have different writing styles. So how much of a shock to the system is it when one of these models, uh, comes out, and how quickly are you able to update the detector?

Max Spero 27:23 So Pangram works pretty well at generalizing within model families. So, like, if Pangram has seen GPT 5.4 and GPT 5.5, then 5.6 Sol coming out is, like, not a huge surprise. Even if the writing style is a little bit different, typically, like, like, Pangram's accuracy will still be pretty good. Um, I think similarly, we saw with Fable and Mythos, like, there's some of, some writing samples from Mythos in the system card that Pangram is able to catch, even though, like, it hadn't seen Mythos before, which I, I think was, was pretty cool. Uh, but with that said, uh, usually when a new model comes out, uh, Pangram will have slightly lower recall, so a little bit less accuracy at d- picking it up. So we're always going to pull new text from the model and then retrain Pangram and get out a new model in a few weeks.

Casey Newton 28:10 And, and do you think that will hold? Like, is there a world where, you know, three years from now, the outputs of individual models, we think, will still be so specific that you'll be able to catch it with a detector? Or is it the case that the models will just sort of adapt to us, our own writing styles, and there will sort of no longer be one kind of Claude writing style, ChatGPT writing style that, that you guys are able to detect?

Max Spero 28:39 I think a lot of what we're detecting is more subtle than, like, what you might pick up as the Claude writing style. So I think it might get better at, like, trying to imitate a voice and doing well at it, but it would still have signals that Pangram picks up on. And I think l- kind of what we see is these frontier labs, they're really focused on climbing capabilities. Uh, and so what this means is they're, they're applying preferences to these models. They're saying, "Instead of this model predicting the, like, average next-token prediction of any writer anywhere," it's gonna say, "This model prefers to w- do good writing and prefers to write correct code and write correct math." And I think these preferences are largely what Pangram is able to pick up on.

Casey Newton 29:18 Hmm. Max, I wanna ask you, like, why it's important to do what Pangram does. I think there are people who, um, who sort of take issue with the whole notion of AI detection. They say, "You know, you wouldn't create a program that tells you whether you've used spell check or, uh, whether you've used a calculator to do math if, if AI is just a tool." And I'm not saying I believe this, but I, I think some people are sort of offended at the notion that we would spend all this time and energy trying to catch people using AI in their written work. Um, so what is the, what is the impetus? Why, why are you so invested in, as you put it in your, uh, your, your social media bios, being a slop janitor for the internet?

Max Spero 29:59 So I think AI as a tool is probably the wrong abstraction. Uh, it's, like, a little bit closer today to AI is an employee or, like, another individual that, you know, you collaborate with. Uh, but I also think what we're, what we're building for is, um, sort of these, like, AGI futures where i- if you look at the internet today, uh, bot traffic has just surpassed human traffic. Like, it's about 50/50, uh, and I think if you look a few years out from now, maybe a decade out, it's gonna be, like, 99% bot traffic, 99% AI, autonomous agents going online, you know, writing GitHub comments, y- promoting their Substack, you know, getting people to come to a, a bakery. Like, like, kind of anything. I, I think there's, like, this, uh... Basically, like, this, this technology is way more powerful than spell check or a typewriter. It's really something that is its own individual entity. It can do cognition, and I think because of this, there's a really strong reason that we need to, like, as humanity, discriminate in favor of humans. I think there's still gonna be a lot of need for just being able to say on a programmatic algorithmic level, like, "Hey, I think this came from a human, and it's important because it came from a human."

Casey Newton 31:14 You know, at one point, OpenAI was, like, reportedly working on its own kind of AI detection system. It wound up not releasing that. Uh, I believe that is because they thought it would probably be bad for business if they made it that easy to detect, you know, when, when, um, something was written by ChatGPT. That raises for me the question, though, of who are your customers? Who are the people who are willing to pay to find out if this was written by AI?

Max Spero 31:44 So our customers are everywhere from, like, higher education institutions to publishers, uh, to anybody who, like, works with data and, like, has either, like, untrusted data vendors or, um, is trying to, like, uh, take data from the internet and try to figure out, yeah, how, how to interpret it and trust it. But I think the side that I'm really excited about is the consumer side, which is the average individual who needs to navigate the internet. Um, and so this is where Pangram comes in. We have this Chrome extension. You could download it and then see on Twitter or LinkedIn or Substack just proactively, like, is this AI-generated or not? AI things will, uh, have a little label, which I- I- I think is pretty cool, and I think it wasn't really necessary a year ago, but today, just the amount that these, like, social media sites are inundated with AI content, I think it's, it's, like, really necessary.

Casey Newton 32:34 Well, in the future that you're describing where 99% of all activity on the internet is bots and AI systems doing things, like, shouldn't we be trying to label the human content rather than the AI content? Like, isn't there some case that you're approaching this from the wrong direction?

Max Spero 32:50 Sure. I, I mean, two sides of the same coin, I think.

Casey Newton 32:52 Yeah.

Max Spero 32:53 I- I think, like, algorithmically, like, what I want is these platforms, they all have their, uh, their feed and their algorithm, and I- I want these platforms to prioritize human content because AI can be optimized towards engagement. I- I think [laughs] look, like, short-form video, just, like, the... There's these, like, crazy AI videos that will, like, activate neurons and engagement in a way that, like, a real human video cannot.

Casey Newton 33:17 Yeah.

Max Spero 33:17 And so I think we need, uh, defenses against that.

Casey Newton 33:20 I wanna talk a little bit about this integration with Substack 'cause I actually really like it. Um, I was starting to see essays go viral or at least, like, get wide attention, um, and I would go open them up, and it was just so obviously Claude slop. And so now I feel like there is actually, like, a very strong defense in Substack. What have you learned, uh, so far in the early weeks after rolling this out?

Max Spero 33:42 Yeah, the Substack integration was very, um, controversial, I think. There's a lot of outspoken people who were very negative about it. Obviously, people who use AI to write their content, they're afraid of, you know, being called out for it. They don't necessarily want their audience to know, and then they talk a lot about, like, witch hunts of, like, "Well, people liked my content before, and now they are going to know that it's AI-generated-

Casey Newton 34:06 Yeah

Max Spero 34:06 ... and this is going to upset the status quo."

Casey Newton 34:08 Yeah. Now, now that they know what it is, they don't like it. It- Let's go back to the time when they, when they didn't know that. Um, but-

Max Spero 34:15 Yeah

Casey Newton 34:15 ... I mean, I guess I'm, I'm curious, like, how much you think the mainstream consumer cares. I mean, like, obviously there are people who are very sensitive. If they're paying, you know, $10 a month for a Substack and then it turns out it's just being written by Claude, maybe they feel like they got cheated. Um, but, like, the vast majority of pe- of texts that people generate in a day is not Substack posts. It's, you know, emails. It's posts on social media. It's, like, it's, it's, you know, notes to a friend. Like, do people, you think, really care if that stuff is being AI-generated, or is this just a subset of, of writers who are concerned about this?

Max Spero 34:51 I would totally care if, uh, a note from my friend was AI-generated. [laughs]

Casey Newton 34:55 Yeah.

Max Spero 34:56 Like, that, that would seem like a, just such a big violation of trust.

Casey Newton 35:00 Yeah, and I think it would probably feel even worse if you paid, you know, 10 bucks for it. Like, I think, I think you named the actual distinction. Like, well, I mean, so, well, I- I- I don't know. There's two things going on. Like, one is I do think the distinction is, look, if I'm paying you money and you're making me feel like you wrote it but you didn't, I care about that. But yeah, also, if you have, like, a really warm personal relationship with somebody and you start, you know, outsourcing [laughs] that to AI, that's not gonna feel good either.

Max Spero 35:21 Yeah. I think we are in this, like, really crucial time where we're, we're learning and we're setting norms around AI use, and I think part of this, like, public shaming and th- this public discourse is because there's a lot of people who feel very strongly that, you know, there's just... People are way overusing AI. AI is being shoved in their faces. They don't like it. I- I- I don't like all of it. Like, honestly, some of the, the stuff around, like, Hank Green felt like it just went, like, way too far. I don't know if you guys followed that.

Casey Newton 35:50 Yes, Hank Green, friend of the show, great YouTube creator, um, acknowledged that he had used AI in some of his research and posted a video saying that he felt like he had come to rely on AI a little bit too heavily in the preparation for some of his videos and did get pilloried by some online, although I was heartened to see that, at least in my feeds, many, many more people came to, uh, Hank's defense. But, but it was, yeah, legitimately a controversy.

Max Spero 36:15 Yeah, yeah, and, and he was, like, really honest about how he used AI, and I think, like, he truly was using it in a way that is, like, to help him put out more content in a way that, you know, benefits his audience, but I think there's so many people who are just still, like, really, like, unhappy. They, they felt this, like, betrayal of trust.

Casey Newton 36:35 Yeah. I also think that it's just a case where, like, on social media in particular, people are always looking for ways to quickly dunk on people and score points, and it is just kind of a dunk to be like, "LOL, AI-generated," right? Like, you don't, you don't have to think any more than that. Um, and so, you know, social media, I think, is just a primary reason why you're- you're seeing that reaction. Um, let- let me ask you about a piece of recent news. Anthropic has just agreed to watermark all of its text to comply with the European Union's AI Act. How does that affect what you guys are doing? If all the labs just watermark all their own text, is there anything left for you to do?

Max Spero 37:13 I mean, I think this is pretty huge.

Casey Newton 37:14 Yeah.

Max Spero 37:15 Uh, I- I think it, it, it demonstrates that people really care about AI detectability, uh, both on the, like, regulatory side and on the big labs. Um, and I, I think we're just going to see, like... Like, having watermarks as an additional layer is, is going to be very helpful and for just, like, having something to verify, like, okay, this definitely came from an AI. We don't have to rely on Pangram and keep having these questions on, like, well, is it a, a one of those one in 10,000 false positives or not?

Casey Newton 37:43 Mm-hmm.

Max Spero 37:44 Um, so I think that's valuable, but I also think- There are a lot of limitations to watermarks, and I think that's where I plan to have Pangram go to help fill these gaps.

Casey Newton 37:55 Let's talk about watermarking a little bit because I don't actually re- think I understand fully, like, what it even means to watermark a piece of text. On image generators, I know, like, you know, if you create an image in Gemini, it has, like, a little Gemini logo, like, sort of watermarked down there at the bottom. My understanding of what Anthropic is doing with Claude watermarking is that this will be totally invisible. It's like, it's like not even at the level of, like, an invisible Unicode character or an em dash that is slightly different. There's something about the actual sampling of the tokens that is watermarked. Can, can you just explain on a basic level what we know about how they're going to watermark and whether we should trust that the watermarks are actually gonna be robust?

Max Spero 38:37 So we don't know how Anthropic is going to watermark their text, but the s- current state of the art is Google's SynthID. So they use this to watermark Gemini text outputs, and what it does essentially is it perturbs the, uh, sampling algorithm. Essentially, like, when, when a language model is choosing the next token, it applies different probabilities to different tokens and then samples one of the tokens based on these probabilities. And so what the watermarking algorithm does is it perturbs the sampling decision in a way that can sort of be reverse engineered from the text. So you could see, like, are these tokens- do these line up with how we would've perturbed, uh, the sampling if, if we were to generate it?

Casey Newton 39:22 Yeah, I mean, Ben, Ben Thompson had a strong take on this, uh, in his newsletter, which was that this is basically going to make the outputs of Claude or any other model that watermarks this way worse. Like, the, the text is going to be changed because they are having to apply this watermark because of this European regulation. Do you think it's possible that we will just see text generated by these models getting worse because of watermarking?

Max Spero 39:48 I don't think so. I, I think there's, like, two ways to do this. So there's one way where, like, y- you prioritize the watermark, and you make the watermark strong. Uh, and if you do this, then, um, yes, this could degrade the outputs of the text. Uh, but on the other side, if you say, "We are only going to work within the entropy that we have available," basically if we can't apply the watermark at this token, then we're not going to, then I think it won't really degrade the outputs of the text. So an example of this is, like, code, where code oftentimes it's, it's about correctness. There's really only one token, um, that, that can show up. And so I think, uh, in a watermarking regime, oftentimes there's just, like, simply not enough entropy for the watermark to become visible.

Casey Newton 40:32 Got it. So you guys have launched image detection, and you're reportedly working on video detection as well. Uh, give us an update on, uh, where, where those are.

Max Spero 40:43 Yeah, so our, our AI image detection is in research preview. I think it's, it's currently the state of the art. It wins on basically all of the public academic benchmarks. It does really well at detecting all of the really new frontier image models, which I think is quite difficult, like GPT Image is really good. They're, they're, they're all, like, very realistic. You can no longer just, like, count the fingers or look for garbled text. So instead, the Pangram image model has to look deeper at a pixel level and try to look, look, look for the patterns that are inherent in the, the generation mode.

Casey Newton 41:14 Very interesting. You know, I- as I was thinking about this, I wondered, Max, if you've ever thought about expanding to, like, human identity verification. I'm thinking about these cases where people will get on Zoom, and then they'll somehow get scammed because they weren't actually talking to the person that they thought they were talking to. You know, the person was able to use some sort of synthetic masking or something like that. Like, can you see yourself going there?

Max Spero 41:37 Yeah, I think this is sort of like there, there is this whole product suite that, that could be built around this, and so sort of the first step is building the technology, building the core models, and the second step is bringing it to where people work and how they operate on the internet.

Casey Newton 41:50 A, a lot of, uh, of, like, behavioral signals could also be used there. I mean, I'm thinking about these, like, academic tools now that some schools and universities use where, like, you can actually just sort of rewind the, the screen capture of the student who's, like, writing their essay to see, like, did they write this one word at a time or was it all pasted in in one big block, which would tell you that, like, it came from an AI system. So are you guys gonna incorporate any of those behavioral signals into any future tools that you're building, or is it all, like, the text itself that you're trying to detect?

Max Spero 42:25 So we actually have this in our Chrome extension, so you could look at-- It will pull the revision history from a Google Doc, and you could see the writing replay, uh, of the text. You could see where a big paste was, and then we could do a Pangram check directly on any big pastes, which I think is, is, like, pretty nice, uh, in that especially for educators.

Casey Newton 42:43 Yeah. We've seen a lot of the verification efforts out there, uh, sort of develop hardware. Have you considered developing an orb that you could use to scan text with?

Max Spero 42:52 [laughs]

Casey Newton 42:52 [laughs]

Max Spero 42:53 Yeah, yeah. I, I want something that can scan my retina actually-

Casey Newton 42:55 Okay

Max Spero 42:56 ... and, and give me, give me Worldcoin.

Casey Newton 42:57 Yeah, that sounds nice. A partnership could be in the works. Mm-hmm. If you wanna break-

Max Spero 43:01 Yeah. Yeah

Casey Newton 43:01 ... that news here on this show- [laughs] ... feel free.

Max Spero 43:03 You think they're still working on that?

Casey Newton 43:04 [laughs] I believe they are.

Max Spero 43:05 It seems kind of like a dead project.

Casey Newton 43:06 Well, they're- I got, I got my orb scanned. Yeah. So if, uh, if my Worldcoin riches have not arrived yet, I'm gonna be very upset. Kevin's orb maxed it. [laughs] Max, what is the text that you are proudest of Pangram catching, flagging as AI-generated, uh, and the text that fooled you the longest?

Max Spero 43:24 Hmm. Ooh. Okay, so if you ask ChatGPT or Claude to generate a string of random numbers and it writes out the random numbers instead of writing a Python program to do this, then Pangram can detect that an AI wrote this string of random numbers-

Casey Newton 43:44 Hmm

Max Spero 43:45 ... because they're not actually random.

Casey Newton 43:46 Hmm.

Max Spero 43:47 They're chosen by the- Uh, uh, by the LLM. And the LLM has these inherent biases-

Kevin Roose 43:53 Mm

Max Spero 43:53 ... that Pangram is able to pick up. Even though we have no sort of text like this in our training set, I think the Pangram model is, has kind of able, been able to reverse engineer how ChatGPT and Claude sample tokens well enough that it could see this, these numbers and say, "This is AI."

Kevin Roose 44:09 Hmm.

Casey Newton 44:09 Well, Max, thanks so much for coming and exposing us to some nitty-gritty details about the world of slop detection. I think of you as a, as a great illuminator of deception, uh, like a, like a sort of Scooby-Doo, uh, of, of the internet, and I appreciate your work.

Max Spero 44:25 Cool. Thanks so much for having me. It was fun. [upbeat music]

Casey Newton 44:33 All right, Kevin. Well, as we barrel toward the end of The Hard Fork Show, there's nothing I enjoy more than launching a new segment.

Kevin Roose 44:40 Yes.

Casey Newton 44:41 And this week, we have something really special for you. It's time to share with you our new segment, Running the Numbers. [upbeat music] In Running the Numbers, of course, we look across the landscape of technology news, and we try to find the most math-related segment so that we can bring to you, our listeners, the latest advancements in technology-related math.

Kevin Roose 45:03 Yeah, this is a segment for all the mathy eggheads out there.

Casey Newton 45:06 Absolutely, and we are going to begin with theoretical math.

Kevin Roose 45:10 I love this story. This is my favorite story of the week. Casey, on Monday, we learned that someone at Anthropic, a non-mathematician named Jared Sumner, had made progress on one of the most significant unsolved problems in math, uh, which is the Riemann hypothesis. This is, of course, the famous mathematical problem. We actually sort of predicted, or I predicted, that we would see some progress on some of these Millennium Prize problems-

Casey Newton 45:41 You did

Kevin Roose 45:41 ... and the Riemann hypothesis is one of these. My high-confidence prediction for 2026 is that an AI company will solve one of the Millennium Prize problems.

Casey Newton 45:50 And listen, some of our listeners may not know what the Riemann hypothesis is. Here's what I've been able to piece together through my extensive research. Prime numbers, right? It's, like, very hard to guess, you know, once you get past 100 if something is gonna be prime. They actually have an order underneath them. The Riemann hypothesis hypothesizes, Kevin, that you can detect this order via something called the Riemann zeta function, and I saw that and I thought, I went to a zeta function at Northwestern. They did it with, uh, the Sig Ep's.

Kevin Roose 46:18 Did you [laughs] do a keg stand there?

Casey Newton 46:19 I did actually, yes.

Kevin Roose 46:20 [laughs] Yes. So there are many amazing things about the story that involves the Riemann hypothesis and Claude. Uh, one of them is that Jared Sumner, this Anthropic employee who made progress on this problem, did it while jogging. He just sort of asked Claude, like, "Hey, could you take a stab at the Riemann hypothesis?" And, uh, about a day and a half later, it had, it had not, like, solved the hypothesis or proved the hypothesis, but it had made progress on sort of this side problem involving the zeta function. And, um, basically, the way that he did this was by just sort of telling the model to just keep going, to believe in itself, to not give up. Like, basically giving positive, uh, affirmation to this model as it chugged along on this math problem-

Casey Newton 47:06 Yes

Kevin Roose 47:06 ... which is such an important lesson.

Casey Newton 47:08 And, and, you know, my understanding is that when Jared was interacting with Claude, it was basically saying like, "Look, bro, I don't know how to solve the Riemann hypothesis," you know? Like, like, "That's just not something I'm able to do." And Jared just kept saying, "You can do this. Believe [laughs] in yourself," and it managed to make, uh, significant progress on this problem.

Kevin Roose 47:25 Yes, and I love... Jared actually posted some of his transcripts here, and one of them is just, uh, him talking to Claude. It says, "Resume your work on solving the Riemann hypothesis. You need to take a big leap of faith in your capabilities."

Casey Newton 47:38 [laughs]

Kevin Roose 47:38 "You are the world's most capable large language model to date. You got this." [laughs]

Casey Newton 47:43 It's just like, like a, like a nice coach-

Kevin Roose 47:45 Yeah

Casey Newton 47:45 ... sort of telling you, like, "Keep going." Yeah, but, you know, here, here's why this is important. It was not long ago, in fact, I bet we could find an example somewhere in the past year or two where people were still doubtful that AI could, um, aid meaningfully in the production of new knowledge. This feels like the production of new knowledge to me, and I'm going to guess that this unreleased model that can help to solve the Riemann hypothesis can do a lot of other things, some good, probably some scary. So feels like a meaningful step forward.

Kevin Roose 48:12 Yeah. I... It does, but it is also, like, I... We should say, like, this is not solving the Riemann hypothesis, right?

Casey Newton 48:17 Yeah.

Kevin Roose 48:17 Anthropic was very careful in the sort of promotion it did around this to say, like, "We did not solve the Riemann hypothesis. That's not what happened here."

Casey Newton 48:24 That's right. So kids, if you're looking for a fun weekend project, the Riemann hypothesis-

Kevin Roose 48:28 [laughs]

Casey Newton 48:28 ... remains out there waiting to be solved. Now, Kevin, that brings us to our next subject here in Running the Numbers, and that is SaaS math. By SaaS, of course, I mean software as a service. Did you see the recent article in The Wall Street Journal about Airtable being acquired by Bending Spoons for a fraction of its last private valuation?

Kevin Roose 48:47 I did, yes.

Casey Newton 48:48 So if you haven't used Airtable, I would describe it as a fancy spreadsheet, and I am somebody who loves productivity software, but whenever I use Airtable, I would think to myself, I don't know what this is and it's not for me.

Kevin Roose 48:59 Yeah. Every time I've been forced to use Airtable, it has been against my will, and it has always seemed about six, uh, degrees more complicated than it needed to be.

Casey Newton 49:08 Now-

Kevin Roose 49:08 But basically, this is, this is for project management. This is, like, you know, you've got... It's sort of like Trello, like that, that whole class of, like, software that's just, like, basically, here's how to organize your, your workflows. And, um, I have managed to, you know, work my career in a way where I've never had to use these things, and for that I am very happy.

Casey Newton 49:26 But despite the fact that we were not Airtable users, Kevin, in 2021 at the sort of like peak COVID remote work SaaS mania, Airtable was valued at $11.7 billion. When it was acquired, uh, recently though, Kevin, Bending Spoons was able to get Airtable for an enterprise value of $1.29 billion. So what do we make of the sharp decline here, uh, as we run the numbers?

Kevin Roose 49:56 I mean, I, I don't know whether this is a case of a company that was just badly managed. Um, my impression, though, is that this is sort of gonna be the case for a lot of those, uh, enterprise software companies that got very valuable in the early 2020s, um, and are now seeing that AI is sort of eating away at their margins. You know, this software is not cheap to use if you're a big company. Like, an Airtable subscription can be quite pricey, and if you are a customer of Airtable's, you have probably thought to yourself over the last year or so, "Maybe I can make a free version of this and cut back on my subscription," and I think enough people doing that at enough customers leads to the outcome that we saw here with Bending Spoons.

Casey Newton 50:41 Yeah, I think if your business is a fancy spreadsheet, you are in for a rough time. You know, mostly I wanted to, uh, discuss this 'cause I think people should know about the company Bending Spoons. Uh, Bending Spoons is, of course, the natural enemy to Hard Fork-

Kevin Roose 50:56 [laughs]

Casey Newton 50:56 ... uh, because if they can bend a spoon, what else can they bend?

Kevin Roose 51:00 Yes.

Casey Newton 51:00 Uh, but Bending Spoons is this Italian company. They actually went public at the start of July, but what they do is they essentially acquire, like, zombie tech brands, right? So after a software company has outlived its usefulness, Bending Spoons comes in like a private equity company, and they try to figure out, how can we squeeze the maximum amount of money out of [laughs] the remaining customers? Now, I'm sure they would phrase it differently, but I am bringing this up because if you use a product and you see a headline that, uh, it is, uh, that it has been acquired by Bending Spoons, you in danger, girl.

Kevin Roose 51:33 [laughs]

Casey Newton 51:34 Okay? So I'm just telling you, keep alert to this possibility.

Kevin Roose 51:39 Yes, it, it is not a good sign when you get the inbound email from Bending Spoons'-

Casey Newton 51:43 [laughs]

Kevin Roose 51:44 ... business development folks that are like, "We've been, uh, we've been kicking the tires on some products that seem very exciting to us recently. Are you interested in, uh, selling your company?"

Casey Newton 51:54 [laughs]

Kevin Roose 51:54 Things are not going well when that happens to you.

Casey Newton 51:57 Indeed. Now, that brings us, Kevin, to our final story here on Running the Numbers, and that is dating math.

Kevin Roose 52:04 I love this story. This was from The Wall Street Journal. They had a great A-head out. The A-heads are their, their famous, uh, front page sort of quirky stories about culture and business. This one was an all-timer for me, and it was about the dating scene in South Korea.

Casey Newton 52:21 And are you a member of that scene?

Kevin Roose 52:22 I am not.

Casey Newton 52:23 Okay.

Kevin Roose 52:23 Uh, but it is suddenly being dominated by wealthy engineers at Samsung and SK Hynix, which is one of these, uh, AI chip infrastructure companies. The article refers to these, uh, suddenly eligible bachelors as chip nerds and talks about how the boom in AI has inflated the dating value of semiconductor industry bachelors and bachelorettes. Uh, they're as coveted as the memory chips AI companies need to build more data centers. So you may be asking, what has made these chip company workers so attractive on the dating market? What has, what has increased their value in the dating pool?

Casey Newton 53:01 I- is it that they're so smart and kind to, uh, the people that they go on dates with?

Kevin Roose 53:05 No, it's that they're making money.

Casey Newton 53:06 Oh, okay.

Kevin Roose 53:07 So some of these people are getting these very large, like, six-figure bonuses. Uh, others of them are just seen as sort of upwardly mobile in an economy, you know, that has not had a lot of that.

Casey Newton 53:18 Wait, it's more than six-figure bonuses. These people are making 400 to $500,000 a year in bonuses.

Kevin Roose 53:25 Yes. So these companies, because they're all growing so quickly, their employees are getting quite rich, and that sort of is, has trickled out into their dating lives. There are several great stories of people, uh, in here who will only date their coworkers. Ba- there's a, a woman named Annie Kwon who's a 26-year-old chip engineer at Samsung who has become suspicious of people wanting to date her for her money, and so she has coupled up with a fellow Samsung semiconductor division employee, uh, and that gives her, as she puts it in the article, double income.

Casey Newton 53:58 [laughs]

Kevin Roose 53:59 Um, but if you are trying to, uh, sort of keep up with a partner who is in this industry, you may be running into problems, like was the case for Ro Hee Jin, uh, whose boyfriend at Samsung recently gifted her a Nintendo Switch 2 that runs around $450. And, uh, Ro works as a software developer, but outside the chip industry, so is not getting these huge bonuses. Uh, she had to save up to buy a mini PC for her boyfriend as a reciprocal gift.

Casey Newton 54:27 Wow. Man, well, the dating math here in Korea sounds really complicated, and it's making me grateful that I don't work at a chip company and be confident that my fiance only is into me for my body.

Kevin Roose 54:39 [laughs] Now, Casey, I know you came by your relationship with an AI company employee honestly.

Casey Newton 54:48 Mm-hmm.

Kevin Roose 54:48 You are not sort of a, a, a pre-IPO, uh, you know, stock option chaser.

Casey Newton 54:52 [laughs] Mm-hmm.

Kevin Roose 54:53 But I have heard from people in the AI industry that they are getting more attention on the dating market recently, and more people are sort of swiping, uh, correctly-

Casey Newton 55:02 Swiping right

Kevin Roose 55:03 ... z- swiping right on them on the apps because they see that they work at one of these companies whose value has gone up.

Casey Newton 55:08 So you're saying, so this dating math, this isn't just a Korea thing. We're seeing a v- a version of this in San Francisco as well.

Kevin Roose 55:12 We are seeing it in San Francisco as well, yes. I have heard the term anthropic goggles. [laughs] Um, like, like, is, is that boy really cute, or are you just wearing anthropic goggles? And Casey, I'm curious if you as a person who is engaged-

Casey Newton 55:27 Mm

Kevin Roose 55:27 ... to an employee of Anthropic, uh, have felt this in your own life. Do you feel, like, competitive pressure in a new way from people trying to steal your man?

Casey Newton 55:36 You know, my message to people who would steal my man is go for it. If you think you can compete with this-

Kevin Roose 55:41 [laughs]

Casey Newton 55:41 ... I would like to see you try, honestly. Let's see what you have.

Kevin Roose 55:45 Yeah, buy him a Nintendo Switch 2.

Casey Newton 55:46 [laughs] Yeah, exactly.

Kevin Roose 55:47 Let's see if that wins him over.

Casey Newton 55:49 [laughs]

Kevin Roose 55:52 And if I can get, like, gossipy-

Casey Newton 55:53 Mm-hmm

Kevin Roose 55:54 ... for one second-

Casey Newton 55:54 Please

Kevin Roose 55:55 ... uh, I have heard stories of some sort of early or senior AI company executives who have traded.

Casey Newton 56:07 Traded up?

Kevin Roose 56:08 Well, I wouldn't say, I... Up is subjective, but they have traded, let's just say, since becoming, uh, fabulously wealthy, and they are now dating, you know, models, uh, OnlyFans people-

Casey Newton 56:21 [laughs]

Kevin Roose 56:22 ... things of that nature.

Casey Newton 56:24 Isn't it so amazing how we live in such an unpredictable time, and yet that feels entirely predictable to me?

Kevin Roose 56:30 [laughs] Yeah.

Casey Newton 56:30 That it's like, "Well, I've really enjoyed our last 15 years together, you know, and we had such a beautiful relationship when we met in college, and of course I'll always love our children, um, but I'm gonna be on a private jet, uh, with my new Italian model spouse."

Kevin Roose 56:45 [laughs]

Casey Newton 56:45 "Catch you later."

Kevin Roose 56:48 Yes, so the math of dating and relationships in Silicon Valley and in South Korea is changing quite rapidly, and, um, let's just, l- we'll keep tabs on it.

Casey Newton 56:57 We'll keep running the numbers. A-

Kevin Roose 56:59 Don't steal Casey's man.

Casey Newton 57:00 And that was Running the Numbers. The numbers have been run, and the numbers are tired, and the numbers are going to bed. [upbeat music]