Can We Detect a Deepfake?

1. 0:00 – 0:49

   The Rise of Deepfakes

   1. VBVijay Balasubramaniyan0:00

      There has been a 1400% increase in the amount of deepfakes we've seen this year in the first six months compared to all of last year.

   2. MCMartin Casado0:09

      So my dad jumps on the line and he's like, "I just talked to you. You were in prison, and I'm leaving to go bring $10,000 of bail money to you." I'm like, "What are you talking about?"

   3. VBVijay Balasubramaniyan0:21

      At the end of last year, there were 120 tools with which you can clone someone's voice, and by March of this year it's become 350. Being able to identify what is real is going to become really important, especially because now you can do all of these things at scale. [upbeat music]

2. 0:49 – 2:23

   The Evolution of Fake Media

   1. VBVijay Balasubramaniyan0:50

      We've been doing deepfake detection for, like, now seven years. Even before that, you have people manipulating audio and people manipulating video, and you saw that Nancy Pelosi slurring in her speech.

   2. SPSpeaker1:03

      ... in the Rose Garden with all this, um, sure sort of visual

   3. VBVijay Balasubramaniyan1:09

      All they did was slow down the audio and, you know, it, it, it, it wasn't a deepfake. It was actually a cheap fake, right? And so what changed is the ability to use, uh, what are known as generative adversarial networks to constantly improve, uh, things like voice cloning or video cloning or, uh, essentially try to get the likeness of a person really close. So it's essentially two systems competing against each other, and the objective function is I'm gonna get really close to Martin's voice and Martin's face, and then the other system is trying to figure out, okay, what are the anomalies? How do I m- uh, you know, can I still detect that it's a, a, a machine as opposed to a human? So it's almost like a reverse Turing test, and so what ended up happening is once you start creating these GANs, which are used in a lot of these spaces when you run them across multiple iterations, the system becomes really, really good 'cause you train a deep learning neural network, and that's where the deepfake comes from. And they became so good that lots of people have extreme difficulty differentiating between what is human and what is machine.

3. 2:23 – 7:34

   The Technology Behind Deepfakes

   1. MCMartin Casado2:23

      Deepfakes are more talked about now than they were in the past, right?

   2. VBVijay Balasubramaniyan2:27

      Yeah.

   3. MCMartin Casado2:28

      And so clearly this seems to have coincided with the generative AI wave, right?

   4. VBVijay Balasubramaniyan2:34

      Yeah.

   5. MCMartin Casado2:34

      And so do you think it's fair to say that there's a new type of deepfake that is drafted on the generative AI wave?

   6. VBVijay Balasubramaniyan2:42

      Yeah. You know, generative AI has allowed for combinations of wonderful things, but when we started, there was just one tool, uh, that, you know, could clone your voice, right? Like, it was called Lyrebird. Incredible tool. Uh, it was used for lots of great applications. Uh, at the end of last year, there were 120 tools with which you can clone someone's voice, and by March of this year it's become 350. Uh, and there's a lot of open source tools that you can use to essentially mimic someone's voice or to mimic someone's likeness, and that's the ease with which this has happened. Uh, uh, essentially the cost of doing this has become close to zero because all it requires for me to clone your voice, Martin, now requires about three to five seconds of your audio, and if I want a really high quality deepfake, it requires about 15 seconds of audio. Compare this to before the generative AI boom where John Legend wanted to become the voice of Google Home, and he spent, like, close to 20 hours recording him saying a whole bunch of things so that Google Home could say, "In San Francisco, your, the weather is 37 degrees," or whatever, right?

   7. MCMartin Casado4:01

      Yeah, yeah, yeah.

   8. VBVijay Balasubramaniyan4:01

      Like, so the fact is that he had to go into a studio, spend 20 odd hours recording his voice in order for you to do that compared to 15 seconds and 300 different tools available to do it.

   9. MCMartin Casado4:16

      I don't know if you remember this, Vijay, but, uh [laughs] this wasn't too long ago when I was, I was in Japan and, um, you know, I got this call from my parents, which I never do, and my mom's like, "Where are you right now?" And I'm like, "I'm in Japan." And my mom's like, "No, you're not." And I'm like, "Yes, I am." She says, "Hold on, let me get your father." So my dad jumps on the line and he's like, "Where are you?" "I'm in Japan." He's like, "I just talked to you. You were in prison, and I'm leaving to go bring $10,000 of bail money to you."

   10. VBVijay Balasubramaniyan4:46

       Oh, God.

   11. MCMartin Casado4:46

       I'm like, "What are you talking about?" And he's like, "Listen, I... Someone called and said that you had a car accident and, you know, you were a bit muffled because, you know, you were, um, you know, hurt, uh, and that I needed to bring cash to a certain area and, like, your mom just thought to call you while I was heading out the door," right?

   12. VBVijay Balasubramaniyan5:08

       Mm-hmm.

   13. MCMartin Casado5:08

       So of course we called the police after this and they said this is a well-known scam that's been going around for a very long time, and it's probably just, you know, someone that tried to sound like you and muffling their voice, right? And so it seems that calling somebody and obfuscating the voice to trick people has been around for a very long time. So, so maybe just from your perspective, like, do we need a new term for, like, these generative AI fakes because they're somehow fundamentally different, or d- is this just kind of more of the same and we shouldn't really worry too much about it because we've been dealing with it for a long time?

   14. VBVijay Balasubramaniyan5:46

       Yeah. So it's interesting it happened to you in Japan, man, because the origin of that scam, this was, like, close to, uh, eight, nine years back when I was talking about voice fraud. The Japanese audience talked to me about ori ori sagi, which is help me grandma. So it's exactly that.

   15. MCMartin Casado6:05

       It's that thing. Wow.

   16. VBVijay Balasubramaniyan6:06

       But if you... At that point in time, it had be- it had started costing Japan close to half a billion dollars in people-

   17. MCMartin Casado6:13

       Wow

   18. VBVijay Balasubramaniyan6:13

       ... losing their life savings to this scam. What's changed is the scale And the ability to actually mimic your voice. The fact is that now you have so many tools that anyone can do it super easily. Two, before if you had some sort of an accent and things like that, they couldn't quite mimic your, uh, your real voice. But now, because it's 15 seconds, your grandson could have a 15-second TikTok video, and that's all that's required. Not even 15 seconds. With five seconds and if, depending upon the demographic, you can get a pretty good clone. So what's changed is the ability to scale this. And then these fraudsters are combining, uh, these systems, these text-to-speech systems with LLM models. So now you have a system that you're saying, okay, when the person says something, give me s- uh, respond back in a particular way crafted by the LLM. And here is the crazy thing, right? Like, in LLMs, hallucination is a problem, so the fact that you're making shit up is a bad idea.

   19. MCMartin Casado7:19

       Yeah, yeah.

   20. VBVijay Balasubramaniyan7:19

       But if you have to make shit up to convince someone-

   21. MCMartin Casado7:22

       [laughs]

   22. VBVijay Balasubramaniyan7:22

       The LLM is created perfect.

   23. MCMartin Casado7:24

       That's right. Yeah, yeah, yeah.

   24. VBVijay Balasubramaniyan7:25

       Yeah, and it's crazy.

   25. MCMartin Casado7:26

       Yeah, yeah.

   26. VBVijay Balasubramaniyan7:26

       We see fraud where the LLM is coming up with crazy, crazy [laughs] ways-

   27. MCMartin Casado7:32

       Wow

   28. VBVijay Balasubramaniyan7:32

       ... to convince you that something

4. 7:34 – 9:50

   AI and Deepfakes in Elections

   1. VBVijay Balasubramaniyan7:34

      bad is happening.

   2. MCMartin Casado7:35

      You probably are the world's expert on voice fraud. You've probably seen more types of voice fraud than, you know, any single person on the planet. [laughs] So I'm just wondering if you could-- Like, we know of the ojioji sagi, which is basically what I got hit with. Can you maybe talk through some other kind of use cases for or uses of deepfakes that are prevalent today?

   3. VBVijay Balasubramaniyan7:55

      Deepfakes right now you can see, right? In the political spectrum, they're there, right? Like, so election misinformation with, uh, President Biden's campaign happened. We were the ones who caught it and identified it and things like that.

   4. MCMartin Casado8:07

      What was the, the specifics? Are, are you allowed to talk about that?

   5. VBVijay Balasubramaniyan8:09

      Yeah. Yeah. No, no, for sure, right? So what happened is early on this year-- And, you know, if you think about deepfakes, they affect three big areas: commerce, uh, media, uh, and communication, right? And so this is n- news media, social media. So what happened is, uh, at the beginning of an election year, you had the first case of election interference with everyone during the Republican pri- primary in New Hampshire got a phone call that said :

   6. SPSpeaker8:38

      "What a bunch of malarkey. We know the value of voting Democratic when our votes count. It's important that you save your vote for the November election."

   7. VBVijay Balasubramaniyan8:47

      And this was made in the voice of the president of the free world, right? President Biden, right? Like, that's the craziness.

   8. MCMartin Casado8:54

      Wow.

   9. VBVijay Balasubramaniyan8:54

      They went for the highest profile target. That came out and, you know, and people were like, "Okay, is this really President Biden?" So not only did we come in and say this was a deepfake, but we identified the-- We have something called source tracing, which tells us which AI application was used to create this deepfake. So we identified the deepfake, and then we worked with that AI application. They're an incredible company. We worked with them, and they immediately found the person who used that script and shut them down so they couldn't create any other problem. And then the, later on, regulation kicked in, and they find the telco providers who distributed these calls. They find the political analyst who, uh, i- intentionally created these deepfakes. But that was, you know, the first case of, you know, political, uh, misinformation. You see this a lot right now.

   10. MCMartin Casado9:45

       Was that this year?

   11. VBVijay Balasubramaniyan9:47

       Yeah. It was this year. It was in January of this year.

   12. MCMartin Casado9:49

       That's amazing.

5. 9:50 – 11:24

   Statistics in Commerce and Banking Deepfakes

   1. VBVijay Balasubramaniyan9:50

      Yeah.

   2. MCMartin Casado9:50

      Okay. We've got, we've got, uh, politics. We've got bilking old people.

   3. VBVijay Balasubramaniyan9:54

      Yeah.

   4. MCMartin Casado9:54

      What are, what are-- Maybe one-- Maybe one more good anecdote before we get into, you know, whether we can detect these things.

   5. VBVijay Balasubramaniyan10:00

      The one thing that's really close home is in commerce, right? Like you, uh, financial institutions. Generative AI came out, uh, in 2022. In 2023, we were seeing essentially one deepfake a month in some customer, right? So it was just one deepfake a month, and some customer would face it. It wasn't a widespread problem. But this year, we've now seen one deepfake per customer per day. So it has rapidly exploded, and we have certain customers, like really big banks, who are getting a deepfake every three hours. Like, it's insane the speed with-- So there, there has been a fourteen hundred percent increase in the amount of deepfakes we've seen this year in the first six months compared to all of last year, and the year is not even over.

   6. MCMartin Casado10:53

      Can you talk to, like, whether these things are detectable at all? Like, is, is this, you know, [laughs] is this the beginning of the end? Or where are we?

   7. VBVijay Balasubramaniyan11:01

      Uh, Martin, you've, you've lived through many such cycles where initially it feels like the sky is falling. Uh, you know, online fraud, email spam, there's a whole bunch of them.

   8. MCMartin Casado11:12

      Yeah.

   9. VBVijay Balasubramaniyan11:12

      Uh, but the situation is the same. They're completely detectable. Right now, we're detecting them with ninety-nine percent detection rate with a one percent false positive rate, so extremely high accuracy on being able to

6. 11:24 – 13:09

   Detection Challenges: Can We Spot a Deepfake?

   1. VBVijay Balasubramaniyan11:24

      detect them.

   2. MCMartin Casado11:24

      Just to put this in context-

   3. VBVijay Balasubramaniyan11:25

      Yeah

   4. MCMartin Casado11:25

      ... what are numbers for, like, detecting, like, identifying voice? Not, not fraud, just, like, whether it's my voice.

   5. VBVijay Balasubramaniyan11:32

      So it's roughly about one in every hundred thousand to one in every million, right? Like, that's-

   6. MCMartin Casado11:36

      I see. I see

   7. VBVijay Balasubramaniyan11:37

      ... uh, that's the ratio. It's, it's a, it's h- much higher precision for sure, uh, and mus- much higher specificity. But yeah. The deepfakes you're, you're detecting with a ninety-nine percent accuracy. Because when you think about even something like voice, you have eight thousand samples of your voice every single second, even in the lowest fidelity channel, which is the contact center. And so you can actually see how the voice changes over time eight thousand times a second. And what we find is these deepfake systems, either on the frequency domain, so spectrally, or on the time domain, make mistakes, and they make a lot of mistakes. And the reason they make mistakes and still it's very clear is because think about it. Your human ear Can't look at anomalies 8,000 times a second. If it did, you'd go mad, right? Like, you'd have some serious problems. So that's the reason you don't-- I mean, like, it's beautiful to your ear. You think it's Martin speaking on the other end, but that's where you can use good AI, which can actually look at things 8,000 times a second. Or in this, like, when we're t- doing most online conferencing, like this podcast, it's usually 16,000. So then you have 16,000 samples of your voice. And if you're doing music, you have 44,000 samples of the musician's voice every single second. So there's so much data and so many anomalies that you can actually detect these pretty comfortably.

7. 13:09 – 15:40

   Watermarking and Cryptography

   1. MCMartin Casado13:10

      I, I see a lot of proposals, particularly from policy circles-

   2. VBVijay Balasubramaniyan13:14

      Yeah

   3. MCMartin Casado13:14

      ... of, um, using things like watermarking or cryptography, you know, which has always seemed kind of a strange idea to me 'cause you're kind of asking criminals to comply by something or, you know, uh, uh, some [chuckles] ... You know. So I don't know. I, like, uh, how, how do you, how do you view more active measures to, like, self-identify either legit or illegitimate traffic?

   4. VBVijay Balasubramaniyan13:42

      Yeah, see, y- this is why you're in security, Martin. Almost immediately you realize that most a- attackers will not comply to you putting in a watermark. But even without putting in a watermark, right? Like, even if you didn't have an active adversary, like the President Biden robocall that I referenced before, when it finally showed up, the system that actually generated it had a watermark in it. But when they tested it aga- against that watermark, they only were able to extract two percent.

   5. MCMartin Casado14:14

      Oh, interesting. So you mean the original Biden call had-

   6. VBVijay Balasubramaniyan14:17

      A watermark because it was generated by an AI app that included a watermark.

   7. MCMartin Casado14:22

      And then they k- and then they copied [laughing] ... That's great.

   8. VBVijay Balasubramaniyan14:25

      And 98% of that watermark went away, largely because when you take that audio, play it across air, play it across telephony channels, they're bits and bytes. They get stripped away. And so once they get stripped away, and audio is a very sparse channel, so even if you add it over and over again, it's, it's not possible to do it. So these watermarking techniques, I mean, they're a great technique. You always think about defense in depth where they're present. Y- you will be able to identify a whole lot more genuine stuff with as a result of these watermarks, but attackers are not going to comply it. When you get videos, like, you know, we are now working with news media organizations, and 90% of the videos and audios they get from, for example, the Israel-Hamas war are fake. So we are-

   9. MCMartin Casado15:16

      How many? How many?

   10. VBVijay Balasubramaniyan15:17

       90% of them are fake.

   11. MCMartin Casado15:18

       What?

   12. VBVijay Balasubramaniyan15:19

       Yeah, like they're all-

   13. MCMartin Casado15:20

       I guess I shouldn't be so surprised, but I've [chuckles] ...

   14. VBVijay Balasubramaniyan15:22

       Yeah. They're all made up. They're a different war. Th- some of them are cheap fakes, some of them are actually deepfakes, some of them are, uh, kludged together stuff. Uh, and so, uh, being able to identify what is real is going to become really important, especially because now

8. 15:40 – 19:15

   The Economics of Deepfake Creation and Detection

   1. VBVijay Balasubramaniyan15:40

      you can do all of these things at scale.

   2. MCMartin Casado15:42

      Can you draw out how the maturation in AI technology impacts this? Because clearly something happened in the last year to make this economic for attackers, which we're seeing a rise, and clearly it's gonna keep getting better.

   3. VBVijay Balasubramaniyan15:58

      Yeah, you know, so one of the things that we, uh, uh, uh, talk about is any deepfake system should have strong resilience built in it. So it should not just be good about detecting deepfakes right now. It should be able to detect what we call zero day deepfakes. A new system gets created, how do you detect that deepfake? And there is essentially the mental model is the following. One, deepfake architectures are not simple monolithic systems. They have, like, several components within them, and what ends up happening is each of these components tend to leave behind artifacts. We call this a fake print. So they all leave behind things that they do poorly, right? And so when you actually create a new system, you often find they've pulled together pieces of other systems, and those leave behind their older fake prints. And so you can actually detect newer systems because they usually only, uh, improvise on one component. The second is we actually run GANs. So you get these GANs to compete. Like, we create our own deepfake detection system. Now we say, "How do you beat that?" And we have multiple iterations of them running, and we're constantly running them.

   4. MCMartin Casado17:09

      Oh, wait, wait, wait. I see. Sorry, I just wanna make sure that I understand here.

   5. VBVijay Balasubramaniyan17:11

      Yep, please.

   6. MCMartin Casado17:12

      So, so you're creating your own deepfake system using the approach you talked about before, which is these general adversarial networks. So then you can, then you can create a good deepfake, and then you can create a detection for that. Is that right?

   7. VBVijay Balasubramaniyan17:24

      Exactly.

   8. MCMartin Casado17:25

      I see.

   9. VBVijay Balasubramaniyan17:25

      And then you beat that detection system, and you run that iteration, iteration, iteration. And then what you find is actually something really interesting, which is if a deepfake system has to serve two masters, that is, one, I need to make the speech legible and sound as much like Martin, and two, I need to, uh, deceive a deepfake detection system, those two objective functions start diverging. So for example, I could start adding noise, and noise is a great way to avoid you from understanding my limitations. But if I start adding too much noise, it stops. I can't hear it. So for example, you know, we were called into one of these deepfakes where LeBron James apparently was saying bad things about the coach during the Paris Olympics. It wasn't LeBron James. It was a deepfake. We actually provided his, uh, organ- uh, uh, his management team the necessary detail so that in X it could be labeled as AI-generated content. But if you look at the audio There was a lot of noise introduced into it, right? To try and avoid detection

   10. SPSpeaker18:37

       Even if you win the gold, it's not on coach. Guys, fuck your ass. I mean, I know him and stuff, but dude, but I don't think I can coach him to do it.

   11. VBVijay Balasubramaniyan18:48

       Lots of people couldn't even hear the audio. They were like, "W- w- is this really..." And so that's where you start seeing these systems diverge, and this is where I have confidence in our ability to detect it, right? Like, which is you run these GANs, you know the architectures that these deepfake generation systems have created, and ultimately you start seeing divergences in one of the objective functions. So either you as a human will be able to detect something's off, or we as a system

9. 19:15 – 21:11

   What Can Spam Teach Us About Deepfakes?

   1. VBVijay Balasubramaniyan19:15

      will be able to detect something's off.

   2. MCMartin Casado19:17

      I mean, one of the reasons that spam works and, and, um, deepfakes work is just the marginal cost of the next call is so low that you can do these things en masse, right?

   3. VBVijay Balasubramaniyan19:28

      Yeah.

   4. MCMartin Casado19:28

      Like the marginal cost of the next spam email or whatever. If it takes me a, a dollar to generate N deepfakes-

   5. VBVijay Balasubramaniyan19:36

      Yeah

   6. MCMartin Casado19:36

      ... how much does it cost to detect N deepfakes? Is it one to one? Is it ten to one? Is it 100 to one?

   7. VBVijay Balasubramaniyan19:42

      Yeah. It's way cheaper to detect deepfakes, right? Because if you think about it, the closest example is Apple released its model, you know, that could run on device. And even that model, which is a small model in order to do lots of things like voice, uh, to text and things like that, our model is about 100 times smaller than that, so it's so much faster in detecting deepfakes. So the, the ratio is about one-hundredth right now, and we're constantly figuring out ways to make it even cheaper, but it's one-hundredth that of generation.

   8. MCMartin Casado20:21

      Wow. I see. So to detect it is two orders of magnitude cheaper than creation.

   9. VBVijay Balasubramaniyan20:27

      Yes.

   10. MCMartin Casado20:28

       Which means in order for anybody to, you know, to economically get... You know, listen, if there is no defense, there's no defense. But if there is a defense, it requires the bad guys to have, you know, two orders of magnitude more resources, which-

   11. VBVijay Balasubramaniyan20:42

       Yeah

   12. MCMartin Casado20:42

       ... is actually pretty dramatic given normally-

   13. VBVijay Balasubramaniyan20:45

       Yeah

   14. MCMartin Casado20:45

       ... you go for parity on these things 'cause there tends to be a lot more good people than bad people. [laughs]

   15. VBVijay Balasubramaniyan20:50

       Yeah. Yeah.

   16. MCMartin Casado20:50

       So.

   17. VBVijay Balasubramaniyan20:50

       And that's the thing.

   18. MCMartin Casado20:51

       Yeah.

   19. VBVijay Balasubramaniyan20:51

       You have two orders of magnitude, and then the fact is that once you know what a deepfake looks like, unless they re-architect the entire system. And the-

   20. MCMartin Casado20:58

       Right

   21. VBVijay Balasubramaniyan20:59

       ... only companies that re-architect full, uh, pipelines, and the last time this was done is back when Google released Tacotron, where they re-architected several pieces of the pipeline. It's a very

10. 21:11 – 24:30

    What Are the Defenses?

    1. VBVijay Balasubramaniyan21:11

       expensive proposition.

    2. MCMartin Casado21:12

       Is the intuitive reason that the cost is so much cheaper to detect is just you just have to do less stuff? Like the-

    3. VBVijay Balasubramaniyan21:17

       Yeah

    4. MCMartin Casado21:18

       ... the person generating the deepfake has to, like, sound like a human, [chuckles] be passable to a human, and evade this, and so, like, that's just more things than detecting it, which just can be a much more narrow focus. So it'll always be cheaper to detect. And then you don't see, you don't see a period in time where the AI is so good no deepfake mechanism can detect it. You don't see that.

    5. VBVijay Balasubramaniyan21:37

       Uh, we don't see that because either you become so good at avoiding detection that you actually start becoming worse at producing human-generated speech, or you're producing human-generated speech and unless, you know, you actually create a physical representation of a human. Because we've had 10,000 years of evolution, and the way we produce speech has vocal cords, has the diaphragm, has your lips and your mouth and your nasal cavity. All of that physical attributes, it's really hard for these systems to replicate all of that. They have generic models, and those generic models are good. You can also think about the more we learn about your voice, Martin, the better we can get at knowing where your voice is deviating.

    6. MCMartin Casado22:26

       And I have an incentive as a good guy to work with you on that, right? So, like, you'll have access to data where, like, the bad people may not have access to data, and it totally makes sense.

    7. VBVijay Balasubramaniyan22:34

       Yeah.

    8. MCMartin Casado22:34

       So it's interesting. So it re- it, it seems to me, like, almost like the spam, the spam lessons learned apply here, which is-

    9. VBVijay Balasubramaniyan22:40

       Yeah

    10. MCMartin Casado22:41

        ... spam can be very effective for attackers.

    11. VBVijay Balasubramaniyan22:43

        Yeah.

    12. MCMartin Casado22:43

        Very effective. Um, defenses can also be incredibly effective.

    13. VBVijay Balasubramaniyan22:48

        Yeah.

    14. MCMartin Casado22:48

        However, you have to put them in place.

    15. VBVijay Balasubramaniyan22:49

        Yeah.

    16. MCMartin Casado22:49

        And so is it this, the same situation here, which is, like, be sure you have a strategy for deepfake detection, but if you do, you'll be okay.

    17. VBVijay Balasubramaniyan22:56

        That's exactly right. And I think, you know, it has to be in each of the areas, right? Like, when you think about deepfakes, you know, you have incredible AI applications that are doing wonderful things in each of these spaces. Like, you know, the voice cloning apps, they've actually given voices to people who've-- who have, uh, throat cancer and things like that, right?

    18. MCMartin Casado23:15

        Yeah.

    19. VBVijay Balasubramaniyan23:15

        But in each of those situations, it was with the consent of the user-

    20. MCMartin Casado23:19

        Yeah

    21. VBVijay Balasubramaniyan23:20

        ... who, you know, who wanted their voice recreated, right? And so that notion that the source AI applications need to make sure that the people using their platform actually are the people who wanna use their platform. That's part A.

    22. MCMartin Casado23:36

        And this, this is where the partnerships that you talked about with the actual generation companies comes in, so that-

    23. VBVijay Balasubramaniyan23:42

        Yeah

    24. MCMartin Casado23:42

        ... then you can help them for, like, the legitimate use cases-

    25. VBVijay Balasubramaniyan23:45

        Yeah

    26. MCMartin Casado23:45

        ... as well as sniffing out the illegitimate one. Is that right?

    27. VBVijay Balasubramaniyan23:47

        Yeah. Absolutely. ElevenLabs, incredible. The amount of work they're doing to, uh, create voices ethically and safely and carefully is incredible. They're trying to get lots of great tools out there. We're partnering with them. They're making their datasets accessible to us. There are companies like that, right? Another company called, uh, Respeecher that's doing, uh-- that did a lot of the Hollywood movies. So all of these companies are starting to partner in order to be able to do this in, in the right way. Uh, and it's similar to, uh, you know, a lot of what happened in the, you know, fraud situation back in the 2000s or the email

11. 24:30 – 28:54

    Policy, Platforms, and Accountability

    1. VBVijay Balasubramaniyan24:30

       spam situation back in the 2000s.

    2. MCMartin Casado24:31

       I wanna shift over to policy.

    3. VBVijay Balasubramaniyan24:34

       Yeah.

    4. MCMartin Casado24:34

       Um, so I [chuckles] I've been in a lot of policy discussions lately, uh, you know, in the California-

    5. VBVijay Balasubramaniyan24:40

       Yeah

    6. MCMartin Casado24:40

       ... uh, in California as well as at the federal level. And so have you given thought to what guidance you would give to policymakers, many of who listen to this podcast, and how they should think about- You know, any regulations or rules around this and maybe how it intersects with things like innovation and free speech, et cetera. I mean, it's a complicated topic.

    7. VBVijay Balasubramaniyan24:59

       I think the, the simple one-liner answer is they should make it really difficult for threat actors and really flexible for creators, right? Like, that's the ultimate difference. And so, and, and, you know, history is rife with a lot of great ways, right? Like, you lived through the email days where the CAN-SPAM Act, uh, was a great way, but it came in combination with better ML technologies, right?

    8. MCMartin Casado25:27

       Maybe just walk through how CAN-SPAM works. I think it's a good analog.

    9. VBVijay Balasubramaniyan25:31

       The CAN-SPAM Act is one where, you know, anyone who's providing unsolicited marketing have to, has to be clear on its headers, has to allow you to opt out, all of those things, and if you don't follow this very strict set of policies, you can be fined. And you also have great detection technologies that allow you to detect these spams, right? Like, now that you follow a particular standard, especially when you're doing unsolicited marketing or you're trying to do, you know, bad things like pornography or things like that, you have detection, AI, ML technologies that, that can detect you well. The same thing happened when, you know, lots of, uh, when banks went online, right? Like, you had a lot of online fraud or e-commerce, right? You had a lot of online fraud, and i- if you remember, the Know Your Customer Act and the anti- anti-money laundering acts came in there. So the onus was you as a organization have to know your customer. You have to put in technology that knows your customer. That's the guarantee. And so you need technology. After that, you can do what you want. And so I think what was really good about both of those cases is they got really specific on, one, what can the technology detect? Because if the technology can't detect it, you can't litigate, you can't, like, fine the people who are misusing it and so on. So what can the technology detect? And two, how do I make it really specific on what you can and cannot do in order to be able to do this? And so I think those two, uh, were great examples of how we should think about litigation. And in deepfake, there is these, this very clear thing, right? Like, you have free speech, but for the longest time, anytime you used free speech for fraud or you were trying to incite violence or you were trying to do obscene things, these are clear places where the free speech, free speech guarantees go away. So as long as you're not doing that, right, like, I think if you're doing that, you should be fined, right? And that's-- And you should have laws that protect you against that, and that's where... That's, that's the model I like to think of.

    10. MCMartin Casado27:43

        But then there's this kind of gray area of unwanted stuff, right? And the unwant- unwanted stuff is like, you know, you didn't ask for it. It may not be illegal, [laughs] but it's super annoying and it's unwanted and it can fill your inbox and it can fill your w- whatever. And for those, you can put in rules, because if somebody crosses those rules, you can litigate them or you can opt out of it. And so it kind of un- regulates the unwanted. I could see that definitely happening here. And then of course there's the wanted stuff which, you know, like, doesn't require any regulation.

    11. VBVijay Balasubramaniyan28:13

        And the only other thing that I'll say is right now, because we consume things through a lot of platforms, platforms should be held accountable at some level to, you know, clearly demarcating what is real and what is not, right? Because otherwise it's going to be really hard for the average consumer to know that this is AI generated versus this is not. So I think, you know, there's, there's a certain amount of accountability there.

    12. MCMartin Casado28:39

        Because the technology is where it is, putting o- the onus on the platforms-

    13. VBVijay Balasubramaniyan28:43

        Yeah

    14. MCMartin Casado28:43

        ... to do best practices, just like we did for spam, right?

    15. VBVijay Balasubramaniyan28:47

        Yeah.

    16. MCMartin Casado28:47

        Like, I rely on Microsoft and Google for, for the spam detection, doing the same type of thing for the pla- this sounds like a very, very

12. 28:54 – 30:26

    Preparing for a Deepfake-Proof Future

    1. MCMartin Casado28:54

       sensible recommendation.

    2. VBVijay Balasubramaniyan28:56

       Yeah.

    3. MCMartin Casado28:56

       So, um, key point number one is, you know, deepfakes have been around for a long time. We probably need a new name for this new generation, and this isn't just, like, some hypothetical thing, but you're seeing a massive increase.

    4. VBVijay Balasubramaniyan29:09

       Yes.

    5. MCMartin Casado29:09

       You said as much as, as, as one per day.

    6. VBVijay Balasubramaniyan29:11

       Yeah.

    7. MCMartin Casado29:12

       And the cost to generate has gone way down.

    8. VBVijay Balasubramaniyan29:14

       Yep.

    9. MCMartin Casado29:14

       Um, good news is that these things are imminently detectable and in your opinion will always be detectable if you have a solution in place.

    10. VBVijay Balasubramaniyan29:24

        Yep.

    11. MCMartin Casado29:24

        And then as a result, I think, you know, any policy should provide the guidance and maybe accountability for the platforms to detect it because we can actually detect it. And so, like, listen, it's something for people to know about, but, you know, it's not the end of the world and, and, you know, policymakers don't have to regulate all of AI for this one specific use case. Is this a fair synopsis?

    12. VBVijay Balasubramaniyan29:47

        This is a beautiful synopsis, Martin. You've captured it really, really well.

    13. SPSpeaker29:51

        Thank you so much for listening to the a16z podcast. If you've made it this far, don't forget to subscribe so that you are the first to get our exclusive video content. Or you can check out this video that we've hand selected for you. [outro music]

Episode duration: 30:26