By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
CHAPTERS
- 0:00 – 2:23
Deepfake Surge and a Cheap-Fake Origin Story
Vijay Balasubramaniyan opens with the sharp rise in deepfakes and the growing number of voice-cloning tools. He explains the Nancy Pelosi cheap fake and how generative adversarial networks, a kind of reverse Turing test, made deepfakes convincing.
- •Deepfakes rose 1400% in the first six months of this year compared to all of last year
- •Voice-cloning tools grew from 120 to 350 between the end of last year and March
- •The Pelosi video was a cheap fake made by slowing audio, not a deepfake
- •GANs pit two systems against each other to get close to a person's likeness
- 2:23 – 4:16
Generative AI Drives the Cost of Cloning Toward Zero
Martin Casado asks whether generative AI has created a new kind of deepfake. Vijay describes how little audio is now needed to clone a voice compared with the 20 hours John Legend recorded for Google Home.
- •Early on there was just one cloning tool, Lyrebird, and now many tools and open source options exist
- •Cloning needs about 3-5 seconds of audio, or about 15 seconds for high quality
- •John Legend spent close to 20 hours recording for Google Home
- •Cost has fallen close to zero
- 4:16 – 7:35
From Japan's Ore Ore Sagi to LLM-Powered Scams
Martin recounts a scam call his parents received claiming he was in prison in Japan. Vijay says the scam has long existed, but scale and voice mimicry have changed, and fraudsters now pair text-to-speech with LLMs that invent persuasive stories.
- •Martin's parents were told he had a car accident and needed $10,000 in bail money
- •The scam traces back to Japan's "help me grandma" fraud, which cost close to half a billion dollars
- •A short TikTok clip is now enough to clone a voice
- •LLM hallucination becomes an asset when the goal is to convince someone
- 7:35 – 10:00
Election Interference: The Biden Robocall
Asked for other uses of deepfakes, Vijay describes the New Hampshire primary robocall in President Biden's voice. His company identified it as a deepfake and used source tracing to find the AI application behind it.
- •Deepfakes affect commerce, media and communication
- •The call went to voters in the Republican primary in New Hampshire in January
- •Source tracing identified which AI application created the fake
- •The application shut down the user, and regulators later pursued telcos and the political analyst
- 10:00 – 10:53
Deepfakes in Banking and Commerce
Vijay explains how deepfakes in financial institutions went from about one a month per customer in 2023 to one per customer per day. Some large banks now see one every three hours.
- •Generative AI arrived in 2022
- •One deepfake a month per customer in 2023 became one per customer per day
- •Some big banks see a deepfake every three hours
- •The 1400% increase is measured over the first six months of this year
- 10:53 – 13:10
Can Deepfakes Be Detected? Accuracy and Sampling
Casado asks whether this is the beginning of the end. Vijay compares it to earlier cycles like spam, saying deepfakes are detectable at 99% with a 1% false positive rate. He says voice has thousands of samples per second where the human ear cannot catch anomalies but AI can.
- •Detection runs at 99% with a 1% false positive rate
- •Voice identification itself is much more precise, around one in 100,000 to one in a million
- •Voice has 8,000 samples a second in contact centers, 16,000 in online conferencing, 44,000 in music
- •Deepfake systems make mistakes in the frequency and time domains that human ears miss
- 13:10 – 15:42
Why Watermarking Falls Short
Casado is skeptical of watermarking and cryptography proposals from policy circles because they depend on criminals complying. Vijay says attackers won't comply and that the Biden call's watermark was 98% stripped, and mentions that 90% of war footage sent to news organizations is fake.
- •Attackers will not comply with watermarking
- •98% of the watermark in the Biden call was lost across air and telephony channels
- •Watermarks still help with defense in depth for genuine content
- •News media organizations report 90% of videos and audio from the Israel-Hamas war are fake
- 15:42 – 19:17
Detecting Zero-Day Deepfakes: Fake Prints and Adversarial Training
Vijay describes how detection stays resilient against new deepfake systems. Components leave artifacts called fake prints, and his team runs GANs against their own detectors, which forces deepfake generators to trade quality against evasion, as in the LeBron James audio.
- •Zero-day deepfake detection is the goal
- •Deepfake systems reuse components that leave behind fake prints
- •Detection is trained by running GANs against it iteratively
- •Evading detection by adding noise makes speech harder to understand, as in the LeBron James clip
- 19:17 – 22:34
The Economics: Detection Is 100 Times Cheaper
Casado asks how the cost of generating deepfakes compares to detecting them. Vijay says detection is about one-hundredth the cost, forcing attackers to spend far more, and that physical limits of human speech make the defender's position durable.
- •Detection is about two orders of magnitude cheaper than generation
- •The detection model is about 100 times smaller than Apple's on-device model
- •Re-architecting a full pipeline is expensive, as with Google's Tacotron
- •Human speech anatomy is hard for generic models to replicate
- •Good actors can supply data about a voice, which improves detection
- 22:34 – 24:31
Spam Lessons and Partnering with Generation Companies
Casado draws the analogy to spam: attacks are effective but so are defenses, if put in place. Vijay says source AI applications must verify consent and mentions partnerships with ElevenLabs and Respeecher.
- •Spam showed that defenses can be effective if deployed
- •Voice cloning has legitimate uses, such as giving voices to people with throat cancer
- •Source AI apps need to ensure users have consent
- •ElevenLabs and Respeecher are working with the detection company
- 24:31 – 28:13
Policy Guidance: CAN-SPAM, KYC and Free Speech
Casado asks what guidance Vijay would give policymakers. Vijay says to make it hard for threat actors and flexible for creators, pointing to CAN-SPAM and Know Your Customer rules as models that were specific and tied to what technology can detect.
- •Make it difficult for threat actors and flexible for creators
- •CAN-SPAM required clear headers and opt-outs, backed by fines
- •Know Your Customer and anti-money laundering rules put the onus on organizations
- •Free speech protections already do not cover fraud, incitement or obscenity
- •Regulation should be specific about what technology can detect
- 28:13 – 30:26
Platform Accountability and Closing Synopsis
Vijay says platforms should be held accountable for demarcating real from AI-generated content, and Casado compares this to relying on Microsoft and Google for spam detection. Casado then summarizes the key takeaways, and Vijay endorses them.
- •Platforms should clearly mark what is real and what is not
- •Platform best practices parallel spam detection
- •Deepfakes are surging and cheap to generate but detectable
- •Policymakers don't need to regulate all of AI for this use case
