Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398
Lex Fridman (host), Mark Zuckerberg (guest)
In this episode of Lex Fridman Podcast, featuring Lex Fridman and Mark Zuckerberg, Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398 explores zuckerberg And Fridman Demo Photoreal Metaverse, Redefining Real Presence Online Lex Fridman interviews Mark Zuckerberg entirely inside Meta’s experimental photorealistic “Codec Avatar” system, demonstrating a sense of presence that feels nearly identical to in‑person interaction. Zuckerberg explains how high‑fidelity face and body scans are compressed into low‑bandwidth, real‑time avatars, and outlines the roadmap to making such avatars scannable via smartphones and usable across work, social, and mixed‑reality contexts. They explore how mixed reality in Quest 3 blends digital objects with physical space, and how Meta’s open‑source Llama 2 and new AI personas will power both assistants and character‑based AIs across Meta’s platforms. Throughout, they dive into philosophical and ethical questions about identity, grief, AI replicas, what counts as “real,” and how a blended physical–digital world could transform work, relationships, and civilization.
Zuckerberg And Fridman Demo Photoreal Metaverse, Redefining Real Presence Online
Lex Fridman interviews Mark Zuckerberg entirely inside Meta’s experimental photorealistic “Codec Avatar” system, demonstrating a sense of presence that feels nearly identical to in‑person interaction. Zuckerberg explains how high‑fidelity face and body scans are compressed into low‑bandwidth, real‑time avatars, and outlines the roadmap to making such avatars scannable via smartphones and usable across work, social, and mixed‑reality contexts. They explore how mixed reality in Quest 3 blends digital objects with physical space, and how Meta’s open‑source Llama 2 and new AI personas will power both assistants and character‑based AIs across Meta’s platforms. Throughout, they dive into philosophical and ethical questions about identity, grief, AI replicas, what counts as “real,” and how a blended physical–digital world could transform work, relationships, and civilization.
Key Takeaways
Photorealistic avatars can cross the uncanny valley and enable true remote intimacy.
Fridman repeatedly forgets Zuckerberg is not physically present, suggesting Codec Avatars capture enough micro‑expressions—especially around the eyes—to recreate the psychological feeling of being in the same room, potentially overturning standard video‑call limitations.
Get the full analysis with uListen AI
Mass adoption depends on cheap, fast scanning—ideally via smartphones.
Today’s hour‑long, high‑end scans are a research process; Meta’s goal is a 2–3 minute phone scan that generates similar fidelity, which Zuckerberg frames as one of the last big technical hurdles before this becomes a consumer feature.
Get the full analysis with uListen AI
Mixed reality will make VR experiences safer, more comfortable, and more social.
Quest 3’s higher‑resolution passthrough, stronger chipset, and hand‑tracking let users see their real environment while interacting with digital objects, alleviating safety concerns and enabling scenarios like hiding behind your real couch from virtual enemies.
Get the full analysis with uListen AI
The ‘real world’ is shifting from purely physical to a physical‑digital blend.
Zuckerberg argues that the modern ‘real world’ will be the fusion of physical and holographic objects, where many current physical artifacts (screens, games, some art and media) are better as instantly instantiated digital holograms.
Get the full analysis with uListen AI
AI personas will be numerous, specialized, and embedded in social contexts.
Instead of one monolithic super‑assistant, Meta is building multiple AIs—from neutral Meta AI to business bots and entertaining characters like Snoop Dogg as a dungeon master—designed to live inside chats, games, and social apps.
Get the full analysis with uListen AI
Creator and celebrity AI replicas require strong control and reliability.
Zuckerberg notes that making an AI ‘version of you’ is far harder than inventing a new fictional character, because the model must stay within behavioral and factual bounds that the real person accepts, demanding more predictable, constrained LLM behavior.
Get the full analysis with uListen AI
Open‑sourcing models like Llama 2 can accelerate innovation with manageable risk.
Meta’s internal red‑teaming led them to conclude the societal upside of releasing Llama 2 outweighed the downsides; they’re already planning successors (e. ...
Get the full analysis with uListen AI
Notable Quotes
“It feels like we're in the same room. It feels like the future.”
— Lex Fridman
“The real world is the combination of the physical world and the digital worlds coming together.”
— Mark Zuckerberg
“We’re basically shortcutting the laws of physics and delivering the social and psychological benefits of being able to be present with another person.”
— Mark Zuckerberg
“I'm already forgetting that you're not real… it feels like I'm seeing color for the first time.”
— Lex Fridman
“I don’t think we want one big superintelligence. We want to empower everyone with a variety of AIs.”
— Mark Zuckerberg
Questions Answered in This Episode
How might widespread access to photorealistic remote presence change where people choose to live and work?
Lex Fridman interviews Mark Zuckerberg entirely inside Meta’s experimental photorealistic “Codec Avatar” system, demonstrating a sense of presence that feels nearly identical to in‑person interaction. ...
Get the full analysis with uListen AI
What psychological impacts—positive and negative—could arise from routinely interacting with perfected or idealized avatar versions of ourselves and others?
Get the full analysis with uListen AI
Where should society draw the ethical line on using avatars and AI to simulate deceased loved ones, and who gets to decide?
Get the full analysis with uListen AI
How will norms about harm and acceptable behavior evolve when acts that are violent or taboo in the physical world feel vivid but are consequence‑free in virtual worlds?
Get the full analysis with uListen AI
If anyone can create an AI version of themselves or others, what new challenges arise for authenticity, consent, and trust in online identity?
Get the full analysis with uListen AI
Transcript Preview
The following is a conversation with Mark Zuckerberg inside the Metaverse. Mark and I are hundreds of miles apart from each other in physical space, but it feels like we're in the same room because we appear to each other as photo-real Kodiak Avatars in 3D with spatial audio. This technology is incredible, and I think it's the future of how human beings connect to each other in a deeply meaningful way on the internet. These avatars can capture many of the nuances of facial expressions that we use- we humans use to communicate emotion to each other. Now, I just need to work on upgrading my emotion-expressing capabilities of the underlying human. This is the Lex Fridman podcast. And now, dear friends, here's Mark Zuckerberg. Wow. (laughs) This is so great. Lighting change. Wow.
Uh, yeah, we can put the light anywhere.
And it doesn't feel awkward to be really close to you?
No, it does. I actually moved you- I moved you back a few feet before you got into the-
Well-
... headset. You were, like, right here.
I don't know if people can see this, but this is incredible. The realism here is just incredible. Where am I? Where are you, Mark? Where, where, where are we?
You're in Austin, right?
No, I mean, this place. (laughs)
Oh. (laughs)
We're, we're, we're shrouded by darkness with ultra-realistic face and it just feels like we're in the same room. This is really the most incredible thing I've ever seen. And sorry to be in your personal space.
I mean...
We have done jujitsu before.
Yeah. No, I was- I was commenting to the team before that even that w- I, I feel like we've choked each other from further distances than it feels-
(laughs)
... like we are right now.
I mean, this is just really incredible. I don't know how to describe it with words. It really feels like... It feels like we're in the same room.
Yeah.
It feels like the future.
Yeah.
This is truly, truly incredible. I just want to take it in. I'm still getting used to it. It's like, it's you, it's really you, but you're not here with me, right? You're there wearing a headset and I'm wearing a headset. It's qu- it's really, really incredible. So what, um... Can you describe what it takes currently for us to appear so photorealistic to each other?
Yeah. So, I mean, for, for background, we both did these scans for, uh, this research project that, that we have at Meta called Kodiak Avatars. And the idea is that instead of actually- instead of our avatars being cartoony, um, and instead of actually transmitting a video, what it does is we've sort of scanned ourselves and a lot of different expressions and we've built a computer model of sort of each of our faces and, and bodies and, um, the different expressions that we make and collapse that into a, a Kodiak that then when you have the headset on your head, it can... It, it sees your face, it sees your expression, and it can basically send, um, an encoded version of what you're supposed to look like over the wire. So, um, so in addition to being photorealistic, it's also actually much more bandwidth efficient than transmitting a, a full video or especially a 3D immersive video of, of a whole scene like this.
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome