Skip to content
The Joe Rogan ExperienceThe Joe Rogan Experience

Joe Rogan Experience #1342 - John Carmack

John Carmack is a computer programmer, video game developer and engineer. He co-founded id Software and was the lead programmer of its video games Commander Keen, Wolfenstein 3D, Doom, Quake, Rage and their sequels. Currently he is the CTO at Oculus.

Joe RoganhostJohn CarmackguestJamie Vernonhost
Aug 29, 20192h 36mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:01 – 2:55

    Oculus Quest: standalone VR and why media apps matter

    1. JR

      Three, two, one. (hands thud) Here we go. How are you, sir?

    2. JC

      I'm doing good.

    3. JR

      Um, I'm super happy to have you here. If there's a Mount Rushmore of video games, you're George Washington. You're up there.

    4. JC

      So, this is great. People have been kind of nudging us for years and years to get this done, so it's great that we're finally able to make it happen.

    5. JR

      Yeah. Um, w- I have been, from day one, a gigantic Quake junkie and a DOOM junkie, so for me to have you in here is a, a giant treat. And everybody ... You know, I've talked about your video games and your creations so many times on this podcast, so it's, it's very cool to have you here. And thank you very much for showing me ... Before the podcast got started, I should tell everybody, you showed me the latest and greatest version of Oculus Rift, which is amazing. It's so small. For people watching the YouTube, this is the entire unit. This, this thing that sits on your head, it's very light, and it's not attached to a computer. Like, you don't have to carry anything around, and there's no extra. Everything is in here.

    6. JC

      Yeah, so this is the Oculus Quest, the standalone device.

    7. JR

      (laughs)

    8. JC

      And it's been kind of the culmination of a bunch of different products that we've been working on. And it's the vision that we had, even six years ago, just the idea of not connected to anything. You put this magic hat on, and you're transported to these different worlds, and it's all coming together.

    9. JR

      And this is available right now. Anyone can buy this, right?

    10. JC

      Yeah.

    11. JR

      Yeah. Okay, so it's avai- ... Jamie just actually pulled up-

    12. JC

      Right.

    13. JR

      ... the, the website. Um, can I push this up to you? Pull that ... Yeah, there you go.

    14. JC

      There you go.

    15. JR

      Um, th- how ... What is the battery life on these things?

    16. JC

      So, it depends on what you're doing, where, uh, if you just sit there and watch Netflix, uh, you know, it'll last a little more than three hours. If you play some of the really hardcore games, it might only last two hours or so.

    17. JR

      Do you have numbers on how many people are watching Netflix on this thing?

    18. JC

      So, our previous one right before this, the Oculus Go, was a little bit more media-focused, and that's one of our more popular ac- applications. I mean, surprisingly, things ... Everybody thought that VR was going to be all about these just amazing gaming experiences, but some of the most popular experiences are doing reasonably conventional things, watching Netflix, watching YouTube, Amazon Prime, stuff like that. Where if you're, like ... If you're you and you've got a great home theater and everything, there's not this much benefit to having a theater like that in VR. But if you're in a situation like you're in a tiny room in Tokyo or something, the idea of being able to put on the VR headset and have this, like, lovely ski lodge atmosphere with a giant screen TV, it has some real benefits. So, in the end, VR should be a replacement for anything you do on screens today, whether it's your phone, your tablet, your TV, your laptop, your PC. All of these should eventually be superseded by just having more flexible screens in VR. I mean, we have lots of challenges now with resolution and comfort for long-term use, but this is the direction that everything's going. Not only do you have things in VR that you couldn't do anywhere else, just experiences that you can't have with that level of immersion, but it should pull along every other thing that people do with screens devices today.

  2. 2:55 – 5:02

    Theme-park VR vs home VR: motion platforms, heat, props, and smells

    1. JR

      Uh, that's ... I didn't consider the television shows, but of, of course, people would be watching Netflix on this if it's possible. Have you done the, uh, Disney World ride, the Avatar ride, Flights of Passage?

    2. JC

      No, I haven't done that yet.

    3. JR

      It's amazing. You sit on this ... It's like a motorcycle-looking thing. It straps you in place, and it's supposed to represent one of those flying dragon things in Avatar. And then you have the headset, you put that on, and the vir- virtual reality experience is second to none. I mean, it's incredible. Super high resolution, and y- the, the motorcycle's moving around. You get wind and smells and all these, uh, sensory-

    4. JC

      Yeah, so that's one of the really interesting things. Like, I think about that whenever I am at amusement parks for things like the Harry Potter rides and stuff like that, where they're doing lots with screens and motion platforms, where ... I think about it from the VR perspective. And anything we're doing visually and audibly, we could go ahead and do a great job in the headset. So, it's cutting it down to these few physical things that you can't do. So, you've got things with motion platforms that actually jostle you around that you can't do in VR. You've got things like smell. And, like, The VOID, um, where they have the Star Wars, uh, experience there, we have a fantastic Star Wars experience on Quest, which in many ways has a lot of that magic, but in The VOID where they set it up and they blow hot air over what's the virtual lava, uh-

    5. JR

      Yeah.

    6. JC

      ... towards you, that's something that you still don't get. But it's kind of like the, the age-old battle of what can you do differently in an arcade that you can't do as good in your, uh, your home system? And VR now takes ... Lets you do all of these amazing things there, but if you're willing to spend millions of dollars and build a theme park attraction, essentially, you can still throw some of these extra things in. People joke about, when is Smell-O-Vision coming to VR? And (laughs) there have actually been real companies that have spun up to say, it's like, "Oh, we want to do scent augmentation."

    7. JR

      Yeah.

    8. JC

      But it's not a great thing, uh, and those are still the last vestiges of things where you have to go someplace. But the promise of VR is to ... You know, the world as you want it, not having to go to some place to do something magical. And if you can get to 90-something percent of that experience staying in your own room, then that's great.

  3. 5:02 – 9:04

    Smell-O-Vision in VR: why scents don’t mix like RGB

    1. JR

      What would they be able to do with Smell-O-Vision?

    2. JC

      (laughs)

    3. JR

      Would you have, like, a standalone unit that, like, has, uh, access to the program? So, like, if you were flying over orange fields, it would spray citrus in the air? Sort of, sort of like Soaring Over The World? Have you ever done that Disneyland ride?

    4. JC

      Yeah.

    5. JR

      Yeah.

    6. JC

      So, they l- somebody literally did make this, where they made a little box that put, uh, that glued or attached to the underside of the head-mounted display. And one of the interesting things about scent as opposed to, uh, like, audio or video ... With video, everybody knows that you just make red, green and blue colors. You can mix them in any way and make all the colors that we can see. Smell isn't like that. Uh, our nose is actually a receptor for a whole lot of discrete different molecules. There's no way to mix up smell like the way we do with light to make red, green and blue primaries with that.

    7. JR

      Oh.

    8. JC

      So, they really had to pick, okay, here's the dozen or so smells that we're going to have with this. And it would just sort of spritz it out on a little blast of air, uh, very close to your nose, so it doesn't need much of it to get in.

    9. JR

      Mm-hmm. So, if you really wanted to do some sort of a jungle experience with, you know, thousands of different smells of plants and dirt and all of ... You would have to have, like, some enormous unit that's spraying these various things.

    10. JC

      Yeah, although I suspect that at least modern people in modern society do not really have that discerning of a level of scent. Like, if you took, uh, some Peruvian indigene or something that ... They probably would complain, it's like, "Oh, the fidelity on this is garbage. I should have-"

    11. JR

      (laughs)

    12. JC

      "... 500 different smells here, and I only detect five."

    13. JR

      Right.

    14. JC

      But you take, you know, a normal city person, and you just ... Three scents of jungle is probably gonna be more than enough to sell the experience.

    15. JR

      Yeah, the guy in the jungle will be like, (sniffs) "This is sterile urine."

    16. JC

      Yeah.

    17. JR

      "This is not real."... the, so what is this, Jamie?

    18. JV

      That's it right there.

    19. JC

      Oh, my goodness.

    20. JV

      It's a home version of it, I guess- ... one of them.

    21. JR

      Oh, my goodness. So this is something she's holding?

    22. JV

      Uh, no, it, and it, she's pulled it off. It attaches to the headset right here.

    23. JR

      It says, "Feel real."

    24. JV

      Yeah.

    25. JR

      I- sensory mask. So it's a sensory mask that s- smell virtual reality. Oh, yeah.

    26. JV

      (laughs)

    27. JR

      It, so it, it simulates h- simulates hundreds of smells to immerse you in the virtual world. Wow.

    28. JV

      Mm-hmm.

    29. JR

      Of course it's coming, but why am I shocked? Look at this guy. It's got, like, a little cartridge?

    30. JC

      Yeah.

  4. 9:04 – 15:21

    Haptics and open sourcing: from impact jackets to Doom living forever

    1. JR

      Now, what about the possibility of a haptic feedback vest, or a suit, or something that you could put on your body?

    2. JC

      So there's the interesting things all the way back into the, the Doom and Quake days. I remember one of the really early, uh, kind of entrepreneur guys that came by, he had made this leather jacket with all of these impact pucks on it, and it had, uh, it had, like, eight or nine different things that were these solenoids that could deliver a pretty sharp thud, and he wanted to in a, you know, get, uh, support added to the games for that. The idea you play that, and was your get- when you're getting shot, it actually feels like you're getting hit in the back.

    3. JR

      Ooh.

    4. JC

      And, you know, I didn't think that was a very likely, um, mass market consumer thing. I mean, not too many people want that level of, uh, fidelity where it actually starts making you sore. But that's one of the wonderful things about being able to open source the various codes after the games are a little bit older, where anybody that wants to can nowadays go and take Doom or Quake or those earlier titles and program in for whatever crazy thing. They don't have to convince someone. They don't have to go convince skeptical John Carmack that this device is going to be a worthwhile thing to add support to the mainline code. They can just go do it, which is a wonderful thing.

    5. JR

      That is very cool that you guys do that. I think that's really cool, and it's-

    6. JC

      Yeah, that was one of those things where-

    7. JR

      ... exciting.

    8. JC

      ... early on, as you can imagine, that was a tough sell in the company, where the people that weren't coming from the sort of hacker ethic background on the programming side, you get the businesspeople, and the artists, and the designers, and they're like, "We wanna just give away our source code? Won't that be a leg up to the competitors? Why do you wanna do this?" And it was one of my, uh, it made me really happy when many years later, Kevin Cloud, one of my early partners, uh, told me that, "Yeah, in retrospect, that was really the right thing to do." And it's great with Doom and Quake now, especially Doom, where anything that has a processor runs Doom. If it's got a 32-bit processor and it can conceivably display an image, people have ported Doom to it, and that code will live forever. 100 years from now, people will be able to dig up and run the Doom source code in some emulator.

    9. JR

      Yeah, that is very cool. Now, what was the conversation like? Like, when you guys, when you were saying, "Hey, this is good for the community. This is good for games overall. It's gonna get people excited about it. It's just gonna generate more business." Like, what did you, how did you sell it?

    10. JC

      So it's an interesting path there, where in our earliest games, uh, I can remember that some of the very first things that happened with Wolfenstein 3D, before Doom, where that was not set up to be easy to be modified. Uh, we were still back in those days of fitting on floppy disks, so I had all the data compressed in this nonstandard thing that I just made up at the time. But people dug through all of that, disassembled the code, figured out how it worked, and started making some level editors, and doing the things like, you know, replacing Hitler with Barney-

    11. JR

      (laughs)

    12. JC

      ... and all of these early mods, and we were all like, "Well, this is fantastic." You know, this is people taking the game, they played through the game, they loved it, and they loved it so much that they wanna keep doing things on it, and they wind up breaking into the game at that time, essentially, to figure out how to make new things. So by the time we were working on Doom, it was an explicit top-line technical goal for me that, okay, I had these graphics things I wanted to do, I wanted to do networking, but I also wanted to really make game modding a first top-level feature. So we added all this ability to do the, the WADs and PWADs, and we documented all of it, and we released a lot of the tools, the early source code, so here's how you go ahead and- it was much harder at that point with the more sophisticated stuff going on, but here's how you build a level f- in Doom, and we even released the code for our level editor. Although, that didn't help the community that much, because we were using these crazy expensive NeXT workstations, uh, and other people had to take the steps to go ahead and make them run on PCs. But-The step beyond that, when we were looking at Quake, I am... So, I knew that I wanted to enable actual changes to the gameplay because in Doom, you could swap out all the different models, you could swap out, I, I... the way things looked, the way things sounded. And some people would go in and actually patch the executable to do a few minor changes in gameplay. But the next step, clearly, was allowing people to really make whole new gameplay modes. So, that was how Quake got this Quake C Extension Language, and we wrote a lot of the game, uh, in that. And that led to all the things like Capture the Flag and Team Fortress and all those, which was-

    13. JR

      Rocket Arena.

    14. JC

      Yeah. All these really, really great things. But there were still things that you couldn't do or couldn't do effectively there, and that's where there was still this desire to be able to say, "Well, what if we just gave them everything? What if we gave them the full source code and let them sort of hack to their heart's content, port to other platforms?" And again, it wasn't a super popular decision, but the way I was pitching it was, "Well, it still helps our titles. It still gives them life. It gives them life after they would have been off the shelf, falling off of people's radar, just falling into some game of nostalgia." But being able to let people make real new versions of it would be... You know, it would keep them current. It would keep them relevant. And so the pitch that I ran for years there was, after our new game came out with brand new technology, then we should be able to open source release the previous generation. So first, when Doom was out, we released Wolfenstein. When Quake was out, we released Doom. And when the later Quakes were out, we released the Quake 1 code. And that worked out really remarkably well. I know at the time, there were some people in the company that are just like, "This is just John's thing," and they were not really happy about it. But I, I was in a position where I could kind of throw my weight around a little bit that, and I was happy that I did it. And in the end, everybody agrees it was a good win. Um, I'm a little sad that more companies weren't able to take that final step. Modding was embraced broadly by a lot of game companies, but only a handful of companies were able to really go the entire way and release full source code, uh, in the years since.

    15. JR

      That's too bad because that is one of the cooler aspects of the Quake community is that, you know, you guys did release that stuff, and there were all those cool extras and, and things you could download, and maps. So many different maps that people had created that were really interesting. Like, I remember one was a guy's apartment. Like, you could, you could play Quake in an apartment. Like, you could shoot... You could get to the top of the toilet and shoot at things off the toilet. It was really amazing. How did you feel about when people would play the game competitively and they would turn all

  5. 15:21 – 20:12

    Competitive Quake design: graphics ‘cheats,’ esports growth, and brutal skill gaps

    1. JR

      the textures off?

    2. JC

      (laughs) So, I... Yeah, that's, uh, that especially is a sore point with the artists that have labored-

    3. JR

      Yeah.

    4. JC

      ... for years to build these glorious textures, and then you get the people that just turn them down. And there's two reasons to turn them down. You turn them down to help performance in some cases. In the early days, and especially the early graphics cards, you would get higher frame rate if you turned them down, so you'd have less latency in your response times. But there's also the even more nefarious thing about turning them all down to improve the contrast on your enemy acquisition.

    5. JR

      Yeah.

    6. JC

      I... So, people want this almost flat-shaded world, so that any moving set of pixels there, just turn and fire at that. And that's... Um, I never came to really great terms with that, where I always thought on... In the early days of esports and gaming, we did always insist that people have to play with at least plausible resolutions there because we want our game to look good.

    7. JR

      Yeah.

    8. JC

      We want people that are looking at it for the first time, seeing these professionals play it. I am... You know, we don't want them to look at that and say, "Well, this game looks like garbage. It's all flat-shaded or blurry." I am... And luckily, computers got fast enough that people could start playing at the frame rates that they wanted, even with the full textures, uh, running in it. I am... But the whole pace of doing the, um, kind of the esports and the competitive gaming was very interesting. We saw the dawn of that with Doom, but it's been pretty surprising. It's been amazing the state that it's taken, it's gotten to today, where I remember when we did the Quake Red Annihilation Tournament. I gave away my first Ferrari as grand prize, and I was thinking-

    9. JR

      Really?

    10. JC

      ... "This is just the most over-the-top thing. This is gonna be unmatched for years." And it was only a year later that there was some other tournament with $100,000 prize coming out. So, that went on, went much quicker than I expected. And then today, you have just the, the amazing celebrity of the top pro players. It's, uh, it's again, it's great to see the path that it's taken.

    11. JR

      It really is, and it's very interesting to see that they're now, like, legitimate sports stars, and they make a ton of money. Whereas if you were a kid 10, 15 years ago, your parents would tell you, "You're wasting your time. This is nonsense. Why are you playing these games?" But now, you have a legitimate opportunity to be a professional game player.

    12. JC

      Although, I do... I get... I- Hey, there's a hazard there, where-

    13. JR

      What's this, Jamie? What are you pulling up?

    14. JV

      Top earners this year-

    15. JC

      Yeah.

    16. JV

      ... uh, compared to the top, uh... Tiger Woods, how much he won in the Masters, and the top Indy 500 earners in winning.

    17. JR

      So, Tiger Woods won $2 million in the Masters. The top Quake players won $3,121,872. Oh, it's not Quake. What game is that?

    18. JV

      Dota.

    19. JR

      What does, what does that? What's Dota?

    20. JV

      Uh, it's a different-

    21. JR

      What's it stand for?

    22. JV

      ... game than Quake, but it's, uh, it's very popular right now.

    23. JR

      What's it stand for? I've never even heard of it.

    24. JC

      It's a Valve game.

    25. JR

      Oh.

    26. JC

      You know, it's a, it's a MOBA, massively online battle arena game-

    27. JR

      Oh.

    28. JC

      ... where you're directing all the different little characters down there. And that, along with, especially League of Legends, even beyond that, uh, not as, uh, popular in the United States, but is just amazing overseas, where, uh, they... Like, if you look at the numbers for things, people think, "Oh, the Super Bowl, like the height of all competitive sports, whatever." And, uh, a lot of these esports games, especially in South Korea, are they dwarf those numbers.

    29. JR

      Really?

    30. JC

      And some people-

  6. 20:12 – 27:04

    Quake obsession, ‘Quake dreams,’ and VR’s memory-as-presence effect

    1. JC

      Some of the interesting things about Quake, where it wasn't really so clear when we were designing it, but it is a brutal game, especially in one-on-one, where a lot of modern games are much more approachable, where if you followed a lot of the Quake games, a lot of them were just blowouts, where you would get somebody that would take control of the level, and they would be running their pattern, denying anyone any foot in the door, and you'd wind up with these 20:1 blowout games, where-

    2. JR

      Yeah.

    3. JC

      ... there are things that you can do in game design to make it more approachable, where, like, if you don't have health packs, where you can keep... Because in a game like Quake, you go in, as long as you come out on top in the fight, you've got this little window to run around and bring yourself, uh, bring your health back up. So, even if you're only 5% better, you might win every engagement, because you have enough time to go back, get yourself back up before you wind up reengaging. Where, in another game, if you didn't have health that, uh, that would continuous... You had the ability to bring it back up, then even if somebody didn't win, if they knocked you down a whole lot, then they might get you the next time around, and scores can be much more even.

    4. JR

      Mm, that makes sense.

    5. JC

      But-

    6. JR

      Yeah.

    7. JC

      ... Quake gameplay winds up brutal, tending towards blowouts, and very frustrating for, uh... It did not have the approachability for new players, where a lot of more modern games, things like Overwatch, uh, can be jumped into a lot more easily, because team play is another aspect of that, where if you've got a team, you could be on the winning team even if you suck, because you might have really great players that are kind of covering for you there. You can jump in and have the chance to say, "Yay, I won," uh, even if you didn't contribute at all, and you might wind up, uh, you know, doing something. You start off being completely useless, and then you slowly work your way up to being able to contribute effectively for your team. So, I can recognize some of these things now about ways to make games more approachable, but the, the kind of brutality of Quake, like, there was a, uh, it was a taste that a lot of people really did like. It didn't, wasn't so much explicitly designed for that, but it worked out that way. And that's one of the interesting things as we look at game design today versus the old days. A lot of people fall into a sort of nostalgia trap about saying, "Well, the games I grew up with were the greatest games ever," and you see it with music and movies and everything. And I tend to be much more optimistic about the state of things today, where the amount of effort that goes into the modern games is extraordinary, just the detail and, uh, all the quality on all the different levels. But there is a little sense of games are so expensive to make now, you know, sometimes in the hundreds of millions of dollars, that they do have to be conservative. So, they have to be careful to make sure that they've got something that has a broad mass appeal, and I think that is the, uh, the upside of some of the older games, where they might have a little bit more of a distinct flavor. They weren't sort of focus-grouped to death in the way-

    8. JR

      Mm.

    9. JC

      ... that some of the more modern games can be.

    10. JR

      That's interesting. Yeah, focus-grouped to death I'm sure is a problem. Um, the, the, the Quake blowouts, although that is a thing, um, i- it's so fun when you're the person blowing-

    11. JC

      (laughs)

    12. JR

      ... the other person out that it's worth learning the maps, and that, that's the... Well, s- for people who don't understand what we're talking about, Quake would have, um, well, it has maps. And on those maps, like, "This is where the rocket launcher is. This is where the real gun is. This is where the health is. This is where the mega armor is." And you had to know where these things were and that they would regenerate every X amount of seconds. And so, you're managing not just your fighting, but you're also managing the resources, so you're running around and trying to control the map and trying to control the mega health and trying to control where the armor is, and don't let the guy get the real gun, don't let the guy get the rocket launcher. And in doing that, it's this incredible game of strategy as well as, like, fast-twitch aiming, and there's so many factors going on.

    13. JC

      Yeah, and the masters would have it timed such that-

    14. JR

      Yeah.

    15. JC

      ... they're just running to where it's gonna spawn, and a half second before they get there, it respawns, they run over it, it's theirs, and it's gone. And the difference between the top-level players is something you see in competitive games a lot, uh, even today, is you get the sense of the big fish in the small pond, you know? It's like, "I am the... I totally beat all my friends' asses," and, or, "The best, I'm the best player anybody's ever seen in my tiny little area." And then you put them in the big pond, uh, with some of the professional players, and they just, they get nothing, you know? They wind-

    16. JR

      Yeah.

    17. JC

      ... up not being able to land a shot. There is that much difference. Of course, you see that in everything, martial arts, where you get the, the dojo hero in one place that then goes in and actually rolls with a professional and just finds out that they weren't all they thought they were. And there's even more layers of that in games because you're not so confined to some of the physical limits of the human body.

    18. JR

      Yeah, and the amount of time you can do it is not confined to the physical limits of the human body. So, there's people out there playing 10, 12 hours a day, where they're thinking and sleeping and dreaming-

    19. JC

      Mm-hmm.

    20. JR

      ... and, you know, catching people with rail shots in the middle of the air.

    21. JC

      Yeah, that sense when you're obsessive about something, how it does invade your dreams, and there have been a number of times in my career when I'm learning new things, when I'm just immersed in whether it's a new programming paradigm, a new piece of technology, and I'm working 13-something hours in a day, and I go to sleep, and I have dreams about what I'm working on. That's when I know I'm really deep in the groove of soaking in this new information. And the dreaming is my mind helping synthesize this into a useful form, so I can apply this in the future. And those are some things that I, I look back on very, very fondly, when I've been that obsessive about something that it's soaked into my dreams.

    22. JR

      Yeah, I used to get that with martial arts. When I was competing, I would, I would throw kicks in the middle of the night, and I would have, like, dreams where I'd be, like, moving, and I'd wake up, like, you know, and, like, thinking that I was in the middle of a fight. And I had a real problem when we...... you know, we have this, uh, LAN room set up here with Quake on it.

    23. JC

      (laughs)

    24. JR

      And when we got into it, where my addiction got re-sparked again, we were playing two, three, four hours a day-

    25. JC

      (laughs)

    26. JR

      ... I was starting to have Quake dreams. It was really weird. Like, I'd have dreams that I was going down corners and, and dodging rockets, and it's just... That game is so immersive, and it's so, um... You, you get done with it, and your heart is pounding. Like, me and Jamie and, and Jeff would play, and then when, when it was over, we would all be out of breath. We'd all... Our, our, our r- heart would be racing. We'd have to get up and walk around. We'd be like, "Jesus," try to shake it off.

    27. JC

      So, that's one of the real interesting things. That gets amplified even more in virtual reality-

    28. JR

      Yeah.

    29. JC

      ... where, uh, when I was doing a bunch of the work on bringing Minecraft into the Gear VR system, I, uh, I played a bunch, and I would play with my family. And it's, you know, it's great when you can have your kids playing with you. And I would have the VR headset on. I'd be playing in my swivel chair, turning around, uh, running, and I had this really weird sense where there were times that I would remember that not as I was playing a game, but I remembered being there. I remembered, like, being in that cavern and breaking through into, uh, into a raining sky, or, uh, you know, having the Creeper run up behind me. And it was not the sense that I remembered playing a video game. I remembered being there.

  7. 27:04 – 30:22

    VR locomotion and simulator sickness: inner ear conflicts and design trade-offs

    1. JR

      Mm.

    2. JC

      And that's... This, that's the power of VR. And in many ways, we don't have as intense of action games, like Quake, in VR, partially because there's a movement control problem, where, in VR, you're fine as long as you're moving in the virtual world the same way your body is in the real world. Uh, but if you go ahead and played a traditional game like Quake, and you were moving your mouse to spin you around, you would get really sick to your stomach quickly, because, uh, parts of your inner ear, uh, detect motion of your physical body, and your brain correlates what your eyes are seeing with what your inner ear is telling you. And when they disagree, that's what causes simulator sickness. Now, some of the theories are that that's your brain saying, "You ate something that's really bad for you, and you should get sick and throw it up." Uh, I... But this idea that you can't play traditionally like that in VR, unless you've just got an absolute iron stomach. So, uh, things like that, the, the dashing down corridors, flipping around, uh, in corners, you don't do very well in VR. But things where you can either stand still or move sort of in a straight line, as long as there's no acceleration. The worst things are sort of parabolic arcs. So, rocket jumping and stuff in-

    3. JR

      Mm.

    4. JC

      ... Quake is bad news in VR.

    5. JR

      (laughs)

    6. JC

      It's still pretty amazing when you go do it, but you do it too many times, no way you're doing that for a three-hour stretch there without having some problems with it.

    7. JR

      So, what you're saying, in terms of the inner ear and y- the visual, uh, w- what, what you're taking in visually, is that when those things are off, your brain thinks maybe you consumed poison, and that's why they're off?

    8. JC

      Yeah, that's one of the theories, because it makes your stomach upset.

    9. JR

      Yeah.

    10. JC

      And people do throw up sometimes.

    11. JR

      Yeah.

    12. JC

      If you've been, uh, if you've... People used to do that even just with traditional video games, where I remember in the early days, uh, in the early days of Quake and Doom, some people would be staring intently enough at the screen, and they would get that sense where, on a small monitor, it's usually not a problem, because, um, usually you see the rest of the stable world around you, and the field of view that you pick, like, traditionally, we used about a 90-degree field of view in the old days. And if you have a little monitor here, it's really only taking maybe 25 degrees or so of your field of view. So, your brain doesn't really buy it as, "I'm looking into a window in this other world."

    13. JR

      Mm.

    14. JC

      You're still kind of controlling something and driving it around. But when you get a giant screen and you get the field of view about right, then your brain starts picking up and saying, "Oh, maybe I should be paying attention to these visuals," and that could cause some people to have, uh, you know, get simulator sickness even on a screen. But in VR, where it's covering your entire field of view, and you don't have a stable reference, it can be a much bigger problem. So, we, we have some of these Band-Aids where people start, would put cages of things in virtual reality, so you've at least got something stable to look at, and that helps a little bit. Like, there's, uh, a game called Omega Agent where you're flying around in a jetpack. And on the one hand, this is the worst thing to do in VR, parabolic flight. It, it checks all the "don't do this in VR" buttons. But it's still really amazingly cool to just jetpack up and kind of coast around, down things.

    15. JR

      Mm.

    16. JC

      So, sometimes you're making these trade-offs with your body on the VR experiences. Uh, we try to push people towards the no-trade-off games. You know, "Okay, you can sit here. You can do this amazing thing, and there's no downside to it." But for the adventurous, there are these other things you can do which might be exactly what hits the right buttons for you, but, uh, you may suffer some for it. (laughs)

  8. 30:22 – 37:27

    Safety systems: Guardian boundaries, passthrough view, and mapping the real world

    1. JR

      Do you guys take into account the possibility of people getting sick and suing? 'Cause then, I think you kinda have to, right?

    2. JC

      So, there's a lot of, uh... What we were more concerned about, health-and-safety-wise, is, on the new system where you can walk around-

    3. JR

      Oh.

    4. JC

      ... is, you know, people banging into walls, falling down steps, and so on. So, we spent a lot of effort building this, we call it the Guardian System. When you start up, you basically trace out what you say the safe area is. And that's on you. If you trace out the safe area over your staircase, um, you know, bad things can happen. But the idea is that, um, it has a known good area. It knows where you are. So, it will tell you. Uh, it'll bring up this little overlay to show you that, "Hey, you're approaching a wall. Do not be charging headlong." Now, people should not be sprinting in VR under almost any circumstances. But you do wind up getting into the action on things, and something jumps, it's behind you, startles you, and you jump around. So, you need a reasonable amount of space to do these, we call them room-scale games, where you're actually walking around. There's a lot of them that are stationary, that you're expected to pretty much stand still. But there's some amazing experiences about being able to clear out a room, walk around, and just not be on task as far as hyper-aware, but just being able to soak in a space and kind of crouch down and look at things. And I always like to say that modern games have so much artistry put into them that is largely wasted or lost on the players, where you have the fire extinguisher sitting in the corner that some artist lovingly crafted for a week, and you just glance at it, see the pixels, and move on. But in VR, you wind up saying... It's like, "Oh, I'm in this room. I'm looking at this." You re- you know, lean down, or even pick something up and turn it around and look at it. And, you know, and that's really great. But the, th- the larger spaces have these hazards, and it turns out that...It's hard to protect people from themselves in some cases, because we started saying, "Well, what if somebody's swinging their hand really hard?" I am... You know, we don't... We want them to be able to stop their hand. And even if we show when the hand is really close to it, they're not gonna have time, so we put in this predictive filter to say, "Based on the velocity of how fast you're moving your hand, maybe we'll start showing up the boundary a second earlier. If we think you're gonna hit it in a second, we'll show the boundary now." But then everybody says, "Oh, I don't want that flashing up. It's distracting me," and we need to put in parameters to turn it down. And it's a real balancing act there, where a lot of developers just kind of go in and say, "Turn off Guardian. Uh, I'll accept the risks for this," but I-

    5. JR

      Oh, wow.

    6. JC

      Yes, of course, the lawyers are very concerned for this. We have, uh... And we have some duty of care to the people that are our customers. Uh, there are some, uh, interesting YouTube videos of people playing VR, not setting things up right, and likes running into walls and so on.

    7. JR

      (laughs)

    8. JC

      And even internally, it's, uh... If you don't set it up right, if you, uh... If you extend it further than you should, there are people that have smacked their knuckles into a table or something, and kind of post the bloody knuckles picture from that-

    9. JR

      (laughs)

    10. JC

      ... but it's on you on how conservative you want to be with it.

    11. JR

      It was interesting when I stepped outside of the... What is it called, the guardian...

    12. JC

      Yeah.

    13. JR

      ... uh, whatever it is, uh, the, the, the matrix that shows up, and you, you walk through, then you see the regular world. Like, you step out of the virtual world, and you're into the regular world, but you're seeing it through this weird lens.

    14. JC

      Yeah, so it's... I... It's black and white because it's using the same cameras. Like, on the Quest, there are four cameras that, um... that are pointed off in different directions, and they have two jobs to do. They, they track where the headset is by looking at the world around it. They find stable little pieces of the world, and based on where they are as you move around, it can figure out where the headset is. And then they also look at the controllers, and the controllers have little invisible IR LEDs on it. Uh, but it takes this black-and-white image of the world from the four different views, and it can pass through the real world. It's pretty low-res, uh, doesn't have color, but it's good enough for important things like finding where you put your controllers and letting you orient yourself. Because depending on what you do in VR, if you spend a couple hours in an en- uh, environment where you're turning around and moving, it can be very shocking to people when they, uh, sort of come out of it, and they... they're in the opposite corner of the room they thought they should be, uh, they're pointed in a different direction. Uh, having the ability to kind of bring the world into place, especially in those cases where you're approaching the boundaries is, uh, it's nice.

    15. JR

      Is there anything that you guys have that can, or maybe in the future can, map out uneven terrain? Like, say if you were at a park, and there was a hill or in your backyard or something like that where you have var- various s- sort of surfaces. You have a sidewalk and then grass and then a hill. Is there ever going to-

    16. JC

      So there's, there's a few aspects to that. Uh, you can set up really large, uh, guardian boundaries. We do have an upper limit on it, uh, but again, some developers have disabled that to do even larger areas. And one of the coolest experiences that I had was at a convention last year. An artist had made a... basically a VR sculpture that was really big. It was like 50 feet long, uh, and you were inside this kind of aquarium-looking thing in a workshop. And we sketched out this giant boundary, and I walked around. Like, I walked literally from room to room in virtual reality. Again, very few people have the space to set something like this up in their house, but we were at a convention center. And the ability to walk through a door in virtual reality, get down on my hands and knees and crawl through a crawl space into another magical little area, that was really something. I mean, it's not clear how we can carry that over to other people, but the idea of doing it outside, uh... There's a few technical issues with it where bright sunlight overpowers, uh, the little tiny LEDs on the controllers. So while it's possible to sort of stop down the exposure on the cameras for tracking the headset, if it's reasonably bright, you're not gonna be able to resolve the controllers, which breaks some things. But people have found that if you get the right overcast day, uh, and you've got the right environment, you can go out to, uh, like a tennis court or, uh, or a big field or something, stretch out a large boundary, and explore some fairly sizable things. Now, really accurate, uh, determination of the world, Facebook Reality Labs has done a lot of research for almost what is the absolute limit of what we could possibly do with the sensors for building the most accurate representation of the world? And it's pretty damn good when you get to the point where they could take even a really cluttered room like this, scan the world for all of this, and they could make, uh, real 3D representations so you could then be good enough that I can go ahead and pick up individual things here. But it's still... Uh, it's pretty expensive to do that both in terms of they need some slightly different sensors, they have to project more structured light out at the world, takes a lot more calculation to do it. It's too much for this generation of products here, but certainly that's something that we're looking at in the future, where eventually we want to not have that one step of drawing out the guardian. You just want it to be both, um, sensible enough that it can tell what's going on in the environment and sort of smart enough to tell what's a hazard, because you want that magic of you just put it on, it does everything, and it just works. We're not there yet, but that is sort of on the roadmap for where we wanna go.

  9. 37:27 – 43:43

    AR vs VR: ‘passthrough’ practicality, Magic Leap hype, and enterprise reality

    1. JR

      So you think there will be a time where the, the, the technology will allow you to maybe possibly have several filter layers? Like, you can see the whole world, and it, it would be more of an augmented reality toward of a s- sort of a situation, or the whole world disappears, and then it could be virtual?

    2. JC

      Yeah, so there's... We have a lot of debates about, um, both the useful things that you might do there and then some real technical aspects where, in terms of augmented reality, this idea that we all buy into this future vision of a world where you've got something that's this... you know, that feels like sunglasses that you put on, and you can pull up all your information, and maybe it clouds over into a virtual experience. There are still, uh...... fundamentally unsolved problems in display technology to do the magic thing that we would really like from all of that. So you step back to saying that, "Well, what you saw in Quest when you saw the world through that, obviously it's low res, it's low frame rate, it's not good." But we could fix all of that. You could say, "Let's go high res color, high refresh rate," and you could make a, we would call that a passthrough rather than a see-through augmented reality system. And we could absolutely build that technology and we could make that pretty good. But then it comes down to what do we expect sort of the user story to be? If you had something like that, uh, would you be wearing this boxy thing, uh, you know, out into the world, riding the bus with it, doing different things? We have a little bit of a hard time seeing the kind of socially acceptable way that you're running around going about your life with sort of this shoebox-sized thing on your head.

    3. JR

      Mm.

    4. JC

      Uh, it's an open question where everybody agrees if it came down to sunglasses, everybody in the world's gonna want something like that if it gives them these magical abilities and it can turn into virtual reality or augmented reality. It's an open question whether there's an in-between layer. If we get down to the point where it's something like swim goggles or very thin, um, sort of ski goggles, something that's half or a quarter of the volume of we- what we've got here, would that be something that people would wanna wear for long periods of the day? I lean towards no, but we haven't built it, so we don't know yet. Um, and then the question of what you wanna do with that in the augmented reality world, where people make these interesting little demos where, "All right, we've mapped the world in this incredible detail. Now we can flood it with water. We can do a simulation of all of this. Isn't this cool?" Or we can turn it into, uh, we can reskin your world as Bilbo Baggins' hobbit hole or something. And I am skeptical of the broad utility of a lot of these things where, like today, there's a ton of AR apps that you can get on your cellphone. You can hold your phone up and kind of look at things and interesting little things happen.

    5. JR

      Pokémon.

    6. JC

      Well, Pokémon Go is an interesting thing where that actually has more of a point for it, and the augmented reality side of it is very small. But the things that actually augment onto the world, I haven't seen anything that I, that I found really compelling there. Interesting technologies, but I think that the, um... I'm still betting more on the fully immersive experiences where you take over, and this idea of bringing part of it into the world, I'm a little more skeptical on, but I, I don't know how it's gonna play out.

    7. JR

      But that's the concept behind Magic Leap, right?

    8. JC

      Yeah.

    9. JR

      And Magic Leap was very, very hyped up a couple of years ago, but it hasn't really come to fruition yet in terms of like a consumer product that people could... I mean, remember they had the little girl and there was a ballerina that was dancing on her hand-

    10. JC

      Yeah.

    11. JR

      ... and it was like, "Is this gonna happen?"

    12. JC

      So the problem with Magic Leap was they, they had... A lot of the augmented reality videos, uh, you wind up with things that are synthetically created, and they're not really what the product does, and they oversell-

    13. JR

      Right.

    14. JC

      ... what they, the actual capabilities are. And that's a slippery slope there where you wanna sell your vision in some way, and you know, you're really, you're very rarely showing exactly what the product does, but they showed something that led people to believe it was much more than it actually was. But even before Magic Leap, Microsoft had been selling the Hololens for a few years, and it's turned out that they have some real wins in, uh, in some enterprise applications for training and for, uh, people like working on jet engines and stuff where it's an expensive product. It costs thousands of dollars for these things. Uh, you're not gonna spend that to get a... It's got a fairly narrow field of view. It's not a great gaming experience. They have a few things that you can do with that, but if it's something that helps you do your job and you're a high-paid specialist for something, it's a tool. You know, it's something that, uh, that has value there. But that's the state that we are in with AR today, where there are some devices that are providing some legitimate value, but they are essentially, they're like socket wrenches. They're a tool you get when you need to do a specific job, where the world that we wanna be in is it's something you put on and it's essentially a part of you, where I want something where I roll out of bed, instead of grabbing my normal see-through glasses, I grab my augmented reality glasses, and then they do something useful for me for 16 hours. And that's where, you know, two hours is not enough. We have a few of, uh, there's a few companies that have made things that are sunglasses form factor that can do some little things with a display, but they run out of battery very quickly. But you want something that you can use all day long. And if it becomes something that is so automatic, like cell phones are augmentations of our power right now, the fact that we can go look anything up, uh, super quickly. If you have that ability and it doesn't even mean pulling your phone out of your pocket, if you can access information, uh, just by potentially even just thinking about it. I mean, there is work, serious work going on about brain-computer interfaces where you could imagine having these glasses, and even if it was all it did was, say, zooming in, say, it was just supervision, if you had the ability to just think zoom and it would zoom in in, for something, that would be a product right there. It could grow into this augmented world and, uh, annotate everything and do all of that other stuff later. But I'm a believer that this, uh, sense of giving you a new power, giving you a new physical ability, and either it needs to be ultra intuitive, I mean, maybe there is something where, um, just tapping your wrist or some super, super fast low-latency interaction, but ideal would be something like, uh, you know, brain-computer interface. Maybe it's some eye tracking or even teeth clicking or something that you could do, but if it just became so natural that just like you think you turn your head to go look at something, if you just do this thing and you zoom in on it and you get more information, that would be pretty magical, that sense of augmenting human capability.

  10. 43:43 – 47:44

    Brain-computer interfaces: Neuralink visit, disability applications, and AI competition

    1. JR

      Have you paid attention to this Elon Musk Neuralink thing?

    2. JC

      So I actually went out to visit the Neuronl- uh, the Neuralink company, uh, it was like the week before they did their big public announcement and I spent a whole day there kind of deep diving with a bunch of their technical people. And it is exciting stuff because it is... (laughs) You know, I, I like to use the word bold for things like that where it's not just this incremental advance where it's just taking something, fixing a few of the flaws, and going out. It is visionary, looking to the future, where the potential upside of this, about being able to make...... the whole automated electronic world, uh, something that is directly accessible on both input and output to your brain, is really remarkable. And, uh, what they've done, I mean, there's a lot of people where you can say, it's like, "Well, other companies might have these capabilities," uh, but clearly they are at the cutting edge. Uh, you could argue how far above or, or behind other companies they might be. But I think it's going to largely work, to some degree here, uh, and they're gonna be working with people that are like seizure patients, people that are, uh, in many cases, profoundly disabled. And they don't expect it's gonna work forever when they put it in. There are all these problems with the brain, uh, kind of rejecting implants eventually, and there are ... People have done experiments like this for quite a while. It starts off with just one neuron. You stick in one sensor, uh, or one actuator, and in the Neuralink case now, they've got tens of thousands that they can put in, so much, much higher fidelity. Uh, even if it doesn't work forever, and they're still working out the different, uh, the codings, tactics, different installation procedures, uh, somebody's going to go in there and be, go from being profoundly disabled to probably being able to play a video game, you know, being able to sort of directly control things with your mind, where you start off being able to do maybe just very slowly driving a cursor. And people have done this, again, for years, where you can slowly move a mouse cursor or something and figure out a click thing. But here, when you have tens of thousands of neurons going in, you could go from this very slow moving something to doing this deep analog multidimensional, like playing a symphony with your brain output potentially.

    3. JR

      Mm-hmm.

    4. JC

      And then potentially feeding information in in a way that we can't right now, that you could have this sort of tactile feel to it.

    5. JR

      Do you remember Lawnmower Man?

    6. JC

      (laughs) Yeah, vaguely. I-

    7. JR

      That was the idea, was that the guy was-

    8. JC

      I've thought my- Yeah, I've thought to myself sometime I should go back and watch that, because I literally have not watched that since it came out when I was a teenager. And the VR is laughable at this point-

    9. JR

      Yeah.

    10. JC

      ... and everything, but it should be good for a chuckle.

    11. JR

      Well, um, it was a clun- I think it was a clunky movie. But the idea was based on a Stephen King book, and the Stephen King book where there was a guy who was sort of slow, and they did something to him, and all of a sudden he became some super genius, almost godlike character. Like, if this Neuralink stuff does work and you can take a person with profound disabilities-

    12. JC

      Mm-hmm.

    13. JR

      ... and all of a sudden, they become the smartest human beings on the planet, that would be really weird.

    14. JC

      So, I know that was one of the pitches that Elon was making early on, that, uh, Elon is very concerned about artificial intelligence, and-

    15. JR

      Very.

    16. JC

      ... part of Neuralink was this pitch that, "Well, maybe we can supercharge humans in a way that the AI won't run away so far, or we can at least interact with them on a more level playing field." I am less, uh, sanguine about that. I am, uh, that seems a little bit more of a reach to me, because I suspect that, okay, even if you do put a million neurons in, uh, when we're making artificial general intelligences, they're gonna have billions and billions of these different connections. And I think that it might be many steps above what a human could be, but if AI becomes possible and takes off, and I, I am a believer in artificial general intelligence. I think it's probably not as far off as many, uh, many people believe, that it is likely to be able to accelerate and advance faster than even a neural-linked human would be able to.

  11. 47:44 – 53:48

    AGI timelines, supercomputers, and quantum computing’s limited upside

    1. JR

      How far away do you think we are from artificial general intelligence?

    2. JC

      So, by nature, I'm an optimist. I'm, you know, I tend to, uh, to underestimate how long things take. But on the other hand, as a programmer, I've usually been able to say, "Well, maybe I missed my estimate by 50% while everybody else blew it by 100%" or something. I think that we will have, we will potentially have, um, clear signs of AGI maybe as soon as a decade from now.

    3. JR

      Wow.

    4. JC

      Now, lots of people disagree. The majority of scientists, uh, working on it think it's like, "Oh, it's gonna be at least a few decades," and you still have a few holdouts that say, "Oh, it can't happen at all." But I'm a strict materialist. I think that, you know, our minds are just our body in action, and there's no reason why we can't wind up simulating that in some way. Um, so I don't think ... The question of how far off, there's a lot of numbers that you can play. Like, the, the brain has something like 85 billion neurons in it, and th- they have something like 10,000 connections between it. And you can multiply those out and compare them to what we have in computer memory and processing time, and you can say that, yeah, within 10 years, those curves should have crossed. But I would even go so far as to say most of those are probably not, uh, completely necessary. We know lots of biological systems. Like, we understand the processing that goes on a lot in the visual side, and we don't need nearly as many computer transistors as, uh, neurons that are used for processing some of those early layers. So, I suspect that even today, some of the government supercomputers, the, the biggest, the top 500 list that they have, those are remarkably, uh, probably useful for doing intel- artificial intelligence work. Where for a long time, for decades, I thought that was sort of just, uh, national chest thumping, the top 500 computers, because so many of them, they relied on replacing what used to be the old big iron Cray vector supercomputers, and they really weren't very easy to program. Most programs people wanna use, you can't run it on a supercomputer and just be a lot faster. One of the shocking things most people don't really appreciate is the fastest way to do most single threaded applications is an overclocked gaming computer today. You can't go spend a million dollars and buy a computer that will do many tasks faster than what you can just run on a gaming computer. And this is not at all the way things were for decades, where for a long time, you would go spend your millions of dollars on a Cray supercomputer and all of your code would run faster than anything you can get. But it turned out that the, uh, the processors that you wind up using in high-end gaming systems are, in many cases, the fastest, or in, in all cases, at least close to as fast for certain serial applications. So, the only thing else you can do is pile lots more of them together. And these big computers are football field sized, uh, systems that are just racks and racks of GPUs and CPUs and nowadays, for a long time I'd be like, "Well, what would you..." I would think, "How could I make a faster Quake map build or something on one of those?" Because we would sometimes have hours and hours spent processing this, and, um, at one point we had a computer that was almost in the top 500, um, at id Software, just for making our maps. Uh, but I looked at a lot of these supercomputers, I'm like, "Ah, these are terrible. Not very useful for what we want." Um, but now as I look at AI work and I think that, "Well, if you're just doing a whole bunch of these, uh, kind of general matrix multiplies, that computer right there is probably pretty good." So-I would suspect that you could do something, if we had the right algorithms, the right training schedule, and the right time to run through it. That it's probably possible on some systems today, and it'll just still take many years for the right algorithms to wind up being developed. Uh, the right training regimens to be run, and faster, cheaper hardware to wind up making it more economical to run all the experiments. Because in so many cases, the trick is not that the minimum requirements exist, but that 1,000 people have thrown themself at the wall of a problem, most of them have bounced off and failed, but eventually somebody gets through.

    5. JR

      Now, what about quantum computing? Is that something that could potentially break the bottleneck that we have with Moore's law?

    6. JC

      So, I don't... I am not an expert on quantum computing, and I, I think that I... Many times I beat myself up about it, where there are some simulators online where you can go and work on it, and I should work through the exercises of doing the basic factoring algorithms on quantum computing. But I... You know, my read on it right now is that it's probably not directly useful for most of the artificial intelligence tasks. The big things that people worry about that are, um, for things like breaking cryptography, uh, breaking the different hashes and encryption methods. That it's possible that... In many ways, that's almost a terrible technology, because it's a technology that doesn't solve so many of the problems that you'd like it to solve. And it does solve one of the problems you kind of wish nobody was able to solve (laughs) . I... So, I-

    7. JR

      What's that?

    8. JC

      By breaking all the encryption.

    9. JR

      Oh. Oh, okay.

    10. JC

      You know, like if somebody winds up, uh, with a quantum computer that they achieve quantum supremacy and it runs past all of our traditional computers, and all of a sudden they can break all of the, all of the secure socket layer stuff, break everybody's signatures, impersonate any public key sign stuff, there's no upside to this. That's all downside-

    11. JR

      Right.

    12. JC

      ... and all bad things that are gonna come from that. While it's not gonna make your video encoding go any faster, and it's probably not going to help artificial intelligence in many ways. So, I haven't found a whole lot to get me really excited about quantum, uh, computing. It may just be that... Uh, and with all these cases, why I beat myself up about not learning more about it, because in most cases when presented with some capability, there's some way to figure out how to apply it usefully to the things that you really want. In fact, I consider that almost the essence of engineering. Engineering is figuring out how to do what you want with what you've actually got (laughs) . And if somebody gives... Anytime somebody gives me new hardware, usually I can figure out some useful way to do things that I want with it, even if it's not immediately obvious. And maybe quantum computing plays out that way, but, um, it is still definitely the domain of big labs with cryogenic cooling and all that stuff. So...

    13. JR

      Mm-hmm.

    14. JC

      It just hasn't been at the top of my radar.

  12. 53:48 – 1:18:58

    Encryption futures, implants, and the reality of phone/social-media addiction

    1. JR

      Now, when we talk about technology and you talk about the exponential increase in the powers of technology, is it possible that we could come to a point in time, somewhere in the future, where there's no way to encrypt anything? Where it's not possible to hide things? Where we won't be able to do banking online, we won't be able to have digital currency, because virtually everyone will have access to all the information? Because essentially, digital currency, or anything that's encrypted, it's just information. Right? It's just ones and zeros. Is it possible that technology will reach a point in time where borders and boundaries are impossible?

    2. JC

      So, uh, one thing that a lot of people don't appreciate about cryptography is there's a really straightforward way to make completely unbreakable cryptography, and that's what's called a one-time pad. Where if you essentially have, um, you know, a long, you know, a very long set of data and it's private, as long as nobody else has it, you can encrypt anything with it, and if it was generated randomly properly... Uh, you always have to worry about flaws in your random number generation or your random number source. But a properly generated one-time pad is unbreakable. Now, the problem is it's finite, so you have a fixed amount of it. And all of the really serious spy craft would use something like that, where you've got a one-time pad, you can send a message through it. In the old days when you were manually doing it, you might only have a, a book with a certain number of pages, and once it's over, it's gone and you can't get more without returning to base. But this is always a possibility. And as we've seen storage densities increase so much, the fact that you can get a little micro-SIM card that's holding hundreds of gigabytes now, which is pretty remarkable, you could imagine a world... Like, if we did have this quantum apocalypse where all of these short, 512, 1,024 bit keys, whatever, all of those just get smashed irrevocably. Uh, you could imagine a world where, I mean, heck, maybe people start implanting the one-time pad inside people so whatever, you know, whatever you need to encrypt that's coming from you has this, uh, you know, this clear, unbreakable key that you're working with.

    3. JR

      Do you think that we're gonna have things implanted in our bodies soon with- that allow you to interface with computers or technology or wireless internet?

    4. JC

      I think that it's possible that it will. Uh, it would... We have people that wanna do that right now. In fact, that was-

    5. JR

      I saw a woman, d- she implanted a Tesla chip in her arm-

    6. JC

      (laughs) .

    7. JR

      ... so that she could just get close to her car and the door would open.

    8. JC

      So, in fact, one of the things that, uh, talking with the Neuralink people, the idea that... Of course, right now, you start off, you say... You take somebody profoundly disabled and you put them in a laboratory and you try to train them how to use this. But we were all saying, like, "Well, what you really need is a programmer to get this interface. You need to be able to let a programmer actually program themselves on their interface and you will make 100 times more progress than this, you know, previously disabled person coming into the lab for a couple hours a day." And it was funny, the conversation there, where, uh, one of their guys was like, "Yeah, we'll give them the basic rules so they don't stroke themselves out." I'm like, "Okay, yeah, that's kind of important." Talk about health and safety rules there.

    9. JR

      Mm-hmm.

    10. JC

      But, uh, yeah, if you start getting a programmer in there that starts running this, so like, all right, instead of just going through these basic exercises they run everybody through, you really understand exactly what you're doing and you change it, you write the code as you're experiencing it. And there are probably people volunteering r- that are ready to go do that, to have something like that. I read an article sometime after that about one of the early, uh, neurosurgeons that did implant himself with some electrodes. He had to go to one of these, uh, fringe countries that didn't have any, uh, ethical guidelines around the, the medical practices or whatever. But he paid a neurosurgeon in one of those countries to implant an electrode into his head, and he even had some complications afterwards. It was like, no, there's a dedicated researcher.

    11. JR

      Sure.

    12. JC

      Although interestingly, there- there's a whole history of a lot of medical science where you would get people that would wind up having the conviction to do the experiments on themselves. And, you know, you've gotta respect that. Where it's one thing to, you know, make a grant proposal to set up a study to do all of this, and it's another one to say, "Damn it, I am so confident in this, I'm gonna have someone cut a hole in my skull and implant this-"

    13. JR

      Mm-hmm.

    14. JC

      "... in me so we can (laughs) learn the lessons."

    15. JR

      What was his complications? Do you remember?

    16. JC

      Uh, he had some, uh, some problems with speech afterwards.

    17. JR

      Oh, Jesus.

    18. JC

      He had to learn how to do that. Like, half of his face had a little bit of a sag to it from the- putting it in there.

    19. JR

      Oh my God.

    20. JC

      So yes, it was life-affecting changes as a result of going with that. And this was-

    21. JR

      Do you remember his name?

    22. JC

      I don't offhand. I, but it w- this was, it was a Wired article, I think, from, uh, a number of years back. But this was, like, a single electrode, and it was just doing very one-bit or one-analog value computation, and he had a little transponder kind of put in under the skin of his skull, had a big lump on his head, uh-

    23. JR

      Mm-hmm.

    24. JC

      ... with that. While interesting, again, the, the Neuralink stuff, all modern high-tech where you kind of power it with RF through the skull, and it's got a little plug. And I, one of the first, when Elon first kind of approached me about, uh, kind of talking with them about that, the idea and the thinking, which was kind of insightful, was this idea that the IO levels that they were doing on the Neuralink, or they were planning on doing with that, was fairly close to what we do on virtual reality, where, okay, we've got theoretically maybe up to a million inputs here and a million outputs. And I can run those numbers and say, "Well, that's kind of like the cameras that we're taking in and the display that we're, uh, we're putting out." And I made the point that, "Well, you probably could run that off of, like, a Qualcomm chip that we've got in here." You'd set it all up as, like, turn them into what are called MIPI lanes for the input and output, make the inputs look like a camera and make the outputs look like a display screen, and you could then run software on something like what we use here to drive your brain. Like, the programmer could then kind of start running some of those experiments with it.

    25. JR

      Well, it's so fascinating being on the outside, watching all this stuff come to fruition, because I remember when virtual reality was sort of, it was something that was tossed around in the '80s. And we, we talked about the future, and that was one of the things that people were really concerned about or looking forward to in the future, was virtual reality. But the technology really wasn't there.

    26. JC

      Yeah, so I have stories from, in the early days I did software, every game from Wolfenstein, DOOM, and Quake, we had at least one VR entrepreneur that wanted to work with us, uh, because finally here was content. Because they were like, everybody had this vision of VR, like it's this hazy vision of the future where cool stuff happens when you put the helmet on, but they didn't have this concrete, uh, instantiation of like, well, what do you actually do? And then people saw the 3D games, like, "Oh, that's what you wanna do inside the virtual reality helmet." So, they would come up, and they'd wanna basically work with us, license the technology, and every time I looked at these, I'm like, "Oh, this is just, isn't gonna work out." I, in many cases, they were people that were very, they were, you know, high on enthusiasm but a little low on the raw technical necessities to make something like this happen, because really it was too early. I mean, there were systems that if you had unlimited money you could get a big SGI refrigerator-sized infinite reality system, uh, that could render images that could be interesting. But people talk about the current resolution limits here or, like, the original, uh, the original DK1 kind of the views for there. Those headsets back then, they were the resolution of, remember the original pixel DOOM? Uh, like 320 by 240-

    27. JR

      Mm-hmm.

    28. JC

      ... sort of resolutions there, but wrapped around your head. Somebody called them football-sized pixels. You know, like you've-

    29. JR

      (laughs)

    30. JC

      ... got this big blurry blob off to the side. So, it was never gonna happen in those early days. So, uh, we always made the point of saying, "Well, okay, we'll license the stuff to you, but, uh, (laughs) you know, I wouldn't put a dime of my investment money into something like that," because I just thought it was too early. And then here we are a couple decades later, and the future has arrived.

Episode duration: 2:36:46

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode udlMSe5-zP8

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

iOSAndroidClaudeChrome