Skip to content
Modern WisdomModern Wisdom

The Future Of Virtual Reality | Dr Sarah Jones | Modern Wisdom Podcast 110

Dr Sarah Jones is an immersive storyteller using AR, VR and 360. For the last 50 years or so, storytelling hasn't massively changed its form, but with the advent of increasingly portable devices, the platforms we use are all changing. How will Virtual Reality enable us to tell stories in a new way? Should we be worried about the control that these technologies could have over us? Are sci-fi predictions about holographic humans realistic? - Extra Stuff: Follow Dr Jones on Twitter - https://twitter.com/VirtualSarahJ Check out everything I recommend from books to products and help support the podcast at no extra cost to you by shopping through this link - https://www.amazon.co.uk/shop/modernwisdom - Listen to all episodes online. Search "Modern Wisdom" on any Podcast App or click here: iTunes: https://apple.co/2MNqIgw Spotify: https://spoti.fi/2LSimPn Stitcher: https://www.stitcher.com/podcast/modern-wisdom - Get in touch in the comments below or head to... Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx Email: modernwisdompodcast@gmail.com

Chris WilliamsonhostDr Sarah Jonesguest
Oct 10, 20191h 3mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:00 – 1:37

    Immersive storytelling vs. “story living”: what Sarah’s PhD actually means

    1. CW

      (wind blowing) I am joined by Dr. Sarah Jones, and before the podcast we've been nerding out on all of the different gadgets and stuff that we've got floating around our house, so that is going to be the continuation of today's episode. Sarah, welcome to the show.

    2. SJ

      Thank you very much for having me.

    3. CW

      It's a pleasure to have you on. You have the world's first PhD in immersive storytelling.

    4. SJ

      Correct.

    5. CW

      What's that? What, what-

    6. SJ

      (laughs) Uh, uh-

    7. CW

      ... what does that mean?

    8. SJ

      ... I'm, I'm sure I won't be the, the, the... I won't be the only one for very long. I'm sure there'll be lots more coming up right behind me, um, with regard to immersive storytelling, and I usually say it's immersive story living, um, as a way to kind of define it a little bit more. But basically, what I do is grab any tech that I can and break it to find a new way of having a story. Um, so story living is all around that kind of immersive experience, um, where it's completely rooted in the experience, in the experiential, so therefore you can't really have a told story. It's not really a directed narrative, um, so you gotta break it apart, and then you live it in whatever way you want, therefore it's story living. Um, so yeah, I, I did my PhD, I got my PhD in story living, um, the first one, um, but yeah, it's pretty good.

  2. 1:37 – 2:43

    From TV journalism to VR: removing the screen as a barrier

    1. CW

      That's fun. So, what are some of the examples of, of the, the kind of technologies that you work with?

    2. SJ

      Um, I've always experimented with, with whatever I can get my hands on, really. So my background was in television journalism, so I was a reporter for, like, 10 years, um, traveled all the way around the world telling stories, and I always wanted to just get closer to those stories. Um, I wanted to find a way of really taking the audience and putting them in the heart of it. So I was right at the start of the mobile phone movement, um, of mobile phone filmmaking, um, and shooting and editing everything on your phones, um, and I found that that could get you closer, um, but it still wasn't close enough 'cause whenever you're watching anything, there's this barrier, and that barrier's the screen, and it's always in the way. Um, and then when I did my first kind of VR experience, I was like, "Ha ha, this is it. Now I can take you, the audience, and I can plonk you in the center of the story, and you're there, and you experience it then yourself." Um, so from all kinds

  3. 2:43 – 3:12

    Tools of immersion: VR, AR, 360 video, and the pull of holograms

    1. SJ

      of things, from normal film to, um, AR to VR to 360 film, which is obviously the easiest way to experiment, um, but at the moment, I'm, I'm really fascinated with holograms and what you can do with those. Um, and I haven't made anything decent.

    2. CW

      (laughs)

    3. SJ

      Um, but I will. Um, so we... it's just a case of experimenting and breaking and trying to find how these technologies suit different mediums.

  4. 3:12 – 4:48

    Art vs. code: why immersive work demands collaboration

    1. CW

      That's interesting. So, uh, is your... are you coming at this from a tech coding side, or are you coming at this from more of a grand perspective, what can it do, um, what's its sort of capabilities side?

    2. SJ

      I think the latter-

    3. CW

      Yeah.

    4. SJ

      ... um, and certainly more of that kind of artistic side.

    5. CW

      Mm-hmm.

    6. SJ

      Um, I tried to learn how to code probably about five, six years ago. I thought, "Okay, I really need to do this. You know, clearly I'm missing that skill." So I started learning, and then I thought, "You know what? There's people that are brilliant at this. I don't need to kill myself trying to do it. I-"

    7. CW

      Yeah.

    8. SJ

      "... just need to find those people-"

    9. CW

      Especially if it's not in your-

    10. SJ

      "... and work with them."

    11. CW

      ... s- s- especially if it's not in your wheelhouse. Like, there's-

    12. SJ

      No.

    13. CW

      ... s- s- some guys-

    14. SJ

      E- e- exactly.

    15. CW

      ... yeah, some guys who f- who eat that stuff for breakfast, but would find-

    16. SJ

      Yeah.

    17. CW

      ... would find the storytelling that you do, like, just crawling through glass, so you might as well.

    18. SJ

      Absolutely.

    19. CW

      Might as well, might as well double down.

    20. SJ

      Yeah, I mean, I, I, I have a basic knowledge. I, I can understand it. I know what I need to do.

    21. CW

      Mm-hmm.

    22. SJ

      Um, but I, I think what's really interesting about working in kind of technology for the kind of artistic side, the narrative side, the experience side, um, is there has to be a collaboration, and that collaboration means you need that specific group of people to make magic, and not one of those people can do it by themselves. You need everybody there. Um, and that's what's really exciting, really, is I feed off everybody else's mind, you know? I need to be stimulated

  5. 4:48 – 7:16

    Stop copy-pasting old media: designing for the platform (Sky News VR example)

    1. SJ

      by all those minds to make something great, um, so it's quite, quite good in that respect. Um, but yeah, certainly that kind of artistic side, looking at what can we really do with it. Um, I think a lot of the problems, um, and I've been quite critical of some people for, for doing it, is that you get a new platform or new technology and you just apply old ways of thinking to it.

    2. CW

      What like?

    3. SJ

      And that's really... and I, I think, well, it... there, there was a prime example, um, and I always name and shame them as an example is Sky-

    4. CW

      Fine. Throw it out there.

    5. SJ

      Um, yeah, so, so Sky News, um, did their first VR experience, and it was the reporter on the, the migrant crisis doing a story on the beach, and they'd obviously filmed it for a live report first off, um, as a flat, s- flat version for, for traditional television. Um, and then they, um, filmed it with a 360 camera, so they filmed it for VR, you know, 360 capturing everything, you can look around, tilt, and rotate on your phone, whatever it might be, but they did exactly the same thing. They didn't think about, "What can we do differently because we have a spherical camera? How can we tell this story differently? What does it mean from the audience perspective of watching a story flat, having the narrative told to you, or actually being there on the beach?"... how does that change what you do? And I was really critical of them for doing that, and then the next day they announced that they had a new Sky, um, VR studio. Um, so my timing wasn't great-

    6. CW

      (laughs)

    7. SJ

      ... um, but I- I- I hold true to- to that belief that it's just really lazy. And, you know, my background is in journalism, and it always reminds me of when online journalism started. And it was just simply, you know, you get your story and you copy and you paste and you put it online, and then you upload the video. And you don't think, it's a different audience, and they're consuming it in a different way, so they need it to be different. Um, so, so I think that's really where a lot of the mistakes are, so what I try and do is start from that blank canvas and think, "What can I do that I can only do because of this technology?" Um, so it's not really technology for technology's sake.

    8. CW

      Mm-hmm.

    9. SJ

      It's more thinking, we, we owe it to the tech to make it great and to break it and find new ways of using it. Um-

  6. 7:16 – 9:46

    How VR planning differs: design the feeling first (Chungking Mansions case study)

    1. CW

      So, how would you, how would you characterize the difference between the use of something for traditional flat screens and that for VR?

    2. SJ

      Um, I think normally-

    3. CW

      If you were, if you were designing it, if you were the- the- the lady that was at Sky that day on that beach, what- what, how does it look different? What do you do?

    4. SJ

      It, it's about the experience. So, when- whenever I'm planning a shoot on anything, um, it's- it's always the first port of call, whereas in flat you would kind of do a storyboard, you think of your start, your middle and your end. When you're doing something for VR, um, or 360, it's closing your eyes and thinking, "What is it, what's that experience that you're trying to get somebody to experience? What is it about being in that specific place that you need to capture, and then how can you capture it?" Um, so I did something a couple of years ago in Hong Kong, um, in Chungking Mansions, uh, which is this crazy, crazy, huge international kind of market scene. Um, it's the most insane place in the world. Um, totally chaotic. Um, 4,000 people staying there in all these really cheap hostels, like the cheapest place that you can stay. Um, and going in there, you know, it's dark, it's dingy, um, there's so many different smells and noises and everything is going on, going on. So for me, being in that environment, I was like, "I need to capture that." And capturing an experience and a feeling is really difficult to do. It's not like just telling the story, like, you know, "Oh, the smells are so intense and these people are doing this." You know, you can't do that. You've got to put someone in the heart of it. So, so to do that, I broke all of the kind of very early, um, introductory editing conventions in VR, um. I had a lot of moving shots, because I wanted e- people to feel disjointed, um, and uneasy. I had various kind of carrying shots so you were at different levels. I had lots of quick edits, um, to make people feel jarred, um, and that was how you could capture the experience. So it's all coming down to that, really. Instead of what is the story that I'm trying to tell, it's what is the experience that I'm trying to create? And that- it's a real shift in the mindset. It's really hard to try and think in that way. Um, it can be easier in some ways, but a lot harder in others.

  7. 9:46 – 12:10

    No frame, no single narrative: viewer agency and rewatchability in VR

    1. CW

      I suppose, thinking about it, the, w- when you are trying to capture something for flat screen, your position in terms of camera is framing the scene as is best appropriate, whereas for VR, it's literally the point of view. It's what is the optimal point of view. Is that, is that the distinction between the two?

    2. SJ

      Yeah, absolutely. And there is no frame. You know, the- the frame has gone.

    3. CW

      Mm-hmm.

    4. SJ

      Um, you- you always have, like, this- this frame in front of you on- on a flat screen, and then suddenly that's gone. Um, I can make something and have 10 different people watch it in 10 completely different ways, taking different narratives from it. Um, it doesn't always have to be that way, and a lot of people do try and direct the narrative and use cues, um, like lights and sounds and- and force the gaze to different places.

    5. CW

      Mm-hmm.

    6. SJ

      Um, but I tend to argue against that and think, "No, I want people to just experience it." And we all experience things in different ways, um, so let's forget about that and put the trust in the audience to just let them create their own experience.

    7. CW

      Mm-hmm. It's not as if-

    8. SJ

      Um, might be me being lazy, but it works.

    9. CW

      Well, it's not as if, it's not as if you can't watch it again, as well. So I- I got a, I got an Oculus Rift actually off an ex-missus. I got an Oculus Rift, I've sold it now. Uh, but, uh-

    10. SJ

      (laughs)

    11. CW

      (laughs) I got, (laughs) I got an Oculus Rift off an- off an ex-missus and, um, one of the things that I found myself doing was watching the same sequence multiple times-

    12. SJ

      Mm-hmm.

    13. CW

      ... because you can only see however much is in your field of vision, and you know, "Well, uh, actually, I'll go back and I'll have a look, I'll see what happens if I look that way for it."

    14. SJ

      Yeah.

    15. CW

      Whereas once you've watched it flat screen, that's it, right? There's no, there's no more-

    16. SJ

      Yeah.

    17. CW

      ... nuance really.

    18. SJ

      Yeah.

    19. CW

      There's a lot less nuance at least.

    20. SJ

      Yeah. I mean, you can go back and I mean, I- I remember, um, watching Breaking Bad and then watching it so many times having read all the- the different color cues and all those kinds of things. Um, but that's very, very different. Um, certainly in VR, you- you can have different people watch it in totally different ways, um, and that's really exciting. It's really hard if you're trying to direct something and you want the control. So for control freaks, it's awful.

    21. CW

      Yeah.

  8. 12:10 – 14:14

    Oculus Quest and the accessibility leap: untethered VR + Beat Saber

    1. SJ

      Um, but it is really interesting just letting go...... and thinking, yeah, people will experience it however they want, so let's let them. Um, so, so yeah. I mean, what's really interesting, you mentioned it, the Rift, the new Oculus Quest is a game changer.

    2. CW

      Ooh, what's that one? What's that?

    3. SJ

      So the Oculus Quest came out, oh, I think maybe about two months ago, um, which is one of the first real standalone VR headsets. You obviously have the Gear.

    4. CW

      I had a Go. I had a Go. Sorry. I had an Oculus Go, but that was it.

    5. SJ

      Yeah. Oh, okay. So the Quest is the next step up from the Go.

    6. CW

      Mm-hmm.

    7. SJ

      Um, so the Quest you've got two controllers. Um-

    8. CW

      Okay.

    9. SJ

      ... and you've got a, a, a space around you. You can play Beat Saber, which is one of the greatest games in VR ever.

    10. CW

      What's that? (laughs) What's Beat Saber?

    11. SJ

      Um, Beat, Beat Saber is like, um, remember the '90s or may- maybe the noughties, um, the dance challenge mats in arcades?

    12. CW

      Oh yeah, yeah, yeah.

    13. SJ

      Where you used to have to jump and, and, you know, your right foot and your left foot and dance around? Um, it's like that but in VR. But you're hitting these things. So, so to watch other people doing it, it's a little bit weird.

    14. CW

      (laughs)

    15. SJ

      Um, but you watch them and you're just like banging to, to different music, those beats.

    16. CW

      Yeah.

    17. SJ

      Um, and it's awesome. It's an incredible exercise. Um, but it's really, really cool. Um, so, so highly recommend anybody playing Beat Saber. Um, and that's probably one of the things that's driving sales of Oculus Quest 'cause everyone wants to play Beat Saber.

    18. CW

      Really? Wow.

    19. SJ

      And now you don't need a huge laptop, um, or PC to run it. It just runs. Um, so I think with that, um, and then VIVE have their own one coming out as well, where again, you don't need to be tethered to a machine. Um, that's all massive, you know. This is huge, really. It might actually allow the tech to be more accessible, um, which is good.

    20. CW

      Yeah.

    21. SJ

      Like that has to be a good thing.

  9. 14:14 – 17:07

    Why VR video looks worse: resolution spread, streaming limits, and adaptation

    1. CW

      Yeah. I think one of the things that I noticed, um, 'cause I've seen floating around for a while, PlayStation VR, um, and when Oculus first came out where I think it was like you needed an Android phone or something to link it up to and other bits. And I was just thinking, it, it just feels quite clunky and it's not really that cool. And then even the Oculus Go, I mean, that's, I didn't pay for it, but even the Oculus Go that I had last year, um, I, I thought was, was cool and I enjoyed watching things. But what I was conscious of, and I, I did a little bit of reading. Um, can you explain to the listeners what happens to the resolution of, of video when you then start to wrap it, it into the, the headset? Because I instantly noticed some of the videos I was watching looked lower quality.

    2. SJ

      Mm-hmm.

    3. CW

      But apparently that was more to do with the, the way that it needs to communicate it to your eyes. Is that right?

    4. SJ

      Yeah. It, it can be. So, um, different, um, headsets, and it'll all be whether you downloaded them or whether you're streaming them and however you're doing it. Um, I always, um, get questions around, you know, "I, I want to make something in 360." "I want a 360 camera." "Which one shall I buy?" Um, and there, there's one camera that I use all the time, um, as a basic entry level kind of prosumer camera. Um, which is the Insta360 ONE X. Um, Insta360 make a great range of cameras, um, really, really great company. Um, and theirs, it now says, the ONE X, is 5.7K. So you think, "Brilliant, this is really good. Like this is gonna be great in a headset."

    5. CW

      Mm-hmm.

    6. SJ

      But those 5.7K pixels are being, you know, if you're watching something flat, it's in front of you in that resolution. We now have to put all of those resolution, th- those pixels all the way around us. So it's the 360-degree image that's made up of all of those pixels.

    7. CW

      Mm-hmm.

    8. SJ

      So it's never gonna be as good as watching uh, a ultra HD television, flat, massive screen, all that kind of thing.

    9. CW

      Yeah.

    10. SJ

      It's just not. Um, but it is getting better. Um-

    11. CW

      I was gonna say, where's the tech at now?

    12. SJ

      I mean-

    13. CW

      Where, what's it progressing to? 'Cause I think, I think the stats when I had a Go were that if you downloaded something, weren't streaming it, uh, had it on optimal resolution, all this sort of stuff, it could get to about 720p. Is that right?

    14. SJ

      Um, no, I think you can go up to 1080 on, on, on the Quest. And it isn't bad.

    15. CW

      Not bad.

    16. SJ

      Like it's, it's good.

    17. CW

      We just got so spoiled with 4K and Samsung-

    18. SJ

      I know.

    19. CW

      ... Super AMOLED, Super Dark Sharp Display.

    20. SJ

      A- absolutely.

    21. CW

      And all that stuff.

    22. SJ

      Yeah. I mean, I would say that after, after a couple of minutes in it, you get used to it.

    23. CW

      Mm-hmm.

  10. 17:07 – 19:52

    Living in VR for 48 hours: embodiment tricks, haptics, and realism vs. cartoons

    1. SJ

      Your eyes adjust to it. Um, and then it all becomes normal. Um, so I, I lived in VR for 48 hours a couple of years ago.

    2. CW

      No way. (laughs)

    3. SJ

      Um, one, one of those, you know, crazy things you decide on a day out you're gonna do.

    4. CW

      How would you go to the bathrooms?

    5. SJ

      Um, I had a five-minute comfort break every hour.

    6. CW

      Okay, cool.

    7. SJ

      Yeah.

    8. CW

      Right. What about like-

    9. SJ

      One of those things.

    10. CW

      ... eating with, with you drinking with it on?

    11. SJ

      Yeah, eating in a headset, drinking in a headset.

    12. CW

      Sleeping with it on?

    13. SJ

      Sleeping in a headset, working in a headset. Um-

    14. CW

      Oh my God.

    15. SJ

      ... racing car driving in a headset in real life, but with like the look-through camera through, through the headset. Um, all those kinds of things. Um, I, I did it with a, a, a, a colleague, a VR colleague of mine, um, Dean Johnson. We, we did it together. Um, he got a tattoo, um, wearing a headset to monitor the difference in heart rate based on your, um, whether it could be a pain reducer.

    16. CW

      Okay. Yeah, yeah, yeah.

    17. SJ

      Um, wearing a headset, which it did. His heart rate went down dramatically when he was wearing the headset. Um, so all those kinds of things. Um, really, really interesting. Um, but when I was doing that, I, I generally don't like cartoony things in VR. I, I like photogrammetry. I like realism. I, I like to feel like it's real.

    18. CW

      Mm-hmm.

    19. SJ

      ... um, but we were doing a boxing thing with this real cartoonish dude. Um, he looked like one of the characters from The Incredibles, um, and I was like, "Oh," you know, "I, I don't believe it's real at all." Um, but after a couple of minutes boxing him, it really felt real. Um, and there was some things there around the haptics, like there was a, a vibration as you were punching him. Um, the screen would go black as if, like, when you got knocked, as if you were like, you know, a little bit starry-eyed and losing your vision slightly. Um, so there were loads of things like that, that really tricked your mind, um, and you kind of get used to it. So, yeah, the resolution's not all that great, but it's getting there.

    20. CW

      Mm-hmm.

    21. SJ

      It's getting better.

    22. CW

      Yeah. I mean, it's-

    23. SJ

      I have faith (laughs) .

    24. CW

      It's a hell of a lot... Uh, uh, don't get me wrong. It's a hell of a lot more immersive than, than watching something on a TV that's at the end of your bed or-

    25. SJ

      Yeah.

    26. CW

      ... or in, or in the... mounted on the wall in the living room.

    27. SJ

      Yeah.

    28. CW

      It's definitely a very different sort of experience. So, what do you think at the moment, if you were to put your money where your mouth was, and someone... let's say someone has a, a, a VR headset at home and they want to see the, the coolest thing or the most impressive thing, or just your favorite thing right now, where would you direct them? What should they watch and what is it and why?

  11. 19:52 – 27:07

    The best VR isn’t at home: immersive theater, events, and multi-sensory design

    1. SJ

      Oh, this is really difficult because for the last couple of years I've been really bored (laughs) .

    2. CW

      (laughs)

    3. SJ

      As a huge advocate for the technology, um, I have been really bored.

    4. CW

      Why?

    5. SJ

      Because I've felt... I've felt like there's, there's been a lot of pressure on people to make stuff, um, obviously for commercial reasons, and to find an audience, and to do all that kind of stuff, and it doesn't allow people the time to be creative and break things a little bit and understand the different things that you can do. Um, and you need that time. You know, it's a new tech... well, it's not a new technology. It's been around since the ni- 1965 was the first headset, so it's been around forever. Um, but you need time to really find the good stuff, um, and that's quite hard to find. So, I mean, there are great... like Beat Saber is a great game, uh, really, really good. Um, if you like to have social VR, you know, you can go into Netflix in a headset and watch something as if you're in the cinema.

    6. CW

      Mm.

    7. SJ

      And that might be to some people's tastes. Um, I don't understand it personally, 'cause I would just watch Netflix on my TV. Um, but, you know, it, it can happen. Um, so there's things like that. Um, I like VR less so for being at home, but in a social experience and an event. So I'm a big fan of that full kind of immersive experience. Um, there's something on in London at the moment, um, that's an immersive experience of War of the Worlds.

    8. CW

      Oh, wow.

    9. SJ

      Um, and it's a two, three-hour experience. Um, it's an event as if you're going to the theater. There are actors there that sweep you arou- along and take you into different places. Um, you go through one area and then suddenly people are throwing a, a rucksack on your back, a sub pack, um, strapping a headset to you-

    10. CW

      Oh my God.

    11. SJ

      ... and then you're off somewhere else. So within-

    12. CW

      Are the tickets still available for this?

    13. SJ

      Yeah. They've just released new dates and it's on until January.

    14. CW

      Oh.

    15. SJ

      So it's definitely worth checking out.

    16. CW

      Right. That's... that is it.

    17. SJ

      Um, yeah.

    18. CW

      S- so I'm gonna put... I'm gonna put it in now.

    19. SJ

      Cool.

    20. CW

      I'm gonna put in War of the Worlds.

    21. SJ

      It's, it's Dot to Dot, um, the, the creators, um, and it's a proper experience.

    22. CW

      (laughs)

    23. SJ

      It, it's as if you're going to the theater.

    24. CW

      Have they got anything to do with the Star Wars one that was on-

    25. SJ

      No.

    26. CW

      Do you remember that one?

    27. SJ

      That, that was-

    28. CW

      Was that similar?

    29. SJ

      ... The VOID.

    30. CW

      Yes.

  12. 27:07 – 32:43

    Interactivity, movement, and photogrammetry: what’s possible now (and what isn’t)

    1. CW

      ... to do that. Um, one of the things that I noticed, again with the Oculus Go, uh, was the file size of these videos is insane, like for, for the, especially for the long ones. There's a Jurassic Park one, did you see that one?

    2. SJ

      Yeah, yeah.

    3. CW

      It was called Blue.

    4. SJ

      (laughs) Yeah.

    5. CW

      Um, and that's, that was awesome. That was my favorite one. I thought that was really cool.

    6. SJ

      (laughs) It's huge, isn't it? (laughs)

    7. CW

      The file size is terrifying. It's only three, three or four minutes long, but this file size is insane. Um, and even with that, you can only move, you don't move, you are moved on a, on a track through this particular f- film.

    8. SJ

      Yeah.

    9. CW

      So you can imagine that you're walking through a jungle and stuff happens, and you can pan around with your headset, but you can't control the motion. If you were to start to create worlds that were rendered to that sort of resolution, plus were, um, being created as you go, I'm gonna guess that's, that's, uh, technology that's not quite ready yet. Is that right?

    10. SJ

      Well, no, I mean, it is if you're tethered to a computer.

    11. CW

      Okay.

    12. SJ

      So if you've got, um, the, the Rift or the Vive, um, and certainly to, to some degree, you've got that on the Quest as well, which is really good. Um, so you are starting to open up those kind of multiple worlds. Um, not infinite, obviously, um, but different ways that you can move around and explore different areas. So that's really happening now.

    13. CW

      Mm-hmm.

    14. SJ

      Um, which is great, um, 'cause-

    15. CW

      Yeah.

    16. SJ

      ... that's what's needed. Um, it's really difficult in, in things where if I work with film and I work with 360, then you can't really do it with that. It, it's not really there. Um, there's ways to add interactivity, but not freedom really o- of space. You need to be using kind of photogrammetry, um, where you're taking like millions and millions of images, um, of one thing and splicing it all together, um, squishing it all together, um, to, to create a, a, a digital, um, image. Um, so you can start doing that, um, but it is moving really quickly, obviously starting back in 1965, but it now it's moving really quickly. (laughs)

    17. CW

      Yeah. (laughs)

    18. SJ

      Um... (laughs)

    19. CW

      It's picking up, picking up pace.

    20. SJ

      Always.

    21. CW

      But-

    22. SJ

      It, it's picking up pace.

    23. CW

      ... it's weird with this stuff though, right, because we're, we're seeing in front of our eyes how, um-

    24. SJ

      Yeah.

    25. CW

      ... multiple areas of technology need to be able to come together to enable something. So with this, let's say that, um, X number of years ago, we would've had the lens capability to have been able to show what was going on, but because of the em- the file sizes, you actually need the storage, and then potentially-

    26. SJ

      Mm-hmm.

    27. CW

      ... if you want to have something which is standalone, you need a battery. The battery on the Oculus Go was terrible.

    28. SJ

      Yeah.

    29. CW

      Like there was periods where I couldn't get the Oculus Go to charge because it didn't have enough battery to start up and it would like-

    30. SJ

      Yeah.

  13. 32:43 – 36:11

    Will we get VR blockbusters? Likely yes—but not ‘films’ as we know them

    1. CW

      Um, so is there ever, is there ever gonna be a time where we're going to see blockbuster films filmed f- in full VR, or do you think that the, uh, inherent difficulty in all of the d- the, um, added elements that you need to be able to control is just going to make that a no-no?

    2. SJ

      No, I think they will. Um, I think there's one feature-length VR film that's come out, um, that one of the longest films kind of that I've seen that I think is a good length, um, was around 18 minutes. That was, um, a piece, um, called CTRL, C-T-R-L, um, which was one of the most powerful pieces that I've seen. Um, and it was a good few years ago. Um, and it was all, it was around the gaming community, um, esports, um, but there was lot, lots of darkness around. I don't wanna say too much in case anyone manages to see it-

    3. CW

      Mm-hmm.

    4. SJ

      ... um, 'cause it's really, really powerful. Um, so, so that's the kind of maximum length that you get for a lot of experiences, that kind of 18 minutes max, most around 8 to 10 for, for kind of a normal experience. Um, but, but I, I think there will, um, but it might not be film as we know it. So, it might not be sitting down and watching something for, you know, 120 minutes. Um, it might involve more movement or activity, um, so that's quite different then. Um, so yes, but not as we know it. Um, and I don't think we would go to the cinema and then on the seat in front of us, there would be a headset that we-

    5. CW

      Pop a headset on.

    6. SJ

      ... would then put on.

    7. CW

      I understand.

    8. SJ

      Um, but, but I could see that happening more in kind of theater companies, that there's part of, and, and I know a lot of theater companies have experimented with that, that there's part of a production that might be augmented or there might be a different layer of technology used. Um, and that's really exciting. For me, the biggest uses will be VR within those kinds of immersive experiences, theater, art galleries, shared experiences, um, and then more use will be with augmented reality, um, because it's easy. It's on our phones.

    9. CW

      Mm-hmm.

    10. SJ

      You know, we can engage with different things on our phones. Um, we will eventually have holograms in our eyes. We can project whatever-

    11. CW

      (laughs)

    12. SJ

      ... we want on the wall. Um, we could be watching the same television, but watching two different programs just through whatever our, our lenses-

    13. CW

      Oh. (laughs)

    14. SJ

      ... what we're programmed to do. Um, but that's what I think will happen.

    15. CW

      Yeah.

    16. SJ

      Like, I think that's where we will go, and that's much more likely to happen than people sticking headsets on-

    17. CW

      Yeah.

    18. SJ

      ... because it's too intrusive.

    19. CW

      Yeah.

    20. SJ

      Um, and they're still heavy and it ruins your hair and it leaves marks around your face and-

    21. CW

      Does look like you've been punched, yeah. Looks like you've been punched a lot.

    22. SJ

      Yeah. Imagine what we'd look, look like after 48 hours (laughs) .

    23. CW

      Oh, my God, yeah. We, did you have to peel it off? You had to like, like (scraping sound) -

    24. SJ

      (laughs) No, they would like freeze it.

    25. CW

      ... pull it off?

    26. SJ

      We, we were swapping with lots of different headsets. We were using the Gear VR headsets. This was in the day when Samsung was still great, um, in VR. Uh, we were using the Rift, we were using the Vive, we were using the PlayStation that had just come out, so, so yeah, we had lots of different headsets.

    27. CW

      Marks, yeah.

    28. SJ

      Um, yeah.

    29. CW

      So you mentioned, you mentioned-

    30. SJ

      Yeah.

  14. 36:11 – 39:48

    Holograms and AR glasses: from stage projections to HoloLens storytelling

    1. CW

      At the very start, you mentioned holograms. Like, I-

    2. SJ

      Yeah.

    3. CW

      For me, a hologram is the security seal on the back of my driving license, and that thing that brought Michael Jackson back to life that one time.

    4. SJ

      (laughs)

    5. CW

      And I think they brought, I think they brought Tupac back as well.

    6. SJ

      (laughs) Yeah, he brought Tupac back.

    7. CW

      What, what's a hologram in your world?

    8. SJ

      Yeah. Uh, well, there's talk at the moment, there was meant to be an Amy Winehouse hologram tour. And, you, you know, the, the, the technology that's being used-... for holograms is really interesting, um, and a little creepy maybe. (laughs)

    9. CW

      How does it work?

    10. SJ

      Um, so, so it works, well, in- in a number of different ways. So they can, um, use old footage and they can also recreate, um, and get mannerisms right. Some use, um, like big green screens and motion capture. Um, others are using loads and loads of digital images, um, from past footage and putting them all together. Um, and then it's just projections. It- it's like a normal hologram that you would do with a, a CD case and projecting, um, lights in and, and creating things that you might, you might do if you're nerdy at home-

    11. CW

      Yeah.

    12. SJ

      ... um, which I'm sure everybody does.

    13. CW

      Mm-hmm.

    14. SJ

      Um, (laughs) so, so, yeah, so, so that, that's, you know, I'd say it's really, really simple. It's obviously not.

    15. CW

      Yeah.

    16. SJ

      It's hugely technical, um, and costs a fortune. Um, but I think with, um, with the technology from HoloLens and obviously Microsoft have been working on, you can really start to think how might you use holograms as a storytelling technique, um, interacting, um, in different ways. And, and there is a really interesting experience which I cannot remember the name of it. I can see the poster for it. It was at Sheffield Doc/Fest around two years ago. I will let you know 'cause it is really, really interesting, um, where it was, um, an immigration officer and you were, you had to decide whether to let people into the country or not based on their stories.

    17. CW

      Oh, wow.

    18. SJ

      So you were grilling them and it was all on the HoloLens, so more of an augmented reality. You can see through with the glasses and it's projecting in front of you, so it's not a VR headset.

    19. CW

      Okay.

    20. SJ

      Yeah.

    21. CW

      So, so, so you're not-

    22. SJ

      Um-

    23. CW

      ... you're not watching something, you're seeing the real world, and then is it overlaying? What's the difference between that hologram-

    24. SJ

      Yeah. It's, it's overlaying.

    25. CW

      ... and AR then?

    26. SJ

      Um, it's very, very similar technology, so, so yeah. The, the HoloLens... Well, the hologram is simply the, the way that it's projected, so-

    27. CW

      Understood.

    28. SJ

      ... the light beam's coming down on there and projecting on there.

    29. CW

      Yeah.

    30. SJ

      Um, where with AR it's more of that digital representation. Um, but they, they were using, um, the HoloLens to, to have these characters who you had to interview to decide whether they would get into the country or not, and determine, and determined by your language that you were choosing to use and the decisions you made. The holograms would get more real, they'd, they'd get more vibrant, um, or they would be slowly withering away.

  15. 39:48 – 50:32

    Always-on vision tech and privacy: Google Glass lessons and the unease of being watched

    1. CW

      'Cause I'm watching, uh, uh-

    2. SJ

      And that might be interesting.

    3. CW

      The other day, I started watching the new Blade Runner which has just gone, gone up on Netflix, and in that-

    4. SJ

      (laughs)

    5. CW

      ... uh, Ryan Gosling. Ryan Gosling?

    6. SJ

      Yeah. Is it-

    7. CW

      Yeah, Ryan Gosling.

    8. SJ

      He's one of the Ryans.

    9. CW

      It's Ryan someone. Has got, um... It's not Ryan Reynolds. Sorry, it's Ryan Gosling.

    10. SJ

      (laughs)

    11. CW

      He's got a, a f- fake girlfriend that's supplied to him by this company-

    12. SJ

      Yeah.

    13. CW

      ... and she's walking through, and we've seen this in films million times, right? Like, is that, is that going to be possible to be able to have that, to be able to project something 3D into the air which is broadly opaque as well and is able to move through? Um, because obviously the hologram at the moment, I didn't quite know that it was still essentially just that CD case projector type thing.

    14. SJ

      Yeah. Yeah.

    15. CW

      You can't have this big backdrop moving around or else it's gonna start clattering into your tables and chairs when it's supposed to be sitting down.

    16. SJ

      Yeah, exactly but-

    17. CW

      Is there anything on the, on the, the horizon that allows us to do anything like that?

    18. SJ

      I'm sure there's lots of people working on that.

    19. CW

      Mm-hmm.

    20. SJ

      Um, and, and certainly when you're, you're looking at kind of lenses in your eyes and, you know, different glasses that you might be wearing, um, which is what HoloLens, Microsoft's HoloLens is, and Magic Leap and all of those. That's what they're working on, um, always hugely top secret. Um, but, you know, that, that kind of technology is allowing that. I mean, simply if you look at, you know, obviously in the real kind of crass basic version, when you've got Pokemon Go-

    21. CW

      Yep.

    22. SJ

      ... and you're playing with the little, little things around and they're in the house and they might be, might be in your room-

    23. CW

      Yeah.

    24. SJ

      ... um, you, you've got them-

    25. CW

      Yeah.

    26. SJ

      ... obviously through the lens of your phone. Um, but if that's being projected in different ways, um, you, you can see, you can see that, but then i- is it any different then to, you know, cyber relationships in the '80s or the '90s-

    27. CW

      Mm-hmm.

    28. SJ

      ... and-

    29. CW

      Mm-hmm. Mm-hmm.

    30. SJ

      ... you know?

  16. 50:32 – 1:03:52

    VR as an ‘empathy machine’? Persuasion power, regulation gaps, and social VR harassment

    1. CW

      So, one of the things that I heard, um, Sam Harris talk about a while ago, and I'm gonna butcher it, but you'll be, you'll know what I'm talking about. Sam's concern was as technology allows us to become more immersive, there'd been some very, um, uh, there'd been some studies done which showed AR and VR's ability to change people's political views, their, uh, views towards I think climate change or tree-felling? There was something as well. Do you know what I'm talking about or am I just-

    2. SJ

      Yeah, yeah.

    3. CW

      ... am I blowing it all in your mind?

    4. SJ

      No, no, no.

    5. CW

      Okay. Can you, can you tell the listeners what, what I'm on about? 'Cause...

    6. SJ

      Yeah. I mean, there, there've been loads a- of, of conversations and discussions and lots of things around this. So, we basically say that VR, as an immersive media, has the capacity, the potential, to change perceptions, to change understandings. Uh, and that's, that's why we like it. So, you know, if you want to experience, you get Clouds over Sidra, an incredible piece done by Chris Milk a few years ago. One of the big, kind of VR 360 kind of experiences, following the journey of a girl in a refugee camp in, um, Jordan. So, that's... was designed to drive empathy. It was meant to, um, invoke that sense of, um, "We need to take action." You know, it was designed to be shown to decision-makers at the UN to get them to understand more and change their minds and get them to do more. So, the technology is being used for all of that. It's been labeled as this empathy machine, uh, which I think is a load of nonsense. Um, but that's what it's been labeled as. Um, if you take that argument that it can change minds, you can put the technology in other people's hands and get them to change minds in different ways. If the technology has that power to change what you're thinking about something, where does it go with subliminal messaging? Where does it go with, um, arguing for something else? Um, there's been really, really interesting studies around gender-swapping in VR, getting people to experience different genders, um, and then seeing how that changes, um, attitudes and behavior in the workplace. You know, feel what it's like for a woman to be the only woman in a boardroom full of men. Those kinds of scenarios. Um, some of them are really powerful. S- some of them have been shown to have impact. Um, there was one around, um, at the Stanford Virtual Human Interaction Lab, where they did something around homelessness. And they had two groups, and one group just received information on flat screens and brochures and that kind of stuff, and the other one experienced it. The other group was experiencing homelessness in, um, immersive experiences and connecting with people. And then they repeated it, and then they, they looked at the data over who was, who was still more empathetic. Who, who was actually, who had changed their behavior? And all of the studies showed that it was the group that were exposed to it in an immersive way.

    7. CW

      Mm-hmm.

    8. SJ

      Because they had that close relationship. So, so yeah, you are looking at technology that can potentially change people's minds if that is for the wrong way. How is that monitored? How is that regulated? Um, we're talking pretty much in quite a unregulated industry.

    9. CW

      Yeah.

    10. SJ

      Um, so yeah, I think that the concept of brain hacking, um, and changing the mind to think, changing the brain and the way that you're thinking, your beliefs and your motivations, is really interesting but a little bit freaky.

    11. CW

      Yeah. Well, I mean, uh, the only thing that I... The analogy that I can kind of draw here is that...With social media, it's ... We had no idea that it was going to come. Social media was a change. The fact that you have a screen in your pocket which is able to layer all sorts of different mental models and cognitive biases on top of itself, like an echo chamber and confirmation bias and, you know, all- all the different things that you have when you follow people on Twitter and then the algorithm starts to redeliver you more content which, you know, either outrage or agreement in the two things, et cetera, et cetera. All that stuff, right? So you- you ... We- we didn't know what was gonna go on with that. That happened, and now I think everyone has realized how we probably fucked up a bit with social media. We've probably ungoverned, or there's not enough governance around, are these things a platform or a pipeline? Are they a publisher? Um, should we be monitoring the content? Should they be allowed to de-platform people if- if they're not breaking the law, all this sort of stuff. And I think a lot of, uh, the general public and sort of techy people are looking forward now with a little bit more mindfulness about what, what's going on.

    12. SJ

      Mm-hmm.

    13. CW

      Are we, are we about to unleash the next thing? And then the analogy Sam Harris uses, it's really, really interesting, is he, he talks about, um, how he sees a ceiling. There's a ceiling on how much behavior change you can get from a, a two-dimensional phone that's in your hand.

    14. SJ

      Mm-hmm.

    15. CW

      And that that ceiling gets raised, and we don't know by how much. How much does it get raised by AR, how much does it get raised by VR, how much does it get raised by, I guess, this always-on VR holographic sort of stuff-

    16. SJ

      Yeah.

    17. CW

      ... which is somewhere between AR, VR and- and-

    18. SJ

      Yeah.

    19. CW

      ... and something else. Um, yeah, I ... A- again, you know, before the Manhattan Project, like you didn't need to have any legislation for nuclear bombs 'cause there was no nuclear bombs.

    20. SJ

      No.

    21. CW

      And inevitably with technology, the legislation always lags behind the actual-

    22. SJ

      Yeah.

    23. CW

      ... technology coming out. You can't-

    24. SJ

      Yeah.

    25. CW

      And on top of that, once the genie's out the bottle, it's not going back in the cage.

    26. SJ

      Yeah.

    27. CW

      So ...

    28. SJ

      Absolutely. Yeah. I mean, I-

    29. CW

      It's a space I'm glad I'm not having to legislate or, or govern in.

    30. SJ

      Yeah.

Episode duration: 1:03:49

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode 4ni60K0774s

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

iOSAndroidClaudeChrome