Gavin Miller: Adobe Research | Lex Fridman Podcast #23

1. 0:00 – 15:00

   The following is a…

   1. LFLex Fridman0:00

      The following is a conversation with Gavin Miller. He's the head of Adobe Research. Adobe has empowered artists, designers, and creative minds from all professions, working in the digital medium for over 30 years with software such as Photoshop, Illustrator, Premiere, After Effects, InDesign, Audition, software that work with images, video, and audio. Adobe Research is working to define the future evolution of these products in a way that makes the life of creatives easier, automates the tedious tasks, and gives more and more time to operate in the idea space instead of pixel space. This is where the cutting edge, deep learning methods of the past decade can really shine more than perhaps any other application. Gavin is the embodiment of combining tech and creativity. Outside of Adobe Research, he writes poetry and builds robots, both things that are near and dear to my heart as well. This conversation is part of the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter at Lex Fridman, spelled F-R-I-D. And now, here's my conversation with Gavin Miller. You're head of Adobe Research, leading a lot of innovative efforts in applications of AI, creating images, video, audio, language, but you're also yourself an artist, a poet, a writer, and even a roboticist. So while I promise to everyone listening that I will not spend the entire time we have together reading your poetry, which I love-

   2. GMGavin Miller1:33

      Oh. (laughs)

   3. LFLex Fridman1:33

      ... I have to sprinkle it in at least a little bit. So some of them are pretty deep and profound, and some are light and silly. Let's start with a few lines from the, uh, the silly variety. You write, uh, in, uh, Je ne regrette rien, a poem that beautifully parodies both, uh, Edith Piaf's Je Ne Regrette Rien and My Way by Frank Sinatra. So it opens with, "And now dessert is near. It's time to pay the final total. I've tried to slim all year, but my diets have been anecdotal." So where does that love for poetry come from for you? And if we dissect your mind-

   4. GMGavin Miller2:17

      Mm-hmm.

   5. LFLex Fridman2:17

      ... how does it all fit together in the bigger puzzle of Dr. Gavin Miller?

   6. GMGavin Miller2:22

      Oh. Well, interesting you chose that one. That was a poem I wrote when I'd been to my doctor and he said, "You really need to lose some weight" and, "Go on a diet."

   7. LFLex Fridman2:30

      (laughs)

   8. GMGavin Miller2:31

      And whilst the rational part of my brain wanted to do that, uh, the irrational part of my brain was protesting and sort of embraced the opposite idea.

   9. LFLex Fridman2:39

      I regret nothing, hence...

   10. GMGavin Miller2:40

       Yes, exactly. Taken to an extreme, I thought it would be funny. Obviously, it's a serious topic for so- some people, but, um, uh... I think for me, I've always been interested in writing since I was in high school, as well as doing technology and invention. And sometimes the parallel strands in your life that carry on and, you know, one is more about your private life and one's more about your technological, um, career, and then at sort of happy moments along the way, sometimes the two things touch, one idea informs the other, and, um, we can talk about that, uh, as we go.

   11. LFLex Fridman3:15

       Do you think your writing, the art, the poetry contribute indirectly or directly to your research, to your work in Adobe?

   12. GMGavin Miller3:22

       Well, sometimes it does if I, say, imagine a future in a science fiction kind of way, and then once it exists on paper I think, "Well, why shouldn't I just build that?" There was an example where when realistic voice synthesis first started in the '90s at Apple where I worked in research, uh, it was done by a friend of mine, um, I sort of sat down and started writing a poem which each line I would enter into the voice synthesizer and see how it sounded-

   13. LFLex Fridman3:50

       Hmm.

   14. GMGavin Miller3:51

       ... and sort of wrote it for that voice. And at the time, the agents weren't very sophisticated, so they'd sort of add random intonation and I kind of made up the poem to sort of match the tone of the voice, and it sounded slightly sad and depressed, so I, I pretended it was a poem written by an intelligent agent, uh, sort of telling the user to go home and leave them alone, but at the same time, they were lonely and wanted to have company and learn from what the user was saying. And at the time, it was way beyond anything that AI could possibly do, but, you know, since then, it's becoming more within the bounds of possibility. And then at the same time, I had a project at home where I did sort of a smart home. Uh, this was probably '93, '94. And, uh, I had the talking voice who'd remind me when I walked in the door of what things I had to do. I had buttons on my washing machine 'cause I was a bachelor and I'd leave the clothes in there for three days and they'd go moldy, so as I got up in the morning, it would say, "Don't forget the washing," and so on. Um, I made photographic photo albums that used light sensors to know which page you were looking at, would send that over w- wireless radio to the agent who would then play sounds that matched the image you were looking at in the book. So I was kind of in love with this idea of magical realism and whether it was possible to do e- that with technology. So that was a case where the sort of the agent sort of intrigued me from a literary point of view and became a personality. I think more recently, I've also written plays, and when plays you write dialogue, and obviously you write a fixed set of dialogue that follows a linear narrative. But with modern agents, as you design a personality or a capability for conversation, you're sort of thinking of, "I kind of have imaginary dialogue in my head," and then I think, "What would it take not only to have that be real, but for it to really know what it's talking about?"

   15. LFLex Fridman5:44

       Mm-hmm.

   16. GMGavin Miller5:44

       So it's easy to fall into the uncanny valley with AI where it says something it doesn't really understand, but it sounds good to the person, but you rapidly realize that it's kind of just stimulus response. It doesn't really have real world knowledge about the thing it's describing. And so...... when you get to that point, it really needs to have multiple ways of talking about the same concept so it sounds as though it really understands it. Now, what really understanding means is in the eye-

   17. LFLex Fridman6:11

       That's right.

   18. GMGavin Miller6:11

       ... of the beholder, right? But if it only has one way of referring to something, it feels like it's a canned response, but if it can reason about it or you can go at it from multiple angles and give a similar kind of response that people would, then it starts to seem more like there's something there that's sentient. Um-

   19. LFLex Fridman6:31

       It can say the same thing, multiple things from different perspectives. I mean, with the, the automatic image captioning that I've seen the work that you're doing.

   20. GMGavin Miller6:38

       Yes.

   21. LFLex Fridman6:39

       There's elements of that, right? Being able to generate different kinds of-

   22. GMGavin Miller6:42

       Right.

   23. LFLex Fridman6:42

       ... statements about the same picture.

   24. GMGavin Miller6:43

       So one, in my team, there's a lot of work on turning a medium from one form to another, whether it's auto-tagging imagery or making up full sentences about what's in the image, um, then changing the sentence, finding another image that matches the new sentence or vice versa. Um, and, uh, in the modern world of GANs, you sort of give it a description and it synthesizes, uh, an asset that matches the description, so. I've sort of gone on a journey. My early days in my career were about 3D computer graphics, the sort of pioneering work sort of before movies had special effects done with 3D graphics, and sort of w- rode that revolution in. That was very much like the Renaissance, where people would model light and color and shape and everything. And now we're kind of in another wave where it's more impressionistic and it's sort of the idea of something can be used to generate an image directly which is, um, sort of the new frontier in, uh, computer image generation using AI algorithms, so.

   25. LFLex Fridman7:43

       So the creative process is more in the space of ideas, or becoming more in the space of ideas versus in the raw pixels?

   26. GMGavin Miller7:50

       Well, it's interesting. It depends. I think at Adobe we really want to span the entire range from really, really good, what you might call low-level tools, by low level, as close to say analog workflows as possible, so. What we do there is we make up systems that do really realistic oil paint and watercolor simulations, so if you want every bristle to behave as it would in the real world and leave a beautiful analog trail of water and then flow after you've made the brushstroke, you can do that, and that's really important for people who want to create something really expressive or really novel, 'cause they have complete control. And then as certain other tasks become automated, it frees the artist up to focus on the inspiration and less of the perspiration, so, um, thinking about different ideas obviously. Once you've finished the design, there's a lot of work to say, do it for all the different aspect ratio of phones or websites and so on, and that used to take up an awful lot of time for artists. It still does for many, what we call content velocity. And one of the targets of AI is actually to reason about from the first example of what are the likely intent for these other formats, maybe if you change the language to German and the l- words are longer, how do you reflow everything so that it looks nicely artistic in that way? And so the person can focus on the really creative bit in the middle which is, what is the look and style and feel and what's the message and what's the story and the human element? So I think creativity is changing. So that's one way in which we're trying to just make it easier and faster and cheaper to do so that there can be more of it, more demand, 'cause it's less expensive, so everyone wants beautiful artwork for everything from a school website to Hollywood movie. On the other side is some of these things have automatic versions of them. People will possibly change role from being the hands-on artisan to being either the art director or the conceptual artist, and then the computer will be a partner to help create polished examples of the idea that they're exploring.

   27. LFLex Fridman10:00

       Let's talk about Adobe products for a second.

   28. GMGavin Miller10:01

       Right.

   29. LFLex Fridman10:01

       AI and Adobe products.

   30. GMGavin Miller10:03

       Yeah.
2. 15:00 – 30:00

   Right. …

   1. GMGavin Miller15:00

      the difference between academic research and industrial research.

   2. LFLex Fridman15:03

      Right.

   3. GMGavin Miller15:03

      So in academic research, it's really about who's the person to have the great new idea that shows promise, and we certainly love to be those people too. But we have sort of two forms of publishing. One is academic peer review, which we do a lot of and we have great success there, as much as some universities. Um, but then we also have shipping, which is a different type of review.

   4. LFLex Fridman15:26

      (laughs)

   5. GMGavin Miller15:26

      And then we get customer review-

   6. LFLex Fridman15:29

      Yeah.

   7. GMGavin Miller15:29

      ... as well as, you know, product critics. And that might be a case where it's not about being perfect every single time, but perfect enough of the time, plus a mechanism to intervene and recover where you do have mistakes. So we have the luxury of very talented customers. We don't want them to be overly taxed doing it every time, but if they can go in and just take it from 99 to 100 with the, the touch of a, of a mouse or something, then for the professional end, that's something that we definitely want to support as well. And for them, it went from having to do that tedious task all the time to much less often, so I- I think that gives us an out. If it had to be 100% automatic all the time, then that would delay the time at which we could get to market. Um...

   8. LFLex Fridman16:19

      So on that thread, maybe you can untangle something. Um, again, I'm sort of just speaking to my own experience. (laughs)

   9. GMGavin Miller16:27

      Yeah. No, that's fine.

   10. LFLex Fridman16:29

       (laughs) Maybe that is the most useful idea.

   11. GMGavin Miller16:30

       Absolutely.

   12. LFLex Fridman16:32

       So I think Photoshop, as per- an example, or Premier, uh, has a lot of amazing features that I haven't touched.

   13. GMGavin Miller16:41

       Mm-hmm.

   14. LFLex Fridman16:42

       And so what's the, in terms of AI helping make my life or the, uh, the, the life of creatives easier, how... This collaboration between human and machine, how, how do you learn to collaborate better? How do you learn the new algorithms? Um, is it something where you have to watch tutorials and you have to watch videos and so on? Or do you, do you ever think of... Do you ever, do you think about the experience itself through exploration, being the teacher?

   15. GMGavin Miller17:11

       We absolutely do. So we... I'm glad that you, uh, brought this up. We sort of think about two things. One is helping the person in the moment to do the task that they need to do. But the other is thinking more holistically about their journey learning a tool, and whether it's like... Think of it as Adobe University, where you use the tool long enough, you become an expert. And not necessarily an expert in everything. It's like living in a city, you don't necessarily know every street, but you know the important ones you need to get to. So we have projects in research which actually look at the thousands of hours of tutorials online and try to understand what's being taught in them. And then, we had one publication at CHI where it was looking at, given the last three or four actions you did, what did other people in tutorials do next? So if you want some inspiration for what you might do next, or you just want to watch the tutorial and see, learn from people who are doing similar workflows to you, you can without having to go and search on keywords and everything. So really trying to use the context of your use of the app to make intelligent suggestions, either about choices that you might make...... or in a more assistive way where it could say, "If you did this next, we could show you." And that's basically the frontier that we're exploring now, which is if we really deeply understand the domain in which designers and creative people work, can we combine that with AI and patent-matching of behavior to make intelligent suggestions? Either through, you know, verbal possibilities or just showing the results of if you try this. And that's, that's really the, sort of the ... You know, I was in a meeting today-

   16. LFLex Fridman18:54

       (laughs)

   17. GMGavin Miller18:54

       ... thinking about these things.

   18. LFLex Fridman18:55

       Wow (overlapping)

   19. GMGavin Miller18:55

       So it's, it's still a grand challenge. You know, we'd all love an artist over one shoulder and a teacher over the other, right?

   20. LFLex Fridman19:02

       Right. Yeah.

   21. GMGavin Miller19:02

       And, uh, we hope to get there. And the right thing to do is to give enough at each stage that it's useful in itself, but it builds a foundation for the next level of expectation.

   22. LFLex Fridman19:14

       Are you aware of this gigantic medium of YouTube that's creating, uh ... just a bunch of creative people, both artists and teachers of different kinds?

   23. GMGavin Miller19:26

       Absolutely. And we, the more we can understand those media types, both visually and in terms of transcripts and words, the more we can bring the wisdom that they embody into the guidance that's embedded in the tool.

   24. LFLex Fridman19:38

       That would be brilliant to-

   25. GMGavin Miller19:39

       Yeah.

   26. LFLex Fridman19:39

       ... to, uh, to remove the barrier from having to, yourself, type in the keyword searching, so on.

   27. GMGavin Miller19:45

       Absolutely. And then in the longer term, an interesting discussion is w- does it ultimately not just assist with learning the interface we have, but does it modify the interface to be simpler? Or do you fragment into a variety of tools, each of which has a different level of visibility of the functionality? I like to say that if you add a, if you add a feature to a GUI, you have to have yet more visual complexity confronting the new user. Whereas if you have an assistant with a new skill, if you know they have it, so you know to ask for it, then it's sort of additive without being more intimidating. So, we definitely think about new users and how to onboard them. Um, many actually value the idea of being able to master that complex interface and keyboard shortcuts like you were talking about earlier, because with great familiarity, it becomes a musical instrument for expressing your visual ideas. And, um, other people just want to get something done quickly in the simplest way possible, and that's where a more assistive version of the same technology might be useful. Maybe on a different class of device, which is more in context for capture, say. Whereas somebody who's in a deep post-production workflow maybe want to be on a laptop or a big screen desktop and have more knobs and dials to really express the subtlety of what they want to do.

   28. LFLex Fridman21:12

       So, there's so many exciting applications of computer vision and machine learning that Adobe is working on.

   29. GMGavin Miller21:17

       Mm-hmm.

   30. LFLex Fridman21:18

       Like scene stitching, sky replacement, foreground/background removal, spatial object-based image search, automatic image captioning, like we mentioned, Project Cloak, Project Deep Fill, filling in parts of the images.
3. 30:00 – 45:00

   And I think Adobe…

   1. GMGavin Miller30:00

      with, with the tool. Either it's helping in the short term to understand their intent so you can make better recommendations, or if they're friendly to your cause or your tool, or they want to help you evolve quickly 'cause they depend on you for their livelihood, uh, they may be willing to share some of their, um, workflows or, um, choices with, with the dataset to be then trained. There are technologies for looking at learning without necessarily storing all the information permanently so that you can sort of learn on the fly but not, um, keep a record of what somebody did. So, we're definitely exploring all of those possibilities.

   2. LFLex Fridman30:39

      And I think Adobe exists in a space where Photoshop-Like, if, if I look at the data I've created and own, you know, I'm mo- mo- less comfortable sharing data with social networks than I am with Adobe because there's a g- a, just exactly as you said, uh, there's an obvious, uh, benefit for sharing the data that I use to create in Photoshop because it's helping improve the workflow in the future.

   3. GMGavin Miller31:06

      Right.

   4. LFLex Fridman31:06

      As opposed to it's not clear what the benefit is in social networks.

   5. GMGavin Miller31:10

      It's nice of you to say that. I mean, I think there are some professional workflows where people might be very protective of what they're doing, such as if I was preparing evidence for a legal case, I wouldn't want (laughs) any, any of that, you know, phoning home to help train the algorithm or anything. Um, there may be other cases where people are, say, having a trial version or they're doing some... I'm not saying we're doing this today, but there's a future scenario where somebody has a more permissive relationship with Adobe where they explicitly say, "I'm fine. I'm only doing hobby projects or things which are non-confidential, um, and in exchange for some benefit, um, tangible or otherwise, I'm willing to share very fine-grain data." So, another possible scenario is to capture relatively crude high-level things from more people and then more detailed knowledge from people who are willing to participate. We do that today with explicit customer studies where, you know, we go and visit somebody and ask them to try the tool and we human observe what they're doing. Um, in the future, to be able to do that enough to be able to train an algorithm, we'd need a more systematic process, but we'd have to do it very consciously because one of the things people treasure about Adobe is a sense of trust and we don't want to endanger that through overly aggressive data collection. So, uh, we have a chief privacy officer and it's definitely front and center of thinking about AI rather than an afterthought.

   6. LFLex Fridman32:37

      Well, when you start that program, sign me up. I'm-

   7. GMGavin Miller32:40

      Okay. Happy to. (laughs)

   8. LFLex Fridman32:41

      (laughs) Is, is there other projects that you wanted to mention that, um, that I didn't perhaps that pop into mind?

   9. GMGavin Miller32:48

      Well, you covered a number. I think you mentioned Project Puppetron.

   10. LFLex Fridman32:52

       Puppetron.

   11. GMGavin Miller32:52

       I think that one is interesting because it's, um... You might think of Adobe as only thinking in 2D, um, and that's a good example where we're actually thinking more three-dimensionally about how to assign features to faces so that we can... You know, if you take... So, what Puppetron does, it takes, um, either a still or a video of a person talking and then it can take a painting of somebody else and then apply the style of the painting to the person who's talking in the video. And, um, it... Unlike a sort of screen door post filter effect that you sometimes see online, um, it really looks as though it's sort of a- somehow attached or reflecting the motion of the face. And so that's a case where even to do a 2D workflow like stylization you really need to infer more about the 3D structure of the world, and I think as 3D computer vision algorithms get better, initially they'll focus on particular domains like faces where you have a lot of prior knowledge about structure and you can maybe have a parameterized template that you fit to the image.

   12. LFLex Fridman33:58

       Mm-hmm.

   13. GMGavin Miller33:58

       But over time this should be possible for more general content. And it might even be invisible to the user that you're doing 3D reconstruction, but... Under the hood, but it might then let you, um, do edits much more reliably or correctly than you would otherwise.

   14. LFLex Fridman34:16

       And, y- you know, face is a very important application, right? So-

   15. GMGavin Miller34:21

       Absolutely.

   16. LFLex Fridman34:21

       ... um, making things work-

   17. GMGavin Miller34:22

       And a very sensitive one. If you do something uncanny, it's very disturbing.

   18. LFLex Fridman34:26

       That's right.

   19. GMGavin Miller34:27

       Yeah.

   20. LFLex Fridman34:27

       (laughs) Because-

   21. GMGavin Miller34:28

       Yeah.

   22. LFLex Fridman34:28

       ... you have to get it, you have to get it right. So, um, in the space of augmented reality and virtual reality-

   23. GMGavin Miller34:36

       Yes.

   24. LFLex Fridman34:37

       ... uh, what do you think is the role of AR and VR in, in the content we consume as people, as consumers, and the content we create as creators today?

   25. GMGavin Miller34:45

       Now, that's a great question. We, we think about this a lot too. So, I think VR and AR serve slightly different purposes. So, VR can really transport you to an entire immersive world, um, no matter what your personal situation is. To that extent, it's a bit like a really, really widescreen television where it sort of snaps you out of your context and puts you in a new one, and I think it's still evolving in terms of the hardware. I actually worked on VR in the '90s trying to solve the latency and sort of nausea problem, which we did, but it was very expensive and a bit early. Um, there's a new wave of that now, I think, uh, and increasingly those devices are becoming all-in-one rather than something that's tethered to a box. I think, uh, the market seems to be bifurcating into things for consumers and things for professional use cases, like for architects and people designing where your product is a building and you really want to experience it better than looking at a scale model or a drawing, I think, um, e- or even than a video. So, I think for that where you need a sense of scale and spatial relationships, it's great.

   26. LFLex Fridman35:52

       Mm-hmm.

   27. GMGavin Miller35:52

       I think AR holds the promise of sort of taking digital assets off the screen and putting them in context in the real world on the table in front of you, on the wall behind you, and that has the corresponding need that the assets need to adapt to the physical context in which they're being placed. I mean, it's... It's a bit like having a live theater troupe come, come to your house and put on Hamlet. Uh-

   28. LFLex Fridman36:18

       (laughs) .

   29. GMGavin Miller36:18

       ... my mother had a friend who used to do this at stately homes in England, uh, for the National Trust, and they would adapt the scenes and even... They'd walk the audience through the rooms to see the action based on the country house they found themselves in for two days, and I think AR will have the same issue that, you know, if you have a tiny table in a big living room or something, it'll try to figure out what can you change and what's fixed, um, and there's a little bit of a tension between fidelity where...... if you captured, say, Nureyev doing a fantastic ballet, you'd want it to be sort of exactly reproduced and maybe all you could do is scale it down. Whereas somebody telling you a story might be walking around the room doing some gestures, and that could adapt to the room in which they were telling the story, so-

   30. LFLex Fridman37:07

       And do you think fidelity is that important in that space? Or is it more about the storytelling?
4. 45:00 – 1:00:00

   I, I think it's…

   1. LFLex Fridman45:00

      using the tooling of, of modifying the images or even with artificial intelligence ideas of deepfakes and creating s- adjusted or fake versions of ourselves and reality.

   2. GMGavin Miller45:14

      I, I think it's an interesting question. You are sort of historical bent on this, so...

   3. LFLex Fridman45:18

      (laughs) So-

   4. GMGavin Miller45:19

      I, I, I actually wonder if 18th century aristocrats who commissioned famous painters to paint portraits of them-

   5. LFLex Fridman45:25

      (laughs)

   6. GMGavin Miller45:25

      ... had portraits that were slightly nicer than they-

   7. LFLex Fridman45:28

      Touché.

   8. GMGavin Miller45:28

      ... actually looked in practice.

   9. LFLex Fridman45:29

      Well played, sir.

   10. GMGavin Miller45:30

       Uh, so-

   11. LFLex Fridman45:31

       (laughs)

   12. GMGavin Miller45:31

       ... uh (laughs) human desire to put your best foot forward has always been true. I, I think it's interesting. You, you sort of framed it in two ways. One is, if we can imagine alternate realities and visualize them, d- is that a good or bad thing? In the old days, you do it with storytelling and words and poetry, uh, which still resides sometimes on-

   13. LFLex Fridman45:51

       (laughs)

   14. GMGavin Miller45:51

       ... websites. But, uh, you know, we've become a very visual culture in particular. In the 19th century, we were very much a text-based culture. Um, people would read long tracts. Political speeches were very long. Nowadays, everything's very kind of quick and visual and snappy. I think it depends on how harmless your intent. It's, a lot of it's about intent. So if you, um, have a somewhat flattering photo that you pick out of the photos that you have in your inbox to say, "This is what I look like." It's, it's probably fine. If someone's gonna judge you by how you look, then they'll decide soon enough when they meet you whether the reality... you know?

   15. LFLex Fridman46:39

       (laughs) Yeah, right.

   16. GMGavin Miller46:39

       Um, I think where it can be harmful is if people hold themselves up to an impossible standard which they then feel bad about themselves for not meeting. I think that's definitely a, uh, can be a, an issue. And, uh, but I think the ability to imagine and visualize an alternate reality, which sometimes of w- sometimes of which you then go off and build later, um, can be a wonderful thing, too. People can imagine architectural styles which they then, you know, have a startup, make a fortune, and then build a house that looks like their favorite video game. Is that a terrible thing? I think, uh, I used to worry about exploration actually, that part of the joy of going to the moon when I was a tiny child, I remember it in grainy black and white, was to know what it would look like when you got there. And I think now we have such good graphics for knowing, for visualizing the experience before it happens, that I slightly worry that it may take the edge off actually wanting to go. You know what I mean?

   17. LFLex Fridman47:42

       (laughs) Yeah.

   18. GMGavin Miller47:42

       Because we've seen it on TV. We kind of, "Oh..." You know, by the time we finally get to Mars, we'll go, "Oh, yeah, so this is Mars. It, this is what it looks like." But then, you know, the outer exploration, I mean, I think, um, Pluto was a fantastic recent discovery where nobody had any idea what it looked like, and it was just breathtakingly varied and beautiful. So I think expanding the ability of the human toolkit to imagine and communicate, on balance, is a good thing. I think there are abuses. We definitely take them seriously and try to discourage them. Um, I think there's a parallel side where the public needs to know what's possible through events like this, right?

   19. LFLex Fridman48:23

       Mm-hmm.

   20. GMGavin Miller48:24

       So that, um, you don't believe everything you read in print anymore. And it may, over time, become true of images as well. Um, or you need multiple sets of evidence to really believe something, rather than a single media asset. So I think it's a constantly evolving thing. It's been true forever. There's a famous story about Anne of Cleves and Henry VIII, where luckily for Anne, they didn't get married, right? Um, so... Or they got married and broke up and it-

   21. LFLex Fridman48:53

       How... What's the story? What's the, uh, how did the-

   22. GMGavin Miller48:54

       Oh, so Holbein went and painted a picture, and then Henry VIII wasn't pleased and, you know-

   23. LFLex Fridman48:59

       Oh, yeah, the ...

   24. GMGavin Miller49:00

       History doesn't record whether Anne was pleased, but I think she was pleased not to, uh, be married more than a day or something.

   25. LFLex Fridman49:05

       Yeah. (laughs)

   26. GMGavin Miller49:06

       So, I mean, this has gone on for a long time, but, um, I think it's just a part of the magnification of human capability.

   27. LFLex Fridman49:14

       You've kind of built up a, an amazing research environment here, research culture, research lab. And you've written that the secret to a thriving research lab is interns. Can you un- unpack that a little bit?

   28. GMGavin Miller49:26

       Oh, absolutely. So, a couple of reasons. As you see looking at my personal history, there are certain ideas you bond with at a certain stage of your career, and you tend to keep revisiting them through time. Um, if you're lucky, you pick one that doesn't just get solved in the next five years, and then you're sort of out of luck. So I think a constant influx of new people brings new ideas with it. From the point of view of industrial research, because a big part of what we do is really taking those ideas to the point where they can ship as very robust features, you end up investing a lot in a particular idea. And if you're not careful, people can get too conservative in what they choose to do next, knowing that the product teams will want it. And interns let you explore the more fanciful or unproven ideas in a relatively lightweight way, um, ideally leading to new publications for the intern and for the researcher. And it gives you then a portfolio from which to draw which idea am I going to then try to take all the way through to being robust in the next year or two to ship. So it sort of becomes part of the funnel. It's also a great way for us to identify future full-time researchers. Many of our greatest researchers were former interns. It builds a bridge to university departments so we can get to know and build an enduring relationship with the professors, and we often do academic gift funds too as well, as an acknowledgement to the value the interns add and their own collaborations. So it's sort of a virtuous cycle. And then, the long-term legacy of a great research lab hopefully will be not only the people who stay, but the ones who move through and then go off and carry that same model to other companies. And so, we believe strongly in industrial research and how it can complement academia, and we hope that this model will continue to propagate and be invested in by other companies. Which makes it harder for us to recruit, of course, but, you know, uh, that's a sign of success, and a, a rising tide lifts all ships in that sense.

   29. LFLex Fridman51:32

       And where, where is the idea born with, with the interns? Is there brainstorming? Is there discussions about, you know, like what d- Adobe ...

   30. GMGavin Miller51:42

       Where do the ideas come from?
5. 1:00:00 – 1:09:11

   Yeah, so you should…

   1. GMGavin Miller1:00:00

      a child, and, uh, from scratch. And I thought, "Well, how hard can it be to build a real one?" And so then started a, what turned out to be like a 15-year obsession with, uh, trying to build better snake robots. And the first one that I built just sort of slithered sideways, but didn't actually go forward. Then I added wheels. And building things in real life makes you honest about the friction. The, the thing that appeals to me is I, I love creating the illusion of life, which is what drewed me to an- drove me to animation. And if you have a robot with enough degrees of coordinated freedom that move in a kind of biological way, then it starts to cross the uncanny valley and to see me like a creature rather than a thing. And I certainly got that with the early snakes by S3. I had it able to sidewind as well as go directly forward. My wife to be suggested that it would be the ring bearer at our wedding, so it actually went down the aisle carrying the rings, and uh (laughs) got in the local paper for that, which was really fun. And this was all done as a hobby. And then I, um... at the time the com- onboard compute was incredibly limited. It was sort of-

   2. LFLex Fridman1:01:08

      Yeah, so you should explain that.

   3. GMGavin Miller1:01:09

      Yeah.

   4. LFLex Fridman1:01:09

      The, these snakes though, the whole idea is that you would... you're trying to run it sta- um, autonomously, so-

   5. GMGavin Miller1:01:15

      Autonomously.

   6. LFLex Fridman1:01:16

      ... all the computers onboard.

   7. GMGavin Miller1:01:16

      Onboard power, onboard bra- yeah, and so...The very first one I actually built the controller from discrete logic, uh, 'cause I used to do LSI, you know, circuits and things when I was a teenager. And then, um, the second and third one, the 8-bit microprocessors were available with, like, a whole 256 bytes of RAM, um, which you could just about squeeze in. So they were radio controlled rather than autonomous, and really were more about the physic- physicality and coordinated motion. Um, I've occasionally taken a sidestep into, "If only I could make it cheaply enough, it would make a great toy," which has been a lesson in how clockwork is its own, uh, magical realm that you venture into and learn things about backlash and other things you don't take into account as a computer scientist, which is why what seemed like a good idea doesn't work. So it was quite humbling. And then more recently, I've been building S9, which is a much better engineered version of S3 where the motors wore out and it doesn't work anymore and you can't buy replacements, which is sad given that it was such a meaningful one. Uh, S5 was about twice as long and, um, looked much more biologically, uh, inspired. I, unlike the typical roboticist, I taper my snakes. Uh, there are good mechanical reasons to do that, but it also makes them look more biological. Although it means every segment's unique rather than, um, a re- repetition, which is why most engineers don't do it. Uh, it actually saves weight and leverage and everything. And that one is currently on display at the International Spy Museum in Washington, DC.

   8. LFLex Fridman1:02:52

      (laughs)

   9. GMGavin Miller1:02:53

      Not that it's done any spying. It was on YouTube and it got its own conspiracy theory where people thought that it wasn't real 'cause I work at Adobe. It must be fake graphics.

   10. LFLex Fridman1:03:02

       (laughs)

   11. GMGavin Miller1:03:03

       And people would write to me, "Tell me it's real." You know, they say, "The background doesn't move." And it's like, "It's, it's on a tripod, you know?" Um, so that one... But you can see the real thing, so it really is true. And then the latest one is the first one where I could put a Raspberry Pi, which leads to all sorts of terrible jokes about pythons and things. But, um...

   12. LFLex Fridman1:03:22

       (laughs) Yeah.

   13. GMGavin Miller1:03:23

       Yeah. But this one can have onboard compute. And then where my hobby work and my work work are converging is you can now add vision accelerator chips which can evaluate neural nets and do object recognition and everything. So both for the snakes and more recently for the spider that I've been working on, um, having, you know, desktop-level compute is now opening up a whole world of true autonomy with onboard compute, onboard batteries, and, um, still having that sort of biomimetic quality that people a- that appeals to children in particular. They're really drawn to them. And adults think they look creepy, but children actually think they look charming. Um, and, uh, uh, I gave a series of lectures at Girls Who Code to encourage people to take an interest in technology. And at the moment, I'd say they're still more expensive than the value that they add, which is why they're a great hobby for me, but they're not really a great product. Um, it makes me think about doing that very early thing I did at Alias with changing the muscle rest lengths. If I could do that with a real artificial muscle material, then the next snake ideally would use that rather than motors and gearboxes and everything. It would be lighter, much stronger, and more continuous and smooth. So, um, it's s- I like to say being in research is a license to be curious, and I have the same feeling with my hobby. It forced me to read biology and be curious about things that otherwise would have just been, you know, a National Geographic special. Suddenly I'm thinking, "How does that snake move? Can I copy it?" I look at the trails that sidewinding snakes leave in sand and see if my snake robots would do the same thing. Um-

   14. LFLex Fridman1:05:09

       So, uh, out of something inanimate, I like the way you put it, try to bring life into it and beauty.

   15. GMGavin Miller1:05:13

       Absolutely.

   16. LFLex Fridman1:05:14

       And-

   17. GMGavin Miller1:05:14

       And then ultimately give it a personality, which is where the intelligent agent research will, uh, converge with the vision and voice synthesis to give it a sense of having, not necessarily human-level intelligence. I think the Turing test is such a high bar, it's-

   18. LFLex Fridman1:05:30

       Mm-hmm.

   19. GMGavin Miller1:05:30

       ... a little bit self-defeating. But having one that you can have a meaningful conversation with, uh, especially if you have a reasonably good sense of what you can say. Um, so not trying to have it so, uh, a stranger could walk up and have one. But so as a, as a pet owner or robot pet owner, you could know what it thinks about and what it can reason about, and...

   20. LFLex Fridman1:05:53

       Or sometimes just a meaningful interaction. If, if you have a, the kind of interaction you have with a dog, sometimes you might have a conversation, but it's usually one way.

   21. GMGavin Miller1:06:02

       Absolutely. Right.

   22. LFLex Fridman1:06:02

       And, uh, nevertheless it feels like a meaningful, uh, connection.

   23. GMGavin Miller1:06:06

       And one of the things that I'm trying to do in the sample audio that we'll play you is beginning to get towards the point where the, the reasoning system can explain why it knows something or why it thinks something. And that, again, creates the sense that it really does know what it's talking about. But also, for debugging. (laughs)

   24. LFLex Fridman1:06:26

       (laughs)

   25. GMGavin Miller1:06:26

       As you get, uh, more and more elaborate behavior, it's like, "Why did you decide to do that? Why, what, you know, how do you know that?" I think it's, the robot's really are my muse for helping me think about the future of AI and what to invent next.

   26. LFLex Fridman1:06:42

       So even a- at Adobe, that's mostly operating in digital world.

   27. GMGavin Miller1:06:47

       Correct.

   28. LFLex Fridman1:06:47

       Do you ever th- Do you see a future where Adobe even expands into the more physical world perhaps? So bringing life not into animations, but bringing life into physical o- objects with whether it's... Well, do you, you, uh-

   29. GMGavin Miller1:07:04

       I have to say at the moment, it's a twinkle in my eye. I think the more likely thing is that we will bring virtual objects into the physical world through augmented reality.

   30. LFLex Fridman1:07:14

       Augmented reality.

Episode duration: 1:09:11