No Priors Ep. 18 | With Kevin Scott, CTO of Microsoft

1. 0:00 – 12:44

   Kevin Scott's Journey to Microsoft CTO

   1. SGSarah Guo0:05

      Microsoft, the behemoth productivity, cloud, and gaming company has taken a massive bet on AI. Everyone's paying close attention to its partnership with OpenAI, and the technical community has been amazed by its release of some of the first truly useful and broadly deployed AI products such as GitHub Copilot. Its full-on attack on web search with the new LLM-powered Bing Chat is making its incumbent competitors dance. Today on No Priors, we're thrilled to speak with Kevin Scott, CTO of Microsoft and the driving force behind their AI strategy. Kevin's leadership, both at Microsoft and prior at LinkedIn, Google, and AdMob as a technologist, is especially inspiring to me given his distance traveled from his childhood home in rural Central Virginia. In 2020, he published a book, Reprogramming the American Dream, about making AI serve us all. Kevin, welcome to No Priors. Thanks so much for joining us.

   2. KSKevin Scott0:54

      Thanks for having me, guys.

   3. SGSarah Guo0:55

      Can you start by sharing with us some of your story? How does one go from a farming community in Virginia where your parents didn't attend college to CTO of Microsoft?

   4. KSKevin Scott1:03

      Uh, I don't know. I think it is a very unlikely journey. Uh, it's, like, certainly not a thing that I, uh, I ever could have imagined. I, I think part of it is I was just super fortunate to be wired like a nerd, uh, and growing up when I grew up. So, you know, when I was a teenager in the early 80s, uh, personal computing was, uh, was happening and, like, that was the thing that I happened to fixate on. Um, and even though we were relatively poor, I managed to, you know, scrape together enough bucks to get myself a personal computer that I could have and just tinker with all the time. And it was, uh, like a, it was a RadioShack Color Computer, too, like one of these things with Chiclet keys that you, uh, you actually connected to a television. Like I had it hooked up to a 13-inch-

   5. SGSarah Guo1:56

      (laughs)

   6. KSKevin Scott1:56

      ... TV and it had a cassette recorder that you, uh, stored and loaded your programs on and, you know, and, and it was just the thing that I was obsessed with and I, I stayed obsessed with computers, uh, from then on. And it was just me trying to find a path at each step where I could work on the most interesting thing that someone was, uh, dumb enough to give me permission to go work on. And, and a- and again, it's, it's a lot of luck. Like there's no way you can, uh, plan a path from rural Central Virginia to CTO of Microsoft, but you know, it, I think it does help to have a high-level vision in your head for what it is that you want to do. Like, just knowing what you're aiming for always helps.

   7. SGSarah Guo2:41

      What was that vision for you? Besides, like, you know, obsessed with computers, wanted to work on them.

   8. KSKevin Scott2:46

      Yeah. I, I more or less had two of them. So the first vision I ha- when I was a teenager was I wanted to be a computer science professor, so I just looked at what computer scientists did and thought, "This is the most amazing stuff I've ever seen." And I went to a science and technology high school, and, and the way that it worked where I lived is, uh, like a really rural area, and so the science and technology, uh, it was a governor's school, so it was centrally located and, uh, each high school in these four or five counties that surrounded the governor's school got to send two students each, and so I was one of the two students that got selected from my high school to go to this thing. And my computer science professor there was this guy, uh, Dr. Tom Morgan, uh, and, like I just sort of felt like he'd opened up this entire new world to me. Like, it was just thrilling to learn all of this stuff and I was like, "Yeah, I want to be like Dr. Morgan." Uh, and, and a lot of this stuff for me is about, you know, like who those influential role models have been, uh, in your life and so as soon as I, like, met Dr. Morgan, I was like, "Oh, I should just go be a computer science professor." And that was the path I was on until I was about 30 years old, um, when I, you know, I was a compiler optimization and computer architecture programming languages person and I got pretty disillusioned with what being a computer science professor actually was relative to what I wanted to do. Like, I just wanted to have a lot of impact and my perception at the time when I was making these decisions was that you can have a lot of impact as a computer science professor, um, and the, and the impact was actually great, but it wasn't the impact that the system appreciated. Uh, so the, so, so, like, the impact that you can actually have is inspire students to go pursue these careers and they will go on to do much greater things than, uh, than you've done yourself. And, like, that, that to me was the greatest impact, but it was the least appreciated part of being a computer science professor, uh, like back in the, you know, 2000s, uh, when I was, you know, making these big decisions, and so I decided to leave and I didn't at the time know what next actually was gonna be. Like, it'd b- it had been my mission for almost 15 years at that point and, like, I was a little bit lost, uh, and I saw that a bunch of my academic buddies, uh, were all working at this startup called Google, um, and I didn't understand why they were working at Google. Like, Google was, you know, was like some little box and you typed a w- keywords in and it gave you 10 things.

   9. SGSarah Guo5:27

      (laughs)

   10. KSKevin Scott5:27

       Like, how is that hard? Uh, but, but you know, Urs Holzle, who was a compiler person and Jeff Dean, who was a compiler person and Alan Eustace, who was a compiler person, like, all of these people who, you know, who I went to conferences with and whose papers I read and, uh, I was like, "All right, well, maybe I should send my resume in." And, like, I sent my resume in and, uh, got called to do a bunch of interviews and they, uh, like it, it was the best interviewing experience I've ever had because they, they took what must have been every compiler person in the company at the time and put them on my interview panel and I was like, "Oh my God, this is amazing." Uh, like I had the best day interviewing there. And I got this job offer and I went, um, I, I got this choice. They'd just started Google New York, uh, which was the first...... office outside of Mountain View and they were like, "You can come to Mountain View or you can go be, you know, the 10th person in this, uh, New York office." And my wife and I wanted to live in New York more than we wanted to live in Mountain View, and so, uh, that's what we did. And after I got there, this is where the new mission came in. I just... So we, we were hiring these brilliant, brilliant people at the time. And, and the way that we did hiring was kind of crazy. It's like, "All right. Well, if you're smart, just come work here," and, like, we have no idea, like, what exactly it is, uh, you're gonna do. And you, like, came in and you sort of sorted yourself out. And we had these people who were so accomplished and so brilliant, and they would come in and choose to work on things that, that just were gonna have no impact at all. Like, they, they were intellectually very interesting, but they were just sort of silly in that they were never gonna connect with anything that moved the needle for the company, which was exactly the problem I was trying to get away from, uh, you know, in, in being a, like, a research computer scientist. Uh, and so I sorted myself out. Like, I found a, like, a pragmatic thing to go work on. Like, you know, we... L- I, I won't go into the details of what it is, but, you know, like, the whole team won a Google Founders award, which was a big deal, uh, for, like, solving this, like, very sort of unsexy problem with a bunch of very fancy computer science, um, which was one of the things I think Google, uh, Google did really well. And then I was like, "Okay. Well, I should just go help more people sort themselves out as well," and that's when I became a manager. And then it... From that point on, it was all about, like, "Hey, I want to... I, I, I wanna help as many engineers as I possibly can, uh, like, make sure that their work lines up with something that's, you know, both interesting and meaningful."

   11. EGElad Gil8:04

       I think that, uh, it's actually a pretty underdiscussed degree to which early Google had so many academics actually running important parts of the company.

   12. KSKevin Scott8:10

       Yeah.

   13. EGElad Gil8:10

       I think urs is a great example, and I think there's others. And so I haven't actually seen anything like that since until maybe now more recently at OpenAI there's more academics where, you know, you feel like the research community is popping back up again. But it's been maybe a decade or two since that's happened.

   14. KSKevin Scott8:24

       Yeah. I mean, I, I think that, that, that's actually a really, really great observation. So when, when I go sit in OpenAI, it really reminds me of early Google days. And it's about the same size Google was when I joined, and so, like, I, I couldn't figure it out for a while, and I was like, "Wow, this is, like, really giving me, like, you know, early Google nostalgia." And, and, you know, the, the conclusion to draw from that is, like, not that they're the same companies or they're trying to solve the same problem, it's just sort of the energy of the place and, like, who they've chosen to hire and, like...

   15. EGElad Gil8:56

       Yeah. Yeah. It's the first time I've seen, like, string theorists getting hired again-

   16. KSKevin Scott8:59

       Yeah.

   17. EGElad Gil8:59

       ... into computer science roles-

   18. KSKevin Scott9:01

       Yeah, 100%.

   19. EGElad Gil9:02

       ... you know, since Google days. Yeah.

   20. KSKevin Scott9:04

       You, you, you and I, uh, like, probably both worked with Yonatan Zunger, who, uh, works at Microsoft right now. And, like, I r- I remember, like, it's like, "Oh, Yonatan's working on this big distributed file system stuff," and, like, "What's his degree? Oh, yeah, he's a, like, s- string theory guy." (laughs)

   21. EGElad Gil9:18

       (laughs) Yeah.

   22. SGSarah Guo9:20

       So a big part of your mission for, you know, the last decades has been, um, helping string theorists and other engineers, uh-

   23. KSKevin Scott9:27

       (laughs)

   24. SGSarah Guo9:27

       ... figure out how to be, how to be useful in their orgs. The other, the other part seems to be, of course, like, um, actual technical direction, right? Deciding, like, what's worth investing in.

   25. KSKevin Scott9:36

       Yeah.

   26. SGSarah Guo9:36

       And, and you've worked on machine learning products for a really long time. Like, ads auctions at Google, recommendations at LinkedIn, et cetera, et cetera. Was there a moment when you decided or you realized personally that AI should be a key technical bet for Microsoft?

   27. KSKevin Scott9:50

       Yeah. I mean, I, I've been at Microsoft a little over six years now. So almost six and a half years. Uh, and, like, pretty, pretty quickly it was obvious that AI was gonna be, like, very, very, very important to the future of the company. I think, you know, Microsoft already understood that before I got there, and then it was just how do you focus all of the energy on the company on the right thing? B- because we had a lot of AI investment and a lot of AI energy, and it was sort of very diffuse, uh, when I got there. Uh, so no lack of IQ and actually no lack of capital spending and everything else, but it was just, you know, kind of getting peanut buttered across, uh, a whole bunch of stuff. And so the, the thing that really catalyzed what we were doing is... I mean, m- maybe this is a little bit too, uh, too technical, but, like, you know, w- we... Before, before I got there, the technical thing that had been happening with some of these AI systems that to me was very interesting is transfer learning was starting to work. So, like, you were going from this mode of, you know, the flavor of statistical machine learning that I cut my teeth on, uh, like, in my first projects at Google, which was, you know, you have a particular domain of data and, like, you have a particular machine learning, uh, model architecture that you are, uh... You know, you're training in a, like, a particular way that you're gonna go do the deployment and measurement and whatnot, and it's all, like, you know, siloed to a, like a, uh, like a use case or a domain or an application, to seeing AI systems that you could train on one set of data and use for things, uh, for multiple purposes. And you saw a little bit of that with some of the cool stuff that DeepMind was doing with reinforcement learning with, uh, you know, play transfer, uh, across some of the gaming applications that they were building. But, like, the really exciting thing was when it started working for language with ELMo and then, uh, you know, BERT and then RoBERTa and Turing and, you know, like, a bunch of things that we were doing. And that was the point where there were so many language, uh, based applications that you could imagine building on top of these things if it continued to get better and better. And so we were just sort of looking for evidence that it was gonna continue to get better and better. And as soon as we found it, like, we just started, like, all in. That was everything from doing a partnership with OpenAI to, uh, you know, like, at one point, I seized the entire GPU budgets for the whole company, and I was like, "We will no longer peanut butter these resources around." Like, we will focus them... Uh, 'cause it's all capital-intensive. It's like we will just allocate these things to things where we have really, really strong evidence-based conviction that, like, a particular path is gonna benefit from adding more capital scale.

   28. SGSarah Guo12:40

       I remember, uh, it must've been like five years back now. We were at

2. 12:44 – 21:18

   Microsoft and Open AI Partnership

   1. SGSarah Guo12:44

      dinner and, you know, now GPU capacity is the talk of the technical town, right? But you were... Like, I asked you what your, like, most pressing issue was and you were like, "How am I gonna spend on GPUs this year?"

   2. KSKevin Scott12:55

      Yeah.

   3. SGSarah Guo12:55

      "And how I'm gonna distribute those GPUs."

   4. KSKevin Scott12:57

      Yeah. Yeah, and it- and it was and it has been. It has... (laughs) Cert- certainly hasn't gotten any easier.

   5. SGSarah Guo13:02

      (laughs)

   6. KSKevin Scott13:02

      But, I mean, so Eli, like, I think, you know, the question you were asking is, like, how we decided to do the OpenAI partnership. And so, like, the- the reason that we did the partnership was two-fold. So one is, with transfer learning actually working, you can imagine building a platform for all of this stuff so that you- you're building single things where you're amortizing the cost of the things across a whole bunch of different applications. And because we have a hyperscale cloud, uh, like, one of the things that I was really, really, uh, interested in and, like, beyond interested, like, it felt, you know, just like an existential thing is how do you make sure that the way that you're building your cloud all the way from, you know, your computing infrastructure, your networks, uh, your software frameworks and whatnot, how can it really serve a whole bunch of interests beyond your own? And so, like, we felt like in addition to the high ambition things that we were doing inside of the company that we needed, uh, like, high ambition partners. And when we looked around, like, OpenAI was clearly the highest ambition partner that was in the field, you know, and I think still their ambition is just breathtaking in what it is that they're trying to accomplish. And so that was one thing, and then the second thing was, like, you know, they- they really had a very similar vision to the one that I had about, like, these things were evolving into platforms and, uh, like, we were able to... Because we were so aligned on vision for the future, like, we could figure out how to do a partnership where, uh, like, even though, like, there's just a ton of difficult things and, like, you know, I- I think there's probably some conservation law of, you know, the stress from difficulty. So it's n- n- not like it ever goes away, but, like, it- it- it- it's stress in service of a common goal and, like, that's the thing that make good partnerships work.

   7. EGElad Gil14:50

      I- I think one of the stunning things about the partnership in some sense was the timing because if I remember correctly, Microsoft made its first investment or its sig- its first significant investment in OpenAI right after GPT-2 launched. Or right around GPT-2 and this was before GPT-3 and there was such a big step function between the two of them that I think it was less obvious in the GPT-2 days that this was gonna be as important as it was. And so I'm a little bit curious, like, what were the signs that made you decide that this was a good partnership to have versus building it internally versus, uh, you know, usually as a larger company there's the old, like, buy/build-

   8. KSKevin Scott15:20

      Yeah.

   9. EGElad Gil15:20

      ... partner kind of thinking. And so I'm just sort of curious, like, how- how you all decided to- to partner in this moment in time where it's very non-obvious and you invested a large sum of money behind that.

   10. KSKevin Scott15:30

       Yeah. There... A- and- and I, like, I don't wanna, uh, like, have revisionist history and, like, paint a rosier picture than there actually was. So the- there was a huge diversity of opinions inside of the company on the wisdom of doing, uh, doing this. And so Satya, uh, like, has this thing that he talks about, uh, like, no- no regrets, uh, investing so, like, there are things where you do the investment and, like, there are multiple ways to win and, like, you, uh, you even win a little bit when you lose. Uh, and so th- this was one of those no regrets things in that, like, the very, very worst thing that could happen is we would go spend a bunch of capital on, uh, computing infrastructure and we would learn, uh, like, what to do at very high scale for building these AI training environments and, you know, you- you'd have to believe something very strange about the world of AI that you wouldn't need, uh, advanced computing infrastructure. Um, and then there were just multiple ways where, you know, like... And- and we had a bunch of evidence, uh, that, you know, we had gathered ourselves and that OpenAI had that gave us, you know, which unfortunately I can't talk about, uh, but, like, that gave us, you know, pretty reasonable confidence that scale-up was actually working. I- you- you've probably seen the, uh, you know, the famous, uh, OpenAI compute scale paper where they sort of plot on the log scale, like, how many, you know, petaflop days or, you know, whatever the, you know, unit of total compute they were using on that graph that shows, uh, you know, from 2012 when we first figured out how to train models with GPUs through, you know, I think the- the plot ends sometime in 2018, uh, yeah, that- that we're, you know, basically consuming ten times compute, more compute every year for, like, training state-of-the-art models. Uh, and- and so, like, you know, you- I just had super, super high confidence that, uh, we were never gonna get to the point where we're like, "All right, we got enough compute." Uh... (laughs)

   11. EGElad Gil17:26

       Mm-hmm. It was a very bold move. I- I think it's very striking all the amazing things Microsoft has done over the last few years in terms of just incredibly smart strategic moves that at the time didn't seem obvious and now are just, in hindsight, you know, really brilliant. I guess a more recent move is you announced a collaboration with NVIDIA to build a supercomputer powered by Azure, um, infrastructure combined with NVIDIA GPUs. Could you tell us a little bit more about your supercomputing efforts in general and then maybe a little bit more about those collaborations both th- both NVIDIA and, um, OpenAI on the supercomputing side?

   12. KSKevin Scott17:55

       Yeah, so we built our f- the first thing that we called an AI supercomputer, uh, I think we started working on it in 2019 and we deployed it, uh, at the end of that year and it was the computing environment that GPT-3 was trained on. And yeah, we- we had been building a, uh, like a progressively more powerful set of these supercomputing environments. Uh, like, we built them in a way where, like, the biggest environments, just because, you know, they're- they're very capital-intensive things, uh, tend to get used for one purpose, but the designs of these systems, like, we- we can build smaller stamps of them and they get used by lots of people. So, like, we have...... you know, tons of people who are training, you know, very big models on, uh, Azure compute infrastructure, both folks inside the company and, uh, you know, partners who can come in and... Y- it was a thing that was, like, not possible to do before where you could sort of say, like, "Hey, I would like a, I would like a compute grid of this size with, like, this powerful network, uh, to do my thing on." And so, you know, NVIDIA's been, y- you know, our compute and network partner since they bought Mellanox, uh, you know, for years now. And the thing that makes that work is generation over generation, like, you're just getting better, uh, you know, uh, price performance from the systems. Um, and, and we work super closely with them, uh, like defining, you know, what the hardware requirements need to be, um, you know, in the coming generations of GPUs, uh, 'cause, like, we have a pretty clear sense of where models are going and, like, what model architectures are evolving towards. Um, and so yeah, I mean, it's just been a super good, super good partnership. Um, yeah, like, we're, we're deploying, uh, Hopper now at scale and, you know, like, a bunch of the features of Hopper, like, you know, eight-bit floating point, uh, you know, arith- arithmetic and, you know, a bunch of other things are, like, things that, uh, you know, like, we've been planning for for a while.

   13. EGElad Gil20:08

       Yeah. I guess one a- one last question on sort of this, about supercomputer as we- as well as platform side of things is I'm a little bit curious how you view the world shifting in terms of closed source and open source models and, you know, the mix that'll exist because obviously from an Azure perspective, lots of people are running open source models on top of Azure right now.

   14. KSKevin Scott20:24

       Yeah, I mean, I, it, it is an interesting thing that people are framing it as some kind of binary thing. Like, I think you're gonna have a lot of both. Um, like, we, we, we still don't see any reason to believe that you're going to want to not build bigger models, uh, but, like, we, we just know in our own deployments, like, if you look at things like Bing Chat or Microsoft 365 Copilot or GitHub Copilot, you, you end up using a portfolio of models to do the work, and, like, you use it for performance and cost optimization reasons and you use it for, uh, you know, just sort of precision and quality reasons, uh, sometimes. Um, and so there's always this, you know, melange of things that you're, uh, that you're doing and it's never either/or. I'm, I'm actually really excited by what's going on

3. 21:18 – 32:12

   The Future of Open Source AI

   1. KSKevin Scott21:18

      with the open source community. I, I think, you know, my biggest question mark there is, like, how you go deal with, uh, like, all of the RAI and safety, uh, safety things. But, like, if you look at the technical innovation inside of the open source community, like, it's really, you know, thrilling and, like, we are, we... You know, like, we are doing some cool stuff right now, like, I was just playing around yesterday with, uh, that 12 billion parameter, uh, DALLE-2.0 model from Databricks, uh, which, like, runs quite nicely on a single machine and, like, yeah, I'm still enough of a dork to, like, love playing around-

   2. EGElad Gil21:52

      (laughs)

   3. KSKevin Scott21:52

      ... with things that run on single machines. Like, it's, you know, really, really impressive work.

   4. EGElad Gil21:56

      Yeah, yeah, yeah. Yeah, it's super cool. How, um, how do you think about that from the context of enabling AI for your business customers, um, outside of your core products? So, is there a specific sort of B2B AI stack that's coming? Are there specific tools coming? To your point, there's safety, there's analytics, there's fine-tuning, there, you know, there's so much stuff that you could potentially provide. I'm just sort of curious how you think about that.

   5. KSKevin Scott22:16

      Yeah, I mean, I don't wanna turn this into some kind of weird marketing spiel, but you know, we, we have this point of view that we started with this assumption that AI is gonna be a platform and the way that people are going to most usefully make, or, or the way that people are gonna make most use of the platform is by building tools that assist people with jobs, so it's, like, less about these fully autonomous scenarios and more about, uh, assistive tech. And so the first thing that we built was GitHub Copilot, which is a coding tool. Uh, it's a thing where you can sort of say in natural language what you would like a piece of code to do and it emits the code and then you, uh, you, you as the developer, uh, like the same way that you would take a suggestion from a pair programmer, like, you scrutinize it and co-review it and, you know, decide whether or not it makes sense for your application. And, you know, and, like, that wa- that was the first version of GitHub Copilot. It does a bunch of other things, uh, now. And so the, the thing that we have observed is this copilot pattern is actually pretty, uh, you know, pretty generic, um, and, and we, we built a bunch of copilots, uh, since then. And the way that we built them, like, there's a, there's, there's a copilot stack that looks almost like one of these OSI, you know, networking, uh, diagrams and it starts with a bunch of user interface, uh, patterns that you have, uh, like, they're now an emerging plug-in ecosystem for, uh, like, how you extend the, you know, the capabilities of a copilot for things that you can't natively get out of the model. And then it is a whole stack of things, uh, you know, sort of an orchestration mechanism, like LangChain is, uh, you know, one of the popular open source, uh, orchestrators, but, like, there are a bunch of open source orchestrators. Like, we have, uh, one that we've developed called Semantic Kernel that we've also open sourced. There is this whole fascinating world right now, uh, that didn't exist nine months ago a- around prompt construction and prompt engineering, and so, like, there's an entire art form and a, a set of tools that, that people have access to to design a meta prompt which is sort of the standing instructions to the model to, like, get it to, uh, conform itself to the application context, uh, that it's in. Uh, like, you have these new things, uh, you know, like new software development patterns, like, uh, retrieval augmented generation, a RAG is, uh, like, we were doing this before it had a name, uh, on it. Uh-... and, you know, so it's basically a way to, like, take the prompt that's flowing from the application and to inject context into the prompt that will help the model better respond. Um, and then there's a whole bunch of safety apparatus that you have, uh, so that looks a- a lot like filtering on both the way down, uh, as the prompt flows through the stack all the way down to the model as well as, you know, as it flows back up. Uh, so what things are you not gonna let a- let the application or the user, uh, send all the way down to the prompt because it's gonna get a bad response back or, like, you know, uh, what things are you gonna filter out at the last, uh, minute because, uh, like, it is a bad, uh, response that has gotten all the way through? Um, you know, and sometimes, like, you have multiple round trips through this cycle before you, like, bubble the thing all the way back up to the user to get them the response that they need. And so, like, you know, we- we have a point of view about what all- what the stack looks like, you know, which Microsoft tools exist that will help people, uh, build these things and, like, what special things you have to go do in the context of an enterprise to, like, answer the actual direct question, uh, where, you know, safety and data privacy and, like, understanding, you know, where the flows of data are and, like, which plug-ins can be enabled and, like, which can't. Uh, like all- all of those things, uh, like, I- I think are getting built out right now. And- and- and, like, the other thing too I'll say is, like, we'll build some of this stuff and, like, the community is gonna build a tremendous amount of it because, like, there's never been a platform or an ecosystem where one company builds all of the useful things. Like, that's just nonsense. Uh, like, it's just never happened. Uh, and- and to me it's the sort of super exciting thing to just see all of the energy that's happening right now. Like I- I just, like, immediately before this call I was doing a review with, uh, Microsoft Research and it's just amazing to watch MSR, uh, which is so many researchers are there, have pivoted what they're doing research on to, like, these AI-adjacent or AI, uh, like, on point things. And it feels a little bit like what MSR was like when I was an intern there in 2001, uh, where you- you had s- all of these super bright people who, like, had the tiniest little glimpse of what the future must look like that no one else had because it was the point where the PC was racing to ubiquity and, like, they were just all orienting their research around, like, what that little glimpse was, uh, that, like, maybe they had the earliest, uh, peek at. And it just s- like, feels magical.

   6. EGElad Gil27:34

      That's massive realignment of the research community right now, sort of in real time. It's- it's very exciting to watch.

   7. KSKevin Scott27:38

      Uh, I mean, and- and it's awe-inspiring. I mean, it's just crazy. It's hard to keep up. Like, super hard. Like, we went from... I mean, th- this has been the biggest surprise for- for me is, like, I just didn't realize that GPT-4 and ChatGPT were gonna catalyze as much of this as they have. Um, like, we'd sort of kind of been expecting a bunch of this stuff. You know, ChatGPT was a 10-month-old model with a little bit of RLHF on top of it and, you know, like, by- by, you know, admission, like, you know, not a beautiful user interface. It was just sort of a way to get something out there because, uh, you know, you needed some- some practice with a handful of things before the big GPT-4 launch was, uh, was coming. And, like, no one really knew that it was gonna blow up this way.

   8. EGElad Gil28:29

      And it's only five months old. That was only five months ago, which is shocking. I think everybody forgets how little time has passed (laughs) .

   9. KSKevin Scott28:35

      Yeah, just shocking. And- but- but it is the open source community and, like, the- the, you know, big tech community I think at its best. Is like, you know, everybody is sort of realigning to, like, what I think is- you know, unlike some of the other, you know, faddish things that have happened over the past, uh, handful of years. Like, I don't think this is a fad. Like, this is- this is real.

   10. SGSarah Guo28:54

       Yeah. Um, I, uh, launched my new fund about six months ago with this AI focus and a few weeks later ChatGPT comes out and-

   11. EGElad Gil29:02

       (laughs) .

   12. SGSarah Guo29:02

       ... I'd say even the people who were very prepared, like hopefully somewhat prepared-

   13. KSKevin Scott29:06

       Yeah.

   14. SGSarah Guo29:06

       ... to go, like, try to keep up or be part of that, uh, massive shift, like, feel constantly upended. But it is- it is very- it's the most fun time to be in technology in decades.

   15. KSKevin Scott29:17

       Yeah. It- look, it's al- it's also I will say a disconcerting time, uh, to be in technology because so many things are changing at once. It's changing at a pace that, you know, you probably... Like e- even me, like I- I'm- I'm- I think I might be in one of the better positions, uh, to, like, feel like I'm kinda in control of what's going on and, like, I'm not in control at all, uh, like, of the pace. Uh, and so it must really be disconcerting to folks, you know, trying to keep up with everything that's going on. And in some cases, like, it's forcing people to change their worldview about things, like worldviews that they've held for a really long time. I think it's, uh, honestly harder for some machine learning people than it is, uh, you know, for, like, a brand new entrepreneur who's, you know, just looking for an interesting thing to go do because it- it is a very different way for a machine learning team to do its work a- and it's, like, been hard, you know, even for some of the people at Microsoft who have had plenty of time to think about the transition to, like, get adjusted to, like, this new way of doing things.

   16. SGSarah Guo30:26

       I wanna ask you one more question, um, that is sort of advice for people making the adjustment in a certain sense and then, uh, you know, talk about your book, talk about the macro and such. Uh, Microsoft has a, um, unbelievably wide portfolio of products and now you're on the other side of all the infrastructure questions, figuring out the, you know, organization of adoption of all these capabilities into that portfolio, right? Um, I talked to, you know, friends who run large companies, started large companies all the time that are also figuring out how to do this. How do you- how do you organize that effort? Uh, what- what advice do you have for them?

   17. KSKevin Scott31:00

       I think you have to be, you have to remember that some things have changed and some things haven't changed at all. Um, and- and so, like one of the confusing things that I think there is for folks, uh, that- that many people get wrong is, like, models aren't products, uh, and infrastructure isn't a product. And so, you know, you- you need to very quickly understand what it is this new type of infrastructure and this new platform is capable of, but, uh, that does not mean that you get to not do the hard work of understanding like what a good product is that uses it. Like, one- one of the things I tell a lot of people is probably the place where the most interesting products are, are where you've made the phase change from im- uh, from impossible to hard. So like, something that like literally you couldn't do at all before this technology exists has become hard now, because like the- the things that have gone from impossible to easy are probably not interesting. And- and like the- the- my- my frivolous example of this is, uh, when smartphones,

4. 32:12 – 45:29

   AI for Everyone

   1. KSKevin Scott32:12

      uh, you know, came on the market 15, 16, 17 years ago now, like it's, uh, yeah, 2007 I- I guess was iPhone launch, right? So, uh, 16 years ago almost. Um, and then a year later you had the app store. So like the first apps were like things that had gone from, uh, impossible to easy. Um, and like they just, you know, we- we barely remember them, like there were all these fart apps. There was like, you know, uh, like this app I had on my phone at one point that was called the Woo Button. You pressed it and it like did a woo, like Ric Flair. Uh, like tho- those are-

   2. SGSarah Guo32:45

      (laughs)

   3. KSKevin Scott32:45

      ... those are not businesses, like they're just, you know, sort of like these explorations that people are doing. Like the- the things that have made the smartphone platform or the hard things that, uh, like went from impossible to hard, they- they also are kind of the non-obvious things, like they weren't even the things that the builders of the platform imagined. Like, you know, we- we- we don't even think the- the original applications on these platforms, like the things that launched, uh, when the platform first launched, like those are not the interesting things anymore. Like your smartphone is way more than just a SMS app and a web browser and a mail client. Uh, like the thing that makes it interesting is TikTok and Instagram and WhatsApp and DoorDash and- and like they were all of these hard things that people had to go build now that they were possible. And so like I think that's thing number one to, you know, hold in your head either as an entrepreneur or as a business that's trying to adopt this stuff. It's not like how I go sprinkle some LLM fairy dust on my existing, you know, products and do some stupid incremental thing. And like, you know, and I shouldn't even call it stupid. Like maybe the incremental things are fine. Uh, but like the really interesting things are- are- are non-obvious and very not incremental. And so th- that is the hard thing for us, is you have an entire group of people who are smart and like they can see all of the things that are possible and so the hard... The- the challenge is to steer them towards like the hard, meaningful, uh, you know, sort of interesting, non-obvious, uh, things that are possible. Like not the, you know, like things that are incremental that, you know, are just gonna burn up a bunch of GPU cycles (laughs) and prevent you from, you know, and a bunch of I- product IQ that will prevent you from doing the things that really matter.

   4. SGSarah Guo34:34

      If we- we sort of zoom out to like non-technical audiences, you wrote a book in 2020, wro- Reprogramming the American Dream. Can you describe who you want to read the book and- and what you hope they'll take away from it?

   5. KSKevin Scott34:46

      I, when I wrote the book, it was not for people like us. Well, th- so the premise of the book is that I- I grew up in rural central Virginia. My, you know, dad was a construction worker. His dad was a construction worker.

   6. SGSarah Guo35:02

      (laughs) .

   7. KSKevin Scott35:02

      His dad was a construction worker. Um, you know, my maternal grandfather like ran an appliance repair business and had been a farmer earlier in his life. So the- the thing- the thing that was true for everyone who was in my life, uh, like, you know, neighbors, members of the community is like, you know, they're just smart, entrepreneurial, ingenious people using the best tools that they could lay their hands on to go do things that mattered to them that like created opportunity for them and, you know, sort of solve problems for, you know, their- their communities. Um, and I believe that like particularly this platform vision of AI where it's sort of getting cheaper and it's getting more accessible all the time, you know, like things, you know, like the stuff that we were chatting about a- uh, you know, few minutes ago about what I did at Google. Like, I, you know, came in with a graduate degree, I was mathematically sophisticated, uh, and yet to do the thing that I... The first project that I did, which was a, you know, like a machine learning, uh, classifier thing in 2003, uh, 2003, 2004, like that was, you know, stacks of like super technical, you know, research papers and you know, uh, elements of statistical machine learning, you know, like you read it cover to cover and then you go write code for six months. Like, high school student could do the whole damn project in four hours on a weekend now. Uh, like it's just, you know, like... And- and what's happening, like that aperture of who can use the tools is just getting bigger and bigger and bigger over time. And so like the book was trying to get people to be inspired by this notion that, uh, like don't be daunted and intimidated or scared by, uh, by AI. Like go embrace it and like try to plug it into the things that you're doing and like maybe, you know, we- we've got a shot at having more equitable distribution of, you know, who's benefiting from the platform as it emerges.

   8. SGSarah Guo37:04

      If you were gonna add an update chapter for the last few years where so much has happened, what, w- w- what would you focus on?

   9. KSKevin Scott37:11

      Well, i- it's, it's really interesting how much of it I think is still true, and like I had this anxiety the whole time that I was writing the book that I was gonna... by the time I had the manuscript and it hit the presses, that all of it was gonna be out of date. Like, the real problem I had is, like, by the time it hit the presses, we had, uh, we had a, a global pandemic and, uh, it literally w- (laughs) i- it hit the presses, uh, the week that everything shut down, so like you literally couldn't buy it. Uh, like Amazon wasn't delivering anything other than essential packages and every bookstore in the country was closed. So, I mean, it's a little bit surprising, uh, you know, to me, like, how many of the i- ideas that, you know, we have a platform, platform's getting more powerful, it's getting more accessible, um, like actually the unit economics of it are getting better, uh, you know, like what you can do for, you know, a per token of inference, uh, is, like, getting higher. Uh, you know, so like I know everybody's, like, in this frenzy around GPUs and like which is this very expensive, uh, expensive thing, but like all of this optimization work is happening where you're, you're able to squeeze more out of the compute that you have and the compute's getting cheaper. So, yeah, I mean, the update that I would, I would add is that... and, and it may be an update that I do, like, it probably won't be this book, but like I'm, I'm sort of contemplating, uh, like writing, uh, something right now. I do, I do think that the, the public dialogue around AI right now is missing so many of the opportunities that we have to go deploy the technology for good. Um, you know, like all of the articles, uh, that you, you know, you read in the newspapers or, you know, around the responsible AI stuff, which is important, and like the regulatory stuff, which is important, uh, but yeah, we should have a few articles in there as well about, uh, Sal Khan's TED Talk, which is just amazing. Like, unbelievably good. And just for, you know, folks who may not have seen it, which they should go see, is like, you know, his problem is perfect for AI. So it's this two sigmas problem, this idea that students who have access to high quality individualized instruction perform substantially better than those who haven't, like controlled for everything else.

   10. SGSarah Guo39:37

       Just for our listeners' sake, the two sigma problem was this study by a guy named Benjamin Bloom which showed that your average tutored student performed above 98% of students in a control class, which is one teacher to 30 students, like a normal American classroom, uh, with reduced variance, which is amazing.

   11. KSKevin Scott39:56

       Yeah, and if you believe that that's true, then you can also believe that every student, uh, every learner in the world, uh, deserves to have access to that individualized high quality instruction at no cost, which seems like a reasonable thing, and then when you think about how you go realize that in the world, like the only way that you can realistically do it is with something like AI. And so there're so many problems that have that characteristic where we can all agree that, like, it is a universal good to do this, and then if you think about how to do it, like, you must conclude that AI is, uh, like part of the solution. Like that, that is the, you know, the reason I get up every morning and deal with people yelling at me about, like, giving my GPUs, uh, you know, for, for the fifth year in a row. Um, i- is because of, of things exactly like that. And it doesn't mean that when you talk about that and you're hopeful and optimistic about those things, or even hopeful and optimistic about all of the things that, you know, venture, venture-backed companies are gonna go do, or like the way that businesses are gonna reinvent themselves, that you are also sa- you know, f- you know giving the middle finger to, you know, the responsible AI concerns or, you know, the things that people care about on the regulatory front. Like you can care about both of those things at the same time. But like the, the thing that I can tell you is, like there is no historical precedent where you get all of these beneficial things by starting from pessimism first. Like pes- pessimism doesn't, doesn't get you to optimistic outcomes (laughs) .

   12. EGElad Gil41:34

       Yeah, it seems like, to your point, a lot of the dialogue is really lacking from global, um, education equity, global health equity, like all these things that AI as a platform should be able to produce because it's, it's cheaper, it's personalized, it can do things at the level of a, of a human in many cases in terms of being a great teacher or great, you know, physician's assistant, etc., and so it really feels like that message is lost and, you know, I think a lot of people don't mention enough how we're almost hopefully gonna enter this golden age if we let this technology actually bloom and be useful. I guess the, the question that I always have on my mind relative to all this stuff is, given the capabilities that AI continues to accumulate, how do you think about 20 years from now in terms of the best roles for people? And in particular, I think about it in the context of my kids. I'm like, okay, normally, two years ago I would've told my, I would've told my, my kids, "Go study computer science. It's the language of the future." What do you think is the right advice to give people, you know, in terms of what to study and, uh, that, that will be the things that will be most durable relative to, to the change that's coming?

   13. KSKevin Scott42:33

       Yeah, I, I think... So 20 years is a tough time horizon. You know, and I, and I-

   14. EGElad Gil42:39

       It's really tough.

   15. KSKevin Scott42:39

       ... I think if any of us are honest with ourselves, like if you rewind 20 years and you sort of imagine the predictions you would've made then, like would you have gotten here and like no- nobody would.But, like, I, I think, you know, there, there are just some sort of obvious things. Like my, my daughter, for instance, like has decided she wants to go be a surgeon, and like I think surgeon is, like, a pretty good job. Uh, like we, we do not have a, a sort of robotics exponentials right now, uh, like we've got a cognitive, uh, exponential. And, and so, like, I think all of the, like the world is just sort of full of these jobs where, um, you know, really, you know, affecting change on a physical system, like doing something in the physical world, like all of those things, uh, like we will need probably many, many more of them than we have right now. Like particularly in medicine, like nurses, surgeons, physical therapists, people who, you know, work in nursing homes. Like we, we have a, you know, rapidly aging population, and so like the burdens on the healthcare system are going to get much higher. And, you know, I do think that AI is gonna have some pretty substantial productivity impacts, uh, but like, you know, maybe it's just enough productivity impact to like, you know, make room for all of the other net new things that we will have to have there. You know, and so I, I think, you know, we got this weird thing in the United States where, like, we apportion less dignity and respect to, like, jobs like the ones that my dad had, uh, than we should. Um, you know, in G- I lived in Germany for a little while, and Germany's a little bit different on this front. Like you can, you know, you, you can go be a machinist in Germany and, you know, like that's a really great career and something that your parents are, you know, celebrate. So like I think there are like all of these careers like, you know, electricians and machinists and, you know, solar installation technicians and, I mean, just so many things that we're gonna need, like especially because we're gonna have to rebuild our entire, uh, power generation and distribution system, like in our children's lifetimes. Um, so like all of those jobs I think are super important. And then I, I would argue even that all of the creative stuff, like th- that we do, uh, there's going to be probably more need for that in the future than less, even though the tools that we're using to do the creative work, whether it's coding or making podcasts or whatnot, are gonna help us be better at it. And the reason that I say that is humans are just extraordinarily good at wanting to put humans at the center of their stories. So, so like we, we, right now, we could be, you know, making, you know, Netflix shows, uh, you know,

5. 45:29 – 51:44

   AI and the Future of Jobs

   1. KSKevin Scott45:29

      like not Queen's Gambit but Machine's Gambit, uh, like about, you know, a, a fleet of computers pla- playing chess among themselves, uh, because they're all better than the, than the very best human. Uh, nobody wants to watch that. Like y- the technology's probably good enough right now where you could have superhuman Formula 1 closed, uh, track drivers in Formula 1 cars that could, you know, do things that humans can't do. Nobody wants to watch that. Um, yeah, and like even go back before computers, like forklifts are stronger than people. You could like go have a strongman or strongperson competition that was about like which forklift could lift the most weight. Like, nobody cares about that. Like, we care about humans, like what are we saying, what do we care about, like what are we trying to express to everyone else? Like, and nothing about that's gonna change. Nothing.

   2. EGElad Gil46:20

      I think that's why people watch the Real Housewives of Dubai.

   3. KSKevin Scott46:23

      (laughs)

   4. EGElad Gil46:23

      So, uh... (laughs)

   5. KSKevin Scott46:24

      Right. (laughs) Yeah, and, and so, like and, and I don't, again, like I d- I don't want to paint too rosy a picture. Uh, every time you have a major technology platform or paradigm shift, like there's disruption, but like what we know from every one of these disruptions is you have like actually a surprising degree of need for human occu- occupation, uh, like all of the, you know, the Industrial Revolution predictions about, you know, four-hour work weeks and we're all gonna live lives of leisure is bullcrap. Um, yeah, like just hasn't happened. And I think some people may say it hasn't happened because, you know, the system, you know, the system, uh, like doesn't want it to happen. But like I think a lot of it is because, uh, like we actually like doing things.

   6. EGElad Gil47:15

      Yeah. And there's a lot to do. I guess, uh, on that note, what, what are some of the areas you're most excited about going forward in terms of the coming year of AI? Or big research areas or big product areas or things that, you know, you're very optimistic about?

   7. KSKevin Scott47:27

      I think sort of two things. Just, um, I think this will be the, maybe the great first year of foundation model deployments where you're just gonna see like lots and lots of companies launch, lots of people trying a bunch of ideas. You're gonna see all of the big tech companies will have substantial things that they're, uh, that they're gonna be building. Um, you know, like I, I got predictions about what other folks will do but, you know, like it will touch all of Microsoft's, uh, product portfolio. Like, the way that you will interact with their software will be substantially different, uh, by the end of this calendar year than it was, uh, coming in. Uh, and I think that'll be true for everyone. I think it, it changes some of the, uh, nature of the competition that you've got between big tech companies and, like, I think it creates new opportunities for small, uh, tech companies to come and drive wedges into, you know, get footholds and do interesting things. Um, you know, one of the things that Sam Altman and I have talked about a lot is like, I, I suspect that this year like the next trillion-dollar company gets founded. Um, it won't be obvious which, uh, you know, which it is. But like, um, we're, we're overdue. Like, long overdue. And, and then I think...... you know, what you're gonna see technically this year is, uh, I, I do think that you will have things like, um, the Red Pajama Project, uh, is like this, a- a- and there are going to be a bunch of others like it, uh, will make really good progress on building more capable open source models. Um, you know, and hopefully, hopefully the community will help build some of the safety, uh, safety solutions that you will need to accompany those things when you d- deploy them. Uh, but like technically, I think you're just gonna see amazing progress there. And then, like it's just, you know, the, the, the frontier will keep expanding out. Um, you know, we, we don't have... O- OpenAI doesn't have GPTV in wide distribution, but like it'll get to wide distribution at some point in the not-too-distant future, and so like you'll have these like mul- very powerful multimodal models the same way that having GBD-4, uh, admitted like all of this exciting energy around new things that you could do with it, like having a, having a model that can like take visual inputs a- and like reason over them, like will also admit a whole bunch of new things, uh, that are gonna be very exciting. So I don't know. Like that, that's, like I just think the, the, the theme of this year is gonna be like progress and activity. Like a- almost too much to track. Like I'm gonna need a, I'm gonna need a co-pilot just to like pay attention to all of this stuff and like make sure I'm not missing important things. 'Cause I feel like I'm, I'm at the... And you all as investors and as people who are watching this closely must feel the same thing. It's like, um, how do I make sure I don't miss, like, you know, the next important thing? How do I see it as soon as humanly possible?

   8. SGSarah Guo50:33

      Actually, just to make completely sure, if you are starting the next trillion-dollar company in our listener base this year, please call me and Elade.

   9. KSKevin Scott50:40

      (laughs)

   10. SGSarah Guo50:40

       And, and Kevin, too. (laughs) Um, wrapping up now, is, is there anything else you would wanna touch on, Kevin?

   11. KSKevin Scott50:48

       Um, I, look, I think the dialogue that we're having right now around regulation is actually really quite important. So a- as we're recording this, Sam Altman was testifying in front of the Senate Judiciary Committee on, uh, like Tuesday of this week. Um, I think more of those conversations are a good thing. I think as fast as things are moving, like you really will need the technology community to come together and to agree on some sensible things that we can do before the regulation even is in place, and I think that's all important and not a thing... Like, the, the thing that none of us should be doing at this point is sort of like looking at the prospect of regulation and saying, "Oh my God, this is a, you know, this is like a pain. Uh, like I don't wanna deal with this." Like the fact that there is a desire for it is like a very good signal

6. 51:44 – 54:56

   The Future of AI and Regulation

   1. KSKevin Scott51:44

      that the things that we're working on actually matter. Because like nobody's trying to regulate frivolous things. And like the purpose of regulation is to make sure that you can build a solid, trusted foundation for things that, you know, maybe b- become ubiquitous in society. Like if you think of this like electricity, for instance, you wanna strike the right balance between allowing the technology to develop and make progress and flourish, but like you also need to make sure that your electric power generators are built safely and you don't allow people to wander in and like stick their finger on the electrode and disintegrate themselves, and like you wanna make sure that, you know, the distribution of electricity is coordinated and that, you know, when it comes into your house, uh, like it doesn't burn your house down, and when you, uh, you know, like you plug your appliances into the wall, that, you know, they function as, as expected. And so like I, I think that is, you know, a similar way. Like it, there's not gonna be one size fits all. Like I think most of the stuff that people ought to be thinking about is deployments, uh, like making sure that like as you deploy the technology, getting, you know, the requirements and the expectations right there is the most important thing. Um, and then, you know, these big engines that we're building that are the, like the largest of the foundation models, like, you know, making sure that you, you have a set of safeguards around those, but like also the way that we are building these things, they don't, they don't get distributed to the world, uh, like by themselves. Uh, like there's a whole layer of things, uh, on top of them to, like render them safe, uh, and then a, a whole set of things per application, per deployment that we do to like make the deployment safe. And so like, you know, I, I think everybody, like a- all the startups, uh, like everyone in the open source community, everybody ought to be thinking about these things, like how am I doing my part to make sure that we are creating as much space as possible for these optimistic, uh, uses, and like we are deterring as many of the harmful ones as possible.

   2. EGElad Gil53:53

      Yeah. I've been impressed by the degree to which the community has self-acted from very early days in terms of AI safety and approaches to that, and so I know OpenAI has done stuff really early, Anthropic has, Google has, Microsoft has. You know, I feel like a lot of the main players have actually been, you know, remarkably thoughtful about this area and, you know, keen to make sure that it's done properly.

   3. KSKevin Scott54:13

      Yeah, I mean, it, it, the thing that I will say is we fiercely compete with a whole bunch of these folks, uh, but like one of the things that I don't do is like look at any of those companies that you just named and like worry that they're going to do something that uh, like is, like I take myself out of my role as CTO of Microsoft and like just think about Kevin citizen of the world. Like I, Kevin citizen of the world is not worried about like what my competitors are gonna do to like do something unsafe. (instrumental music plays) Like I'm just not.

   4. SGSarah Guo54:43

      Thanks so much for being with us, Kevin. We really appreciate it.

   5. KSKevin Scott54:45

      Yeah. Thanks for inviting me. This was awesome.

   6. EGElad Gil54:47

      Yeah. Thanks so much for the time. It was great. (instrumental music plays)

Episode duration: 54:56