Dwarkesh PodcastNat Friedman (Github CEO) — Reading ancient scrolls, open source, & AI
EVERY SPOKEN WORD
155 min read · 30,720 words- 0:00 – 30:00
Vesuvius Challenge
- NFNat Friedman
... we have 600 plus kind of roughly intact scrolls that we can open. And I heard, I heard about this and I thought that was incredibly exciting, like the idea that there is information from 2,000 years in the past, we don't know what's in these things. If we could read all of them, then that would give us approximately a doubling of the total texts that we have from antiquity. If there are thousands more papyrus scrolls in there and we, and we now have the techniques to read them, then there's gold in that mud, and you know, it's gotta be dug out. I just fundamentally don't believe the world is efficient and so if I see an opportunity to do something, I used to but I no longer have a reflexive reaction that says, "Oh, that must not be a good idea. If it were a good idea, someone would already be doing it."
- DPDwarkesh Patel
Okay. Today, I have the pleasure of speaking with Nat Friedman, who was the CEO of GitHub from 2018 to 2021. Before that he started and sold two companies, uh, companies, Simeon and Xamarin, and he is also the founder of AI Grant and California Yimby. And most recently he is the organizer and funder of the Scroll Prize, which is where we'll start this conversation. So Nat, do you wanna tell the audience about what the Scroll Prize is?
- NFNat Friedman
Well, we're, we're calling it the Vesuvius Challenge.
- DPDwarkesh Patel
Oh, okay. Got it.
- NFNat Friedman
And, uh, this is just this crazy and exciting thing I feel like incredibly honored to have gotten caught up in, but, uh, a couple of years ago, I was reading... It was in the midst of COVID and, uh, I think we were in lockdown and like everybody else, we were falling into internet rabbit holes, and I just started reading about the eruption of Mount Vesuvius in Italy about 2,000 years ago. And it turns out that when Vesuvius erupted, uh, it was AD 79, it destroyed all the nearby towns. Everyone knows about Pompeii, but there was another nearby town called Herculaneum and Herculaneum was sort of like the Beverly Hills to Pompeii, so big villas, big houses, fancy people. And in Herculaneum there was one villa in particular, it was enormous, and it had once been owned by the father-in-law of Julius Caesar, and so well-connected guy. And it was full of beautiful statues and marbles and art but it was also the home to a huge library of papyrus scrolls, and so when the villa was buried, the, the volcano actually, it spit out enormous quantities of mud and ash and it buried Herculaneum, in particular, in something like 20 meters of material. So it wasn't like a thin layer, it was a very thick layer. Those towns were buried and forgotten for hundreds of years. No one even knew exactly where they were until the 1700s, and so in 1750, a farm worker who was digging a well kind of in the outskirts of Herculaneum struck this marble paving stone of a path that had been at this huge villa, and of course he, he was pretty far down when he did that, he was, you know, 60 feet down, and then subsequently this Swiss engineer came in and started digging tunnels from that well shaft and they found all these treasures, and, and that was sort of the spirit at the time, was like looting.
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
You know, they were taking out like incredible... They would, you know, if they encountered a wall they would just bust through it, and they were taking out these beautiful bronze statues that had survived, and along the way they kept encountering these lumps of what looked like charcoal. They weren't sure what they were and many were apparently thrown away until someone noticed a little bit of writing on one of them and they realized they were papyrus scrolls, and there were hundreds, there may have been thousands of them. And so they had uncovered really the, this enormous library, it was the only ex- library ever to have sort of survived in any form, even though it was badly damaged, you know, they were sort of carbonized, very fragile, deformed, the only one that's survived since antiquity. It- it- in the open air, these papyrus scrolls in like a Mediterranean climate, they rot and they declay- decay quickly and so they'd have to be recopied by monks like every hundred years or so, maybe even less, and so we only have... It's estimated, you know, something like less, one percent, less than one percent of all the writing from that period. And so to find underground the hundreds of definitely not in good condition but still present (laughs) you know, papyrus scrolls where on a few of them you can make out the lettering, was like this enormous discovery. People immediately, in a well meaning attempt to read them, started trying to open them but they're, they're really fragile, like they, you know, they like, they turn to ash in your hand, and so hundreds were destroyed. People did things like, uh, cut them with daggers down the middle and you know, a bunch of little pieces would flake off and they'd try to like get a few letters off of a couple of pieces, and then eventually there was a monk, uh, named Biagio, who's an Italian monk, and he devised this machine kind of under the care of the Vatican to unroll these things very, very slowly, like half a centimeter a day, something like that. And a typical scroll I think would be 15 or 20 or 30 feet long, and managed to successfully unroll a few of these, and they found on them Greek philosophical texts in the Epicurean tradition by this little known philosopher named Philodemus. But, but we got kind of new text from antiquity which is, you know, not a thing that happens all the time. Eventually people stopped trying to physically unroll these things because so many were destroyed and in fact some attempts to physically unroll the scrolls continued even into like the 2000s, like 1990s, 2000s, and, and they were d- destroyed. So the current situation is we have 600 plus kind of roughly intact scrolls that we can open. And I heard, uh, I heard about this and I thought that was incredibly exciting, like the idea that there is information from 2,000 years in the past, we don't know what's in these things, and obviously people are trying to develop new ways and new technologies to open them, and-I read about a professor at the University of Kentucky, Brent Seales, who had been trying to scan these using increasingly advanced imaging techniques, and then use computer vision techniques and machine learning to kind of virtually unroll them without ever opening them, and they tried a lot of different things. But their most recent attempt in 2019 was to take the scrolls to a particle accelerator in Oxford, England, uh, called the Diamond Light Source and to make essentially an incredibly high resolution CT scan of them, sort of 3D X-ray scan, and they needed really high energy photons in order to do this, and they were able to take scans at eight microns, so these- these really quite tiny voxels... which they thought would be sufficient. And I thought this was like the coolest thing ever, you know, we're using technology to sort of read this lost information from the past, and I sort of waited for the news that they had been decoded successfully. Um, so that was, you know, 2020. And then I think COVID hit, everybody got a little bit slowed down by that, and last year I just found myself wondering, "I wonder what happened to, you know, Dr. Seales and his scroll project?" And I reached out and it turned out they'd been making really good progress, you know, they'd gotten some machine learning models to start to identify ink inside of the scrolls but they hadn't yet extracted words or- or passages. It's very challenging, and I invited him to come out to California and hang out, and to my shock, he did. And, um, we got to talking and decided to team up and try to crack this thing, and the approach that we've settled on to do that is to actually launch an open competition. We're gonna... we've done a ton of work with his team to get the data into a shape where... and- and the tools and techniques and just the broad understanding of the materials, into a shape where smart people can kind of approach it and get productive easily, and then I'm putting up, together with Daniel Gross, a prize, you know, sort of like an XPRIZE or something like that, uh, for the first person or team who can actually read, like, substantial amounts of real text from one of these scrolls without opening them. And so we're launching that this week, you know, I guess maybe it's when- when this airs, I don't know. Um, the stakes are kind of big, like, this... like, what gets me excited are the stakes. So the six or 800 scrolls that are there, it's estimated that if we could read all of them and... you know, like, f- somehow if the technique works and it generalizes to all the s- scrolls, then that would give us approximately a doubling of the total text that we have from antiquity. This is what historians and classicists tell me. So it's not like, oh, we would get like a 5% bump or a 10% bump in the total ancient Roman or Greek text, it would be like, no, we get all of the text that we have again. You know, multiple Shakespeares is sort of one of the units that I've heard, so... so that- that would be significant. I mean, we don't know what's in there, you know, we've got a few Philodemus texts, those are of some interest, um, but there could be lost epic poems or God knows what, so I'm really excited and I think, you know, my bet is there's like a 50% chance that someone will encounter this opportunity and get the data and get nerd sniped by it and we'll solve it this year.
- DPDwarkesh Patel
I mean, really, it is something out of a science fiction novel, you know, it's like something you'd read in Neal Stephenson or something. Um, I was talking to, uh, Professor Seales before and apparently the sh- shock went both ways because the first few emails, he goes like, "This has got to be spam." (laughs) Like, "No way Nat Friedman is reaching out and has found out about this prize and..." Uh...
- NFNat Friedman
That- that's really funny because he was really (clears throat) pretty hard to get in touch with, um...
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
So like I- I emailed him a couple times, s- just, like, didn't respond, and so I was like, "Ugh," so I asked my admin, Emily, to call the secretary of his department-
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
... and say like, uh, "Mr. Friedman requested me," and then, like, he knew there was something, like, actually going on there and so he finally got on the phone with me and we got on Zoom and, uh, he's like, "Why are you interested in this?" (laughs)
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
(laughs) I mean, I love Brent, he's fantastic, and um, I think I... you know, we're- we're like friends now and, uh, I think we found that we think alike about this and I think he's reached the point where he just really wants to f- crack the... you know, they've- they've taken this right up to the one yard line, like, ah, this is doable at this point. They've demonstrated, I think, every key component, but putting it all together, improving the quality, doing it at the scale of a whole scroll, this is still very hard work.
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
And an open competition seems like the most efficient way to get it done.
- DPDwarkesh Patel
Before we get into the state of the data and the different-
- NFNat Friedman
Yeah.
- DPDwarkesh Patel
... possible solutions, um, I wanna make tangible, like, what could be gained if we can unwrap these. So you said there's a few more thousand scrolls, are we talking about the ones in, uh, Philodemus' lair or are we talking about the ones in other lairs?
- NFNat Friedman
Yeah. Well,
- NANarrator
(...)
- NFNat Friedman
You know, you'd think if you find this crazy villa that was owned by Julius Caesar's father-in-law, that we'd just like dig the whole thing out, um, but in fact most of the exploration occurred in the 1700s through the Swiss engineers' tunnels underground, so it was never... the villa was never dug out and exposed to the air. You went down 50, 60 feet and then you'd dig tunnels and, you know, again, they were looking for treasure, not, like, a full archeological exploration, so they mostly got treasure. In the '90s, some additional excavations were done kind of at the edge of the villa and they discovered a couple of things. First they discovered this, like... it was a seaside villa, it faced the ocean, it was right on the water before the volcano erupted. The eruption actually pushed the shoreline out by depositing so much additional mud there so it's no longer right by the ocean apparently. I've actually never been. And, uh, they also found that there were two additional floors in the villa that the tunnels apparently had never excavated, and so at most a third of the villa has been excavated. Now they also know when they were, uh, discovering these papyrus scrolls that they- they found basically one little room-... where most of the scrolls were. And these were m- mostly these Philodemus texts, at least that's what we know. And they found apparently several revisions sometimes of the same text. And so they think, the hypothesis is this was actually Philodemus's working library. He worked here, this, this sort of Epicurean philosopher. And in the hallways though, they occasionally found other scrolls, including crates of them. And the belief is, at least this is what historians have told me, I'm, I'm no expert, but what they have told me is they think that the main library in this villa has probably not been excavated.
- DPDwarkesh Patel
Mm.
- NFNat Friedman
And that the main library may be a Latin library and may contain, you know, literary texts, th- you know, historical texts, other things, and that it could be much larger. Now, I don't know how prone these classicists are to wishful thinking. Uh, it is a romantic idea but they have some evidence, you know, in the presence of these, uh, partly evacuated sort of scrolls that were found in hallways and, and that sort of thing. So there are descriptions, and I've since gone and read a bunch of the, uh, like firsthand accounts, uh, of, of the excavations. And there were these heartbreaking descriptions of them finding, like, an entire case of scrolls in Latin and, like, accidentally destroying it as they tried to get it out of the mud and, you know, there were maybe 30 scrolls or something in there. So th- there clearly was some other stuff that we just haven't got to.
- DPDwarkesh Patel
I mean, you, you made some, uh-
- NFNat Friedman
Yeah.
- DPDwarkesh Patel
... some scrolls in, uh-
- NFNat Friedman
We do so, okay, so-
- DPDwarkesh Patel
... ferns, right?
- 30:00 – 37:39
Finding points of leverage
- NFNat Friedman
rest.
- DPDwarkesh Patel
I- I- I wonder if the budget for archeological movies and games like Uncharted or Indiana Jones is bigger than the actual budget to do real world archeology. But, you know, I was talking to some of the people before this interview, and that's one thing they emphasized, is your ability to find these leverage points. For example, with California Yimby, I don't know the exact amount you seeded it with-... but, um, uh, for that amount of money, it is, uh, and for an institution that is that new, it is one of the very few institutions that has had a significant amount of political influence, right? Like, if you look at the state of EMB in California and nationally today. I guess, how do you identify these things? Like how do you see... I mean-
- NFNat Friedman
Yeah.
- DPDwarkesh Patel
... there's plenty of people who have money who get into history or get into every subject.
- NFNat Friedman
Yeah.
- DPDwarkesh Patel
Very few do something about it, right? Like how do you figure out where to-
- NFNat Friedman
I don't... You know, I'm a little bit mystified by why people don't do more things too. Um, like I think, first of all, I don't know, maybe you can tell me, why are more people doing things? Like I think most rich people are boring-
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
... and they should do more cool things. Um, so I'm hoping that they do that now. Um, but yeah, I mean, I don't know, like, um, I think part of it is I just fundamentally don't believe the world is efficient. And so if I see an opportunity to do something, I don't have a ref-... I used to, but I no longer have a reflexive reaction that says, "Oh, that must not be a good idea. If it were a good idea, someone would already be doing it." Like, someone must be taking care of housing policy in California, right? Or somebody must be, you know, taking care of this or that. And so like I think, uh, you know, first I like, I don't have that filter that says the world's efficient and don't bother, someone's probably got it covered. And then the second thing is, I kind of have learned to trust my enthusiasm. You know, it was... This gets me in trouble too, but if I get like really enthusiastic about something and that enthusiasm kind of persists, um, I just indulge it and just think, oh yeah, I'm gonna go like... You know, like I like doing the things I'm enthusiastic about and so I, I just kind of let myself be impulsive and... So frequently what you do, uh, there's this great, you know, image that I found and tweeted which said, um, "We do these things not because they are easy, but because we thought they would be easy."
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
And so yeah, like that's frequently what happens is like the commitment to do it is impulsive 'cause... And it's done out of enthusiasm and then you get into it and you're like, "Oh my God, this is like really much harder than we expected." But then you're sort of committed and you're stuck and you're gonna have to get it done. Like, I thought this project would be relatively straightforward, we're just gonna take the data and put it up and... But of course everything is... And, and truly 99% of the work has already been done by Dr. Seales and his team, uh, at the University of Kentucky. I, I am a kind of carpetbagger, I've shown up at the end here, you know, to like try to do a new piece of it. But-
- DPDwarkesh Patel
The last mile is often the hardest.
- NFNat Friedman
Well, I mean, it's- it turned out to be fractal anyway, like the... You know, just like all the little bits that you have to get right to do a thing and have it work and, you know, I, I hope we got all of them, but... So I think that's part of it is just like, yeah, not believing the world's efficient, then just like allowing your enthusiasm to cause you to commit to something that turns out to be a lot of work and really hard and then you just are like stubborn and don't want to fail and so you keep at it. I don't know, I think that's it.
- DPDwarkesh Patel
Yeah. I, I, I don't know. I, I feel like the efficiency point, do you think that's particularly true just of things like, um, California EMB or this where there isn't a direct monetary incentive or...
- NFNat Friedman
No. I mean, look, certainly parts of the world are more efficient than others-
- DPDwarkesh Patel
Right.
- NFNat Friedman
... and, uh, you can't assume equal levels of inefficiency everywhere. But I'm, I'm like constantly surprised by how even in areas you expect to be very efficient there, there are things that are sort of in plain sight that no... And it's not that I see them and others don't, you know, there's lots of stuff I, I don't see too. I was talking to some traders at a hedge fund recently and I asked them... I was trying to understand the role secrets play in the success of a hedge fund and the reason I was interested in that is because I think the AI labs are going to enter a new similar dynamic where their secrets are very valuable. Like if you have a 50% training efficiency improvement and your training runs cost $100 million, that is a $50 million secret that you have that you want to keep and hedge funds do that kind of thing routinely. And so I asked, uh, some traders at a very successful hedge fund, uh, "If you had maybe your smartest trader get on Twitch for 10 minutes once a month and on that Twitch stream describe their 30-day old trading strategies," right? "So not your current ones, but the ones that are a month old, what would that... How would that affect your business after 12 months of doing that?" So 12 months, 10 minutes a month, 30-day look back, so it's two hours in a year and to my shock they told me 80% reduction-
- DPDwarkesh Patel
Huh.
- NFNat Friedman
... in their profits. Like it would have a huge impact. And then I asked, "Okay, so how long would the look-back window have to be before it would have like a relatively small effect on your business?" And they said 10 years. So like that I think is just quite strong evidence that the world's not perfectly efficient 'cause you know these folks make billions of dollars using secrets that could be related in like an hour or something like that and yet others don't have them or their secrets wouldn't work. And so I think there are different levels of efficiency, um, in the world, but on the whole, our like default estimate of how efficient the world is is far too charitable.
- DPDwarkesh Patel
Mm. On, on the particular point, by the way, of AI labs potentially sharing secrets, I mean you have this sort of strange norm of different people from different AI labs not only being friends but like often living together, right? So it would be like Oppenheimer living with somebody working on the Russian atomic bomb or something like that. Do, do you think those norms will persist once the value of these secrets is realized?
- NFNat Friedman
Yeah, I m- I was just wondering about that some more today. I mean, it's, it seems to be sort of slowing, you know, they seem to be trying to close the valves-
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
... um, but I think there's a lot of things working against them in this regard. So one is again that the secrets are relatively simple. Two is that you're coming off this academic norm of publishing and really o-... Like, like the entire culture is based on sort of sharing and publishing. You know, three is as you said they all live in group houses, some are in polycules, you know, there's just a lot of, um, intermixing. And then it's all in California and California's a non- you know, non-compete state-
- DPDwarkesh Patel
Right.
- NFNat Friedman
... we don't have non-competes. And so we'd have to change the culture-... get everybody their own house and move to Connecticut, and then, you know-
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
... maybe it would work. You know, I- I- I think ML engineer salaries and compensation packages will probably be adjusted to try to, you know, address this, because you don't want your secrets walking out the door.
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
There are engineers, you know, uh, Igor Babushkin, for example, um, who has just, I believe, joined Twitter, I think. Is that right? Uh, Elon hired him to train... I think th- that's public. Is that right? I think it is.
- DPDwarkesh Patel
It will be now. (laughs)
- 37:39 – 40:32
Open Source in AI
- DPDwarkesh Patel
As- as somebody who has been involved in open source their entire life, are you happy that this is the way that AI has turned out, or do you think that this is less than optimal?
- NFNat Friedman
Well, I don't know. My- my opinion's been changing. I'm- I have increasing worries about kind of safety issues, like, um, not- not the hijacked version of safety, but, uh, some industrial accident type situations or- or misuse. And so I do think there's- there's some... We're not in that world, and I'm- I'm not particularly concerned about it in the- in the short term, but in the long term, I do think there are worlds that we should be a little bit concerned about, although I don't know what to do about, um, where, yeah, like, bad things happen. Mo- the probability mass of my belief is- though is that it is probably better on the whole for more people to get to tinker with and use these models, at least in their current state. And so, for example, when, you know, Georgy Gerganov this weekend did a four-bit quantization of the LLaMA model and got it, you know, inferencing on an M1 or M2, I was very excited and I got that running and it's, like, fun to play with. Now I've got a model that, you know, is, like, very good, it's almost GPT-3 quality, runs on my laptop and... You know, I've- I've sort of grown u- grown up in this world of the tinkerers and open source folks, and the more access you have, the more things you can try. And so I- I- I think I do find myself, you know, very attracted to that.
- DPDwarkesh Patel
Uh, I- I guess that is the scientist, uh, and the ideas part of what is being shared, but there's also another part about the actual substance, right? So, like, the uranium and the sort of atom bomb analogy. As, I guess, different sources of data realize how valuable their data is for s- uh, training newer models, do you think that these things are going harder to scrape, LibGen Archive? Are these going to become rate limited in some way or... What- what are you expecting there?
- NFNat Friedman
Well, first, there's so much data on the internet.
- DPDwarkesh Patel
Right.
- NFNat Friedman
I mean, the two kind of primitives that you need to build models are you need lots of data, we- we have that in the form of the internet, we digitized the whole world into the internet, and then you, and then you have, you need these GPUs, which we have because of video games. So you take, like, the internet and video game hardware and you smash them together and you get machine learning models, and they're both commodities. And so I think the data, I don't- I don't think anyone in the open source world is really gonna be data limited for a long time. There's so much that's out there. Probably people who have, like, proprietary datasets that are readily scrapable have been shutting those down. (laughs) You know?
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
So get your scraping in now if you, uh, if you need to do it. But, um, that's just on the margin. I- I still think there's- there's quite a lot that's out there to work with. So I- I think, look, there's gonna be a ton... This is the year of proliferation. This is a week of proliferation. Like, we're gonna see four or five major AI announcements this week. You know, new models, new APIs, new platforms, new tools from all the different vendors. Um, in a way, this, you know, they're all looking forward. My Herculaneum project is looking backwards. (laughs)
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
You know? But I think it's extremely exciting and cool, but it is sort of a funny contrast.
- 40:32 – 50:18
Github Acquisition
- NFNat Friedman
- DPDwarkesh Patel
Okay, so that, uh... I- I guess before I delve deeper into AI, I- I do want to talk about GitHub.
- NFNat Friedman
Yeah.
- DPDwarkesh Patel
So I think we should start with, you were at Microsoft, and at some point you realized that GitHub is very valuable and worth acquiring. How did you realize that and how did you convince Microsoft to purchase GitHub?
- NFNat Friedman
Well, so I had started a company called Xamarin together with, uh, Miguel de Icaza and Joseph Hill, and, uh, we had built kind of mobile tools and platforms. And, uh, Microsoft acquired the company in 2016. And, uh, I was excited about that and I thought it was great. Uh, but to be honest, I didn't actually expect or plan to spend, you know, more than a, kind of a year or so there. But when I got in there, I got exposed to what Satya was doing and just the quality of his leadership team. I was really impressed. And, um, actually, I think I saw him in the first week or so I was there and he asked me, "What do you think we should do at- at Microsoft?" And I said, "Well, I think we should buy GitHub." And he said, "Can't-"
- DPDwarkesh Patel
When would this have been?
- NFNat Friedman
This was, like, my first week. It was, like, a- March or April of 2016.
- DPDwarkesh Patel
Of... Okay.
- NFNat Friedman
And then he said, um, "Yeah, it's a good idea. We thought about it. I'm not sure we can get away with it." Or something like that.
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
And then it was about a year later, little more than a year later, yeah, I wrote a- I wrote him an email, just a memo, you know, I sort of said, like, "I think it's time to do this." There was some noise that Google was sniffing around. I think that may have been manufactured by the GitHub team. But it was a good catalyst because it was something I thought made a lot of sense for Microsoft-
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
... to do anyway. And so I wrote an email to Satya, sort of a little memo saying, you know, "Hey, I think we should buy GitHub. Here's why, here's what we should do with it." And the basic argument was developers are making IT purchasing decisions now. It used to be this sort of IT thing, you know, and now developers are leading that purchase and- and it's, you know, this- this sort of major shift in how software products are- are acquired. And Microsoft really was an IT company.
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
It was not a developer company in- in the way most of its purchases were made.... but it was founded as a developer company, right? And so, you know, the, Microsoft's first product was a programming language. Um, yeah, I said, "Look, the challenge that we have is there's an entire new generation of developers who have no affinity with Microsoft, and the largest collection of them is at GitHub. And if we acquire this, and we do a merely competent job of running it, we can earn the right to be considered by these developers for all the other products that we do." And to my surprise, Satya replied in like six or seven minutes, and said, "I think this is very good thinking. Let's meet next week or so and talk about it." And I ended up at this conference room with him and Amy Hood and Scott Guthrie and Kevin Scott and several other people, and, uh, they said, "Okay, make, you know, tell us what you're thinking," and I kind of did a little 20-minute ramble on it. And Satya said, "Yeah, I think we should do it, and, uh, why don't we run it independently like LinkedIn? Nat, you'll be the CEO." And he said, "Do you think we can get it for two billion?"
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
And I said, "Well, s- we could try." (laughs)
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
And, uh, three weeks later, and he said, "Okay, go, go do this, Scott, you know, Scott'll support you on this." Three weeks later, we had like a signed term sheet and, and a announced deal. Um, and then it was an amazing experience for me. I'd been there less than two years and, you know, Microsoft was made up of and run by a lot of people who'd been there for many years, and they trusted me with this really big project, and, uh, it made me feel really good, you know, to be trusted and empowered. And I had grown up in the open source world, and so for me to get an opportunity to run GitHub, it's like, I don't know, getting appointed mayor of your hometown or something like that.
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
It, it felt cool. Um, and I really wanted to do a good job for developers. And so, that's, that's how it happened.
- DPDwarkesh Patel
That's actually, uh, one of the things I wanna ask you about. Because often when something succeeds, we kind of think it was inevitable that it would succeed. But at the time, I remember, I mean, uh, I, it was like a while back, but I remember that there was a huge amount of skepticism. I would go on like Hacker News and like the top thing would be the blog post about how Microsoft's gonna mess up GitHub. And I guess people are, have... But those concerns have been ev- alleviated throughout the years.
- NFNat Friedman
Yep.
- DPDwarkesh Patel
But how did you get, you know, deal with that skepticism and deal with that distrust?
- NFNat Friedman
Well, I was really paranoid about it.
- DPDwarkesh Patel
Yeah.
- NFNat Friedman
And I really cared about what developers thought. I think there's always this question of, who are you performing for? Like, who do you actually really care about? Sort of who's in... Who's the audience that's in your head that you're trying to, you know, do a good job for, impress, earn the respect of, whatever it is. And though I love Microsoft and care a lot about Satya and the, everyone there, I really cared about the developers, you know. I'd, I'd grown up in this open source world, and so for me to do a bad job with this central institution in open source would've been a devastating feeling for me. It was very important to me not to. So that was sort of first thing, is just that I cared. And then the second thing is that the deal leaked. Uh, it was gonna be announced, I think, on a Monday, it leaked on a Friday and, uh, Mi- Microsoft's buying GitHub. And, uh, the whole weekend, there were like terrible posts online, you know, people saying, "We gotta evacuate GitHub as quickly as possible." And, and, uh, we're like, "Oh my god, this is terrible." And then Monday, we put the announcement out and we said, "We're acquiring GitHub. It's going to run as an independent company." And then it said, "Nat, you know, Nat Friedman's gonna be CEO." And, you know, I, I had... I don't wanna overstate or whatever, like, uh, but I think a couple people were like, "Oh, Nat comes from open source. You know, he spent some time in open source, so, you know, it's gonna be run independently." So I don't think they were really that, that calmed down. But at least so- a few people thought like, "Well, maybe I'll give this a few months and just see what happens before I migrate off." (laughs) And then my first day as CEO after we got the deal closed, at 9:00 AM the first day, uh, I, you know, I, I was in this room, and we got on Zoom, and all the heads of engineering and product, and I think maybe... I don't know what people were expecting, but I think maybe they were expecting some kind of longer term strategy or something. But I came in and I said there was this, uh... GitHub had no official feedback mechanism that was publicly available, but there were s- several GitHub repos that community members had started. Isaac from NPM had started one, uh, where he'd just been allowing people to give GitHub feedback, and people had been voting on this stuff for years. And I kind of shared my screen and put that up, sorted by votes, and said like, "We're gonna pick one thing from this list and fix it by the end of the day and ship that. Like, just one thing." And I, you know, I think people were like, like, "This is the new CEO strategy?"
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
(laughs) Yeah. You know, and they were like, "I don't know. We can't, you know, you have to do database migrations. Uh, can't do that in a day." And like, uh, and then someone's like, "Well, maybe we can do this. You know, we... This is sort of... We actually have a half implementation of this." And, uh, we eventually found something that we could fix by the end of the day. And what I'm thinking is, what I'm thinking, what I, I hope I said was, what we need to show the world is that GitHub cares about developers, not that it cares about Microsoft. Like, if the first thing we did after the acquisition was to add Skype integration, developers would've said, "Oh, we're not your priority. Like, you have new priorities now." And so the idea was just to find ways to make it better for the people who use it and have them see that we cared about that immediately. And so I said, "We're gonna do this today, and then we're gonna do it every day for the next 100 days." And it was cool because I think it created some really good feedback loops, at least for me. One, one was, you know, you ship things and then people are like, "Oh, hey, I've been wanting to see this fixed for years and now it's fixed." It's a relatively simple thing. So you get this sort of nice dopaminergic, you know, feedback loop going there, and then people in the team feel the, you know, excitement of shipping stuff. Um, I think GitHub was a company that had a little bit of stage fright about shipping previously, and so to break that static friction and, and ship a little bit more, I think felt good. And then the other one is just the learning loop. By trying to do lots of small things, I got exposed to like, okay, this team is really good, you know, or this part of the code has a lot of tech debt, or, hey, we shipped that and it was actually kind of bad. How, how come that design got out? Like, where... You know? And so you... Whereas if the project had been some six-month thing, I'm not sure my learning would've been quite as quick about the company, and there's still things I missed and mistakes I made for sure.... but that was part of how I think. You know, and I, you know, no one knows counterfactually what, uh, whether that made a big difference or not, but I do think that earned some trust.
- DPDwarkesh Patel
I mean, most acquisitions don't go well. Not only do they not go as well, but, like, they don't go well at all, right? Like, as we're seeing in, in the last few months, uh, with a certain one. Well, why do most acquisitions fail, or f- fail to go well?
- 50:18 – 1:11:47
Copilot origin Story
- NFNat Friedman
- DPDwarkesh Patel
And then, so I wanna go into the story of Copilot because, uh, un- until ChatGPT, I guess, it was, like, the, the most widely used application of the modern AI models. What, whatever part of the story you're willing to share in public.
- NFNat Friedman
Yeah, I mean, um, I've talked about this a little bit. I mean, so look, i, uh, GPT-3 came out in May, I think, of 2020 and I saw it and it really blew my mind. Um, I thought it was amazing and I was CEO of GitHub at that time and I, um, I thought like, "I don't know what, but we've gotta build some product with this." This is, you know, "We've got to build something." So Satya had, at I think Kevin Scott's urging, um, already invested in OpenAI, like a year before GPT-3 came out. Like this is, like, quite amazing. And he invested, like, a billion dollars. So-
- DPDwarkesh Patel
By the way, do you know why he knew that OpenAI would be worth investing in at that point?
- NFNat Friedman
I don't know, actually. I s- I've never asked him. But, uh, yeah, I'm not sure. That's a good question. I mean, I think OpenAI had already had some successes that were noticeable and I think if you're Satya and you're running this multi-trillion dollar company, you're trying to execute well and serve your customers, but you're always looking for the next gigantic wave that is gonna upend the technology industry. It's not just about trying to win cloud. It's like, okay, what comes after cloud? And so you want, you have to make some big bets and I think he thought AI could be one. Um, and I think Kevin Scott deserves a lot of credit for really advocating for that aggressively. And I think Sam Altman did a good job of building that partnership, uh, because he knew that he needed access to the resources of a company like Microsoft to, to build, you know, large-scale AI and eventually AGI. And so I think it was some combination of those three people kind of coming together to make it happen, but I still think it was a very prescient bet. You know, people, I've said that to people and they say, "Well, a billion dollar's not a lot for Microsoft." Yeah, but like there were a lot of other companies that could have spent a billion dollars to do that and did not. (laughs) And so I still think, like, that deserves a lot of credit. Okay, so GPT-3 comes out. Uh, you know, I pinged Sam and Greg, I think, Brockman at, at OpenAI and, um, they were like, "Yeah, like, let's, we've already been experimenting with GPT-3 and derivative models in coding context. Like, let's, let's definitely work on something." And to me at least and a few other people, it was not incredibly obvious what the product would be. Um, now I think it's trivially obvious. You know, autocomplete, my gosh, isn't that what the models do? But at the time actually, my first thought was that it was probably gonna be like a Q&A chatbot stack overflow type of thing, and so that was actually the first thing we prototyped. Um, so, uh, we grabbed a couple of engineers, um, uh, this guy Olga who had come in from a, a, an acquisition that we'd done, a- and Alex Graveley, and, uh, started prototyping and the first prototype was a chatbot and, you know, what we discovered first was that the demos were fabulous. Like, every AI product has a fantastic demo. You get this sort of wow moment. So like, that is, turns out to be maybe not a sufficient condition for a product to be good because it was just at, at the time the models were just not reliable enough, they were not good enough. You know, I ask you a question, 25% of the time you give me an incredible answer that I love, 75% of the time your answer is useless or wrong. It's not a great product experience. And so then we started thinking about code synthesis and our first attempts at this were actually large chunks of code synthesis, like synthesizing whole function bodies, and we built some tools to do that and put them in the editor and that also was not really that satisfying. And so the next thing that we tried was to just do simple, small scale autocomplete with the large models and we used the kind of IntelliSense drop-down UI to do that. And that was better, like, definitely pretty good, but the UI was not quite right and we lost the ability to do this large-scale synthesis. You know, we, we still have that but the UI for that wasn't good and, uh, we had it I think so that you, to get a function body synthesized you would hit a key and then, I don't know why this was the idea everyone had at the time but several people had this idea that it should display, um, multiple options for the function body and then the user would read them and pick the right one and I think the idea was that we would use that human feedback to improve the model. But that turned out to be a bad experience because first you had to hit a key and explicitly request it, then you had to wait for it-... and then you had to read, you know, three different versions of a block of code. Reading one version of a block of code takes some cognitive effort. Doing it three times takes more cognitive effort. And then most often, the result of that was like you... none of them were good or you didn't know which, you know, (laughs) which one to pick and the... and so, that was also like, you're putting a lot of energy and you're not getting a lot out, it's sort of frustrating. So once we had that sort of single line completion working, I think Alex had the idea of saying we can use the cursor position in the AST to figure out heuristically whether you're at the beginning of a block in the code or not. And if it's not the beginning of a block, just complete a line. If it's the beginning of a block, show inline a full, you know, block completion. So the- the sort of number of tokens you request and when you stop gets- gets altered automatically with no user interaction. And then the idea of using the sort of gray text, like Gmail had done in the editor. And so we got that implemented, and it was really only kind of once all those pieces came together and we started using a model that was small enough to be low latency but big enough to be accurate that we reached the point where like the median new user loved CoPilot and wouldn't stop using it. And that took four months, five months of just tinkering and sort of exploring, you know, there were other dead ends that we hit along the way. And, um, and then, yeah, I think that... Then it was- became p- quite obvious that it was good because we had hundreds of internal users who were GitHub engineers. And I remember the first time I looked at the retention numbers, they were extremely high. It was like, I don't know, it was like 60 plus percent after 30 days from first install. Like, if you installed it, the chance that you were still using it after 30 days was like over 60%. And it's a very intrusive product. I mean, it's sort of always popping UI up, and so y- if you don't like it, you will disable it. Um, and four- if indeed, 40 something percent of people did disable it. But those are very high retention numbers for like an alpha first version, you know, of a product that you're using all day. And so then I was, you know, just incredibly excited to- to launch it and then now it's, you know, it's improved dramatically since then.
- DPDwarkesh Patel
Yeah, yeah. Sounds very similar to the Gmail story, right? Of it's, uh, incredibly valuable inside and then it becomes obvious that it needs to go outside. Okay. We'll- we'll go back to the AI stuff in a second, but, uh, some- some more GitHub questions. By what point will, if ever, will GitHub profiles replace resumes for programmers?
- NFNat Friedman
That's a good question. I mean, I- I think they're a contributing element to how people, like, try to understand a person now, but I don't think they're like a definitive resume. You know, we introduced READMEs on profiles when I was there and I was excited about that, 'cause I thought it gave people like some degree of personalization. I think some people have, you know... I mean, many thousands of people have- have done that. Um, yeah, I don't know. I... ther- there's forces that push in the other direction too on that one, where people like don't want their activity and skills to be as legible, and there may be some adverse selection as well where the people with the most elite skills, you know, it's rather gauche for them to signal their competence on their profile. So there's some weird like social dynamics that feed into it too, but I will say, I think it effectively has this role for people who are breaking through today. Like, one of the best ways to break through... I- I know many people who are in this situation. You w- you were born in Argentina, you, um, you're a very sharp person, but you didn't grow up in like a highly connected or- or prosperous network, family, et cetera, and yet you know you're really capable and you just wanna get connected to kind of the most elite part... communities in the world. And so if you're good at programming, you can join open source communities and contribute to them, and you can very quickly accrete a global reputation for your talent, which is legible to many companies and individuals around the world.
- DPDwarkesh Patel
Mm.
- NFNat Friedman
And suddenly you find yourself getting a job and moving maybe to the US or maybe not moving, or you end up at a squit startup. I mean, I know a lot of people who've, like, deliberately pursued this strategy of, you know, building reputation in open source and then kind of you've got the sail up and the wind catches you and, you know, you're- you've got a career. Um, and so I- I think it plays that role in that sense, but in other communities, like in machine learning research, this is not how you... You know, there's- there's a thousand people, their reputation is more on arXiv, you know, than it is on GitHub. So I don't know that it'll ever be comprehensive.
- DPDwarkesh Patel
Are there any other industries for which proof of work of this kind will eat more into the way in which people are hired?
- NFNat Friedman
Well, I think there's a labor market dynamic in software where the really high quality talent is so in demand and the supply is so much less than the demand that it shifts power onto the developers such that they can require of their employers that they be allowed to work in public.
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
And, uh, because... And then when they do that, they develop an external reputation-
- DPDwarkesh Patel
Yeah.
- NFNat Friedman
... which is this asset they can port between companies. And if the labor market dynamics weren't like that, if- if programming well were less economically valuable, then they would not... la- you know, the- the... y- like, th- th- uh, they would not have... Companies wouldn't let them do that. They wouldn't let them, like, publish a bunch of stuff publicly. They'd say like, "We're not g- that's a rule." And that used to be the case, in fact. And so as software's become more valuable, developers, um, the- the- uh, the leverage of like a single super talented developer has gone up and they've been able to demand, over the last several decades, the ability to work in public. And, um, I think that's not going away.
- DPDwarkesh Patel
Other than that, what has been... I mean, we- we talked about this a little bit, but what has been the impact of developers being more empowered in organizations, even ones that are not traditionally IT organizations?
- NFNat Friedman
Yeah. I mean, software is- is- is kinda magic, right? I mean, the... You can write a for loop and do something a lot of times and like... You know, like when you build large organizations at scale, one of the things that does surprise you is the degree to which you need to systematize the behavior of the people who are working. Like, when I first was starting companies and building sales teams-I had this wrong idea coming from the world as a programmer, that salespeople were, like, hyper-aggressive, hyper-entrepreneurial, you know, making promises to the customer that the product wouldn't do, and that the main challenge you had with salespeople was, like, restraining them from going out and, like, you know, aggressively cutting deals that shouldn't be cut. And what I discovered is that that does exist sometimes but, like, the much more common case is that you need to build a systematic sales playbook, which is almost a script that you run on your sales team, where your sales reps know the process they need to follow to, like, exercise this repeatable sales motion and get a deal closed. And so, you know, I just had bad ideas there, I didn't know that that was how the world worked, but software is a way to, like, systematize and scale out a valuable process extremely efficiently. And I think the more digitized the world has become, the more valuable software becomes, and the more valuable become the developers who can create it, essentially.
- DPDwarkesh Patel
Would 25-year-old Nat be surprised with how well open source worked and how pervasive it is?
- NFNat Friedman
Yeah. I think that's true. Yeah. I- I, um, I think we all have this image when we're young that these institutions are these implacable edifices that are evil and all-powerful and, you know, are able to, like, substa- with master plans, substantially orchestrate the world. And that is som- sometimes a little bit true but, like, they're very vulnerable to these, um, yeah, like new ideas and new forces and new communications media and stuff like that. So right now, I think, like, maybe I wouldn't... Right now, I think our institutions overall look relatively weak and certainly they're weaker than I thought they were back then. So I thought Microsoft... Honestly, I thought Microsoft could stop open source. I thought that was a possibility. You know, they can do some patent move and kind of there's a master plan to ring fence open source in. And, um, yeah, that didn't, that didn't end up being the case. In fact, Microsoft, when we bought GitHub, we, um, we pledged all of our (laughs) patent portfolio to open source. That was one of the things that we did as part of it (laughs) , and so that was a kind of poetic moment for me, having been on the other side of patent discussions in the past, to, uh, be a part of an instrumental in, in Microsoft making that pledge. Like, that was, that was quite crazy.
- DPDwarkesh Patel
Oh, that's really interesting. It wasn't that there was, like, some business strategic reason, more so it was just like an idea whose time had come?
- NFNat Friedman
Well, um, GitHub had made such a pledge, and so I think in part in acquiring GitHub, we had to either try to annul that pledge or sign up to it ourselves, and so there was sort of a moment of a forced choice. But, you know, everyone at Microsoft thought, thought it was a good idea too. I, so I think in, in many senses, it was a moment whose time had come and the kind of GitHub acquisition was a forcing function.
- DPDwarkesh Patel
What, what, what do you make of, um, critics of modern open source like, uh, Richard Stallman, or people who advocate for free software saying that, um, well, corporations might advocate for open source because of, like, practical reasons for getting good code. The real value of software, um, and the real way the software should be made, it should be free in that you can replicate it, you can, um, you can change it, you can modify it and you can completely view it, and that the ethical values about that should be more important than the practical values. Like, what do you make of that critique of open source?
- NFNat Friedman
Yeah, I think those are the things that he wants-
- DPDwarkesh Patel
Right.
- NFNat Friedman
... um, and I think the thing that maybe he hasn't updated is that maybe not everyone else wants that. Um, you know, he has this idea that people want freedom from the tyranny of a proprietary intellectual property license but what people really want is freedom from having to configure their graphics card or sound driver or something like that. (laughs)
- DPDwarkesh Patel
(laughs) Yeah.
- NFNat Friedman
You know, they want their computer to kind of work and-
- DPDwarkesh Patel
Right.
- NFNat Friedman
... there are places where freedom is really valuable but there's al- there's always this thing of, like, I have a prescriptive ideology that I'd like to impose on the world versus this thing of, like, I will try to develop the best observational model for what people actually want-
- DPDwarkesh Patel
Mm-hmm.
- 1:11:47 – 1:32:56
Nat.org
- DPDwarkesh Patel
we, we'll get to it in a second, but first I wanna ask about nat.org and the 300 words-
- NFNat Friedman
Oh, yeah.
- DPDwarkesh Patel
... uh, the list there, which is, I, I think, like, one of the most interesting, um, uh, sort of like and I guess very Straussian, uh, list I've seen. Uh, t- list of 300 words I've seen anywhere. But, um, I'm just gonna, like, mention some of these and get some of your commentary. "You should probably work on raising the ceiling, not the floor."
- NFNat Friedman
Yeah.
- DPDwarkesh Patel
Why?
- NFNat Friedman
Yeah, I mean ... Um, well first I, I say probably.
- DPDwarkesh Patel
(laughs)
- NFNat Friedman
But what does it mean to raise the ceiling or the floor? I mean, I, I just observed a lot of projects that set out to raise the floor.
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
Meaning, "Gosh, we are fine, but they are not and we need to go help them-"
- DPDwarkesh Patel
Mm-hmm.
- NFNat Friedman
"... with our superior prosperity and understanding of their situation." And many of those projects fail. So for example, there were a lot of attempts to bring internet to Africa by large and wealthy tech companies and American universities. And I won't say they all had no effect. That's not true. But m- many of them were far short of successful. Like, uh, there were satellites, there were balloons, there were, you know, high altitude drones, um, there were mesh network laptops that were pursued by all these companies. And by the way, by perfectly well-meaning, incredibly talented people who I think did in some cases see some success, but overall probably much less than they ever hoped.... but if you go to Africa, there is internet now. And the way internet got there is the technologies that we developed to raise the ceiling in the richest part of the world, which were cellphones and cell towers. I mean, in the movie Wall Street from the '80s, you know, he's got that gigantic brick cellphone. That thing cost like 10 grand at the time. That was a ceiling-raising technology. It eventually, uh, went down the learning curve and became cheap, and the cell towers and cellphones eventually, you know, we've got now hundreds of millions, uh, or billions of them in Africa, and it was sort of, it was that e- initially ceiling-raising technology and then the sort of force of, of, of capitalism that, that made it work in the end. And it was not any deus ex machina technology solution that it was intended to kind of raise the floor. And so I think there's something about that that's not just an incidental example, um, but I say on, on my website, I say probably, 'cause there are, there are some examples where I think people set out to kind of raise the floor and say, "No one should ever die of smallpox again," right? "No one should ever die of Guinea worm again," and they succeed, and I, I wouldn't want to discourage that from happening, but I think on balance, we have too many attempts to do that that look good, feel good, sound good, and don't matter, and in some cases have the opposite of the effect they intend to.
Episode duration: 1:38:23
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode qcvMjoJdck4
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome