Lex Fridman PodcastChris Lattner: The Future of Computing and Programming Languages | Lex Fridman Podcast #131
EVERY SPOKEN WORD
150 min read · 30,126 words- 0:00 – 2:25
Introduction
- LFLex Fridman
The following is a conversation with Chris Lattner, his second time on the podcast. He's one of the most brilliant engineers in modern computing, having created LLVM Compiler Infrastructure project, the Clang compiler, the Swift programming language, a lot of key contributions to TensorFlow and TPUs as part of Google. He's served as vice president of Autopilot Software at Tesla, was a software innovator and leader at Apple, and now is at SiFive as senior vice president of platform engineering, looking to revolutionize chip design to make it faster, better, and cheaper. Quick mention of each sponsor, followed by some thoughts related to the episode. First sponsor is Blinkist, an app that summarizes key ideas from thousands of books. I use it almost every day to learn new things or to pick which books I want to read or listen to next. Second is Neuro, the maker of functional sugar-free gum and mints that I use to supercharge my mind with caffeine, L-theanine, and B vitamins. Third is MasterClass, online courses from the best people in the world on each of the topics covered, from rockets to game design, to poker, to writing, and to guitar. And finally, Cash App, the app I use to send money to friends for food, drinks, and unfortunately lost bets. Please check out the sponsors in the description to get a discount and to support this podcast. As a side note, let me say that Chris has been an inspiration to me on a human level because he is so damn good as an engineer and leader of engineers, and yet he's able to stay humble, especially humble enough to hear the voices of disagreement and to learn from them. He was supportive of me and this podcast from the early days. And for that, I'm forever grateful. To be honest, most of my life, no one really believed that I would amount to much. So when another human being looks at me and makes me feel like I might be someone special, it can be truly inspiring. That's a lesson for educators. The weird kid in the corner with a dream is someone who might need your love and support in order for that dream to flourish. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter, @lexfridman. And now, here's my conversation with Chris Lattner.
- 2:25 – 7:55
Working with Elon Musk, Steve Jobs, Jeff Dean
- LFLex Fridman
What are the strongest qualities of Steve Jobs, Elon Musk, and the great and powerful Jeff Dean since you've gotten the chance to work with each?
- CLChris Lattner
(laughs) You're starting with an easy question there. Um, these are three very different people. I guess you could do maybe a pairwise comparison between them instead of a group comparison.
- LFLex Fridman
(laughs) Yeah.
- CLChris Lattner
So if you look at Steve Jobs and Elon, um, I worked a lot more with Elon than I did with Steve. Um, they have a lot of commonality. They're both, um, visionary in their own way. They're both very demanding in their own way. Um, my sense is Steve is much more human factor focused, where Elon is more technology focused.
- LFLex Fridman
What does human factor mean?
- CLChris Lattner
Steve's trying to build things that feel good, that people love, that affect people's lives, how they live. He's looking into, into the future a little bit in terms of, um, what people want, um, where I think that Elon focuses more on, uh, learning how exponentials work and predicting the development of those.
- LFLex Fridman
Steve worked with a lot of engineers.
- CLChris Lattner
Yeah.
- LFLex Fridman
That was one of the things, sort of reading the biography, and... How, how can a designer, essentially, talk to engineers and, like, get their respect?
- CLChris Lattner
I think... So I did not work very closely with Steve. I'm not an expert at all. My sense is that he, uh, pushed people really hard, but then when he got an explanation that made sense to him, then he would let go. And, um, he did actually have a lot of respect for engineering and s- but he also knew when to push, and, you know, when you can read people well, you can know when they're holding back and when you can get a little bit more out of them. And I think he was very good at that. I mean, if you, if you compare the other, the other folks, so Jeff Dean, right? Jeff Dean's an amazing guy. He's super smart, um, as, as are the other guys. Um, Jeff is a really, really, really nice guy, well-meaning. He's a classic Googler. He, uh, uh, wants people to be happy. He, uh, combines it with brilliance, so he can pull people together in a, in a really great way. He's definitely not a CEO type. I don't think he would even want to be that. Um-
- LFLex Fridman
Do you know if he still programs? Like is he-
- CLChris Lattner
Oh yeah, he definitely programs. Jeff is an amazing engineer today, right? And that has never changed. So, um, it's really hard to compare Jeff to, to either of those two. Um, he, uh... I think that Jeff leads through technology and building it himself and then pulling people in and inspiring them. And so I think that that's, um, a gr- one of the amazing things about Jeff. But each of these people, you know, with their pros and cons, all are really inspirational and have achieved amazing things. So-
- LFLex Fridman
Yeah.
- CLChris Lattner
... it's, it's been a, it's been... I've been very fortunate to get to work with these guys.
- LFLex Fridman
For yourself, you, you've led large teams. You've done (laughs) so many incredible difficult technical challenges. Is there something you've picked up from them about how to lead?
- CLChris Lattner
Yeah, well, so I mean, I think leadership is really hard. It really depends on what you're looking for there. Um, I think you really need to know what you're talking about. So being grounded on the product, on the technology, on the business, on the mission is really important. Being... Uh, understanding what people are looking for, why they're there. One of the most amazing things about Tesla is the unifying vision, right? People are there because they believe in clean energy and electri- electrification and all these kinds of things. Um, uh, the other is to understand what really motivates people, how to get the best people, how to, how to build a plan that actually can be executed, right? There's so many different aspects of leadership, and it really depends on the time, the place, the problems, you know? You know, there's a lot of issues that don't need to be solved. And so if you focus on the right things and prioritize well, that can really help move things.
- LFLex Fridman
Two interesting things you mentioned. One is you really have to know what you're talking about. How... You've, uh...... you've worked on a lot of very challenging technical things.
- CLChris Lattner
Sure.
- LFLex Fridman
So I kind of assume you were born, uh, technically savvy. But (laughs) -
- CLChris Lattner
(laughs)
- LFLex Fridman
... a- assuming that's not the case, uh, how did y- how did you develop, uh, technical expertise? Like, even at Google, you worked on I don't know how many projects, but really challenging-
- CLChris Lattner
Yeah.
- LFLex Fridman
... very varied t-
- CLChris Lattner
Compilers, TPUs, hardware, cloud stuff, a bunch of different things. Um, the thing that I've become comfortable is I've, w- more comfortable with as I've, uh, gained experience is, uh, being okay with not knowing. And so a major part of leadership is actually, it's not about having the right answer, it's about getting the right answer. And so if you're working on a team of amazing people, (laughs) right? And many of these places, many of these companies all have amazing people, it's the question of how do you get people together? How do you get, how do you build trust? How do you get people to open up? How do people get people to, you know, b- be vulnerable sometimes with an idea that maybe isn't good enough, but it's the start of something beautiful? How do you, um, how do you provide an environment where you're not just like top down, "Thou shalt do the thing that I tell you to do," right? But you're encouraging people to be part of the solution. And, uh, and providing a safe space-
- LFLex Fridman
Sure.
- CLChris Lattner
... where if you're not doing the right thing, they're willing to tell you about it. Right? So-
- LFLex Fridman
So you're okay asking dumb questions?
- CLChris Lattner
Oh, yeah. Dumb questions are my specialty. (laughs) Yeah. Well, I've, so I've been in the hardware room recently, and I don't know much at all about how chips are designed. I know a lot about using them. I know some of the principles and the ars technica level of this, but it turns out-
- LFLex Fridman
(laughs) Ars technica level.
- CLChris Lattner
... it turns out that if you ask a lot of dumb questions, you get smarter real, really quick. And when you're surrounded by people that wanna teach and learn th- themselves, uh, it can be a beautiful thing.
- 7:55 – 13:55
Why do programming languages matter?
- CLChris Lattner
- LFLex Fridman
Uh, so let's talk about programming languages, if it's okay.
- CLChris Lattner
Sure, sure.
- LFLex Fridman
At the highest absurd philosophical level 'cause I, I-
- CLChris Lattner
Don't get romantic on me, Lex.
- LFLex Fridman
(laughs) I won- I will forever get romantic (laughs) and, uh, torture you. I apologize. Uh, w- why do programming languages even matter?
- CLChris Lattner
Okay. (laughs) Well, thank you very much. So you're saying why should, why, why should you care about any one programming language, or why do you, why do we care about programming computers, or-
- LFLex Fridman
No. Why, why do we c- (laughs) why do we care about programming language design, creating-
- CLChris Lattner
Yeah. Okay.
- LFLex Fridman
... effective programming languages? Uh, m- choosing a, you know, one programming languages versus another programming language, why we keep struggling and improving through the evolution of these programming languages.
- CLChris Lattner
Sure, sure, sure. Okay. So, so I mean, I think you have to come back to what, what are we trying to do here, right? So we have these, these, uh, beasts called computers that are very good at specific kinds of things, and we think it's useful to have them do it for us, right? Uh, now you have this question of how best to express that because you have a human brain still that has an idea in its head, and you wanna achieve something, right? So, well, there's lots of ways of doing this. You can go directly to the machine and speak assembly language, and then you can express directly what the computer understands. That's fine. Um, you can then have higher and higher and higher levels of abstraction up until machine learning, and you're designing a neural net to do the work for you. Um, the question is where, where along this way do you want to stop, and what benefits do you get out of doing so? And so programming languages in general, you have C, you have Fortran and Java and ADA, Pascal, Swift. You have lots of different things. Um, they all have different trade-offs, and they're ta- tackling different parts of the problems. Now, one of the things that most programming languages do is they're trying to make it so that you have pretty basic things like portability across different hardware. So you've got, "I'm gonna run it on an Intel PC. I'm gonna run it on a RISC-V PC. I'm gonna run it on a, uh, ARM phone or something like that." Fine. Um, I wanna write one program and have it portable, and this is something that assembly doesn't do. Now, when you start looking at the space of programming languages, this is where I think it's fun because programming languages all have trade-offs, and most people will walk up to them, and they look at the surface level of syntax and say, "Oh, I like curly braces," or, "I like tabs," or, "I like, you know, semicolons or not," or whatever, right? It's subjective, uh, uh, fairly subjective, very shallow things. But programming languages, when done right, can actually be very powerful. And the, the benefit they bring is expression. (laughs) Okay? And if you look at programming languages, there's really kinda two different levels to them. One is the down in the dirt nuts and bolts of how do you get the computer to be efficient, stuff like that, how they work, type systems, compiler stuff, things like that. The other is the UI, and the UI for a programming language is really a design problem, and a lot of people don't think about it that way.
- LFLex Fridman
And the UI, you mean all that stuff with the braces and-
- CLChris Lattner
Yeah.
- LFLex Fridman
... the act-
- CLChris Lattner
All that stuff's the UI, and what it is, and UI means user interface. Um, and so what, what's really going on is it's the interface between the guts and the human. And humans are hard, right? Humans have feelings. They have things they like. They have things they don't like. And a lot of people treat programming languages as though humans are just kind of abstract creatures that cannot be predicted. But it turns out that actually there are, there is better and worse. (laughs) Like, people can tell-
- LFLex Fridman
Yeah.
- CLChris Lattner
... when a programming language is good or when it was an accident, right? And, uh, one of the things with Swift in particular is that a tremendous amount of time by a trem- endous number of people have been put into really polishing and making it feel good. But it also has really good nuts and bolts underneath it.
- LFLex Fridman
You, you said that, uh, Swift makes a lot of people feel good. How do you get to that point? So how do you predict that, um, you know, tens of thousands, hundreds of thousands of people are going to enjoy using this, the user experience of this programming-
- CLChris Lattner
Yeah.
- LFLex Fridman
... language?
- CLChris Lattner
Well, you can, you can look at it in terms of better and worse, right? So if you have to write l- lots of boilerplate or something like that, you will feel unproductive. And so that's a bad thing. You can look at it in terms of safety. If like C, for example, is what's called a memory unsafe language, and so you get dangling pointers, and you get all these kind of bugs that then you have to spend tons of time debugging, and it's a real pain in the butt-
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
... and you feel unproductive. And so by subtracting these things from the experience, you get, um, you know, happier people.
- LFLex Fridman
But, uh, uh, again, keep interrupting. I'm sorry. Uh, but-
- CLChris Lattner
You're so hard to deal with. (laughs)
- LFLex Fridman
(laughs) If you look at the people, people that are most productive on Stack Overflow-
- CLChris Lattner
Mm-hmm.
- LFLex Fridman
... they are, uh, they have a set of priorities-
- CLChris Lattner
Yeah.
- LFLex Fridman
... that may not always correlate perfectly with the experience of the majority of users. You know, like-
- 13:55 – 24:48
Python vs Swift
- CLChris Lattner
okay?
- LFLex Fridman
Yeah.
- CLChris Lattner
And so you look at Python, for example. Python is really good at assembling things, but it's not so great at building all the libraries. And so what you get, because of performance reasons, other things like this, is you get Python layered on top of C.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
For example. And that means that doing certain kinds of things, well, it doesn't really make sense to do in Python. Instead, you do it in C, and then you wrap it, and then you have- you're living in two worlds. And two worlds never is really great because tooling and the deb- debugger doesn't work right, and, like, all these kinds of things.
- LFLex Fridman
Can you clarify a little bit wha- uh, what you mean by Python is not good at building libraries? Meaning, it doesn't make a conducive-
- CLChris Lattner
Certain kinds of, certain kinds of libraries.
- LFLex Fridman
No, but just the con- the actual meaning of the sentence.
- CLChris Lattner
Yeah.
- LFLex Fridman
Uh, meaning, like, it's not conducive to developers to come in and add libraries? Or, it's- it's- or the langu- or is it the- the duality of the- it's a dance between Python and C and you can-
- CLChris Lattner
Well, so, Python's amazing. Python's a great language. I did not mean to say that Python is-
- LFLex Fridman
No offense (laughs) .
- NANarrator
(laughs)
- CLChris Lattner
... is bad for libraries. What- what I meant to say is, um, there are Python- there are libraries that Python's really good at- that you can write in Python, but there are other things, like if you wanna build a machine learning framework, you're not gonna build a machine learning framework in Python because of performance, for example.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
Or you want GPU acceleration or things like this. Instead, what you do is you write a bunch of C or C++ code, or something like that, and then you talk to it from Python, right? And so this is because of decisions that were made in the Python design. And, um, and those decisions have other counterbalancing forces. But- but the trick, when you start looking at this from a programming language perspective, is you sort of say, "Okay, cool. How do I build this catalog of libraries that are really powerful? And how do I make it so that then they can be assembled into ways that feel good and they generally work the first time?" Because when you're talking about building a thing, you have to include the debugging, the fixing, the turnaround cycle, the development cycle, all that kinda stuff in- in- into the process of building the thing. It's not just about pounding out the code. And so this is where things like, um, you know, catching bugs at compile time is valuable, for example. Um, but if you dive into the details in this, Swift, for example, has certain things like value semantics, which is this fancy way of saying that when you, uh, treat a- treat a variable like a value, um, uh, it acts like a mathematical object would. Okay? So, and y- y- you have used PyTorch a little bit?
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
In PyTorch, you have tensors. Tensors are, uh, n-d- n-dimensional grid of numbers. Very simple. You can do plus and other operators on them. It's all totally fine. But why do you need to clone a tensor sometimes?
- LFLex Fridman
(laughs)
- CLChris Lattner
Have you ever run into that?
- LFLex Fridman
Uh, yeah.
- CLChris Lattner
Okay. And so why is that? Why do you need to clone a tensor?
- LFLex Fridman
It's the usual object thing that's in Python.
- CLChris Lattner
Yeah. So, in Python, and just like with Java and many other languages, this isn't unique to Python. In Python, it has a thing called reference semantics, which is the nerdy way of explaining this. And what that means is you actually have a pointer to a thing-
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
... instead of the thing. Okay? Now, this is due to a bunch of implementation d- details that you don't wanna go into. But in Swift, you have this thing called value semantics. And so when you have a tensor in Swift, it is a value. If you copy it, it looks like you have a unique copy. And if you go change one of those copies, then, uh, it doesn't update the other one 'cause you just made a copy of this thing. Right? So-
- LFLex Fridman
So that- that's, like, highly error-prone in, uh, at least computer science, math-centric disciplines, uh, about Python.
- CLChris Lattner
Yep.
- LFLex Fridman
That, like, the- the thing you would expect to behave-
- CLChris Lattner
Like math.
- 24:48 – 30:06
Design decisions
- LFLex Fridman
And, uh, how many people are involved with good design. So, if we look at Swift-
- CLChris Lattner
Yeah.
- LFLex Fridman
... but look at historically, I mean, (laughs) this might touch, like, uh, it's almost like a Steve Jobs question too. Like, how much dictatorial decision-making is required (laughs) versus, um, collaborative, and we'll talk about how all that can go wrong or right.
- CLChris Lattner
Yeah.
- LFLex Fridman
But-
- CLChris Lattner
Yeah. Well, well, Swift, uh, so I can't speak to in general all design everywhere. Uh, so the way it works with Swift is that, um, there's a core team.
- LFLex Fridman
Yeah.
- CLChris Lattner
And so a core team is, uh, six or seven people-ish, something like that, that is people that have been working with Swift since the very early days. And so I-
- LFLex Fridman
And by early days, it's not that long ago.
- CLChris Lattner
Okay. Yeah. So it's-
- LFLex Fridman
(laughs)
- CLChris Lattner
It, it became public in 2014, so it's been six years public now. But, um, but still that's enough time that there's a story arc there. (laughs)
- LFLex Fridman
Okay. Yeah.
- CLChris Lattner
Right? And there's mistakes have been made that then get fixed and you learn something and then you, you know. And so, uh, the, what the core team does is it provides continuity. And so you wanna have a, "Okay, well, um, there's a big hole that we wanna fill. We know we wanna fill it, so don't do other things that invade that space until we fill the hole," right? There, there's a boulder that's missing here, we want to, do... We will do that boulder even though it's not today, keep, keep out of that space.
- LFLex Fridman
And the whole team remembers of the... remembers the myth of the boulder that's there.
- CLChris Lattner
Yeah. You know, there's a general sense of what the future looks like in broad strokes and a shared understanding of that, combined with the shared understanding of what has happened in the past that worked out well and didn't work out well. Um, the next level out is you have the, uh, what's called the Swift Evolution Community, and you've got, in that case, hundreds of people that really care passionately about the way Swift evolves. And that's, like, an amazing thing to, again, uh, the core team doesn't necessarily need to come up with all the good ideas. You got hundreds of people out there that care about something and they come up with really good ideas too. And that provides this, like, tumbling, rock tumbler for ideas.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
And so the, the evolution process is, you know, a lot of people in a discourse forum, they're like hashing it out and trying to, like, talk about, "Okay, well, what... should we go left or right? Or if we did this, what would be good?" And, you know, here you're talking about hundreds of people, so you're not gonna get consensus necessarily or not obvious consensus. And so there's a proposal process that, uh, then allows the core team and the community to work this out. And what the core team does is it aims to get consensus out of the community and provide, uh, gua- guardrails, but also provide long term... make sure we're going the right direction kind of things.
- LFLex Fridman
So does that group represent like the, how much people will love the user interface?
- CLChris Lattner
Yeah.
- LFLex Fridman
Like do you think they're able to capture that?
- CLChris Lattner
Well, I mean, it's something we talk about a lot, something we care about. How well we, how well we do that is up for debate, but I think that we've done pretty well so far. Yeah.
- LFLex Fridman
Is, is the beginner in mind?
- CLChris Lattner
Yeah. Yeah.
- LFLex Fridman
Like, because you said the progressive disclosure ........................
- CLChris Lattner
Yeah. So we care a lot about, uh, a lot about that, a lot about power, a lot about efficiency, a lot about... There are many factors to good design and you have to figure out a way to kind of work your way through that and-
- LFLex Fridman
So if you, like, think about, like, the language I love is Lisp.
- CLChris Lattner
Okay. Yeah.
- LFLex Fridman
Probably still because I use Emacs, but I haven't done anything, any serious work in Lisp, but it has a ridiculous amount of parentheses.
- CLChris Lattner
Yeah.
- 30:06 – 33:54
Types
- CLChris Lattner
- LFLex Fridman
What about like the decision of whether, like in Python, having one type or having, you know, uh, strict typing.
- CLChris Lattner
Yeah. Okay.
- LFLex Fridman
Many types.
- CLChris Lattner
Yeah. Let's talk about this. So, so, um, I, I like how you put that, by the way, like so, so many people would say that Python doesn't have types.
- LFLex Fridman
Doesn't have types. Yeah.
- CLChris Lattner
But you're right.
- LFLex Fridman
Well, I've listened to you enough to where... (laughs)
- CLChris Lattner
Yeah. (laughs)
- LFLex Fridman
I'm, I'm a fan of yours and I've listened to way too many podcasts and videos-
- CLChris Lattner
There you go. (laughs)
- LFLex Fridman
... of you talking about this.
- CLChris Lattner
Oh, yeah. So I would argue that Python has one type. And so, um, so like when you import Python into Swift, which by the way works really well, you have everything comes in as a Python object.
- LFLex Fridman
Yeah.
- CLChris Lattner
Now, here there are trade-offs because, um, uh, you know, it depends on where you're optimizing for. And Python is a super successful language for a really good reason, um, because it has one type. Uh, you get duck typing for free and things like this, but also you're pushing... you're making it very easy to, to pound out code on one hand, but you're also making it very easy to introduce, uh-... complicated bugs that you have to debug, and you pass the string into something that expects an integer, and it doesn't immediately die, it goes all the way down the stack trace and you find yourself in the middle of some code that you really didn't wanna know anything about, and it blows up, and you're just saying, "Well, what did I do wrong?" Right? And so types are good and bad and they have trade-offs, they're good for performance and certain other things depending on where you're coming from, but it's, it's all about trade-offs. And so this is, this is what design is, right? Design is about weighing trade-offs and trying to understand the ramifications of the, the things that you're weighing, like types or not, or one type or many types. Um, but also within many types, how powerful do you make that type system is another very complicated question, uh, with lots of trade-offs. It's very interesting, by the way. Uh, but, uh, but that's like one, one dimension.
- LFLex Fridman
Yeah.
- CLChris Lattner
And there's a bunch of other dimensions. JIT-compiled versus static compiled, garbage collected versus reference counted, versus memory mana- manual memory management versus, you know, like in, like all these different trade-offs and how you balance them are what make a programming language good.
- LFLex Fridman
Concurrency.
- CLChris Lattner
Yeah.
- LFLex Fridman
So and all those things, I guess, uh, when you're designing a language, you also have to think of how that's gonna get all compiled down to, uh-
- CLChris Lattner
If you ca- care about performance, yeah. Well, and, and go back to Lisp, right? So Lisp, also I would say JavaScript are, is another example of a very simple language, right? And so one of the, uh ... So I also love Lisp. I don't use it-
- LFLex Fridman
Yeah.
- CLChris Lattner
... as much as maybe you do-
- LFLex Fridman
Yeah.
- CLChris Lattner
... or you did. But-
- LFLex Fridman
No, I think we're both everyone who loves Lisp, it's like you, you love ... It's like, uh, I don't know, you know, I love Frank Sinatra, but m- like how often do I seriously listen to Frank Sinatra?
- CLChris Lattner
Sure. Sure.
- LFLex Fridman
(laughs) So.
- CLChris Lattner
But, but, but you look at that or you look at JavaScript, which is another very different but relatively simple language, and there's certain things that don't exist in the language. But there's, there is inherent complexity to the problems that we're trying to model. And so what happens to the complexity? In the case of, uh, both of them, for example, you say, "Well, what about large scale software development?" Okay. Well, you need something like packages. Neither language has a, like language affordance for packages, and so what you get is patterns. (laughs)
- LFLex Fridman
Mm-hmm.
- 33:54 – 36:26
Programming languages are a bicycle for the mind
- CLChris Lattner
Well, so, so the, the, the, the key thing to think about with, uh, with programming languages, and you, you think about what a programming language is there for, is it's about making a human more productive, right? And so like there's an old, I think it's a Steve Jo- Jobs quote about, um, it's a bicycle for the mind, right? You can, you can, you can definitely walk, (laughs) but you'll get there a lot faster if you can bicycle on your way. And-
- LFLex Fridman
A, a programming language is a bicycle for the mind?
- CLChris Lattner
Yeah.
- LFLex Fridman
It's, it's basically a ... Wow, that's really an interesting way to think about it.
- CLChris Lattner
By, by raising the level of obstruction now you can fit more things in your head. By being able to just directly leverage somebody's library, you can now get something done quickly. Um, in the case of Swift, SwiftUI is this new framework that Apple has released recently for doing UI programming, and it has this declarative programming model which defines away entire classes of bugs, it's ma- it builds a value somatics and many other nice Swift things. And what this does is allows you to just get way more done with le- way less code, and now your productivity as a developer is much higher, right? And so that, that's really the, what programming languages should be about, is it's not about tabs versus spaces or curly braces or whatever, it's about how productive do you make the person. And you can only see that when you have libraries that were built with the right intention that the language was designed for, and with Swift I think we're still a little bit early, um, but SwiftUI and many other things that are coming out now are really showing that, and I think that they're opening people's eyes.
- LFLex Fridman
It's kinda interesting to think about like how that, you know, the knowledge of something, of how good the bicycle is, how people learn about that, you know. So I've used C++. Now this is not going to be a trash talking session about C++, but I used C++ for a really long-
- CLChris Lattner
I can go there if you want.
- LFLex Fridman
(laughs)
- CLChris Lattner
I have the scars. (laughs)
- LFLex Fridman
I, I feel like I spent many years without realizing like there's languages that could, for my particular lifestyle, brain style, thinking style, there, there's languages that could make me a lot more productive, uh, in the debugging stage, in the, just the development stage, in thinking, like the bicycle for the mind, that I could fit more stuff into my-
- CLChris Lattner
Python's a great example of that, right?
- LFLex Fridman
Yeah.
- CLChris Lattner
I mean, a machine learning framework in Python is a great example of that. It's just s- very high abstraction level, and so you can be thinking about things on a, like very high-level alg- algorithmic level instead of thinking about, "Okay, well, am I copying this tensor to a GPU or not?" (laughs) Right? It's not, it's not what you wanna be thinking about.
- LFLex Fridman
And as I was
- 36:26 – 42:25
Picking what language to learn
- LFLex Fridman
telling you, I mean, I g- I guess the question I had is, uh, you know, how does a person like me or in general people discover more productive, uh, you know, languages? Like how ... I was, as I've been, uh, telling you offline, I've been looking for like a project to work on in Swift so I can really, uh, try it out as-
- CLChris Lattner
Yeah.
- LFLex Fridman
I mean, my intuition was like doing a "Hello World" is not going to get me there, uh, to-
- CLChris Lattner
Right, right.
- LFLex Fridman
... to, to, to get me to experience the power of the language. So I was-
- CLChris Lattner
Yeah. You need, you need a few weeks to change your metabolism.
- LFLex Fridman
Exactly. I ... beautifully put. Uh, that, that's one of the problems with people with diets. Like I, I'm, I'm actually currently ... To go in parallel but in a small tangent is I've been recently eating only meat.... okay?
- CLChris Lattner
Okay.
- LFLex Fridman
(laughs) Okay. So most people are like, th- uh, the thing that's horribly unhealthy or whatever, you have like a million ... it, the, whatever the science is, it just doesn't sound right.
- CLChris Lattner
Well, so, so back when I was in college, we did the Atkins diet. That was, that was the thing.
- LFLex Fridman
Yeah, yeah. Similar.
- CLChris Lattner
Yeah.
- LFLex Fridman
And but if you ... you have to always give these things a chance. I, I mean, with dieting, always ... not dieting, but it's just the things that you like. If I eat, personally, if I eat meat, just everything, I'm, I can super fo- well, more focused than usual. (laughs) I, I just feel great. I got, I've been, I've been running a lot, you know, doing pushups and pull-ups and so on. I mean, Python is similar in that sense for me.
- CLChris Lattner
Where are you going with this? (laughs)
- LFLex Fridman
(laughs) I mean, literally, I just, I felt, I had like a stupid smile on my face when I first started-
- CLChris Lattner
Yeah. Yeah.
- LFLex Fridman
... using Python. I could, uh, code up really quick things. Like I-
- CLChris Lattner
Yeah.
- LFLex Fridman
... like I, I would see the world, I'll be empowered to write a script to, to, um, you know, to do some basic data processing.
- CLChris Lattner
Yeah.
- LFLex Fridman
To rename files on my computer.
- CLChris Lattner
That's great. Yeah.
- LFLex Fridman
Right? And like, Pearl didn't do that for me. Uh ...
- CLChris Lattner
Yeah.
- LFLex Fridman
... uh, I mean, kind ... a little bit. It is. But-
- CLChris Lattner
Well, and, and, and again, like n- none of these are about which, which is best or something like that, but there, there's definitely better and worse here, right?
- LFLex Fridman
But it clicks, right? Well, yeah. It, it-
- CLChris Lattner
And if you, if you look at Pearl, for example, you get bogged down in, uh, scalers versus arrays versus hashes versus type globs and like all that kind of stuff. And, and Python's like, "Yeah, wait, let's not do this." Right?
- LFLex Fridman
And some of it is debugging, like everyone has different priorities, but for me, it's connect, create systems for myself that empower me to debug quickly.
- CLChris Lattner
Mm-hmm.
- 42:25 – 51:50
Most beautiful feature of a programming language
- LFLex Fridman
uh ... there you go. This, this is a question, uh, to ask, what is the most beautiful feature in a programming language? Before I ask it, let me say like, with Python, I remember I saw list comprehensions.
- CLChris Lattner
Yeah.
- LFLex Fridman
Uh, was like ... uh, when I like really took it in-
- CLChris Lattner
Yeah.
- LFLex Fridman
... it, I don't know, I just loved it. It was like fun to do. Like, it was fun to do that kind of, um, uh ... yeah. There was something about it to be able to filter through a list and to create a new list all in a single line, was elegant, and I could all get into my head.
- CLChris Lattner
Yeah.
- LFLex Fridman
And it just made me, um, fall in love with the language.
- CLChris Lattner
Yup.
- LFLex Fridman
So is there ... let me ask you a question. Uh, is there ... what do you use the most beautiful feature in a, in a programming languages that you've ever encountered? In Swift maybe, and then outside of Swift.
- CLChris Lattner
I think the thing that I like the most from a programming language... So, so I think the thing you have to think about with the programming language, again, what is the goal? You're trying to get people to get things done quickly. And so you need libraries, you need high-quality libraries, and then you need a user base around them that can assemble them and do cool things with them, right? And so to me, the question is, what enables high-quality libraries? Okay.
- LFLex Fridman
Yeah.
- CLChris Lattner
And there's a huge divide in the world (laughs) between libraries who enable high-quality libraries versus, um, the ones that put special stuff in the language.
- LFLex Fridman
So programming languages that enable-
- CLChris Lattner
High-quality libraries.
- LFLex Fridman
... high-quality libraries? Got it. Uh-
- CLChris Lattner
So, so and, and what I mean by that is expressive libraries that then feel like a natural integrated part of the language itself.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
So, um, a- an example of this in Swift is that int and float and also array and string, things like this, these are all part of the library. Like int is not hardcoded into Swift. And so what that means is that because int is just a library thing defined in the standard library, along with strings and arrays and all the other things that come with the standard library, um... Well, hopefully you do like int. (laughs) But anything that, any language features that you needed to define int, you can also use in your own types. So if you wanted to define a, uh, a quaternion or something-
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
... like this, right, um, well, it doesn't come in the standard library. Um, there's a various special set of people that care a lot about this. But those people are also important. It's not, it's not about classism, right? It's not about the people who care about ints and floats are more important than the people who care about quaternions. And so to me, the beautiful things about programming languages is when you allow those communities to, to build high-quality libraries that feel native, that feel like they're built into the, built into the compiler without having to be.
- LFLex Fridman
What does it mean t- for the int to be part of a, not hardcoded in?
- CLChris Lattner
Yeah.
- LFLex Fridman
So is it like... How, so what is an, what is an int?
- CLChris Lattner
Okay. (laughs) Int is just a integer. In this case, it's like a, you know, like a 64-bit integer or something like this. But-
- LFLex Fridman
So like the 64-bit is hardcoded, or no?
- CLChris Lattner
No, none of that's hardcoded. So int, int if you go look at how it's implemented is it's just a struct in Swift.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
And so it's a struct, and then how do you add two structs? Well, you define plus, and so you can define plus on int. Well, you can define plus on your thing too. You can define... Uh, int has like an is_odd method or something like that on it. And so yeah, you can add methods onto things.
- LFLex Fridman
Is_odd? (laughs)
- CLChris Lattner
Yeah. So-
- 51:50 – 1:01:16
Walrus operator
- CLChris Lattner
- LFLex Fridman
Well, let me ask you about, uh, the most, uh, sort of this, this, uh, this, this podcast isn't about information, it's about drama. So-
- CLChris Lattner
(laughs) Okay.
- LFLex Fridman
... let me, let me talk to you about some drama. So you mentioned Pascal and, uh, colon equals. Uh, there's something ca- that's called the walrus operator.
- CLChris Lattner
Okay.
- LFLex Fridman
And, uh, Python, uh, in Python 3.8 added the walrus operator. And the reason I think it's interesting, uh, is not just 'cause of the feature. It does, it's, it has the same kind of expression feature you can mention in C that it returns the value of the assignment. And maybe you can comment on that in general. But on the other side of it, it's also (laughs) the thing that f- that, uh, uh, toppled the dictator. Uh, so... (laughs)
- CLChris Lattner
(laughs) Okay.
- LFLex Fridman
It finally drove Guido to, uh, step down from BDFL-
- CLChris Lattner
Yeah.
- LFLex Fridman
... the toxicity of the community. So maybe, um, what do you think about the walrus operator in, in Python? Is there an equivalent thing in Swift that really te- uh, stress-tested the community? And, uh, and then on the flip side, what do you think about Guido stepping down over it?
- CLChris Lattner
Yeah, if you, well, if he, like if I look past the details of the wal- walrus operator, one of the things that makes it most polarizing is that it's syntactic sugar.
- LFLex Fridman
Okay. What do you mean by syntactic sugar?
- CLChris Lattner
It means you can take something that already exists in the language and you can express it in a more concise way.
- LFLex Fridman
So, okay, I'm gonna play devil's advocate. So, uh-
- CLChris Lattner
Yeah.
- LFLex Fridman
(laughs) This is great. Uh, is that a objective or subjective statement? Like, uh-
- CLChris Lattner
Uh-
- LFLex Fridman
... can you, can you argue that basically anything is syntactic sugar or no?
- CLChris Lattner
Uh, y- no. You, not everything is, is syntactic sugar. So for example, um, the type system, like can you have classes versus, uh, versus, uh, like h- do you have types or not? Right? So s- so one type versus many types is not something that affects syntactic sugar. And so if you say, "I want to have the ability to define types," now I have to have all this like language mechanics to define classes and, oh, now I have to have inheritance, now I have to like, like I have all this stuff. That's just making the language more complicated.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
That's not, that's not about sugaring it. Um, Swift has sugar. (laughs) So like Swift has this thing called if let, and it has, uh, various operators that are used to concisify, uh, specific use cases. So the problem with syntactic sugar, when you're talking about, "Hey, I have a thing that takes a lot to write and I have a new way to write it," you have this h- like horrible trade-off, which becomes almost completely subjective, which is how often does this happen and does it matter? And one of the things that is true about human psychology, particularly when you're talking about introducing a new thing, is that, uh, people over- overestimate the burden of learning something. And so it looks foreign when you haven't gotten used to it. But if it was there from the beginning, of course, it's just part of Python. Like, unquestionably, like this is, this is just a thing I know. And it's not a new thing that you're worried about learning, it's just part of, part of the deal. Now with Guido, uh, I, I don't know Guido well. (laughs) Um-
- LFLex Fridman
Yeah. Have you passed across much?
- CLChris Lattner
Yeah, I've met him a couple of times, but I, I don't kn- know Guido well. But the, the sense that I got out of that whole dynamic was that he had put the, not just the decision-maker weight on his shoulders, but it was so tied to his personal identity that, um, he took it personally and he felt the need and he kind of put himself in the situation of being the person instead of building a base of support around him. I mean, he, this is probably not quite literally true, but by-
- LFLex Fridman
There's too much, so there, there was too much p-
- CLChris Lattner
Too much concentrated on him. Right? And so...
- LFLex Fridman
And that can wear you down.
- CLChris Lattner
Well, yeah. Particularly because people then say, "Guido, you're a horrible person. I hate this thing. Blah, blah, blah, blah, blah, blah, blah." And sure, it's like, you know, maybe 1% of the community that's doing that. But-... Python's got a big community.
- LFLex Fridman
Yeah.
- CLChris Lattner
And 1% of it, of millions of people is a lot of hate mail. And that, just from a human factor, will just wear on you.
- LFLex Fridman
Well, to, to clarify, it looked from just what I saw in the messaging, for the, let's not look at the million Python users, but at the Python core developers. It feels like the majority, the big majority, on a vote, were opposed to it. So-
- 1:01:16 – 1:06:28
LLVM
- CLChris Lattner
- LFLex Fridman
Are you still BDFL? You've been BDFL of some stuff. Uh-
- CLChris Lattner
Yeah, so-
- LFLex Fridman
You're, you're very heavy on the B. (laughs) Uh, the benevolent, uh, benevolent dictator for life. Uh, I guess LLVM?
- CLChris Lattner
Yeah, so-
- LFLex Fridman
You're still in the-
- CLChris Lattner
So I still l- lead the LLVM world.
- LFLex Fridman
Uh, I mean what's the role of, uh, uh, so then on Swift you said that there's a group of people.
- CLChris Lattner
Yeah. So if you contrast Python with Swift, right, one of the reasons w-... so everybody on the core team takes his role really seriously, and I think we all really care about where Swift goes. But you're almost delegating the final decision-making to the wisdom of the group, and so it doesn't become personal. And also, when you're talking with the community, so yeah, some people are very annoyed as certain decisions get made. Um, there's a certain faith in the process because it's a very transparent process. And when a decision gets made, a full rationale is provided, things like this. These are almost defense mechanisms to help both guide future discussions and provide case law, kind of like the Supreme Court does, about this decision was made for this reason, and here's the rationale and what we wanna see more of or less of. Um, but it's also a way to provide a defense mechanism so that when somebody's griping about it, they're not saying, "That person did the wrong thing."
- LFLex Fridman
Right.
- CLChris Lattner
They're saying, "Well, this- this thing sucks and grr," and so later- later they move on and they- they get over it.
- LFLex Fridman
Yeah, the analogy of the Supreme Court I think is really- is really good, but then, okay, not to get personal on the Swift team-
- CLChris Lattner
Yeah.
- LFLex Fridman
... but, like is there- is there a divi- like it just seems like it's impossible for there- for division not to emerge.
- CLChris Lattner
Well each- each of the humans on the- the Swift core team, for example, are different, and the membership of the Swift core team changes slowly over time, which is, I think, a- a healthy thing. And so each of these different humans have different opinions. Trust me, it's not- it's not a sing- singular consciousness of, by any stretch of the imagination. You've got three major organizations, including Apple, Google and Sci-Fi have all-
- LFLex Fridman
Oh, wow. Cool.
- CLChris Lattner
... kinda working together and, um, and it's a small group of people, but you need high trust. You need, again, it comes back to the principles of what you're trying to achieve, and understanding, you know, what- what you're optimizing for. And I think that starting with strong principles and working towards decisions is always a good way to both make wise decisions in general, but then be able to communicate them to people so that they can buy into them. And that- that is hard. And so y- you mentioned LLVM. LLVM is, uh, gonna be 20 years old, uh, this December, so it's- it's showing its own age.
- LFLex Fridman
Do you have like- like a- like a- like a dragon cake planned, or do you have, uh-
- CLChris Lattner
Oh, we should definitely do that. Yeah, if we can have a- a pandemic cake.
- LFLex Fridman
(laughs) Pandemic cake.
- CLChris Lattner
(laughs) Everybody gets a slice of cake-
- LFLex Fridman
Sure.
- CLChris Lattner
... and it gets, you know, sent through email. Um, (laughs) but the, uh, uh, but LLVM has had tons of its own challenges over time too, right? And one of the challenges that, um, the LLVM community has, in my opinion, is that it has a whole bunch of people that, um, have been working on LLVM for 10 years, right? 'Cause this happens s- somehow.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
And LLVM's always been one way, but it needs to be a different way, right? And they've worked on it for, like 10 years is a long time to work on something. And, you know, you- you suddenly can't see the faults in the thing that you're working on. And LLVM has lots of problems and we need to address them and we need to make it better, and if we don't make it better, then somebody else will come up with a better idea. Right? And so it's just kind of of that age where the community i- is, like in danger of getting too calcified, and, um, and so I'm happy to see new projects joining and new things mixing it up. You know, Fortran is now a new- a new thing in the LLVM community-
- LFLex Fridman
Oh, yeah?
- CLChris Lattner
... which is hilarious and good.
- LFLex Fridman
I've been trying to find, uh, on this little tangent, find people who program in Cobalt or Fortran, F- Fortran especially, to- to talk to. They're hard to find. (laughs)
- CLChris Lattner
Yeah.
- LFLex Fridman
(laughs)
- 1:06:28 – 1:10:35
MLIR compiler framework
- CLChris Lattner
of this is we- we built this new compiler framework called MLIR.
- LFLex Fridman
Yes.
- CLChris Lattner
MLIR is this, a whole new framework. It's not, many people think it's about machine learning. The ML stands for multi-level, because compiler people can't name things very well, I guess.
- LFLex Fridman
Can we- could we dig into what MLIR is?
- CLChris Lattner
Yeah, so when you look at compilers, compilers have historically been solutions for a given space. So LLVM is a, it's really good for dealing with CPUs, let's just say, at- at a high level. You look at, um, Java. Java has a JVM. The JVM is very good for garbage collected languages that need dynamic compilation, and it's very optimized for a specific space. And so HotSpot is one of the compilers that gets used in that space, and that compiler's really good at that kind of stuff. Um, usually when you build these domain-specific compilers, you end up building the whole thing from scratch for each domain.
- LFLex Fridman
W- uh, what's a domain? So w- what- what- what's the ver- uh, what's the scope of a domain?
- CLChris Lattner
Well, so here I would say, like if you look at Swift, there's several different parts to the Swift compiler, um, one of which is covered by, um, the LLVM part of it. There's also a high level piece that's specific to Swift, and there's a huge amount of redundancy between those two different infrastructures, and a lot of re- reimplemented stuff that is similar but different.
- LFLex Fridman
What does LLVM define?
- CLChris Lattner
LVM is effectively an infrastructure, so you can mix and match it in different ways. It's built out of libraries. You can use it for different things. But it's really good at CPUs and GPUs. CPUs and like the tip of the iceberg on GPUs. It's not really great at GPUs. Okay. Um, but it turns out-
- LFLex Fridman
There's a bunch of languages that, uh-
- CLChris Lattner
That then use it to talk to CPUs.
- LFLex Fridman
Got it.
- CLChris Lattner
Um, and so it turns out there's a lot of hardware out there that is custom accelerators. So machine learning, for example. There are a lot of, uh, matrix multiply accelerators and things like this. There, there's a whole world of hardware synthesis. So we- we're using MLIR to build circuits. (laughs) Okay. And so you're compiling for a domain of transistors. And so what MLIR does is it provides a tr- tremendous amount of compiler infrastructure that allows you to build these domain-specific compilers in a much faster way and have the result be good.
- LFLex Fridman
If we're, if we're thinking about the future... Now we're talking about like ASICs, like so anything?
- CLChris Lattner
Yeah, yeah.
- LFLex Fridman
So if we project into the future, it's very possible that the number of these kinds of ASICs, very s- specific, um, infrastructure thing, uh, architecture things, uh, like multiplies exponentially.
- CLChris Lattner
I hope so. Yeah.
- LFLex Fridman
So that's MLIR.
- CLChris Lattner
So what MLIR, what MLIR does is it allows you to build these compilers very efficiently, right? Now, one of the things that coming back to the LLVM thing, and then we'll go to hardware, is, um, LLVM is a, is a specific compiler for a specific domain. MLIR is now this very general, very flexible thing that can solve lots of different kinds of problems. So M- LLVM is a subset of what MLIR does.
- LFLex Fridman
So MLIR is, I mean, it's an ambitious project then.
- CLChris Lattner
Yeah, it's a very ambitious project. Yeah. And so to make it even more confusing, MLIR has joined the LLVM umbrella project. So it's part of the LLVM family.
- LFLex Fridman
Right.
- CLChris Lattner
Um, but where this comes full circle is now folks that work on the LLVM part, the classic part that's 20 years old, um, aren't aware of all the cool new things that have been done in the new th- the new thing that, you know, M- MLIR was built by me and many other people that knew a lot about LLVM, and so we fixed a lot of the mistakes that lived in LLVM.
- LFLex Fridman
I mean, what, what-
- CLChris Lattner
So now you have this community dynamic where it's like, well, there's this new thing, but it's not familiar. Nobody knows it. It feels like it's new. And so let's not trust it. And so it's just really interesting to see the cultural, social dynamic that comes out of that. And, and, you know, I think it's super healthy because we're seeing the ideas percolate and we're seeing the technology diffusion happen. As people get more comfortable with it, they start to understand things in their own terms. And this just gets to the, it takes a while for ideas to propagate, even though, um, they may be very different than what people are used to.
- 1:10:35 – 1:23:09
SiFive semiconductor design
- CLChris Lattner
- LFLex Fridman
Maybe let's talk about that a little bit, the world of ASICs and-
- CLChris Lattner
Yeah.
- LFLex Fridman
Well, actually, your, um, your- you have a new role at SiFive. What's that place about? What is the vision-
- CLChris Lattner
Sure.
- LFLex Fridman
... uh, for their vision for, I would say, the future of computing?
- CLChris Lattner
Yeah. So I lead the engineering product teams at SiFive. SiFive is a company who's, was founded with this architecture called RISC-V. RISC-V is a new instruction set. Instruction sets are the things inside of your computer that tell it how to run things. Um, x86 from Intel and ARM from the ARM company and things like this are other instruction sets.
- LFLex Fridman
I've talked to ............................ I talked to Dave Patterson, who's super excited about RISC-V.
- CLChris Lattner
Yeah. Dave, Dave is awesome.
- LFLex Fridman
Yeah, he's brilliant. Yeah.
- CLChris Lattner
Yeah. The, uh, RISC-V is distinguished by not being proprietary.
- LFLex Fridman
Mm-hmm.
- CLChris Lattner
And so x86 can only be made by Intel and AMD. ARM can only be made by ARM. They sell licenses to build ARM chips to other companies, things like this. MIPS is another instruction set that is owned by the MIPS company, now Wave, and then it gets licensed out, things like that. Um, and so RISC-V is an open standard that anybody can build chips for. And so SiFive was founded by three of the founders of RISC-V that designed and built it in Berkeley working with Dave. Um, and so that was the, the genesis of the company. SiFive today has some of the world's best RISC-V cores and we're selling them and that's really great. They're going to tons of products. It's very exciting. Um-
- LFLex Fridman
So they're taking this, uh, thing that's open source and just being, tr- uh trying to be or are the best in the world at building these things?
- CLChris Lattner
Yeah. So here it's the specification's open source. It's like saying TCPIP is an open standard or C is an open standard. But then you have to build an implementation of the standard. And so SiFive, on the one hand, pushes forward and defined and pushes forward the standard. On the other hand, we have implementations that are best in class for different points in the space, depending on if you want a really tiny CPU or if you want a really big beefy one that, that, uh, is faster but it uses more area and things like this.
- LFLex Fridman
What about the actual manufacturer of the chip? So like what-
- CLChris Lattner
Yeah.
- LFLex Fridman
Where does that all fit? I'm gonna ask a bunch of dumb questions.
- CLChris Lattner
That's okay.
- LFLex Fridman
(laughs)
- CLChris Lattner
This is how we learn, right? Uh, and so, uh, what, the, the way this works is that there's generally a separation of the people who design the circuits and then the people who manufacture them. And so the, you'll hear about fabs like TSMC and Samsung and things like this that actually produce the chips, but they take a design coming in and that design specifies how, um, how the, you know, you turn, uh, code for the chip into, uh, little rectangles that then use photolitho- lithography to make, uh, mask sets and then burn transistors onto a chip or onto a, onto silicon rather.
Episode duration: 2:42:40
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode nWTvXbQHwWs
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome