Lex Fridman PodcastGilbert Strang: Linear Algebra, Teaching, and MIT OpenCourseWare | Lex Fridman Podcast #52
EVERY SPOKEN WORD
75 min read · 15,440 words- 0:00 – 3:45
Introduction
- LFLex Fridman
The following is a conversation with Gilbert Strang. He's a professor of mathematics at MIT, and perhaps one of the most famous and impactful teachers of math in the world. His MIT Open Courseware lectures on linear algebra have been viewed millions of times. As an undergraduate student, I was one of those millions of students. There's something inspiring about the way he teaches, that is at once calm, simple, and yet full of passion for the elegance inherent to mathematics. I remember doing the exercises in his book, Introduction to Linear Algebra, and slowly realizing that the world of matrices, of vector spaces, of determinants and eigenvalues, of geometric transformations, and matrix decompositions reveal a set of powerful tools in the toolbox of artificial intelligence; from signals to images, from numerical optimization to robotics, computer vision, deep learning, computer graphics, and everywhere outside AI, including, of course, a quantum mechanical study of our universe. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, give it five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D-M-A-N. This podcast is supported by ZipRecruiter. Hiring great people is hard, and to me, is the most important element of a successful mission-driven team. I've been fortunate to be a part of and to lead several great engineering teams. The hiring I've done in the past was mostly through tools that we built ourselves, but reinventing the wheel was painful. ZipRecruiter's a tool that's already available for you. It seeks to make hiring simple, fast, and smart. For example, Codable co-founder, Gretchen Huebner, used ZipRecruiter to find a new game artist to join her education tech company. By using ZipRecruiter's screening questions to filter candidates, Gretchen found it easier to focus on the best candidates and finally hiring the perfect person for the role in less than two weeks from start to finish. ZipRecruiter, the smartest way to hire. See why ZipRecruiter's effective for businesses of all sizes by signing up, as I did, for free at ziprecruiter.com/lexpod. That's ziprecruiter.com/lexpod. This show is presented by Cash App, the number one finance app in the App Store. I personally use Cash App to send money to friends, but you can also use it to buy, sell, and deposit Bitcoin. Most Bitcoin exchanges take days for a bank transfer to become investable. Through Cash App, it takes seconds. Cash App also has a new investing feature. You can buy fractions of a stock, which to me is a really interesting concept. So you can buy of $1 worth, no matter what the stock price is. Brokerage services are provided by Cash App Investing, a subsidiary of Square and member SIPC. I'm excited to be working with Cash App to support one of my favorite organizations that many of you may know and have benefited from called FIRST, best known for their FIRST Robotics and Lego competitions. They educate and inspire hundreds of thousands of students in over 110 countries and have a perfect rating on Charity Navigator, which means the donated money is used to maximum effectiveness. When you get Cash App from the App Store or Google Play and use code LEXPODCAST, you'll get $10 and Cash App will also donate $10 to FIRST, which again is an organization that I've personally seen inspire girls and boys to dream of engineering a better world. And now here's my conversation with Gilbert Strang.
- 3:45 – 5:10
Math rockstar
- LFLex Fridman
How does it feel to be one of the, uh, modern day rockstars of mathematics?
- GSGilbert Strang
(laughs) I don't feel like a rockstar. That's kind of crazy for old math person. But, uh, it's true that, um, the videos in linear algebra that I made way back in 2000, I think, uh, have been watched a lot. And, uh, well, partly the importance of linear algebra, uh, which we- I'm sure you'll ask me and give me a chance to say that linear algebra as a subject has just surged in importance. But also, I, it was a class that I taught a bunch of times, so I kind of got it organized and, uh, an- and enjoyed doing it. It was just the videos were just the class, so they're on Open Courseware and on YouTube and translated-
- LFLex Fridman
But th-
- GSGilbert Strang
... as one.
- LFLex Fridman
But there's something about that chalkboard and the, and the simplicity of the way you explain the basic concepts in the beginning. I, you know, to be honest, when I went to undergrad, you know...
- GSGilbert Strang
You didn't do linear algebra probably.
- LFLex Fridman
Of course, I did linear algebra.
- GSGilbert Strang
You did? Okay, yeah.
- LFLex Fridman
Yeah, yeah, yeah, of course.
- GSGilbert Strang
Right.
- LFLex Fridman
But I, before going through the course at my university, I li- there was going through Open Course where I was, you were my instructor for linear algebra.
- GSGilbert Strang
Oh, I see. Right, yeah.
- LFLex Fridman
(laughs) And that, uh, I mean, we were using your book, and, I mean, that, that,
- 5:10 – 7:29
MIT OpenCourseWare
- LFLex Fridman
the fact that there is thousands, you know, hundreds of thousands, millions of people that watch that video, I think that's-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... that's really powerful. So, uh, how do you think the idea of putting lectures online wo- would really... MIT Open Courseware has innovated?
- GSGilbert Strang
That was a wonderful idea. You know, I think, uh, uh, the story that I've heard is the committee, uh, committee was appointed by the president, President Vest at that time, a wonderful guy. And, uh, the idea of the committee was to figure out how MIT could make, uh, be like other universities market, uh, market the work we were doing. And then they didn't see a way and after a weekend and they had an inspiration and came back to the President Vest and said, "What if we just gave it away?" And, uh, he decided that was g- okay, good idea.... so...
- LFLex Fridman
You know, that's a crazy idea, that's, uh-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... if we think of a university as a thing that creates a product-
- GSGilbert Strang
Yes.
- LFLex Fridman
... isn't knowledge-
- GSGilbert Strang
Right.
- LFLex Fridman
... the, uh, you know, the kind of educational knowledge, isn't the product, and giving that away?
- GSGilbert Strang
Yeah.
- LFLex Fridman
Are you surprised that (laughs) -
- GSGilbert Strang
Th- the-
- LFLex Fridman
... that it went through?
- GSGilbert Strang
... uh, th- the result that it w- that he did it? Well, knowing a little bit, President Vest, it was like him, I think.
- LFLex Fridman
(laughs)
- GSGilbert Strang
And, uh, and it was really the right idea. You know, uh, um, MIT is a kind of... It's known for being high-level technical things. And, and this is the best way we can, say, w- tell, we can show what MIT really is like, uh, 'cause th- the, the, v- in my case, those 18.06 videos are just teaching the class. They were there in 26.100. They're kind of fun to look at. People write to me and say, "Oh, you've got a sense of humor," but I, I don't know where (laughs) that comes through. Somehow, I've been friendly with the class. I like students-
- LFLex Fridman
Yeah.
- GSGilbert Strang
... and, uh... And linear algebra, uh, uh, the subje- we gotta give the subject most of the credit. It, it, it really has come forward in importance, uh, in, in
- 7:29 – 13:11
Four Fundamental Subspaces of Linear Algebra
- GSGilbert Strang
these years.
- LFLex Fridman
So let's talk about linear algebra a little bit 'cause it is such a... It's both a powerful and a beautiful, uh, s- uh, subfield of mathematics. So what's your s- favorite specific topic in linear algebra, or even math in general, to give a lecture on, to, to-
- GSGilbert Strang
I see.
- LFLex Fridman
... convey, to tell a story-
- GSGilbert Strang
I see.
- LFLex Fridman
... to teach students?
- GSGilbert Strang
Okay. Well, on the teaching side, so it's not deep mathematics at all, but I, I'm kind of proud of the idea of the four subspaces, the, the four fundamental subspaces, w- which were, of course, known before, long before m- my name for them. But, uh, th-
- LFLex Fridman
Can you go through them? Can you go through the four subspaces?
- GSGilbert Strang
Sure I can, yeah. So the first one to understand is, so the matrix is... Maybe I should say the matrix is-
- LFLex Fridman
What is a matrix?
- GSGilbert Strang
What's a matrix? Well, so we have a, like, a rectangle of numbers. So it's got N columns, it's got a bunch of columns, and also got, um, M rows, let's say. And the relation between... So, of course, the columns and the rows, it's the same numbers, so there's gotta be connections there. But they're not simple. The, uh, they're, they're mi- the columns might be longer than the rows, and they're all different. The numbers are mixed up. First space to think about is, take the columns, so those are vectors, those are points in N dimensions.
- LFLex Fridman
What's a vector?
- GSGilbert Strang
So a physicist would imagine a vector, or might imagine a vector as a arrow, you know, in space, or the point it ends at i- in space. For me, it's a column of numbers that-
- LFLex Fridman
This is- i- it... You often think of... This is very interesting in terms of linear algebra-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... in terms of a vector. You think a little bit more abstract than the... how it's very commonly used perhaps.
- GSGilbert Strang
Yeah.
- LFLex Fridman
You, you think this arbitrary s- spa- multidimensional space, that there's a-
- GSGilbert Strang
Yeah. I'm, I'm... Right away, I'm in high dimensions and in the-
- LFLex Fridman
(laughs) Dreamland (laughs) .
- GSGilbert Strang
Yeah, that's right. In a lecture, I try to... So if you think of two vectors in 10 dimensions, I'll do this in class. And I'll readily admit that I have no good image in my mind of a vector, of a arrow in 10-dimensional space, but whatever. Y- you can, uh, you can add one bunch of ten numbers to another bunch of ten numbers. So you can add a vector to a vector, and you can multiply a vector by three. And that's... If you know how to do those, you've got linear algebra.
- LFLex Fridman
You know, 10 dimensions-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... you know, there's this beautiful thing about math if we look at string theory and-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... all these theories which are really fundamentally derived through math-
- GSGilbert Strang
Yes.
- LFLex Fridman
... but are very difficult to visualize. And-
- GSGilbert Strang
Yeah.
- LFLex Fridman
How do you think about the things, like a 10-dimensional vector, that we can't really visualize?
- 13:11 – 15:03
Linear Algebra vs Calculus
- GSGilbert Strang
- LFLex Fridman
So planes in these multidimensional spaces, how, how difficult of an idea is that to, to come to? Do you think if you-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... if you look back in time-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... I, I think, uh, mathematically, it makes sense. But I don't know if it's intuitive for our... uh, us to imagine-
- GSGilbert Strang
Right.
- LFLex Fridman
... just as we were talking about. It feels like calculus is easier to-
- GSGilbert Strang
I see.
- LFLex Fridman
... intuit.
- GSGilbert Strang
Well, calcul- I have to admit, calculus came earlier-
- LFLex Fridman
(laughs)
- GSGilbert Strang
... uh, earlier than linear algebra. So Newton and Leibniz were the great men to understand the key ideas of calculus. But linear algebra, to me, is like, okay, it's the starting point 'cause it's all about flat things.
- LFLex Fridman
(laughs)
- GSGilbert Strang
Calculus has got... All the complications of calculus come from the curves-
- LFLex Fridman
Yeah.
- GSGilbert Strang
... the bending, the s- the curved surfaces. Linear algebra, the surfaces are all flat.
- LFLex Fridman
(laughs)
- GSGilbert Strang
Nothing bends in linear algebra.
- LFLex Fridman
(laughs)
- GSGilbert Strang
So, uh, it should've come first but it didn't. And calculus also comes first in, in high school classes, in, in college class. It'll be freshman math, it'll be calculus. And then I say, "Enough of it." Like, "Okay, get to, get to the good stuff." And, uh, that-
- LFLex Fridman
Do you think linear algebra should come first?
- GSGilbert Strang
Well, it really... Uh, I- I'm okay with it not coming first but it should. Yeah, it should. It's simpler. It- i- i-
- LFLex Fridman
'Cause everything is flat (laughs) .
- GSGilbert Strang
Yeah, everything is flat.
- LFLex Fridman
(laughs)
- GSGilbert Strang
Well, of course, for that reason, your calculus sort of sticks to one dimension or s- or eventually you do multivariate, but that basically means two dimensions. Linear algebra, you take off into 10 dimensions, no problem.
- LFLex Fridman
It just feels scary and dangerous to go beyond two dimensions.
- GSGilbert Strang
Well-
- LFLex Fridman
That's all. (laughs)
- 15:03 – 19:47
Singular value decomposition
- GSGilbert Strang
- LFLex Fridman
So w- what concept or theorem in linear algebra or in math you find most beautiful, that gives you pause-
- GSGilbert Strang
Most beautiful.
- LFLex Fridman
... that leaves you in awe?
- GSGilbert Strang
Well, I'll stick with linear algebra here. Uh, I hope the viewer knows that really mathematics is amazing, amazing subject and deep, deep, uh, connections between ideas that didn't look connected. Some, they turned out they were. But if we stick with linear algebra, so we have a matrix. That, that's like the basic thing, a rectangle of numbers, and might be a rectangle of data. You're probably gonna ask me later about data science-
- LFLex Fridman
Yeah.
- GSGilbert Strang
... where an often data comes in a matrix. You have, you know, the... uh, maybe every column corresponds to a, to a drug, and every row corresponds to a patient. And, and, uh, if the patient, uh, uh, reacted favorably to the drug then you put up some positive number in there. Anyway, m- m- rectangle of n- of numbers, a matrix is basic. So, uh, the big problem is to understand all those numbers. You got a big, big set of numbers and what are the patterns, what's going on? And, uh, so one of the ways to break down that matrix into simple pieces is uses something called singular values.
- LFLex Fridman
Mm-hmm.
- GSGilbert Strang
And that's come on as fundamental in the last... in, certainly in my lifetime. Uh, eigenvalues pro- if you have viewers who've done engineering math or, or, uh, or basic linear algebra, eigenvalues were in there. Uh, but those are restricted to square matrices. And data comes in rectangular matrices, so you gotta take that... you gotta take that next step.
- LFLex Fridman
(laughs)
- GSGilbert Strang
I'm, I'm always pushing-
- LFLex Fridman
(laughs)
- GSGilbert Strang
... math faculty, "Get on, do, do, do it. Do it," uh, singular values. So those are a way to break, to, to make, to find the es- the important pieces of the matrix w- which add up to the whole matrix. So, so you're breaking a matrix into simple pieces and, uh, the first piece is the most important part of the data, the second piece is the second most important part. And, uh, then often... So a data scientist will have to like... if you, if a data scientist can find those first and second pieces, stop there, the rest of, of the data is probably round off, you know, er- um, experimental error maybe. So you're looking for the important part.
- LFLex Fridman
Yeah. So what do you find beautiful about singular values? What, what is the problem-
- GSGilbert Strang
Well, yeah, I didn't give the theorem. Yeah, so here's the, here's the idea of singular values. Every matrix, every matrix, uh, rectangular, square, whatever-... can be written as a product of three very simple special matrices. So that's the theorem. Every matrix can be written as a rotation, times a stretch, which is a s- just a matrix, a diagonal matrix, otherwise all zeros except on the one diagonal, and then a thir- and the third factor is another rotation. So rotation, stretch, rotation is the breakup of a, of a, of any matrix.
- LFLex Fridman
The structure that, uh, the ability that you can do that, what- what- what do you find appealing? What do you find beautiful about it?
- GSGilbert Strang
Well, geometrically, as I freely admit, the- the ma- action of a matrix, this is not so easy to visualize. But everybody can visualize a rotation. Take- take- take two-dimensional space and just turn it around the, around the center. Take three-dimensional space. So a pilot has to know about, well, what are the three, the yaw is one of them. I've forgotten all of the three turns that a pilot makes. Uh, up to ten dimensions, you got ten ways to turn. But, uh, you can visualize a rotation. Take the space and turn it. And you can visualize a stretch. So to break a- a- a- a matrix with all those numbers in it into something you can visualize, rotate, stretch, rotate, is pretty neat.
- LFLex Fridman
Yeah.
- GSGilbert Strang
Pretty neat.
- LFLex Fridman
That's pretty powerful.
- 19:47 – 23:38
Why people like math
- LFLex Fridman
On YouTube, just consuming a bunch of videos and just watching what people connect with and what they really enjoy and are inspired by, math seems to come up again and again. I- I'm trying to understand why that is. Perhaps you can help-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... me, give me clues. So it's not just the lec- the kinds of lectures that y- you give, but it's also just the other folks, like with Numberphile, there's a channel-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... where they just chat about things that are extremely complicated, actually.
- GSGilbert Strang
Yeah.
- LFLex Fridman
People, nevertheless, connect with them.
- GSGilbert Strang
Yeah.
- LFLex Fridman
What do you think that is? What-
- GSGilbert Strang
It's wonderful, isn't it?
- LFLex Fridman
Yeah.
- GSGilbert Strang
I mean, I wasn't really aware of it. Do s- we're- we're conditioned to think math is hard, math is abstract, math is just for a few people. But it isn't that way. A lot of people quite like math. And they like to... I get messages from people saying, "You know, now I'm retired, I'm gonna learn some more math." I get a lot of those. It's really encouraging. And I think what people like is that there's some order, you know, a lot of order and org- you know, things are, uh, not obvious, but they're true. So it's really cheering to think that, that, uh, so many people really want to learn more about math. Yeah.
- LFLex Fridman
Well, in terms of truth-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... again, sorry to, uh, slide into philosophy at times-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... but, uh, math does reveal pretty strongly what things are true.
- GSGilbert Strang
Yeah.
- LFLex Fridman
I mean, that's the whole point of proving things.
- GSGilbert Strang
It is, yeah.
- LFLex Fridman
Wh- what... And yet, sort of our real world is messy and complicated.
- GSGilbert Strang
It is.
- LFLex Fridman
What do you think about the nature of truth that math reveals? Is-
- GSGilbert Strang
Oh, wow. (laughs)
- LFLex Fridman
Because it is a source of comfort, like you- you've mentioned-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... in our lives.
- GSGilbert Strang
That's right. Well, I have to say, I'm- I'm not much of a philosopher. I just like numbers, you know?
- LFLex Fridman
(laughs) Yeah.
- GSGilbert Strang
As a kid, I would, you'd... This was before you had, uh, uh, you had to go in when you're, when you had a filling in your teeth. You had to kind of just take it. Yeah. So I, what I did was think about math, you know, like take powers of two, two, four, eight, 16, up until the time the tooth stopped hurting-
- 23:38 – 25:04
Teaching by example
- LFLex Fridman
do you think for teaching and in general, thinking about new concepts, do you think it's better to plug in the numbers-
- GSGilbert Strang
(laughs)
- LFLex Fridman
... or to think more abstractly? So, uh, l- looking at theorems and proving the theorems, or actually, you know, building up a basic intuition of the theorem or the method, the approach, and then just plugging in numbers and seeing it work?
- GSGilbert Strang
Yeah. Well, certainly, uh, uh, many of us like to see examples first. We- we understand it might be a pretty abstract sounding example, like a three-dimensional rotation. How- how are you gonna, how are you gonna understand a rotation in 3D, uh, or in 10D?... or, uh, uh, but, uh... The, and then some of us like to keep going with it to the point where you got numbers, where you got-
- LFLex Fridman
Mm-hmm.
- GSGilbert Strang
... 10 angles, 10 axes, 10 angles, uh, but, uh, the best, the great mathematicians probably... I don't know if they do that 'cause they, they... for them, uh, uh, uh, uh, uh, for them, uh, an example would be a highly abstract thing to the rest of us.
- LFLex Fridman
Right. But nevertheless, it's working within the space of examples.
- GSGilbert Strang
Yeah, examples.
- LFLex Fridman
It seems to-
- GSGilbert Strang
Examples of, of structure.
- LFLex Fridman
Our brains seem to connect with that.
- GSGilbert Strang
Yeah,
- 25:04 – 26:46
Andrew Yang
- GSGilbert Strang
yeah.
- LFLex Fridman
So, uh, not sure if you're familiar with him, but A- Andrew Yang is a presidential candidate currently running-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... with, uh, "MATH" in all capital letters in his hats as a slogan.
- GSGilbert Strang
I see.
- LFLex Fridman
Stands for "Make America Think Hard."
- GSGilbert Strang
Okay.
- LFLex Fridman
(laughs)
- GSGilbert Strang
I'll vote for him. (laughs) .
- LFLex Fridman
(laughs) So, uh... And his name rhymes with yours, Yang, Strang, so. But he also loves math-
- GSGilbert Strang
Yeah.
- LFLex Fridman
And he comes from that world of-
- GSGilbert Strang
Right.
- LFLex Fridman
But he also... Looking at it makes me realize that math, science, and engineering are not really part of our politics, uh-
- GSGilbert Strang
Right.
- LFLex Fridman
... political discourse, about political life-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... government in general.
- GSGilbert Strang
Yeah.
- LFLex Fridman
Why do you think that is?
- GSGilbert Strang
Well-
- LFLex Fridman
What are your thoughts on that in general?
- GSGilbert Strang
Well, certainly somewhere in the system we need people who are comfortable with numbers, comfortable with quantities. You know, if you, if you say this leads to that, they see it and it's, it's undeniable.
- LFLex Fridman
But isn't that strange to you that we have almost no... I mean, I'm pretty sure we have-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... no elected officials in Congress or obviously the pr- president-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... that is a... either has an engineering degree or a math degree (laughs) .
- GSGilbert Strang
Yeah, well, uh, that's too bad, in... No, a few could, uh... a few who could make the connection. Yeah, it would have to be people who are at the... who are, uh, who understand engineering or science and at the same time can make speeches and, uh, and, uh, lead, yeah.
- LFLex Fridman
And inspire people.
- 26:46 – 29:21
Society for Industrial and Applied Mathematics
- GSGilbert Strang
yeah.
- LFLex Fridman
So you were, speaking of inspiration, the president of The Society for Industrial and Applied Mathematics. (laughs)
- GSGilbert Strang
Oh, yes.
- LFLex Fridman
It's a major organization in, in, in math, in applied math. What do you see as the role of that society, you know, in our public discourse-
- GSGilbert Strang
Right.
- LFLex Fridman
... in public life?
- GSGilbert Strang
Yeah, so... Well, it was fun to be president at the time- (laughs)
- LFLex Fridman
(laughs) .
- GSGilbert Strang
... at the time.
- LFLex Fridman
Couple of years? A few years?
- GSGilbert Strang
Y- two years around, around 2000. Um, uh, it was all... That's president of a pretty small society, but nevertheless, it was a time when math was getting some m- more attention in Washington. But yeah, I got to give a, a little ten minutes to, uh, in- to, uh, Committee of the House of Representatives-
- LFLex Fridman
Nice.
- GSGilbert Strang
... talking about why ma- And then a- actually it was fun because, uh, one of the members of the House, uh, y- uh, uh, uh, had been a student, had been in my class.
- LFLex Fridman
(laughs)
- GSGilbert Strang
What do you think of that? Yeah, as you say, pretty rare. Most, most members of the House have had a different training, different background, but, uh, there was one from New Hampshire who, who was, uh, my friend really by, by being in the class, so, yeah. So tha- those years were good. Uh, then, of course, other things take, take over in importance in Washington and math-
- LFLex Fridman
Yes.
- GSGilbert Strang
... math, uh, just, at this point, is not so visible, but for-
- LFLex Fridman
Well-
- GSGilbert Strang
... a little moment it was.
- LFLex Fridman
There's some excitement, uh, some concern about artificial intelligence in Washington now-
- GSGilbert Strang
Yeah, sure.
- LFLex Fridman
... about the future.
- GSGilbert Strang
Yeah, uh-
- LFLex Fridman
And I think at the core of that is math (laughs) .
- GSGilbert Strang
Well, it is, yeah.
- LFLex Fridman
Yeah.
- GSGilbert Strang
But-
- LFLex Fridman
Maybe it's hidden, maybe it's wearing a, a different hat.
- GSGilbert Strang
Well, that's right.
- LFLex Fridman
But, uh-
- 29:21 – 37:28
Deep learning
- GSGilbert Strang
this."
- LFLex Fridman
So maybe from your perspective, let me ask the, this, the most basic question.
- GSGilbert Strang
Yeah.
- LFLex Fridman
H- how do you think of a neural network? What is a neural network?
- GSGilbert Strang
What... Yeah, okay. So can I s- start with the, uh, i- idea about deep learning? What does that mean?
- LFLex Fridman
Sure.
- GSGilbert Strang
Uh...
- LFLex Fridman
What is deep learning? (laughs)
- GSGilbert Strang
What is deep learning? Yeah. So, so w- we're trying to learn... From all this data, we're trying to learn what's important, what, what's, um, what's it telling us. So you've, you've got data. You've got some inputs for which you know the right outputs. The question is, can you see the pattern there? Can you figure out a way for a new input, which we haven't seen, to, to get the, to, to understand what the output will be from that new input? So we've got a million inputs with their outputs. So we're trying to create some pattern, some rule that will take those inputs, those million training inputs which we know about, to the correct million outputs.... and, uh, this idea of a neural net is part of the structure of the, of our new way to create a, create a rule. We're looking for a rule that will take these training inputs to the known outputs, and then we're gonna use that rule on new inputs that we don't know the output and, and see what comes.
- LFLex Fridman
Linear algebra is a big part of defining- of finding that rule.
- GSGilbert Strang
That's right. Linear algebra is a big part, not all the part. People were leaning on matrices, that's good, still do. Linear is something special. It's, it's all about straight lines and flat planes, and, uh, and, and data isn't quite like that, you know? It's, uh, it's, it's more complicated. So you gotta introduce some complication, so you have to have some function that's not a straight line-
- LFLex Fridman
Nonlinear.
- GSGilbert Strang
... and it turned out that... Nonlinear, nonlinear-
- LFLex Fridman
Scary.
- GSGilbert Strang
... not linear. And it turned out that, uh, it was enough to use the function that's one straight line and then a different one halfway-
- LFLex Fridman
That's- (laughs)
- GSGilbert Strang
... so piecewise linear.
- LFLex Fridman
Piecewise linear.
- GSGilbert Strang
One piece of- one piece has one slope, one piece, the other piece has a second slope.
- LFLex Fridman
Yeah.
- GSGilbert Strang
And, uh, so that introdu-
- LFLex Fridman
That's-
- GSGilbert Strang
... getting that nonlinear, simple nonlinearity in, uh, blew the problem open.
- LFLex Fridman
That little piece makes it sufficiently complicated to make things interesting.
- GSGilbert Strang
Exactly, 'cause you're gonna use that piece over and over a million times, so you, so you... It has a, it has a fold in the, in the graph, the graph two pieces, and, but when you fold something a million times, you got, you've got a pretty complicated function that's pretty realistic.
- LFLex Fridman
So that's the thing about neural networks is they have a lot of these-
- GSGilbert Strang
A lot of these, that's right.
- LFLex Fridman
... so why do you think neural networks, by using, uh, sort of formulating an objective function, very not a plane-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... uh, uh, function-
- 37:28 – 38:54
Theory vs application
- LFLex Fridman
what excites you more? The... We just talked about a little bit of the application.
- GSGilbert Strang
Yeah.
- LFLex Fridman
What excites you more, theory or the application of mathematics?
- GSGilbert Strang
Well, for myself, I'm probably a theory person. I'm not, uh... I'm speaking here pretty freely about applications, but I'm not a person who really... I'm not a physicist or a chemist or a neuroscientist. Uh, so for myself, I like the, the structure and the su- s- flat sub-spaces and, and, and the, and the relation of matrices, columns, the rows. That's m- my part, uh, uh, in the spectrum. So there really, uh, r- really, science is a big spectrum of people from asking practi- practical questions and answering them using some math, then some math guys, like myself, who are in the middle of it, and then the geniuses of math and physics and chemistry and... who are finding fundamental rules and then doing, doing the... really, uh, understanding nature. That's, that's incredible.
- LFLex Fridman
At its lowest, uh, simplest level.
- 38:54 – 39:00
Open problems in mathematics
- LFLex Fridman
Maybe just a quick, in broad strokes, from your perspective, what is, uh,
- 39:00 – 41:52
Linear algebra as a subfield of mathematics
- LFLex Fridman
where does l- linear algebra sit as a sub-field of mathematics? What, what are the various sub-fields that you're-
- GSGilbert Strang
Okay.
- LFLex Fridman
... that you think about in relation to linear algebra?
- GSGilbert Strang
So the big fields of math are algebra as a whole, and problems like c- calculus and differential equations, so that's a second, quite different field. Then maybe geometry, uh, deserves to be under- thought of as a different field under "Stand the geometry of high-dimensional surfaces." So I think... Am I allowed to say this here? I- I think-
- LFLex Fridman
Uh-oh.
- GSGilbert Strang
... m-
- LFLex Fridman
Calculus?
- GSGilbert Strang
... this, this is, this is-
- LFLex Fridman
Discussion? (laughs)
- GSGilbert Strang
... this is where, uh, personal view comes in. I think, uh, m- math, if we're thinking about undergraduate math-
- LFLex Fridman
Mm-hmm.
- GSGilbert Strang
... what, what millions of students study, I think we o- uh, overdo the calculus at, at the cost of the algebra, at the cost of linear.
- LFLex Fridman
So you have this talk-
- GSGilbert Strang
Linear.
- LFLex Fridman
... titled Calculus Versus Linear Algebra.
- GSGilbert Strang
That's right. That's right.
- LFLex Fridman
No, and s- and, and you say that linear algebra wins, so-
- GSGilbert Strang
Uh, well...
- LFLex Fridman
... can you, can you, uh, can you dig into that a little bit?
- GSGilbert Strang
Yeah.
- LFLex Fridman
Why does linear algebra win? (laughs)
- GSGilbert Strang
(laughs) Right. Well, okay, I'm, uh... The viewer is gonna think this guy is biased.
- LFLex Fridman
(laughs)
- GSGilbert Strang
Not true. I'm, I'm just telling the truth as it is. Yeah, so I- I feel linear algebra is, is, is just a nice part of math that people can get the idea of. They can understand something that's a little bit abstract, 'cause once you get to 10 or 100 dimensions, um... And very, very, very useful. That's what's happened in, in my lifetime is the, the, the importance of data which does come in matrix form, so it's really set up for algebra. It's not set up for differential equations. And, uh, uh, let me f- fairly add probability. The ideas of probability and statistics have become very, very important, have also jumped forward. So... And that's not... That's different from linear algebra, quite different. So now, we really have three major areas to me. Uh, calculus, uh, linear algebra, matrices, and probability statistics, and, uh, they all deserve a important place.
- LFLex Fridman
Oh, yeah.
- GSGilbert Strang
And, and, and calculus is traditionally had a, had a, it's a lion's share of the time-
- LFLex Fridman
A disproportionate share.
- GSGilbert Strang
... a dis- thank you.
- LFLex Fridman
Yeah.
- GSGilbert Strang
Disproportionate, that's a good word.
- 41:52 – 46:19
Favorite matrix
- LFLex Fridman
I know it's hard to pick favorites, but what is your favorite matrix?
- GSGilbert Strang
What's my (laughs) favorite matrix?
- LFLex Fridman
(laughs)
- GSGilbert Strang
Okay. So my favorite matrix is square, I admit it. It's a square bunch of numbers, and it has twos running down the main diagonal. T- and, uh, on the next diagonal, so think of top left to bottom right, twos down, down the middle of the matrix, and minus ones just above those twos, and minus ones just below those twos. And otherwise, all zero. So mostly zeros. Just three non-zero diagonals coming down.
- LFLex Fridman
What is interesting about it?
- GSGilbert Strang
Well, all the different ways it comes up.You see it in engineering, you see it as analogous in calculus to second derivative. So, calculus learns about taking the derivative, the figuring out how much, how fast something's changing. But second derivative, now that's also important. That's how fast the change is changing, how fast the graph is bending. How, how fast it's, it's curving. Uh, and Ei- Einstein showed that that's fundamental to understand space. So, second derivatives should have a bigger place in calculus, second mice, matrices, which are like the linear algebra version of second derivatives, are neat in, in linear algebra. Yeah. Just everything comes out right with those guys. (laughs)
- LFLex Fridman
(laughs)
- GSGilbert Strang
Yeah.
- LFLex Fridman
Beautiful. What have you learned about the process of learning by having taught so many students math over the years?
- GSGilbert Strang
Ooh, that is hard. I'll have to admit here that I'm not, I'm not really a good teacher.
- LFLex Fridman
(laughs)
- GSGilbert Strang
Because I don't get into the exam part. The exam's the part of my life that I don't like, and grading them, and giving the students A or B or whatever. I do it because it's, I'm supposed to do it. But, uh, but I tell the class at the beginning, I don't know if they believe me, probably they don't, I, I tell the class, "I'm here to teach you. I'm here to teach you math, and not to grade you." And, but they're thinking, "Okay, this guy is gonna, you know, when's he gonna, is he gonna give me an A minus? Is he gonna give me a B plus? What?"
- LFLex Fridman
What, what have you learned about the process of learning?
- GSGilbert Strang
Of learning? Yeah. Well-
- LFLex Fridman
So the students-
- GSGilbert Strang
... maybe to be the, to give you a legitimate answer about learning, I should've paid more attention to the assessment, the evaluation part at the end. But I like the teaching part at the start, that's the sexy part-
- LFLex Fridman
(laughs)
- GSGilbert Strang
... to, to tell somebody for the first time about a matrix. Wow.
- LFLex Fridman
But is there, are there moments, so you, you are teaching a concept, are there moments of learning that y- you just see in the students' eyes, you don't need to look at the grades-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... but you see in their eyes that, that you hook them. That, you know, that you connect with them in a way where, you know what, they, they f- they fall in love with this-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... with this beautiful world of mathematics.
- GSGilbert Strang
They see that it's got some beauty there.
- LFLex Fridman
It g- it, it, see-
- GSGilbert Strang
Yeah.
- LFLex Fridman
Or conversely-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... that they give up at that point-
- GSGilbert Strang
Uh-huh.
- 46:19 – 47:37
Advice for students on their journey through math
- LFLex Fridman
- GSGilbert Strang
Like-
- LFLex Fridman
... see it.
- GSGilbert Strang
Yeah.
- LFLex Fridman
What advice do you have to a student just starting their journey in mathematics today? How do they get started? (laughs)
- GSGilbert Strang
(laughs) Oh, yeah, that's hard. Well, I hope you, you have a teacher, professor who, uh, is still enjoying what he's doing.
- LFLex Fridman
Mm-hmm.
- GSGilbert Strang
What he's teaching. He's still looking for new ways to teach and to, and to understand math. Uh, 'cause that's the pleasure to, to, the moment when you see, "Oh, yeah, that works."
- LFLex Fridman
So it's less about the material, you-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... you study, it's more about the source of the teacher being full of passion for the subject.
- GSGilbert Strang
Yeah, more about the fun. Yeah.
- LFLex Fridman
The fun.
- GSGilbert Strang
The, the, the moment of un- of getting it.
- LFLex Fridman
But in terms of topics?
- GSGilbert Strang
Well-
- LFLex Fridman
Linear algebra?
- GSGilbert Strang
Well, that's my, my topic. But, oh, there's beautiful things in geometry to understand. What's wonderful is that in the end, there, uh, there's a pattern, there, there's a, there are rules that, that, uh, that are followed in, uh, biology as there are in every field.
- 47:37 – 49:47
Looking back
- GSGilbert Strang
- LFLex Fridman
You describe the life of a mathematician as, as, uh, 100% wonderful.
- GSGilbert Strang
(laughs)
- LFLex Fridman
Uh, except for the grade stuff-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... and having good grades.
- GSGilbert Strang
Except for grades.
- LFLex Fridman
Yeah. When you look back at your life-
- GSGilbert Strang
Yeah.
- LFLex Fridman
... what memories bring you the most joy and pride?
- GSGilbert Strang
Well, that's a good question. I certainly feel good when I, maybe I'm giving a class in, in 18.06, that's MIT's linear algebra course that I started, so sort of there's a good feeling that, "Okay, I started this course. A lot of students take it, quite a few like it." Yeah. So I'm, I'm sort of happy when I feel I'm helping, helping make a connection between ideas and students. Between, uh, theory and the reader. Uh, yeah, it's, uh, I get a lot of very nice messages, uh, from, uh, people who've watched the videos, and it's, uh, inspiring. I just, and I'll maybe just take this chance to say thank you.
- LFLex Fridman
Well, there's millions of students who you've taught, and I am grateful to be one of them, so Gilbert-
- GSGilbert Strang
Thank you.
- LFLex Fridman
... thank you so much. It's been an honor. Thank you for talking today.
- GSGilbert Strang
I- it was a pleasure. Thanks.
- LFLex Fridman
Thank you for listening to this conversation with Gilbert Strang. And thank you to our presenting sponsor, Cash App. Download it, use code LEXPODCAST, you'll get $10, and $10 will go to FIRST, a STEM education nonprofit that inspires hundreds of thousands of young minds to learn and to dream of engineering our future. If you enjoy this podcast, subscribe on YouTube, give it five stars on Apple Podcasts, support on Patreon, or connect with me on Twitter. Finally, some closing words of advice from the great Richard Feynman, "Study hard what interests you the most, in the most undisciplined, irreverent, and original manner possible." Thank you for listening, and hope to see you next time.
Episode duration: 49:52
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode lEZPfmGCEk0
