Gilbert Strang: Linear Algebra, Teaching, and MIT OpenCourseWare | Lex Fridman Podcast #52

1. 0:00 – 15:00

   The following is a…

   1. LFLex Fridman0:00

      The following is a conversation with Gilbert Strang. He's a professor of mathematics at MIT, and perhaps one of the most famous and impactful teachers of math in the world. His MIT Open Courseware lectures on linear algebra have been viewed millions of times. As an undergraduate student, I was one of those millions of students. There's something inspiring about the way he teaches, that is at once calm, simple, and yet full of passion for the elegance inherent to mathematics. I remember doing the exercises in his book, Introduction to Linear Algebra, and slowly realizing that the world of matrices, of vector spaces, of determinants and eigenvalues, of geometric transformations, and matrix decompositions reveal a set of powerful tools in the toolbox of artificial intelligence; from signals to images, from numerical optimization to robotics, computer vision, deep learning, computer graphics, and everywhere outside AI, including, of course, a quantum mechanical study of our universe. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, give it five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D-M-A-N. This podcast is supported by ZipRecruiter. Hiring great people is hard, and to me, is the most important element of a successful mission-driven team. I've been fortunate to be a part of and to lead several great engineering teams. The hiring I've done in the past was mostly through tools that we built ourselves, but reinventing the wheel was painful. ZipRecruiter's a tool that's already available for you. It seeks to make hiring simple, fast, and smart. For example, Codable co-founder, Gretchen Huebner, used ZipRecruiter to find a new game artist to join her education tech company. By using ZipRecruiter's screening questions to filter candidates, Gretchen found it easier to focus on the best candidates and finally hiring the perfect person for the role in less than two weeks from start to finish. ZipRecruiter, the smartest way to hire. See why ZipRecruiter's effective for businesses of all sizes by signing up, as I did, for free at ziprecruiter.com/lexpod. That's ziprecruiter.com/lexpod. This show is presented by Cash App, the number one finance app in the App Store. I personally use Cash App to send money to friends, but you can also use it to buy, sell, and deposit Bitcoin. Most Bitcoin exchanges take days for a bank transfer to become investable. Through Cash App, it takes seconds. Cash App also has a new investing feature. You can buy fractions of a stock, which to me is a really interesting concept. So you can buy of $1 worth, no matter what the stock price is. Brokerage services are provided by Cash App Investing, a subsidiary of Square and member SIPC. I'm excited to be working with Cash App to support one of my favorite organizations that many of you may know and have benefited from called FIRST, best known for their FIRST Robotics and Lego competitions. They educate and inspire hundreds of thousands of students in over 110 countries and have a perfect rating on Charity Navigator, which means the donated money is used to maximum effectiveness. When you get Cash App from the App Store or Google Play and use code LEXPODCAST, you'll get $10 and Cash App will also donate $10 to FIRST, which again is an organization that I've personally seen inspire girls and boys to dream of engineering a better world. And now here's my conversation with Gilbert Strang. How does it feel to be one of the, uh, modern day rockstars of mathematics?

   2. GSGilbert Strang3:51

      (laughs) I don't feel like a rockstar. That's kind of crazy for old math person. But, uh, it's true that, um, the videos in linear algebra that I made way back in 2000, I think, uh, have been watched a lot. And, uh, well, partly the importance of linear algebra, uh, which we- I'm sure you'll ask me and give me a chance to say that linear algebra as a subject has just surged in importance. But also, I, it was a class that I taught a bunch of times, so I kind of got it organized and, uh, an- and enjoyed doing it. It was just the videos were just the class, so they're on Open Courseware and on YouTube and translated-

   3. LFLex Fridman4:38

      But th-

   4. GSGilbert Strang4:38

      ... as one.

   5. LFLex Fridman4:38

      But there's something about that chalkboard and the, and the simplicity of the way you explain the basic concepts in the beginning. I, you know, to be honest, when I went to undergrad, you know...

   6. GSGilbert Strang4:50

      You didn't do linear algebra probably.

   7. LFLex Fridman4:52

      Of course, I did linear algebra.

   8. GSGilbert Strang4:53

      You did? Okay, yeah.

   9. LFLex Fridman4:54

      Yeah, yeah, yeah, of course.

   10. GSGilbert Strang4:55

       Right.

   11. LFLex Fridman4:56

       But I, before going through the course at my university, I li- there was going through Open Course where I was, you were my instructor for linear algebra.

   12. GSGilbert Strang5:04

       Oh, I see. Right, yeah.

   13. LFLex Fridman5:05

       (laughs) And that, uh, I mean, we were using your book, and, I mean, that, that, the fact that there is thousands, you know, hundreds of thousands, millions of people that watch that video, I think that's-

   14. GSGilbert Strang5:16

       Yeah.

   15. LFLex Fridman5:17

       ... that's really powerful. So, uh, how do you think the idea of putting lectures online wo- would really... MIT Open Courseware has innovated?

   16. GSGilbert Strang5:27

       That was a wonderful idea. You know, I think, uh, uh, the story that I've heard is the committee, uh, committee was appointed by the president, President Vest at that time, a wonderful guy. And, uh, the idea of the committee was to figure out how MIT could make, uh, be like other universities market, uh, market the work we were doing. And then they didn't see a way and after a weekend and they had an inspiration and came back to the President Vest and said, "What if we just gave it away?" And, uh, he decided that was g- okay, good idea.... so...

   17. LFLex Fridman6:08

       You know, that's a crazy idea, that's, uh-

   18. GSGilbert Strang6:10

       Yeah.

   19. LFLex Fridman6:10

       ... if we think of a university as a thing that creates a product-

   20. GSGilbert Strang6:14

       Yes.

   21. LFLex Fridman6:14

       ... isn't knowledge-

   22. GSGilbert Strang6:16

       Right.

   23. LFLex Fridman6:16

       ... the, uh, you know, the kind of educational knowledge, isn't the product, and giving that away?

   24. GSGilbert Strang6:22

       Yeah.

   25. LFLex Fridman6:22

       Are you surprised that (laughs) -

   26. GSGilbert Strang6:25

       Th- the-

   27. LFLex Fridman6:25

       ... that it went through?

   28. GSGilbert Strang6:25

       ... uh, th- the result that it w- that he did it? Well, knowing a little bit, President Vest, it was like him, I think.

   29. LFLex Fridman6:32

       (laughs)

   30. GSGilbert Strang6:33

       And, uh, and it was really the right idea. You know, uh, um, MIT is a kind of... It's known for being high-level technical things. And, and this is the best way we can, say, w- tell, we can show what MIT really is like, uh, 'cause th- the, the, v- in my case, those 18.06 videos are just teaching the class. They were there in 26.100. They're kind of fun to look at. People write to me and say, "Oh, you've got a sense of humor," but I, I don't know where (laughs) that comes through. Somehow, I've been friendly with the class. I like students-
2. 15:00 – 30:00

   ... I mean if…

   1. LFLex Fridman15:00

      (laughs)

   2. GSGilbert Strang15:01

      ... I mean if everything is flat, you can't go wrong.

   3. LFLex Fridman15:03

      So w- what concept or theorem in linear algebra or in math you find most beautiful, that gives you pause-

   4. GSGilbert Strang15:13

      Most beautiful.

   5. LFLex Fridman15:13

      ... that leaves you in awe?

   6. GSGilbert Strang15:15

      Well, I'll stick with linear algebra here. Uh, I hope the viewer knows that really mathematics is amazing, amazing subject and deep, deep, uh, connections between ideas that didn't look connected. Some, they turned out they were. But if we stick with linear algebra, so we have a matrix. That, that's like the basic thing, a rectangle of numbers, and might be a rectangle of data. You're probably gonna ask me later about data science-

   7. LFLex Fridman15:46

      Yeah.

   8. GSGilbert Strang15:47

      ... where an often data comes in a matrix. You have, you know, the... uh, maybe every column corresponds to a, to a drug, and every row corresponds to a patient. And, and, uh, if the patient, uh, uh, reacted favorably to the drug then you put up some positive number in there. Anyway, m- m- rectangle of n- of numbers, a matrix is basic. So, uh, the big problem is to understand all those numbers. You got a big, big set of numbers and what are the patterns, what's going on? And, uh, so one of the ways to break down that matrix into simple pieces is uses something called singular values.

   9. LFLex Fridman16:36

      Mm-hmm.

   10. GSGilbert Strang16:37

       And that's come on as fundamental in the last... in, certainly in my lifetime. Uh, eigenvalues pro- if you have viewers who've done engineering math or, or, uh, or basic linear algebra, eigenvalues were in there. Uh, but those are restricted to square matrices. And data comes in rectangular matrices, so you gotta take that... you gotta take that next step.

   11. LFLex Fridman17:05

       (laughs)

   12. GSGilbert Strang17:05

       I'm, I'm always pushing-

   13. LFLex Fridman17:08

       (laughs)

   14. GSGilbert Strang17:08

       ... math faculty, "Get on, do, do, do it. Do it," uh, singular values. So those are a way to break, to, to make, to find the es- the important pieces of the matrix w- which add up to the whole matrix. So, so you're breaking a matrix into simple pieces and, uh, the first piece is the most important part of the data, the second piece is the second most important part. And, uh, then often... So a data scientist will have to like... if you, if a data scientist can find those first and second pieces, stop there, the rest of, of the data is probably round off, you know, er- um, experimental error maybe. So you're looking for the important part.

   15. LFLex Fridman17:59

       Yeah. So what do you find beautiful about singular values? What, what is the problem-

   16. GSGilbert Strang18:03

       Well, yeah, I didn't give the theorem. Yeah, so here's the, here's the idea of singular values. Every matrix, every matrix, uh, rectangular, square, whatever-... can be written as a product of three very simple special matrices. So that's the theorem. Every matrix can be written as a rotation, times a stretch, which is a s- just a matrix, a diagonal matrix, otherwise all zeros except on the one diagonal, and then a thir- and the third factor is another rotation. So rotation, stretch, rotation is the breakup of a, of a, of any matrix.

   17. LFLex Fridman18:46

       The structure that, uh, the ability that you can do that, what- what- what do you find appealing? What do you find beautiful about it?

   18. GSGilbert Strang18:51

       Well, geometrically, as I freely admit, the- the ma- action of a matrix, this is not so easy to visualize. But everybody can visualize a rotation. Take- take- take two-dimensional space and just turn it around the, around the center. Take three-dimensional space. So a pilot has to know about, well, what are the three, the yaw is one of them. I've forgotten all of the three turns that a pilot makes. Uh, up to ten dimensions, you got ten ways to turn. But, uh, you can visualize a rotation. Take the space and turn it. And you can visualize a stretch. So to break a- a- a- a matrix with all those numbers in it into something you can visualize, rotate, stretch, rotate, is pretty neat.

   19. LFLex Fridman19:44

       Yeah.

   20. GSGilbert Strang19:45

       Pretty neat.

   21. LFLex Fridman19:45

       That's pretty powerful. On YouTube, just consuming a bunch of videos and just watching what people connect with and what they really enjoy and are inspired by, math seems to come up again and again. I- I'm trying to understand why that is. Perhaps you can help-

   22. GSGilbert Strang20:05

       Yeah.

   23. LFLex Fridman20:05

       ... me, give me clues. So it's not just the lec- the kinds of lectures that y- you give, but it's also just the other folks, like with Numberphile, there's a channel-

   24. GSGilbert Strang20:15

       Yeah.

   25. LFLex Fridman20:15

       ... where they just chat about things that are extremely complicated, actually.

   26. GSGilbert Strang20:19

       Yeah.

   27. LFLex Fridman20:19

       People, nevertheless, connect with them.

   28. GSGilbert Strang20:22

       Yeah.

   29. LFLex Fridman20:23

       What do you think that is? What-

   30. GSGilbert Strang20:24

       It's wonderful, isn't it?
3. 30:00 – 45:00

   Linear algebra is a…

   1. GSGilbert Strang30:00

      you see the pattern there? Can you figure out a way for a new input, which we haven't seen, to, to get the, to, to understand what the output will be from that new input? So we've got a million inputs with their outputs. So we're trying to create some pattern, some rule that will take those inputs, those million training inputs which we know about, to the correct million outputs.... and, uh, this idea of a neural net is part of the structure of the, of our new way to create a, create a rule. We're looking for a rule that will take these training inputs to the known outputs, and then we're gonna use that rule on new inputs that we don't know the output and, and see what comes.

   2. LFLex Fridman30:56

      Linear algebra is a big part of defining- of finding that rule.

   3. GSGilbert Strang30:59

      That's right. Linear algebra is a big part, not all the part. People were leaning on matrices, that's good, still do. Linear is something special. It's, it's all about straight lines and flat planes, and, uh, and, and data isn't quite like that, you know? It's, uh, it's, it's more complicated. So you gotta introduce some complication, so you have to have some function that's not a straight line-

   4. LFLex Fridman31:27

      Nonlinear.

   5. GSGilbert Strang31:27

      ... and it turned out that... Nonlinear, nonlinear-

   6. LFLex Fridman31:30

      Scary.

   7. GSGilbert Strang31:30

      ... not linear. And it turned out that, uh, it was enough to use the function that's one straight line and then a different one halfway-

   8. LFLex Fridman31:39

      That's- (laughs)

   9. GSGilbert Strang31:39

      ... so piecewise linear.

   10. LFLex Fridman31:40

       Piecewise linear.

   11. GSGilbert Strang31:41

       One piece of- one piece has one slope, one piece, the other piece has a second slope.

   12. LFLex Fridman31:47

       Yeah.

   13. GSGilbert Strang31:47

       And, uh, so that introdu-

   14. LFLex Fridman31:50

       That's-

   15. GSGilbert Strang31:50

       ... getting that nonlinear, simple nonlinearity in, uh, blew the problem open.

   16. LFLex Fridman31:56

       That little piece makes it sufficiently complicated to make things interesting.

   17. GSGilbert Strang32:00

       Exactly, 'cause you're gonna use that piece over and over a million times, so you, so you... It has a, it has a fold in the, in the graph, the graph two pieces, and, but when you fold something a million times, you got, you've got a pretty complicated function that's pretty realistic.

   18. LFLex Fridman32:19

       So that's the thing about neural networks is they have a lot of these-

   19. GSGilbert Strang32:23

       A lot of these, that's right.

   20. LFLex Fridman32:25

       ... so why do you think neural networks, by using, uh, sort of formulating an objective function, very not a plane-

   21. GSGilbert Strang32:36

       Yeah.

   22. LFLex Fridman32:37

       ... uh, uh, function-

   23. GSGilbert Strang32:38

       Lots of folds, yeah.

   24. LFLex Fridman32:39

       Lots of folds, of the inputs, the outputs, why do you think they work to be able to find a rule that we don't know is optimal but is just- seems to be pretty good in a lot of cases? What's your intuition? Is it surprising to you as it is to many people? Do you have an intuition of why this works at all?

   25. GSGilbert Strang33:01

       Well, I'm beginning to have a better intuition. This idea of things that are piecewise linear, flat pieces but, but with folds between them, like think of a roof of a complicated, i- infinitely complicated house or something, that, that, that curve, it almost curve, but it- but every piece is flat. Uh, that, that's been used by engineers, that idea has been used by engineers, uh, is used by engineer- big time, something called the finite-element method. If you wanna, if you wanna design a bridge, design a building, d- design a pl- airplane, you're, you're using this idea of piecewise flat as, as, as a good, a simple computable approximation.

   26. LFLex Fridman33:52

       So, but you're- you have a sense that, um, that there's a lot of expressive power in this kind of piecewise linear-

   27. GSGilbert Strang33:58

       Yeah. That's-

   28. LFLex Fridman33:59

       ... functions combined together?

   29. GSGilbert Strang34:00

       That- you used the right word. Ex- if you measure the expressivity-

   30. LFLex Fridman34:04

       Yeah.
4. 45:00 – 49:47

   Yeah. …

   1. LFLex Fridman45:00

      you are teaching a concept, are there moments of learning that y- you just see in the students' eyes, you don't need to look at the grades-

   2. GSGilbert Strang45:08

      Yeah.

   3. LFLex Fridman45:08

      ... but you see in their eyes that, that you hook them. That, you know, that you connect with them in a way where, you know what, they, they f- they fall in love with this-

   4. GSGilbert Strang45:19

      Yeah.

   5. LFLex Fridman45:19

      ... with this beautiful world of mathematics.

   6. GSGilbert Strang45:21

      They see that it's got some beauty there.

   7. LFLex Fridman45:23

      It g- it, it, see-

   8. GSGilbert Strang45:24

      Yeah.

   9. LFLex Fridman45:24

      Or conversely-

   10. GSGilbert Strang45:25

       Yeah.

   11. LFLex Fridman45:26

       ... that they give up at that point-

   12. GSGilbert Strang45:28

       Uh-huh.

   13. LFLex Fridman45:28

       ... is the opposite, the, the, the dark, say, the math, "I'm just not good at math. I wanna"-

   14. GSGilbert Strang45:32

       Yeah.

   15. LFLex Fridman45:32

       "... walk away."

   16. GSGilbert Strang45:33

       Yeah. Yeah. Maybe because of the approach in the past, they were discouraged. But don't be discouraged, it's, it's too good to miss. Um, yeah, I, uh, uh, uh, well, if I'm teaching a big class, do I know when, I think maybe I do, sort of, uh, I mentioned at the very start the, uh, four fundamental sub-spaces and the structure of the, the fundamental theorem of linear algebra. The fundamental theorem of linear algebra. That t- is the relation of those four sub-spaces, those four spaces. Yeah. So I think that, m- I, I feel that the class gets it.

   17. LFLex Fridman46:17

       When they, when they-

   18. GSGilbert Strang46:19

       Like-

   19. LFLex Fridman46:19

       ... see it.

   20. GSGilbert Strang46:19

       Yeah.

   21. LFLex Fridman46:20

       What advice do you have to a student just starting their journey in mathematics today? How do they get started? (laughs)

   22. GSGilbert Strang46:27

       (laughs) Oh, yeah, that's hard. Well, I hope you, you have a teacher, professor who, uh, is still enjoying what he's doing.

   23. LFLex Fridman46:39

       Mm-hmm.

   24. GSGilbert Strang46:40

       What he's teaching. He's still looking for new ways to teach and to, and to understand math. Uh, 'cause that's the pleasure to, to, the moment when you see, "Oh, yeah, that works."

   25. LFLex Fridman46:55

       So it's less about the material, you-

   26. GSGilbert Strang46:57

       Yeah.

   27. LFLex Fridman46:57

       ... you study, it's more about the source of the teacher being full of passion for the subject.

   28. GSGilbert Strang47:03

       Yeah, more about the fun. Yeah.

   29. LFLex Fridman47:06

       The fun.

   30. GSGilbert Strang47:06

       The, the, the moment of un- of getting it.

Episode duration: 49:52