Stanford CS229: Machine Learning Lecture 1 - Andrew Ng (Autumn 2018)

1. 0:00 – 5:21

   Introduction

   1. ANAndrew Ng0:04

      Welcome to CS229 Machine Learning. Uh, some of you know that this is a class that's taught at Stanford for a long time, and this is often the class that, um, I most look forward to teaching each year because this is where we've helped, I think, several generations of Stanford students become experts in machine learning, go out, build many of their products and services and startups that I'm sure many of you, or probably all of you, are using, uh, uh, today. Um, so what I want to do today was spend some time talking over, uh, logistics and then, uh, spend some time, you know, giving you a beginning of an intro, talk a little bit about machine learning. So about 229, um, you know, all of you have been reading about AI in the news, uh, about machine learning in the news. Um, and you've probably heard me or others say AI is the new electricity. Uh, much as the rise of electricity about a hundred years ago transformed every major industry, I think AI already-- we call it machine learning, but the rest of the world seems to call it AI. You know, um, machine learning and, and AI and deep learning will change the world. And I hope that through 229, uh, we'll give you the tools you need so that you can be many of these future titans of industries, that you can be one to go out and build, you know, help the large tech companies do the amazing things they do, or build your own startup, or go into some other industry. Go, go transform healthcare, or go transform transportation, or go build a self-driving car, um, and do all of these things that, um, after this class, maybe you'll be able to do. You know, um, the majority of students applying-- The, the demand for AI skills, the demand for machine learning skills is so vast. I think you all know that. Um, and I think it's because machine learning has advanced so rapidly in the last few years that there are so many opportunities, um, to apply learning algorithms, right? Both in industry as well as in academia. I think today we have, um, the English department professors trying to apply learning algorithms to understand history better. Uh, we have lawyers trying to apply machine learning to process legal documents. Uh, and off-campus, every company, both the tech companies as well as a lot of, lot of companies that you wouldn't consider tech companies, everything from manufacturing companies to healthcare companies to logistics companies are also trying to apply machine learning. So I think that, um, uh, uh, if you look at it on a, on a factual basis, the number of people doing very valuable machine learning projects today is much greater than it was six months ago, and six months ago was much greater than it was twelve months ago. And the amount of value, the amount of exciting, meaningful work being done in machine learning is, is, is very strongly going up. Um, and I think that given the rise of, you know, the, the, the amount of data we have as well as the new machine learning tools that we have, um, it will be a long time before we run out of opportunities, you know, before, before society as a whole has enough people with a machine learning skill set. Um, so just as maybe, I don't know, twenty years ago was a good time to start working on this internet thing, and a lot of people that started working on the internet, like, twenty years ago have fantastic careers. I think today is a wonderful time to jump to machine learning, uh, and, and, and the number of-- and the opportunities for you to do unique things that no one is, no one else is doing, right? The opportunity for you to go to a logistics company and find an exciting way to apply machine learning, uh, will be very high because chances are that logistics company has no one else even working on this because, you know, they probably can't... They, they may not be able to hire a fantastic Stanford student as a graduate of CS229, right? Because there, there just aren't a lot of CS229 graduates around. Um, so what I want to do today is, um, do a quick intro, talk a little about logistics, um, and then, uh, we'll, we'll spend the second half of today, you know, giving an overview and, and talk a little bit more about machine learning. Okay? And, uh, oh, and I apologize. I, I think that, uh, this room, according to that sign there, seats, what? Three hundred something students? Uh, I th-- uh, we have, uh, uh, like, uh, not quite eight hundred people enrolled in this class. [laughing] Um, so there are people outside. And, and all, all of the classes are, are recorded, broadcast on SCPD. Uh, they usually-- the video is usually made ava-ava-available same day. So for those of you that can't get into the room, my apologies. Um, there, there were some years, um, where even I had trouble getting into the room. [laughing] But I'm glad they let me in. Uh, but, but I'm-- but, but hopefully you can watch-- you, you'll be able to watch all of these things online shortly. Oh, I see. Yes. Yeah. I don't know. Uh, it's a bit complicated. [laughing] Yeah. Uh, yeah. Thank you. I think it's okay. Yeah. I, I... We could... Yeah. Yeah. May-may-maybe for the next few classes, people can squeeze in and use up the MCC. So for now, it might be too complicated. Uh, okay. So quick intros. Um, oh, I'm sorry. I should have introduced myself. My name's Andrew Ng. Uh, uh, uh, and I want to introduce some of the rest of the teaching team as well.

2. 5:21 – 6:42

   Teaching team introductions

   1. ANAndrew Ng5:22

      Is the class coordinator. Um, she has been playing this role for many years now and helps, uh, keep the trains run on time and make sure that everything in class happens when it's supposed to. Uh, uh, so, so, so she'll be, uh... And then, uh, we're thrilled to have... Do you guys wanna stand up? Uh, be the co-head TAs. Uh, are respectively are PhD students working with me, uh, and so bring a lot of, um, uh- Uh, uh, technical experience, uh, technical experience in machine learning as well as practical know-how on how to actually make these things work. And with the large class that we have, we have a large TA team. Um, uh, maybe I won't introduce all of the TAs here today, but you meet many of them throughout this course here. But the TAs' expertise span everything from computer vision to natural language processing to computer technology to robotics. And so, um, through this quarter, as you work on your class projects, I hope that you get a lot of, uh, help and advice and mentoring from the TAs, uh, all of which, all of whom have deep expertise, not just in machine learning, but often in a specific vertical application area, um, of machine learning. So depending on what your project is, we try to match you to a TA that can give you advice, uh, uh, that most relevant to whatever project you end up working on. Um, so, you know, goal

3. 6:42 – 10:09

   Goals for the course and the state of machine learning across research and industry

   1. ANAndrew Ng6:42

      of this class. I hope that after the next ten weeks, uh, you will be an expert in machine learning. Um, it turns out that, uh, uh, you know, um... And, and I hope that after this class, you'll be able to go out and, uh, build very meaningful machine learning applications, uh, either in academic setting where, uh, hopefully you can apply it to your problems in mechanical engineering, electrical engineering and, uh, English and law and, um, uh, and, and, and education, all of this wonderful work that happens on campus. Uh, as well as after you graduate from Stanford, to be able apply it to whatever jobs you find. Um, one of the things I find very exciting about machine learning is that it's no longer a sort of pure tech company only kind of thing, right? I think that many years ago, um, machine learning, it was like a thing that, you know, computer science department would do and that the elite AI companies like Google and Facebook and Baidu and Microsoft would do. Uh, but now it is so pervasive that even companies that are not traditional sort of tech companies see a huge need to apply these tools. And I find a lot of the most exciting work, uh, these days, um... And, and, and maybe some of you guys know my history. So I'm a little bit biased, right? I, I led the Google Brain team, which helped Google transform from what was already a great company ten years ago to today, which is, you know, a great AI company. And then I also led the AI group at Baidu and, you know, led the company's technology and strategy to help Baidu also transform from what was already a great company many years ago to today arguably China's greatest AI company. So having led the, you know, built the teams that led the AI transformations of two large tech companies, I, I, I feel like that's a great thing to do. Uh, but even beyond tech, I think that, um, there's a lot of exciting work to do as well to help other industries, to help other sectors, uh, embrace machine learning and use these tools effectively. Um, but after this class, I hope that each one of you will be well qualified to get a job at a shiny tech company and do machine learning there, or go into one of these other industries and do very valuable machine learning projects there. Um, and in addition, if any of you, um, are taking this class with the primary goal of, uh, being able to do research, uh, in machine learning. You know, so, so actually, uh, some, some of you I know are PhD students. Um, I hope that this class will also leave you well equipped to, um, be able to read and understand research papers, uh, as well as, uh, you know, be qualified to start pushing forward, um, the state-of-the-art. Great. Um, so let's see. Um, so today, uh, uh, so, so just as machine learning is evolving rapidly, um, the whole teaching team, we've been, uh, constantly updating CS229 as well. So, um, it- it's actually very interesting. I feel like the pace of progress in machine learning has accelerated. So it, it, it actually feels like that, uh, the amount we change the class year over year has been increasing over time. So, so if you have friends that took the class last year, you know, things are a little bit different this year because we're, we're constantly updating the class to keep up with what feels like still accelerating progress in the whole field of machine learning. Um, so, so, um, so, so there are some logistical changes. For example, uh, uh, we've gone from, uh, we-

4. 10:09 – 11:53

   Prerequisites for the course

   1. ANAndrew Ng10:09

      we used to hand out paper copies of handouts. Uh, but we're, we're trying to make this class digital only. Um, but let me talk a little bit about, uh, prerequisites as well as in case your friends have taken this class before, some of the differences for this year, right? Um, so prerequisites. Um, we are going to assume that, um, all of you have, uh, knowledge of basic computer skills and principles. Uh, so, you know, Big O notation, queues, stacks, binary trees. Uh, hopefully you understand what all of those concepts are. And, uh, assume that all of you have a basic familiarity with, um, uh, probability, right? That hopefully you know what's a random variable, what's the expected value of a random variable, what's the variance of a random variable. Um, and if for some of you, maybe especially the SCPD students taking this remotely, if it's been, you know, some number of years since you last had a probability and statistics class, uh, we will have, uh, review sessions, uh, on, on, on Fridays, uh, where we'll go over some of this prerequisite material as well. But, but so hopefully, you know what a random variable is, what expected value is. But if you're a little bit fuzzy on those concepts, we'll, we'll go over them again, um, at a, at a discussion section, uh, on Friday. Um, also assume that you're familiar with basic linear algebra. So hopefully that you know what's a matrix, what's a vector, how to multiply two matrices or multiply a matrix and a vector. Um, if you know what is an eigenvector, then that's even better. Uh, if you're not quite sure what an eigenvector is, we'll go over it but, but you, you, you... But, uh, uh, yeah, we'll, we'll, we'll go over it, I guess. Uh, um- And then, um, a large part of this class, uh, uh, is, um, having you practice these ideas, uh, through the homeworks, uh, as well as I'll mention later,

5. 11:53 – 16:57

   Homework, and a note about the Stanford honor code

   1. ANAndrew Ng11:53

      a, uh, open-ended project. And so, um, one, uh, there-- we-we've actually, uh, until now, we used to use, uh, MATLAB, uh, and Octave for the programming assignments. Uh, but this year, we're trying to shift the programming assignments to, uh, Python. Um, and so, um, I think for a long time, uh, even today, you know, I sometimes use Octave to prototype because the syntax of Octave is so nice and just run, you know, very simple experiments very quickly. But I think the machine learning world, um, is, you know, really migrating, I think, from, um, MATLAB, Python world to increasing... Excuse me, MATLAB, Octave world to increasingly a Python maybe, and, and then eventually for production, Java, C++ kind of world. And so, uh, we're rewriting a lot of the assignments for this course this quarter, um, have been, have been driving that process, uh, so that, so that this course here you could do more of the assignments, uh, uh, maybe most- maybe all of the assignments in, um, Python, uh, NumPy instead. Um, now, a note on the honor codes, um, we ask that, you know, we, we, we actually encourage you to form study groups. Uh, so, so, you know, I've been, um, fascinated by education for a long time. Spent a long time studying education and pedagogy and how instructors like us can help support you to learn more efficiently. And one of the lessons I've learned from the educational research literature is that for highly technical classes like this, if you form study groups, uh, you will probably have an easier time, right? So, so CS229 will go through highly technical material. There's a lot of math. Some of the programs are hard. And if you have a group of friends to study with, uh, you probably have an easier time, uh, uh, because you can ask each other questions and work together to help each other. Um, where we ask you to draw the line or what we ask you to, to, to do relative to the Stanford's, uh, honor code is, um, we ask that you do the homework problems by yourself, right? Uh, and, and, and more specifically, um, it's okay to discuss the homework problems with friends. But if you, um, but after discussing homework problems with friends, we ask you to go back and write up the solutions by yourself, uh, without referring to notes that, you know, you and your friends had developed together. Okay? Um, the class's honor code is written clearly on the class, um, handouts posted digitally on the website. So if you ever have any questions about what is allowed collaboration and what isn't allowed, uh, please refer to that written document on the course website where we describe this more clearly. But, um, out of respect for the Stanford honor code, as well as for, uh, uh, you know, for, for, for students kind of doing their own work, we ask you to basically do your own work, uh, for the... Um, it's okay to discuss it, but after discussing homework problems with friends, ultimately, we ask you to write up your problems by yourself so that the homework submissions ref-reflect your own work, right? Um, and I care about this because it turns out that, uh, having CS229, you know, CS229 is one of those classes that employers recognize. Uh, uh, I don't know if you guys know, but there have been, um, companies that have put up job ads that say stuff like, "So long as you got-- So long as you completed CS229, we guarantee you get an interview," right? I-I-I've seen stuff like that.

   2. SPSpeaker15:09

      [laughing]

   3. ANAndrew Ng15:09

      And so I think, you know, in order to, to maintain that sanctity of what it means to be a CS229 completer, I think, um, and I ask that all of you sort of really do your own work, um, uh, or stay within the bounds of accepted of, accepted collaboration relative to the honor codes. Um, let's see. And I think that, um, uh, if, uh... You know what? This is, um... Yeah. And I think that, uh, one of the best parts of CS229, it turns out, is, um... Excuse me. So try... Sorry, I'm gonna try looking for my mouse cursor. Uh, all right. Sorry about that. My, my, my displays are not mirrored, so this is a little bit awkward. Um, so one of the best parts of the class is... Oh, shoot. Sorry about that. All right. Never mind. I won't do this. Um, you could do on- you could do it yourself online later. Um, yeah. I started using, I started using Firefox recently in addition to Chrome. Anyway, just, just to mix it up. Um, one of the best parts of, um, the class is, um, the class project.

6. 16:57 – 25:57

   Overview of the class project

   1. ANAndrew Ng16:57

      Um, and so, you know, one of the goals of the class is to leave you well-qualified to do a meaningful machine learning project. And so, uh, one of the best ways to make sure you have that skill set is through this class and hopefully with the help of some of our TAs. Uh, we wanna support you To work on a small group to complete a meaningful machine learning project. Um, and so one thing I hope you start doing, you know, later today, uh, is to start brainstorming, maybe with your friends, um, some of the, some of the class projects you might work on. Uh, and the most common class project that, you know, people do in CS229 is to pick a area, pick an application that excites you and to apply machine learning to it and see if you can build a good machine learning system for some application area. And so, um, if you go to the course website, you know, cs229.stanford.edu, and look at previous years' projects, you, you see machine learning projects applied to pretty much, you know, pretty much every imaginable application under the sun. Everything from, I don't know, diagnosing cancer to creating art to, uh, lots of, um, uh, projects applied to other areas of engineering, uh, applying to application areas in EE or mechanical engineering or civil engineering or earthquake engineering and so on, uh, to applying it to understand literature, to applying it to, um, uh, I don't know. And, and, and, and so, uh, if you look at the previous years' projects, uh, many of which are posted on the course website, you can use that as inspiration to see the types of projects students complete-completing this class are able to do. And I also encourage you to, um, uh, you can look at that for inspiration, you know, to, to get a sense of what you'll be able to do at the conclusion of this class, and also see if, uh, looking at previous years' projects gives you inspiration for what, um, you might do yourself. Uh, so we ask you to-- we, we invite you, I guess, to do class projects in small groups. And so, um, after class today, also encourage you to start making friends in the class, both for the purpose of forming study groups as well as for the purpose of maybe finding a small group to do a class project with. Um, uh, we ask you to form project groups of, um, up to size three. Uh, uh, most project groups end up being size two or three. Um, if you insist on doing it by yourself, right, without any partners, that's actually okay, too. You're welcome to do that. But, uh, but, but I think often, you know, having one or two others to work with may give you an easier time. And, uh, for projects of exceptional scope, if you have a very, very large project that just cannot be done by three people, um, uh, sometimes, you know, let us know and we're open to, uh, w-with, with to some project groups of size four. But our expectation-- But we do hold projects, you know, with a group of four to a higher standard than projects with size one to three. Right? So, so what that means is that if your project team size is, uh, one, two, or three persons, the grading is one criteria. If your project group is, uh, bigger than three persons, we use a stricter criteria when it comes to grading class projects. Okay? Um, oh, and that, that, that reminds me. Um, uh, I know that, uh, let's see. So for most of you, since this, since this starts at nine thirty AM on the first day of the quarter, uh, for many of you, this may be-- this is probably your very first class at Stanford. For how many of you, this is your very first class at Stanford? Wow, cool. Okay, awesome. Great. Welcome to Stanford. Uh, and there's someone next to you just raised their hand. Uh, actually, rai-raise your hand again. So I hope that, you know, maybe after class today, if someone next to you raised their hand, uh, help welcome them to Stanford and then say hi, introduce yourself and make friends afterward. Yeah. Cool. Nice, nice to see so many of you here. Um, all right. So, um, just a bit more on logistics. Uh, so, um, let's see. In addition to the main lectures that we'll have here, uh, on Mondays and Wednesdays, um, CS229 also has discussion sections, uh, on-- held on Fridays that are-- And everything we do, including the-- So all the, all the lectures and discussion sections are recorded and broadcast through SCPD, uh, through the online website. Um, and one of-- And, uh, discussion section, uh, are taught, uh, usually by the TAs on Fridays, and attendance at discussion sections is optional. Uh, and what, what I mean is that, um, you, you know, you hundred percent promise there won't be material on the midterm that will sneak in from the discussion section, so it's a hundred percent optional. Uh, and you will be able to do all the homework and the pro-projects without attending the discussion section. But what we'll use the discussion section for, uh, for the first three discussion sections, so, you know, this week, next week, uh, the week after that, we'll use the discussion sections to go over prerequisite material in greater depth. So, uh, go over, uh, linear algebra, basic problem statistics, uh, teach a little bit about Python NumPy in case you're less familiar with those programming frameworks. Uh, so do that for the first few weeks. And then for the discussion sections that are held later this quarter, we'll usually use them to go over more advanced optional material. Uh, for example, um, CS229, a lot of the learning algorithms you, you, you hear about in a class rely on convex optimization algorithms, but we want to focus the class on the learning algorithms and spend less time on convex optimization. So if you want to come and hear about more advanced concepts in convex optimization, we'll defer that to discussion section. Uh, and then there, there are a few other, uh, advanced topics, uh, hidden Markov models, time series, uh, that we're planning to defer to the, um, Friday discussion sections. Okay? Um, so, uh, let's see. Um, cool. And, uh, oh, and, um, uh, final bit of logistics, um, uh, for there'll be-- there are digital tools that some of you will have seen, but, um, for this class, we'll drive a lot of the discussions through the, uh, online website Piazza. How, how many of you have used Piazza before? Okay, cool. Most of-- Wow, all of you? That's pretty amazing. Um, good. So, so, uh, online discussion board for those of you that haven't seen it before, but, um, definitely encourage you to, uh, participate actively on Piazza and also to answer other students' questions. Right? I think that one of the best ways to learn, uh, as well as contribute, you know, back to the course as a whole is if you see someone else ask a question on Piazza, if you jump in and help answer that, uh, uh, that, that often helps you and helps your classmates. So I strongly encourage you to do that. Uh, for those of you that have a private question, you know, sometimes we have students, um, uh, reach out to us to w-w-with a personal matter, uh, or something that, you know, is not appropriate to share on a public forum, in which case you're welcome to email us at the class email address as well. Uh, and we also-- And, and the class email address, the cl- teaching staff's email address is on the course website. You can find it there and contact us. But for anything technical or anything reasonable to share with the class, uh, which includes most technical questions and most logistical questions, right? Questions like, you know, "Gee, can you confirm what date is the midterm?" Or, or, you know, "What happens" uh, uh, "Can, can you confirm when's the handout for this going out?" and so on. For questions that are not personal or private in nature, I strongly encourage you to post on Piazza rather than emailing us because, uh, statistically you actually get a faster answer, uh, posting it on post-posting on Piazza than, than, you know, if, if you wait for one of us to respond to you. Um, and we'll be using Gradescope as well, um, uh, to-- for, for online grading. Right? And, and if you, if you don't know what Gradescope is, don't worry about it. We'll, we'll, we'll send you links and show you how to use it later. Um, oh, and, uh, uh, again, relative to... One, one last logistical thing to plan for, um, com- unlike, uh, previous, um, uh, years where we taught CS229, uh, so we're constantly updating the syllabus, right? The technical content to try to show you the latest machine learning algorithms. Uh, and, uh, uh, the two big logistical changes we're making this year, I guess one is, uh, Python instead of MATLAB, and the other one is, uh, um, instead of having a, uh, a midterm exam, you know, that's a timed midterm, uh, we're planning to, uh, have a take-home midterm, uh, this quarter instead. So I, I don't know. Some, some people just breathed in sharply when I said that. I don't know what that means. Was that shock or happiness? I don't know. Okay. Don't worry. Midterms are fun. You, you, you, you'll love it. [laughing] All right. Um, so that's it for the, that's it for the logistical aspects. Um, let me check with the... And so let, let me check if there are any

7. 25:57 – 1:15:19

   Questions

   1. ANAndrew Ng25:57

      questions. Oh, yeah. Go ahead.

   2. SPSpeaker26:04

      Uh, on campus, this course is offered every quarter. Are all the renditions identical?

   3. ANAndrew Ng26:10

      Oh, um-

   4. SPSpeaker26:11

      I think so. It mentions the spring quarter with Chris Re.

   5. ANAndrew Ng26:15

      Yeah. So that's interesting. Uh, let's see. I think it's offered in spring, uh, and one other person. Uh, uh, yes, and are teaching it. So someone else is teaching it in spring quarter. Um, uh, I actually did not know it was gonna be offered in winter.

   6. SPSpeaker26:34

      No, not this quarter.

   7. ANAndrew Ng26:34

      Yeah.

   8. SPSpeaker26:34

      I don't believe it's offered in winter.

   9. ANAndrew Ng26:35

      Yeah. Right. Yeah.

   10. SPSpeaker26:39

       Yes.

   11. ANAndrew Ng26:40

       Yeah. So, so I think, like, great guy and teaching it, uh, in... Sorry, Uh, we actually capped the enrollment of CS229A at a relatively low number, at 100 students, so I actually don't wanna encourage too many of you to sign up because, uh, uh, I think we might be hitting the enrollment cap already. So, so please don't all sign up for CS229A because, um, uh, we-- T-Two-Twenty-Nine A does not have the capacity this quarter. But, uh, CS229A is a, um, much less mathematical and, uh, much more applied, uh, uh, uh, a relatively more applied version of machine learning. And, uh, so I, I guess, uh, I'm teaching CS229A, CS230, and CS229 this quarter. Of the three, CS229 is the most mathematical. Um, it is a little bit less applied than CS229A, which is more applied machine learning, and CS230, which is deep learning. My advice to students is that, um, CS229, uh, CS229A is... Let me write this down. I think I'm... Right. So CS229A, uh, is taught in a flipped classroom format, which means that, uh, students taking it will mainly watch videos, um, on the Coursera website and do a lot of, uh, programming exercises, and then meet for weekly discussion sections. Uh, but it's a smaller class with capped enrollment. Um, I, I would advise you that, um, if you feel ready for CS229 and CS230 to do those, uh, but, uh, CS229, you know, because of the math we do, this is a, this is a very heavy workload and pretty challenging class. And so if you're not sure if you're ready for CS229, uh, CS229A may be a good thing to, to, to take first. Um, uh, and then, um, CS229 and CS229A cover a broader range of machine learning algorithms, uh, and CS230 is more focused on deep learning algorithms specifically, right? Which is a much narrower set of algorithms, but it is, you know, one of the hottest areas of deep learning. Uh, there is not that much overlap in content between the three classes, so if you actually take all three, you learn relatively different things from all of them. Uh, in the past, we've had students simultaneously take Two-Twenty-Nine and Two-Twenty-Nine A, and there is a little bit of overlap. You know, they, they do kind of cover related algorithms, but from different points of view. So, so some people actually take multiple of these classes at the same time. Um, uh, but Two-Twenty-Nine A is more applied, a bit more, you know, practical know-how, hands-on and so on, and, uh, and, and, uh, uh, much less mathematical. Uh, and, and CS230 is also less mathematical and more applied, more about kind of getting it to work, whereas CS229A, um, we do much more mathematical derivations in CS229. Cool. Any other questions? Uh, yes. Uh, someone had a hand up.

   12. SPSpeaker32:20

       So my question was about Two-Twenty-Nine, Two-Thirty, but then I guess, um, well, my next question is, am I permitted to ask a question in class?

   13. ANAndrew Ng32:23

       So, uh, uh, why don't you say the-- What I would generally prefer students not do that in the interest of time, but what, what do you want?

   14. SPSpeaker32:33

       I just want to know how many people in here were enrolled in both Two-Twenty-Nine and Two-Thirty.

   15. ANAndrew Ng32:37

       Oh, I see. Sure, go for it. Who's enrolled in Two-Twenty-Nine, Two-Thirty? I... Oh, not that many of you. Interesting. Oh, that's actually really interesting. Cool. Yeah. Thank you. Yeah. I, I just didn't wanna set a precedent of students using this as a forum to run surveys. So that was, that was a, that was a, that was an interesting question, so thank you. [laughs] Yeah. Um, cool. All right. Uh, and by the way, I think, uh, you know, just one, one thing about Stanford is, um, the AI world, the machine learning world... AI is bigger than machine learning, right? And machine learning is bigger than deep learning. Um, one of the great things about being a Stanford student is you can, and I think should, take multiple classes, right? I think that, you know, CS229 has, for many years, been the core of the machine learning world at Stanford. Uh, but even beyond CS229, it's worth your while to take multiple classes and gain multiple perspectives. So, so if you want to, uh, be really effective, you know, after you graduate from Stanford, you do wanna be an expert in machine learning, you do wanna be an expert in deep learning, uh, and you probably wanna know probability and statistics, maybe you wanna know a bit of convex optimization, maybe you wanna know a bit more about reinforcement learning, know a little bit about planning, know a bit about lots of things. So, so I, I, I actually encourage you to, to take multiple classes, I guess. Cool. All right. Good. Um, if there are no more questions, let's go on to talk a bit about, um, machine learning. Okay. So, um, all right. So in the remainder of this class, what I'd like to do is, um, give a quick overview of, uh, you know, the major, uh, uh, areas of machine learning and also, um, and, and, and also give you a sort of overview of the things you'll learn, uh, in the next ten weeks. So, you know, what, what is machine learning, right? It seems to be everywhere these days, and it's useful for so many places and, and I think that, um- Uh, and, and, you know, and, and, uh, uh, and I, I, I feel like, uh, by the way, just to share with you my personal bias, right? You, you, you read the news about these people making so much money building learning algorithms. I think that's great. I hope, I hope all of you go make a lot of money. But the thing I find even more exciting is, is the meaningful work we could do, right? I think that, you know, I think that every time there's a major technological disruption, which there is now through machine learning, um, it gives us an opportunity to remake large parts of the world. And if we behave ethically in a principled way and use these superpowers of machine learning to do things that, you know, helps people's lives, right? Maybe we could, um, uh, maybe you could improve the healthcare system. Maybe we can improve, give every child a personalized tutor. Uh, maybe we can make our democracy run better rather than make it run worse. But I think that, um, the meaning I find in machine learning is that there's so many people that are so eager for us to go in and help them with these tools that, um, it, it-- if, if you become good at these tools, it gives you an opportunity to really remake some piece, some meaningful piece of the world, uh, hopefully in a way that helps other people and makes the world kind of... Makes the world a better place is very clichéd in Silicon Valley. But, but I think, you know, with these tools, you actually have the power to do that. And if you go make a ton of money, that's great too. But I find a much greater meaning in the work we could do. Um, uh, it gives us a unique opportunity to do these things. But, um, despite all the excitement about machine learning, what is machine learning? So let me give you a couple, um, definitions of machine learning. Um, Arthur Samuel, whose claim to fame was, uh, building a checkers playing program, uh, defined it as follows. Uh, "The field of study that gives computers the ability to learn without being explicitly programmed." Um, and, uh, you know, interesting, when, when Arthur Samuel, many, many decades ago, wrote a checkers playing program, uh, the debate of the day was, uh, can a computer ever do something that it wasn't explicitly told to do? And Arthur Samuel, uh, wrote, um, a checkers playing program that through self-play learned what are the patterns of a checkerboard that are more likely to lead to a win versus more likely to lead to a loss and learn to be even better than Arthur Samuel, the author himself, at playing checkers. So back then, this was viewed as a remarkable result that a computer programmer, you know, that, uh, could write a piece of software to do something that the computer programmer himself could not do, right? Because this program became better at Arthur Samuel, um, at, uh, uh, uh, at, uh, at, at the task of playing checkers. Um, and I think today we, um, are used to computers or machine learning algorithms outperforming humans on so many tasks. Uh, uh, but it turns out that when you choose a narrow task, like speech recognition on a certain type of task, you can maybe surpass human level performance. If you choose a narrow task like playing the game of Go, then by throwing really tons of computation power at it and self-play, uh, uh, uh, you can have a computer, you know, become very good at, at these narrow tasks. But this was maybe one of the first such examples in the history of computing. Um, uh, and I think this is still one of the most widely cited, um, uh, definitions, right? Uh, gives computers the ability to learn without being explicitly programmed. Um, my friend Tom Mitchell, in his textbook, uh, defined this as a well-posed learning problem. Uh, program said to learn from experience E with respect to task T on some performance measure P if performance on T as measured by P improves with experience E. You know, and I, I, I asked Tom this. I asked Tom if, um, he wrote this definition just because he wanted it to rhyme. And, and he, he- [laughing] ... he, he, he did not say yes, but I, I, I, I, I don't know. Um, but in this definition, the experience E, uh, for, for the case of playing checkers, the experience E would be the experience of having the checkers play program play tons of games against itself. Uh, so, you know, computer has lots of patience. It sit there for days playing games of checkers against itself. So that's experience E. Uh, the task T is the task of playing checkers, and the performance measure P may be, um, what's the chance of this program winning the next game of checkers it plays against the next opponent, right? So, so we say that, uh, this is a well-posed learning problem, learning to play checkers. Now, within this, um, set of ideas of machine learning, there are many different tools we use in machine learning. And so in the next ten weeks, you'll learn about a variety of these different tools. Um, and so the first of them, and the most widely used one, is supervised learning. Um, let's see. I wanna switch to the whiteboard. Do you guys know how do I erase the screen? Oh, like that. Okay. All right, cool. So what I wanna do today is really go over some of the major categories of, uh, machine learning tools and, uh, and, and some of what you learn in the next, um, uh, by the end of this quarter. So the most widely used machine learning tool is, um, today is supervised learning. Actually, let me check. How, how many of you know what supervised learning is? Uh, like, uh, two-thirds, half of you, maybe. Okay, cool. Let me, let me just briefly define it. Um, here's one example. Let's say you have a database of, uh, housing prices. And so I'm gonna plot your dataset where on the horizontal axis, um, I'm gonna plot the size of the house in square feet. You know, and on the vertical axis, we'll plot the price of the house. Right? And, um, maybe a data set looks like that. Um, and so horizontal axis, I guess we'll call this X, and vertical axis, we'll call that Y. So, um, the supervised learning problem is, given the data set like this, to find the relationship mapping from X to Y. Right? And so, um, for example, let's say you have-- let's say, uh, l-let's say you have a h-- Let's say you're fortunate enough to own a house in Palo Alto, right? [laughing] Uh, uh, and you're trying to sell it, and you want to know how to price the house. So maybe your house has a size, you know, of that amount on the horizontal axis. I don't know. Maybe this is, uh, what, five hundred square feet, one thousand square feet, fifteen hundred square feet. So your house is, uh, twelve fifty square feet, right? And you want to know, you know, how do you price this house. So given this data set, one thing you can do is, um, uh, fit a straight line to it, right? And then you could estimate or predict the price to be whatever value you read off on the, um, vertical axis. So in supervised learning, you are given a data set with, uh, inputs X and labels Y, and your goal is to learn a mapping from X to Y. Right? Now, um, fitting a straight line to data is maybe the simplest possible, maybe the simplest possible learning algorithm, maybe the, uh, one of the simplest poss-- learning algorithms. Um, given a data set like this, there are many possible ways to learn a mapping, to learn a function mapping from the input size to the estimated price. And so, um, maybe you wanna fit a quadratic function instead. Maybe that actually fits the data a little bit better. And so how do you choose among different models will be, uh, either automatically or manual intervention will be, will be something we'll spend a lot of time talking about. Now, to give a little bit more, um, to define a few more things, this particular example is a problem called a regression problem. And the term regression refers to that the, uh, value y you're trying to predict is continuous, right? Um, in contrast, here is a c-- here's a different type of problem. Um, so a problem that some of my friends were working on, and, and I'll simplify it, was, was a healthcare problem where, uh, they were looking at, uh, breast cancer, uh, breast tumors, um, and trying to decide if a tumor is benign or malignant. Right? So a tumor, you know, so a lump in a, in a woman's breast, um, is, uh, uh, can be m-uh, uh, malign or cancerous, um, or benign, meaning, you know, roughly that it's not that harmful. And so if on the horizontal axis you plot the size of a tumor, um, and on the vertical axis you plot is it malignant or not. So malignant means harmful, right? Um, and some tumors are harmful, some are not. And so whether it's malignant or not takes only two values, one or zero. And so you may have a data set, um, like that. Right? Uh, and given this, can you learn a mapping from X to Y, so that if a new patient walks into your office, uh, walks into the doctor's office and their tumor size is, you know, say, this. Can the learning algorithm figure out from this data that, you know, it's probably, well, based on this data set, looks like there's a, there's a high chance that that tumor is, um, malignant. Um, so, uh, so this is an example of a classification problem. And the term classification refers to that y here takes on a discrete number of variables. Right? So for a regression problem, y is a real number. I guess technically prices can be rounded off to the nearest dollar and cent, so prices aren't really real numbers. Uh, uh, you know, but because you would probably not price a house at, like, pi times one million or whatever. Uh, but so, so, but, but, but for all practical purposes, prices are continuous, so we call them housing price prediction to be a regression problem. Whereas if you have, uh, two values of possible output, zero, one, we call it a classification problem. Um, if you have k discrete outputs, so, uh, if the tumor can be, um, malignant, uh, or if there are five types of cancer, right? So you have one of five possible outputs, then that's also a classification problem, if the output is discrete. Now, um, I wanna find a different way to visualize this data set, which is, um, let me draw a line on top. And I'm just gonna, you know, map all this data on the horizontal axis upward onto a line. Um, but, well, let me just show you what I'm gonna do. I'm going to use a symbol O to denote... Right? Um, I hope what I did was clear. So I took the two sets of examples, uh, the positive and negative examples. Positive examples is one, negative examples is zero. And I took all of these examples and, and kinda pushed them up onto a straight line. And I used two symbols. I used Os to denote negative examples, and I used crosses to denote positive examples. Okay? So this is just a different way of visualizing the same data, um, but drawing it on a line and using, you know, two symbols to denote the two discrete values, zero and one. Right? So, um, it turns out that, uh- Uh, in both of these examples, the input X was one-dimensional. It was a single real number. For most of the, um, machine learning applications you work with, the input X will be multidimensional. You won't be given just one number and asked to predict another number. Instead, you'll often be given, uh, multiple features or multiple numbers to predict another number. So for example, instead of just using, uh, tumor size to predict-- to estimate malignancy of a ma-malignant versus benign tumors, um, you may instead have two features where one is tumor size and the second is age of the patient and be given a dataset. Maybe I'll, right, and be given a dataset that looks like that, right? Where now your task is, um, given two input features, so X is tumor size and age, you know, like a two-dimensional vector. Um, and your task is given, uh, these two input features, um, to predict whether a given tumor is malignant or benign. So if a new patient walks in a doctor's office, and if the tumor size is here and the age is here, so that point there, then hopefully you can conclude that, you know, this patient's tumor is probably benign, right? Corresponding to O, the, the negative example. Um, and so one thing, one thing you learn, uh, next week is a learning algorithm that can fit a straight line to the data as follows, kind of like that, to separate out the positive and negative examples, separate out the O's and the crosses. And so next week you'll learn about the logistic regression algorithm, which, um, which can do that. Okay? So, um, one of the most interesting things you learn about is, uh-- Let's see. So in this example, I drew a dataset with two input features. Um, when-- So I have friends that actually worked on the breast cancer, uh, prediction problem. And in practice, you usually have a lot more than one or two features. And usually you have so many features you can't plot them on the board, right? And so for an actual breast cancer prediction problem, my friends are working on this, we're, we're using many other features such as... Don't worry about what these me-mean. I guess clump thickness, uh, you know, uniformity of cell size, uniformity of cell shape, right? Um, uh, adhesion, how well the cells stick together. Don't worry about what this means. But, uh, if you're actually doing this in a, in a actual medical application, there's a good chance that you'll be using a lot more features than just two. Uh, and this means that you actually can't plot this data, right? It's too high-dimensional. You can't plot things higher than three-dimensional, maybe four-dimensional or something, right? And so when you have a lot of features, it's actually difficult to plot this data. I'll come back to this in a second in learning theory. Um, and, uh, one of the things you learn about-- So as we develop learning algorithms, you learn how to build, um, regression algorithms or classification algorithms that can deal with these, uh, relatively larger number of features. One of the, uh, most fascinating results you learn is that, um, you also learn about an algorithm called the support vector machine, which uses not one or two or three or ten or a hundred or a million input features, but uses an infinite number of input features, right? And so, so, so just to be clear, if in this example, the state of a patient were re-represented as one number, you know, tumor size. Uh, in this example, you had two features. So the state of a patient was represented using two numbers, the tumor size and the age. If you use this list of features, maybe a s- patient is represented with five or six numbers. Uh, but there's an algorithm called the support vector machine that allows you to use an infinite dimensional vector, um, to represent a patient. And, um, how do you deal with that and how can a computer even store an infinite dimensional vector, right? I mean, you know, computer memory, you can store one real number, two real numbers, but you can't store an infinite number of real numbers in a computer without running out of memory or process speed or whatever. So, so how do you do that? Uh, so when we talk about support vector machines, uh, and specifically the technical method called kernels, you learn how to build learning algorithms that work with, uh, sort of an infinitely long list of features, infinitely long list of feature, uh, for, for which, which-- And you can imagine that if you have an infinitely long list of numbers to represent a patient, that might give you a lot of information about that patient. And so that is one of the relatively effective learning algorithms of some problems. Okay? Um, so that's supervised learning. And, you know, let me just, um, uh, play a video, um, show you a fun, slightly older example of supervised learning and give you a sense of what this means. But at the heart of supervised learning is the idea that during training, uh, you are given inputs X together with the labels Y, and you're given both at the same time. And the job of your learning algorithm is to, uh, find a mapping so that given a new X, you can map it to the most appropriate output Y. Um, so this is a very old video, uh, made by, um, uh, Dean Pomerleau, known for a long time as well, on using supervised learning for autonomous driving. Uh, this is not state-of-the-art for autonomous driving anymore, but it, it actually does remarkably well. Oh, and, uh, um, as you, uh, you hear a few technical terms like back propagation, you learn all those techniques in this class. Uh, and by the end of the class, you'll be able to build a learning algorithm much more effective than what you see here. But let's, let's, let's see this application. Oh, uh, could you turn up the volume if you have that? Are you guys getting volume? Oh, I see. All right. I'll narrate this. Uh, [laughs] so I'll be using artificial neural network to drive this vehicle that, uh, was built at Carnegie Mellon University, uh, many years ago. And what happens is, uh, during training, it watches the human, um, drive the vehicle, and I think ten times a second, uh, it digitizes the image in front of the vehicle. And, um, so that's a picture taken by a front-facing camera. Um, and what it does is in order to collect label data, the car, while the human is driving it, records both the image, such as you're seeing here, as well as the steering direction that was chosen by a human. So at the bottom here is the image turned into grayscale and low res, and, uh, on top... Let me pause this for a second. Um, this is the driver direction. The font's kinda blurry, but this text says driver direction. So this is the y label, the label y, that the human driver chose. Um, and so the position of this white bar, of this white blob, shows how the human is choosing to steer the car. So in this, in this image, the white blob is a little bit to the left of center, so the human is, you know, steering just a little bit to the left. Um, this second line here is the output of the neural network. And initially, the neural network doesn't know how to drive, and so it's just outputting this white schmear everywhere. It's saying, "Oh, I don't know. Do I drive left, right, center? I don't know." So it's kinda outputting this gray blur everywhere. Um, and as the algorithm learns using the back propagation learning algorithm or gradient descent, which you'll learn about, uh, you actually learn about gradient descent this Wednesday. Um, you see that the neural network's outputs becomes less and less of this white schmear, this white blur, but starts to become sharper, um, and starts to mimic more accurately the human-selected driving direction. Right? So this, um, is an example of supervised learning because the human driver demonstrates inputs x and outputs y, uh, namely, if you see this in front of the car, steer like that. So that's x and y. And, uh, after the learning algorithm has learned, um, you can then, uh, well, he pushes a button, takes his hand off the steering wheel, um, and then it's using this neural network to drive itself, right? Digitizing the image in front of the road, taking this image, and passing it through the learning algorithm, through the trained neural network, letting the neural network select the steering direction, uh, and then using a little motor to turn the wheel. Um, this is a slightly more advanced version which has trained two separate models, one for a, I think a two-lane road, one for a four-lane road. Uh, so that's the, um, uh, so the second and third lines, this is for a two-lane road, this is a four-lane road, and the arbitrator is, is another algorithm that tries to decide whether the two-lane or the four-lane road model is the more, more appropriate one for a particular given situation. Um, and so as Alvin is driving-- Uh, excuse me, a one-lane road or, uh, a two-lane road. So it says driving from a one-lane road here, uh, toward an intersection. Um, the, uh, the, the algorithm realizes it should switch over from, um, I think, I forget, the, I think the one-lane neural network to the, to the two-lane neural network. The one we'll see is there, right? All right. Um, okay. Oh, oh, all right, fine. We just see the final dramatic moment of switching from a one-lane road to a two-lane road. All right. Um, uh, and I think, you know, so this is just using supervised learning to take as input what's in front of your car to decide on the steering direction. This is not state-of-the-art for how self-driving cars are built today, but, you know, it could do some things in some limited context. Uh, uh, and I think, uh, w- in, in, in several weeks, you'll actually be able to build something that is more sophisticated than this. Right. Um, so after supervised learning, uh, we w- will in this class spend a bit of time talking about machine learning strategy or some- uh, or I think on the class notes, we, um, annotate this as, uh, learning theory. But what that means is, um, I wanna give you the tools to go out and apply learning algorithms effectively. And I think I've been fortunate to have, uh, you know, to know a lot of, uh, uh, uh... I, I, I think that, um, I've been fortunate to have, you know, over the years constantly visited lots of great tech companies, uh, uh, more than ones that I've, that, that I've been publicly associated with, right? But often, just to help friends out, I visit various tech companies, uh, of the sort whose products I'm sure are installed on your cell phone. Uh, but I often visit tech companies and, you know, talk to the machine learning teams and see what they're doing and see if I can help them out. And what I see is that there's a huge difference in the effectiveness of how two different teams could apply the exact same learning algorithm. Right? Uh, and I think that, um, what I've seen sadly is that sometimes there will be a team, uh, even in some of the best tech companies, right? The, the, the, the, the elite AI companies, right? In, in, in multiple of them, where you go talk to a team and they'll tell you about something they've been working on for six months, and then you can quickly take a look at the data and, and hear that they're not-- the algorithm isn't quite working. And sometimes you can look at what they're doing and go, "Yeah, you know, I could have told you six months ago that this approach was never gonna work." Right? Um, and, um, what I find is that the most skilled machine learning practitioners are very strategic, by which I mean that your skill at deciding, um... When you work on a machine learning project, you, you, you have a lot of decisions to make, right? Do you collect more data? Do you try a different learning algorithm? Uh, do you run faster GPUs to train your learning algorithm for longer? Or if you collect more data, what type of data do you collect? Or for all of these architecture choices, use neural networks, support vector machine, logistic regression, which one do you pick? Um, but there are a lot of decisions you need to make when building these learning algorithms. So one thing that's quite unique to the way we teach is, uh, we want to help you become more systematic in driving machine learning as a, as a systematic engineering discipline so that when one day you are working on a machine learning project, you can efficiently figure out what to do next, right? Um, and I sometimes make an analogy to how, um, to, uh, uh, to, to, to software engineering. Um, you know, I-- many years ago, I had a friend, um, that would debug code by compiling it and then, um, uh, this friend would look at all the syntax errors, right? That, you know, C++ compiler outputs. And they thought that the best way to eliminate the errors is to delete all the lines of code with syntax errors, and that was their first serious thing.

   16. SPSpeaker1:01:03

       [laughing]

   17. ANAndrew Ng1:01:03

       So that did not go well, right? Um, uh, uh, it took me a while to persuade them to stop doing that. Uh, but, but, but, but so it turns out that, um, when you run a learning algorithm, you know, it almost never works the first time, right? That's, that's just life. Uh, uh, and, and, and the way you go about debugging the learning algorithm will have a huge impact on your efficiency on, on, on how quickly you can build effective learning systems. And I think until now, too much of the-- of this process of, uh, making your learning algorithms work well has been a black magic kinda process where, you know, has worked on this for decades. So when you run something, you don't know why it's not working, you go, "Hey, what do I do?" And Ron says, "Oh yeah, do that." And then, and then because he's so experienced, it works. But I think, um, what we're trying to do with the discipline of machine learning is to evolve it from a black magic, tribal knowledge, experience-based thing to a systematic engineering process, right? And so, um, later this quarter, as we talk about machine learning strategy or talk about learning theory, we try to systematically give you tools on how to, um, uh, go about strategizing, uh, so, so you can be very efficient in, um, how you, how you yourself or how you can lead a team to build an effective learning system. 'Cause I, I don't want you to be one of those people that, you know, waste six months on some direction that maybe could have relatively quickly figured out was not promising. Or maybe one last analogy. If you, um, if you're used to optimizing code, right? Making code run faster, not sure how many of you have done that. Uh, uh, uh, less experienced software engineers will just dive in and optimize the code to try to make it run faster, right? Let's take the C++ and code an assembly or something. But more experienced people will run a profiler to try to figure out what part of your code is actually the bottleneck and then just focus on changing on that. So, uh, one of the things I hope to do this quarter is, uh, uh, uh, convey to you some of these more systematic engineering principles, right? And yeah. Oh, and actually, if any of you are interested, uh, this is, uh, uh, uh, yeah, actually, I've been, um, I've been, I've been writ- so actually, so, uh, how many of you have heard of machine learning journey? Oh, just a few of you. Interesting. Uh, so actually to, to, if any of you are interested, um, just in my, uh, spare time, uh, I've been writing a book, um, uh, to try to codify systematic engineering principles for machine learning. And so if you, uh, uh, and, and so, uh, if you want a, you know, free draft copy of the book, sign up for a mailing list here. I tend to just write stuff and put it on the internet for free, right? So if you want a free draft copy of the book, uh, uh, uh, uh, you know, go to this website, uh, enter your email address, and the website will send you a copy of the book. That'll talk a little bit about these engineering principles as well. Okay. All right. So, uh, so first subject, machine learning. Second subject, learning theory. Um, and, uh, the third major subject we'll talk about is, uh, deep learning, right? And so, you know, there are a lot of tools in machine learning, and many of them are worth learning about, and I use many different tools in machine learning, you know, for many different applications. There's one subset of machine learning that's really hot right now because it's just advancing very rapidly, which is deep learning. And so we'll spend a bit of time talking about deep learning so that you can understand the basics of, uh, how to train a neural network as well, right? Uh, but I think that, um, whereas 229 covers a much broader set of algorithms, which are all useful, uh, CS230 more narrowly covers just deep learning, right? Um, so, uh, other than deep learning slash-- after, after deep learning slash neur- neural networks, the other, the, the four or fourth of the five major topics we'll cover will be, um, unsupervised learning. Um, so what is unsupervised learning? So you saw me draw a picture like this just now, right? And this would be a classification problem, like the, uh, tumor, malignant, benign problem. This is a classification problem. And that was a supervised learning problem because you had to learn a function mapping from X to Y. Um, unsupervised learning would be if I give you a data set like this with no labels, so you're just given inputs X and no Y, and you're asked to find me something interesting in this data or figure out, you know, interesting structure in this data. Um, and so in this data set, it looks like there are two clusters, and an unsupervised learning algorithm, which you learned about, called k-means clustering, will discover this, um, this structure in the data. Um, other examples of unsupervised learning, you know, if, if you... Actually, Google News is a, is a very interesting website. Sometimes I use it to look up, right, latest news. Uh, this is an old example. But Google News every day, uh, crawls or reads, uh, uh, uh, I don't know, uh, uh, many, many thousands or tens of thousands of news articles on the Internet and groups them together, right? For example, this is a set of articles on the, uh, BP oil well spill, and it has, uh, taken a lot of the articles written by different reporters and grouped them together. So you can, you know, figure out that, uh, what, uh, BP, uh, Macondo oil well, right? That this is a CNN article about the oil well spill. This is a Guardian article about the oil well spill. And this is an example of a clustering algorithm, where it's, uh, taking these different news sources and figuring out that these are all stories kind of about the same thing, right? Um, and other examples of clustering, just getting data and figuring out what groups belong together. Um, a lot of work on, um, genetic data. This is a visualization of, of, uh, so genetic microarray, array data, where given data like this, you can group individuals into different types of, uh, in-into individuals with different, uh, characteristics. Um, or, uh, clustering algorithms, grouping this type of data together is used to, um, organize computing clusters, uh, you know, figure out what machines workloads are more related to each other and organize computing clusters appropriately. So take a social network, uh, like, you know, LinkedIn or Facebook or other social networks and figure out which are the groups of friends or which are the cohesive communities within a social network. Um, or market segmentation. Um, actually, many companies I've worked with look at the customer database and cluster the users together. So you can say that looks like we have four types of users. You know, looks like that, um, there are the, uh, uh, young professionals looking to develop themselves. There are the, you know, soccer moms and soccer dads. There are the discount and discovery. So you can then market to the different market segments, um, separately. Uh, and, and, and actually many years ago, my friend Andrew Moore, uh, uh, uh, was using this type of data for astronomical data analysis to group together galaxies. Uh, question? So is all unsupervised learning sort of clustering classification? Oh, is all unsupervised learning clustering? Uh, no, it's not. So unsupervised, unsupervised learning broadly is the concept of using unlabeled data, so just X, and finding interesting things about it, right? Um, so, uh, for example, uh, actually, here's... Shoot. This won't work without audio. We'll do this later in the cla- in, in the class, I guess. Um, maybe we'll see if we can do this later. Cocktail party problem, um, uh, is another unsupervised learning problem. Really need audio for this to explain this, though. Um, let me think how to explain this. Uh, you know, cocktail party problems. I'll, I'll, I'll try to do the demo when we can get audio working on this laptop. Is a problem where, um, uh, if you have a noisy room and you stick mult-multiple microphones in the room and record overlapping voices, uh, so there are no labels. We just have multiple microphones, a mic-- uh, an array of microphones in a room with lots of people talking. Uh, how can you have the algorithm separate out the people's voices? So that's an unsupervised learning problem because, um, there are no labels. You just stick microphones in the room and have it record different people's voices, overlapping voices. You have multiple people talking at the same time, and then have it try to separate out people's voices. And one of the program exercises you do, uh, later is if we have, you know, five people talking, so each microphone records five people's overlapping voices, right? Because, you know, each microphone hears five people at the same time. How can you have an algorithm separate out these voices, so you get clean recordings of just one voice at a time? So that's called the cocktail party problem. And the algorithm you use to do this is called ICA, independent components analysis, and that's something you implement in one of the latest, uh, homework exercises. Right. Um, and there are other examples of unsupervised learning as well. Uh, uh, the Internet has tons of unlabeled text data. You just suck down data from the Internet. Uh, there are no labels, uh, necessarily. But can you learn interesting things about language? Figure out what-- figure out, uh, uh... I don't know. One of the best-cited results recently was learning analogies. Like, you know, a man is to woman as king is to queen, right? Um, uh, uh, or, uh, what, uh, Tokyo is to Japan as, uh, Washington DC is to United States, right? To learn analogies like that. Turns out you can learn analogies like that from unlabeled data, just from text on the Internet. So that's also unsupervised learning. Okay? Um, so after unsupervised learning... Oh, and unsupervised learning. So, you know, machine learning is very useful today. Turns out that most of the recent wave of economic value created by machine learning is through supervised learning. Uh, but there are important use cases for unsupervised learning as well. So I use them in my work occasionally. Uh, uh, and it's also leading edge for a lot of exciting research And then the final topic, final of the five topics we cover. So we talk about supervised learning, um, machine learning strategy, deep learning, unsupervised learning, and then the fifth one is reinforcement learning is this, which is, um, let's say I give you the keys to Stanford autonomous helicopter. Uh, this helicopter is actually sitting in my office. I'm trying to figure out how to get rid of it. Um, and, and I ask you to write a program to, to, to make it fly, right? So how do you do that? Um, so this is a video of a helicopter flying. Uh, the audio is just of a lot of helicopter noise, so that's not important. But, uh, we'll zoom out the video. You can see trees flying in the sky, right? But so, um, you can use learning algorithm. Yeah, that's kinda cool, right? I, I, I was the, I was the cameraman that day. Um, but so you can use learning algorithms to get, you know, robots to do pretty interesting things like this. Um, and it turns out that a good way to do this is through reinforcement learning. So, so what's reinforcement learning? Um, it turns out that no one knows what's the optimal way to fly a helicopter, right? If you fly a helicopter, you have, uh, two control sticks that you're moving. Um, but no one knows what's the optimal way to move the control sticks. So the way you can get a helicopter to fly itself is, um, let the helicopter do whatever... Think of it as training a dog, right? Like, how-- You, you can't teach a dog the optimal way to behave. But actually, how, how many of you have had a pet dog or pet cat before? Huh, not that many. That's fascinating. Okay. So I had a pet dog when I was a kid, and my family made it my job to train the dog. So how do you train a dog? You let the dog do whatever it wants, and then whenever it behaves well, you go, "Oh, good dog." And when it misbehaves, you go, "Bad dog." Um, and then over time, the dog learns to do more of the good dog things and fewer of the bad dog things. And so reinforcement learning is a bit like that, right? I don't know what's the optimal way to fly a helicopter, so you let the helicopter do whatever it wants, and then whenever it flies well, you know, does a maneuver you want or flies accurately without jetting around too much, you go, "Oh, good helicopter." And, and when it crashes, you go, "Bad helicopter." And it's the job of the reinforcement learning algorithms to figure out how to control it over time so as to get more of the good helicopter things and fewer of the bad helicopter things. Um, and I think, um... Well, just one more video. Um, oh. Oh, interesting. Yeah, all right. And so again, given a robot like this, I, I actually don't know how to program a... I actually have... You know, a robot like this has a lot of joints, right? So how do you get a robot like this to climb obstacles? So, well, this is actually a robot dog, so you can actually say good dog or bad dog. Um, but by giving those signals, called a reward signal, uh, you can have a learning algorithm figure out by itself how to optimize the reward, therefore climb over these types of obstacles. Um, and I think recently, the most famous applications of reinforcement learning have been for game playing, uh, playing Atari games or playing, you know, game of Go, like in AlphaGo. Um, I think that, uh, uh, I, I, I think that, uh, uh, game playing has made for some remarkable stunts, a remarkable PR, but I'm also, uh, equally excited or maybe even more excited about the inroads that reinforcement learning is making into robotics applications, right? So I think, uh, uh, I, I think, yeah, reinforcement has been proven to be fantastic for playing games. It's also getting, making real traction in, um, optimizing robots and optimizing sort of, uh, uh, logistics system and things like that. Um, so you learn about all these things. Um, last thing for today, uh, I hope that you will start to, uh, talk, meet people in the class, make friends, form project partners and study groups. Uh, and if you have any questions, you know, dive on the Piazza or ask your questions. Let's help others answer their questions. So let's break for today, and I look forward to seeing you on Wednesday. Welcome to two twenty.

Episode duration: 1:15:19