Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
CHAPTERS
- 0:04 – 5:08
Welcome and why machine learning is a major opportunity
Andrew Ng welcomes the class and frames AI as the new electricity. He describes how demand for machine learning skills is growing across industry and academia, and notes the overflow enrollment in the room.
- •CS229 has trained generations of Stanford students in machine learning
- •AI compared to electricity transforming every major industry
- •English professors, lawyers, and non-tech companies are all applying machine learning
- •Today is likened to a good time to start on the internet twenty years ago
- •Not quite 800 enrolled, so lectures are recorded and posted online
- 5:08 – 10:12
Teaching team and goals for the course
Ng introduces the class coordinator, co-head TAs and the wider TA team, whose expertise spans several application areas. He then sets out his goals: that students become machine learning experts able to build applications in academia, industry, or research.
- •TAs have expertise from computer vision to robotics and will be matched to projects
- •Goal is for students to be experts after ten weeks
- •Ng mentions leading the Google Brain team and the AI group at Baidu
- •Machine learning has spread beyond elite tech companies
- •The class also prepares PhD students to read papers and push the state of the art
- 10:12 – 12:57
Prerequisites and the move to Python
Ng lists the assumed background in computer science basics, probability, and linear algebra, with Friday review sessions for those who need them. He explains that programming assignments are moving from MATLAB/Octave to Python and NumPy.
- •Assumes Big O notation, queues, stacks, and binary trees
- •Assumes basics of random variables, expected value, and variance
- •Assumes matrices, vectors, and ideally eigenvectors
- •Assignments shift from MATLAB/Octave to Python with NumPy
- •Handouts are going digital only
- 12:57 – 16:50
Honor code and study groups
Ng encourages study groups for this technical class but asks students to write up homework solutions on their own. He explains the importance of preserving what completing CS229 means to employers.
- •Study groups are encouraged based on education research
- •Discussing homework is fine, but write-ups must be done alone, without shared notes
- •The honor code is documented on the course website
- •Employers recognize CS229, with some job ads promising interviews to completers
- 16:50 – 20:46
Class project guidelines and group size
Ng describes the class project as one of the best parts of the course, usually applying machine learning to an application area. He explains group size rules and the stricter grading for groups larger than three.
- •Brainstorm projects and look at previous years' projects on the course website
- •Past projects range from diagnosing cancer to art and engineering
- •Groups of up to three are standard; solo work is allowed
- •Groups of four are held to a higher standard
- •Many students are in their first Stanford class
- 20:46 – 26:04
Discussion sections, Piazza, and the take-home midterm
Ng explains that Friday discussion sections are optional, covering prerequisites first and advanced topics later. He also covers Piazza, Gradescope, and email, and announces the take-home midterm.
- •Discussion sections are optional and nothing on the midterm comes only from them
- •First three sections review linear algebra, statistics, and Python/NumPy
- •Later sections cover convex optimization, hidden Markov models, and time series
- •Piazza is preferred for non-private questions because it gets faster answers
- •The midterm will be take-home rather than timed
- 26:04 – 34:08
Q&A on CS229, CS229A, and CS230
In response to questions, Ng compares CS229 with CS229A and CS230. He describes how they differ in mathematical depth and focus, and encourages students to take multiple classes.
- •CS229 is the most mathematical of the three and has a heavy workload
- •CS229A is more applied, taught in a flipped classroom, and has capped enrollment
- •CS230 focuses specifically on deep learning
- •Little overlap in content between the three classes
- •Ng encourages taking multiple classes for different perspectives
- 34:08 – 40:13
Why machine learning matters and what it is
Ng shares his view that the meaningful work enabled by machine learning matters more than the money. He then gives Arthur Samuel's and Tom Mitchell's definitions, illustrated with checkers.
- •Ng finds more excitement in meaningful work like healthcare and education than in money
- •Arthur Samuel's definition: computers learning without being explicitly programmed
- •Samuel's checkers program learned through self-play and beat its author
- •Tom Mitchell's definition uses experience E, task T, and performance measure P
- •Checkers example mapped onto E, T, and P
- 40:13 – 51:54
Supervised learning: regression and classification
Ng defines supervised learning with housing price prediction and tumor classification, distinguishing regression from classification. He extends this to multiple input features and previews logistic regression and support vector machines with infinite-dimensional features.
- •Supervised learning learns a mapping from inputs X to labels Y
- •Housing prices are a regression problem because y is continuous
- •Benign vs. malignant tumors are a classification problem with discrete outputs
- •Real problems usually have many features that cannot be plotted
- •Support vector machines and kernels allow infinitely many features
- 51:54 – 58:11
Autonomous driving video as a supervised learning example
Ng plays an old video of a neural network trained to drive a vehicle at Carnegie Mellon. He narrates how it learns steering from human demonstrations, as an example of supervised learning.
- •The car records camera images along with the human's steering direction as the label
- •The network's output starts as a blur and sharpens to mimic the human
- •After training, the network steers the vehicle itself
- •Separate models for one-lane and two-lane roads were used with an arbitrator
- •Not state of the art, but students will build something more sophisticated
- 58:11 – 1:04:05
Machine learning strategy and learning theory
Ng argues that teams applying the same algorithm differ greatly in effectiveness, and that skilled practitioners are strategic. He wants to turn machine learning from tribal knowledge into a systematic engineering discipline, using debugging and profiling analogies, and mentions his book draft.
- •Teams sometimes spend six months on an approach that was never going to work
- •Practitioners face many decisions: more data, different algorithms, faster GPUs
- •Debugging learning algorithms should be systematic rather than black magic
- •Analogy to profiling code instead of blindly optimizing
- •Ng offers a free draft of a book on engineering principles for machine learning
- 1:04:05 – 1:11:08
Deep learning and unsupervised learning
Ng introduces deep learning as the third major topic, noting that CS230 covers it more narrowly. He then explains unsupervised learning with unlabeled data, covering clustering examples, the cocktail party problem, and word analogies.
- •Deep learning is a fast-advancing subset of machine learning
- •Unsupervised learning finds structure in data with no labels
- •k-means clustering finds clusters, as in Google News grouping, genetic data, and market segmentation
- •The cocktail party problem separates overlapping voices using independent components analysis
- •Most recent economic value comes from supervised learning, but unsupervised learning matters too
- 1:11:08 – 1:15:19
Reinforcement learning and closing remarks
Ng introduces reinforcement learning as the fifth topic, using the autonomous helicopter and dog training as analogies. He covers reward signals, applications in game playing and robotics, and closes by encouraging students to make friends and use Piazza.
- •No one knows the optimal way to fly a helicopter, so it learns from good and bad signals
- •Reinforcement learning is compared to training a dog with praise and scolding
- •A reward signal lets a robot dog learn to climb obstacles
- •Game playing like Atari and Go is famous, but robotics and logistics are also exciting
- •Students are urged to form study groups and project partners
