Stanford OnlineStanford CS230 | Autumn 2025 | Lecture 3: Full Cycle of a DL project
At a glance
WHAT IT’S REALLY ABOUT
Deep learning projects: iterate fast, engineer data, monitor drift continuously
- AI projects differ from traditional software because behavior is driven by both code and unpredictable data, making development inherently iterative and empirical.
- For early-stage teams, prioritizing speed of execution (e.g., collecting a “good enough” dataset in days) enables rapid model/data iteration and reveals what truly matters in the data.
- Data improvement should be guided by error analysis and targeted collection (data-centric AI) rather than indiscriminately gathering more of everything.
- Real deployments require system-level engineering choices (e.g., edge visual activity detection plus cloud face recognition) that balance accuracy, latency, bandwidth, and cost.
- After launch, models must be monitored and maintained because concept/data drift and changing environments can break test-set performance in production.
IDEAS WORTH REMEMBERING
5 ideasTreat ML as an empirical, iterative discipline—not a one-shot build.
Because you rarely know what’s in the data (or future data), progress comes from building something quickly, observing failures, and iterating on model, data, and system design.
Optimize for speed early: collect data in days, not months.
Fast iteration (e.g., a 48-hour data sprint) surfaces real bottlenecks and prevents overinvesting in datasets whose value you can’t predict before training and testing.
Use error analysis to decide what data to collect next.
When performance is weak on specific conditions (e.g., hats, scarves, blur), collect or engineer data for those slices; “more data” works best when it’s the right kind of data.
Data quality and relevance matter, but perfect distribution matching is often unnecessary.
Large-capacity models can absorb somewhat “off-distribution but not wrong” data without harm, though the benefit is problem-dependent and must be validated experimentally.
Start with the simplest deployment component that gives learning fastest.
A non-ML VAD like pixel-difference thresholding can be implemented in minutes and used to learn real failure modes (trees swaying, cars passing) before investing in a learned VAD.
WORDS WORTH SAVING
5 quotesAI projects are different than traditional software engineering projects.
— Kian Katanforoosh
In traditional software projects, you write code and you control your code, right? Write whatever code you want, compile it, your code does what you tell it to. But AI projects, you know, involve both code as well as data that you train your algorithm on, and you almost never know what strange and wonderful things there are in your data.
— Kian Katanforoosh
One of the biggest predictors for the chance of success is just the sp-she-- speed of execution. It's just speed of getting stuff done.
— Kian Katanforoosh
I instead tend to go to my teams and say, "We have two days or one day or maybe a week," right? Some short time span like that, and say, "What's the most creative, you know, respectful, responsible, but creative way you can use to collect data in this short time span?"
— Kian Katanforoosh
I encourage you to think of yourselves if you're building a machine learning system, I think of my job as building something that works, and that can be different than building something that works on the test set.
— Kian Katanforoosh
High quality AI-generated summary created from speaker-labeled transcript.
