Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Episode Details
EPISODE INFO
- Released
- October 21, 2025
- Duration
- 1h 47m
- Channel
- Stanford Online
- Watch on YouTube
- ▶ Open ↗
EPISODE DESCRIPTION
For more information about Stanford’s graduate programs, visit: https://online.stanford.edu/graduate-education October 17, 2025 This lecture covers:
- Pretraining
- Quantization
- Hardware optimization
- Supervised finetuning (SFT)
- Parameter-efficient finetuning (LoRA)
To follow along with the course schedule and syllabus, visit: https://cme295.stanford.edu/syllabus/ Chapters: 00:00:00 Introduction 00:07:19 Pretraining 00:13:26 FLOPs, FLOPS 00:16:34 Scaling laws, Chinchilla law 00:24:49 Training optimizations overview 00:31:09 Data parallelism with ZeRO 00:35:51 Model parallelism 00:38:26 Flash Attention 00:52:37 Quantization 00:56:00 Mixed precision training 01:02:31 Supervised finetuning 01:09:21 Instruction tuning 01:37:53 Parameter-efficient finetuning with LoRA 01:45:16 QLoRA Afshine Amidi is an Adjunct Lecturer at Stanford University. Shervine Amidi is an Adjunct Lecturer at Stanford University. View the course playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rOCXd21gf0CF4xr35yINeOy
EPISODE SUMMARY
In this episode of Stanford Online, Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 4 - LLM Training explores How LLMs are pretrained, optimized, and fine-tuned at scale The lecture outlines the LLM training pipeline as pretraining on vast next-token data followed by smaller fine-tuning stages that shape the model into a helpful assistant or task-specific system.
RELATED EPISODES
