Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 3 - Tranformers & Large Language Models

ℹ️ EPISODE INFO

Released

October 17, 2025

Duration

1h 48m

Channel

Stanford Online

Watch on YouTube

▶ Open ↗

📝 EPISODE DESCRIPTION

For more information about Stanford’s graduate programs, visit: https://online.stanford.edu/graduate-education October 10, 2025 This lecture covers:

• Definition and architecture
• Mixture of experts
• Context length, temperature
• Sampling strategies
• Prompting, in-context learning
• Chain of thought
• Self-consistency

To follow along with the course schedule and syllabus, visit: https://cme295.stanford.edu/syllabus/ Chapters: 00:00:00 Introduction 00:01:02 Recap of Transformers-based models 00:03:43 LLM definition 00:07:37 Mixture of Experts 00:12:43 Dense & Sparse MoE 00:15:47 MoE in LLMs 00:36:35 Response generation 00:38:34 Greedy decoding & beam search 00:46:36 Sampling-based methods 00:52:24 Impact of temperature on predictions 01:04:53 Guided decoding 01:07:07 Prompting strategies 01:14:09 In-context learning 01:18:35 Chain-of-thought, self-consistency 01:25:00 Inference optimizations with KV cache 01:33:09 PagedAttention, MLA Afshine Amidi is an Adjunct Lecturer at Stanford University. Shervine Amidi is an Adjunct Lecturer at Stanford University. View the course playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rOCXd21gf0CF4xr35yINeOy

🧾 EPISODE SUMMARY

In this episode of Stanford Online, Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 3 - Tranformers & Large Language Models explores Lecture surveys LLM basics, decoding, prompting, and inference optimizations The lecture defines large language models as large, decoder-only next-token predictors and contrasts them with encoder-only and encoder–decoder transformer variants discussed in earlier lectures.

🔗 RELATED EPISODES

Stanford CS230 | Autumn 2025 | Lecture 4: Adversarial Robustness and Generative Models

Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 2 - Transformer-Based Models & Tricks

Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 1 - Transformer

Stanford CS230 | Autumn 2025 | Lecture 3: Full Cycle of a DL project

Stanford CS230 | Autumn 2025 | Lecture 2: Supervised, Self-Supervised, & Weakly Supervised Learning

Stanford CS230 | Autumn 2025 | Lecture 1: Introduction to Deep Learning