Stanford OnlineStanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 9 - Recap & Current Trends
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
CHAPTERS
Jump to publisher timestampsChapter summaries
Lecture 9 roadmap: recap, trends, and course wrap-up
The instructor explains this final lecture will be structured differently: a recap of the quarter, a set of trending topics, and closing thoughts with next steps. They frame the session as connecting the pieces of the course rather than introducing lots of brand-new material.
- •The lecture is split into three parts: course recap, current/future trends, and concluding next steps
- •The goal is to connect concepts from across the quarter into a coherent picture
- •This is the final lecture of the course
From tokens to self-attention: building blocks behind Transformers
The recap starts from the basics of processing text: tokenization and vector representations. It then motivates why earlier sequence models struggled and how self-attention addressed long-range dependency issues.
- •Tokenization as splitting text into atomic units, with subword tokenization highlighted as common
- •Word2vec-style embeddings learned from proxy prediction tasks, but not context-aware
- •RNNs process tokens sequentially but struggle with long-range dependencies
- •Self-attention enables tokens to attend to each other regardless of distance
- •Query/key/value dot-product attention produces weighted combinations of token representations
Transformer architecture and post-2017 improvements (RoPE, attention variants, norm choices)
The instructor revisits the encoder–decoder Transformer and how it underpins modern models. They then review key architectural refinements that emerged after the original publication, especially around positional encoding, attention computation, and normalization placement.
- •Transformer encoder–decoder framing, originally motivated via translation
- •Absolute vs relative position handling; Rotary Position Embeddings (RoPE) as a relative-position method inside attention
- •Grouped query attention as a way to share or group projections across heads (especially for keys/values)
- •Post-norm versus pre-norm as a design choice in Transformer blocks
- •Derived model families: encoder-only (BERT), decoder-only (GPT), and encoder–decoder (T5)
What makes an LLM “large”: scaling, MoE sparsity, and decoding randomness
The recap transitions to decoder-only, text-to-text Transformer models commonly called LLMs. It highlights scaling-related tricks like Mixture of Experts for compute efficiency and discusses sampling-based decoding controls.
- •LLMs described as Transformer-based, decoder-only, autoregressive text generation models
- •Mixture of Experts activates only a subset of expert feedforward networks per input (sparse compute)
- •Token-level routing enables distributing experts across different GPUs for more parallelism
- •Next-token prediction leads to a probability distribution over tokens; sampling vs greedy decoding
- •Temperature controls randomness: low temperature more deterministic, high temperature more varied
How LLMs are trained at scale: compute/data/parameter tradeoffs and FlashAttention
The instructor explains the practical realities of training very large models: scaling laws, compute budgets, and systems techniques. They review efficiency methods like FlashAttention and parallelism strategies used during pre-training.
- •Scaling trends: more compute, more data, and more parameters generally reduce test loss
- •Compute budgets motivate choosing an “optimal” model size and dataset size pairing
- •Rule of thumb discussed: training tokens on the order of ~20× the parameter count
- •FlashAttention optimizes GPU memory reads/writes (HBM vs SRAM) while remaining exact
- •Recomputation can be beneficial: discarding intermediates and recomputing later can improve runtime
- •Data parallelism and model parallelism as core distributed training approaches
- •Pre-training produces an autocomplete-capable model trained on very large token counts
From pre-training to helpfulness: SFT and preference tuning (RLHF framing)
After pre-training, the recap describes how models become helpful via supervised fine-tuning and then aligned via preference-based tuning. The instructor explains the RL analogy that motivates preference optimization and why staying close to a base model matters.
- •Supervised fine-tuning (SFT) trains on desired input–output pairs to shape behavior
- •Preference tuning introduces negative/relative feedback: preferring one output over another
- •LLM generation is compared to an RL policy acting over token sequences and receiving a reward signal
- •Reward modeling via a Bradley–Terry-style pairwise formulation to learn preferences
- •A reward model may be trained pairwise but used to score single outputs at inference
- •Regularization constraints keep the tuned model close to the SFT reference and limit step-to-step drift
- •Reward hacking risk motivates staying near the base model
Reasoning-focused training: chain-of-thought prompting and GRPO vs PPO
The recap turns to the recent push for reasoning-capable models trained to produce reasoning chains before final answers. The instructor contrasts GRPO with PPO and explains why GRPO is popular for verifiable-reward settings like math.
- •Reasoning chains are trained and prompted to appear before final answers to improve performance
- •An example benchmark is discussed as a way people track reasoning gains during training
- •PPO is described as relying on a value model and advantage estimation machinery
- •GRPO avoids training/maintaining a value model by comparing rewards across multiple completions
- •Verifiable rewards (e.g., tasks with known correct answers) can remove the need for a learned reward model
- •Only the policy model and a reference model are emphasized for certain GRPO setups
- •Extensions mentioned address length-related biases that can push models toward long incorrect answers
Agentic workflows: RAG retrieval pipeline and tool calling loops
The instructor explains how models become more useful by interacting with external information and systems. They detail Retrieval-Augmented Generation (RAG) as a multi-stage retrieval process and tool calling as structured API usage with results fed back to the model.
- •RAG addresses knowledge cutoffs and non-continuous training by fetching up-to-date or missing info
- •Candidate retrieval via a bi-encoder semantic search over embeddings and similarity scoring
- •Re-ranking via a cross-encoder scoring query–document pairs more precisely
- •Top results are added to the prompt (augmentation) before generation
- •Tool calling: the model selects an API and arguments, an intermediate executes it, then results are returned to the model
- •Modern agentic workflows combine RAG and tool calling across multiple steps before producing an answer
Evaluating LLMs: from BLEU/ROUGE to “LLM as a judge” and its biases
The recap ends with evaluation, emphasizing why older overlap-based metrics fall short for open-ended language generation. The instructor describes LLM-as-judge setups and the biases that can distort judgments.
- •Rule-based metrics like BLEU, ROUGE, and METEOR don’t capture valid linguistic variation well
- •LLM as a judge: judge model is given prompt, response, and criteria to produce a score and rationale
- •Binary judgments (pass/fail, true/false) are described as common for simplicity
- •Producing rationale before the score is described as improving judge performance
- •Biases discussed: position bias (ordering effects), verbosity bias (prefers longer outputs), self-enhancement bias (prefers its own outputs)
- •Benchmarks span multiple dimensions such as knowledge, reasoning, coding, and safety
Transformers beyond text: Vision Transformer (ViT) and vision-language model integration
The lecture’s trend section begins by asking whether self-attention can work on non-text data. The instructor explains how ViT tokenizes images into patches for encoder-based classification and then discusses common ways to combine image and text tokens for question answering over images.
- •Self-attention operates on vectors, motivating applying Transformers to non-text inputs like image patches
- •ViT uses the Transformer encoder: split images into patches, embed them with position information, and classify via a CLS representation
- •ViT is described as outperforming traditional convolutional approaches when trained on lots of image data
- •Discussion of inductive bias: CNNs impose strong locality structure, Transformers have lower inductive bias
- •Vision-language models can concatenate image-derived tokens with text tokens into a decoder-only LLM to generate answers
- •A less common alternative is injecting image information through cross-attention rather than concatenation
- •The instructor encourages staying open to Transformer applications across domains (recommendation, speech, and more are mentioned)
Diffusion-based LLMs: replacing autoregressive decoding with masking-and-denoising
The instructor introduces diffusion-based language modeling as a hot topic, motivated by faster inference than token-by-token generation. They explain diffusion intuitions from images and the text analogue where masking plays the role of noise, enabling coarse-to-fine refinement in fewer steps.
- •Autoregressive inference is inherently sequential (not parallelizable at inference time), even though training can be parallelized with causal masking
- •Diffusion in images is explained as learning to remove noise added gradually during a forward noising process
- •Text is discrete, so “noise” is reframed as masking: progressively mask tokens, then learn to reconstruct/unmask them
- •Terms mentioned for this family include masked diffusion models and diffusion-based LLMs
- •Diffusion decoding uses a fixed number of denoising steps, often fewer than the number of output tokens, yielding lower latency
- •Coarse-to-fine analogy: drafting and refining a speech rather than writing strictly left-to-right
- •Potential advantages noted: speedups (especially for long outputs) and suitability for fill-in-the-middle coding tasks
- •Open work includes matching frontier autoregressive performance and adapting common LLM techniques (e.g., reasoning-style methods) to diffusion
Cross-pollination between vision and text, and why Transformer design is still evolving
Shervin takes over to reflect on how ideas travel between modalities: diffusion concepts moving into text, and Transformers becoming common in modern image generation pipelines. He emphasizes that even core Transformer components remain active research areas with competing design choices.
- •Shervin highlights bidirectional inspiration: diffusion ideas from images influencing text, and Transformers replacing convolutions in many image-generation approaches
- •An example is given where image patch tokens can reconstruct text tokens from relatively few vision tokens
- •RoPE is revisited as a trick from text that can be adapted to 2D settings for images and multimodal layouts
- •Ongoing iteration areas listed include optimizer choices, normalization placement and type, attention variants, and activation functions
- •Architectural and hyperparameter decisions (including MoE vs dense) are portrayed as not settled across papers
Data, efficiency, and hardware: curation, smaller models, and specialized acceleration
Shervin discusses constraints shaping the next phase: data quality in a world full of model-generated text, pressure to reduce serving costs, and opportunities for hardware innovation. He argues these forces will increasingly guide research beyond pure benchmark chasing.
- •Data concerns: internet content is increasingly model-generated, driving more emphasis on dataset curation
- •A multi-stage pipeline is described: pre-training, mid-training on higher-quality corpora, and fine-tuning
- •Training on model-generated text is linked to concerns about reduced diversity and “model collapse” (as described in a referenced line of work)
- •A shift is suggested toward a new frontier: achieving high quality more cost-effectively, including interest in smaller language models
- •Hardware is framed as underexplored in the class: GPUs excel at matrix multiply but Transformers have additional expensive operations (e.g., attention)
- •FlashAttention is referenced as an example of memory-movement optimization on existing hardware
- •A proof-of-concept hardware direction is described where operations are embedded in the physical hardware behavior, improving latency and energy in simulation
Where LLM products are going: everyday agents, security, and unresolved challenges
Shervin surveys how LLMs are already used—from coding assistance to brainstorming and creative drafting—and what may come next as agentic systems become more accessible. He also outlines major open problems like reliability over long action chains, continuous learning, and the broader set of safety and interpretability concerns.
- •Current use cases mentioned include coding assistants with agent modes, web browsing assistance, creativity support, and learning/brainstorming during lectures
- •A trend is suggested toward democratizing agentic workflows so non-experts can create useful tools via natural language
- •Agent reliability is framed as compounding failure probability with more steps; stabilization is an ongoing need
- •Security concerns are raised for browsing/agents: prompt injection and data exfiltration risks
- •Longer-term test: whether AI-powered customer service becomes genuinely helpful compared to humans, including dimensions like empathy and groundedness
- •Open questions listed include continuous learning beyond fixed weights, hallucinations as tied to next-token training, personalization, interpretability, and safety
Staying up to date and closing thanks
Shervin closes with practical ways to keep learning after the course, spanning papers, implementations, and community discussion. The instructors thank both in-person and online participants and conclude the course on an appreciative note.
- •Suggested ways to stay current include reading arXiv and conference highlights and studying accompanying codebases
- •Hugging Face trending papers are mentioned as a discovery resource
- •Social/community channels like Twitter/X and YouTube paper walkthroughs are recommended
- •Specific educator channels are called out as helpful for deep dives into papers and concepts
- •The instructors express gratitude to students attending in person and viewers online, and formally conclude the course
