By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Latent SpaceWhy FLUX 3 Trains on Images, Video and Sound — Andreas Blattmann & Dustin Podell, Black Forest Labs
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Great conversations run long. Their best ideas don't have to.
uListen is free on iPhone, iPad, Android, Chrome, and Claude.
See howCHAPTERS
Jump to publisher timestampsChapter summaries
Meet Black Forest Labs: guests’ credentials, why this episode matters
The host introduces Andreas Blattmann and Dustin Podell and frames the conversation around their shared history with Stable Diffusion and the broader diffusion ecosystem. He also previews themes including image editing, video generation, and robotics as an emerging direction.
- •The host situates Black Forest Labs as formed by former core Stable Diffusion contributors
- •The host highlights Dustin’s work across SDXL, Control LoRA, SD3 post-training, and FLUX variants
- •The host explains why image editing models (like FLUX Kontext) could reshape everyday creative workflows
- •The host tees up robotics/world-model directions as a recurring theme across recent episodes
Cold open: funding, revenue, and the shift beyond image generation
Short, high-signal quotes preview the episode’s stakes: the lab’s funding and revenue, and the thesis that video modeling is central to understanding the world. The guests also hint at fast training surprises and ambitions in robotics.
- •Andreas says the lab has raised $450 million in funding
- •Andreas references a published milestone of $100 million in revenue and says they’ve crossed it since
- •Andreas states BFL was not founded to be only an image generation lab and emphasizes video modeling
- •Dustin mentions seeing surprising generality from a very short training run
Black Forest roots: from Heidelberg latent diffusion to founding in Freiburg
Andreas explains how the founding team’s physics background and early research collaborations led to latent diffusion. He also tells the story behind the name Black Forest Labs and why they chose to start in their home region.
- •Andreas describes years of diffusion work and the philosophical appeal of diffusion as a physical phenomenon
- •He recounts meeting collaborators during PhDs and developing latent diffusion at the university
- •He explains the company name and the decision to start near Freiburg at the foot of the Black Forest
- •The host shares how the podcast name “Latent Space” connects to latent representations and a literary reference
Dustin’s path: from freelance developer to researcher via open-source obsession
Dustin recounts how Stable Diffusion motivated him to quit his job and go all-in on generative AI. He details how he learned by reading papers, hacking on models, and working his way from engineer to research inside Stability.
- •Dustin says Stable Diffusion’s release prompted him to resign and focus on understanding the technology
- •He describes participating in open communities and experimenting with models hands-on
- •He joined Stability as an engineer and learned by working closely with researchers and mentors
- •Alessio emphasizes the team’s diverse mix of academic and open-source backgrounds
Visual intelligence: why BFL extends from pixels to multimodal + actions
The guests lay out the transition from image generation to what they call visual intelligence: models that learn intuitive, system-one style understanding from natural signals. They argue that images, video, audio, and actions can be learned within a unified framework using diffusion/flow matching.
- •Dustin says BFL started as image-focused but recognized the technology’s broader potential
- •Andreas describes visual intelligence as intuitive understanding learned from images, video, and audio
- •Andreas references Kahneman’s “system one” framing and contrasts it with language-model intelligence
- •They describe FLUX 3 as a single model trained across images, videos, audio, and actions
World understanding without explicit 3D: observation, implicit structure, and critics
The discussion tackles whether models can learn the world without explicit 3D structure or embodied experience. Andreas and Dustin argue that learning from natural video provides implicit 3D understanding, while acknowledging interaction and long-context memory as key future challenges.
- •Andreas argues no explicit 3D model is needed; training on 2D video can yield implicit 3D understanding
- •Dustin compares model learning from signals to human perception limits and reconstruction
- •They discuss observation-first human learning and the role of interaction for closing the loop later
- •The host raises a common critique about modeling correlations vs ‘the thing itself,’ prompting rebuttal
Inside SDXL: multi-aspect training, the refiner, and structure vs texture
Dustin walks through the technical and product motivations behind SDXL, including training on multiple aspect ratios and developing post-training data. He explains the SDXL refiner as a separate diffusion model used to improve high-frequency details and shares ‘secret lore’ about an unreleased architecture that became the refiner.
- •Dustin describes SDXL as a push to make models feel better to use, including multi-aspect training
- •He explains the refiner as a second diffusion model focused on high-frequency detail rather than resolution changes
- •Dustin says the refiner originated from a different internal model that had better texture but poor structure
- •Andreas frames this as an early exploitation of how structure emerges earlier in denoising than fine texture
Learning motion with video diffusion: temporal consistency and data filtering
Andreas recounts early work mapping latent diffusion to video and explains why temporal learning is essential to avoid flicker. He emphasizes that data quality and filtering are underappreciated levers and describes experiments exploring the quality–quantity trade-off and how video pretraining supports later capabilities.
- •Andreas describes adapting latent diffusion to video and the importance of compressing temporal dimensions
- •He highlights temporal attention/transformers as crucial for consistent motion and avoiding flicker
- •He argues data filtering and dataset quality shape what models can learn, independent of algorithm strength
- •He discusses searching for the sweet spot between data quantity and quality for training runs
Flow matching and SD3: simplifying math, allocating compute, and the role of data
The conversation moves to SD3 and the shift from classic diffusion to flow matching/rectified flow, plus architectural changes toward transformers. Andreas explains why flow matching is mathematically simpler and more general, and they discuss training efficiency: where along the noise-to-data trajectory compute matters most.
- •Andreas explains flow matching as learning a vector field between distributions and integrating an ODE
- •They describe SD3 as analyzing which parts of the denoising trajectory matter most for compute allocation
- •Dustin links this to the SDXL refiner intuition about emphasizing later steps for fine detail
- •Text-in-image improvements are attributed primarily to better data/captions and better modeling rather than a single architectural trick
Starting Black Forest Labs: frustrations at Stability, rapid FLUX.1 progress, partnerships
Andreas describes the early BFL stealth period, the small founding team, and an offsite that kicked off training what became FLUX.1. Dustin explains motivations for founding the company, including a belief that Stability couldn’t capture the models’ value, and they discuss leveraging partnerships for distribution and post-training.
- •Andreas recalls an early offsite where the team aligned and began training FLUX.1
- •He says the company started small and focused, enabling rapid execution and a strong launch reception
- •Dustin says they believed they could both build frontier models and better capture their value as a business
- •They mention partnerships as a way to solve distribution and accelerate post-training/distillation work
From image gen to editing: FLUX Kontext and the push for controllability
The guests argue the next frontier is controllability, not just raw visual realism. Dustin tells the story of FLUX Kontext starting as his side project, driven by the idea that models already contain spatial relationships and can be trained with targeted data to support iterative, character-consistent editing.
- •They identify controllability and character consistency as major pain points beyond visual quality
- •Dustin says Kontext began as an internal pitch and then a small side project with targeted data creation
- •He describes a short early training run that revealed surprising generality for editing tasks
- •They position editing as moving away from ‘slot machine’ prompting toward iterative workflows
Editing evaluation and references: Flux.2 multi-image context, human/arena-style benchmarks
They discuss limitations of single-image editing and the need for multi-reference conditioning, then describe building a public evaluation set to compare models. The group emphasizes that many generative tasks require human judgment and that model builders must actively use and test their own models.
- •Dustin says single-image edits were limiting and motivated Flux.2 support for multiple reference images
- •They discuss practical limits on references from both usability and evaluation complexity perspectives
- •They describe a public context evaluation set designed for arena-style comparisons rather than fixed ground truth
- •They argue evaluation is inherently hard in generative tasks and must combine multiple methods plus hands-on testing
Autoregressive vs diffusion for images (and text): trade-offs and unsettled debates
The group discusses the apparent trend toward autoregressive image generation and whether conversational editing is tied to autoregression. Andreas and Dustin argue capabilities depend heavily on training data and systems design, outline the data-efficiency vs sampling-speed trade-off, and touch on diffusion approaches for language.
- •Andreas argues conversational capability is not exclusive to autoregressive models; it can be trained into diffusion too
- •He states autoregressive models are more data-efficient in training, while diffusion can be faster at sampling via distillation
- •They say the field hasn’t settled the diffusion vs autoregressive debate and active research continues on both fronts
- •Dustin distinguishes discrete diffusion-language models from continuous denoising-style approaches
FLUX.3 Action: world action models, chunked prediction, and robotics roadmap to 2027
Andreas explains how a multimodal pre-trained model can be extended to action prediction by adding robot state/action representations alongside video and instructions. They discuss real-time constraints, fine-tuning vs in-context learning aspirations, and predict major commercialization in manufacturing—calling 2027 a pivotal year for robotics.
- •Andreas describes pretraining on images/video/audio then adding action prediction via robot state parameterizations
- •He says the model predicts actions alongside video in chunked steps and can run in a loop on a robot
- •They emphasize aiming beyond fine-tuning toward in-context robotics learning—show once, then do the action
- •They predict early production deployments and call 2027 the year robotics breaks out, especially in manufacturing contexts
Who uses FLUX and how it spreads: brand awareness, responsibility, and unexpected viral uses
In closing, they discuss distribution challenges and public perception of AI in generative media. They emphasize professional workflow value, responsible positioning, and cite image restoration as an unexpectedly meaningful use case that created strong emotional impact for users.
- •Andreas asks for broader awareness and marketing support so more people know the lab beyond the tech community
- •A speaker notes generative media can face backlash when labeled ‘AI,’ making go-to-market tricky
- •Andreas frames the goal as productivity and workflow improvement rather than flooding the world with low-quality outputs
- •Dustin highlights image restoration and colorization of old family photos as a standout positive use case
Publisher timestamps
- 0:00Host introduction
- 3:39Cold open
- 4:23Black Forest roots
- 9:23Dustin’s path
- 17:04Visual intelligence
- 27:38Inside SDXL
- 33:59Learning motion
- 39:57Flow matching
- 46:57Starting BFL
- 50:36Deepgram sponsor break
- 51:43Early use cases for FLUX
- 54:58Editing with Kontext
- 1:03:15Training data
- 1:07:32Testing an edit
- 1:13:12Autoregression
- 1:20:50World action models
- 1:30:08Robotics in 2027
- 1:32:16Who uses FLUX?