How I AIHow Emmy Award–winning filmmakers use AI to automate the tedious parts of documentaries
CHAPTERS
- 0:00 – 3:57
Automating documentary post-production media management with AI
Tim explains why documentaries generate a chaotic mix of assets—stills, archival footage, interviews, transcripts—and why manual logging becomes a major bottleneck. He frames AI as a practical tool for automating tedious data entry rather than generating creative content.
- •Post-production is dominated by media management complexity
- •Documentary workflows involve many file types and sources
- •Manual logging/metadata entry is time-consuming and error-prone
- •AI is most immediately valuable as a tooling layer vs. content generation
- 3:57 – 5:21
The documentary shooting ratio: why databases get enormous
Claire and Tim quantify the scale of documentary research and captured media using the ‘shooting ratio’ concept. Tim shares concrete numbers from a major production to illustrate why searchability and organization are mission-critical.
- •Documentaries can have very high shooting ratios
- •Example: Muhammad Ali series—20,000 stills, 100+ hours footage, ~35 interviews
- •Teams must manage hundreds of hours and tens of thousands of images
- •The organizational challenge directly impacts production efficiency
- 5:21 – 5:52
Inside the archival asset database: what needs to be logged
Tim shows the type of database used on productions and the fields required for each asset—descriptions, tags, dates, sources, and technical details. The goal is to automate this end-to-end because it has historically required repetitive manual entry.
- •Database includes stills, footage, music, and other assets
- •Logging requires both creative and technical metadata
- •Descriptions/tags/source data are essential for later discovery
- •Automation target: replace years of manual data entry
- 5:52 – 8:59
Early vision workflow: a single script that describes an image
Tim recreates the initial approach: upload an image to an AI model and get a plain-language description. He demonstrates how he used prompt-driven scripting (via Cursor + dictation) to quickly prototype a working pipeline.
- •First prototype: simple Python script calling an AI vision endpoint
- •Used Cursor and dictation tooling to speed up prototyping
- •Basic image descriptions are helpful but often too generic
- •Early outputs can hallucinate or omit critical factual identifiers
- 8:59 – 11:57
Adding metadata guardrails to reduce hallucinations and improve accuracy
Tim extends the script to extract embedded metadata (e.g., Library of Congress fields) and append it to the model prompt. This shifts the output from generic guesses to verifiable specifics like place, date, and photographer credit.
- •Archival images often contain embedded metadata not visible to the eye
- •Script extracts metadata first and feeds it as source-of-truth context
- •Guardrails encourage factual specificity (location/date/creator)
- •Workflow can later expand to scraping trusted web sources for verification
- 11:57 – 13:14
Model choices and switching costs: Claude for coding, OpenAI for vision
Claire asks why Tim mixes models: Claude for generating code and OpenAI for image analysis. Tim explains it’s partly historical—OpenAI had early vision APIs—and partly pragmatic due to accumulated infrastructure and switching costs.
- •Claude models used for coding assistance inside Cursor
- •OpenAI used for vision because it was first in Tim’s pipeline
- •Switching costs matter once infrastructure is built
- •Model choice is task- and reliability-dependent (esp. live demos)
- 13:14 – 15:16
From one-off scripts to a production REST API with job pipelines
Tim shows the evolved system: a dedicated REST API running on a remote machine that handles asset ingestion and metadata tasks automatically. The workflow includes file spec extraction, server copy/ID renaming, metadata parsing, URL scraping, and description generation.
- •Architecture shift: single script → reusable REST API service
- •API is triggered by the database tool via webhooks/automation
- •Pipeline steps: file specs → copy/rename → parse metadata → scrape URL → generate description
- •System generalizes across images, video, music, and more
- 15:16 – 17:45
Video understanding at scale: frame sampling + reasoning over events
Tim explains how the same approach scales to video by sampling frames at intervals to control cost. He generates lightweight frame captions with a cheaper model, then sends the aggregate (frames + transcript) to a stronger reasoning model to summarize what happens.
- •Video processed as sampled frames plus audio
- •Frame sampling chosen for cost control (e.g., every 5 seconds)
- •Cheap model for per-frame captions; stronger reasoning model for holistic interpretation
- •Prompting evolved into a ‘parent prompt’ that compiles observed events
- 17:45 – 19:45
Audio transcription with Whisper and time-aligned video context
Tim details the audio pipeline: Whisper-based speech-to-text plus alignment to time intervals that match frame sampling. The result is a synchronized view of what’s seen and said, enabling better downstream reasoning and database searchability.
- •Whisper used for robust speech-to-text transcription
- •Audio segmented to align with frame timestamps
- •Frames + time-pegged transcript improve event understanding
- •Back-end processing supports team-wide usage via the API
- 19:45 – 22:51
Semantic discovery with vector embeddings (CLIP + text) and fusion search
Tim introduces embeddings as the next leap: enabling semantic search beyond exact keyword matching. He creates image embeddings from thumbnails using CLIP and text embeddings from descriptions, then fuses them for stronger retrieval.
- •Embeddings enable semantic search (puppy vs. dog problem)
- •Two embeddings per asset: visual (CLIP) + textual (OpenAI text embeddings)
- •Fusion improves retrieval robustness and relevance
- •Moves the system from ‘logging’ to ‘discovery’
- 22:51 – 24:44
Reverse image search in the archive: ‘Find Similar’ for editors
Tim demonstrates using embeddings for a ‘Find Similar’ feature that acts like internal reverse image search. This helps editors quickly surface thematically or visually related assets and identify duplicates or near-duplicates.
- •One-click similarity search within the organization’s collection
- •Useful for editors seeking consistent visual ‘vibe’
- •Can reveal duplicates and related portrait sets
- •Embeddings unlock navigation patterns not possible with tags alone
- 24:44 – 28:14
Field research capture: the ‘Flip Flop’ iOS app for front/back photos
Tim explains the real-world archive workflow: shooting thousands of iPhone photos of documents and image backs for notes/accession numbers. ‘Flip Flop’ structures capture into collections, pairs front/back reliably, and automates transcription of backside text.
- •Physical archive visits produce huge, messy camera rolls
- •Key need: reliably pair the front and back of each item
- •App organizes by collection/folder to maintain provenance
- •AI transcribes backside notes to reduce later manual cleanup
- 28:14 – 32:20
Embedding AI results into file metadata + enforcing structured file naming
Instead of only storing text in a database, Flip Flop writes captions/transcriptions directly into image EXIF metadata. Tim emphasizes that consistent naming conventions and embedded metadata make assets portable and usable across tools and teams.
- •AI output written into EXIF metadata on the image file itself
- •Assets remain self-describing even outside the app/database
- •Structured filenames keep front/back pairs sortable and importable
- •Workflow improvement scales (e.g., 1,400 images returned from a trip)
- 32:20 – 37:59
Precision OCR for historical documents: ‘OCR Party’ desktop utility
Tim demos a macOS menu bar app that lets users crop a region of a scanned document and OCR only that portion. It supports a local macOS Vision option and an AI-powered option for harder cases like creased, low-quality, or partially obscured text.
- •Problem: traditional OCR struggles with old scans and noisy documents
- •Solution: crop-to-target OCR for just the relevant article/section
- •Two modes: macOS Vision vs AI API for quality/comfort tradeoffs
- •AI can infer missing text but outputs still require later fact-checking
- 37:59 – 47:36
Learning mindset, industry concerns, and practical prompting tactics
Tim and Claire zoom out on how ‘vibe coding’ mirrors learning complex creative tools and why creative professionals may be well-suited to these workflows. They discuss ethical boundaries in nonfiction, job displacement fears, and Tim’s tactic of generating a ‘resume work’ prompt to reset stuck AI sessions.
- •Learning AI tools parallels mastering Photoshop/Premiere-style complexity
- •Nonfiction ethics: avoid generating fake archival footage; follow strict guidelines
- •Commercial video generation raises stronger job displacement concerns
- •Prompting tip: ask for a ‘resume work’ summary, prune it, and restart in a new chat
