By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
EO StudioThis Insane AI Video Search Technology Selected by NVIDIA and Snowflake | Twelve Labs, Jae Lee
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Great conversations run long. Their best ideas don't have to.
uListen is free on iPhone, iPad, Android, Chrome, and Claude.
See howCHAPTERS
Jump to publisher timestampsChapter summaries
Building video foundation models and a developer API for semantic video understanding
Jae Lee introduces Twelve Labs and explains what the company builds: large AI models that understand video in a human-like way and expose that capability via APIs. He frames the core value as mapping natural language to what happens in video to enable search, classification, and summarization.
- •Twelve Labs is an AI research and product company based in San Francisco and Seoul
- •The product focus is video foundation models delivered to developers and enterprises via APIs
- •The models aim to understand video like humans and connect language to video content
- •Capabilities described include semantic search, classification, and summarization
- •Jae cites an active developer base using the search API and mentions adoption across multiple sectors
Early curiosity in data and AI: from MATLAB to studying computer science
Jae shares his personal background and how early exposure to working with data sparked his interest in understanding the world through distributions and systems. He describes moving between Seoul and the U.S. and then studying computer science at Berkeley, where he “geeked out” on AI and engineering.
- •Jae grew up in Seoul and later moved to the U.S. as a child
- •An uncle’s MATLAB work with plots and distributions sparked his interest in data and systems
- •He became curious about capturing data and building systems to understand the world
- •He studied computer science at Berkeley and focused heavily on AI and software engineering
- •He describes living roughly half his life in Seoul and half in the U.S.
Founding roots in Korean Cyber Command: choosing startups over academia
Jae recounts meeting future co-founders during military service in Korean Cyber Command, where they bonded over AI research and debate. The team weighed academia versus starting a company and ultimately decided they wanted to build together beyond the military setting.
- •Jae served in Korean Cyber Command and met co-founders including the chief architect, SJ, and Aiden
- •They spent significant time reading papers, discussing, and arguing about AI progress
- •They saw two paths: pursue academia or start something themselves
- •The shared experience and intensity of military life reinforced their desire to build a company
- •They committed to exploring what frontier they could push as young AI researchers
Why video understanding: an underexplored frontier with massive demand
The founders identify video understanding as a major, less-explored research and product opportunity compared with text and image. Jae explains their belief that developers and enterprises need better tooling than object detection or transcription to make sense of exploding video data.
- •They spent about a year and a half deciding what frontier to pursue
- •Jae references the impact of the transformer-era shift and contrasts it with video’s research gap
- •They believed video and complex multimedia would become core internet data infrastructure
- •They saw a need beyond object detection and transcription for understanding video at scale
- •Jae describes the model goal: mapping human language precisely to what happens in video
Starting while still enlisted: bagel-shop prototyping and surviving unstable office setups
Jae details the logistical difficulty of launching with founders discharged at different times and describes building prototypes in a bagel shop outside the base. After brief access to a shared office that went bankrupt, the team moved into a tiny office until they could raise a seed round.
- •Founders were discharged on staggered timelines, making early collaboration difficult
- •They worked from a bagel shop near the base, carrying laptops in and out to prototype
- •A friend offered office space, but that company went bankrupt within weeks, forcing a quick move
- •They relocated to a very small office where five people worked for about six months
- •Jae reflects that naivete and excitement helped them push through early uncertainty
Betting on market acceptance while broke: making foundation models real to customers
Jae explains that early on even employees struggled to describe the company, and the team was uncertain whether the market would accept video foundation models. He describes starting with very limited money and focusing on getting exposure by working with early believers and finding a credible way to get noticed.
- •Early foundation-model concepts were hard to explain to people outside the space
- •The team believed the technology would work, but market acceptance was uncertain
- •They started with about $2,000 and had to choose tactics that maximized impact with minimal resources
- •They talked to customers and found early believers who built with their APIs
- •They decided they needed broader exposure beyond early customer conversations
ICCV competition gamble: spending scarce compute to win attention and credibility
Twelve Labs enters an ICCV video understanding competition to earn recognition and prove capability. Jae describes the fear of spending significant compute money quickly and credits the team’s execution for winning, which then sparked inbound interest from investors.
- •They chose to participate in an ICCV video understanding competition to gain visibility
- •Aiden spearheaded the effort with strong internal team support
- •Compute costs were a major risk, with Jae describing $200,000 in compute as a scary bet
- •Winning helped validate the work and made the company more visible to the broader ecosystem
- •Jae frames the lesson as needing a way to get on the map when others may share the same thesis
Fundraising after momentum: inbound VC interest, a rushed first deck, and scaling hard questions
After the competition win, Jae describes inbound conversations with investors and the scramble to create a first pitch deck for an early call. He shares a memorable scaling question about indexing massive volumes of video and how it forced the team to confront technical limits behind their big vision.
- •Post-win, investors with a multimodal AI thesis reached out inbound
- •Jae built an early pitch deck overnight for a very early-morning call
- •The storyline emphasized the size of the problem and the lack of adequate solutions for making sense of video data
- •A VC question about indexing extremely large volumes of content highlighted gaps between ambition and current capability
- •Jae says the company raised about $30 million in seed funding
Early customer lessons: don’t force a “yes,” find innovators who already see the product’s value
A co-founder, Soyoung, describes an early customer who paid but didn’t use the product, leading to a lesson about not pushing sales before customers are ready. She explains the importance of focusing on true innovators and using demos that let customers connect the technology to their own vision without heavy selling.
- •Soyoung recounts winning a customer who paid but didn’t actually use the product
- •She says the team over-optimized for turning “no” into “yes,” even when readiness wasn’t there
- •She suggests optimizing for hearing “no” sooner to find the right direction faster
- •The best early customers are described as true innovators across different fields
- •She contrasts overselling with showing a use-case demo that helps innovative customers define their own applications
Resisting mentor-driven distraction: staying grounded in infrastructure over “TikTok 2.0” ideas
Jae shares a regret about spending time chasing narrow application ideas influenced by mentors, despite having conviction about building foundational infrastructure. He emphasizes the need for founders to rely on core principles and say “no” even to respected advisors to avoid drifting away from what they truly want to build.
- •Jae says they made frequent mistakes and highlights one major regret about early strategic distraction
- •He describes blindly following startup mantras about narrow problems despite their foundation-model conviction
- •Mentor suggestions pushed them toward various “2.0” application concepts that didn’t excite the team
- •He argues founders need grounding and gut judgment to reject misaligned advice
- •He warns that accommodating all mentor advice can transform the company into something unintended
From cloud partnership to NVIDIA connection: how product fit turned into a deeper relationship
Jae recounts meeting Jensen at an Oracle event tied to Twelve Labs’ compute partnership and says the interaction led to greater visibility, including being featured at NVIDIA’s GTC. He describes subsequent conversations with NVIDIA’s venture team and frames the relationship as a natural match between NVIDIA’s direction in vision/video and Twelve Labs’ roadmap.
- •Twelve Labs has a multi-year compute partnership with Oracle Cloud Infrastructure using NVIDIA chips
- •Jae and Aiden met Jensen briefly at an Oracle partner event and discussed Twelve Labs
- •Twelve Labs was featured in NVIDIA’s 2023 GTC, which increased internal interest at NVIDIA
- •NVIDIA’s venture team reached out for casual discussions about the future of multimodal video understanding
- •Jae describes alignment beyond investment, including potential product partnership and synergy
Hiring philosophy and what makes a great engineer: perspective and values beyond coding
Jae explains he spends much of his time hiring and describes how he evaluates candidates through long conversations focused on values and product thinking. He argues that many people can code well, but the best engineers bring strong perspective on how the product and interfaces should evolve.
- •Jae says his current focus is mostly on hiring and meeting great people
- •He describes spending hours in conversation to understand a candidate’s core values
- •He looks for candidates he can learn from and who can articulate impact on the company
- •He states that some of the best engineers are not the best coders
- •He emphasizes product perspective, groundedness, and thinking about product/interface evolution
Near-term vision: horizontal video understanding infrastructure and real-time streaming
Jae closes with Twelve Labs’ vision over the next two years: becoming a horizontal layer of video understanding infrastructure for organizations working with video. He also mentions moving into real-time streaming to act as a “visual cortex” for modern video applications.
- •The goal is to become horizontal video understanding infrastructure for businesses and developers
- •He wants Twelve Labs to serve organizations working broadly with video data
- •He mentions entering streaming for real-time video data understanding
- •He frames the ambition as becoming a “visual cortex” for modern video applications
- •The focus remains on foundational capability that many applications can build upon
