Inside the expert network training every frontier AI model | Garrett Lord

Inside the expert network training every frontier AI model | Garrett Lord

Lenny's PodcastAug 24, 20251h 9m

Garrett Lord (guest), Lenny Rachitsky (host), Garrett Lord (guest), Garrett Lord (guest)

Pre-training vs. post-training in modern AI model developmentRise of expert-driven data labeling and RLHF/SFTHandshake’s pivot from student job marketplace to AI data providerStructural advantages of owning a trusted, at-scale expert networkBuilding and operating a zero-to-one startup inside a 10-year-old companyImpact of AI on early-career jobs and “AI-native” workersFuture of human-in-the-loop data, synthetic data, and evolving data types

In this episode of Lenny's Podcast, featuring Garrett Lord and Lenny Rachitsky, Inside the expert network training every frontier AI model | Garrett Lord explores handshake turns student network into explosive AI training data powerhouse Handshake CEO Garrett Lord explains how the company leveraged its massive network of students, graduates, and experts to build a hypergrowth AI data-labeling business on top of its decade-old core marketplace. As frontier labs shifted from broad internet pre‑training to post‑training with highly specialized human data, Handshake realized its 18M+ users—especially 500K PhDs and 3M master’s students—were an ideal expert supply. In under a year, the new Handshake AI unit scaled from zero to over $50M ARR, on track to surpass $100M and rival the $200M ARR of the original jobs platform. The conversation breaks down how post‑training works, why expert data is now the bottleneck, how Handshake structurally outcompetes legacy data-labeling firms, and how they successfully incubated a startup‑style business inside a mature company.

Handshake turns student network into explosive AI training data powerhouse

Handshake CEO Garrett Lord explains how the company leveraged its massive network of students, graduates, and experts to build a hypergrowth AI data-labeling business on top of its decade-old core marketplace. As frontier labs shifted from broad internet pre‑training to post‑training with highly specialized human data, Handshake realized its 18M+ users—especially 500K PhDs and 3M master’s students—were an ideal expert supply. In under a year, the new Handshake AI unit scaled from zero to over $50M ARR, on track to surpass $100M and rival the $200M ARR of the original jobs platform. The conversation breaks down how post‑training works, why expert data is now the bottleneck, how Handshake structurally outcompetes legacy data-labeling firms, and how they successfully incubated a startup‑style business inside a mature company.

Key Takeaways

AI gains have shifted from scraping the internet to post-training with expert data.

Frontier labs have largely exhausted the public web for pre‑training, so most performance improvements now come from post‑training—supervised fine‑tuning, preference ranking (RLHF), trajectories, and multimodal data built by domain experts.

Get the full analysis with uListen AI

The bottleneck is no longer cheap generalist labor but high-quality experts.

Models are now strong enough that low-cost generalist labelers add limited value; labs need top physicists, chemists, teachers, coders, and other specialists to locate failure modes and produce ground-truth reasoning paths in advanced domains.

Get the full analysis with uListen AI

Owning an engaged audience is the core moat in human data.

Where competitors spend heavily on recruiters and ads to find experts, Handshake already has trusted relationships and rich profiles for millions of students and professionals, enabling near-zero CAC, better targeting, and higher retention on projects.

Get the full analysis with uListen AI

Expert work is curated, trained, and measured like a scientific process.

Handshake builds training cohorts, instructional design, internal post-training teams, and GPU-backed evals so each unit of data is checked for quality and actual model gain before it’s scaled or sold across multiple labs.

Get the full analysis with uListen AI

Building a new AI business inside an existing company requires separation and founder-level focus.

Handshake AI was spun up with its own org, metrics cadence, hiring bar, compensation, and physical space, with Lord spending ~80% of his time on it and redeploying top talent solely to the new unit to avoid legacy drag.

Get the full analysis with uListen AI

AI is likely to amplify young, AI-native workers rather than eliminate them.

Lord argues employers see AI as an “Iron Man suit” that lets one person do the work of several—e. ...

Get the full analysis with uListen AI

Human-in-the-loop data will keep evolving rather than disappearing soon.

As models improve, demand will shift toward more complex data—tool-use trajectories, multimodal audio/video, CAD, scientific instruments—but labs and leaders still expect humans to be necessary for at least the next decade of frontier progress.

Get the full analysis with uListen AI

Notable Quotes

The only moat in human data is access to an audience.

Garrett Lord

The models have gotten so good that the generalists are no longer needed. What they really need is experts.

Garrett Lord

There will never be a time like this. I've never seen anything like it… where there's unlimited demand.

Garrett Lord

For as long as models are improving, humans will be needed in this process.

Garrett Lord

Being AI-native, young people are at a huge advantage.

Garrett Lord

Questions Answered in This Episode

How defensible is an expert-based data-labeling business once more companies try to build or aggregate their own expert networks?

Handshake CEO Garrett Lord explains how the company leveraged its massive network of students, graduates, and experts to build a hypergrowth AI data-labeling business on top of its decade-old core marketplace. ...

Get the full analysis with uListen AI

What specific metrics and internal evals does Handshake use to decide whether a given data pipeline is actually improving a lab’s model?

Get the full analysis with uListen AI

How might the balance between human-generated and synthetic data shift over the next five years across different domains (e.g., coding vs. medicine)?

Get the full analysis with uListen AI

What are the ethical and economic implications of students helping train systems that may later automate parts of their own prospective careers?

Get the full analysis with uListen AI

How can other mature companies systematically identify and incubate similar AI-native businesses built on assets they already own but underutilize?

Get the full analysis with uListen AI

Transcript Preview

Garrett Lord

There will never be a time like this. I've never seen anything like it. I doubt I'll ever feel anything like this in business again, where there's unlimited demand. How do you make sure that three months from now, six months from now, you have, like, no regrets? Get on a plane to go talk to a customer. Make the late-night push. Check the data six times over again.

Lenny Rachitsky

Your company creates new data to continue advancing the intelligence of models. This is a business that you built on top of a business you've already had.

Garrett Lord

We're the largest expert network in the world. We have this massive strategic advantage, which is like no customer acquisition costs. The only moat in human data is access to an audience.

Lenny Rachitsky

You guys come in after the model's trained to tweak the weights based on additional data that you create.

Garrett Lord

The models have gotten so good that the generalists are no longer needed. What they really need is experts.

Lenny Rachitsky

There's this tension between all these students training models to become smarter, and then there's the, they will have harder time potentially finding jobs.

Garrett Lord

That's not what we're hearing from our employers. This is just enabling human beings to be even more productive. You used to put, like, Google Search on a skill on your resume 'cause you, like, grew up with Google. Being, like, AI-native, young people are at a huge advantage.

Lenny Rachitsky

Today my guest is Garrett Lord. Garrett is the co-founder and CEO of Handshake, which is one of the most interesting and incredible AI success stories that you probably haven't heard of. Handshake has been around for over 10 years. They're essentially LinkedIn for college students. It's a place for students to connect with companies to find a job. They are the platform of choice for every single Fortune 500 company, over 1,500 colleges, over 20 million students and alumni, and over one million companies use them to hire graduates. At the start of this year, Garrett and his team realized that their huge proprietary network of students, including tens of thousands of PhDs and master's students, is extremely valuable to AI labs to help them create and label high-quality training data. So, they launched a new business from zero to one in January. Four months later, they hit 50 million ARR. They're now on pace to blow past 100 million ARR within just 12 months. They'll exceed the revenue that they're making with their decade-old business in under two years. This is a truly incredible and rare story, and one that I think a lot of teams can learn from, because AI is creating a lot of opportunity, but also a lot of potential disruption. And this is an amazing story where the company basically disrupted themselves. This episode is packed with insights, including a primer on what the heck are people actually doing when they're labeling and creating data to train models? A huge thank you to Garrett for making time for this. His wife just had a baby this week. He's also in the middle of scaling this insane new business. So thank you, Garrett. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. Also, become an annual subscriber of my newsletter. You get a year free of a bunch of incredible products, including Lovable, Replit, Bolt, N8n, Linear, Superhuman, Descript, Whisperflow, Gamma, Perplexity, Warp, Granola, Magic Patterns, Raycast, ChatPRD, and Mobbing. Check it out at lennysnewsletter.com and click bundle. With that, I bring you Garrett Lord. This episode is brought to you by CodeRabbit, the AI code review platform transforming how engineering teams ship faster with AI without sacrificing code quality. Code reviews are critical, but time-consuming. CodeRabbit acts as your AI co-pilot, providing instant code review comments and potential impacts of every pull request. Beyond just flagging issues, CodeRabbit provides one-click fix suggestions and lets you define custom code quality rules using AST grep patterns, catching subtle issues that traditional static analysis tools might miss. CodeRabbit also provides free AI code reviews directly in the IDE. It's available in VSCode, Cursor, and Windsurf. CodeRabbit has so far reviewed more than 10 million PRs, installed on one million repositories, and is used by over 70,000 open source projects. Get CodeRabbit for free for an entire year at coderabbit.ai using code Lenny. That's coderabbit.ai. This episode is brought to you by Orkes, the company behind open source Conductor, the orchestration platform powering modern enterprise apps and agentic workflows. Legacy automation tools can't keep pace. Siloed low-code platforms, outdated process management, and disconnected API tooling fall short in today's event-driven, AI-powered agentic landscape. Orkes changes this. With Orkes Conductor, you gain an agentic orchestration layer that seamlessly connects humans, AI agents, APIs, microservices, and data pipelines in real time at enterprise scale. Visual and code-first development, built-in compliance, observability, and rock-solid reliability ensure workflows evolve dynamically with your needs. It's not just about automating tasks. It's orchestrating autonomous agents and complex workflows to deliver smarter outcomes faster. Whether modernizing legacy systems or scaling next-gen AI-driven apps, Orkes accelerates your journey from idea to production. Learn more and start building at orkes.io/lenny. That's O-R-K-E-S.io/lenny. Garrett, thank you so much for being here. Welcome to the podcast.

Install uListen to search the full transcript and get AI-powered insights

Get Full Transcript

Get more from every podcast

AI summaries, searchable transcripts, and fact-checking. Free forever.

Add to Chrome