By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
Y CombinatorThings That Don't Scale, The Software Edition – Dalton Caldwell and Michael Seibel
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
CHAPTERS
- 0:00 – 1:32
Turning the Water On: Doing Things That Don't Scale in Software
Michael Seibel opens with a house-and-pipes analogy for launching a product and introduces the episode's theme of software hacks used when there was no time to build the right thing. He and Dalton Caldwell introduce Paul Buchheit's 90/10 solution idea.
- •Pipe analogy: turn the water on to learn what work needs doing
- •Episode covers software and product hacks from real companies
- •Paul Buchheit coined the 90/10 solution
- •Buchheit asks how to get 90% of the benefit for 10% of the work
- •Founders tend to dislike this pushback
- 1:32 – 5:44
Gmail's Scrappy Beginnings and the Invite System
Seibel and Caldwell recount how Paul Buchheit built Gmail by putting his own email into Google's Groups UI and adding features as he needed them. They also cover the outage story and how the invite system came from a hardware shortage rather than growth hacking.
- •First version reused the Google Groups UI to read his own email
- •Features like writing email were built only when he wanted them
- •A desk mate's feedback was that the inbox had Buchheit's email, not theirs
- •A corrupted hard drive took Gmail down one morning, showing how much people value email
- •The invite system existed because Google lacked servers, not as a virality hack
- 5:44 – 10:04
Facebook's Per-School Servers
Caldwell describes the era of physically driving to data centers and what he saw in his data center in Santa Clara when TheFacebook.com moved in next to Imeem. Facebook scaled by running a separate code and database stack for each school.
- •Startup founders had to physically reboot servers in data centers
- •Each school had its own labeled servers
- •Separate PHP, MySQL and Memcache instances per school
- •Users logged in at school-specific URLs like harvard.thefacebook.com
- •It took years to build a global users table
- 10:04 – 12:03
Twitch: Static Pages for 20X Traffic Peaks
Seibel explains why live video is hard: traffic peaks reach 20X the steady state, which can't be tested in advance. Their first hack was a button that turned any Justin.tv page into a cached static page.
- •Normal sites peak at 2-4X steady traffic; live video peaked at 20X
- •20X peaks can only be experienced, not tested
- •A button made any page static so it could be cached
- •Video and chat ran on separate systems and kept working
- •Smarter partial caching came much later
- 12:03 – 13:00
Friendster vs. MySpace: Faking the Extended Network
Caldwell shares an anecdote about Friendster computing friends-of-friends counts on every login, which strained its MySQL setup. MySpace's shortcut was to just label people as in your friends list or in your extended network.
- •Friendster computed second-degree network size live at login
- •Its founder considered the feature important
- •Friendster struggled with MySQL threads and Linux kernel issues
- •MySpace simply said someone was in your extended network
- 13:00 – 15:04
Twitch: Pre-Propagating Streams to Video Servers
Seibel describes the second Twitch hack: celebrity streams could crash video servers because the system didn't know how many viewers were waiting. The fix was for the website to tell the video system the viewer count so the stream could be propagated before being shown to everyone.
- •Popular streams need to be spread across many video servers
- •Slow-growing streams were handled by an algorithm, but sudden celebrity arrivals killed servers
- •Overly complicated modeling solutions were considered
- •The website knew viewer counts and passed them to the video system
- •Streams were pre-populated, then switched on with a delay of a couple of seconds
- 15:04 – 16:58
Imeem: A Video Site with No Video
Caldwell explains how Imeem played music in the browser by reusing open source Flash video tooling. Every music file was transcoded into a video file with no video, which he ties to the theme that good product decisions are often made fast and under duress.
- •YouTube's in-browser Flash playback was a major innovation over RealPlayer
- •Imeem reused video tooling rather than building music-specific tools
- •Uploads were transcoded into FLV files with audio only
- •Many good product decisions happen under pressure
- 16:58 – 20:21
Twitch: Free Peering Ultimatum and Crowdsourced Translation
Seibel covers two more Twitch hacks driven by lack of money. They blocked video for viewers on an ISP that refused free peering, and they built a community translation site instead of paying translation companies.
- •Live video was expensive and Twitch lacked funding
- •A former YouTube network ops hire taught them free peering with ISPs
- •Viewers on a non-peering ISP saw a block message after 10 minutes
- •Translation companies quoted prohibitive costs
- •Community volunteers translated strings, a solution borrowed from Reddit
- 20:21 – 23:36
Google's Stale Index and the Birth of MapReduce
Caldwell tells, as a secondhand story, how Google's batch indexing process began failing as the web grew, leaving the index stale for months without users noticing. That crisis led to MapReduce, which Google only built once it had to.
- •Caldwell notes the story is secondhand and details may be off
- •The batch re-index took weeks and started failing
- •Google served stale results for perhaps three to four months
- •MapReduce was created under duress, and Hadoop descended from it
- •Google never shut down search and built MapReduce only after it had users
- 23:36 – 25:55
Startup Lessons: Earning the Privilege to Scale
Seibel returns to the pipes analogy, saying startups mostly turn the water on, fix what breaks, and repeat. Caldwell adds that companies earn the privilege to build scalable things by first making something people want, and Seibel wraps up by tying the software examples to non-software ones.
- •Big companies are built by fixing what breaks, not by following a written plan
- •Caldwell contrasts Wozniak's hand soldering with the AirPods
- •Google earned the right to build MapReduce through a good search product
- •Don't make the perfect the enemy of the good
- •Airbnb and DoorDash examples show things that don't scale work outside software too
