Lex Fridman PodcastTim Sweeney: Fortnite, Unreal Engine, and the Future of Gaming | Lex Fridman Podcast #467
EVERY SPOKEN WORD
150 min read · 30,191 words- 0:00 – 3:06
Episode highlight
- TSTim Sweeney
Humans are by far the hardest part of computer graphics because millions of years of evolution have given us dedicated brain systems to detect patterns in faces and infer emotions and intent. Because cavemen had to, uh, when they see a stranger, determine whether they were likely friendly or they were- might be trying to kill them. Um, and so people in the world have extraordinarily detailed expectations of a, a face, and we can notice imperfections, especially imperfections arising from computer graphics limitations. Okay, one part is capturing humans, and so we've built really advanced, dedicated hardware that puts a human in a capture sphere with dozens of cameras and then taking high-resolution, high-frame-rate video of them as they go through a range of motions. And then capturing the human face is complicated because the nuanced detail of our faces and how all of the muscles and sinews and fat work together to give us different expressions. So it's not only about the shape of a person's face, but it's also about the entire range of motion that they might go through. So that's the data problem. There's a lot of other problems with computer graphics, you know. There's technology for rendering hair w- which is really hard 'cause you can't render every... A- again, we know the laws of physics. It would be easy to just render every hair, it would just be a billion times too slow. Um, so you need approximations that capture the net effect of hair on rendering and on pixels without calculating every single interaction of every light with every strand of hair. Um, that's one part of it. There's detailed features for different parts of faces. There's subsurface scattering because we think of humans as opaque, but really our- our skin is... Light travels through it. It's not completely opaque, and the way in which light travels through skin has a huge impact on our appearance. You know, this is why you... There's no way you can paint a mannequin to look realistic for a human, uh-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... you know? It's just a solid surface, um, and will never have the sort of detail you- you see.
- LFLex Fridman
That kind of blew my mind, like, thinking through that. Hmm, I think I heard that sort of the oiliness of the skin creates very specific nuanced, complex reflections, and then some light is absorbed and travels through the skin and that creates textures that our human eye is able to perceive, and it creates the thing that we consider human, whatever that is. All of that while considering all the muscles involved in making a nuanced expression, just the subtle squinting of the eyes or the subtle formation of a smile. It's the subtlety of human faces that you have to capture (laughs) . Like, the difference between a real smile and a fake smile. But the way they show, like, beginning of a formation of a smile that actually reveals a deep sadness. All of that, like when I watch a human face, I can, like, read that. I could see that. You have to have the tools that in real time can render something like that, and that's incredibly difficult.
- TSTim Sweeney
That's right. Getting faces right requires interplay of literally dozens of different systems and aspects of computer graphics. And if any one of them is wrong, your eye is completely drawn to that, and you find it on the wrong side of Uncanny Valley. (graphics whooshing)
- 3:06 – 3:39
Introduction
- TSTim Sweeney
- LFLex Fridman
The following is a conversation with Tim Sweeney, a legendary video game programmer, founder and CEO of Epic Games that created many incredible games and technologies, including the Unreal Engine and Fortnite, which both revolutionized the video game industry and the experience of playing and creating video games. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's Tim Sweeney.
- 3:39 – 6:56
10,000 hours programming
- LFLex Fridman
When did you first fall in love with computers and maybe with programming?
- TSTim Sweeney
I had a brother, Steve Sweeney, who, 16 years older than me, and, um, at some point when I was a little kid he went off, uh, to work in California for a tech company, and he'd gotten one of the first IBM PCs. And so for one summer, I think I was about 11, I went to visit him in California. It was my first, like, trip away from my family just to hang out with him, and he had this brand new IBM computer, and I- I learned to program over the course of a few days in BASIC. And I was just blown away with the capabilities of computers at the time. It was unbelievable what they could accomplish, and, uh, I was just hooked from that point onward and very much wanted to be a programmer.
- LFLex Fridman
Do you remember what you wrote in BASIC? Was it, uh, video game-type things? Is it, like, for loop some numerical thing? What, do you remember?
- TSTim Sweeney
Yeah, it's funny. I- I have a perfectly vivid memory of all of the first things I learned to program. (laughs)
- LFLex Fridman
Okay.
- TSTim Sweeney
I have a hard time remembering people's names, but, like, code really sticks with me. At every step and every challenge, there were lessons learned. And, you know, some of which I- I've come to realize were just, like, me, uh, getting over some learning hurdles. But other things were actually shortcomings of programming languages, uh, and the realization that there are actually better ways than what, uh, programmers learning to program for the first time. Uh, you know, a lot of what they're facing isn't the challenge of learning a new art. It's the friction introduced by failures of programming language design. And so I've- I've constantly come back to those early lessons there as I've- as I've progressed and done more and more things, including building programming languages.
- LFLex Fridman
Yeah, the friction and the pain is- is the guide to learning in programming. Like if- if I were to describe programming journey, uh, that would (laughs) be marked by pain. And that pain, you- you shouldn't escape the pain. The pain is instructive for you to understand programming languages. But do you (laughs) , do you remember what kind of stuff you were writing, um, at that time, just the early programs?
- TSTim Sweeney
Yeah, in the early days, I wrote a little bit of everything. I wrote some games. Uh, the first game I wrote on the Apple II was, uh, since I only knew how to program in text mode, um, the computer would, uh, throw asterisks across the screen. They'd flow from left to right, and you'd have a parentheses on the right-hand side of the screen and, you know, looked like a baseball mitt, and you're supposed to catch the asterisks. That was- that was my very first game. It took about a couple hours, uh, to build and tune, um, and I went from there. But I built a lot of things. I built...... databases at different points. I built a programming language and a full compiler for a language like Pascal, 'cause, uh, like, I couldn't... I didn't know where you went to buy one (laughs) of those, so I made my own. Um, and, you know, one of the things of, one of the fun things of that time was bulletin boards. Uh, before we had the internet in the hands of consumers, you used your modem, um, and you dialed into a, a local phone number, um, and connected to whoever was running the computer there. And every town or city had hundreds of these bulletin boards run by different people with their own personalities and themes. Um, and so I spent a lot of time building a bulletin board program and learning how to deal with database management and user interface and dealing with multiple users concurrently and things. And so, uh, I'd... I don't know. I probably found about 10 or 15,000 hours, uh, writing code just on my own as a kid between, like, age 10 and, uh, and, you know, age, uh, age 20 before I actually shipped a program to the outside world.
- 6:56 – 15:07
Advice for young programmers
- TSTim Sweeney
- LFLex Fridman
10 to 15,000 hours. What was the value of the hours as a kid you put in, in programming that led to the success you've had in later life? Maybe this is by way of advice to younger people in terms of how they allocate the hours of their early life.
- TSTim Sweeney
Yeah. You know, it's not just hours. It's really striving to learn, to understand what knowledge you have, what knowledge you lack, and to continually do experiments and work on projects that improve your knowledge base. And I didn't do this with a great about of structure or planning. I was rather just going from project to project, doing things that I thought would be fun and cool. And with each project, I learned new things, you know, learning about how to store and manage data, learning how to deal with advanced data structures, how to, uh, write complex programs that have deeply nested, uh, data and control flow. Um, each one does, you know, provide a lesson, uh, which were later essential. You know, when, uh... In 1991, I released my first game, and over the, the course of that decade, we went from, you know, zero commercial releases to the first generation Unreal Engine. But, you know, this was largely just using the knowledge that I'd built up over the previous decade, um, just doing fun hobby projects. Um, if I hadn't done all that work, there's no way I could have ever built the things that came later.
- LFLex Fridman
All the experimentation and all the exploration somehow contributed, somehow made sense later on. Like, all of that is integrated somehow in the stuff you build. It's funny how life works. Like, it... the pieces kinda come together eventually.
- TSTim Sweeney
Yeah. You know, there are definitely Karate Kid moments, uh-
- LFLex Fridman
(laughs)
- TSTim Sweeney
... 'cause, you know, all this time I was learning math in high school and in college. I studied mechanical engineering. And so, you know, you learn all kinds of math, vector calculate- calculus, and vector math, um, and matrices, and, uh, you know, all these related fields. Uh, physics and stress and strain, and, uh, how to, you know, deal with complex physical systems. Um, and yeah, I wasn't really sure how, uh, engineers would actually make use of that knowledge (laughs) . Do you just, like, forget about it when you actually go off to do work, or is it... do you write down equations on paper? It was actually not clear (laughs) as an early engineering student what you do. But when I started writing the first generation Unreal Engine and I was dealing with 3D maps, I was like, "Wait, I know this stuff."
- LFLex Fridman
(laughs)
- TSTim Sweeney
"I learned this (laughs) ." And, you know, so, you know, suddenly like the Karate Kid, you know, you get to paint the fence and, uh-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... wax the car, and suddenly p- put all the pieces together into, you know, a 3D engine based on whole lot of accumulated programming language and math knowledge that often, uh, often knowledge gained without ever anticipating that I might use it in that way.
- LFLex Fridman
Also, I think what's useful is over and over learning a hard thing and then showing to yourself, you know, uh, that you can do it, that you can learn a hard thing. So then when you come to having to write a 3D engine that, uh... in ways that haven't been done before, you're like, "I've d- I've been here (laughs) . I've been here in this experience. Like, I don't know what to do, but we'll figure it out. We'll learn. I'll learn all the necessary components." So just not being afraid of something new.
- TSTim Sweeney
Yeah, that's right. And constantly striving to make connections between these fields and look for their applications. Long after I shipped on Unreal Engine, it was like going back through an engineering textbook and looking at, "Oh, yeah, I used that, I used that, I used that."
- LFLex Fridman
(laughs)
- TSTim Sweeney
And then I got to the section on-
- LFLex Fridman
Yeah.
- TSTim Sweeney
... eigenvalues. I'm like, "Don't know what the hell this is." Um, but, you know, it turns out eigenvectors and eigenvalues were the critical breakthrough that made the Google search engine technology work and stand apart from the rest, because they found if you throw all the links that exist in the web and, you know, links from and to different sites, and you put them in a giant matrix and you conclude it, uh... You found the dominant eigenvalues, then those eigenvectors described the best search results for different things. And so constantly picking up knowledge and looking for ways to put it together is, is the thing to do. And if you aspire to be a programmer, you've got to write a lot of code and you've got to continually learn new things and improve. If you wanna be an artist, you've gotta continually draw artwork, all styles and all kinds, and constantly push yourself to learn more and more, uh, because you, you never know exactly what you're gonna end up doing in the long run. But the more knowledge you have and the more skills, the more chance you have putting it together and being successful.
- LFLex Fridman
And whether you're a programmer or an artist, you should probably take linear algebra, even though it doesn't make sense at the time.
- TSTim Sweeney
I found getting engineering, an engineering degree and then never working in an engineering field, uh, you know, just being a computer programmer was immensely valuable. Um, yeah, I went to University of Maryland, which, uh, for some disciplines, it's kind of known as a party school, but they work the engineers to death.
- LFLex Fridman
(laughs)
- TSTim Sweeney
Work really hard, and if you learn any engineering discipline, you learn massive amounts of math and you learn the rigor of problem-solving. You know, not just what you find from the Wikipedia article, but going through all the exercises of solving, you know, complex problems and building up series of solutions to, to arrive at an answer. It's, it's valuable, and it is... it embodies the knowledge that you need as a programmer. And, you know, people often go to university and think, "Okay, my goal here is to get good grades so I get a diploma and I prove to an employer that I'm valuable." Like-... no, that's just kind of the superficial bookkeeping of the university. The real purpose of all of this is to learn. And whether you learn formally or you learn on your own, it's the learnings that are really valuable in a career. Um, and especially if you're going to be an entrepreneur, it's really knowing the stuff that matters and not having the, the diplomas and the... yeah. Th- there's ever more pressure to make a- s- build- rebuild society more and more around credentials. Do you have this certificate? Do you have that proof? But, like, you know, companies th- are focused on just building great products and doing great things, uh, gravitate towards people who do the great work.
- LFLex Fridman
Yeah, one of the great things about youth (laughs) is, uh, there's more freedom, there's just more time to learn. And people, when they go to high school, they sometimes think, "Wow, I can't wait to get out of this and be an adult and be free," but it's not quite freedom. When you get a job and you start a family, all wonderful things, but you get le- more and more busy and less and less time to learn in the general sense, learn whatever the hell you want. And that, that is a wonderful time in life, th- the teenage years, the early 20s, the 20s, when you could just learn random shit.
- TSTim Sweeney
Yeah, you know, and I think this is something that's kind of changing in America. Um, there's so much focus on grades and homework and, um, structure around kids' lives. You know, when I was growing up, you know, my mom would feed me and my neighbors bre- you know, my ne- their- my neighbors' moms would feed them breakfast and they'd, you know, be like, "Well, be back by dark." Um-
- LFLex Fridman
(laughs) Yeah.
- TSTim Sweeney
... and, you know, we'd go out and we'd play and we'd do all sorts of things. We'd, you know, explore the woods, we'd build g- go-karts, we'd, uh, you know, salvage old pieces of electronics and build what we thought were our space shu- uh, spacecraft control panels, um, for the (laughs) you know, spaceships we were building as play. And, uh, we'd have an enor- enormous amount of freedom. And, uh, you know, from basically being a little kid through, um, through the time I went off to college, um, I had an enormous amount of free time. A- and some people would just use that and waste it and watch TV, some people socialized, um, and some people really got into serious projects. Uh, so many people, at all times, were doing cool things. You know, I was programming, I was learning to build things, I was, uh... You know, before I was releasing games to the world, I'd be, like, you know, having neighborhood folks over to play the things I was working on and check them out, and sometimes they're impressed and sometimes they weren't. Um, and they'd have their own projects, and often we'd have spare time jobs and everybody was entrepreneurial. Like, everybody, you know, had a side gig. Sometimes you'd go around and mow people's lawns, uh, or you'd, you know, you know, rake the leaves up and, you know, earn money. And th- the freedom there and the, the organic learning that occurred there, I, I think is something that is really critical to the American experience that I, I worry is increasingly going away as, uh, society's ever more protective and sheltering, um, and makes it harder to get these experiences.
- 15:07 – 17:16
Video games in the 80s and 90s
- LFLex Fridman
So, on the video game side, when did you first fall in love with video games?
- TSTim Sweeney
I've had a funny relationship with games, because my real aspiration has always been to program cool stuff. Uh, I get more enjoyment out of programming than anything else in the world. Um, and so, you know, th- my first really two formative experience with games were playing, uh, this game called Adventure for the Atari 2600. It was like, you moved this dot around the screen and picked up objects like swords and fought dragons and invaded castles and solved puzzles.
- LFLex Fridman
Nice.
- TSTim Sweeney
Uh, very, very simple, iconic stuff, you know, rather than realistic graphics. And then the other game that I really got immersed in was Zork, uh, which was a text adventure game. It would tell you where you are and what you see and you'd type in commands like, "Go north," or, "Pick up sword," or, "Open door," and explore a world that way. So the game didn't have any graphics, but in your mind you had this elaborate picture of what you were seeing there and, uh, it really brought in and inspired imagination more than other things. And playing those games led me to go off and want to learn to program everything that I saw there, um, and that drove a lot of my programming. I learned how to move a player around the screen, I learned how to, you know, build a design tool so I could build castles and save them off and play them in a game. And I realized there was a separation between the tools that you use to build a game and the game itself, and that if- the more powerful tools you had, the more creativity you could unleash in yourself or others. Um, and, you know, I learned all the programming techniques that supported games. How to parse text, you know, "Pick up sword and go north." How do you make that sentence into, uh, an actual series of commands on the computer? Um, and that was really, really exciting. Um, I have to say, u- until the time that Fortnite came out, I played video games primarily to learn the- what they were doing so I could go off and do it myself.
- LFLex Fridman
(laughs)
- TSTim Sweeney
You know, I'd, I'd sit down, you know, when Wolfenstein came out and then Doom came out, um, and I'd go through and look at it pixel by pixel. I'd move the mouse very slightly and look at exactly what was happening to figure out-
- LFLex Fridman
(laughs) That's fun. That's great.
- TSTim Sweeney
... what technique was being used there. Th- that was puzzle solving at a grand scale, and it was so fun.
- 17:16 – 29:54
Epic Games origin story
- LFLex Fridman
Uh, so, so take me there in, uh, the early '90s. So you launched Epic Games in 1991. (sighs) So you're... The writing of your first big video game, uh, ZZT, what was it like? What was the technical challenges? What were the s- the psychological challenges of building that?
- TSTim Sweeney
It was a funny project because I didn't start out to build a video game. Um, I'd just moved from an Apple II that... Uh, so my, my brother bought my family an Apple II right after I visited him in California, so I'd been programming on that for a few years, learned a lot of techniques. But there weren't many Apple II users around still, uh, by the time that cycle came to an end. Um, and so I'd just gotten an IBM PC, uh, of my own. Um, and I was learning to program and I realized I needed a text editor, so I started writing a text editor. You know, a text editor is a program, uh, to ex- edit text files. You have logic to move the cursor around and let people type things and backspace and delete, and do all those, you know, mundane actions. And, you know, one night I was like... I finished it up and I was like, "Well, okay. I have a text editor, but this is pretty boring." And so I made the cursor, uh, into a smiley face character, and I had the... like, different characters you could place in this document performed different gameplay actions. Some would be walls, and some would kill you, and some would, uh, be moving objects that could, uh, fly around the screen. And so this text editor I made evolved into a little game editor as I was building these levels for a game. I put a lot of time into, like, building an editor and a, a primitive set of objects, uh, about 20 or 30 different objects. Enough to build a really cool and compelling game, but not so many that players would lose track of what they're seeing. And I started off just building different, uh, game levels. You know, the idea is you'd be on a series of board, they'd be connected by, you know, going north. Past the end of the current board would take you to a new one if it was open, or maybe it was blocked and you couldn't go there. I built this whole game world around that, and, you know, this was the game that became ZZT. And, uh, I was having fun with it, uh, building it and playing it, but I didn't know if it would really work, so, uh, I did this experiment. I started inviting neighbors over, uh, like some adults, some kids, all different ages, and I sat them down in front of it and said, like, "Here's a game I made. Uh, figure it out." (laughs)
- LFLex Fridman
Yeah.
- TSTim Sweeney
And, you know, I had to force myself not to tell them what they needed to do, right? Because, uh, I really wanted to learn if, if they were able to, uh, you know, discover it all for themselves. You know, today we would call this a, you know, user experience test, um, and there's a whole field of research around user experience-
- LFLex Fridman
Yeah.
- TSTim Sweeney
... research. But back then, it was just inviting some kids over to play the game. I took notes about what they got stuck on and what they enjoyed and where they felt bored, um, and just iteratively polished the game until I felt it was good, and I put it out, um, and released it on... well, this was before the internet, so there were bulletin boards. I uploaded it to a bunch of local bulletin boards and, um, from there, it started spreading because, you know, the way to build up cred for bulletin board users was to upload new files and to claim that, "Hey, I was the first that brought this to you."
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
And, uh, you know, so there was a natural tendency of the software to spread. And I decided to use the shareware model, you know, so I didn't just build this one game. I built, uh, a trilogy of three games. Um-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... and I released the first one for free, and I said, "Hey, if you like this, buy the two sequels." Um, and, and I, I included my parents' mailing address and said, you know, "Send us, uh, $30 and, uh, you can get the sequels to this game." And, uh, the checks started coming in, uh, within a few days. (laughs) I was making, like... getting three or four orders a day. I was making, like, $100 a day and I'm like, "Whoo! I'm rich!" (laughs) 'Cause, you know, being a 20-year-old, that was, like, (laughs) a pretty big deal.
- LFLex Fridman
What did that feel like just getting money and probably feeling this immense success from something you created?
- TSTim Sweeney
Well, you know, I always looked at money always just as a tool to help you fund accomplishing cool things, um, and, you know, having enough to do the things you wanna do is the critical thing. Um, it's always been just very utilitarian. But the knowledge that other people all around the country and all... then, you know, and a month later, all around the world were playing the game, that was, that was mind-boggling. You know, that me, like, this, this little kid who had put out a game on a local bulletin board, uh, could be doing international business and shipping discs all over the world, um, to players, you know, because the software was spreading on its own. Uh, it was just magical. Like, that... and that was a new thing for software. Like, that did not happen with mechanical devices. Like, you manufactured one, you sold it to somebody and they had it and that was it. But software could spread. Um, that was just really cool to see, and it made me realize there's really no upward limit on the potential for a business like that. You know, we saw Microsoft as a big juggernaut company at, at the time, but I was like, "Hey, you know, if Epic does games good enough, you know, we could accomplish, uh, what they have accomplished with operating systems," and the sky was the limit. Uh, and I, I think this is the, the age we live in now. It's... you don't have to be an industrialist manufacturing physical products. Uh, anybody who builds anything, um, digitally, if it's good enough, you can reach the entire world and build a, you know, next Microsoft or Meta or Apple or Google or, or Epic Games.
- LFLex Fridman
That's such a cool origin story though. So you start out building a text editor. So you're looking at this project, you're playing around with it, you're building up the tools. It's, it's such an inspiring moment 'cause, uh, a lot of us start out building a project and to allow yourself to see the potential pivots, the potential trajectories it can go is really nice. Just sit back, allow yourself to be bored, and like, "Ah, I'm gonna go this way." I mean, that's like a crossroads. (laughs) You came to a crossroads. I mean, you built, uh, you know, compilers of... You p- you designed your own programming language, you built compilers, databases, all these things you mentioned, and you started building a text editor, and then here came to this crossroad. "I wanna make this fun." (laughs) And then from there, you know, one of the most legendary gaming companies was created. It's kinda cool. Like, that, that's an inspiring thing for sort of developers. Like, be open to the possibility of creating something you didn't plan to create and just go with it, right? That's cool.
- TSTim Sweeney
Yeah. And it was... a bunch of learnings emerged really quickly there. The neat thing I did with ZZT was I didn't just release the game. I also released the editor with it.
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
The... I built this tool so I could make these ZZT boards that people could play, but I also gave it to all of the players themselves, and, um...... you know, like, 30 years later, I still run into people. You know, when I go to a game industry event, it was like, "You know, I grew up playing ZZT." (laughs) And you know, "Here's an adult who grew up playing my game." (laughs) And it was because it enabled anybody to become a creator too. It had, you know, this little board editor and it also had a little scripting language, so you could learn a little bit of programming in it too, and, um, it- it kind of impressed and... It really set a formative principle with Epic, which was that, you know, the company's mission is to make awesome entertainment, but also awesome tools and to share those tools with everybody so that they can build their own amazing things too and, um, yeah, when we got into Unreal Engine a few years later, uh, the interplay between us building a game and us building the tools, uh, tools that were widely used by others was a critical part of that. And I think that's the sole reason that Epic has been massively successful and the f- actually the reason that we've survived all of this time is that by serving both creators and gamers, um, we've been able to weather the ups and downs of the game industry. It's a, a brutal place, uh, for companies. Um, we've been able to survive every financial downturn and sometimes the engine's been funding the business because we didn't have a game and sometimes the games have been funding the business, and, uh, it really set a principle in our culture that, uh, that's per- persevered and is continually brought to the forefront.
- LFLex Fridman
But on the editor front, that's such a fascinating philosophy that you always allow people to create their own worlds. You have an engine from which you simulate the world that the game is in, you have the actual game, and you also have the freedom for creators to create various, you know, in Fortnite, islands, uh, of their own. So it's like with- with everything you ship, that f- that freedom to create is always there. That's really interesting.
- TSTim Sweeney
Yeah. And something we- we aim to do more and more fully over time, you know, in the course of building Fortnite, we've built a lot of other tools that are useful for us too, 'cause it's not just a game powered by Unreal Engine, but it's also, you know, a social ecosystem where people can make friends and voice chat and g- get together in parties, and we've opened up all of those social features into Epic Online Services and we give them away to all developers for free because we all benefit from growth in that user base, um, and, you know, our- our goal is ultimately to build the company's products on the same technology that we share with everybody else and to hope that foster a bigger and bigger ecosystem over time where everybody benefits.
- LFLex Fridman
If we could just linger on the '90s. (laughs)
- TSTim Sweeney
(laughs)
- LFLex Fridman
Uh, so you said bulletin boards. Maybe you can explain what that's like and also explain the birth of the internet, what that was like. What was the- what was the internet like in the '90s?
- TSTim Sweeney
So the internet is a funny thing. It started out as this defense department, um, research project called the ARPANET, the A- Advanced Research Project Agency Network, and, um, it was kind of like this revered secret thing. (laughs)
- LFLex Fridman
(laughs) Yeah.
- TSTim Sweeney
Uh, then it became more and more open as they connected universities. Uh, universities connected to the internet in the, you know, mid-1980s. And so if you were at a prestigious institution with access to computers, you could get on there, but the consumer back then, we just had these modems. You know, this thing you plug into your phone line, um, and it dials up a phone number and then, you know, it sends g- wild sound effects over the (laughs) over the telephone line to, uh, send digital signals back and forth, and these were really slow. Three- you know, their first modem I had was 300 baud, so that means 30 characters per second-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... of data. So you're, like, sitting there w- watching a sentence, like, slowly emerge character by character as you're going online. But yeah, that's how we got online and we talked with each other. So you dial up, uh, to a local bulletin board, it'll be run by a person, usually they have a computer or two sitting in their kitchen or something that's running the bulletin board and, um, they have a small community of a few hundred users, um, all competing to connect to that one phone line. Um, it was often busy and you couldn't get in and, uh, the more popular bulletin boards were hardest to get to. Um...
- LFLex Fridman
Nice.
- TSTim Sweeney
But yeah, all kinds of communities develop, you know, and you could see, like, there was the programming communities where people talked about programming, there was the news and events, you know, uh, community. I was lived in the outskirts of Washington D.C. so that was, like, a big thing, but then there was, like, the pirate community where they were sharing pirated Apple II games and, you know, very different, uh, community eth- ethos and- and mantras out there, but all, uh, you know, all really nice and also very small. Um, these things, these bulletin boards couldn't grow to the size of Facebook 'cause your phone line couldn't take that many calls, um, and you know, then, uh, then later in the 1990s, the- the internet, which had been fostered in these colleges started opening up to the public and anybody could connect to it and suddenly the world took on a life of its own. It became much, much easier to reach global audience faster.
- LFLex Fridman
And you would start shipping games to the internet, which is a bit of a crazy thing to do 'cause you're s- supposed to have, like, a, you know, a physical copy but to- to post on the internet is pretty innovative. Even shareware is pretty innovative.
- TSTim Sweeney
Yeah. You know, it's been a funny transition for the game business, you know, Epic started out making shareware games distributed digitally, um, but you know, as- as the first 3D games took off, like, you know, Wolfenstein and then DOOM from id Software and then Unreal from us, um, took off, you know, to reach a huge audience of millions of users, we had to go into retail stores, so we worked with a retail publisher and they made a box and we put CD-ROMs in the box and, um, and, you know, then the world started transitioning back to digitally, like, and- and that transition didn't start well, right? The- the ori- initial transition of gaming to digital was all BitTorrent, all piracy, um, and the other horror stories about games that would, uh, you know, uh, sell, like, 100,000 copies but have two million users-
- 29:54 – 35:47
Indie game development
- LFLex Fridman
Uh, but maybe just staying in the '90s, uh, there are gonna be a lot of indie game developers who listen to us talking today. Can you, uh, go back to that mindset and try to derive some wisdom and advice to those folks when you were just a solo developer, maybe just a small group of people, uh, creating your early games that eventually became this, uh, huge gaming company? But in the early days, what, what, uh, what were you going through? What were the ups and downs? Uh, what did it take to sort of stay strong and persevere?
- TSTim Sweeney
Well, you know, one of the critical things that Epic always worked hard to do was to make something different, um, that nobody else was doing, um, and to, uh, try to satisfy a small audience rather than competing globally with the game juggernauts. You know, back in the 1990s, Epic was new but Electronic Arts and Activision and the other big publishers had been around for a decade and they were huge companies, um, that had giant retail distribution networks. You know, if I tried to make a game and then convince them to publish it, I, I doubt I could've had a chance and I doubt, uh, th- if, even if I had made a successful game that, uh, I would've made much money from it, though they might have. Um, and yeah, so the really unique angle to Epic then was shareware. And that was just the idea that if we distribute our game differently then we can reach a much larger audience than these bigger competitors by virtue of this first episode of the game being free. You know, it was kind of the advent of what later became free-to-play, um, and the logic of that is just as true now as it was then. It's if the thing is free and anybody can get into it, then it's gonna spread from friend to friend-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... as people bring, uh, you know, their real world friends into, into the games they're playing and, uh, you know, have the opportunity to build up a community around that. Uh, you know, so the other li- lesson there was minimize the friction of people getting into your game. Make it easy to get into and make it fun. And I think the other, well, I was very fortunate. Uh, ZZT was a funny game. It was not like (laughs) , much like any other game. It was, uh, had much worse graphics because it was all just text characters, uh, smiley faces, and, you know, other (laughs) Greek letters and things participating in this game simulation. They were kind of iconic representations of characters rather than real ones, and, you know, this was decades into the age of real graphical games with interesting graphics. Um, and so it wasn't even trying to compete in that area, but it was able to compete in a different area which is that, you know, it wasn't just my, the three, uh, games I'd made and shipped as a trilogy that were successful and drove the success of the product. It was the fact that I released an editor and there was a whole community around it. And you see that, that, that trend has repeated itself. Like there was, you know, ZZT was one of them. Before that there was Bill Budge's Pinball Construction Set. That was a 1980s Apple game that let users build their own pinball tables. And since then you've had some of the world's most successful games follow that path, like Minecraft, you can build your own stuff, Roblox, now Fortnite Creative and Unreal Editor for Fortnite. You know, games that become platforms for other people to build stuff was a real opportunity. You know, I think the big thing to realize th- is for indie developers right now is like there's m- massive, massive competition in every major genre and so, uh, it's very unlikely unless y- you just happen to be the world's best at a particular thing that you're gonna release a game in an existing highly competitive genre and when, um, a much better chance of success, uh, is in releasing something that hasn't been done before. Uh, being really unique and reaching an audience even if big or medium-sized or small, um, reaching an audience and becoming really popular with that. Making some money from it and being able to reinvest and then expand towards your ultimate dream. You know, I think the one shot, uh, go from idea to commercial success at massive scale is a lot less likely, uh, than the multi-step process of continually build better and better stuff over time until you get into a position of excellence.
- LFLex Fridman
And constantly try to do something that others aren't doing.
- TSTim Sweeney
Yeah, that's right. It, 'cause if you look at every market, um, there's a few markets where the current leader came late to the space, um, usually because the, the prior leader had failed so horribly, um, but most of the time the, you know, the company that's succeeding and winning in a market is the first or second entrant there, um, they've just continually buoyed their success.
- LFLex Fridman
Great advice and fascinating, but on a human level, was it lonely? Was it scary? Y- you sitting there as a developer?
- TSTim Sweeney
I'd say it was, uh, it was the opposite of lonely because, uh, you know, the thing that spurred me to actually release this was seeing kids playing the game in my neighborhood and having fun and being like, "This is really good." Um, and seeing them enjoying it and laughing and pointing at the screen and, you know, getting together and just wanting to play more.
- LFLex Fridman
That's awesome.
- TSTim Sweeney
Um, yeah, so, and, and the human element was always pervasive, you know, 'cause I'd, I'd not only receive orders but people would actually write letters, you know. We wrote letters back then in the (laughs) 1990s. Um, people would say how much they're enjoying the game and how their kids were playing the game and so on and so on. Um, so, you know, it felt, felt very connected. Um, and, you know, I think a lot of businesses have to make scary decisions, uh, because you're spending, you know, potentially all of the money you have to take a shot at something that you're not sure will succeed. Uh, I was very fortunate starting a business like this because it didn't really need any capital. The capital was, well, the several thousand dollars in computers I'd bought by mowing lawns (laughs) .
- LFLex Fridman
Okay.
- TSTim Sweeney
Um, and, uh, it wasn't much risk. If that hadn't succeeded, uh, I guess I could've figured out how people get mechanical engineering jobs and pursued that. But, um, once it took off...
- LFLex Fridman
Wow.
- TSTim Sweeney
... and once the, once the orders started coming in and people started writing letters saying they're enjoying the game, I knew I was gonna go all out and try to build a company there and succeed and that was like gonna be, you know, my big goal.
- 35:47 – 1:01:44
Unreal Engine
- LFLex Fridman
So I'm sure people know, but, uh, Epic Games was created in 1991 and went on to, uh, transform the gaming industry several times, uh, one of which is Unreal Engine. So let's talk to the origin story of that. You said that, uh, when Wolfenstein and, uh, Doom came out, that changed everything. So take me to that moment.
- TSTim Sweeney
Yeah, that, that was a very interesting time. E- Epic had, uh, after my first couple of games, uh, had recruited developers, you know, usually college students, high school students who were just working on their own, had real skills, uh, but didn't have an outlet for their work. Um, Epic had been matchmaking the best artists and programmers together from all over the world. Like, Jaz Jackrabbit was Cliff Bleszinski, a high school kid in California who'd made a really cool adventure game together with Arjan Brussee, a demo coder from Holland who would make amazing graphical stuff, and had built a 2D game engine. Um, they connected them together and a musician, Robert Allen in California, and they, you know, by telephone and modem and so on, were, were building these little 2D games and, uh, having quite a lot of success. You know, there are a bunch of people making thousands of dollars a month, uh, while they were still students, um, in royalties from the games that Epic was kind of producing and by coordinating people, with people and publishing, um, through shareware. Um, e- and that was all going great. Uh, the company had a little office and we were, you know, copying floppy d- floppy disks and mailing them out. But, um, when Wolfenstein came out, we realized, like, the future of gaming is gonna be 3D. Um, it, there had been a lot of experiments in 3D before that hadn't been great. You know, there were 2D ma- there were 3D renderings of mazes that were not in real time and you were always looking north, south, east or west. Um, and then there were vector graphics with little wire frames moving around and things. But, uh, you know, Wolfenstein was the first game that was fast enough, you know, running at 30 frames per second, it really felt immersive. It felt like you were there, like you were, you know, in this castle, Wolfenstein, fighting Nazis, and that was a really amazing and immersive experience. 3D graphics were pretty primitive then. Id Software followed shockingly fast with Doom, which was a much, much more, uh, capable 3D engine which had, you know, stairs and though it was still what we call 2.5D, it was environments that were very realistic, textures that were very realistic. Uh, you know, form of lighting, uh, that was approximate but incredibly realistic and, uh, just such great artistry and sound effects that it felt completely visceral, um, and, and real. Um, yeah.
- LFLex Fridman
(laughs)
- TSTim Sweeney
You might, uh, you might look at it today from a, you know, point of view of a, a modern, uh, you know, game player with, uh, you know, 20 teraflops of computing power in your device and say, "Oh, that's not very impressive." But it was amazing at the time.
- LFLex Fridman
I mean, for me, just sorry to, to pause on that, I think Wolfenstein was one of the most, uh, amazing moments of my own life. Just being able to, like you said, in real time, move about a three-dimensional world. I just remember just, like, just moving around just in like... What is that feeling like? I mean, you feel transported into another world.
- TSTim Sweeney
You feel that you're there, yeah.
- LFLex Fridman
Yeah.
- TSTim Sweeney
Especially when you turn the lights down in your room and you turn the sound up on your speakers and it will scare you. (laughs) Uh, and you'll, you'll, you'll feel like, you know, that fireball that's coming at you is gonna kill you.
- LFLex Fridman
Yeah.
- TSTim Sweeney
Uh, that was an amazing time 'cause we hadn't experienced that before. There was nothing like that. Uh, you know, you'd watch a movie, a scary movie or whatever, you know, it was just this thing that was happening. Uh, this was you. This was you in a 3D world.
- LFLex Fridman
So, uh, how did that, how did that change Epic, this realization that the future of gaming is going to be 3D?
- TSTim Sweeney
Well, at first I was really depressed.
- LFLex Fridman
(laughs)
- TSTim Sweeney
Uh, 'cause-
- LFLex Fridman
Yeah.
- TSTim Sweeney
... the wizardry of Doom, especially, was so incredible that I gave up on programming for, like, six months. I was like, "I don't know if we will compete with this. I have no idea what we're gonna do." Um, we just keep making 3D, 2D games and hope that the business goes on, but, um, uh, that was the nature of Carmack's wizardry. He had, had done things that were, like, not just one innovation leap ahead, but, like, a dozen simultaneously-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... interplaying in a way that you couldn't pick them apart into their component pieces. But, um, funny thing happened. Uh, Michael Abrash, a long-timer in computer graphics that wrote a book on the techniques for 3D, uh, graphics and texture mapping, and he wrote some articles in a, in one of the programming magazines of the day and, um, explained it and showed assembly code to do texture mapping. You know, drawing these 3D graphics on the screen, and it was actually really simple stuff. I was like, "Oh, I can do that."
- LFLex Fridman
(laughs)
- TSTim Sweeney
And, uh-
- LFLex Fridman
(laughs)
- TSTim Sweeney
... and, you know, so a bunch of us at Ep- Epic independently went off and, uh, wrote our, started writing our own 3D graphics code to figure it out, and, um, uh, we, we found at one point we had a number of people dabbling in this, doing different parts of it and, uh, at that point we decided, okay, this is, 3D graphics and 3D gaming is gonna completely change the world. We need to go all in on this. And so we took the best people from our best 2D game development teams and put them all together to make a 3D game. Um, we didn't really know what we were doing at the time. None of us had ever shipped a 3D game and most of us were still learning, but, um, everybody was, like, trying different disciplines to see what they were best at, and, um, you know, it was a combination of a bunch of people, uh, who came together to make Unreal. I'd initially volunteered to make the 3D editor, um, for the thing, and James Schmotz, who'd made Epic Pinball, Epic Pinball, now that wasn't a crazy game. This was one of the 2D shareware games. He made it while he was in college and he was making, like, $30,000 a month (laughs) from, you know, the royalties from this game because-
- LFLex Fridman
Wow.
- TSTim Sweeney
... everybody had wanted an awesome pinball game.
- LFLex Fridman
(laughs)
- TSTim Sweeney
Massively successful. But, uh, he was, he was a multi-disciplinary person. He wrote the code for the game, the art for the game, and did basically everything, and, and the code was 30,000 lines of assembly language. (laughs)
- LFLex Fridman
(laughs)
- TSTim Sweeney
And so, uh, he was initially gonna write the 3D engine, um, and I was gonna write the editor and he sent me the, his code so I could integrate it into the editor and it was like-... just a giant pile of (laughs) assembly code. I was like, "Hmm. (laughs) Why don't I just write this myself?" And so James instead started going off and building 3D models and 3D animations using the tools at the time. And so, uh, Cliff, uh, who had done a lot of design work and built the levels on Jazz Jackrabbit, went off and started, uh, learning basics of level design. And so I was writing this editor, and Cliff Bleszinski was customer number one for it. Um, starting to go off and build levels, and James Schmalz was building awesome creatures, sending them to me. I'd get them in and implement it in-game. And then we brought in an animator to bring them to life, and we brought in, you know, more and more people until at the peak of Unreal 1 development, um, we had about 20 people, um, working on it, which was a huge team for the time, and it was, uh, really stretching Epic's finances nearly to the breaking point.
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
Um, we barely survived and almost ran out of money a number of times, but, uh, somehow we always pulled through, and, um, it was a crazy project because it was three and a half years of development on a game that we always thought was six months from shipping.
- 1:01:44 – 1:06:36
Technical details of Unreal Engine
- LFLex Fridman
So Unreal Engine is this fascinating creation, it's a big bold crazy bet that you made. Maybe it's good to actually explain what Unreal Engine is for people sort of outside this world. I would say, (sighs) it transformed the gaming industry. But that was a big bet in 1995 that, uh, most of the effort would be on creating the gaming engine, not the game.
- TSTim Sweeney
Yeah. Unreal Engine is a big bundle of code and tools, uh, uh, huge software package, um, that provides all the functions you need to build any sort of a 3D graphics application. Um, game developers use it to make games and that's the predominant use, uh, but it's also used in, um, Hollywood film and television production to create 3D scenery, um, in real time for production sets, um, to do pre-visualization. Um, it's used by car makers to visualize their cars before they're, uh, constructed or manufactured. Um, it's used by architects to preview buildings before they're made, and industrial desi- designers of all sorts. And it provides, you know, the- all of the 3D simulation features you need both for creating highly realistic 3D graphics, uh, but also, uh, physics and interactions between objects, and making things happen like you might, uh, see in the real world, um, and supports a huge variety of styles from Pixar-stylized movies to cel shading, um, to photorealism. Um, and it can be used for anything that needs a- needs real-time 3D graphics.
- LFLex Fridman
Including humans that populate those three-dimensional worlds. And, uh, we'll probably talk a bunch of the details involved in the, uh, in the process of creating ul- creating ultra-realistic humans because we humans care about how other humans look and how they convey emotion and- and express, how they speak, all that kind of stuff. Uh, but, so yes, it's the 3D objects that are static, the 3D objects that are dynamic, and, uh, on the dynamic front including humans that are ultra-dynamic. So all of that, you have to create this engine that simulates that world, the world as we, uh, this beautiful world that we know and love. Okay, so that, but, you know, you're early. (laughs) So here you see Doom, and you're trying to create this world and trying to create an engine that would not just power Unreal the video game but future video games. So how do you go about it? What are you thinking? And that- that I should sort of linger on that. That is a crazy bet that we're going to build an engine as a company.
- TSTim Sweeney
Yeah. Well, you know, the philosophy began with, uh, ZZT and continued onward. Um, we're not just building, uh, a game, uh, for players to play, we're also building tools that could be used for- for building that game or any other game, um, and catering to all of the artists and designers who would use the tool. And so that philosophy started with the very early parts of Unreal, uh, development. Um, I was building the tools, uh, for level designers like Cliff Bleszinski and artists like James Schmalz. Um, and, uh, you know, as we began marketing the game thinking it was six months away, we were constantly, uh, releasing screenshots and things like that. Um, other companies started calling us and saying they wanted to build 3D games too, um, but they didn't have the expertise for that and they wanted to license our 3D engine. Um, and this was one of the coolest, uh, piv- pivots in Epic's history that Microprose called up Mark Rein, our vice president and longtime, uh, business guy, and said they wanted to license our engine. And Mark Rein was like, "Oh, what? You want to license what? What? An engine? What's the- what engine?"
- LFLex Fridman
(laughs)
- TSTim Sweeney
And they explained to him what they wanted to license. He's like, "Oh, that engine. Yeah, yeah. That's very expensive." (laughs) But this was one of the critical things that kept E- Epic going through that three and a half years. Uh, we were lic- starting to license our engine out to other developers. Microprose took two licenses, and we got in half a million dollars from that, um, and, uh, a company GT Interactive licensed our engine to build, uh, another game and, uh, we got paid for that. And so we- we had this revenue stream funding the development of Unreal Engine, uh, from other games that were being developed by other developers and, um, because they were the lifeline for the company, we- we took the engine business very seriously from the start. Um, we set up, uh, you know, mailing lists so that our partners could ask us questions and all of the- all of the developers and artists working on our games were participating in helping customers. Um, everybody took that very seriously because it was our funding source and, um, you know, that's kind of set the store spirit of Epic of building technology and supporting game developers simultaneous with building games and supporting gamers. Um, it's continued onward and just grown over time.
- 1:06:36 – 1:12:35
Constructive solid geometry
- TSTim Sweeney
- LFLex Fridman
Can you just go back to that? Y- you programming.What, what are some interesting technical challenges you had to overcome? You mentioned dynamic lighting. Like, create, you know, create this th- three-dimensional world and, and, uh, try to figure out the puzzle of how d- you actually do that a- at a time when nobody... just Carmack and you (laughs) , uh, doing this kinda thing. It's, it's a totally open Wild West. So, what are some interesting technical challenges you had to, uh, you had to try to solve?
- TSTim Sweeney
There's a lot. Um, some of them are visible on screen. Um, and some are behind the scenes-
- LFLex Fridman
Yeah.
- TSTim Sweeney
... and still require a lot of innovation. Um, all the graphical techniques were, were really interesting challenges. Um, and Unreal Engine, uh, in those early days, went a lot further than the Quake engine in building, uh, environments using constructive solid geometry with a real-time editor, and, uh, th- that was a, that was a really interesting technical challenge. You know, the idea is building is extremely tedious if you are only adding objects to the world. If you wanna build a door, then you need to add, like, a dozen different pieces of door frames and add a bunch of different walls together to fit together in the right shape. It sure would be easier if you could just start with a wall and subtract the door out. Um, and so we had this way of adding geometry to the world and subtracting geometry, and the engine would perform all of the calculations on that. Um, and, uh, this is something that I'd been anticipating was possible for a long time, but, uh, when I finally got around to it, it took this 30-hour coding session, um, to, like, figure out all the special cases of the code that needed to be implemented to make that work. But, uh, you know, in the course of 30 hours, I got constructive geo- solid geometry up and running. I started doing, uh, uh, you know, like handed it to James Schmalz, uh, the next time we were together and, um, he's like, "Okay, I think you're cheating here." So you create a giant torus-
- LFLex Fridman
(laughs)
- TSTim Sweeney
... and then add another giant torus interlocked with it-
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
... um, and then subtracted a cylinder from it and, um, like, created this really advanced composite object, uh, with just three operations. He was like, "Whoa, I can't believe this." It's like, "Yeah, (laughs) we figured it out." And that, and that was cool to see for the first time. Uh, it was probably the first time somebody had done constructive solid geometry in real time, um, but it was also a really useful artist tool that all the artists appreciated and immediately began making use of.
- LFLex Fridman
Can you actually speak to that, the 30-hour session? I mean, this is not, from everything I know about computation geometry, doing this kinda thing from your perspective is not, that's not easy. That's... what is it? The uncertainty, the open questions involved, the, like, the, I mean, even just, uh, on the algorithm front, how to do that efficiently. Uh, and then plus the usual programming thing of debugging. Like, s- suffering through the trickiness of it, and we don't have really, at that time, you don't have the tooling to really visualize everything that's going on really well, and you're probably, like, using some crappy editor. I mean, there's just a lot of (laughs) , like, friction here. So the, the 30-hour session is one that's probably, uh, rough. It's a rough one.
- TSTim Sweeney
Your, your brain works in different ways and depending on your, your state, right? Uh-
- LFLex Fridman
Right.
- TSTim Sweeney
... there are some things that re- require really working on a problem fresh, um, where you've put together a bunch of logical pieces and now you just need to write a whole lot of code to make it all work together and, you know, plumb a whole lot of data between a whole lot of different algorithms. Um, but, you know, I- I think our brains have vastly more horsepower than we're able to directly access by thinking of what code to type next. Um, and, you know, after you've been working for a very long time, you, you can get into a sleep-deprived state where you have much, much more direct access to that low-level knowledge.
- LFLex Fridman
(laughs) That's great. Yeah.
- TSTim Sweeney
(laughs) You know, because, you know, th- there are symptoms that are well studied of sleep deprivation. One of them is, um, short-term memory loss. And so you're working without, like, the easy recall of the code you just typed, uh, but your brain is then free to, to think about other problems and, uh, and yet I'd built up this intuition over a very long, uh, period of time, you know, so the foundation for the subject is the binary space partition tree, the status structure invented by a computer sci- graphics researcher, Bruce Naylor.
- LFLex Fridman
Mm-hmm.
- TSTim Sweeney
Carmack had picked up on that and, uh, had used the technique in, in Doom, uh, to really great effect. Um, and, and I picked up on that and Unreal Engine was using, uh, this technique for all of its graphics and rendering, but it, uh, you know, it was just additive geometry everywhere and it had a lot of overlapping polygons and was pretty inefficient. Um, you know, so I had the idea that if we had a BSP tree, there was a really efficient way to do constructive solid geometry and it... to do that, you had to break down the ways that different pieces of geometry can fit together. I broke it d- down into, like, 14 different cases, um, and most of them are pretty simple, crank them out. Um, anyways, I got towards the end, you know, there were some pretty complicated things, like, what, how do you deal with co-planar polygons? They're in the same plane-
- LFLex Fridman
(laughs) yeah.
- TSTim Sweeney
... um, and pointing in the same direction versus the other direction. In what cases should you keep them? When, what cases should you eliminate them? And, and so on and so on to create really efficient, uh, geometry output and, you know, uh, just, uh, plowing through it, uh, eventually, uh, through mostly, uh, deduction, but some trial and error too. Like, sometimes you just have to try the possibilities and see what works. Um, you know, I, I cranked it out and it worked, and the next day I came in, like, kind of weary and it's like, "Oh wow, this actually did work. It wasn't just a dream."
- LFLex Fridman
So you're considering the edge cases also. I mean, that's the problem with geometry is, like, there's probably just gonna be all kinds of weird polygons that you have to... so you're, like, thinking th- you're imagining the edge cases and trying to see how do I not create inefficiencies in th- in this algorithm while still considering the edge cases, allowing for the edge cases.
- TSTim Sweeney
Yeah. You know, it, it's pretty easy to write software that's, like, 99% correct.
- LFLex Fridman
Yeah.
- TSTim Sweeney
It's the 1% that's the really hard part and, uh, where the devil lies in the details.
- 1:12:35 – 1:17:05
Dynamic lighting
- TSTim Sweeney
- LFLex Fridman
What about, like, lighting? Is there other interesting-
- TSTim Sweeney
Well, the funny answer is, like, we know the laws of physics-So it's actually really easy to do everything in computer graphics.
- LFLex Fridman
Yeah.
- TSTim Sweeney
But, uh, the direct solution of the laws of physics is immensely so. And so what we're finding are approximations rather than complete solutions, um, 'cause you need something that's a million times faster than the brute force answer.
- LFLex Fridman
We should say that the, the physics of the (inaudible) is you just take a bunch of photons and bounce them around. That's how light works.
- TSTim Sweeney
Yeah.
- LFLex Fridman
That's going to be very inefficient because there's (laughs) it's a lot of bouncing and a lot of photons.
- TSTim Sweeney
Yeah, yeah. Photon tracing is the subject matter that does brute force calculation of pixels on a screen from all of the light in the scene, and it- it works and it's correct and it just is an implementation of laws of physics, and it's millions or billions of times slower than what we do. But Carmack had, uh, figured out how to do, uh, really cool lighting algorithms, including real-time lighting with objects moving around, um, and, uh, but hadn't taken it very far. So I- with Unreal Engine, I'd- I'd realized, like, it's- we don't have nearly enough computing performance on our CPU to compute the light of every pixel on the screen from all of the light sources that affect it. Um, yeah. We're at a six cycle texture mapper and we couldn't afford 30 more cycles for lighting. And so the answer had to be some approximation, and, uh, the one that, uh, Carmack had picked up on in the Quake engine was light mapping, that if we, uh, instead of calculating all the lighting on every pixel, what if we, like, made a big texture that we placed over all of the walls in the scene that was like wallpaper? And what if we say at every foot we're gonna compute a lighting value for just that one, one-foot grid on the object rather than computing it everywhere? Um, and then if we, what if we just linear interpolate that over the course of it? You know, you get a lighting and, uh, solution that actually works pretty well, um, and is fast enough to work. And so a lot of Unreal Engine's lighting techniques were based on light mapping. We introduced colored lighting, um, and so you could have colored light sources. Then we realized, oh, since we're doing this and we're doing it on light maps, we can actually do some pretty expensive calculations, hundreds of cycles, since we're only calculating it for every one foot of world space rather than every pixel. And so we introduced a whole bunch of elaborate lighting effects, um, like torch flickering, um, and, you know, the caustic effects of water bouncing off of a surface, uh, and so on, um, so, and li- pulsing lights and blinking lights and everything else, and created a system... I created a system for compositing them together. So if you had an arbitrary number of light sources, they could all do that. And-
Episode duration: 4:25:19
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 477qF6QNSvc
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome