Lex Fridman PodcastGreg Brockman: OpenAI and AGI | Lex Fridman Podcast #17
EVERY SPOKEN WORD
150 min read · 30,404 words- 0:00 – 15:00
The following is a…
- LFLex Fridman
The following is a conversation with Greg Brockman. He's the co-founder and CTO of OpenAI, a world-class research organization developing ideas in AI, with a goal of eventually creating a safe and friendly artificial general intelligence, one that benefits and empowers humanity. OpenAI is not only a source of publications, algorithms, tools, and datasets, their mission is a catalyst for an important public discourse about our future with both narrow and general intelligence systems. This conversation is part of the Artificial Intelligence Podcast at MIT and beyond. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter @lexfridman, spelled F-R-I-D. And now, here's my conversation with Greg Brockman. So in high school and right after, you wrote a draft of a chemistry textbook.
- GBGreg Brockman
(laughs)
- LFLex Fridman
I saw that. It covers everything from basic structure of the atom to quantum mechanics. So it's clear you have an intuition and a passion for both the, uh, the physical world with chemistry and now robotics, to the digital world with, uh, AI, deep learning, reinforcement learning, so on. Do you see the physical world and the digital world as different? And what do you think is the gap?
- GBGreg Brockman
A lot of it actually boils down to iteration speed, right? That I think that a lot of what really motivates me is, is building things, right, is the... Uh, you know, think about mathematics, for example, where you think really hard about a problem, you understand it, you write it down in this very obscure form that we call proof. But then, this is in humanity's library, right? It's there forever. This is some truth that we've discovered. And, you know, maybe only five people in your field will ever read it, uh, but somehow you've kinda moved humanity forward. And so I actually used to really think that I was going to be a mathematician, and, uh, then I actually started writing this chemistry textbook. One of my friends told me, "You'll never publish it because you don't have a PhD." So instead, I, I decided to build a website and try to promote my ideas that way, and then I discovered programming and, uh, I... You know, that in programming, you think hard about a problem, you understand it, you write it down in a very obscure form that we call a program. But then once again, it's in humanity's library, right? And anyone can get the benefit from it, and the scalability is massive. And so I think that the thing that really appeals to me about the digital world is that you can have this, this, this h- insane leverage, right? A single individual with an idea is able to affect the entire planet, um, and that's something I think is really hard to do if you're moving around physical atoms.
- LFLex Fridman
But you said, uh, mathematics, so if you look at the, the wet thing, eh, o- over here, our mind, do you, you ultimately see it as just math, as just information processing? Or, or is there some other magic as you've seen, if you've seen through biology and chemistry and so on?
- GBGreg Brockman
Yeah, I think it's really interesting to think about, uh, humans as just information processing systems and, uh, that it seems like it's actually a pretty good, uh, way of describing a lot of, of kind of how the world works or a lot of what we're capable of to think that, that... You know, again, if you just look at, at technological innovations over time-
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
... that in some ways, the most transformative innovation that we've had has been the computer, right? In some ways, the internet, you know, that, what has the internet done, right? The internet is not about these physical cables. It's about the fact that I am suddenly able to instantly communicate with any other human on the planet. I'm able to retrieve any piece of knowledge that, in some ways, the human race has ever had, uh, and that those are these insane transformations.
- LFLex Fridman
Do you see the, our society as a whole, the collective as another extension of the intelligence of the human being? So if you look at the human being as an information processing system, you mentioned the internet, the networking. Do you see us all together as a civilization, as a, as a kind of intelligence system?
- GBGreg Brockman
Yeah. I think this is actually a really interesting perspective, uh, to take and to think about, that you sort of have this collective intelligence of all of society. The economy itself is this superhuman machine that is optimizing something, right? And it's almo- in some ways, a company has a will of its own, right? That you have all these individuals who are all pursuing their own individual goals and thinking really hard and thinking about the right things to do. But somehow, the company does something that is this emergent thing, uh, and that is, is y- it's, it's a really useful abstraction.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And so I think that in some ways, you know, we think of, of, of ourselves as the most intelligent things on the planet and the most powerful things on the planet, but there are things that are bigger than us that are these systems that we all contribute to. Um, and so I think actually, you know, it's a, it's interesting to think about, uh, if you've read Isaac Asimov's Foundation, right? That, uh, that there's this concept of psychohistory in there, uh, which is effectively this, that if you have trillions or quadrillions of, of beings, then maybe you could actually predict what that being, that, that huge macro being will do, uh, and, uh, almost independent of what the individuals want. And I actually have a sec- a second angle on this that I think is interesting-
- LFLex Fridman
Okay.
- GBGreg Brockman
... which is thinking about, uh, technological determinism. One thing that, that I actually think a l- a lot about with, with OpenAI, right, is that we're kinda coming on, onto this insanely transformational technology of, of general intelligence, right? That will happen at some point. And there's a question of, how can you take actions that will actually steer it to go better rather than worse? And that I think one question you need to ask is, as a scientist, as an inventor, as a creator, what impact can you have in general, right? You look at things like the telephone invented by two people on the same day. Like, what does that mean? Like, what does that mean about the shape of innovation? And I think that what's going on is everyone's building on the shoulders of the same giants. And so you can kind of... You can't really hope to create something no one else ever would. You know, if Einstein wasn't born, someone else would have come up with relativity. You know, you change the timeline a bit, right? That maybe it would have taken another 20 years, but it wouldn't be that fundamentally, humanity would never discover these, these fundamental truths.
- LFLex Fridman
So there's some kind of invisible momentum that some people like Einstein or OpenAI is plugging into, uh, that anybody else can also plug into and ultimately, it, that wave takes us into a certain direction. That's what you mean by determinism?
- GBGreg Brockman
That, that, that's right. That's right. And, you know, this kinda seems to play out in a bunch of different ways, uh, that there's some exponential that is being ridden and that the exponential itself, which one it is changes. Think about Moore's Law. An entire industry set its clock to it for 50 years. Like, how can that be, right? How is that possible? And yet somehow, it happened. And so I think you can't hope to ever invent something that no one else will. Maybe you can change the timeline a little bit. But if you really want to make a difference, I think that the thing that you really have to do, the only real degree of freedom you have is to set the initial conditions under which a technology is born. And so you think about the internet, right? That there are lots of other competitors trying to build similar things, and the internet won, and that the initial conditions where that was created by this group that really valued people being able to be...... uh, you know, anyone being able to plug in this very academic mindset of, of being open and connected.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
Um, and I think that the internet for the next 40 years really played out that way. Um, you know, maybe today, uh, things are starting to, to shift in a, in a different direction. But I think that those initial conditions were really important to determine the next 40 years worth of, of progress.
- LFLex Fridman
That's really beautifully put. So another example of that I think about, you know, I recently looked at it. I looked at Wikipedia, the formation of Wikipedia, and I wonder what the internet would be like if Wikipedia had ads.
- GBGreg Brockman
Hmm.
- LFLex Fridman
You know, there's a interesting argument that, uh, why they chose not to make it, uh, put advertisement on Wikipedia. I think it's, uh, I think Wikipedia's one of the greatest resources we have on the internet. It's, it's extremely surprising how well it works and how well it was able to aggregate all this kind of, uh, good information. And they, e- essentially, the creator of Wikipedia, I don't know, there's probably some debates there, but set the initial conditions.
- GBGreg Brockman
Yes.
- LFLex Fridman
And now it carried itself forward. That's really interesting. So you're, the way you're thinking about AGI or artificial intelligence is you're focused on setting the initial conditions for, for the progress.
- GBGreg Brockman
That's right.
- LFLex Fridman
That's powerful. Okay, so looking to the future, if you create an AGI system, like one that can ace the Turing test in natural language, what do you think would be the interactions you would have with it? What do you think are the questions you would ask? Like, what would be the first question you would ask it, her, him?
- GBGreg Brockman
That's right. I think that at that point, if you've really built a powerful system that is capable of shaping the future of humanity, the first question that you really should ask is, how do we make sure that this plays out well? Um, and so that's actually the first question that I would ask a powerful AGI system is-
- LFLex Fridman
So you wouldn't ask your colleague. You wouldn't ask, like, Ilya. You would ask the AGI system.
- GBGreg Brockman
Oh, we've already had the conversation with Ilya-
- LFLex Fridman
Okay.
- GBGreg Brockman
... right? And everyone here. Uh, and so you want as many perspectives and, uh, uh, pieces of wisdom as you can for, for answering this question. So I don't think you necessarily defer to whatever your powerful system tells you. Um, but you use it as one input, uh, to try to figure out what to do. But... And I, I guess fun- fundamentally, what it really comes down to is if you built something really powerful and you think about, think about, for example, the creation of, uh, of shortly after the creation of nuclear weapons, right? The most important question in the world was, what's the world order going to be like? How do we set ourselves up in a place where we're going to be able to s- survive as a species? With AGI, I think the question's slightly different, right? That there is a question of, how do we make sure that we don't get the negative effects? But there's also the positive side, right? You imagine that, you know, like, like, what will an AGI be like? Like, what will it be capable of? And I think that, that one of the core reasons that an AGI can be powerful and transformative is actually due to, uh, technological development.
- 15:00 – 30:00
To even say this…
- LFLex Fridman
so align with, um, the ethics and the morals of human beings?
- GBGreg Brockman
To even say this in a different way, I mean, think about how, how do we align humans, right? Think about, like-
- LFLex Fridman
Right.
- GBGreg Brockman
... a human baby can grow up to be an evil person or a great person. And a lot of that is from learning from data, right? That you have some feedback as a child who's growing up, they get to see positive examples. And so I think that, that just like the, um, that the, the only example we have of a general intelligence, uh, that is able to learn from data, uh, to align with human values and to learn values, um, I think we shouldn't be surprised that we can do the same sorts of, of, of techniques or whether the same sort of techniques end up being how we, we, we solve value alignment for AGIs.
- LFLex Fridman
So let's go even, uh, higher. I don't know if you've read the book, Sapiens.
- GBGreg Brockman
Mm-hmm.
- LFLex Fridman
But, uh, there's an idea that, you know, um, that as a collective, as us human beings, we kind of develop together an, uh, ideas that we hold. There's no, uh, in that context, objective truth, we just kind of all agree to certain ideas and hold them as a collective.
- GBGreg Brockman
Mm-hmm.
- LFLex Fridman
Did you have a sense that there is, in the world of good and evil, do you have a sense that to the first approximation, there are some things that are good and that we could teach systems to behave to be good?
- GBGreg Brockman
So I think that, that this actually blends into our third team, right? Which is the policy team. And this is the one, th- the aspect I think people really talk about way less than they should, right? 'Cause imagine that we build super powerful systems that we've managed to figure out all the mechanisms for these things to do whatever the operator wants. The most important question becomes, who's the operator? What do they want? And how is that going to affect everyone else, right? And, and I think that this question of what is good, what are those values, I mean, I think you, you don't even have to go to those, those, those very grand existential places to start to, to realize how hard this problem is. You just look at different countries and cultures across the world-
- LFLex Fridman
Right.
- GBGreg Brockman
... and that there's, there's a very different conception of how the world works and, you know, what, what, what kinds of, uh, of ways that society wants to operate. And so I think that, that the, that the really core question is, is, is, is actually very concrete. Um, and I think it's not a question that we have ready answers to, right? Is how do you have a world where all the different countries that we have, United States, China, Russia, and, you know, the, the, the hundreds of other countries out there are able to continue to not just operate in the way that, that they, that they see fit, but in, in, uh, the, the, the world that emerges, um, in these, wh- where you have these very powerful systems, uh, operating alongside humans ends up being something that empowers humans more, that makes, like, exist- human existence be a more meaningful thing and, uh, that people are happier and wealthier and, uh, able to live more fulfilling lives. It's not an obvious thing for how to design that world once you have that very powerful system.
- LFLex Fridman
So if we take a little step back, and we're having, like, a fascinating conversation, and, uh, OpenAI is in many ways a tech leader in the world, and yet we're thinking about these big existential questions, which is fascinating, really important. I think you're a leader in that space, and that's a really important space of just thinking how AI affects society in a, in a big picture view. So Oscar Wilde said, "We're all in the gutter, but some of us are looking at the stars." And I think OpenAI has a charter that looks to the stars, I would say, to create intelligence, to create general intelligence, make it beneficial, safe, and collaborative. So can you tell me, uh, how that came about, how a mission like that and the path to creating a mission like that at OpenAI was founded?
- GBGreg Brockman
Yeah. So I think that in some ways, it really boils down to taking a look at the landscape, right? So if you think about the history of AI that basically for the past 60 or 70 years, people have thought about this goal of what could happen if you could automate human intellectual labor.
- LFLex Fridman
Right.
- GBGreg Brockman
Imagine you could build a computer system that could do that. What becomes possible? We have a lot of sci-fi that tells stories of various dystopias and, you know, increasingly you have movies like Her that tell you a little bit about maybe more of a little bit utopic vision. Uh, you think about the impacts that we've seen from being able to, uh, have bicycles for our minds in computers, uh, and that, I think that the, that the impact of, of computers and the internet has just far outstripped what anyone really could have predicted. And so I think that it's very clear that if you can build an AGI, it will be the most transformative technology that humans will ever create. And so what it boils down to then is a question of, well, is there a path? Is there hope? Is there a way to build such a system? And I think that for 60 or 70 years that people got excited and, uh, uh, that, you know, ended up not being able to deliver on the hopes that, that, that people had pinned on them. And I think that then, you know, that after, you know, two, two winters of AI development, uh, that people, uh, you know, I think kind of almost stopped daring to dream, right?
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
Th- that really talking about AGI or thinking about AGI became a- almost this taboo in the community.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
But I actually think that people took the wrong lesson from AI history. And if you look back, starting in 1959 is when the Perceptron was released, and this is basically, you know, one of the earliest neural networks. Um, it was released to what was perceived as this massive over-hype. So in the New York Times in 1959, you have this article, uh, saying that, you know, the Perceptron will one day recognize people, call out their names, instantly translate speech between languages.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And y- people at the time looked at this and said, "This is ... like, your system can't do any of that." And basically spent 10 years trying to discredit the whole Perceptron direction and suc- and succeeded. And all the funding dried up and, you know, people kind of, uh, went in other directions and, you know, in the '80s there was this resurgence and I'd always heard that the resurgence in the '80s was due to the invention of back propagation and these, these algorithms that got people excited. But actually the causality was due to people building larger computers; that you can find these, these articles from the '80s saying that the democratization of computing power suddenly meant that you could run these larger neural networks. And then people started to do all these amazing things. The back propagation algorithm was invented. And, you know, the, the neural nets people were running were these tiny little, like, 20 neuron neural nets.
- LFLex Fridman
Right.
- GBGreg Brockman
Right? Like, what are you supposed to learn with 20 neurons?
- LFLex Fridman
Yeah.
- GBGreg Brockman
And so of course they weren't able to get great results. And it really wasn't until 2012 that this approach that's almost the most simple, natural approach, that people had come up with in the '50s, right, in some ways even in the '40s before there were computers with Pitz-McCullon n- neuron, suddenly this became the best way of solving problems. Right? And I think there are three core properties that deep learning has that I think are very worth paying attention to. The first is generality. We have a very small number of deep learning tools, SGD, deep neural net, maybe some, some, you know, RL, and it solves this huge variety of problems.
- LFLex Fridman
Hm.
- GBGreg Brockman
Speech recognition, machine translation, game playing, all these problems, s- small set of tools. So there's the generality. There's a second piece, which is the competence. You wanna solve any of those problems? Throw out 40 years worth of normal computer vision research, replace it with a deep neural net, it's gonna work better. And there's a third piece, which is the scalability, right? That one thing that has been shown time and time again is that you, if you have a larger neural network, throw more compute, more data at it, it will work better. Those three properties together feel like essential parts of building a general intelligence. Now it doesn't just mean that if we scale up what we have that we will have an AGI, right? There are clearly missing pieces, there are missing ideas. We need to have answers for reasoning, but I think that the core here is that for the first time, it feels that we have a paradigm that gives us hope that general intelligence can be achievable. And so as soon as you believe that, everything else becomes, comes into focus, right? If you imagine that you may be able to, and you know that the timeline I think remains uncertain, um, th- but I think that, that, you know, certainly within our lifetimes and possibly within a, a much shorter period of time than, than people would expect, if you can really build the most transformative technology that will ever exist, you stop thinking about yourself so much, right? You start thinking about just like, how do you have a world where this goes well? And that you need to think about the practicalities of how do you build an organization and get together a bunch of people and resources, um, and to make sure that people feel, uh, motivated and ready to, to do it. But I think that then you start thinking about, well, what if we succeed? Um, and how do we make sure that when we succeed, that the world is actually the place that, that, that we want ourselves to exist in and, you know, almost in the, the Rawlsian bale sense of the word. And so that's kind of the, the, the broader landscape and OpenAI was really formed in 2015 with that high level picture of AGI might be possible sooner than people think and that, uh, we need to try to do our best to make sure it's going to go well. And then we spent the next couple years really trying to figure out what does that mean? How do we do it? Um, and you know, I think that typically with a company you start out very small, so you and a co-founder and you build a product, you get some users, you get product market fit, you know, then at some point you raise some money, you hire people, you scale and then, uh, you know, down the road then the big companies realize you exist and try to kill you. Um, and for OpenAI it was basically everything in exactly the opposite order. Uh-
- LFLex Fridman
(laughs) Let me just pause for a second. You said a lot of things and l- let me just admire the jarring aspect of what OpenAI stands for, which is, uh, daring to dream. I mean, you said it, it's pretty powerful. It caught me off guard because I think that's very true. The, the, the step of just daring to dream about the possibilities of creating intelligence in a positive and a safe way, but just even creating intelligence is, uh, a much needed refreshing, uh, catalyst for the AI community. So that's, that's the starting point. Okay, so then formation of OpenAI, uh, what's-
- GBGreg Brockman
Uh, I just, I would just say that, you know, when we were starting OpenAI, uh, that kind of the first question that we had is, is it too late to start-
- 30:00 – 45:00
I imagine OpenAI, uh,…
- GBGreg Brockman
was the most important step towards figuring out how do we structure a company that can actually raise the resources to do what we need to do.
- LFLex Fridman
I imagine OpenAI, uh, the decision to create OpenAI LP was a really difficult one and there was a lot of discussions, as you mentioned, for a year and, uh, there was different ideas, perhaps detractors within OpenAI, uh, sort of, uh, different paths that you could have taken. What were those concerns? What were the different paths considered? What was that process of making that decision like?
- GBGreg Brockman
Yep. Um, but so if you look actually at the OpenAI charter, the- there's almost two paths embedded within it.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
There is, we are primarily trying to build AGI ourselves, but we're also okay if someone else does it, and this is a weird thing for a company.
- LFLex Fridman
It's really interesting actually.
- GBGreg Brockman
Yeah.
- LFLex Fridman
That there is an element of competition that you do want to be the one that does it, but at the same time you're okay if somebody else does it. And we'll talk about that a little bit, that trade-off, that's- that dance that's really interesting.
- GBGreg Brockman
And I think this was the core tension as we were designing OpenAI LP and really the OpenAI strategy, is how do you make sure that both you have a shot at being a primary actor, which really requires...... building an organization, raising massive resources, and really having the will to go and execute on some really, really hard vision, right? You need to really sign up for a long period to go and take on a lot of pain and a lot of risk. Um, and to do that, normally you just import the startup mindset, right? And that you think about, okay, like how do we out execute everyone? You, you have this very competitive angle. But you also have the second angle of saying that, well, the true mission isn't for OpenAI to build AGI. The true mission is for AGI to go well for humanity.
- LFLex Fridman
Right.
- GBGreg Brockman
And so how do you take all of those first actions and make sure you don't close the door on outcomes that would actually be positive and, and f- fulfill the mission? And so I think it's a very delicate balance, right? And I think that going 100% one direction or the other is clearly not the correct answer. And so I think that even in terms of just how we talk about OpenAI and think about it, there's j- just like, like one thing that's always in the back of my mind is to make sure that we're not just saying OpenAI's goal is to build AGI, right? That it's actually much broader than that, right? That first of all, uh, you know, it's not just AGI, it's safe AGI that's very important. But secondly, our goal isn't to be the ones to build it. Our goal is to make sure it goes well for the world. And so I think that figuring out how do you balance all of those, um, and to, to get people to really come to the table and compile the, the, like a single document that, that encompasses all of that w- wasn't trivial.
- LFLex Fridman
So part of the challenge here is, uh, your mission is, I would say, beautiful, empowering, and a beacon of hope for people in the research community and just people thinking about AI. So your decisions are scrutinized more than I think a, a regular profit-driven company. Do you feel the burden of this in the creation of the charter and just in the way you operate?
- GBGreg Brockman
Yes.
- LFLex Fridman
(laughs) Uh, so why do you, uh, lean into the burden (laughs) by creating such a charter?
- GBGreg Brockman
Yeah.
- LFLex Fridman
Why not keep it quiet?
- GBGreg Brockman
I mean, it just boils down to the, to the mission, right? Like, like I'm here and everyone else is here because we think this is the most important mission, right?
- LFLex Fridman
Dare to, dare to dream. All right. So my, uh, do you think you can be good for the world or create an AGI system that's good when you're a for-profit company? From my perspective, I don't understand why profit, uh, interferes with a positive impact on society, right? I don't understand, uh, why, uh, Google that makes most of its money from ads can't also do good for the world, or other companies, Facebook, anything. I don't, uh, I don't understand why those have to interfere. You know, you can, um... Profit isn't the thing, in my view, that affects the impact of a company. What affects the impact of the company is the charter, is the culture, is the, you know, the people inside, and profit is the thing that just fuels those people. So y- what, what are your views there?
- GBGreg Brockman
Yeah. So I thi- I think that's a, is a really good question. And there's, there's, there's some, some, you know, real, like long, longstanding debates in human society that are wrapped up in it. Uh, the way that I think about it is just think about what, what are the most impactful nonprofits in the world? What are the most impactful for-profits in the world?
- LFLex Fridman
Right. Yeah. It's much easier to list the for-profits.
- GBGreg Brockman
That's right. And I think that there's, there's some real truth here that the system that we set up, the system for kind of how, you know, today's world is organized, um, is one that, that really allows for huge impact. Um, and that, that, you know, kind of part of that is that you need to be... The, the, the, you know, for, for-profits are, are self-sustaining and able to, to kind of, you know, build on their own momentum. Um, and I think that's a really powerful thing. It's something that when it turns out that we haven't set the guardrails correctly causes problems, right? Think about logging companies that go and deforest, uh, you know, the, the rainforest. That's really bad. We don't want that. Um, and it's actually really interesting to me that kind of this, this question of how do you get positive benefits out of a for-profit company, it's actually very similar to how do you get positive benefits out of an AGI, um, right? That you have this like very powerful system, it's more powerful than any human and, uh, it's kind of autonomous in some ways. You know, it's superhuman in a lot of axes. And somehow you have to set the guardrails to get good things to happen. But when you do, the benefits are massive. And so I think that, that when, when I think about nonprofit versus for-profit, I think just not enough happens in nonprofits. They're very pure, but it's just kind of, you know, it's just hard to do things there. Um, in for-profits in some ways, like too much happens. Um, but, um, if, if kind of shaped in the right way, it can actually be very positive. And so with OpenAI LP, we're, we're picking a road in between. Now, the thing that I think is really important to recognize is that the way that we think about OpenAI LP is that in the world where AGI actually happens, right? In a world where we are successful, we build the most transformative technology ever, the amount of value we're gonna create will be astronomical.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And so then in that case that the, that the, that the cap that we have will be a small fraction of the value we create. Um, and that the amount of value th- that goes back to investors and employees looks pretty similar to what would happen in a, in a pretty successful startup. And that's really the case that we're optimizing for, right? That we're thinking about in this success case, making sure that the value we create doesn't get locked up. And I expect that in other, you know, for-profit companies that it's possible to do something like that. I think it's not obvious how to do it, right? And I think that as a for-profit company, you have a lot of fiduciary duty to your shareholders and that there are certain decisions that you just cannot make. Um, in our structure, we've set it up so that we have a fiduciary duty to the charter, right? That we always get to make the decision that is right for the charter, um, rather than... Even if it comes at the expense of our own stakeholders. And, uh, and so I think that when I think about what's really important, it's not really about nonprofit versus for-profit. It's really a question of if you build AGI and y- you kind of, you know, humanity's now in this new age, who benefits? Whose lives are better? Um, and I think that what's really important is to have an answer that is everyone.
- LFLex Fridman
Yeah, which is one of the core, um, aspects of the charter. So one concern people have not just with OpenAI, but with Google, Facebook, Amazon, a- anybody, uh, really, uh, that's a- that's creating impact at scale is how do we avoid, as your charter says, avoid enabling the use of AI or AGI to unduly concentrate power? Why would not a company like OpenAI keep all the power of an AGI system to itself?
- GBGreg Brockman
The charter.
- LFLex Fridman
The charter. So, you know, how does the charter actualize itself in, uh, day-to-day?
- GBGreg Brockman
So I think that, that first, to, to zoom out, right, that the way that we structure the company is so that the, the power for sort of, you know, dictating the actions that OpenAI takes ultimately rests with the board, right? The board of, of, of the nonprofit. Um, and the board is set up in certain ways with certain, certain restrictions that you can read about in the OpenAI LP blog post. Um, but effectively the board is the, is the governing body for OpenAI LP. Um, and th- the board has a duty to fulfill the mission of the nonprofit. Um, and so that's kind of how we tie... H- how we thread all these things together. Um, now there's a question of, so day-to-day, how do people, the individuals who in some ways are the most empowered ones, right? You know, the board sort of gets to call the shots at the high level, but the people who are actually executing are the employees, right? The people here on a day-to-day basis who have the, you know, the, the keys to the technical kingdom. And there, I think that the answer looks a lot like, well, how does any company's values get actualized, right? And I think that a lot of that comes down to that you need people who are here because they really believe in that mission, um, and they believe in the charter and that they are willing to take actions, uh, that maybe are worse for them, but are better for the charter. Um, and that's something that's really baked into the culture. And honestly, I think it's, uh, you know, I think that that's one of the things that we really have to work to preserve as time goes on. Um, and that's a really important part of how we think about hiring people and bringing people into OpenAI.
- LFLex Fridman
So there's people here, there's people here who could speak up and say like, hold on a second, this is totally against what we stand for.
- GBGreg Brockman
Uh-
- LFLex Fridman
Culture-wise.
- 45:00 – 1:00:00
Yeah. So again, I…
- LFLex Fridman
release the full model because you had concerns about the possible negative effects of the availability of such model. It's, uh, outside of just that decision, it's super interesting because of, uh, the discussion as, at a societal level, the discourse it creates, so it's fascinating in that aspect. But if you think, at the specifics here at first, what are some negative effects that you envisioned? And of course, what are some of the positive effects?
- GBGreg Brockman
Yeah. So again, I think to zoom out, like, the way that we thought about GPT-2 is that with language modeling, we are clearly on a trajectory right now where we scale up our models and we get qualitatively better performance, right? GPT-2 itself was actually just a scale up of a model that we've released pre- in the previous June, right? And we just ran it at, you know, much larger scale and we got these results where suddenly starting to write coherent prose, which was not something we'd seen previously.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And what are we doing now? Well, we're gonna scale up GPT-2 by 10X, by 100X, by 1,000X and we don't know what we're gonna get. And so it's very clear that the model that we, that we released last June, you know, I think it's kind of like, it's, it's, it's a good academic toy. It's not something that we think is something that can really have negative applications or, you know, to the extent that it can, that the positive of people being able to play with it, uh, is, is, you know, far, far outweighs the, the, the possible harms. You fast forward to not GPT-2 but GPT-20-
- LFLex Fridman
Right.
- GBGreg Brockman
... and you think about what that's gonna be like, and I think that the capabilities are going to be substantive. And so there needs to be a point in between the two where you say, "This is something where we are drawing the line, um, and that we need to start thinking about the safety aspects." And I think for GPT-2, we could have gone either way. And in fact, when we had conversations internally, uh, that we had a bunch of pros and cons, um, and it wasn't clear which one, which one outweighed the other. Um, and I think that when we announced that, "Hey, we decide not to release this model," um, then there was a bunch of conversation where various people said, "It's so obvious that you should've just released it." There were other people said, "It's so obvious you should not have released it." And I think that that almost definitionally means that holding it back was the correct dec- decision, right? If it's con- if there's, if it's not obvious whether something is beneficial or not, you should probably default to caution. And so I think that the, that the, that the overall landscape for how we think about it is that this decision could have gone either way. There's great arguments in both directions. But for future models down the road, um, and possibly sooner than, than, than you'd expect, 'cause, you know, scaling these things up doesn't have to take that long, those ones we're, you're definitely not going to want to, to release into the wild. And so, um, I think that we, we almost view this as a test case and to see can we even design, you know, how, how do you have a society or how do you have a system that goes from having no concept of responsible disclosure, where the mere idea of not releasing something for safety reasons is unfamiliar, to a world where you say, "Okay, we have a powerful model. Let's at least think about it. Let's go through some process"? And you think about the security community, it took them a long time to design responsible disclosure, right? You know, you think about this question of, "Well, I have a security exploit. I send it to the company. The company is like, tries to prosecute me or just sit, uh, just ignores it. What do I do?" Right? And so, you know, the alternatives of, "Oh, I just, just always publish your exploits," that doesn't seem good either, right? And so it really took a long time and took this, this, uh, it was bigger than any individual, right? It's really about building a whole community that believe that, okay, we'll have this process where you send to the company, you know. If they don't act in a certain time, then you can go public. And you're not a bad person. You've done the right thing. Um, and I think that in AI, part of the, the response at GPT-2 just proves that we don't have any concept of this. Um, so that's the high-level picture.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
Um, and so I think that, I think this was, this was a really important move to make, um, and we could've maybe delayed it for GPT-3 but I'm really glad we did it for GPT-2. And so now you look at GPT-2 itself and you think about the substance of, okay, what are potential negative applications? So you have this model that's been trained on the internet, which, you know, it's also going to be a bunch of very biased data, a bunch of, you know, very, uh, offensive content in there. Uh, and, uh, you can ask it to generate content for you on basically any topic, right? You just give it a prompt and it'll just start, start writing and writes content like you see on the internet, you know, even down to, like, saying advertisement (laughs) in the middle-
- LFLex Fridman
Yeah.
- GBGreg Brockman
... of, of some of its generations. And, uh, you think about the possibilities for generating fake news or abusive content. And, you know, it's interesting seeing what people have done with, uh, you know, we released a smaller version of GPT-2 and, uh, that people have done things like try to generate...... I, you know, take, take my own Facebook message history and generate more Facebook messages like me.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
Um, and, uh, people generating fake politician, uh, uh, content or, uh, you know, there's- there's a bunch of- of things there where you at least have to think, "Is this going to be good for the world?" There's the flip side which is, I think, that there's a lot of awesome applications that we really want to see like creative, uh, applications in terms of if- if you have sci-fi authors that can work with this tool and come up with cool ideas. Like, that seems- that seems awesome. If we can write better sci-fi through the use of th- these tools... And we've actually had a bunch of people writing to us asking, "Hey, can we use it for, y- you know, a variety of different creative applications?"
- LFLex Fridman
So, the- the positive are actually pretty e- easy to imagine. Uh, they're f- uh, uh, you know, the- the usual NLP applications are really interesting. But l- let's go there. It's kinda interesting to think about a world where, uh, look at Twitter, where not just fake news but smarter and smarter bots being able to, uh, spread in an interesting complex networking way in- information that just floods out us regular human beings with our original thoughts. So, uh, what are your views of this world with GPT-20, right?
- GBGreg Brockman
Mm-hmm.
- LFLex Fridman
What do you... How do we think about it? Again, it's, like, one of those things about in the '50s trying to describe the, uh, the internet or the smartphone. What do you think about that world? The nature of information? Do we, uh, d- one possibility is that we'll always try to design systems that identify robot versus human and will do so successfully, and so we will authenticate that we're still human. And the other world is that we just accept the p- the fact that we're swimming in a sea of fake news-
- GBGreg Brockman
Mm-hmm.
- LFLex Fridman
... and just learn to swim there.
- GBGreg Brockman
Well, have- have you ever seen the, uh, there's- there's always, you know, pop- popular, uh, meme of, uh, of, uh, robot, uh, with- with a physical, physical arm and pen clicking the "I'm not a robot" button?
- LFLex Fridman
Yeah. (laughs)
- GBGreg Brockman
I think, I think that the truth is that, uh, that really trying to distinguish between robot and human is a losing battle.
- LFLex Fridman
Ultimately, you think it's a losing battle?
- GBGreg Brockman
I think it's a losing battle ultimately, right? I think that that is the, in terms of- of the content, in terms of the action that you could take. I mean, think about how CAPTCHAs have gone, right? The CAPTCHAs used to be a very nice simple, you just have this image, all of our OCR is terrible, you put a couple of- of artifacts in it. You know, humans are gonna be able to tell what- what- what it is, uh, an AI system wouldn't be able to. Today, like, I can barely do CAPTCHAs.
- LFLex Fridman
Yeah.
- GBGreg Brockman
And I think that- that- that this is just kinda where we're going. I think CAPTCHAs were- were a moment in time thing, and as AI systems become more powerful that there being human capabilities that can be measured in a very easy automated way, uh, that- that AIs will not be capable of. I think that's just, like, it's just an increasingly hard technical battle. But it's not that all hope is lost, right? When you think about, uh, how do we already authenticate ourselves, right? That, you know, we have systems, we have social security numbers if you're in the U.S. or, you know, you have- you have, uh, uh, you know, ways of identifying individual people, um, and having real world identity tied to- to digital identity seems like a step, uh, towards, you know, authenticating the source of content rather than the content itself. Um, now, there are problems with that. How can you have privacy and anonymity in a world where the only content you can really trust is, or the only way you can trust content is by looking at where it comes from? Um, and so I think that building out good reputation networks, uh, may be- may be one possible solution. But, yeah, I think that this- this question is- is not an obvious one. And I think that we, you know, maybe sooner than we think will be in a world where, you know, today I often will read a tweet and be like, "Hmm, do I feel like a real human wrote this?" Or, you know, "Do I feel like this is, like, genuine?" I feel like I can kinda judge the content a little bit. Um, and I think in the future, it just won't be the case. You look at, for example, the FCC comments on net neutrality. Uh, it came out later that millions of those were autogenerated and that the researchers were able to do various statistical tech- techniques to do that. What do you do in a world where those statistical techniques don't exist? It's just impossible to tell the difference between humans and AIs. And in fact the, uh, the- the- the most persuasive arguments are written by- by AI. All that stuff, it's not sci-fi anymore. You look at GPT-2 making a great argument for why recycling's bad for the world. You gotta read that and be like, "Huh, you're right."
- LFLex Fridman
(laughs)
- GBGreg Brockman
"We are addressing different symptoms."
- LFLex Fridman
Yeah. That's- that's quite interesting. I mean, ultimately it boils down to the physical world being the last frontier of proving, as you said, like, basically networks of people, humans vouching for humans in the physical world, and somehow the authentication, uh, ends there. I mean, if I had to ask you, I mean, you're way too eloquent for a human. So if I had to ask you to authenticate, like prove how do I know you're not a robot and how do you know I'm not a robot-
- GBGreg Brockman
Yeah.
- LFLex Fridman
... I- I think that's so far we're p- we're this, in this space, this conversation we just had, the physical movements we did, is the biggest gap between us and AI systems, is the physical m- manipulation. So maybe that's the last frontier.
- GBGreg Brockman
Well, here's another question is- is, you know, why- why is- why is solving this problem important, right? Like, what aspects are really important to us? And I think that probably where we'll end up is we'll hone in on what do we really want out of knowing if we're talking to a human, um, and, uh, and I think that, again, this comes down to identity. And so I think that- that the internet of the future, I expect to be one that will have lots of agents out there, uh, that will interact with- with you. But I think that the question of is this, you know, flesh, real flesh and blood, uh, human or is this an automated system, uh, may- may actually just be less important.
- 1:00:00 – 1:12:28
Mm-hmm. …
- GBGreg Brockman
kind of like, it's spent a long time in its, like, evolutionary history baking in all this information, getting very, very good at this predictive process. And then at runtime, I just kind of do one forward pass and, uh, and am able to generate stuff. And so, you know, there might be small tweaks to what we do in order to get the type signature right. For example, well, you know, it's not really one forward pass, right? You know, you generate symbol by symbol.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And so maybe you generate, like, a whole sequence of, of, of thoughts and you only keep, like, the last bit or something.
- LFLex Fridman
Right.
- GBGreg Brockman
Um, but I think that at the very least I would expect you have to make changes like that.
- LFLex Fridman
Ye- yeah, just exactly how we th- you said think is the process of generating thought by thought in the same kind of way, like you said, keep the last bit, the thing that we converge towards.
- GBGreg Brockman
Yep. Uh, and I think there's, there's another piece which is, which is interesting which is this out of distribution generalization, right?
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
That, like, thinking somehow lets us do that, right? That we haven't experienced a thing and yet somehow we just kind of keep refining our mental model of it. Um, this is again, something that feels tied to whatever reasoning is and maybe it's a small tweak to what we do, maybe it's many ideas and will take us many decades.
- LFLex Fridman
Yeah, so the, the assumption there, uh, j- generalization out of distribution is that it's possible to create, uh, new, new ideas.
- GBGreg Brockman
Mm-hmm.
- LFLex Fridman
Uh, the pos- you know, it's possible that nobody's ever created any new ideas and then with scaling GPT-2 to GPT-20, uh, you would, y- you would essentially generalize to all possible thoughts-
- GBGreg Brockman
Yeah.
- LFLex Fridman
... that us humans can have. (laughs)
- GBGreg Brockman
(laughs) I mean-
- LFLex Fridman
Just to, just to play devil's advocate here.
- GBGreg Brockman
Right, right, right. I mean, how many, how many, uh, new, new story ideas have we come up with since Shakespeare, right?
- LFLex Fridman
Yeah, exactly. It's... (laughs) It's just all different forms of love and drama and so on. Okay.Not sure if you read Bitter Lesson, a recent blog post by Ray Sutton.
- GBGreg Brockman
Yep, I have.
- LFLex Fridman
He basically says, uh, something that echoes some of the ideas that you've been talking about which is, uh, he says, "The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately going to, uh, ultimately win out." Uh, do you agree with this? So basically, of, and OpenAI in general, but the ideas you're exploring about coming up with methods, whether it's GPT-2 modeling or whether it's, uh, OpenAI 5 playing DOTA or a general method is, uh, better than a more fine-tuned, expert tuned method.
- GBGreg Brockman
Yeah. So I, I think that, well, one thing that I think was really interesting about the reaction to that blog post was that a lot of people have read this as saying that compute is all that matters.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And that's a very threatening idea, right? And I don't think it's a true idea either, right? It's very clear that we have algorithmic ideas that have been very important for making progress and t- to really build AGI, you wanna push as far as you can on the computational scale, and you wanna push as far as you can on human, human ingenuity. And so I think you need both. But I think the way that you phrased the question is actually very good, right? That it's really about what kind of ideas should we be striving for? And absolutely, if you can find a scalable idea, you pour more compute into, you pour more data into it, it gets better. Like, that's, that's the real holy grail. And so I think that, uh, that, that the answer to, to the question I think is, is yes.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
Um, that, that, that's really how we think about it. And that part of why we're excited about the power of deep learning, the potential for building AGI is because we look at the systems that exist in the most successful AI systems, and we realize that y- you scale those up, they're gonna work better.
- LFLex Fridman
Mm-hmm.
- GBGreg Brockman
And I think that that scalability is something that really gives us hope for being able to build transformative systems.
- LFLex Fridman
So I'll tell you this, uh, partially an emotional, m- you know, a thing that, a response that people often have is compute is so important for state-of-the-art performance, individual developers, maybe a 13-year-old sitting somewhere in Kansas or something like that, you know, they're sitting, they, they might not even have a GPU and, or may have a single GPU, a 1080 or something like that, and there's this feeling like, "Well, how can I possibly compete or contribute to this world of AI if, uh, scale is so important?" So for, if you can comment on that, and in general, do you think we need to also in the future focus on, uh, democratizing compute resources more, more or as much as we democratize the algorithms?
- GBGreg Brockman
Well, so the way that I think about it is that there's this space of, of possible progress, right? There's a space of ideas and sort of systems that, that will work, that will move us forward, and there's a portion of that space, and to some extent, an increasingly significant portion of that space, that does just require massive compute resources.
- LFLex Fridman
Right.
Episode duration: 1:25:06
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode bIrEM2FbOLU
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome