Michael I. Jordan: Machine Learning, Recommender Systems, and Future of AI | Lex Fridman Podcast #74

Lex Fridman 0:00 The following is a conversation with Michael I. Jordan, a professor at Berkeley and one of the most influential people in the history of machine learning, statistics, and artificial intelligence. He has been cited over 170,000 times and he has mentored many of the world-class researchers defining the field of AI today, including Andrew Ng, Zoubin Gharahmani, Ben Taskar, and Yoshua Bengio. All this, to me, is as impressive as the over 32,000 points and the six NBA championships of the Michael J. Jordan of basketball fame. There's a non-zero probability that I talk to the other Michael Jordan, given my connection to and love of the Chicago Bulls of the '90s, but if I had to pick one, I'm going with the Michael Jordan of statistics and computer science, or as Yann LeCun calls him, the Miles Davis of machine learning. In his blog post titled, Artificial Intelligence: The Revolution Hasn't Happened Yet, Michael argues for broadening the scope of the artificial intelligence field. In many ways, the underlying spirit of this podcast is the same: to see artificial intelligence as a deeply human endeavor, to not only engineer algorithms and robots, but to understand and empower human beings at all levels of abstractions, from the individual to our civilization as a whole. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, give us five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter @LexFridman, spelled F-R-I-D-M-A-N. As usual, I'll do one or two minutes of ads now and never any ads in the middle that could break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code LEXPODCAST. Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App does fractional share trading, let me mention that the order execution algorithm that worked behind the scenes to create the abstraction of the fractional orders is, to me, an algorithmic marvel. So big props for the Cash App engineers for solving a hard problem that, in the end, provides an easy interface that takes a step up to the next layer of abstraction over the stock market, making trading more accessible for new investors and diversification much easier. So once again, if you get Cash App from the App Store or Google Play and use the code LEXPODCAST, you'll get $10 and Cash App will also donate $10 to FIRST, one of my favorite organizations that is helping to advance robotics and STEM education for young people around the world. And now, here's my conversation with Michael I. Jordan.

Lex Fridman 3:02 Given that you're one of the greats in the field of AI, machine learning, computer science, and so on, you're trivially called the Michael Jordan of machine learning. Although, as you know, you were born first, so technically MJ is the Michael I. Jordan of basketball. But anyway, my, my favorite is Yann LeCun calling you the Miles Davis of machine learning, because as he says, you reinvent yourself periodically and sometimes (laughs) leave fans scratching their heads after you change directions. So, can you put, at first, your historian hat on and give a history of computer science and AI as you saw it, as you experienced it, including the four generations of AI successes that I've seen you talk about?

Michael I. Jordan 3:48 Sure. Yeah, first of all, I much prefer Yann's, uh, metaphor. Um, Miles Davis is, uh, was a real explorer in jazz and, um, he had a coherent story. So I think I have one and... But it's not just the one you lived, it's the one you think about later, right? What a good historian does is they look back and they revisit. Um, I think what's happening right now is not AI. That was an intellectual aspiration, um, that's still alive today as an aspiration. But I think this is akin to the development of chemical engineering from chemistry or electrical engineering from, from electromagnetism. So if you go back to the '30s or '40s, there wasn't yet chemical engineering. There was chemistry, there was fluid flow, there was mechanics, and so on. Um, but people pretty clearly viewed, uh, interesting goals to try to build factories that, um, you know, made chemicals, products, and do it viably, safely, make good ones, do it at scale. Uh, so people started to try to do that, of course, and some factories worked, some didn't. You know, some were not viable, some exploded. But in parallel, uh, developed a whole field called chemical engineering. Right? And chemical engineering's a field. It's, it's no, no bones about it. It has theoretical aspects to it, it has practical aspects. It's not just engineering, quote unquote, it's the real thing, and real concepts are needed. Now, same thing with electrical engineering. You know, there was Maxwell's equations, which in some sense were everything you need to know about electromagnetism, but you needed to figure out how to build circuits, how to build modules, how to put them together, how to bring electricity from one point to another safely, and so on and so forth. So a whole field was developed called electrical engineering. All right? I think that's what's happening right now, is that we have-

Lex Fridman 5:28 But-

Michael I. Jordan 5:28 ... we have a proto-field, which is statistics, comput- more the theoretical side of, the algorithmic side of computer science. That was enough to start to build things. But what things? Systems that bring value to human beings and use human data and mix in human decisions. The engineering side of that is all ad hoc. That's what's emerging. In fact, if you wanna call machine learning a field, I think that's what it is. It's a proto-form of engineering based on statistical and computational ideas of previous generations.

Lex Fridman 5:56 But it... Do you think there's something deeper about AI in its dreams and aspirations as compared to chemical engineering and electrical engineering?

Michael I. Jordan 6:03 No. Well, the dreams and aspirations, maybe, but those are from, those are 500 years from now. I think that that's like the Greeks sitting there and saying, "It would be neat to get to the moon someday."

Lex Fridman 6:11 Right.

Michael I. Jordan 6:12 Um, I think we have no clue how the brain does computation. Uh, we're just clueless. We're like, we're even worse than the Greeks, uh, on most anything interesting, uh, scientifically o- of our era.

Lex Fridman 6:23 Can you linger on that just for a moment? Because you stand...... not completely unique, but a little bit unique in that, in the clarity of that. Can you, can you elaborate your intuition of why we're, like, where we stand in our understanding of the human brain? And a lot of people say, you know, scientists say we're not very far in understanding the human brain.

Michael I. Jordan 6:41 Yeah.

Lex Fridman 6:41 But you're like, you're saying, "We're in the dark here."

Michael I. Jordan 6:44 Well, I know I'm not unique, I don't even think in the clarity, but if you talk to real neuroscientists that really study real synapses or real neurons, they agree. They agree. It's, it's a 100 year, hundreds of year task and they're building it up slowly but surely. What the signal is there is not clear. We think we have all of our metaphors. We think it's electrical, maybe it's chemical. It's, it's a whole soup. It's ions and proteins and it's a cell. And that's even around like, a single synapse. If you look at an electromyograph of a single synapse, it's a, it's a city of its own, and that's one little thing on a dendritic tree which is extremely complicated, you know, electrochemical thing, and it's doing these spikes and voltages have been flying around, and then proteins are taking that and taking it down into the DNA and who knows what. Um, so it is the problem of the next few centuries. It is fantastic, but we have our metaphors about it. Is it an economic device? Is it like the immune system or is it like a layered, you know, set of comput- you know, arithmetic computations? What... We have all these metaphors and they're fun, um, but that's not real science, uh, per se. There is neuroscience. That's not neuroscience, all right? That- that's, that's like the Greeks speculating about how to get to the moon. Fun, right? And I think that I like to say this fairly strongly 'cause I think a lot of young people think that we're on the verge, because a lot of people who don't talk about it clearly let it be understood that yes, we kind of... This is brain-inspired, we're kind of close, you know, breakthroughs are on the horizon. And then scrupulous people sometimes who need money for their labs, um, that's why I'm saying scrupulous, but people will oversell. "Um, I need money for my lab. I'm gonna... I'm studying, you- you know, computational neuroscience, um, I'm gonna oversell it." And so there's been too much of that.

Lex Fridman 8:25 So, uh, let's step into the slight, the gray area between metaphor and engineering with, uh, I'm not sure if you're familiar with, uh, brain computer interfaces. So, a company like Elon Musk has Neuralink that's working on con- putting electrodes into, into the brain and trying to be able to read, both read and send electrical signals. Just as you said, even the basic mechanism of communication in the brain is not something we understand, but do you hope without understanding the fundamental principles of how the brain works, we'll be able to do something interesting at that gray area of metaphor and-

Michael I. Jordan 9:06 It's not my area, so I- I hope in the sense like anybody else hopes for some interesting things to happen from research.

Lex Fridman 9:12 Yeah.

Michael I. Jordan 9:12 I would expect more something like Alzheimer's will get figured out from modern neuroscience. That, you know, a lot of, there's a lot of human suffering based on brain disease, and we throw things like lithium at the brain. It kind of works. No one has a clue why. That's not quite true, but you know, mostly we don't know, and that's even just about the biochemistry of the brain and how it leads to mood swings and so on. How thought emerges from that, we just, we- we're really, really completely dim, so that you might want to hook up electrodes and try to do some signal processing on that and try to find patterns, fine. You know, by all means go for it. It's just not scientific at this point. It's just, it's... So it's like kind of sitting in a satellite and watching the emissions from a city and trying to infer things about the microeconomy even though you don't have microeconomic concepts. I mean, it's really that kind of thing. And, and so yes, can you find some signals that do something interesting or useful? Can you control a cursor, uh, or mouse with your brain? Yeah, absolutely. You know, and, and I can imagine business models based on that, and even, you know, medical applications of that. But from there to understanding algorithms that allow us to really tie in deeply to- from the brain to the computer, you know, I just... No, I don't agree with Elon Musk. I don't think that's even, that's not for our generations, not even for this century.

Lex Fridman 10:26 So, just, uh, in the hopes of getting you to dream, uh, you've mentioned Kolmogorov and Turing might pop up. Do you think that there might be breakthroughs that will get you to sit back in five, 10 years and say, "Wow"?

Michael I. Jordan 10:43 Oh, I- I'm sure there will be, but I don't think that there'll be demos that impress me.

Lex Fridman 10:48 Mm-hmm.

Michael I. Jordan 10:49 I don't think that having a computer call a restaurant and pretend to be a human is a breakthrough.

Lex Fridman 10:56 Right, right.

Michael I. Jordan 10:56 And people, you know, some people present it as such. Uh, it's imitating human intelligence. It's even putting coughs (laughs)

Lex Fridman 11:03 Yeah.

Michael I. Jordan 11:04 ... in the thing to make a bit of a PR stunt. And so fine, the, the world runs on those things too, uh, and I don't want to diminish all the hard work and engineering that goes behind things like that and, and the ultimate value to the human race, but that's not scientific understanding. And, and I know the people that work on these things, they are after scientific understanding. You know, in the meantime, they've got to kind of, you know, the train's got to run and they got mouths to feed and they got things to do, and there's nothing wrong with all that. Um, I would call that, though, just engineering, and I want to distinguish that between an engineering field like electrical engineering ...and chemical engineering that original- that originally emerged that had real principles and you really knew what you were doing and you had a little scientific understanding, maybe not even complete-

Lex Fridman 11:43 Mm-hmm.

Michael I. Jordan 11:43 ... so it became more predictable and it was, really gave value to human life because it was understood. And, and so we have to, we don't want to muddle too much these waters of, uh, you know, what we're able to do versus what we really can do, uh, in a way that's gonna impress the next... So I don't, I don't need to be wowed, but I- I think that someone comes along in 20 years, a younger person who's absorbed all the, uh, the technology and- and for them to be wowed, I think they have to be more deeply impressed. A young Kolmogorov would not be wowed by some of the stunts that you see right now coming from the big companies.

Lex Fridman 12:13 The demos.

Michael I. Jordan 12:14 The demos.

Lex Fridman 12:14 But do you think the breakthroughs from Kolmogorov-... would be, and give this question a chance, do you think they'll be in the scientific fundamental principles arena? Or do you think it's possible to have fundamental breakthroughs in engineering? Meaning, you know, I would say some of the things that Elon Musk is working with, SpaceX and then others, sort of trying to revolutionize the fundamentals of engineering, of manufacturing, of, of saying, "Here's a problem we know how to do a demo of," and actually-

Michael I. Jordan 12:43 Yeah.

Lex Fridman 12:43 ... taking it to scale.

Michael I. Jordan 12:44 Yeah. So, so there's gonna be all kinds of breakthroughs, I just don't like that terminology. I'm a scientist and I work on things day in and day out and things move along and then eventually say, "Wow, something happened," but it's, I don't like that language very much. Uh, also, I don't like to prize, uh, theoretical breakthroughs over practical ones. Um, I, I tend to be more of a theoretician and I think there's lots to do in, in that arena right now. Um, and so I wouldn't point to the Kolmogorovs, I might point to the Edisons of the era, and maybe Musk is a bit more like that. But, um, you know, Musk, God bless him, also will, will say things about AI that he knows very little about a- and, and he doesn't know what he... He, he, he is, you know, leads people astray when he talks about things he doesn't know anything about. Trying to program a computer to understand natural language, to be involved in a dialogue like we're having right now, ain't gonna happen in our lifetime. You could fake it. You can mimic, sort of take old sentences that humans use and retread them, but the deep understanding of language? No, it's not gonna happen. And so from that, you know, I hope you can perceive that the deeper, yet deeper kind of aspects in intelligence are not gonna happen. Now, will there be breakthroughs? You know, I think that Google was a breakthrough.

Lex Fridman 13:48 Right.

Michael I. Jordan 13:48 I think Amazon's a breakthrough. You know? I think Uber is a breakthrough. You know, that bring value to human beings at scale in new, brand new ways based on data flows and, and so on. A lot of these things are slightly broken because there's not an em- a kind of a engineering field that takes economic value in context of data and, and that, you know, planetary scale and, and worries about all the externalities, the privacy. You know, we, we don't have that feel, so we don't think these things through very well. But I see that as emerging and that will be con- that will... You know, looking back from 100 years, that will be cons- a breakthrough in this era. Just like electrical engineering was a breakthrough in the early part of the last century and chemical engineering was a breakthrough.

Lex Fridman 14:24 So the scale, the markets that you talk about and we'll get to, uh, will be seen as sort of breakthrough. And we're in the very early days of really doing interesting stuff there. And we'll get to that, but it's... Just taking a quick step back, can you give, (laughs) uh... We'll kind of throw off the historian hat. I mean, you briefly said that, uh, im- the history of AI kind of mimics the history of chemical engineering. But-

Michael I. Jordan 14:49 I keep saying machine learning and you keep want to say AI. Just to let you know, I don't... You know? I, I, I resist that.

Lex Fridman 14:54 So that's-

Michael I. Jordan 14:54 I don't think this is about... AI really was John McCarthy, as almost a philosopher-

Lex Fridman 15:00 Got it.

Michael I. Jordan 15:00 ... saying, "Wouldn't it be cool if we could put thought in a computer? If we could mimic the human capability to think or put intelligence in in some sense into a computer?" That's an interesting philosophical question and he wanted to make it more than philosophy. He wanted to actually write down logical formula and algorithms that would do that. And that is a perfectly valid, reasonable thing to do.

Lex Fridman 15:19 Yeah.

Michael I. Jordan 15:20 That's not what's happening in this era. Right?

Lex Fridman 15:22 So, so the reason I keep saying AI actually, and I'd love to hear what you think about it. Machine learning has, um, has a very particular set of methods and tools.

Michael I. Jordan 15:34 Maybe your version of it is. Mine doesn't.

Lex Fridman 15:36 No, it doesn't.

Michael I. Jordan 15:36 Mine is very, very open. It does optimization, it does sampling, it does, uh-

Lex Fridman 15:40 So systems that learn is what machine learning is?

Michael I. Jordan 15:42 Systems that learn and make decisions.

Lex Fridman 15:44 And make decisions. So we're just-

Michael I. Jordan 15:45 So it's not just pattern recognition and li- fr-

Lex Fridman 15:47 Got it.

Michael I. Jordan 15:47 ... you know, finding patterns. It's all about making decisions in real worlds and having closed feedback loops.

Lex Fridman 15:52 So s- something like symbolic AI, expert systems, reasoning systems, knowledge-based representation, all of those kinds of things, search. Does that neighbor fit into what you think of as machine learning?

Michael I. Jordan 16:04 So I don't even like the word machine learning. I think that what the field you're talking about-

Lex Fridman 16:07 Yeah.

Michael I. Jordan 16:07 ... is all about making large collections of decisions under uncertainty by large collections of entities.

Lex Fridman 16:12 Yes.

Michael I. Jordan 16:13 Right? And there are principles for that, at that scale. You don't have to say the principles are for a single entity that's making decisions, a single agent or a single human. It really immediately goes to the network-

Lex Fridman 16:22 Distribution.

Michael I. Jordan 16:22 ... of decisions.

Lex Fridman 16:22 Is there a good word for that? Or not?

Michael I. Jordan 16:24 No, there's no good words for any of this. That's kind of part of the problem. Um, so, uh, we can continue the conversation and use AI for all of that. I just want to kind of raise-

Lex Fridman 16:31 Clarify, yeah.

Michael I. Jordan 16:31 ... the flag here that this is not about... We don't know what intelligence is. A real intelligence. We don't know much about abstraction and reasoning at the level of humans. We don't have a clue. We're not trying to build that because we don't have a clue. Eventually, it may emerge. There may... I don't know if there'll be breakthroughs, but eventually, we'll start to get glimmers of that. It's not what's happening right now though, okay? We're taking data, we're trying to make good decisions based on that. We're trying to do it at scale, we're trying to do it economically viably. We're trying to build markets, we're trying to create value at that scale. Um, and s- aspects of this will look intelligent. They will look... Computers were so dumb before, they will seem more intelligent. We will use that buzzword of intelligence, so we can use it in that sense. But, you know. So machine learning, uh, you can scope it narrowly as just learning from data and pattern recognition. But whatever I... When I talk about these topics, I... Maybe data science is another word you could throw in the mix. Um, it really is important that the decisions are i- is... As part of it, it's consequential decisions in the real world. Or am I gonna have a medical operation? Am I gonna drive down this street? You know, things that where there's scarcity, uh, things that impact other human beings or other, you know, the environment and so on. How do I do that based on data? How do I do that adaptively? How do I use computers to help those kind of things go forward? Whatever you wanna call that. So let's call it AI, let's agree to call it AI. But it's, um... Let's, let's not say that what the goal of that is is intelligence. The goal of that is really good working systems at planatory scale that we've never seen before.

Lex Fridman 17:56 So reclaim the word AI from the Dartmouth conference from many decades ago of the dream of human-

Michael I. Jordan 18:01 I don't wanna reclaim it. I want a new word.

Lex Fridman 18:03 A new word.

Michael I. Jordan 18:03 I, I think it was a bad choice. I mean, I, I... You know, I... If you read one of my little things, um, the history was basically that, uh, McCarthy needed a new name 'cause cybernetics already existed.

Lex Fridman 18:11 (laughs)

Michael I. Jordan 18:12 And he didn't like... You know, no one really liked Norbert Wiener. Norbert Wiener was kind of an island to himself and he felt that he had encompassed all this, and in some sense, he did. You look at the language of cybernetics, it was everything we're talking about. It was control theory and signal processing and some notions of intelligence and closed feedback loops and data. It was all there.... it's just not a word that lived on partly 'cause of the, maybe the personalities. Um, but McCarthy needed a new word to say, "I'm different from you. I'm not part of your, your show. I got my own." Invented this word, um, and again, as a kind of a t- t- t- thinking forward about the movies that would be made about it, uh, it was a great choice. But thinking forward about creating a sober, academic, and real-world discipline, it was a terrible choice because it, it led to promises that are not true, that we understand, we understand artificial perhaps, but we don't understand intelligence.

Lex Fridman 18:59 As a small tangent,

Lex Fridman 19:00 because you're one of the great personalities of machine learning, whatever the heck you call the field, the... Do you think science progresses by personalities or by the fundamental principles and theories and research that's outside of personalities?

Michael I. Jordan 19:14 Uh, both. A- and I wouldn't say there should be one kind of personality. I have mine and I have my preferences and, uh, uh, I have a kind of network around me, uh, that feeds me. And, and some of them agree with me and some of them disagree, but, you know, all kinds of personalities are needed. Um, right now, I think the personality that is a little too exuberant, a little bit too ready to promise the moon is, is a little bit too much in ascendance.

Lex Fridman 19:34 Yeah.

Michael I. Jordan 19:35 Um, and I do, I do think that that's... There's some good to that. It certainly attracts lots of young people to our field. But a lot of those people come in with strong misconceptions and they have to then unlearn those and then find something, you know, to do. Um, and so I think there's just gotta be some, you know, multiple voices and there's... I didn't, I wasn't hearing enough of the more sober voice.

Lex Fridman 19:54 So,

Lex Fridman 19:55 uh, as a continuation of a fun tangent, and speaking of vibrant personalities, uh, what would you say is the most interesting disagreement you have with Yann LeCun?

Michael I. Jordan 20:07 So Yann's an old friend and I just say, say that, uh, I, I don't think we disagree about very much really.

Lex Fridman 20:12 Right.

Michael I. Jordan 20:13 Uh, h- he and I both kind of have a let's build it kind of mentality and does it work k- kind of mentality, and, uh, kind of concrete. Um, we both speak French and we speak French when we're together and we have, we have a lot o', a lot in common. Um, and so, uh, you know, if one wanted to highlight a, uh, a disagreement, it's not really a fundamental one. I think it's just kind of what we're emphasizing. Um, Yann has, uh, emphasized pattern recognition and, uh, has emphasized prediction. All right? So, you know, um... And it's interesting to try to take that as far as you can. If you could do perfect prediction, what would that give you, kind of as a thought experiment? Um, and, um, I think that's, um, way too limited. Um, we cannot do perfect prediction. We will never have the data sets that allow me to figure out what you're about ready to do, what question you're gonna ask next.

Lex Fridman 21:00 Right.

Michael I. Jordan 21:00 I have no clue. I will never know such things. Moreover, most of us find ourselves during the day in all kinds of situations we had no anticipation of, that are kind of very, very s- that are novel in various ways. And in that moment, we want to think through what we want. And also there's gonna be market forces acting on us. Uh, I'd like to go down that street, but now it's full because there's a crane in the street. I gotta, I gotta think about that. I gotta think about what I might really want here. And I gotta sort of think about how much it costs me to do this action versus this action. I gotta think about, uh, the risks involved. You know, a lot of our current pattern recognition and prediction systems don't do any risk evaluations. They have no error bars, right? I gotta think about other people's decisions around me. I gotta think about a collection of my decisions. Even just thinking about, like, a medical treatment. You know, I'm not gonna take a, the prediction of a neural net about my health, about something consequential, am I about ready to have a heart attack because some number is over .7. Even if you had all the data in the world that had ever been collected about heart attacks, uh, better than any doctor ever had, I'm not gonna trust the output of that neural net to predict my heart attack. I'm gonna wanna ask what if questions around that.

Lex Fridman 22:06 Right.

Michael I. Jordan 22:06 I'm gonna wanna look at some us- or other possible data I didn't have, causal things. I'm gonna wanna have a dialogue with a doctor about things we didn't think about when we gathered the data. You know, it... I could go on and on, I hope you can see.

Lex Fridman 22:17 Yeah.

Michael I. Jordan 22:17 And, and I don't... I think that if you say prediction's everything, that, that, that you're missing all of this stuff. Um, and so prediction plus decision making is everything, but both of them are equally important. And so the field has emphasized prediction. Yann, rightly so, has seen how powerful that is. But, um, at the cost of people not being aware that decision making is where the rubber really hits the road, where human lives are at stake, where risks are being taken, where you gotta gather more data, you gotta think about the error bars, you gotta think about the consequences of your decisions on others, you gotta think about the economy around your decisions, blah, blah, blah, blah, blah. I'm not the only one working on those, but we're a smaller tribe, and right now we're not the, the one that people talk about the most. Um, but, you know, if you go out in the real world, in industry, um, you know, at Amazon, I'd say half the people there are working on decision making and the other half are doing, you know, the pattern recognition. It's important.

Lex Fridman 23:05 And the words of pattern recognition and prediction, I think the distinction there, not to linger on words, but the distinction there is more a constrained sort of in the lab data set versus decision making is talking about consequential decisions in the real world under the messiness and the uncertainty of the real world, and just the whole of it. The whole mess of it that actually touches human beings and scale, like you said, market forces. That's the, that's the distinction.

Michael I. Jordan 23:30 Yeah. I- it helps add those, that perspective, that broader perspective.

Lex Fridman 23:33 Right.

Michael I. Jordan 23:33 Y- You're right. I totally agree. Uh, on the other hand, if you're a real prediction person, of course you want it to be in the real world.

Lex Fridman 23:38 Right.

Michael I. Jordan 23:38 You wanna predict real world events. I'm just saying that's not possible with just data sets, uh, that it has to be in the context of, you know, uh, strategic things that someone's doing, data they might gather, things they could have gathered, the reasoning process around data. I- it's not just taking data and making predictions based on the data.

Lex Fridman 23:53 So one of the, the things that you're working on, uh, I'm sure there's others working on it, but I don't hear often, uh, it talked about, especially in, in the clarity that you talk about it, and I think is both the most exciting and the most concerning area of AI i- in terms of decision making. So you've talked about AI systems that help make decisions that scale in a distributive way, millions, billions decisions, in sort of markets of decisions. Can you, as a starting point, sort of give an example of a system that you think about when you're thinking about these kinds of systems?

Michael I. Jordan 24:27 Uh, yeah. So first of all, you're, you're absolutely getting into some territory which will... I, I will be beyond my expertise and the... and there are lots of things that are gonna be very non-obvious to think about. Just like, just the... Uh, again, I like to think about history a little bit, but think about, put yourself back in the '60s. There was kind of a banking system that wasn't computerized really.

Lex Fridman 24:43 Mm-hmm.

Michael I. Jordan 24:43 There was... Then there was database theory emerging.... and database people had to think about, "How do I actually not just move data around, but actual money, and have it be, you know, valid, and have transactions at ATMs happen that are actually, you know, all valid," and, and so on and so forth. So, that's the kind of issues you get into when you start to get serious about sorts of things like this. Um, I like to think about as kind of almost a thought experiment to help me think, uh, something simpler, which is, um, the music market. And, uh, 'cause there is, uh, to first order, there is no music market in the world right now, in the co- in our country, for sure. Uh, there are, uh, something called, things called record companies, and they make money, uh, and they prop up a few, um, really good musicians and make them superstars, and they all make huge amounts of money. Um, but there's a long tail of huge numbers of people that make lots and lots of really good music that is actually listened to by more people than the famous people. Um, um, uh, they are not in a market. They cannot have a career. They do not make money.

Lex Fridman 25:43 The creators, the creator's side.

Michael I. Jordan 25:45 The creators, the so-called influencers or whatever. That diminishes who they are, right? So, there are people who make extremely good music, especially in the hip hop or Latin wor- world these days. Uh, they do it on their laptop, that's what they do, um, on the weekend, uh, and they have, uh, uh, another job during the week. They put it up on SoundCloud or other sites. Eventually it gets streamed. It now gets turned into bits. It's not economically valuable. The, the information is lost. It gets put up there, people stream it. Y- you walk around in, uh, a big city, you see people with headphones on, you know, especially young kids listening to music all the time. If you look at the data, none of the m- very little of the music they listen to is, is the famous people's music. And none of it's old music, it's all the latest stuff. But the people who made that latest stuff are like, some 16-year-olds somewhere who will never make a career out of this, who will never make money. Of course, there'll be a few counterexamples, the record companies incentivized to pick out a few and, and highlight them. Long story short, there's a missing market there. There is not a consumer-producer relationship at the level of the actual creative acts. Um, the Pipelines and Spotifys of the world that take this stuff and stream it along, they make money off of subscriptions, or advertising and those things. They're making the money, all right? And then they will offer bits and pieces of it to a few people, again, to highlight that, you know, they're ... they simulate a market. Anyway, a real market would be if you're a creator of music, that you actually are somebody who's good enough that people want to listen to you, uh, you should have the data available to you. There should be a dashboard showing a map of the United States, showing last week, here's all the places your songs were listened to. It should be transparent, um, vettable, so that if someone in, down in Providence sees that, uh, you're being listened to 10,000 times in Providence, that they know that's real data, you know it's real data. They will have you come give a show down there. They will broadcast to the people who've been listening to you that you're coming. If you do this right, you could, you could, you know, go down there and make $20,000. You do that three times through your career, you start to have a career. So, in this sense, AI creates jobs. It's not about taking away human jobs, it's creating new jobs because it creates a new market. Once you've created a market, you've now connected up producers and consumers. You know, the mu- person who's making the music can say to someone who comes to their shows a lot, "Hey, I'll play at your daughter's wedding for $10,000." You'll say, "$8,000." They'll say, "$9,000." Um, y- then you, you again, you, you can now get an income up to $100,000. You're not gonna be a millionaire, all right? And, and now even think about really the value of music is in these personal connections, even so much so that, um, a, a young kid wants to wear a T-shirt with the, their favorite musician's signature on it, right? So, if, if they listen to the music on the internet, the internet should be able to provide them with a button that they push and the merchandise arrives the next day. We can do that, right? And now, why should we do that? Well, because the kid who bought the shirt'll be happy, but more the person who made the music will get the money. There's no advertising needed, right? So, you could create markets between producers and consumers, take 5% cut. Your company will be perfectly, uh, sound. It'll go forward into the future and it will create new markets, and that raises human happiness. Um, now, this seems like, well, this is easy. Just create this dashboard, kind of create some connections and all that. But, uh, you know, if you think about Uber or whatever, you think about the, the challenges in the real world of doing things like this. And there are actually new principles going to be needed. If you're trying to create a new kind of two-way market at a different scale that's ever been done before, there's gonna be, um, you know, uh, unwanted aspects of the market. There'll be bad people. There'll be, you know, um, the data will get used in wrong ways. You know, it'll fail in some ways. It won't deliver val- You have to think that through. Just like anyone who, like, ran a big auction or, you know, or ran a big matching service in economics will think these things through. And so, that maybe doesn't get at all the huge issues that can arise when you start to create markets, but it starts for at least, uh, for me, solidify my thoughts and let- allow me to move forward in my own thinking.

Lex Fridman 29:28 Yeah. So, uh, I talked to head of research at Spotify actually, and I think their long-term goal they've said is to, uh, have at least one million creators h- make, uh, make a comfortable living putting on Spotify. So, in, and ... I, I think you articulate a really nice vision of, uh, the world in the digital s- and the cyberspace of markets. What, what do you think companies like Spotify or YouTube or Netflix can do to create such markets? Is it an AI problem? Is it an interface problem, so interface design? Is it, um, some other kind of ... What, is it an economics problem? Who, who should they hire (laughs) to solve these problems?

Michael I. Jordan 30:15 Well, part of it's not just top down. So, the Silicon Valley has this attitude that they know how to do it. They will-

Lex Fridman 30:20 Right.

Michael I. Jordan 30:20 ... create the system, just like Google did with the search box, that will be so good that they'll just, everyone will adopt that.

Lex Fridman 30:25 Yeah.

Michael I. Jordan 30:25 Right? Um, it's not, it's, it, it's everything you said, but really, I think missing the kind of culture.

Lex Fridman 30:31 Mm-hmm.

Michael I. Jordan 30:31 All right? So, it's literally that 16-year-old who's cr- who's able to create the songs. You don't create that as a Silicon Valley entity. You don't hire them, per se.

Lex Fridman 30:38 Okay.

Michael I. Jordan 30:39 Right? You have to create an ecosystem in which they are wanted and that they're belong, right? And so you have to have some cred- cultural credibility to do things like this. You know, Netflix, to their credit, wanted some of that s- credibility. They created shows, you know, content. They call it content. It's such a terrible word, but it's cul- it's culture.

Lex Fridman 30:56 Yeah.

Michael I. Jordan 30:56 Right? And so with movies you can kind of go give a large sum of money to somebody graduating from the USC film school, uh-... uh, it's a whole thing of its own, but it's kind of, like, rich white people's thing to do, you know? And, you know, American culture has not been so much about rich white people. (laughs) It's been about all the immigrants, all the af- all the Africans who came and brought that culture and those, th- those rhythms and, and that, that, to, to, to this world and created this whole new thing, you know, American culture. And, and so, companies can't artificially create that. They can't just say, "Hey, we're here. Uh, we're gonna buy it up." You got a partner.

Lex Fridman 31:30 Right.

Michael I. Jordan 31:31 And, um, so, but anyway, you know, not to denigrate, these companies are all trying and they should, and they, they, they're pr- they... I'm sure they're asking these questions and some of them are even g- making an effort. But it, it, it is partly a respect the culture as you are a te- as a technology person. You gotta blend your technology with cultural, with, with cultural, uh, you know, meaning.

Lex Fridman 31:49 How much of a role do you think the algorithm through machine learning has in connecting the consumer to the creator, sort of, uh, the recommender system aspect of this?

Michael I. Jordan 31:59 Yeah. It's a great question. I think pretty high. Recomme- y- you know, um, there's no magic in the algorithms, but a good recommender system is way better than a bad recommender system. And, uh, recommender systems was a billion-dollar industry back even, you know, 10, 20 years ago. Um, and it continues to be extremely important going forward.

Lex Fridman 32:17 What's your favorite recommender system, just so we can put something-

Michael I. Jordan 32:19 Well, just historically, I was one of the, you know, when I first went to Amazon, uh, you know, I first didn't like Amazon 'cause they put the bookkeeper out of business or the library, you know, the local, uh, booksellers went out of business. Um, I've come to accept that there, you know, there probably are more books being sold now and more people reading them than ever before. Uh, and then lo- local book st- stores are coming back. So, you know, that's how economics sometimes work. You go up and you go down. Um, but, uh, anyway, when I finally started going there (laughs) and I bought a few books, I was really pleased to see another few books being recommended to me that I never would've thought of. Uh, and I bought a bunch of them, so they obviously had a good business model. Um, but I learned things, and I still, to this day, kind of browse using that service. Um, and I think lots of people get a lot, you know, they're... That, that, that is a good aspect of a recommendation system. I'm learning from my peers in a, in a, in an indirect way. Um, and their algorithms are not meant to th- have them impose what we, what we learn. It really is trying to find out what's in the data. Uh, it doesn't work so well for other kind of entities, but that's just the complexity of human life, like shirts. You know, I'm not gonna g- get recommendations on shirts. And, uh, but that's, that's, that's interesting. Uh, if you try to recommend, um, uh, restaurants, it's, it's, it's a, it's, it's di- it's hard. It's hard to do it at scale. And, and, um, but, uh, a blend of recommendation systems with other, um, uh, economic ideas, uh, matchings and so on is really, really still very open research-wise, and there's new companies that are gonna emerge that do that well.

Lex Fridman 33:48 What, what do you think is going to the messy difficult land of, say, politics and things like that, that YouTube and Twitter have to deal with in terms of recommendation systems, being able to suggest, uh, I think Facebook just launched Facebook News. So they're having, uh, be- recommend the kind of news that are most likely for you to be interesting. Do you think this is a S- AI solvable, again, whatever term we wanna use, do you think it's a solvable problem for machines or is it a deeply human problem that sounds solvable?

Michael I. Jordan 34:18 Uh, so I don't even think about it at that level. I think that what's broken with some of these companies, it's all monetization via advertising. They're not... At least Facebook. Let's... I wanna critique them, but they didn't really try to connect a producer and a consumer in an economic way, right? No one wants to pay for anything.

Lex Fridman 34:34 Right.

Michael I. Jordan 34:34 And so they all, you know, starting with Google, then Facebook, they went back to the playbook of, you know, the, the television companies back in the day. No one wanted to pay for this signal. They will pay for the TV box, but not for the signal, at least back in the day. And so advertising kind of filled that gap. And advertising was new and interesting and it somehow didn't take over our lives quite, right? Uh, fast forward, Google provides a service that people don't wanna pay for. Um, and so, sur- somewhat surprisingly in the '90s, they made, ended up making huge amounts. They cornered the advertising market. It didn't seem like that was gonna happen, at least to me. Um, these little things on the right-hand side of the screen just did not seem all that economically interesting. But that, companies had maybe no other choice. The TV market was going away and billboards and so on. Um, so they've... they got it. And I think that sadly that, uh, Google just has m- it was doing so well with that at making such money. They didn't think much more about how... Wait a minute, is there a producer-consumer relationship to be set up here, not just, uh, between us and the advertisers market to be created? Is there an actual market between the producer and consumer? They're the producers, the person who created that video clip, the person that made that website, the person who could make more such things, the person who could adjust it and as a function of demand, the person on the other side who's asking for different kinds of things, you know? So, you see glimmers of that now. There's influencers and there's kind of a little glimmering of a market, but it should have been done 20 years ago. It should have been thought about. It should have been created in parallel with the advertising ecosystem. Uh, and then Facebook inherited that, and I think they also didn't think very much about that. So fast forward, and now they are making huge amounts of money off of advertising and the news thing and all these clicks is just, is feeding the advertising. It's all connected up to the advertising. So you want more people to click on certain things because that money flows to you, Facebook. You're very much incentivized to do that. And when you start to find it's breaking, so people are telling you, "Well, we're getting into some troubles." You try to adjust it with your smart AI algorithms.

Lex Fridman 36:26 Mm-hmm.

Michael I. Jordan 36:27 Right? And figure out what are bad clicks. "Oh, maybe it shouldn't be click through rated. Should we..." So, I find that pretty much hopeless. Uh, it does get into all the complexities of human life. And you can try to fix it. You should. But you could also fix the whole business model. And the business model is that really what are... are there some human producers and consumers out there? Is there some economic value to be liberated by connecting them directly? Is it such that it's so valuable that, uh, people are willing to pay for it?... all right? And when-

Lex Fridman 36:53 Like micropayments, like small payments.

Michael I. Jordan 36:55 Micro, but even not to be micro. So, uh, I like the example. Suppose I'm going, next week, I'm going to India. Never been to India before, right? Uh, I have a couple of days in, in, in Mumbai. Um, I have no idea what to do there, right? And I could go on the web right now and search. It's gonna be kind of hopeless. I'm not gonna find, you know. Um, I'll have lots of advertisers in my face, all right? What I really want to do is broadcast to the world that I am going to Mumbai and have someone on the other side of a market look at me and, and there's a recommendation system. So they're not looking at all possible people coming to Mumbai. They're looking at the people who are relevant to them. So someone my age group, someone who kind of knows me at some level. Um, I give up a little privacy by that but I'm happy because what I'm gonna get back is this person's gonna make a little video for me or they're gonna write a little two-page paper on here's the cool things that you want to do in Mumbai this week especially, right? I'm gonna look at that. I'm not gonna pay a micropayment. I'm gonna pay, you know, $100 or whatever for that. It's, it's real value. It's like journalism. Um, as, and not a subscription. It's that I'm gonna pay that person in that moment. Company's gonna take 5% of that and that person has now got a, it's a gig economy, if you will. But, you know, done for in, you know, thinking about a little bit behind YouTube, there was actually people who could make more of those things. If they were connected to a market, they would make more of those things independently. You don't have to tell them what to do. You don't have to incentivize them any other way. Um, and so yeah, these companies, I don't think have thought long, long and hard about that. So I, I do d- distinguish on, you know, Facebook on the one side who's just not thought about these things at all, I think, uh, thinking that AI will fix everything, uh, and Amazon who thinks about them all the time because they were already out in the real world. They were delivering packages to people's doors. They were, they were worried about a market. They were worrying about sellers and, you know, they worry and some things they do are great, some things maybe not so great, but, you know, they're in that business model. And then I'd say Google sort of hovers somewhere in between. I don't, I don't think for a long, long time, they, they got it. Uh, I think they probably see that YouTube is more pregnant with possibility than, than, than they might have thought and that they're probably heading that direction. Um, but uh, it's, you know, Silicon Valley's been dominated by the Google-Facebook kind of mentality and the subscription and advertising, and that is, that's the core problem, right? That the fake news actually rides on top of that 'cause it, it means that you're monetizing with click-through rate and that is the core problem. You gotta remove that.

Lex Fridman 39:06 So advertisement, if we can linger on that, I mean, that's an interesting thesis. I don't know if everyone really deeply thinks about that. So you're right. The thought is the advertisement model is the only thing we have, the only thing we'll ever have, so we have to fix... We have to build algorithms that despite that business model, you know, find the better angels of our nature and do good by society and by the individual. But you think we can slowly... You think, first of all, there's a difference between should and could. So you're saying we should slowly move away from the advertisement model and have a direct connection between the consumer and the creator. The, the question I also have is, can we? Because the advertisement model is so successful now in terms of just making a huge amount of money and therefore being able to build a big company that provides... has really smart people working there to create a good service. Do you think it's possible? And just to clarify, you think we should move away from the advertisement-

Michael I. Jordan 40:08 Well, I think we should, yeah, but, uh, we is the, you know, not me, so, um-

Lex Fridman 40:11 Society.

Michael I. Jordan 40:12 Yeah, well, the companies. Um, I mean, so first of all, full disclosure, I'm doing a day a week at Amazon 'cause I kind of want to learn more about how they do things. So, you know, I'm not speaking for Amazon in any way, but, um, you know, I did go there because I actually believe they get a little bit of this or trying to create these markets.

Lex Fridman 40:26 And they don't really use... Advertisement is not a crucial part of Amazon?

Michael I. Jordan 40:29 That's a good question. So y- it has become, not crucial, but it's become more and more present if you go to Amazon website and, uh, with, you know, without revealing too many deep secrets about Amazon, I can tell you that, you know, a lot of people in the company question this and there's a huge questioning going on. You do not want a world where there is zero advertising. That actually is a bad world, okay? So here's a way to think about it. You're, um, a company that, like Amazon, is trying to bring products to customers, all right? And the customer at any given moment, you want to buy a vacuum cleaner, say. You want to know what's available for me, you know? It's not gonna be that obvious. You have to do a little bit of work at it. The recommendation system will sort of help, all right? But now suppose this other person over here has just made the world... you know, they spent a huge amount of energy. They had a great idea. They made a great vacuum cleaner. They know. They, they really did it. They nailed it. It's an MIT, you know, whiz kid that made a great new vacuum cleaner. All right? It's not gonna be in the recommendation system. No one will know about it. The algorithms will not find it, and AI will not fix that, okay? At all, right? How do you allow that vacuum cleaner to start to get in front of people, be sold? Well, advertising. And here what advertising is, it's a signal that you're, you believe in your product enough that you're willing to pay some real money for it. And to me as a consumer, I look at that signal and I say, well, first of all, I know these are not just cheap little ads 'cause we have now right now. There, I know that, you know, these are super cheap, you know, pennies. Uh, if I see an ad where it's actually I know the company is only doing a few of these and they're making, you know, m- real money is kind of flowing, and I see an ad, I might pay more attention to it and I actually might want that.

Lex Fridman 41:56 Mm-hmm.

Michael I. Jordan 41:56 Because I see, hey, that guy spent money on his vacuum cleaner. Uh, oh, maybe there's something good there so I will look at it. And, and so that's part of the overall information flow in a good market. Uh, so advertising has a role, um, but the problem is of course that that signal is now completely gone because it just, you know, dominated by these tiny little things that add up to big money for the company, you know? So I, I think it will just, I think it will change because societies just don't, you know, stick with things that annoy a lot of people. Oh, and advertising currently annoys people more than it provides information.

Lex Fridman 42:28 So-

Michael I. Jordan 42:28 And I think that, uh, Google probably is smart enough to figure out that this is a dead, this is a bad model even though it's a hard, huge amount of money and they'll have to figure out how to pull it away from it slowly. And I'm sure the CEO there will figure it out but, um, they need to do it and, uh, they need to de- to, uh... So if you reduce advertising, not to zero, but you reduce it and at the same time you bring up producer consumer actual real value being delivered so real money is being paid and they take a 5% cut, that 5% could start to get big enough to cancel out the lost revenue from the, the kind of the poor kind of advertising and I think that a good company will, will do that, will realize that.... um, and they're a comp- I- you know, Facebook, you know, again, God bless them. They, they bring, you know, grandmothers, uh, you know, uh, they bring children's pictures into grandmothers' lives. It's fantastic. Um, but they need to think of a new business model and, and they, that's, that's the core problem there. Um, until they start to connect producer-consumer, I think they will just, just continue to make money and then buy the next social network company, and then buy the next one. And the innovation level will not be high and the health, the health issues will not go away.

Lex Fridman 43:34 So, I apologize that we've kind of returned to words. Uh, I don't think the exact terms matter, but in sort of defense of advertisement, do- don't you think the kind of direct connection between consumer and, uh, creator-producer is the best, like the, is what advertisement strives to do? Right? So at its best, advertisement is literally now Facebook is listening to our conversation and heard that you're going to India and will be able to actually start automatically for you making these connections and start giving this offer. So like, uh, I apologize if it's just a matter of terms, but just to draw a distinction, is it possible to make advertisements just better and better and better algorithmically to where it, it actually becomes a connection? Almost like direct marketing.

Michael I. Jordan 44:25 That's a good question. So let's keep on that- push on that. First of all, I, I... What we just talked about, I was defending advertising. Okay? So I was defending it as a way to get signals into a market that don't come any other way, uh, especially algorithmically. It's a sign that someone's putting money on it. It's a sign they think it's valuable.

Lex Fridman 44:41 It's the-

Michael I. Jordan 44:41 And if I think that if other things- someone else thinks it's valuable, then if I trust other people, I might be willing to listen.

Lex Fridman 44:47 It's the-

Michael I. Jordan 44:47 I don't trust that Facebook, though, is- who's an intermediary between this. I don't think they care about me. Okay? I don't think they do. And I find it creepy that they know I'm going to India next week because of our conversation.

Lex Fridman 45:01 Why do you think that is? Can we... So what... C- can you just, uh, put your PR hat on (laughs) . Why do you think you find Facebook, uh, creepy and not trust them as, as do majority of the population? So they're, uh, out of the Silicon Valley companies, I saw like, uh, not approval rate, but there's, there's ranking of how much people trust companies and Facebook is in, in the gutter.

Michael I. Jordan 45:23 In the gutter, including people inside of Facebook (laughs) .

Lex Fridman 45:25 Yeah, so what, what, uh, what do you attribute that to? Because f- when I-

Michael I. Jordan 45:28 Come on. You don't find it creepy that right now we're talking, that I might walk out on the street right now, that some unknown person who I don't know kind of comes up to me and says, "I hear you're going to India."

Lex Fridman 45:37 I-

Michael I. Jordan 45:37 I mean, that's not even Facebook. That's just a... If... I- I want transparency in human society. I want to have, if you know something about me, there's actually some reason you know something about me, that's something that if I look at it later and audit it, kind of, I'm, I approve. You know something about me 'cause, uh, you care in some way. There's a caring relationship even, or an economic one or something.

Lex Fridman 45:58 Right.

Michael I. Jordan 45:58 Not just that you're someone who could exploit it in ways I don't know about or care about, or, or I- I'm troubled by, or, or whatever.

Lex Fridman 46:05 But a lot of companies-

Michael I. Jordan 46:05 And we're in a world right now where that happens way too much. And that Facebook knows things about a lot of people and could exploit it, and does exploit it at times. I think most people do find that creepy. It's not for them. It's not, it's not that... Facebook does not doing it 'cause they care about them, right? In any real sense. And they shouldn't. They should not be a big brother caring about us. That is not the role of a company like that.

Lex Fridman 46:28 Why not?

Michael I. Jordan 46:29 Right?

Lex Fridman 46:29 Wait, wait. Not the big brother part-

Michael I. Jordan 46:31 Yeah.

Lex Fridman 46:31 ... but the caring, the trusting. I mean, don't those companies... J- just to linger on it because a lot of companies have a lot of information about us. I, I would argue that there's companies like Microsoft that has more information about us than Facebook does and yet-

Michael I. Jordan 46:44 Yeah.

Lex Fridman 46:44 ... we trust Microsoft more.

Michael I. Jordan 46:45 Well, Microsoft is pivoting. Microsoft, you know, under Satya Nadella has decided this is really important. We, we don't wanna do creepy things. Really want people to trust us to actually only use information in ways that they really would approve of, that we don't decide. Right? And, um, I'm just kind of adding that the health, health of a, a market is that, uh, when I connect to someone who producer-consumer, it's not just a random producer-consumer. It's pe- people who see each other. They don't like each other, but they sense that if they transact, some happiness will go up on both sides. If a company helps me to do that in moments that I choose, of my choosing, um, then fine. So... And also, think about the difference between, you know, browsing versus buying, right? There are moments in my life, I just wanna buy, you know, a gadget or something. I need something for that moment. I need some ammonia for my house or something 'cause I got a problem, a spill. Um, I wanna just go in. I don't wanna be advertised at- at that moment. I don't wanna be led down various direct- I- you know, that's annoying. I want to just go and have it be extremely easy to do what I want. Um, uh, other moments, I might say, no, I'm... It's like in the... Today, I'm going to the shopping mall. I wanna walk around and see things and see people and be exposed to stuff. So I want control over that, though. I don't want the company's algorithms to decide for me. Right? And I think that's the thing. We- it's a total loss of control if Facebook thinks they should take the control from us of deciding when we want to have certain kinds of information and when we don't, what information that is, how much it relates to what they know about us that we didn't really want them to know about us. They're not... I don't want them to be helping me in that way. I don't want them to be helping them by they decide, well, they have control over, um, um, what I want and when.

Lex Fridman 48:22 I totally agree. So it's the Facebook... By the way, I have this optimistic thing where I think Facebook has the kind of personal information about us that could create a beautiful thing. So I- I'm really optimistic of what Facebook could do. Uh, not what it's doing, but what it could do. So-

Michael I. Jordan 48:39 I don't see that... I think that optimism is misplaced because there's not a bu- you have to have a business model behind these things.

Lex Fridman 48:44 Yes. No, you have to-

Michael I. Jordan 48:45 Cre- cre- create a beautiful thing is really-

Lex Fridman 48:46 Yeah.

Michael I. Jordan 48:46 ... let's be f- let's be clear. It's about som- something that people would value. And, and I don't think they have that business model.

Lex Fridman 48:53 Okay.

Michael I. Jordan 48:53 And I don't think they, they will suddenly discover it by what, you know, um, by having a long, hot shower.

Lex Fridman 48:58 ... I disagree. I disagree in terms of, uh, a sh- you can discover a lot of amazing things in, in a shower. So... (laughs)

Michael I. Jordan 49:05 I didn't say that. I said they won't cover it.

Lex Fridman 49:07 They won't.

Michael I. Jordan 49:07 They won't do it-

Lex Fridman 49:08 But, uh... (laughs)

Michael I. Jordan 49:08 ... in the shower. I think a lot of other people will discover it. I think that this... Right, so I, I should also, uh, full disclosure, there's a company called United Masters, which I'm on their board, and they've created this music market.

Lex Fridman 49:18 Yes.

Michael I. Jordan 49:18 They have 100,000 artists now signed on, and they've done things like gone to the NBA, and the NBA, the music you find behind NBA clips right now is their music. Right? That's a company that had the right business model and mind from the get-go, right, executed on that and, and from day one, there was value brought to... So here, here you have a kid who made some songs who suddenly their songs are on the NBA website, right? That, that's real economic value to people and, uh, so, you know.

Lex Fridman 49:45 So you and I differ on the optimism of being able to, sort of, uh, sh- um, change the direction of the Titanic, right?

Michael I. Jordan 49:53 Mm-hmm.

Lex Fridman 49:54 Uh, so I...

Michael I. Jordan 49:55 Yeah.

Lex Fridman 49:56 (laughs)

Michael I. Jordan 49:56 I'm older than you so I've seen some Titanics crash.

Lex Fridman 49:59 (laughs) Got it. But, uh, so and just to elaborate 'cause I, I totally agree with you and I just want to know how difficult you think this problem is of... So for example, I, um, I wanna read some news and I would... There's a lot of times in the day where something makes me either smile or think in a way where I, like, consciously think this really gave me value. Like I sometimes listen to, uh, the daily podcast from The New York Times, way better than The New York Times themselves, by the way, for people listening. That's like real journalism is happening for some reason in the podcast space. It doesn't make sense to me. But, uh, often I listen to it, 20 minutes, and I s- I would be willing to pay for that like $5, $10 for that experience.

Michael I. Jordan 50:41 Yeah, absolutely.

Lex Fridman 50:41 And s- how difficult... That's kind of what you're getting at is that little transaction. What... How difficult is it to create a frictionless system like Uber has, for example, for other things? What's your intuition there?

Michael I. Jordan 50:55 Uh, so I... First of all, I, I pay little bits of money to, you know, to s- there's something called Quartz that does financial things. I like Medium as a site. I don't pay there but, um, I would.

Lex Fridman 51:04 You had a great post on Medium. I would have loved to pay you a dollar and-

Michael I. Jordan 51:09 But I wouldn't want it-

Lex Fridman 51:10 ... not others.

Michael I. Jordan 51:10 I wouldn't want it per se because, um, there should be also sites where that's not actually the goal. The goal is to actually have a broadcast channel that I monetize in some other way if I chose to.

Lex Fridman 51:20 Right.

Michael I. Jordan 51:20 I mean, I could now. People know about it. I could. I'm not doing it, but, um, that- that's fine with me. There are also the musicians who are making all this music, I, I don't think the right model is that you pay a little subscription fee to them, all right? Because, because people can copy the bits too easily and it's just not that... It's not where the value is. The value is that a connection was made with r- between real human beings then you can follow up on that, all right, and create yet more value. So no, I, I think, um, uh-

Lex Fridman 51:45 So there's a lot of open questions here, huh?

Michael I. Jordan 51:46 Lot open questions but also, yeah, I do want good recommendation systems that recommend cool stuff to me and... But it's pretty hard, right? I, I don't like them to recommend stuff just based on my browsing history. I don't like that they're based on stuff they know about me, quote unquote. What's unknown about me is the most interesting. Right?

Lex Fridman 52:00 So the... This is the, this is the really interesting question. We may disagree, maybe not, I think that I love recommender systems and I wanna give them everything about me in a way that I trust.

Michael I. Jordan 52:13 Yeah, but you-

Lex Fridman 52:14 S-

Michael I. Jordan 52:14 ... but you don't because... So for example, this morning, I clicked on a, you know, I was pretty sleepy this morning, um, I clicked on a, a story about the Queen of England.

Lex Fridman 52:23 Yes.

Michael I. Jordan 52:23 Right? I do not give a damn about the Queen of England. I really do not. But it was clickbait, it kinda looked funny and I had to say, "What the heck are they talking about there?" I don't wanna have my life, you know, heading that direction. Now that's in my browsing history. The system, in any reasonable system, will think that ...that's browsing history ... I care about the Queen of England. Right, but, but you're saying all the trace, all the digital exhaust or whatever, that's been kind of the model. So if you collect all this stuff, you're gonna figure all of us out. Well, if you're trying to figure out like kind of one person, like Trump or something, maybe you could figure him out. But if you're trying to figure out, you know, 500 million people, you know? No way, no way.

Lex Fridman 52:58 You think so? You think-

Michael I. Jordan 52:58 No, I do. I think so. I think we are... Humans are just amazingly rich and complicated. Every one of us has our little quirks, every one of us has our little things that could intrigue us that we don't even know it will intrigue us, and there's no sign of it in our past but, by God, there it comes and, and, you know, you fall in love with it and I don't want a company trying to figure that out for me and anticipate that.

Lex Fridman 53:15 Okay, well, let me-

Michael I. Jordan 53:16 I want them to provide a forum, a, a market, a place that I kind of go and by hook or by crook this happens, you know? I, I'm walking down the street and I hear some Chilean music being played and I never knew I liked Chilean music but wow. So there is that side and I want them to provide a, a limited but, you know, interesting place to go, right? And so don't try to use your AI to kinda, you know, figure me out and then put me in a world where you've figured me out, you know? No, create hu- spaces for human beings where our creativity and our style will be enriched and come forward and it'll be a lot more transparency. I won't have people randomly anonymously putting comments up and especially based on stuff they know about me, facts that you know. We are so broken right now if you're, you're... You know, especially if you're a celebrity but, you know, just about anybody that... Uh, anonymous people are hurting lots and lots of people right now and that's part of this thing that Silicon Valley is thinking that, you know, just collect all this information and use it in a, in a great way. So no, I, I'm, I'm not, I'm not a pessimist but I'm very much an optimist by nature but I think that's just been the wrong path for the whole technology to take. Uh, be more limited, uh, create... Let humans rise up don't, don't try to replace them, that's the AI mantra. Don't try to anticipate them, don't try to predict them 'cause you're, you're, you're not gonna, you're not gonna be doing those, those things. You're gonna make things worse.

Lex Fridman 54:34 Okay so, right now, just give this a chance, uh, right now the recommender systems are the creepy people in the shadow watching your every move. So they're looking at traces of you, they're not directly interacting with you. Sort of the... Your close friends and family, the way they know you is by having conversation, by actually having interactions back and forth. Do you think there's a place...... for recommender systems, sort of the step, 'cause you, you just emphasized the value of human-to-human connection, but-

Michael I. Jordan 55:04 Yeah.

Lex Fridman 55:05 ... just give it a chance, AI-human connection. Is there a role for an AI system to have conversations with you-

Michael I. Jordan 55:11 Yeah.

Lex Fridman 55:11 ... in terms of w- to try to figure out what kind of music you like, not by just watching what you listen to-

Michael I. Jordan 55:16 Yeah.

Lex Fridman 55:16 ... but actually having a conversation, natural language or otherwise?

Michael I. Jordan 55:19 Yeah. No, I'm, I'm, so I'm not against any of this. I just wanted to push back against the maybe is you, you're saying now you have options for Facebook. So, there, I think it's misplaced. But, but, um, (laughs) I think that distributing-

Lex Fridman 55:29 (laughs) I'm the one guy defending Facebook.

Michael I. Jordan 55:30 Yeah, no, so good for you. Um, go for it.

Lex Fridman 55:33 That's a hard spot to be in.

Michael I. Jordan 55:34 Yeah, no, good. Human interaction, like, on our daily, the context around me in my own home is something that I don't want some big company to know about at all, but I would be more than happy to have technology help me with it.

Lex Fridman 55:44 Which kinda technology? What-

Michael I. Jordan 55:45 Well, you know, just-

Lex Fridman 55:46 ... Alexa, Amazon?

Michael I. Jordan 55:47 ... uh, well, a good... Alexa's done right, I think Alexa's a research platform right now more than anything else. But Alexa done right, you know, could do things like I, uh, I leave the water running in my garden, and I say, "Hey, Alexa, the water's running in my garden," um, and even have Alexa figure out that that means when my wife comes home that she should be told about that. That's a little bit of a reasoning. You, I would call that AI, and by any kind of stretch, it's a little bit of reasoning, and it actually kind of would make my life a little easier and better. And, you know, I don't, I wouldn't call this a wow moment, but I kinda think that overall rises hum- human happiness up to have that kind of thing. Um-

Lex Fridman 56:18 But not when you're lonely. Alexa knowing loneliness-

Michael I. Jordan 56:21 No.

Lex Fridman 56:22 ... sadness, depression.

Michael I. Jordan 56:22 No, there, I don't want Alexa to be, feel intrusive, and I, I, and I don't want just the designer of the system to kind of work all this out. I really wanna have a lot of control, and I want transparency and control. And if the company can stand up and give me that in the context of new technology, um, I think they're gonna, first of all, be way more successful than our current generation. And like I said, I was, uh, mentioning Microsoft earlier on. I really think, uh, they're, they're pivoting to kind of be the trusted old uncle, but, you know-

Lex Fridman 56:45 (laughs)

Michael I. Jordan 56:45 ... I think that they get that this is the way to go, that if you let people find technology empowers them to have more control and have, and have control not just over privacy, but over this rich set of interactions, um, that that, people are gonna like that a lot more. And that's, that's the right business model going forward.

Lex Fridman 57:00 What does control over privacy look like? Do you think you should be able to just view all the data that-

Michael I. Jordan 57:05 No, it's much more than that. I mean, first of all, it should be an individual decision. Some people (laughs) don't want privacy. They want their whole life out there. Other peoples want it. Um, uh, privacy is not a zero-one. It's not a legal thing. It's not just about which data's available and which is not. Um, I like to re- recall to people that, you know, couple hundred years ago, everyone, there was not really big cities. Everyone lived in, on the countryside in villages. Um, and in villages, everybody knew everything about you. Very, you didn't have any privacy. Is that bad? Are we better off now? Well, you know, arguably no, because what did you get for that loss of at least certain kinds of privacy? Um, well, uh, people helped y- each other. If they, 'cause they know everything about you, they know something's bad's happening, they will help you with that, right? And now you live in a big city, no one knows anything about that you get no help. Um, so, uh, kind of depends is the answer. I want certain people, uh, who I trust and there should be relationships, I should kind of manage all those. But who, who knows what about me? I should have some agency there. It shouldn't, I shouldn't just be adrift in a sea of technology where I have n- I have no agency. And I don't wanna go reading things and checking boxes. Uh, so I don't know how to do the, you know, I'm not a privacy researcher per se. I just, I recognize the vast complexity of this. It's not just technology. It's not just l- legal scholars meeting technologists. Uh, it, there's gotta be kind of whole layers around it. And so I, I, when I allude to this emerging engineering field, this is a big part of it. Um, like w- when electrical engineering come, came, I'm not, wasn't around in the time, but, uh, you just didn't plug electricity, you know, into walls and it all kind of worked. You now had to have, like, you know, Underwriters Laboratory that reassured you that that plug's not gonna burn up your house and that that machine will do this and that and everything. There'll be whole people who can install things. There'll be people who can watch the installers. There'll be a whole layers, you know, an onion of these kind of things. And for things as deep and the, interesting as privacy, which is as least as interesting as electricity, um, that, that's gonna take decades to kind of work out, but it's gonna require a lot of s- new structures that we don't have right now. So it's gonna be hard to talk about it.

Lex Fridman 59:02 And you're saying there's a lot of money to be made if you get it right, so-

Michael I. Jordan 59:05 Absolutely.

Lex Fridman 59:05 ... companies should look at it.

Michael I. Jordan 59:06 A lot of money to be made in all these things that provide human services and people recognize them as useful parts, uh, uh, y- of their lives. Uh, so, yeah. Um, so, yeah, the dialogue sometimes goes from the exuberant technologists, uh, to the no technology is good kind of. And that's w- you know, in our public discourse, you know, in, in news stories, you see too much of this kind of thing. And the, and the sober discussions in the middle, which are the challenging ones to have, are where we need to be having our conversations. And, you know, there's just not, actually there's not many forum, fora for those. Um, you know, there's, that's, that's kind of what I would look for. Maybe I could go and I could read a comment section of something, and it would actually be this kind of dialogue going back and forth. You don't see much of this, right?

Lex Fridman 59:45 Which is why actually there's a resurgence of podcasts out of all, because-

Michael I. Jordan 59:49 Good.

Lex Fridman 59:49 ... people are really hungry for conversation.

Michael I. Jordan 59:51 Yeah.

Lex Fridman 59:51 But there's, technology is not helping much, so comment sections of anything, including YouTube-

Michael I. Jordan 59:58 Yeah.

Lex Fridman 59:59 ... is not help-

Michael I. Jordan 1:00:00 Hurting.

Lex Fridman 1:00:00 Hurting and not helping.

Michael I. Jordan 1:00:01 Hurting, yeah.

Lex Fridman 1:00:03 And you think, technically speaking, it's possible to help?

Michael I. Jordan 1:00:07 I, I don't know the answers, but it's, uh, it's, uh, it's, uh, the, less anonymity, a little more locality, um, you know, worlds that you kind of enter in and you trust the people there in those worlds, so that when you start having-

Lex Fridman 1:00:18 Mm-hmm.

Michael I. Jordan 1:00:18 ... a discussion, you know not only is it people not gonna hurt you, but it's not gonna be a total waste of your time, 'cause there's a lot of wasting of time that, you know, a lot of us... I, I pulled out of Facebook early on 'cause it was clearly gonna waste a lot of my time-

Lex Fridman 1:00:28 Yeah.

Michael I. Jordan 1:00:28 ... even though there was some value. Um, and so, yeah, worlds that are somehow you enter in, you know what you're getting, and it just kind of appeals to you. You might, new things might happen, but you kinda have some, some trust in that world.

Lex Fridman 1:00:40 And there's some deep, interesting, complex psychological aspects around anonymity, uh, how that changes human behavior that's-

Michael I. Jordan 1:00:48 Indeed.

Lex Fridman 1:00:48 ... quite dark and-

Michael I. Jordan 1:00:49 Quite dark, yeah.

Lex Fridman 1:00:50 Yeah.

Michael I. Jordan 1:00:50 I think a lot of us are, uh, who, especially those of us who really love the, the advent of technology... I loved social networks when they came out. I was just, I didn't see any negatives there at all.... but then I started, uh, seeing comment sections, I think it was maybe, you know, on CNN or something-

Lex Fridman 1:01:04 Mm-hmm.

Michael I. Jordan 1:01:04 ... and I started to go, "Wow, this- this darkness I just did not know about," and, um, and our technology is now amplifying

Michael I. Jordan 1:01:11 it. Um...

Lex Fridman 1:01:11 So, sorry for the big philosophical question, but on that topic, do you think human beings, 'cause you've also, out of all things, had a foot in psychology too, uh, the, do you think human beings are fundamentally good? Like, all of us have good intent that could be mined? Or is it, depending on context and environment, everybody could be evil?

Michael I. Jordan 1:01:34 So, my answer is fundamentally good, um, but fundamentally limited. All of us have very, you know, blinkers on. We don't see the other person's pain that easily. We don't see the other person's point of view that easily. We're very much in our own head, in our own world. Um, and on my- my good days, I think that technology could open us up to, you know, more perspectives and more- less blinkered and more understanding. You know, a lot of wars in human history happened because of just ignorance. They didn't- they- they thought the other person was doing this when the other person wasn't doing this and we have huge amounts of that. Um, but in my lifetime, I've not seen technology really help in that way yet. And I- I- I do- I do believe in that, but, you know, no, I think fundamentally human- humans are good. People suffer, people have grievances, people have grudges and those things cause them to do things they probably wouldn't want. They regret it often, um, so no, I- I- I think it's a, you know, part of the progress of technology is to indeed allow it to be a little easier to be the real good person you actually are.

Lex Fridman 1:02:31 Well

Lex Fridman 1:02:32 put. Do you think individual human life or society can be modeled as an optimization problem?

Michael I. Jordan 1:02:41 Um, not to the way I think typically. I mean, that's you're talking about one of the most complex phenomena in the whole, you know, in all of the universe.

Lex Fridman 1:02:47 Which? The individual human life or society as a whole?

Michael I. Jordan 1:02:50 Both, both. I mean, individual human life is- is amazingly complex and, um, so, uh, you know, optimization's kind of just one branch of mathematics that talks about certain kind of things and, uh, it just, it feels way too limited for the complexity of, uh, such things.

Lex Fridman 1:03:04 What properties of optimization problems... Do you think so- do you think most interesting problems that could be solved through optimization, uh, what kind of properties does that surface have? Non-convexity, convexity, linearity, all those kinds of things? Saddle points?

Michael I. Jordan 1:03:20 Well, so optimization's just one piece of mathematics, you know, there's like, you just, even in our era we're aware that, say, sampling, um, is coming up with examples of something, um, coming up with a distribution-

Lex Fridman 1:03:30 What's- what's optimization? What's sampling?

Michael I. Jordan 1:03:33 Well, you- they- you can, if you're kind of a certain kind of mathematician, you can try to blend them and make them see- seem to be sort of the same thing, but optimization is, roughly speaking, trying to, uh, find a point that, um, a single point that is the optimum of a criterion function of some kind. Um, and sampling is trying to, from that same surface, treat that as a distribution or density and find prob- points that have high density. So, um, I- I want the entire distribution in a sampling paradigm, and I want the, um, you know, the- the single point that's the best point in the para- in the samp- in the, uh, optimization paradigm. Now, if you were optimizing in the space of probability measures, the output of that could be a whole probability distribution. So, you can start to make these things the same, but in mathematics, if you go too high up that kind of abstraction arc, you start to lose the, uh, you know, the ability to do the interesting theorems, so you kind of don't try to- you don't try to overly- over abstract.

Lex Fridman 1:04:26 So,

Lex Fridman 1:04:27 as a small tangent, what kind of world view do you find more appealing? One that is deterministic or stochastic, uh, statistic-

Michael I. Jordan 1:04:36 Well, that's easy.

Lex Fridman 1:04:36 ...ian?

Michael I. Jordan 1:04:36 I mean, I'm a statistician, you know, the- the world is highly stochastic. We- I don't know what's going to happen in the next five minutes, right? What you're gonna ask, what we're gonna do, what I'll- I'll say.

Lex Fridman 1:04:44 Due to the uncertainty? Due to the-

Michael I. Jordan 1:04:45 Massive uncertainty.

Lex Fridman 1:04:47 Yeah.

Michael I. Jordan 1:04:47 You know, massive uncertainty. And so the best I can do is have kind of a rough sense, or probability distribution on things, and somehow use that in my reasoning about what to do now.

Lex Fridman 1:04:58 So,

Lex Fridman 1:05:00 how does the distributed at scale when you have multi-agent systems, uh, look like? So, optimization can optimize sort of, it makes a lot more sense sort of, uh, at least from my- from a robotics perspective for a single robot, for a single agent trying to optimize some objective function. Uh, wha- when you start to enter the real world, this game theoretic concept starts popping up, uh, that... H- how do you see optimization in this? 'Cause you've talked about markets and the scale, what does that look like? Do you see it as optimization? Do you see it as sampling? Do you see- like how- how should you match up, match up...

Michael I. Jordan 1:05:38 Yeah. These all blend together, um, and a system designer thinking about how to build an incentivized system will have a blend of all these things. So, you know, a particle in a potential well is optimizing a function called a Lagrangian, right? The particle doesn't know that. There's no algorithm running that does that. It just happens.

Lex Fridman 1:05:55 Okay.

Michael I. Jordan 1:05:55 It's- it's, so it's a description mathematically of something that helps us understand as analysts what's happening, right? And so the same thing will happen when we talk about, you know, mixtures of humans and computers and markets and so on and so forth, there'll be certain principles that allow us to understand what's happening whether or not the actual algorithms are being used by any sense is not clear. Um, now at- at some point I may have set up a multi-agent or market kind of system, and I'm now thinking about an individual agent in that system, and they're asked to do some task and they're incentivized in some way. They get certain signals and they- they have some utility. Maybe what they will do at that point is they- they just won't know the answer, they may have to optimize to find an answer, okay? So, and optimizers could be embedded inside of an overall market, uh, you know, and game theory is- is very, very broad. Um, it is often studied very narrowly for certain kinds of problems, um, but it's roughly speaking this is just the, "I don't know what you're gonna do, so I kind of anticipate that a little bit, and then you anticipate what I'm anticipating," and we kind of go back and forth in our own minds. We run kind of thought experiments.

Lex Fridman 1:06:55 You've talked about this interesting point in terms of, uh, game theory, so, you know, most optimization problems really hate saddle points. Maybe you can describe what saddle points are? But-I've heard you kind of mention that there's a- there's a branch of optimization that you could try to explicitly look for saddle points as a good thing.

Michael I. Jordan 1:07:15 Oh, not optimization. That's just game theory. That- that's so, uh, y- there's all kinds of different equilibria in game theory, and some of them are highly explanatory behavior. They're- they're not attempting to be algorithmic, they're just trying to say, "If you happen to be at this equilibrium, you would see certain kind of behavior," and we see that in real life. That's what an economist wants to do, especially a behavioral economist. Um, uh, in- in continuous, uh, differential game theory, you're in continuous spaces, a, um, some of the simplest equilibria are saddle points. A Nash equilibrium is a saddle point. It's a special kind of saddle point. So, classically in game theory, you were trying to find Nash equilibria, and in an algorithm of game theory, you were trying to find algorithms that would find them. Uh, and so you're trying to find saddle points. I mean, so that's- that's literally what you're trying to do. Um, but, you know, any economist knows that Nash equilibria, uh, have their limitations. They are definitely not that explanatory in many situations. They're not what you really want. Um, there's other kind of equilibria, and there's names associated with