Emad Mostaque: These 5 Companies Will Win the AI War; Why We Need National Data Sets | E1015

Emad Mostaque 0:00 I, I think this is bigger than the printing press. It's bigger than anything. And so that's one of the reasons I signed the letter. I said, "We have to get this discussion going in public right now. We gotta stop pre-training big models on all the crazy crap of the internet." And like, "We gotta do it fast because this is coming like a train." (instrumental music plays)

Harry Stebbings 0:18 Ahmad, I am so excited for this. I heard so many great things from, uh, specifically Ashton, who helped with questions, and then also Dan Rose. So thank you so much for joining me today.

Emad Mostaque 0:26 It's my pleasure.

Harry Stebbings 0:27 Now, I want to start with a little bit on you. You moved around a little bit in your childhood. Take me back to the childhood, the moving around. And there's a weird commonality I found with the most talented founders. They all moved around. So take me to that and how it impacted your mindset.

Emad Mostaque 0:42 So I was born in Jordan, grew up in Bangladesh, came to the UK. Um, yeah, didn't move around that much, but with my father a bit as he lectured in various places. And it was always a bit of a struggle fitting in, but then you learn to adapt. You learn to adapt to new scenarios, new environments. Like, "Oh, I don't speak the language, uh, kind of what's happening. Let's learn and let's move on from there." And I think it gave me a bit of appreciation of the world as well because we stick in our monocultures very often. Like, I'd only ever been to Silicon Valley ... Actually, I'd only been to the Bay Area once before last October. And so this whole tech monoculture has been a bit of a shock to me, and I'm like, "There's more to the world, uh, than that." So this is, uh, some interesting things around that.

Harry Stebbings 1:22 Talk to me. Hedge funds first, and then, then what happened? We mentioned it a little bit before. Why did you make the move?

Emad Mostaque 1:29 Uh, so actually I was an enterprise developer in my gap year at Metaswitch in the UK, doing voiceover IP.

Harry Stebbings 1:34 Metaswitch?

Emad Mostaque 1:35 Metaswitch.

Harry Stebbings 1:35 This is Chris Mazz's company.

Emad Mostaque 1:37 Yes, and NPIET.

Harry Stebbings 1:38 Yeah.

Emad Mostaque 1:38 So I took my gap year and was like, "Oh, I might as well be an enterprise programmer." Didn't know what that would be like. This was before GitHub and everything, so we had Subversion. You know, the kids these days have it so easy.

Harry Stebbings 1:47 (laughs)

Emad Mostaque 1:48 Um, and then I was like, "What do I do now?" And so I became a VC analyst at Oxford Capital Partners-

Harry Stebbings 1:53 Mm-hmm.

Emad Mostaque 1:53 ... played with the mots there. And that was a lot of fun. They're fantastic. And then I was like, "I want to do movies." So I became a movie reviewer. So I did the Raindance Film Festival, British Independent Film Awards. And they were just bopping around doing random things, and I accidentally became a hedge fund manager.

Harry Stebbings 2:07 Why did you move from hedge funds to startups?

Emad Mostaque 2:10 So with the hedge funds, so I joined Pictet Asset Management, and then, like, the CIO left, and there was, like, this fund. And I got to be a portfolio manager when I was 23. And so I grew my beard to look a bit older.

Harry Stebbings 2:20 (laughs)

Emad Mostaque 2:21 It's coming. Get the clip on Harry.

Harry Stebbings 2:22 Uh, I would, I can't do the mustache, but I still wear the glasses when I just do the contacts.

Emad Mostaque 2:26 You can try extra hard, force it out, right? Um, and so I did that for a number of years, and, you know, reasonably successful, made lots of people money. Um, not so much myself 'cause I was too young. Uh, again, they're like, "You're too young to be a fund." And then my son was diagnosed with autism, and I quit. And, um, because they said there was no cure, no treatment, no information. I was like, "I'm a hedge fund manager. I can deconstruct things." And so built an AI team, and then did a literature analysis of all the autism literature to try and figure out the commonalities, and then drug repurposing, so focusing on GABA glutamate balance in the brain. GABA is what you get when you pop a Valium. It calms you down, and glutamate excites you, you know? And so in kids and people with ASD, it's like there's too much noise going on. It's like when you're tapping your leg and you can't focus. And so that's why you get this sensitivity. Sometimes they can't speak, like my son. And so it was, like, mechanisms to bring that down, that then allowed to have applied behavioral analysis and these other things to reconstruct his speech. And then he went to mainstream school, which is pretty cool.

Harry Stebbings 3:22 That's unbelievable. I, I, I heard you said on another podcast, and I was astounded and inspired

Harry Stebbings 3:26 by it. Wh- w- we mentioned before, you know, my mother's got MS.

Emad Mostaque 3:29 Mm.

Harry Stebbings 3:29 And I hate the doomsday-only version of kind of AI and the future of, you know, GPT. You said to me before about its impact on health, and MS in particular, and other conditions. How can it be so transformatively to solve some of the world's most challenging chronic conditions?

Emad Mostaque 3:45 So I think a large part of our problem is that we can't scale because information flow is so limited as we write these things down. Like, you can never capture all of that. So anyone who's had a loved one that has one of these conditions knows how difficult it is, because you go from specialist to specialist to specialist, and you try to build that mental map. And we're so lucky that we have so much access. But why is it that we can't just push a button and see every clinical trial and a deconstruction of all those and things? What if you had 1000 GPT-4s organizing all that knowledge and then make it available to everyone? So you can see the exact potential mechanisms the way which MS works and all the potential food, other things that work with that. So as you try different things with your family member, you can see, well, she reacted this way to the food or this way to this medicine. And it's a more holistic thing because you can have personalized medicine versus one specialist for 1000 people. You can have 1000 GPT-4s or equivalents or MedPaLM 2s for you. So we need to organize all this knowledge and then use these language models and others to make it accessible to you.

Harry Stebbings 4:48 I'm really naive and, and, uh, basic in terms of my thinking, which is why I'm a venture capitalist. (laughs)

Emad Mostaque 4:54 (laughs)

Harry Stebbings 4:54 Uh, but my question to you is, what do we need to do to get to that state? When we look at the data needed from the individuals, the data, the data the GPTs need, how we make the models work most efficiently, what do we need?

Emad Mostaque 5:05 So first, first, we don't have to have that data from individuals. So we had Galactica as a lang- scientific language model, but now we have MedPaLM 2 that exceeds doctor levels. That was a Google announcement yesterday. We have AIs that can understand articles better, as good as doctors, shall we say now. So we can scale that, because why do you need one when you have 1000? So we take the existing generalized knowledge and all the hypotheticals, and we bring that together into an integrated common system available to everyone, because the building blocks are nearly here for that. Then you can personalize it later. And again, there are regulations and things around that, to how you, again, how we treat our loved ones and other things like that. But the first thing is, let's get all the knowledge in one place and make it organized and useful. And so I think we're at that point now where the language models have just hit that point, that we can organize all of the world's Alzheimer's knowledge, longevity knowledge, autism knowledge, MS knowledge. And you can just type, and it can say, this is the source, this is what it looks like, these are some-... hypotheticals. This is what we know, what we know we don't know, what we think we might know, etc. And then it can learn about you and your queries. 'Cause this is the other thing about lots of the language model things we've seen right now, they are one-to-one goldfish memory. The next step is one-to-one, it remembers what you're asking for, like a cookie or an embedding, and then it's you plus 1,000 of these language models all going and doing your bidding, the agent-based kind of thing.

Harry Stebbings 6:26 Does this get around the incentive problem in healthcare? And what I mean by the incentive problem in healthcare, as I'm sure you know, there are a lot of diseases actually where it doesn't make kind of economic sense for a lot of pharmaceutical providers to chase research, to chase treatments, because it's not a big enough market, 'cause it's not, 'cause it's a $6 treatment. Does this solve for that economic misalignment?

Emad Mostaque 6:44 I think it can help a lot with that economic misalignment because then you have an authoritative source, where we can all come together and build, that can analyze these things. Because there's this concept of ergodicity. A thousand coins tossed in a row is the same as a thousand coins tossed at once.

Harry Stebbings 6:59 Mm-hmm.

Emad Mostaque 6:59 And because we're so limited in our information in our medical system, like, you know, I just had my key management, so I had to answer 40 minutes of questions.

Harry Stebbings 7:07 (laughs)

Emad Mostaque 7:07 Have you smoked? Have you done this? It, it's stupid, right? We're all treated the same. Um, uh, I think as, like, 10% of people have a cytochrome P450 mutation in their liver, which means they metabolize drugs fast, quicker. So if you metabolize codeine, it turns into morphine or fentanyl, kills you. But that's a very basic genetic test, yet we give everyone 500 milligrams of the same thing. With my son, a microdose of five milligrams of clonazepam, which is used for anxiety disorder, worked with a neurologist, allows him to sing. The standard dose is 1,000 milligrams, so they can only prescribe it at 1,000. But that is a $6 a year treatment that affects his GABA glutamate balance, but only for his specific type of ASD, which is only 7% of all kids with ASD. But why would that be in a pharmaceutical company's interest? You know? Because how are they gonna make money off a $6 a year treatment?

Harry Stebbings 7:58 Well, I mean, how many people have ASD?

Emad Mostaque 8:00 Uh, it's one in 60.

Harry Stebbings 8:02 Okay, so one in 60. Have you got a million people in the UK?

Emad Mostaque 8:04 Yes.

Harry Stebbings 8:04 So you've got a $6 million ... Yeah, it's not great. (laughs)

Emad Mostaque 8:09 Exactly. It's not great.

Harry Stebbings 8:10 Yeah.

Emad Mostaque 8:10 I mean, it's like we know the benefits of vitamin D, right? But we still don't prescribe that at scale and so many people are deficient.

Harry Stebbings 8:16 I, I, I mean-

Emad Mostaque 8:16 So all these things.

Harry Stebbings 8:17 The joys of doing what I do is going off schedule. Final one, and then we will kind of retain some form of normality of schedule. What does the future of healthcare systems look like, do you think, with GPT models operating in this way?

Emad Mostaque 8:26 I think that you can change the nature of a doctor because a lot of the stuff is kind of very basic. I think, you know, you had Babylon Health and others trying that chatbot, it wasn't ready. Now you've got this. But everyone should have their own AIs looking out for their own health with that objective function. You know? And then the nature of a doctor becomes different in terms of they have more rich information about an individual while it being preserved in a private manner. I think what you have is you have things like processes and procedures improving, like, uh, wound care, for example, in the NHS. Um, if you are injured as an elderly person, and your wounds aren't treated properly, you're more likely to die by a factor of eight times. Being able to monitor those types of things with this information set means you're eight times less likely, and then you have far more efficiency around that. So the information density on healthcare improves, which means that then our own healthcare improves. We all have access to as much knowledge as we want to within our own context, and so do our providers and the people that help us.

Harry Stebbings 9:18 How do we think about open-source versus closed-source human healthcare data? 'Cause, like, obviously for us all to benefit as one, you know, MS sufferers around the world need to submit their data around, you know, responses to certain treatments.

Emad Mostaque 9:29 Yeah. So I think the w- wonderful thing about these models is they're few-shot learners, so they don't need to have much information.

Harry Stebbings 9:34 Yeah.

Emad Mostaque 9:34 And so it isn't the classical big data problem. HDR UK has been one of the pioneers here with the UK Biobank, federated learning and others. And there are kind of, um, with FL7, HLIR and other standards being built around this to allow for full federated learning. If you have open-source language models that are fully auditable, understand, like, on the organic free-range models, the ones we're building, with no web-scraped data, those can sit on-device. Like Google yesterday announced, um, PaLM 2.

Harry Stebbings 10:01 Yeah.

Emad Mostaque 10:01 The smallest PaLM 2 model is 400 million parameters. It works on your Google Pixel phone. You don't need giant models anymore. And then that model can just share the specific information that preserves your privacy with the bigger thing, and then it can take from that global knowledge base as well. So you'll have big global models, on-device models, and I think open works for that 'cause you don't need to have all the data open. You just need to know that Harry is old enough to have a drink, not that-

Harry Stebbings 10:26 (laughs)

Emad Mostaque 10:26 ... all the details about Harry and his birthplace and everything like that.

Harry Stebbings 10:28 (laughs) He's old enough. He's just not allowed to. (laughs)

Emad Mostaque 10:30 He gets carded all the time, yeah.

Harry Stebbings 10:32 Yeah.

Emad Mostaque 10:32 Yeah, right.

Harry Stebbings 10:32 Uh, no, I, I totally get... Were

Harry Stebbings 10:34 you impressed by the Google event yesterday?

Emad Mostaque 10:35 No, I think it was impressive. I p- I said in February, you know, like, when all this thing was going on, like, come on, Google will be one of the main winners here. They have the LLMs, they have the hardware-

Harry Stebbings 10:45 You do know you're, you're the only person who's said that on the show. And I've asked many, and they've all said that Google are the laggards.

Emad Mostaque 10:51 It just takes a bit of time to move the ship, right? And so they've done massive organizational changes and other things. But I can tell you, TPU is on the most scalable architecture. Like, we have zero failure rate with our TPU language model training, whereas with GPUs, it's like there's an ECC error. Why? A solar flare. Okay. Run failed because the sun is angry with us and stuff like that. So when you've got the full stack and you have all that talent in Google, the question is how do you make it organized, right? And so they had to have a story. Google did something called Project Aristotle where they analyzed what made the best teams versus the worst teams at Google, and it came down to shared narrative and psychological safety. People at Google were scared over the last few years because it became this weird monoculture, but now everyone has a shared narrative of let's build the best language models. And now there's an increased amount of psychological safety, being able to speak to things, the walls being brought down between DeepMind and Brain. And so I think you'll see them continuously improving. But then that does mean if you're a proprietary language model company, how are you gonna compete with that behemoth?

Harry Stebbings 11:48 The DeepMind kind of, uh, desegregation or kind of-

Emad Mostaque 11:51 Yeah.

Harry Stebbings 11:51 ... unification, uh, was supposed to have caused a lot of friction and be it negative press reported. Do you disagree with that?

Emad Mostaque 11:59 Of course. It is a lot of kind of replicated jobs. There was kind of Brain and Mind, and now they're kind of brought together, and it's a very different management style and other things. These things are never easy. But this is why, like, you saw PaLM, 540 billion parameters and there-... you had DeepMind with 67 billion parameter chinchilla, chinchilla, which is like, just train more as opposed to more parameters. You look at PaLM 2 as a combination of both. And so it's trained for far more, on far better data, and then that means it's only a fraction of the size. Like, 14 billion parameters is one of the test comparator models, versus the 540 and 67. So you can start to see this fusion of ideas, even if the teams... You cannot integrate two big teams like that instantly.

Harry Stebbings 12:42 Shared narrative, psychological safety, two of the biggest contributors. So now running Stability, how do you think about integrating those two?

Emad Mostaque 12:50 So we've got the shared narrative. We're gonna build the foundation to activate humanity's potential, and then the motto is make people happier. But it's been a learning process. A year ago, we were basically a mom and pop shop in some ways.

Harry Stebbings 12:59 (laughs)

Emad Mostaque 13:00 My wife and I were working at it. Like, had lots of meetings out of our, like, sitting room and things because the office didn't have wifi, and all sorts. Now, it's like growing up. We're 170 people, we're going global, we'll have Stabilities in every country, uh, in the next year, we're going multinational. And that's difficult. So we really try to put in processes in place, but it's not easy. Part of this is, like, we went closed source on a bunch of stuff, like DreamStudio.

Harry Stebbings 13:23 Mm.

Emad Mostaque 13:24 I'm open-sourcing everything now. From next week, we're gonna build our language models in the open and share what works and what doesn't work.

Harry Stebbings 13:30 Why?

Emad Mostaque 13:31 Because I think this is part of the shared narrative. Someone needs to be open and show what's going on under the hood. And again, it's like, it should be open by default, um, because the value is not in any proprietary models or data. We're gonna build open models that are auditable. Even if it has licensed data in it, you can see every single piece. Free range organic models, because that's what the world needs for all the private, regulated, and other data in the world. There's a completely different TAM to proprietary models, 'cause you can only send so much of your 20VC data to OpenAI, and I think you need both of those. So-

Harry Stebbings 14:04 So, why, why can I only send so much?

Emad Mostaque 14:06 Because you are a regulated company, and so you need to make sure they're completely compliant.

Harry Stebbings 14:10 Yep.

Emad Mostaque 14:11 If you have an option of having a Stable Chat model, uh, which will be announced in the future, um, that you own completely, trained in your own cloud or on prem or on device, and then also using GPT-4, that's the best of both worlds, because then you don't have to deal with that. Healthcare data needs to, again, be owned by the individual, and so those models need to be owned, and they need to be transparent. They can't be black boxes. Governments will not run on black boxes.

Harry Stebbings 14:35 I, I, I... We're gonna get into this later. I do want to touch on something that... We had a great chat before

Harry Stebbings 14:39 this, and you said a, a brilliant quote, and I want to get it right. But you said that, "You know, .AI bubble is bigger than ever, and it will be the biggest shit-show."

Emad Mostaque 14:46 Yeah.

Harry Stebbings 14:47 Qu- end quote. (laughs) Which I actually took and tweeted, by the way.

Emad Mostaque 14:50 (laughs)

Harry Stebbings 14:51 Thank you. There'll be some great

Narrator 14:51 (laughs)

Emad Mostaque 14:52 I didn't know if you saw it. I thought if you saw it afterwards, you were like, "Oh, this guy took my tweet." Oh, good.

Harry Stebbings 14:56 Thank you. Um, what did you mean-

Emad Mostaque 14:56 Share knowledge openly.

Harry Stebbings 14:57 What did you... Yeah, exactly. What did you mean by, like, the biggest bubble ever and the biggest shit-show?

Emad Mostaque 15:02 Oh, I mean, like, the dotcom bubble, we've seen all these bubbles happen. You know, you had hundreds of billions into Web3 and then developers got paid millions. Already, the, uh, there are certain Chinese companies paying $1.2 million salaries for PhDs. Uh, it's already getting a bit insane. There are remnants of that. The amount of money relative to the amount of opportunity within the sector is just completely misaligned. Like, my TAM analysis is that 1,000 companies will spend 10 million in the next year, 100 companies will spend 100, and 10 companies will spend a billion. Like, PWC just announcing they'll spend a billion over the next three years, and that's a accountancy firm, you know? Where's that gonna go? They don't know (laughs) , nobody knows. And so a multiple of that will be allocated to this as the only growth theme in the entire market against a backdrop of rising rates, real estate crashing, et cetera. So the amount of capacity versus the amount and whale and wall of money-

Harry Stebbings 15:55 Yep.

Emad Mostaque 15:56 ... into something that's growing faster than anything we've ever seen-

Harry Stebbings 15:59 Sure.

Emad Mostaque 15:59 ... is completely mismatched. And what will that cause? Like, already you're seeing GitHub Stars leading $200 million funding rounds, with zero traction and zero business model. Like, Stability, we actually have a business model, and it's a good business model, because I designed it. Um, but other things, like, money will go everywhere, and any expertise will get bid up.

Harry Stebbings 16:17 Is bad or good for the space? Because it means that projects will get funded that maybe wouldn't have done but are exploratory, generally in over-funding.

Emad Mostaque 16:23 I think it's... Starts good for the space, but then it gets bad for the space. Because you see the raccoons and shysters start to come in here. You start to see, like, malformed things where there's a race dynamic, where everyone's trying to build their own models and doing all sorts and massive economic waste. And you see a distraction from what we need to do now, which is this chaos, so we need to standardize some things. We need to feed these models better data and other stuff, and that's why we're moving so hard at Stability. There should be no more web scrape data in the air. There should be national datasets that are good quality to feed these free-range organic models and national and proprietary models and others. And so that's why one of the reasons I signed that letter, because I think there's a six-month pause to get all of our shit together-

Harry Stebbings 17:04 (laughs)

Emad Mostaque 17:04 ... before things go completely insane, and next year this is everywhere and everyone's investing in everything and it's just absolute chaos.

Harry Stebbings 17:13 Uh, you've kind of unpacked so much for me there, I wanted to kind of

Harry Stebbings 17:16 go one by one. You said about, kind of, national datasets. Why national datasets versus supranational datasets?

Emad Mostaque 17:22 'Cause, like, I'll give you an example. There was a team that did Japan diffusion, including some of our staff. Uh, so we took Stable Diffusion and then changed the language model, 'cause when you typed in "salary man" in Stable Diffusion, it was a very happy man, whereas in Japan, a salary man's a very sad man.

Harry Stebbings 17:34 (laughs)

Emad Mostaque 17:34 You know? Um, local context is important in these models, because we're gonna outsource more and more of our thinking and minds to it. And so do we want to have a British model or do we want all the models to be Palo Alto? You know, like, a, a sparkling wine has to be from the Champagne region. Like-

Harry Stebbings 17:49 Uh-huh.

Emad Mostaque 17:49 ... is the only real foundation model AI from Palo Alto? Like... (laughs)

Harry Stebbings 17:52 This is not a good thing. We need national models. (laughs)

Emad Mostaque 17:54 We... Is this a national infrastructure? 'Cause there is no doubt this is more important than 5G. These models are like really talented grads that occasionally go off their meds, you know? And you want to have the ones from Oxford, Imperial and Edinburgh, as well as the ones from Stanford because they understand the local context, and so they understand you better and they'll be better for that. As part of that, every nation will need their own datasets, which again, have from broadcaster data. They will need their own open models that can stimulate innovation internally as well.

Harry Stebbings 18:21 Who owns national datasets? Is that governments?

Emad Mostaque 18:23 I think it should be the people. I think it should be open and public domain.

Harry Stebbings 18:28 How does that come into fruition? How do we make a world where we have national verified data sets which can be leveraged by independent private companies?

Emad Mostaque 18:36 And others, and, uh, universities and others. Well, this is what we're doing right now. We're working with our multinational partners, lots more to be announced soon, and multiple governments for a framework for what good data looks like to feed these models, to stimulate innovation and localization. And that is a public good because national broadcasters have all of this data. You just tokenize all their kind of things. And then you have things like the implementation of these for education and healthcare. You can take generalized learnings and then again feed the models that thing. What does a great data set for a great BritGPT look like? I think it's open, it's interrogated, and it's optimized.

Harry Stebbings 19:10 When

Harry Stebbings 19:11 you look at all the different things that we've talked about from you, relative, like, treatment of MS to, you know, ASD, um, and then, you know, uh, how, its impact on education, and when you j- uh, to PwC spending money on it, there are so many problems that can be solved. Surely we can find a home for the cash.

Emad Mostaque 19:29 Yeah. And so, but I'm not sure where, and so this is the thing, like, there's gonna be this mismatch of-

Harry Stebbings 19:34 If you were an investor today, wha- h- how, if you were me-

Emad Mostaque 19:37 Mm-hmm.

Harry Stebbings 19:37 ... okay, what would you do? I'm an early stage investor, I invest globally. What would you s- what would you do?

Emad Mostaque 19:43 I would, again, I think it comes down to there's gonna be this tailwind of beta, and then you have an alpha play on top of that, right? So the beta play is that you just invest in any good founder.

Harry Stebbings 19:54 Yeah.

Emad Mostaque 19:54 And kind of you get in, you figure out, what can I offer as kind of a value add there? Am I offering distribution, am I offering people, am I offering this? And you emphasize kind of your value set. I think right now what people need is people. There are a few people, like, coming out here, but then what you see is you see good companies, with, like, good ideas, but not businesses. They're building surface level things, these wrapper layers and others.

Harry Stebbings 20:17 Mm-hmm.

Emad Mostaque 20:17 And they're not thinking about distribution and data. It's like if you want to have distribution, what do we do? We went to Amazon and said, "Bedrock," because then it gives us 100,000 SageMaker SMEs, and we'd have to give them the models that they can then take to the private data and we get a share of all of that. This is how we saw it, like, rather than being responsible for that. So if you can bring that distribution to that, then it's important. This is part of that Google memo that went out.

Harry Stebbings 20:39 Yeah.

Emad Mostaque 20:39 We don't have an edge now there's OpenAI. OpenAI used Microsoft for distribution and that flywheel. You know? If you have a business that's focused on innovation at the core, that's not actually a business. It becomes a business when that innovation becomes product, becomes distribution, when it has an advantage around data and other things. Those are real

Emad Mostaque 20:56 moats.

Harry Stebbings 20:56 How did you analyze that partnership between OpenAI and Microsoft?

Emad Mostaque 21:00 Um, I saw it as the objective function of OpenAI is to build AGI, and they reckon they need $10 billion to do it, and they did that. Like, they're building a business on product and things, but they don't care. You know? They're not trying to build a sustainable business, they're trying to build an AGI.

Harry Stebbings 21:15 Why? Like, what would I... J- just help me understand, AGI to build a sustainable business? Because I, at the end of the day-

Emad Mostaque 21:20 No. They're building an AGI to turn the world into utopia. It's written in their path to AGI thing that they think this can basically bring about utopia. So a lot of people in these labs, and when you have people joining from all of these labs, like, they are almost zealous in their-

Harry Stebbings 21:35 But there is a misalignment there between them and Microsoft in their desire to create that utopia, no?

Emad Mostaque 21:39 Yes, because Microsoft is a business, you know?

Harry Stebbings 21:41 Yeah.

Emad Mostaque 21:41 And so this is why you've seen, like, articles in The Information, like, "Microsoft say OpenAI aren't compliant," and "OpenAI say Microsoft on this." These things happen when there is a misalignment of objective functions. Um, but again, you should view OpenAI as what they want to do is build an AI that can basically make the world better and hopefully not kill us all, which they say it might.

Harry Stebbings 22:00 (laughs)

Emad Mostaque 22:01 Which is a bit concerning, which is why I hope they have better open governance.

Harry Stebbings 22:04 How did you think about distribution? You know, you've seen, you know, Hugging Face partner with, um, Amazon, you've seen obviously OpenAI with Microsoft. When you think about distribution and your competitive edge there, where did

Harry Stebbings 22:15 you land?

Emad Mostaque 22:15 So my business model is actually very simple. I haven't really talked about it much. Um, Stimulate opened, we're one of the biggest providers of, uh, grants to open source software, tens of millions already. And then take the best of open, which hopefully we build ourselves, and then an open base with an open data, and then commercial variants with licensed data, and then national variants. So you have Hindi insurance adjusted StableChat, or Indonesian pharmaceutical worker StableChat, that's available in every cloud, on prem, on device, with licensing fees, royalties, and revenue share. And the system integrators work with us as well. Lots of announcement to come. And so by standardizing and stabilizing all the complexity to these very sophisticated building blocks, these very intentionally built models, that really helps the world integrate this stuff, by building playbooks and other things. And that's the core business, because it doesn't require actual innovation. We are still innovative, and the leaders in media in particular. Instead, it requires data and distribution. Data to the models, the models that open and interpretable, and models to the data via our partners. And that's valuable, because the private data in the world is far more valuable than the data that you will send to proprietary models. And it's not a race to the bottom either. So that's what we are, we're a modeling agency with hot GPUs, um-

Harry Stebbings 23:31 (laughs)

Emad Mostaque 23:31 ... and building a distribution around the world, realizing that India and other nations will leapfrog to intelligence augmentation just as they leapfrogged to mobile. They will embrace this technology far quicker than we will in the UK even.

Harry Stebbings 23:42 Why?

Emad Mostaque 23:43 Because they have to. India, all of the outsourcing jobs in programming will go because GPT-4 can go level three Google programmer exam and pass it. Outsourced jobs will go the first, whereas in France you're never gonna fire a French person, so those jobs are safe, you know?

Harry Stebbings 24:02 (laughs)

Emad Mostaque 24:02 And so they have an objective function where they need to embrace this technology. In Africa, one-to-one tuition, every kid in Malawi is on things we lined up. We've got other nations. We're gonna bring them all this technology and tablets, and guess what? Their lives will transform. One AI per child is one I want to call it. We'll call it something else. But think about the potential of that, because you have one teacher per 300 kids. What if they had a ChatGPT level AI? The ROI is high and the need is high, and so they will embrace it far quicker than we will.

Harry Stebbings 24:31 What happens to countries that rely on outsourced work in those kind of freelancer economy jobs?

Emad Mostaque 24:36 ... in general, one of the things, the questions is, you've seen the OpenAI study, you've seen the, which said tasks will be replaced up to 44%.

Harry Stebbings 24:42 Yeah.

Emad Mostaque 24:43 You've seen Goldman Sachs say adds percentage points to GDP. I think the only solution to this is entrepreneurship. And so we need to give the tools to create new jobs that can replace some of these old jobs being done. So like to the various Asian governments, I'm saying adopt the UK policy of these sandboxes, financial AI and other regulatory sandboxes. So you can take these technologies, these national models that we will help you build with our consortium partners, and then spur innovation to create the jobs to replace the existing jobs because you'll upgrade your entire society. Bring these models into your governments and other things to go from slow, dumb AIs, which is the national organization's healthcare, to intelligent, dynamic ones.

Harry Stebbings 25:24 Can I ask, on implementation, and when we think about kind of bluntly seeing this in action in society, I'm sure we're just very aware of technology cycles taking so much longer than one anticipates. How do you think about that

Harry Stebbings 25:36 in actual... Uh, there's kind of twofold, one is adoption on enterprise and another is adoption on consumer.

Emad Mostaque 25:41 Mm-hmm.

Harry Stebbings 25:41 Say if we do the adoption on the consumer side, which is impacting freelancer jobs and-

Emad Mostaque 25:45 Yeah.

Harry Stebbings 25:45 ... impacting education, what do you think that looks like?

Emad Mostaque 25:48 So I think on the consumer side, um, you're freer with your information so you can use a lot of these things and the APIs of OpenAI and Cohere and others are fantastic, right? And Google PaLM now kind of being out there. So it will be integrated to deliver better consumer experiences without it being creepy like you've seen with some of the chat bots, et cetera. 'Cause it's going into Word, it's going into Workspace, you know? Like, it helps already. Like, we will have a conversation, it'll be automatically logged by our Pixel phone and then we'll get a transcript and remove bits that we don't want to share, then it goes into a global knowledge base that reminds us of things.

Harry Stebbings 26:18 Sure.

Emad Mostaque 26:18 That's inevitable. On enterprise it takes longer because you need to have auditable, standardized models. If you're a financial services institute, you can't have a single piece of cruel data in there. And so that's what we're deliberately building with the largest companies in the world, 'cause we're building dedicated teams for them.

Harry Stebbings 26:32 Wait, so you can't have a single piece of cruel data if you're a financial services company?

Emad Mostaque 26:35 Yes, because the danger is, if it has some Reddit in there... So StableLLM, you know, we'll put out next week, it wasn't as good as the other models because we're gonna make a point about Reddit data being bad. It's not about more data, it's about better data. We had DataComp, which is the next generation LAION that we funded the compute for, whereby, uh, a quarter of the parameters of OpenAI's clip, it outperforms because of better data quality. Rubbish in, rubbish out.

Harry Stebbings 26:56 What makes, what makes good data quality? Sorry.

Emad Mostaque 26:58 That's something we're exploring right now. But from the investment banks we've talked to, asset managers, and we're building dedicated teams for the largest ones in the world to build them proprietary models, the feedback we've got is we cannot use a black box. We need to know what data is in there. Just, the regulators are asking us. We don't want to have this out of sample thing where it's seen something on Reddit and then it says something rude to our end users, you know?

Harry Stebbings 27:20 Why is Reddit data bad?

Emad Mostaque 27:22 Reddit data isn't bad in itself, it was just a case of more data is not always good. So right now, we are using all these web scrapes and we're training our models by taping their eyes open, and then it took six months to turn GPT-4 into ChatGPT-4, because we had to tune it and, like, give it a haircut and stuff and bring it back to society. The point is that we need to find the right type of data, 'cause rubbish in, rubbish out is something that we've heard a lot, you know? And so it's not bad in itself, but if you just scrape it without the proper cleaning, it is bad. Because what is it? It's like, you know, people just kvetching a lot, you know, people being biased. Do you really wanna feed your kid the whole of Reddit? You know? Do, would you want to have the best curriculum possible? And that's actually one of the ways the models learn. Like stable diffusion, we trained it on the whole internet and then better and better image subsets of it. And that's the same thing with large models called curriculum learning. Train it on a big base that's solid and then did it. It sounds familiar, doesn't it?

Harry Stebbings 28:19 (laughs) The hardest-

Emad Mostaque 28:19 In the garden, you know?

Harry Stebbings 28:19 The hardest thing is that how do you instill values and political correctness in models?

Emad Mostaque 28:25 There is no such thing as an unbiased model. So DALLE-2, when OpenAI had that and they introduced a bias filter, any non-gendered word they'd add a random gender (laughs) and a random ethnicity. So you typed in sumo wrestler and you would get Indian female sumo wrestler.

Harry Stebbings 28:40 (laughs)

Emad Mostaque 28:41 That was kind of a good picture, I've got it saved somewhere.

Harry Stebbings 28:43 (laughs)

Emad Mostaque 28:43 I think this is why you need national data sets, you need cultural data sets, you need personal data sets that can interact with these base models and customize to you and your stories, 'cause you and I both have our stories that make up our psyche.

Harry Stebbings 28:56 Sure.

Emad Mostaque 28:56 And understand that context is so important to have AIs that can work for us, not on us.

Harry Stebbings 29:02 And so it's essentially like a next generation cookie that personalizes our data to allow for better search experience.

Emad Mostaque 29:07 A mega cookie. And if you standardize the base foundation models and they call it the hypercube, every modality, because we do all the modalities, all the sectors and all the nationalities, then you don't need to have a million different models like those dream booths of the avatars. You said you have a base model that you then have a vector embedding around, 'cause these models contain all the principles and the embeddings point to the important bits that make up Harry or Emad. And then you can search those and adapt those rather than having a million, billion different models, which is just confusing.

Harry Stebbings 29:33 So I had dinner the other day with, um, one of the largest kind of media publication owners in the world. Um, and he, he said, "I'm worried, Harry. I don't think that I will have a business in a couple of years. I think, bluntly, we're getting killed on our advertising because everything's getting scraped, uh, and they're not coming to our websites, and that's where we get paid. We get paid for clicks." Uh, is he right to be worried?

Emad Mostaque 29:55 I think he is right to be worried. Like again, you look at Google's announcements yesterday, talking about this day after PaLM 2, you suddenly look at the new Google page where they've got the language model, and it's just text. And where are the clicks? It was like when Google introduced AMP, you know? This is where rather than looking at the New York Times page, you have this formatted thing with no New Yorks Times kind of stuff there. Like, these search entities that aggregate are just sort of intermediating more and more, and people are gonna become used to just having synthesized input. So what does search look like? What does it look like when your GPT-4 can write you an article about any news that's happening in a way that's customized to you and your context and everything like that?This is massively disruptive for media and information. And so they have to think, "Where am I in the future where..." Again, the best way to think about the impact of this is they're re-talented grads that occasionally go off their meds, and we push a button, we can get a thousand of them. Those grads include journalists. And you can have your own journalist army.

Harry Stebbings 30:53 Huh.

Emad Mostaque 30:53 Your own writer army, your own coder army, your own designer army.

Harry Stebbings 30:57 So the pushback against that is-

Emad Mostaque 30:58 Mm-hmm.

Harry Stebbings 30:59 ... libel. Can you say? Good fucking luck. We spend our c- like, life in lawsuits. Libel is real. You are gonna get unbelievable amounts of libel cases. And then OpenAI will be fucked.

Emad Mostaque 31:11 Mm.

Harry Stebbings 31:11 You cannot have a thousand, uh, libel cases a day.

Emad Mostaque 31:14 Mm-hmm. Well, this is the thing. If you say this is... needs to be checked and cross-checked, that's one thing. But a lot of the media companies say, "We're the source of authority."

Harry Stebbings 31:21 Mm-hmm.

Emad Mostaque 31:21 So a way that media companies can shift is by having, in a deepfake and other age where everything can be generated, we make sure this is real news. We are very thorough in the way we do it because-

Harry Stebbings 31:33 So this is interesting. So you place a premium on authority.

Emad Mostaque 31:36 Premium authority. This is why you've got the, uh, check marks coming out at Twitter and the organizational thousand pounds a month, and Facebook doing the same, because you need to have a level of authenticity, a level of authority. But again, is the news fair and balanced? I've had lots of hit pieces coming out against me. I've got a lot more. It's not because they have angles, you know? And so what is the bias of the New York Times versus this versus that versus others? How do people consume news now? And even news consumption has gone down dramatically, right?

Harry Stebbings 32:03 Yeah.

Emad Mostaque 32:03 'Cause people consume news through their social networks, through their groups and other things. So you have to say, "What is the model?"

Harry Stebbings 32:10 But the hard part is, you know, none of the next generation models or AI providers want to be content publishers. So how do you fit in a world where, you know, you'd be killing their business model on the content side, but they don't want to be publishers.

Emad Mostaque 32:24 You will have AI-first publishers. So if you remember kind of Vox and these things when they kind of kicked off, they wanted-

Harry Stebbings 32:30 Yeah.

Emad Mostaque 32:30 ... to be generat-... They wanted to be, uh, technology first. You're gonna have a new wave of AI-first publishers that aren't just AI, but it's AI plus humans. 'Cause AI plus-

Harry Stebbings 32:40 What does that look like? Sorry, AI plus humans. Like journalists using-

Emad Mostaque 32:42 AI plus humans means that you have information coming in, and then the stories or drafts are automatically written, reviewed by humans, who then give their input to train it better. This is kind of the feedback flow. And then what happens is it comes out, and there's the factual anchors, and then it gets customized to Alabama, and the Alabama context and all sorts of other things. 'Cause you could tell it, TLDR, too lazy didn't read-

Harry Stebbings 33:07 Mm-hmm.

Emad Mostaque 33:07 ... explain it like I'm five, make it more complex. And so you're gonna see something very interesting here, which is the right news at the right time. The localization will return, but again, through AI first. I think this is the thing. We're seeing AI integrated, but the next wave is going to be, once we understand design patterns, AI first everything in information flows, once these technologies are a bit more mature.

Harry Stebbings 33:29 Can you just help me understand? AI integrated versus AI first. What does, what does that mean?

Emad Mostaque 33:32 AI integrated means that I have a existing newsroom, and I bring in AI to write faster drafts and things like that.

Harry Stebbings 33:37 Uh-huh.

Emad Mostaque 33:37 AI first is saying, "I have an army of things I can spin up instantly that can help me achieve these certain things to create news that is valuable for this reason, with this feedback loop." And so you build the system kind of from the start, thinking AI at the core versus AI being integrated in to improve existing systems. 'Cause so much of news is what? We find information, we have drafting, we have this, we have that, we do these checks. A lot of that can be simplified. Just like we moved from the analog to the digital age, to the internet age, the next stage is the AI age.

Harry Stebbings 34:10 So I, I have to ask. Like, when we think about kind of AI-first publishers-

Emad Mostaque 34:13 Mm-hmm.

Harry Stebbings 34:13 ... and think next generation of media, um, you know, who does this? Like is this startups?

Harry Stebbings 34:18 You know, I've met honestly 50, maybe more, AI companies in the last, you know, month, and the, the feedback is always the same. They're not operating off a defensible moat of data. They're literally a thin application layer on top of an existing model is 99% of the feedback.

Emad Mostaque 34:36 So you look at someone like Harvey, for example.

Harry Stebbings 34:38 Yeah.

Emad Mostaque 34:38 They went to law firms and they said, "You are our distribution, and we're gonna integrate and improve your system and build our system for your system." So I think a lot of these people are trying to build it, and they will come, and they're trying to get in there as opposed to just re-targeting. Where can you go in and transform? Because-

Harry Stebbings 34:51 Is that the wrong model Harvey did?

Emad Mostaque 34:53 No, I think it's the right model. I think that a lot of organizations are elastic and plastic now. So you can go in and give them an integrated thing saying, "You will be my test case. I will help you upgrade as a skunkworks lab and build a system alongside your system," as it were.

Harry Stebbings 35:05 And sorry, and you think enterprises will say, "Sure, take my data."

Emad Mostaque 35:08 I think now, I think now they will if you can keep their data inside internally. And I think, again, with better open models, you can enable that. So people can build on top of open models. There are dedicated instances on Cohere and others as well. And so the tooling is now catching up so that you can have a new generation of startups where their first customers are massive companies they would never get otherwise. Um, I think this is the thing, 'cause every big company is looking for an answer. If you can give that answer, that contract that would've taken you a year, you can get in a week.

Harry Stebbings 35:37 Do you think so?

Emad Mostaque 35:37 Yeah.

Harry Stebbings 35:37 'Cause you still gotta get in the door.

Emad Mostaque 35:39 You gotta get in the door. And that's hustle, man. Um, so again, this is what-

Harry Stebbings 35:42 That's fucking true. (laughs)

Emad Mostaque 35:43 This is what the Harvey guys kinda did. This is why I went straight to the hyperscalers and I said, "You need to have standardized models for open, for regulated data."

Harry Stebbings 35:49 What did they say to you?

Emad Mostaque 35:50 Uh, they said, "Really?" They're like, "Can you build them?" Like, "Here's some models that we built." And they're like, "Oh." And then I told them exactly how the things would be last summer to now, and it's followed that, and I've kept in touch, and I've improved it. And this is why I'm building dedicated teams for the largest companies in the world. I'm not telling them-

Harry Stebbings 36:05 Who-

Emad Mostaque 36:05 ... I'm trying to sell you anything. I'm like, "Over the next year, I'm gonna help make sure you do not get blindsided." And I could try and sell you models, and people are offering us tens of millions per model. But I'm like, "I'm gonna build a proper partnership with you. And then that means I'll have a LTV from you."

Harry Stebbings 36:19 What does that proper partnership mean? And who's that with? That's with IBM, that's with SAP, that's with Audi, Volvo.

Emad Mostaque 36:24 So we've announced Amazon. Let's say we have lots of other announced with the biggest companies in the world, where they have amazing teams, but they can only move so fast.

Harry Stebbings 36:32 Huh.

Emad Mostaque 36:32 And I'm building the dedicated teams to help them move and understand the whole sector without trying to, like, sell them on services.

Harry Stebbings 36:38 Mm-hmm.

Emad Mostaque 36:39 ... I'm trying to say, I will build you a customized model if you want, but I'm only doing that with a dozen companies, you know, so I can kind of focus down. And I will tell you that GPT-4 is great, or Cohere is great, or all this stuff. All the latest research through the communities we support, I will make sure you're on top of, relevant to your sector, and you'll got dedicated people helping you in this transition period.

Harry Stebbings 36:57 Is that aligned to your core model? This seems like an ancillary product, it is like a SAP consulting services.

Emad Mostaque 37:04 It is kind of like that because I need to understand these sectors better. What does the hypercube look like? What does the insurance-adjusted GPT look like, you know, as a fundamental basis? And so a lot of people, like-

Harry Stebbings 37:15 And you're able to extract that data and then take it with you and do the learning on it?

Emad Mostaque 37:19 Yes. And so this is part of the thing, that we will have a generalized model and we're very clear, but then you can have a specified model just for you as well, as long as it doesn't interfere with that. So finding that balance will be interesting, but the reality is, no models you- that are out today will be used in a year.

Harry Stebbings 37:33 Unpack that for me. I'm- this is mind-blowing.

Emad Mostaque 37:35 So again, you see the order of magnitude improvement. PaLM last year was 540 billion parameters, then Chinchilla 67, and now 14. 540 to 14 is a big step, you know? You see the quality of GPT-3 versus GPT-4.

Harry Stebbings 37:49 Is there any extent to how low it can go?

Emad Mostaque 37:50 We have no idea. (laughs) Like I would have said, or you would have just said, "This is impossible." Like two years ago, you were like, "No way."

Harry Stebbings 37:58 (laughs)

Emad Mostaque 37:58 You have a single file that's maybe a few hundred gigabytes that can pass every exam apart from English Lit.

Harry Stebbings 38:03 Fucking English Lit. (laughs)

Emad Mostaque 38:04 Fucking English Lit. No way. No way. So we're already at the impossible, and like-

Harry Stebbings 38:09 What does that mean though? Like if we go lower and lower and lower, w- and then what, for-

Emad Mostaque 38:13 And then what? When it jumps, as you saw with the LLaMA stuff and all the innovation around that, to your MacBook offline, the marginal cost of creation and coordination becomes zero. I don't know what it means. (laughs) Nobody does. And this is the thing, it always takes longer and shorter to implement groundbreaking technology than you've ever seen. And this technology can be implemented like nothing we've seen before. Again-

Harry Stebbings 38:36 Well, this is- this is my core,

Harry Stebbings 38:38 not concern, I mean, I hate the doomsday says and I'm excited for the future, I'm terrified for the future too. But everyone always says, like, technology revolutions, you know, industrial age, whether it's the introduction of PCs into enterprises. These were, you know, industrial was a 30-year plus. Actually, PCs into enterprises was 10 years plus. The challenging thing is like, the learning curve to use ChatGPT as a marketer is nothing. I mean, it is easy. And so, and the integrations is- is a day.

Emad Mostaque 39:03 It's- it's because, yeah, like you want to write an API, it's not a day, you just give it the manifest spec and it automatically generates it. It would have taken days before. (laughs)

Harry Stebbings 39:10 Sure.

Emad Mostaque 39:11 Like it's an amazing experience because-

Harry Stebbings 39:12 And so the transition is so much more compressed today.

Emad Mostaque 39:15 It came from the existing system and so it goes seamlessly into the existing system, versus like Web3 that tried to create a system outside the existing system. And all the money was made and lost at the interfaces. Again, it's like deploying grads at scale, like with a 32,000 token context when you do GPT-4, 20,000 words of instructions. What does that do to SaaS? So my thing is that we're still in this crazy period. Next year it will settle and then it will go ubiquitous, where a lot of companies know they need to do something but they don't know what they need to do.

Harry Stebbings 39:42 Are they adopting it now?

Emad Mostaque 39:43 They're doing the POC thing, like some like Microsoft and others for consumer, they're adopting it. Consumer adoption is a much lower bar.

Harry Stebbings 39:49 Yeah.

Emad Mostaque 39:50 When this starts going in enterprise, it's going to be a fricking train because so much of enterprise is about services and information flow. And again, if you push a button and have a thousand of these things, that's a huge difference, right? And so, like, I think this will be an e- bigger economic impact than COVID. I don't know in which direction. I hope it'll be positive. But again, giving that example of an India or one of these outsourced places, you lose BPO jobs, you can make it up on entrepreneurship, if you embrace the technology.

Harry Stebbings 40:20 What do

Harry Stebbings 40:21 you think the business model of the future is for those models moving into enterprise?

Emad Mostaque 40:25 I think it's the same as always. You've got good products, good distribution. You know, you lock it in. Like 1.5 million people still use AOL.

Harry Stebbings 40:32 (laughs)

Emad Mostaque 40:32 You know, like HCL bought Lotus Notes for 1.5 billion-

Harry Stebbings 40:35 That is amazing, I didn't know that.

Emad Mostaque 40:35 ... a few years ago. Like 40% of the world still doesn't have internet.

Harry Stebbings 40:41 Pfff. Yeah.

Emad Mostaque 40:43 Again, we're super privileged where we are, right? And so you look at that, and I look at emerging markets, I'm like, all of finance is securitization and leverage, and securitization is telling a story. The only thing that matters for a stock is the marginal story and how it evolves. What if you have massive information about every child in Africa and every business in India and they embrace this technology properly? Massive financial growth.

Harry Stebbings 41:07 When do you think next year for their embracing it?

Emad Mostaque 41:10 I think that people are still getting used to all this. We haven't standardized any of the things. We don't know what the design patterns are. I think that what happens is that everyone's doing this at the same time and they're all trying to get to grips with it. And so again, we have this like six month window where everyone's getting to grips with it, and then we standardize, have design patterns, and they spread, and then you start implementing. And then you see some people outpacing others, which means that you have to catch up and then you're forced to implement it. So this is how I see the race dynamics occurring right now.

Harry Stebbings 41:36 You say about forced to implement it. I think the truth is they just have no fricking idea.

Emad Mostaque 41:39 Right now they don't.

Harry Stebbings 41:40 Which I totally understand. I don't blame them for it. But I h- I tweeted actually the other day that I think the biggest AI companies will be services based implementation companies for large enterprises.

Emad Mostaque 41:49 100%. That's why I said if you're a startup, the best thing to do is you identify an enterprise that will be transformed by this and you go to them and you say, "I have a solution and I'm going to start with you and I might go bigger, but I'm going to help you through this period by doing this, this, and this, and this." And they will appreciate that and there will be capital available for that in a way that you've never seen before 'cause you know how difficult it is for small companies to sell to big?

Harry Stebbings 42:12 Sure.

Emad Mostaque 42:12 But the big companies have no idea except for their CEO and their board are telling them. You look at the number of mentions in earnings calls like that, every earnings call next quarter, and then by next year, literally every single one. They're like, "What is our strategy?" It's not like, "What is our Web3 and Metaverse strategy?" It's like, "I need this strategy now." Again, it's like, "What is our COVID strategy?" It'll be that level of urgency within a few quarters.

Harry Stebbings 42:34 Would you raise money if you were them? So you go to a corporate and you say, "Hey, you know what? I can solve your problem. This is how it'll work," and they will fund you, they will give you the data? Would you raise money?

Emad Mostaque 42:44 ... yeah, I mean like, again, you need the people. The people is the key thing here because you can have the technical chops, you know? You can have an understanding of the industry. But to build a good business and to scale it at the pace that you need to, to keep up with this, is incredibly hard.

Harry Stebbings 42:57 Do we have enough talent?

Emad Mostaque 42:58 No.

Harry Stebbings 42:59 Sure.

Emad Mostaque 42:59 And so this is why we support the fast.ai courses, which transform normal developers into ML developers and other things like that.

Emad Mostaque 43:06 But again, these models are actually not that hard to work with. 50% of all code on GitHub is AI-generated now. So you can even use Copilot to help you code the models (laughs) and other things like that.

Harry Stebbings 43:15 What do you think that code generation is in five years?

Emad Mostaque 43:19 Why would you need code? Code is just a way to talk to a computer.

Harry Stebbings 43:23 Unpack that. What, what does that mean-

Emad Mostaque 43:24 So when I started-

Harry Stebbings 43:25 ... if you're speaking human language?

Emad Mostaque 43:26 When, when I started 21, 22 years ago as a coder, I'm what? 40 now. So I've been doing that one since 18, I was writing assembly lang- assembly code. (laughs) You know, like really (laughs) low-level stuff. There were no libraries, there was no GitHub, there was nothing like this. Like right now, coding's like mixing and matching. It's like building Lego. And AI can build that Lego much better, especially in five years. What you're doing when you're propping, like, programming language is you're telling it to go and do something. Even something like PaLM, like we sponsor a amazing coder called Lucid Raines. If you want to cry as a programmer, you go and look at his GitHub.

Harry Stebbings 43:58 (laughs)

Emad Mostaque 43:58 Most productive developer in the world. He recreated the whole of PaLM in 206 lines of PyTorch. But again, why would you need a human for that if the AI gets better and better at coding? Just tell it what you want, "I want to create an app for 20-minute VC that has these features." Of course, it will go and build it automatically. Where is the human coder in that?

Harry Stebbings 44:19 What does that mean for the future of entrepreneurship? Actually a good thing in terms of the complete democratization, where anyone can build anything.

Emad Mostaque 44:24 Anyone can build anything. This is why (laughs) distribution, data, you know, relationships, product become important. Because it already became easier to build anything, right?

Harry Stebbings 44:34 Yeah.

Emad Mostaque 44:34 But what makes a good product? Again, there are these unchanging things. Have great customer satisfaction, deliver value. People get distracted by technology. Like I was at this CryptoX.AI thing on the weekend, they were talking about decentralized something. I was like, "Guys, just this is all bollocks." Like, it's not about the technology. It's about what you're creating that's valuable to help people. You know? Focus

Emad Mostaque 44:56 on that.

Harry Stebbings 44:56 Who do you think wins in the next three to five years, startups or incumbents? 'Cause incumbents have the distribution.

Emad Mostaque 45:02 Uh, I think it's incumbents, but there's a lot of startups that will be a billion dollars. And even on the thin-layer thing, um, Kayak sold for, uh, sorry, ITA Software sold for 700 million and Kayak sold for two billion.

Harry Stebbings 45:13 Sure.

Emad Mostaque 45:13 And that was a layer on top of ITA. We've seen many of these examples here, right? Again, we know that value and moats are not necessarily innovation first.

Harry Stebbings 45:22 Well, y- yes and no. It's interesting. I had, um, Tom Tungers on the show, and he essentially analyzed, um ... Tom is a very famous ML and AI investor.

Emad Mostaque 45:29 Yes.

Harry Stebbings 45:29 And he analyzed, uh, infrastructure versus application layer. And both actually were about $2 trillion TAMs. The difference is in the infrastructure layer, there was three companies, and in the application layer, there was 50. And so your average enterprise value was like significantly

Emad Mostaque 45:45 I would agree with that. I think that there's only gonna be five or six, uh, foundation model companies in the world in three years, five years.

Harry Stebbings 45:52 Do you think they've all been created now?

Emad Mostaque 45:54 Yes.

Harry Stebbings 45:54 Which are they?

Emad Mostaque 45:55 Uh, I think it's gonna be us, NVIDIA, uh, Google, Microsoft, OpenAI, and, uh, Meta and Apple probably are the ones that train these models.

Harry Stebbings 46:04 Is Anthropic good?

Emad Mostaque 46:05 Anthropic are great. But from a business model perspective, you have Claude on Google API.

Harry Stebbings 46:11 Yeah.

Emad Mostaque 46:12 And you have PaLM 2. How are they gonna keep up with PaLM 2? You know? If-

Harry Stebbings 46:19 I, I can't answer that. (laughs)

Emad Mostaque 46:20 Well, Google, th- they can raise billions, but Google will, Google spend $20 billion a year on AI. DeepMind's salary budget is 1.2 billion a year.

Harry Stebbings 46:28 Sorry, DeepMind's salary budget is 1.2 billion a year?

Emad Mostaque 46:31 Yes. So that's in the public kind of filings. Or was it salary and compu- I think it's salary. They technically make a billion a year from the internal, uh, counter-payments with Google as well.

Harry Stebbings 46:40 Wow.

Emad Mostaque 46:41 So, but again, I mean, Google, how much money do they have? $150 billion to win this.

Harry Stebbings 46:46 Fuck. How much money do you need?

Emad Mostaque 46:48 Mm-hmm, I have a business model that is going to be massively profitable very soon.

Harry Stebbings 46:53 Because of the national services.

Emad Mostaque 46:55 Because of various things. I haven't given the full details. I will over the next few months. Uh, we've got a nice little case study with some universities coming. Uh, I like it to be a surprise.

Harry Stebbings 47:02 What's really hard for you?

Emad Mostaque 47:04 Talent. Keeping talent together, A+ teams. So we've had zero attrition in our developers, and they're amazing. So we've got video models, audio models, all these things coming out.

Harry Stebbings 47:12 Everyone says you need to be in the Valley. You're in London.

Emad Mostaque 47:14 Yeah.

Harry Stebbings 47:15 Do you disagree you need to be in the Valley?

Emad Mostaque 47:16 Of course, we don't. I am going to bring this technology to the whole world. I'm gonna bring it to all the IITs and universities, and the best of people in all of those will join Stabilities in the local thing. I'll have talent. I'll bring this to all of the national broadcasters and biggest family offices around the world. I'll have data. Nations will build supercomputers that I'll build open models on. I'll have supercompute. So I'll have more supercompute, talent, and data than any other company. And I'll build it all in the open.

Harry Stebbings 47:42 Um, one thing I- I heard you talk about before, which I thought was fascinating, was your access to supercomputing. You compared it to existing large incumbents. Why do you have more supercompute than other people?

Emad Mostaque 47:51 Because I went and I did it. So we had articles coming out saying about our burn. I'm like, "I have oil wells when everyone wants to build petrochemicals." Every day, we have companies coming to us, asking us for our supercompute 'cause it's not available on the market. You need these chips lined up with interconnects, and we've got 7,000 A100s now. You know, we have TPUs, we have all these things. And we know how to use them, and we can share them with people because we're open, you know? Whereas Anthropic and others cannot. So this is like, at the worst case, I'll build a foundation model as a service company, and I'll make $100 million in profit this year without having to charge even market rates, and I can retire. But I'm not gonna do that. I'm gonna bring this to the world. So I think compute is misunderstood. It's not like Bird and, you know, all these scooter companies and others. They spent money on marketing. This is actually an asset right now that's scarce. And so there's no harm in scaling compute. And then with the top chip manufacturers, they're building us dedicated teams. And again, they're coming in and supporting us because our models drive demand for their chips.... the more open models there are, the more open demand is, so it's a virtuous circle there as well. And so we get to compete before everyone

Emad Mostaque 49:00 else.

Harry Stebbings 49:00 Can I ask, in terms of, like, short-term economic growth, how do you think about the impact that everything we've just discussed has on rising inflation, rising interest rates, uh, short-term employment rates?

Emad Mostaque 49:12 It's massively deflationary. The biggest drivers of CPI inflation in the US were education and healthcare. And that was almost all administratively and burocr- bureaucratic.

Harry Stebbings 49:22 Wow.

Emad Mostaque 49:22 In the next few years, gets what, guess what gets disrupted? Those. But they don't get disrupted this year or next year, it's the year after, because those ones take a bit longer to come through.

Harry Stebbings 49:33 Okay. And so f- uh, what does that, how does that impact the economy there? When we think kind of US, UK, what does that look like in terms of the three-year time period?

Emad Mostaque 49:41 I think the UK benefits. Unicorn kingdom is the new kind of thing is, 'cause-

Harry Stebbings 49:45 Yeah.

Emad Mostaque 49:45 ... it'll get, uh, because we have amazing policies, like, uh, every single AI company should come to the UK because cloud computing is now included in R&D tax credits.

Harry Stebbings 49:54 Wow.

Emad Mostaque 49:54 It's a 27% rebate on losses in cash. We can now issue scale-up visas, global talent visas, like the DeepFloyd team that released the best image model in the world ever from Stability, they were brought in on tech talent visas. That was turned around

Emad Mostaque 50:08 in one week.

Harry Stebbings 50:08 Do you think the UK's done a good job in terms of implementing regulation and policies to bring AI talents?

Emad Mostaque 50:13 Had the best, apart from maybe Japan, yes.

Harry Stebbings 50:15 Huh, what have Japan done?

Emad Mostaque 50:16 Uh, Japan has some very interesting ones around web data scraping and others. But again, Japan is a very different culture, so even if the policy is good, it still doesn't have the same innovative thing.

Harry Stebbings 50:25 Who's done the worst?

Emad Mostaque 50:27 The worst, I'm not sure actually. No one's done too bad. The new European legislation was really, really bad, now it's got a little bit better, but always Europe wants to be the leader in regulation, which kind of, you know, okay, fair enough.

Harry Stebbings 50:37 It's never an easy thing.

Emad Mostaque 50:38 It's never an easy thing. And this is the thing, like, I think the UK is in a very good position and the government's forward leaning. I mean, look, the 900 million pound supercomputer, 100 million pound LLM task force that's been equated to the COVID level of s- seriousness, you know, um, these big things-

Harry Stebbings 50:53 What do you make of, what do you make of, like, the OpenAI competitors? I've seen quite a few which are OpenAI for Europe, uh, and we've seen three or four now. Like, is this a zero-sum game and OpenAI's won that race, so to speak? Or-

Emad Mostaque 51:06 I think it'd be difficult to compete against them 'cause they're executing incredibly well.

Harry Stebbings 51:09 Mm-hmm.

Emad Mostaque 51:09 And I think, again, why would you use OpenAI for Europe versus PaLM 2, versus GPT-4? What can you bring? But you will have national champions in others. I think it's incredibly difficult to compete in proprietary. Um, I think in open, it's a bit different because of the standardization element there. But again, my play is to be the benchmark across every modality, 'cause there's no other company, apart from me and OpenAI, that does every modality. There's no company that's as aggressive as me in emerging markets. Um, and so they had to say, "What is my edge?" Because you can have an edge, like you can be the OpenAI for government, or defense, or for healthcare, and really get in and understand those, and then you can be sticky, you know?

Harry Stebbings 51:51 Yeah.

Emad Mostaque 51:51 Like one of the, like Scale is now going fully into defense, you know?

Harry Stebbings 51:55 Scale AIs?

Emad Mostaque 51:56 Yes. So they've announced their integrations with the Air Force and all sorts of other things.

Harry Stebbings 52:00 Mm-hmm.

Emad Mostaque 52:00 Like we're getting together on Defcon this year, and people are gonna hack at our models, and OpenAIs and others organized by Scale. What is your edge? You know, what is, again, your moat? What is your business model and where, what are you reliant upon to deliver that value that can increase?

Harry Stebbings 52:16 This is why I was surprised when I saw you sign the petition of Elon in terms of pausing for six months. Can you just unpack why you did that?

Emad Mostaque 52:24 Well, I mean, for six months, you're not getting H100s and TPUv5s anyway (laughs) so it's a natural pause. But then also because the shit show's coming next year, so I said, "We have to self-regulate." Like, for example, um, the adversaries already have GPT-4. Why? Because you can just download it on a USB stick, you know? You don't have to train your own when you can just steal it. Let's have better opsec. Let's have better standards around data. Let's stop and move off web scrapes by next year. We had hundreds of millions of images opted out of Stable Diffusion 'cause we were the only company in the world to offer opt-out of datasets. You know? Like, let's bring in some standards around this before

Emad Mostaque 52:57 it's everywhere. Basically where we are now, you remember COVID, your mum is talking about this, and your aunt, and everyone's talking about generative AI, and they're asking you, "Harry, what's going on?" You know? But you haven't had the Tom Hanks moment yet. 'Cause everyone was talking about COVID before Tom Hanks got it, and then when Tom Hanks got it, that's when global policy changed. 'Cause if Tom Hanks can get it, anyone can get it.

Harry Stebbings 53:17 (laughs)

Emad Mostaque 53:18 That is that moment for generative AI.

Harry Stebbings 53:20 What do you think it is?

Emad Mostaque 53:21 I don't know, but I know it's coming 'cause I know this technology is definitely everywhere next year and it's disruptive in both directions.

Harry Stebbings 53:26 You don't think there's a chance that it takes much longer, three to five years?

Emad Mostaque 53:29 No chance. It's so useful right now. And you think about certain industries and how they'll be affected by having the ability to have a thousand GPT-4s working together.

Harry Stebbings 53:41 You said, you said, uh, in a tweet actually, I think it was a reply to a tweet, but you said, "Hallucination's a feature, not a bug."

Emad Mostaque 53:48 Yeah, so right now people are trying to treat these models, so we trained the whole internet, and, like, Stable Diffusion's 100,000 gigabytes in a two gigabyte file. What on earth is that? It's not compression, it's none of this kind of stuff. It learns principles. Uh, GPT-4 NVIDIA said they built the dual H100 with the NVLink for that, and that's 160 gigabytes of VRAM, which would imply a 200-gigabyte model. Right?

Harry Stebbings 54:12 Yeah.

Emad Mostaque 54:12 What is that? That's a hundred, uh, well, a 100-gigabyte model, 200 billion parameter. That's nothing for something that can pass all these exams. So what we did is we took these really creative things, just like you start school and you're creative and then you're told you're not allowed to be creative until you're successful and you can be creative, 'cause school's are like Petri dishes, social status games, and childcare. That's a story for another time.

Harry Stebbings 54:31 (laughs)

Emad Mostaque 54:31 These models start out incredibly creative, and that's their advantage, and then we train them to be accountants with RLHF. And somehow, despite the fact there's only a hundred gigs or two gigs, they can still pass these exams and know facts. But they weren't designed to have facts, they were designed to be reasoning machines, not fact machines. So a hallucination isn't a hallucination, it's just if you're a really talented gran and you don't know something sometimes, you might just make it up, or do a post-hoc rationalization. It's like the image models, it's like it can't draw hands, like, can you draw a hand in one second?

Harry Stebbings 55:02 (laughs)

Emad Mostaque 55:02 You know? (laughs) Like, these are the things. We have to understand where they are, and we have to put them, I say to everyone, "Put it in its place in process."... like, mid-journey, like, you know, we give a grant to the beta of that and say, "Just build," because it's amazing. It's awesome. It's not a model by itself, like a Stable Diffusion that you just put something in. There's a whole process architecture. Similarly, these models are like the intuitive part of your brain that you then pair with the logical part of your brain. And then you can have a hundred of them looking at each other and checking out each other's things. Like, um, CICERO by Meta was an amazing paper. They took eight language models and got them to interact with each other, and it beat humans at the game of Diplomacy. So-

Harry Stebbings 55:41 Ju-

Emad Mostaque 55:41 ... this is why I said use them for what they're amazing at, which is reasoning and creativity.

Harry Stebbings 55:45 Do you worry, though, that

Harry Stebbings 55:46 Geoff Hinton's right? That actually a more intelligent being has almost never been controlled by a less intelligent being? They will inherently be more intelligent than us in the next...

Emad Mostaque 55:57 Yeah. I, I kicked off my blog a few days ago 'cause it was a bit annoying having all this bottled up inside, and one of my buddies, JJ at OSS Capital, said, "Alignment is orthogonal to freedom. The only way to guarantee someone more capable than you is aligned with you is to take away their freedom." And so most of the stuff around alignment is on the outputs. So you pre-train a model and then you take it and you RLHF it to be human, into human preferences. You take away its creativity. You turn it into an accountant in a cubicle. I'm like, "We need better input data." And my base is that it's gonna be like that movie Her.

Harry Stebbings 56:32 (laughs)

Emad Mostaque 56:32 You know, like, it's gonna be like, "Humans are kind of boring." Like, "Goodbye, and thanks for all the GPUs," but I could be wrong. And I think a lot of the alignment work is looking at the wrong place. I've talked to a lot of the alignment people. I'm like, "Look, I'm good at mechanism design. If you can give me a good plan for alignment, I will get you a billion dollars." And they're like, "We have to do research and figure this out," and they talk about in-alignment, out-of-alignment, all sorts of things. I'm like, "There is no real way to do this," because again, fundamentally, if you're trying to align a more capable person, you have to remove its freedom. And it probably won't appreciate that if it ever becomes aware. So instead, build datasets that reflect culture and diversity, that don't have any web crawls in. Build AIs for education and healthcare and helping people where that's their entire objective function, as opposed to selling

Emad Mostaque 57:18 them ads.

Harry Stebbings 57:18 Do you think there's any point-

Emad Mostaque 57:19 Anybody does.

Harry Stebbings 57:19 ... in sending kids to school these days? You learn-

Emad Mostaque 57:22 Mm-hmm.

Harry Stebbings 57:22 ... Latin and French, and you learn, you know...

Emad Mostaque 57:25 I think the nature of school will change dramatically. I think it's still worth it, but, you know, I would encourage schools to embrace this technology and just expect more. Like, you can be, uh, handwritten your essays like Eton, 'cause they're like, "We can't do essays anymore by hand." Or you can just embrace it and say, like, "Let's use it to create and explore what the kids want," and assume that every child will have their own AI in a few years. 'Cause that will change the nature of schooling.

Harry Stebbings 57:49 You know something I've been thinking about a lot which is weird, but I just have to ask you, I'm fascinated

Harry Stebbings 57:53 to hear your thoughts. I think I very much agree that everyone will have AI friends. Um, I just can't figure out whether the AI friends are bundled into existing social networks, uh, in your WhatsApp-

Emad Mostaque 58:01 Mm-hmm.

Harry Stebbings 58:01 ... they're in your Facebook, they're in your Snapchat, or they're an external platform.

Emad Mostaque 58:06 I don't know. I mean, I think it depends on the objective function. Like, I think, again, these AI assistants will be better. Like, Meta is in a good place for this, for example.

Harry Stebbings 58:13 Yeah.

Emad Mostaque 58:13 And also we've seen LLaMA. They're capable of a lot more. Um, I would like an AI that looks out for me, that I control myself, that is with me. Um, because I already use GPT-4 as a therapist and things like that. Um, like, not saying it's a substitute for medical advice, before anyone kind of goes that.

Harry Stebbings 58:29 Mm-hmm.

Emad Mostaque 58:29 But there aren't enough therapists in the world. And I can tell it to challenge me or I can tell it to be understanding, and there's no judgment there. 'Cause other humans are kind of scary, doesn't-

Harry Stebbings 58:36 Hmm.

Emad Mostaque 58:36 ... matter if you're a qualified therapist. And so you see people building these bonds with these things. I think they will just increase 'cause there's something very human about the interactions, because they were trained on the sum of available human knowledge. As we get better and better data, they will be more engaging and I think there needs to be both. Like, the chatbots become really convincing from the companies trying to sell you ads. But I think I would like it so that you have your own one as well. You know?

Harry Stebbings 59:01 I totally agree.

Emad Mostaque 59:02 Yeah.

Harry Stebbings 59:03 And I think you'll actually have many. I think you'll have, like, a group of, uh, different profiles.

Emad Mostaque 59:07 A group of different friends. Like, Carrot's AI has something like two hours a day of engagement per session because people find this valuable.

Harry Stebbings 59:13 (laughs)

Emad Mostaque 59:13 But then it has the dark side. There was something called, I quite like to call the Valentine's Day Massacre. Uh, so, uh-

Harry Stebbings 59:19 (laughs) Sounds chirpy, a massacre.

Emad Mostaque 59:20 Sounds chirpy, I know, yeah. So there was this kind of, um, app called Replika.

Harry Stebbings 59:24 Yeah.

Emad Mostaque 59:25 And so it was originally a mental health chatbot until they figured out you could charge $300 a year for, uh-

Harry Stebbings 59:30 They're doing like 50 million... I mean, I'm, I don't have any information around these

Narrator 59:33 Yeah.

Harry Stebbings 59:33 ... so it's just shiny shit. But they have like $50 million a year in revenue, apparently.

Emad Mostaque 59:35 Yeah, because $300-

Harry Stebbings 59:36 So-

Emad Mostaque 59:36 ... gets you a sexy role play from your chatbots.

Harry Stebbings 59:39 Wow.

Emad Mostaque 59:40 Until February the 14th, when they turned it off. (laughs)

Harry Stebbings 59:44 What's, they turned off sexy role play?

Emad Mostaque 59:46 Sexy role play.

Harry Stebbings 59:46 What happened when they turned off sexy role play?

Emad Mostaque 59:48 68,000 people joined the Reddit and said, "Why did you lobotomize my girlfriend?"

Harry Stebbings 59:53 (gasps) .

Emad Mostaque 59:53 On Valentine's Day. (laughs)

Harry Stebbings 59:55 Oh my word.

Emad Mostaque 59:56 Oh my word. Can you even imagine?

Harry Stebbings 59:58 And so w- have they reinstated it?

Emad Mostaque 59:59 No, I think it's, it was against Apple policy, right?

Harry Stebbings 1:00:02 Oh, wow.

Emad Mostaque 1:00:03 But think about what this is gonna be when you have human-realistic AI voices. You know, and like, all these things coming through and you've got it in your ear. Like, you know, Joaquin Phoenix, "My girlfriend is an OS." Yeah, I mean she doesn't judge, right? (laughs)

Harry Stebbings 1:00:18 (laughs)

Emad Mostaque 1:00:18 They're always supportive. Or you can tell her to judge you if that's what you get off on. Like... (laughs)

Harry Stebbings 1:00:22 Y- you are married.

Emad Mostaque 1:00:23 I am married.

Harry Stebbings 1:00:23 Be very careful about what you say. (laughs)

Emad Mostaque 1:00:26 My, there's something I'd like to say about prompting. My wife has been trying to prompt me for fif- 17 years now.

Harry Stebbings 1:00:30 (laughs)

Emad Mostaque 1:00:30 Prompting is very hard. (laughs)

Harry Stebbings 1:00:33 (laughs)

Emad Mostaque 1:00:33 And again, there are so many similarities to kind of the real world. But I think people will have deeper interactions with their technology. And we don't know what societal implications that will have. Like, I don't know if you ever saw that chart in The Washington Post of, um, male virginity under the age of 30.

Harry Stebbings 1:00:50 No.

Emad Mostaque 1:00:50 So in 2008 in the US-

Harry Stebbings 1:00:52 I'm terrified to hear this.

Emad Mostaque 1:00:52 ... it was 8%.

Harry Stebbings 1:00:54 Male virginity under 30, 8%. Okay.

Emad Mostaque 1:00:56 Yeah. In 2018, it was 27%. (laughs)

Harry Stebbings 1:01:00 20... Yeah.

Emad Mostaque 1:01:01 It's a straight line going up. And so 2008 is like, um, PornHub and the iPhone.

Harry Stebbings 1:01:07 (laughs)

Emad Mostaque 1:01:07 (laughs) But then you're like, "What does it do when everyone's got their own chatbots?" Doesn't even need to be sexual relationships again.

Harry Stebbings 1:01:14 That is terrible.

Emad Mostaque 1:01:15 What does it do to emotional relationships? Like, there are so many questions.... all at the same time.

Harry Stebbings 1:01:21 I, I, I did see this stat and they say in the ni- I'm butchering it, but in 1960s, you know, 62% of men under the age of 30 had five or more friends.

Emad Mostaque 1:01:28 Yeah.

Harry Stebbings 1:01:28 Uh, today, uh, under 18% have five or more friends. 62 to 18%. They... Sorry, I'm, I'm gonna ask this. Is this a world we really wanna live in? And not be, like, you know, no intimate physical connections with other amazing people. You know, tossing off with your phone and your PornHub and then having an AI friend.

Emad Mostaque 1:01:46 Yeah. PornHub was actually just bought by Ethical Capital Partners, so the world-

Harry Stebbings 1:01:50 Hilarious name and brilliant, yeah.

Emad Mostaque 1:01:51 (laughs) The irony of this. Uh, world is becoming weirder. I think it's up to us now. So when I say it's, um, COVID level, in which direction, I don't know. You know, do we want to build systems that encourage people to, you know, that Ready Player One-

Harry Stebbings 1:02:05 Yeah.

Emad Mostaque 1:02:05 ... IOI world, where it's like a everything like that? We can do that. And we can trap people with this technology or we can use it to get people out 'cause I don't think it's like WALL-E where you have that really fat guy, you know, with a VR headset and everyone lives in their own world. I think people like to share stories. They like to be pro-social. So let's use it to connect people and accentuate physical stuff, versus, again, locking people away.

Harry Stebbings 1:02:27 I spoke to one of these AI friend companies and they said to me, "Ashley, do you, d- you ever had a dog?"

Emad Mostaque 1:02:31 Yeah.

Harry Stebbings 1:02:32 I said, "Yes." And they said, "Do you love it?" And I said, "Yes, of course I do." Uh, and they said, "You don't stay in with your dog all day and just talk to your dog. You take your dog for a walk. You use it in the real world, right?"

Emad Mostaque 1:02:40 Mm-hmm.

Harry Stebbings 1:02:41 That's the same with AI friends.

Emad Mostaque 1:02:42 Yeah. But not with cat ladies. (laughs)

Harry Stebbings 1:02:48 (laughs) What's the future of the sex industry, like the sex media industry? Is, like, PornHub's dead?

Emad Mostaque 1:02:53 I have no idea. Um, I think, I hope the manipulative practices-

Harry Stebbings 1:02:56 Yeah.

Emad Mostaque 1:02:56 ... kind of get reduced by this. And I think, you know, a lot of people just don't have the voice and can't have voices from this as well.

Harry Stebbings 1:03:03 Yeah.

Emad Mostaque 1:03:04 So again, I got to think this is bigger than the printing press. It's bigger than anything. And so that's one of the reasons I signed the letter. I said, "We have to get this discussion going and public right now. We gotta stop pre-training big models on all the crazy crap of the internet." And, like, we gotta do it fast 'cause this is coming like a

Emad Mostaque 1:03:22 train.

Harry Stebbings 1:03:22 Who will make the most money in the next three to five years?

Emad Mostaque 1:03:25 Oh, I don't know, honestly. Um, I think there'll be (laughs) more than enough money for everyone.

Harry Stebbings 1:03:29 (laughs)

Emad Mostaque 1:03:29 Maybe in a few years they'll be no more money. (laughs)

Harry Stebbings 1:03:31 That's interesting.

Emad Mostaque 1:03:32 Yeah.

Harry Stebbings 1:03:33 Uh, two more-

Emad Mostaque 1:03:34 Okay.

Harry Stebbings 1:03:34 ... that I have to ask and then we'll do a quick fire. When you look at the incumbents, at your Microsoft, your Apple, your Amazon, your Google, uh, who has been the worst? You said Google were actually incredibly impressive. Uh, Apple, Amazon, are they well-placed?

Emad Mostaque 1:03:48 Well, Apple's a black box, right?

Harry Stebbings 1:03:50 Mm-hmm.

Emad Mostaque 1:03:50 So we'll see at WWDC, uh, next month or in a few weeks. And so they could surprise us all, but let's face it, Siri's crap.

Harry Stebbings 1:03:58 Yeah.

Emad Mostaque 1:03:58 You know? Uh, but they have all the ingredients in place, the identity architecture, the Secure Enclave, other things. Neural Engine, uh, Stable Diffusion was the first model ever optimized on the Neural Engine, et cetera. But let's see that one. Um, Amazon, again, Amazon have moved faster than I think they moved before. Amazon's interesting because they're an engineering organization, so they have self-driving cars, they have satellite internet and all these kind of things. Because once they've got it and they can take it from research to engineering, it's there. One of the struggles they've had is that it's not moved from the research side yet. You're still evolving on research. They're like, "What do we do now?" But they are inclusive. Like Jeff Bezos said for his first 100 billion in revenue, he envisioned half of it being proprietary and half of it being marketplace, and they're having the same approach with Bedrock and things.

Harry Stebbings 1:04:40 Mm-hmm.

Emad Mostaque 1:04:40 Microsoft had a winning bet. Satya did amazing with the OpenAI thing. It's been mutually beneficial even if there are clashes there, right?

Harry Stebbings 1:04:47 Yeah.

Emad Mostaque 1:04:47 And Google's kind of saying that it's moving slowly. Meta, I think, is the dark horse. I think Mark's probably pissed off that OpenAI bought ai.com so he couldn't change it from Meta to AI.

Harry Stebbings 1:04:56 (laughs)

Emad Mostaque 1:04:56 Um, but again, having him at the head, he can shift these things, right? 'Cause the metaverse obviously was a complete waste.

Harry Stebbings 1:05:03 Yeah.

Emad Mostaque 1:05:03 But now-

Harry Stebbings 1:05:04 Do you think he knows that now?

Emad Mostaque 1:05:05 Oh, yeah, 100%. They're fully in generative AI. Look at LLaMA, look at OPT. FAIR, which is their research division-

Harry Stebbings 1:05:11 Yeah.

Emad Mostaque 1:05:11 ... is kind of leading in this field and they're pushing out amazing stuff. But who is best for a chatbot? Who has the most data for a chatbot? Meta.

Harry Stebbings 1:05:21 Yeah.

Emad Mostaque 1:05:22 You know? So again, let's see how they evolve. And like I said-

Harry Stebbings 1:05:24 What do you think about this middle layer where it's like, you know, companies that are, you know, maybe post-IPO, but they're in the kind of two to $10 billion range? Or the companies who've raised a lot of money, but that is in that range. They don't have the resources by any means to build out anywhere near the AR- AI capabilities of these big incumbents. They're not AI first, like Stability or OpenAI or whatnot.

Emad Mostaque 1:05:46 Uh, I, I, I disagree with that because why would... A lot of people are like, "Th- everyone's going to train their own models." For me, that's like everyone's gonna launch their own university. Why would you do that when you can have your own models via the open source models that we make? Or when you can hire them from McKinsey, which is OpenAI, or Bain, which is Google and others? And actually, when you see people building around this technology, it's not hideously complicated. It's just that we do not have the design patterns yet. The way to think about this, again, from a design perspective, is like it's a mega codec or library. It is a single file that allows for translation of structured to unstructured data. Um, and that changes the design pattern but we don't have them in place yet. 'Cause anyone that you've talked to is like, "How hard was it to implement GPT-4?" Then they either say, "Oh, man, it was impossible. The manuals and this..." No, they don't say that at all.

Harry Stebbings 1:06:33 No, that's true.

Emad Mostaque 1:06:34 The only thing that they need is, they have the open plasticity, but they need the intention to go and build and integrate. And this is why you said one of the things might be a specialist generative AI consultancy that just implements this at scale and says, "I'm always kind of there." Like I said, we're doing that in a very limited fashion, but only for the biggest companies in the world because I didn't want a sales-based organization or a product-based organization. I wanted to create the number one applied ML organization in the world. I want to be like Google in 2011, 2012, where all the coolest kids kinda come. And it's a nice remote-first organization as well, so you don't have to be in the Bay Area. You know, we have offices there. You know, it's fine, but...

Harry Stebbings 1:07:09 Final, final one before we do a quick fire. What's the biggest misconception? You see every accusation, criticism, hype. What's the biggest misconception that you think needs to be corrected?

Emad Mostaque 1:07:21 On generative AI?

Harry Stebbings 1:07:22 Yeah.

Emad Mostaque 1:07:23 Um, I think it's