Emad Mostaque: These 5 Companies Will Win the AI War; Why We Need National Data Sets | E1015

1. 0:00 – 0:42

   Intro

   1. EMEmad Mostaque0:00

      I, I think this is bigger than the printing press. It's bigger than anything. And so that's one of the reasons I signed the letter. I said, "We have to get this discussion going in public right now. We gotta stop pre-training big models on all the crazy crap of the internet." And like, "We gotta do it fast because this is coming like a train." (instrumental music plays)

   2. HSHarry Stebbings0:18

      Ahmad, I am so excited for this. I heard so many great things from, uh, specifically Ashton, who helped with questions, and then also Dan Rose. So thank you so much for joining me today.

   3. EMEmad Mostaque0:26

      It's my pleasure.

   4. HSHarry Stebbings0:27

      Now, I want to start with a little bit on you. You moved around a little bit in your childhood. Take me back to the childhood, the moving around. And there's a weird commonality I found with the most talented founders. They all moved around. So take me to that and how it impacted your mindset.

2. 0:42 – 3:26

   Who is Emad Mostaque?

   1. HSHarry Stebbings0:42

   2. EMEmad Mostaque0:42

      So I was born in Jordan, grew up in Bangladesh, came to the UK. Um, yeah, didn't move around that much, but with my father a bit as he lectured in various places. And it was always a bit of a struggle fitting in, but then you learn to adapt. You learn to adapt to new scenarios, new environments. Like, "Oh, I don't speak the language, uh, kind of what's happening. Let's learn and let's move on from there." And I think it gave me a bit of appreciation of the world as well because we stick in our monocultures very often. Like, I'd only ever been to Silicon Valley ... Actually, I'd only been to the Bay Area once before last October. And so this whole tech monoculture has been a bit of a shock to me, and I'm like, "There's more to the world, uh, than that." So this is, uh, some interesting things around that.

   3. HSHarry Stebbings1:22

      Talk to me. Hedge funds first, and then, then what happened? We mentioned it a little bit before. Why did you make the move?

   4. EMEmad Mostaque1:29

      Uh, so actually I was an enterprise developer in my gap year at Metaswitch in the UK, doing voiceover IP.

   5. HSHarry Stebbings1:34

      Metaswitch?

   6. EMEmad Mostaque1:35

      Metaswitch.

   7. HSHarry Stebbings1:35

      This is Chris Mazz's company.

   8. EMEmad Mostaque1:37

      Yes, and NPIET.

   9. HSHarry Stebbings1:38

      Yeah.

   10. EMEmad Mostaque1:38

       So I took my gap year and was like, "Oh, I might as well be an enterprise programmer." Didn't know what that would be like. This was before GitHub and everything, so we had Subversion. You know, the kids these days have it so easy.

   11. HSHarry Stebbings1:47

       (laughs)

   12. EMEmad Mostaque1:48

       Um, and then I was like, "What do I do now?" And so I became a VC analyst at Oxford Capital Partners-

   13. HSHarry Stebbings1:53

       Mm-hmm.

   14. EMEmad Mostaque1:53

       ... played with the mots there. And that was a lot of fun. They're fantastic. And then I was like, "I want to do movies." So I became a movie reviewer. So I did the Raindance Film Festival, British Independent Film Awards. And they were just bopping around doing random things, and I accidentally became a hedge fund manager.

   15. HSHarry Stebbings2:07

       Why did you move from hedge funds to startups?

   16. EMEmad Mostaque2:10

       So with the hedge funds, so I joined Pictet Asset Management, and then, like, the CIO left, and there was, like, this fund. And I got to be a portfolio manager when I was 23. And so I grew my beard to look a bit older.

   17. HSHarry Stebbings2:20

       (laughs)

   18. EMEmad Mostaque2:21

       It's coming. Get the clip on Harry.

   19. HSHarry Stebbings2:22

       Uh, I would, I can't do the mustache, but I still wear the glasses when I just do the contacts.

   20. EMEmad Mostaque2:26

       You can try extra hard, force it out, right? Um, and so I did that for a number of years, and, you know, reasonably successful, made lots of people money. Um, not so much myself 'cause I was too young. Uh, again, they're like, "You're too young to be a fund." And then my son was diagnosed with autism, and I quit. And, um, because they said there was no cure, no treatment, no information. I was like, "I'm a hedge fund manager. I can deconstruct things." And so built an AI team, and then did a literature analysis of all the autism literature to try and figure out the commonalities, and then drug repurposing, so focusing on GABA glutamate balance in the brain. GABA is what you get when you pop a Valium. It calms you down, and glutamate excites you, you know? And so in kids and people with ASD, it's like there's too much noise going on. It's like when you're tapping your leg and you can't focus. And so that's why you get this sensitivity. Sometimes they can't speak, like my son. And so it was, like, mechanisms to bring that down, that then allowed to have applied behavioral analysis and these other things to reconstruct his speech. And then he went to mainstream school, which is pretty cool.

   21. HSHarry Stebbings3:22

       That's unbelievable. I, I, I heard you said on another podcast, and I was astounded and inspired

3. 3:26 – 10:34

   AI & Medicine

   1. HSHarry Stebbings3:26

      by it. Wh- w- we mentioned before, you know, my mother's got MS.

   2. EMEmad Mostaque3:29

      Mm.

   3. HSHarry Stebbings3:29

      And I hate the doomsday-only version of kind of AI and the future of, you know, GPT. You said to me before about its impact on health, and MS in particular, and other conditions. How can it be so transformatively to solve some of the world's most challenging chronic conditions?

   4. EMEmad Mostaque3:45

      So I think a large part of our problem is that we can't scale because information flow is so limited as we write these things down. Like, you can never capture all of that. So anyone who's had a loved one that has one of these conditions knows how difficult it is, because you go from specialist to specialist to specialist, and you try to build that mental map. And we're so lucky that we have so much access. But why is it that we can't just push a button and see every clinical trial and a deconstruction of all those and things? What if you had 1000 GPT-4s organizing all that knowledge and then make it available to everyone? So you can see the exact potential mechanisms the way which MS works and all the potential food, other things that work with that. So as you try different things with your family member, you can see, well, she reacted this way to the food or this way to this medicine. And it's a more holistic thing because you can have personalized medicine versus one specialist for 1000 people. You can have 1000 GPT-4s or equivalents or MedPaLM 2s for you. So we need to organize all this knowledge and then use these language models and others to make it accessible to you.

   5. HSHarry Stebbings4:48

      I'm really naive and, and, uh, basic in terms of my thinking, which is why I'm a venture capitalist. (laughs)

   6. EMEmad Mostaque4:54

      (laughs)

   7. HSHarry Stebbings4:54

      Uh, but my question to you is, what do we need to do to get to that state? When we look at the data needed from the individuals, the data, the data the GPTs need, how we make the models work most efficiently, what do we need?

   8. EMEmad Mostaque5:05

      So first, first, we don't have to have that data from individuals. So we had Galactica as a lang- scientific language model, but now we have MedPaLM 2 that exceeds doctor levels. That was a Google announcement yesterday. We have AIs that can understand articles better, as good as doctors, shall we say now. So we can scale that, because why do you need one when you have 1000? So we take the existing generalized knowledge and all the hypotheticals, and we bring that together into an integrated common system available to everyone, because the building blocks are nearly here for that. Then you can personalize it later. And again, there are regulations and things around that, to how you, again, how we treat our loved ones and other things like that. But the first thing is, let's get all the knowledge in one place and make it organized and useful. And so I think we're at that point now where the language models have just hit that point, that we can organize all of the world's Alzheimer's knowledge, longevity knowledge, autism knowledge, MS knowledge. And you can just type, and it can say, this is the source, this is what it looks like, these are some-... hypotheticals. This is what we know, what we know we don't know, what we think we might know, etc. And then it can learn about you and your queries. 'Cause this is the other thing about lots of the language model things we've seen right now, they are one-to-one goldfish memory. The next step is one-to-one, it remembers what you're asking for, like a cookie or an embedding, and then it's you plus 1,000 of these language models all going and doing your bidding, the agent-based kind of thing.

   9. HSHarry Stebbings6:26

      Does this get around the incentive problem in healthcare? And what I mean by the incentive problem in healthcare, as I'm sure you know, there are a lot of diseases actually where it doesn't make kind of economic sense for a lot of pharmaceutical providers to chase research, to chase treatments, because it's not a big enough market, 'cause it's not, 'cause it's a $6 treatment. Does this solve for that economic misalignment?

   10. EMEmad Mostaque6:44

       I think it can help a lot with that economic misalignment because then you have an authoritative source, where we can all come together and build, that can analyze these things. Because there's this concept of ergodicity. A thousand coins tossed in a row is the same as a thousand coins tossed at once.

   11. HSHarry Stebbings6:59

       Mm-hmm.

   12. EMEmad Mostaque6:59

       And because we're so limited in our information in our medical system, like, you know, I just had my key management, so I had to answer 40 minutes of questions.

   13. HSHarry Stebbings7:07

       (laughs)

   14. EMEmad Mostaque7:07

       Have you smoked? Have you done this? It, it's stupid, right? We're all treated the same. Um, uh, I think as, like, 10% of people have a cytochrome P450 mutation in their liver, which means they metabolize drugs fast, quicker. So if you metabolize codeine, it turns into morphine or fentanyl, kills you. But that's a very basic genetic test, yet we give everyone 500 milligrams of the same thing. With my son, a microdose of five milligrams of clonazepam, which is used for anxiety disorder, worked with a neurologist, allows him to sing. The standard dose is 1,000 milligrams, so they can only prescribe it at 1,000. But that is a $6 a year treatment that affects his GABA glutamate balance, but only for his specific type of ASD, which is only 7% of all kids with ASD. But why would that be in a pharmaceutical company's interest? You know? Because how are they gonna make money off a $6 a year treatment?

   15. HSHarry Stebbings7:58

       Well, I mean, how many people have ASD?

   16. EMEmad Mostaque8:00

       Uh, it's one in 60.

   17. HSHarry Stebbings8:02

       Okay, so one in 60. Have you got a million people in the UK?

   18. EMEmad Mostaque8:04

       Yes.

   19. HSHarry Stebbings8:04

       So you've got a $6 million ... Yeah, it's not great. (laughs)

   20. EMEmad Mostaque8:09

       Exactly. It's not great.

   21. HSHarry Stebbings8:10

       Yeah.

   22. EMEmad Mostaque8:10

       I mean, it's like we know the benefits of vitamin D, right? But we still don't prescribe that at scale and so many people are deficient.

   23. HSHarry Stebbings8:16

       I, I, I mean-

   24. EMEmad Mostaque8:16

       So all these things.

   25. HSHarry Stebbings8:17

       The joys of doing what I do is going off schedule. Final one, and then we will kind of retain some form of normality of schedule. What does the future of healthcare systems look like, do you think, with GPT models operating in this way?

   26. EMEmad Mostaque8:26

       I think that you can change the nature of a doctor because a lot of the stuff is kind of very basic. I think, you know, you had Babylon Health and others trying that chatbot, it wasn't ready. Now you've got this. But everyone should have their own AIs looking out for their own health with that objective function. You know? And then the nature of a doctor becomes different in terms of they have more rich information about an individual while it being preserved in a private manner. I think what you have is you have things like processes and procedures improving, like, uh, wound care, for example, in the NHS. Um, if you are injured as an elderly person, and your wounds aren't treated properly, you're more likely to die by a factor of eight times. Being able to monitor those types of things with this information set means you're eight times less likely, and then you have far more efficiency around that. So the information density on healthcare improves, which means that then our own healthcare improves. We all have access to as much knowledge as we want to within our own context, and so do our providers and the people that help us.

   27. HSHarry Stebbings9:18

       How do we think about open-source versus closed-source human healthcare data? 'Cause, like, obviously for us all to benefit as one, you know, MS sufferers around the world need to submit their data around, you know, responses to certain treatments.

   28. EMEmad Mostaque9:29

       Yeah. So I think the w- wonderful thing about these models is they're few-shot learners, so they don't need to have much information.

   29. HSHarry Stebbings9:34

       Yeah.

   30. EMEmad Mostaque9:34

       And so it isn't the classical big data problem. HDR UK has been one of the pioneers here with the UK Biobank, federated learning and others. And there are kind of, um, with FL7, HLIR and other standards being built around this to allow for full federated learning. If you have open-source language models that are fully auditable, understand, like, on the organic free-range models, the ones we're building, with no web-scraped data, those can sit on-device. Like Google yesterday announced, um, PaLM 2.
4. 10:34 – 12:50

   Google’s AI

   1. HSHarry Stebbings10:34

      you impressed by the Google event yesterday?

   2. EMEmad Mostaque10:35

      No, I think it was impressive. I p- I said in February, you know, like, when all this thing was going on, like, come on, Google will be one of the main winners here. They have the LLMs, they have the hardware-

   3. HSHarry Stebbings10:45

      You do know you're, you're the only person who's said that on the show. And I've asked many, and they've all said that Google are the laggards.

   4. EMEmad Mostaque10:51

      It just takes a bit of time to move the ship, right? And so they've done massive organizational changes and other things. But I can tell you, TPU is on the most scalable architecture. Like, we have zero failure rate with our TPU language model training, whereas with GPUs, it's like there's an ECC error. Why? A solar flare. Okay. Run failed because the sun is angry with us and stuff like that. So when you've got the full stack and you have all that talent in Google, the question is how do you make it organized, right? And so they had to have a story. Google did something called Project Aristotle where they analyzed what made the best teams versus the worst teams at Google, and it came down to shared narrative and psychological safety. People at Google were scared over the last few years because it became this weird monoculture, but now everyone has a shared narrative of let's build the best language models. And now there's an increased amount of psychological safety, being able to speak to things, the walls being brought down between DeepMind and Brain. And so I think you'll see them continuously improving. But then that does mean if you're a proprietary language model company, how are you gonna compete with that behemoth?

   5. HSHarry Stebbings11:48

      The DeepMind kind of, uh, desegregation or kind of-

   6. EMEmad Mostaque11:51

      Yeah.

   7. HSHarry Stebbings11:51

      ... unification, uh, was supposed to have caused a lot of friction and be it negative press reported. Do you disagree with that?

   8. EMEmad Mostaque11:59

      Of course. It is a lot of kind of replicated jobs. There was kind of Brain and Mind, and now they're kind of brought together, and it's a very different management style and other things. These things are never easy. But this is why, like, you saw PaLM, 540 billion parameters and there-... you had DeepMind with 67 billion parameter chinchilla, chinchilla, which is like, just train more as opposed to more parameters. You look at PaLM 2 as a combination of both. And so it's trained for far more, on far better data, and then that means it's only a fraction of the size. Like, 14 billion parameters is one of the test comparator models, versus the 540 and 67. So you can start to see this fusion of ideas, even if the teams... You cannot integrate two big teams like that instantly.

   9. HSHarry Stebbings12:42

      Shared narrative, psychological safety, two of the biggest contributors. So now running Stability, how do you think about integrating those two?

5. 12:50 – 14:39

   Stability AI

   1. HSHarry Stebbings12:50

   2. EMEmad Mostaque12:50

      So we've got the shared narrative. We're gonna build the foundation to activate humanity's potential, and then the motto is make people happier. But it's been a learning process. A year ago, we were basically a mom and pop shop in some ways.

   3. HSHarry Stebbings12:59

      (laughs)

   4. EMEmad Mostaque13:00

      My wife and I were working at it. Like, had lots of meetings out of our, like, sitting room and things because the office didn't have wifi, and all sorts. Now, it's like growing up. We're 170 people, we're going global, we'll have Stabilities in every country, uh, in the next year, we're going multinational. And that's difficult. So we really try to put in processes in place, but it's not easy. Part of this is, like, we went closed source on a bunch of stuff, like DreamStudio.

   5. HSHarry Stebbings13:23

      Mm.

   6. EMEmad Mostaque13:24

      I'm open-sourcing everything now. From next week, we're gonna build our language models in the open and share what works and what doesn't work.

   7. HSHarry Stebbings13:30

      Why?

   8. EMEmad Mostaque13:31

      Because I think this is part of the shared narrative. Someone needs to be open and show what's going on under the hood. And again, it's like, it should be open by default, um, because the value is not in any proprietary models or data. We're gonna build open models that are auditable. Even if it has licensed data in it, you can see every single piece. Free range organic models, because that's what the world needs for all the private, regulated, and other data in the world. There's a completely different TAM to proprietary models, 'cause you can only send so much of your 20VC data to OpenAI, and I think you need both of those. So-

   9. HSHarry Stebbings14:04

      So, why, why can I only send so much?

   10. EMEmad Mostaque14:06

       Because you are a regulated company, and so you need to make sure they're completely compliant.

   11. HSHarry Stebbings14:10

       Yep.

   12. EMEmad Mostaque14:11

       If you have an option of having a Stable Chat model, uh, which will be announced in the future, um, that you own completely, trained in your own cloud or on prem or on device, and then also using GPT-4, that's the best of both worlds, because then you don't have to deal with that. Healthcare data needs to, again, be owned by the individual, and so those models need to be owned, and they need to be transparent. They can't be black boxes. Governments will not run on black boxes.

   13. HSHarry Stebbings14:35

       I, I, I... We're gonna get into this later. I do want to touch on something that... We had a great chat before

6. 14:39 – 17:16

   Why the AI Bubble Will Be Bigger Than the Dot-Com Bubble

   1. HSHarry Stebbings14:39

      this, and you said a, a brilliant quote, and I want to get it right. But you said that, "You know, .AI bubble is bigger than ever, and it will be the biggest shit-show."

   2. EMEmad Mostaque14:46

      Yeah.

   3. HSHarry Stebbings14:47

      Qu- end quote. (laughs) Which I actually took and tweeted, by the way.

   4. EMEmad Mostaque14:50

      (laughs)

   5. HSHarry Stebbings14:51

      Thank you. There'll be some great

   6. NANarrator14:51

      (laughs)

   7. EMEmad Mostaque14:52

      I didn't know if you saw it. I thought if you saw it afterwards, you were like, "Oh, this guy took my tweet." Oh, good.

   8. HSHarry Stebbings14:56

      Thank you. Um, what did you mean-

   9. EMEmad Mostaque14:56

      Share knowledge openly.

   10. HSHarry Stebbings14:57

       What did you... Yeah, exactly. What did you mean by, like, the biggest bubble ever and the biggest shit-show?

   11. EMEmad Mostaque15:02

       Oh, I mean, like, the dotcom bubble, we've seen all these bubbles happen. You know, you had hundreds of billions into Web3 and then developers got paid millions. Already, the, uh, there are certain Chinese companies paying $1.2 million salaries for PhDs. Uh, it's already getting a bit insane. There are remnants of that. The amount of money relative to the amount of opportunity within the sector is just completely misaligned. Like, my TAM analysis is that 1,000 companies will spend 10 million in the next year, 100 companies will spend 100, and 10 companies will spend a billion. Like, PWC just announcing they'll spend a billion over the next three years, and that's a accountancy firm, you know? Where's that gonna go? They don't know (laughs) , nobody knows. And so a multiple of that will be allocated to this as the only growth theme in the entire market against a backdrop of rising rates, real estate crashing, et cetera. So the amount of capacity versus the amount and whale and wall of money-

   12. HSHarry Stebbings15:55

       Yep.

   13. EMEmad Mostaque15:56

       ... into something that's growing faster than anything we've ever seen-

   14. HSHarry Stebbings15:59

       Sure.

   15. EMEmad Mostaque15:59

       ... is completely mismatched. And what will that cause? Like, already you're seeing GitHub Stars leading $200 million funding rounds, with zero traction and zero business model. Like, Stability, we actually have a business model, and it's a good business model, because I designed it. Um, but other things, like, money will go everywhere, and any expertise will get bid up.

   16. HSHarry Stebbings16:17

       Is bad or good for the space? Because it means that projects will get funded that maybe wouldn't have done but are exploratory, generally in over-funding.

   17. EMEmad Mostaque16:23

       I think it's... Starts good for the space, but then it gets bad for the space. Because you see the raccoons and shysters start to come in here. You start to see, like, malformed things where there's a race dynamic, where everyone's trying to build their own models and doing all sorts and massive economic waste. And you see a distraction from what we need to do now, which is this chaos, so we need to standardize some things. We need to feed these models better data and other stuff, and that's why we're moving so hard at Stability. There should be no more web scrape data in the air. There should be national datasets that are good quality to feed these free-range organic models and national and proprietary models and others. And so that's why one of the reasons I signed that letter, because I think there's a six-month pause to get all of our shit together-

   18. HSHarry Stebbings17:04

       (laughs)

   19. EMEmad Mostaque17:04

       ... before things go completely insane, and next year this is everywhere and everyone's investing in everything and it's just absolute chaos.

   20. HSHarry Stebbings17:13

       Uh, you've kind of unpacked so much for me there, I wanted to kind of

7. 17:16 – 19:11

   National Data Sets

   1. HSHarry Stebbings17:16

      go one by one. You said about, kind of, national datasets. Why national datasets versus supranational datasets?

   2. EMEmad Mostaque17:22

      'Cause, like, I'll give you an example. There was a team that did Japan diffusion, including some of our staff. Uh, so we took Stable Diffusion and then changed the language model, 'cause when you typed in "salary man" in Stable Diffusion, it was a very happy man, whereas in Japan, a salary man's a very sad man.

   3. HSHarry Stebbings17:34

      (laughs)

   4. EMEmad Mostaque17:34

      You know? Um, local context is important in these models, because we're gonna outsource more and more of our thinking and minds to it. And so do we want to have a British model or do we want all the models to be Palo Alto? You know, like, a, a sparkling wine has to be from the Champagne region. Like-

   5. HSHarry Stebbings17:49

      Uh-huh.

   6. EMEmad Mostaque17:49

      ... is the only real foundation model AI from Palo Alto? Like... (laughs)

   7. HSHarry Stebbings17:52

      This is not a good thing. We need national models. (laughs)

   8. EMEmad Mostaque17:54

      We... Is this a national infrastructure? 'Cause there is no doubt this is more important than 5G. These models are like really talented grads that occasionally go off their meds, you know? And you want to have the ones from Oxford, Imperial and Edinburgh, as well as the ones from Stanford because they understand the local context, and so they understand you better and they'll be better for that. As part of that, every nation will need their own datasets, which again, have from broadcaster data. They will need their own open models that can stimulate innovation internally as well.

   9. HSHarry Stebbings18:21

      Who owns national datasets? Is that governments?

   10. EMEmad Mostaque18:23

       I think it should be the people. I think it should be open and public domain.

   11. HSHarry Stebbings18:28

       How does that come into fruition? How do we make a world where we have national verified data sets which can be leveraged by independent private companies?

   12. EMEmad Mostaque18:36

       And others, and, uh, universities and others. Well, this is what we're doing right now. We're working with our multinational partners, lots more to be announced soon, and multiple governments for a framework for what good data looks like to feed these models, to stimulate innovation and localization. And that is a public good because national broadcasters have all of this data. You just tokenize all their kind of things. And then you have things like the implementation of these for education and healthcare. You can take generalized learnings and then again feed the models that thing. What does a great data set for a great BritGPT look like? I think it's open, it's interrogated, and it's optimized.

   13. HSHarry Stebbings19:10

       When

8. 19:11 – 20:56

   Which AI companies should VCs invest in?

   1. HSHarry Stebbings19:11

      you look at all the different things that we've talked about from you, relative, like, treatment of MS to, you know, ASD, um, and then, you know, uh, how, its impact on education, and when you j- uh, to PwC spending money on it, there are so many problems that can be solved. Surely we can find a home for the cash.

   2. EMEmad Mostaque19:29

      Yeah. And so, but I'm not sure where, and so this is the thing, like, there's gonna be this mismatch of-

   3. HSHarry Stebbings19:34

      If you were an investor today, wha- h- how, if you were me-

   4. EMEmad Mostaque19:37

      Mm-hmm.

   5. HSHarry Stebbings19:37

      ... okay, what would you do? I'm an early stage investor, I invest globally. What would you s- what would you do?

   6. EMEmad Mostaque19:43

      I would, again, I think it comes down to there's gonna be this tailwind of beta, and then you have an alpha play on top of that, right? So the beta play is that you just invest in any good founder.

   7. HSHarry Stebbings19:54

      Yeah.

   8. EMEmad Mostaque19:54

      And kind of you get in, you figure out, what can I offer as kind of a value add there? Am I offering distribution, am I offering people, am I offering this? And you emphasize kind of your value set. I think right now what people need is people. There are a few people, like, coming out here, but then what you see is you see good companies, with, like, good ideas, but not businesses. They're building surface level things, these wrapper layers and others.

   9. HSHarry Stebbings20:17

      Mm-hmm.

   10. EMEmad Mostaque20:17

       And they're not thinking about distribution and data. It's like if you want to have distribution, what do we do? We went to Amazon and said, "Bedrock," because then it gives us 100,000 SageMaker SMEs, and we'd have to give them the models that they can then take to the private data and we get a share of all of that. This is how we saw it, like, rather than being responsible for that. So if you can bring that distribution to that, then it's important. This is part of that Google memo that went out.

   11. HSHarry Stebbings20:39

       Yeah.

   12. EMEmad Mostaque20:39

       We don't have an edge now there's OpenAI. OpenAI used Microsoft for distribution and that flywheel. You know? If you have a business that's focused on innovation at the core, that's not actually a business. It becomes a business when that innovation becomes product, becomes distribution, when it has an advantage around data and other things. Those are real

9. 20:56 – 22:15

   Microsoft & OpenAI’s Partnership

   1. EMEmad Mostaque20:56

      moats.

   2. HSHarry Stebbings20:56

      How did you analyze that partnership between OpenAI and Microsoft?

   3. EMEmad Mostaque21:00

      Um, I saw it as the objective function of OpenAI is to build AGI, and they reckon they need $10 billion to do it, and they did that. Like, they're building a business on product and things, but they don't care. You know? They're not trying to build a sustainable business, they're trying to build an AGI.

   4. HSHarry Stebbings21:15

      Why? Like, what would I... J- just help me understand, AGI to build a sustainable business? Because I, at the end of the day-

   5. EMEmad Mostaque21:20

      No. They're building an AGI to turn the world into utopia. It's written in their path to AGI thing that they think this can basically bring about utopia. So a lot of people in these labs, and when you have people joining from all of these labs, like, they are almost zealous in their-

   6. HSHarry Stebbings21:35

      But there is a misalignment there between them and Microsoft in their desire to create that utopia, no?

   7. EMEmad Mostaque21:39

      Yes, because Microsoft is a business, you know?

   8. HSHarry Stebbings21:41

      Yeah.

   9. EMEmad Mostaque21:41

      And so this is why you've seen, like, articles in The Information, like, "Microsoft say OpenAI aren't compliant," and "OpenAI say Microsoft on this." These things happen when there is a misalignment of objective functions. Um, but again, you should view OpenAI as what they want to do is build an AI that can basically make the world better and hopefully not kill us all, which they say it might.

   10. HSHarry Stebbings22:00

       (laughs)

   11. EMEmad Mostaque22:01

       Which is a bit concerning, which is why I hope they have better open governance.

   12. HSHarry Stebbings22:04

       How did you think about distribution? You know, you've seen, you know, Hugging Face partner with, um, Amazon, you've seen obviously OpenAI with Microsoft. When you think about distribution and your competitive edge there, where did

10. 22:15 – 24:31

    Stability AI’s Business Model

    1. HSHarry Stebbings22:15

       you land?

    2. EMEmad Mostaque22:15

       So my business model is actually very simple. I haven't really talked about it much. Um, Stimulate opened, we're one of the biggest providers of, uh, grants to open source software, tens of millions already. And then take the best of open, which hopefully we build ourselves, and then an open base with an open data, and then commercial variants with licensed data, and then national variants. So you have Hindi insurance adjusted StableChat, or Indonesian pharmaceutical worker StableChat, that's available in every cloud, on prem, on device, with licensing fees, royalties, and revenue share. And the system integrators work with us as well. Lots of announcement to come. And so by standardizing and stabilizing all the complexity to these very sophisticated building blocks, these very intentionally built models, that really helps the world integrate this stuff, by building playbooks and other things. And that's the core business, because it doesn't require actual innovation. We are still innovative, and the leaders in media in particular. Instead, it requires data and distribution. Data to the models, the models that open and interpretable, and models to the data via our partners. And that's valuable, because the private data in the world is far more valuable than the data that you will send to proprietary models. And it's not a race to the bottom either. So that's what we are, we're a modeling agency with hot GPUs, um-

    3. HSHarry Stebbings23:31

       (laughs)

    4. EMEmad Mostaque23:31

       ... and building a distribution around the world, realizing that India and other nations will leapfrog to intelligence augmentation just as they leapfrogged to mobile. They will embrace this technology far quicker than we will in the UK even.

    5. HSHarry Stebbings23:42

       Why?

    6. EMEmad Mostaque23:43

       Because they have to. India, all of the outsourcing jobs in programming will go because GPT-4 can go level three Google programmer exam and pass it. Outsourced jobs will go the first, whereas in France you're never gonna fire a French person, so those jobs are safe, you know?

    7. HSHarry Stebbings24:02

       (laughs)

    8. EMEmad Mostaque24:02

       And so they have an objective function where they need to embrace this technology. In Africa, one-to-one tuition, every kid in Malawi is on things we lined up. We've got other nations. We're gonna bring them all this technology and tablets, and guess what? Their lives will transform. One AI per child is one I want to call it. We'll call it something else. But think about the potential of that, because you have one teacher per 300 kids. What if they had a ChatGPT level AI? The ROI is high and the need is high, and so they will embrace it far quicker than we will.

11. 24:31 – 25:36

    AI’s Impact on Developing Countries

    1. EMEmad Mostaque24:31

    2. HSHarry Stebbings24:31

       What happens to countries that rely on outsourced work in those kind of freelancer economy jobs?

    3. EMEmad Mostaque24:36

       ... in general, one of the things, the questions is, you've seen the OpenAI study, you've seen the, which said tasks will be replaced up to 44%.

    4. HSHarry Stebbings24:42

       Yeah.

    5. EMEmad Mostaque24:43

       You've seen Goldman Sachs say adds percentage points to GDP. I think the only solution to this is entrepreneurship. And so we need to give the tools to create new jobs that can replace some of these old jobs being done. So like to the various Asian governments, I'm saying adopt the UK policy of these sandboxes, financial AI and other regulatory sandboxes. So you can take these technologies, these national models that we will help you build with our consortium partners, and then spur innovation to create the jobs to replace the existing jobs because you'll upgrade your entire society. Bring these models into your governments and other things to go from slow, dumb AIs, which is the national organization's healthcare, to intelligent, dynamic ones.

    6. HSHarry Stebbings25:24

       Can I ask, on implementation, and when we think about kind of bluntly seeing this in action in society, I'm sure we're just very aware of technology cycles taking so much longer than one anticipates. How do you think about that

12. 25:36 – 29:33

    Enterprise vs Consumer Adoption

    1. HSHarry Stebbings25:36

       in actual... Uh, there's kind of twofold, one is adoption on enterprise and another is adoption on consumer.

    2. EMEmad Mostaque25:41

       Mm-hmm.

    3. HSHarry Stebbings25:41

       Say if we do the adoption on the consumer side, which is impacting freelancer jobs and-

    4. EMEmad Mostaque25:45

       Yeah.

    5. HSHarry Stebbings25:45

       ... impacting education, what do you think that looks like?

    6. EMEmad Mostaque25:48

       So I think on the consumer side, um, you're freer with your information so you can use a lot of these things and the APIs of OpenAI and Cohere and others are fantastic, right? And Google PaLM now kind of being out there. So it will be integrated to deliver better consumer experiences without it being creepy like you've seen with some of the chat bots, et cetera. 'Cause it's going into Word, it's going into Workspace, you know? Like, it helps already. Like, we will have a conversation, it'll be automatically logged by our Pixel phone and then we'll get a transcript and remove bits that we don't want to share, then it goes into a global knowledge base that reminds us of things.

    7. HSHarry Stebbings26:18

       Sure.

    8. EMEmad Mostaque26:18

       That's inevitable. On enterprise it takes longer because you need to have auditable, standardized models. If you're a financial services institute, you can't have a single piece of cruel data in there. And so that's what we're deliberately building with the largest companies in the world, 'cause we're building dedicated teams for them.

    9. HSHarry Stebbings26:32

       Wait, so you can't have a single piece of cruel data if you're a financial services company?

    10. EMEmad Mostaque26:35

        Yes, because the danger is, if it has some Reddit in there... So StableLLM, you know, we'll put out next week, it wasn't as good as the other models because we're gonna make a point about Reddit data being bad. It's not about more data, it's about better data. We had DataComp, which is the next generation LAION that we funded the compute for, whereby, uh, a quarter of the parameters of OpenAI's clip, it outperforms because of better data quality. Rubbish in, rubbish out.

    11. HSHarry Stebbings26:56

        What makes, what makes good data quality? Sorry.

    12. EMEmad Mostaque26:58

        That's something we're exploring right now. But from the investment banks we've talked to, asset managers, and we're building dedicated teams for the largest ones in the world to build them proprietary models, the feedback we've got is we cannot use a black box. We need to know what data is in there. Just, the regulators are asking us. We don't want to have this out of sample thing where it's seen something on Reddit and then it says something rude to our end users, you know?

    13. HSHarry Stebbings27:20

        Why is Reddit data bad?

    14. EMEmad Mostaque27:22

        Reddit data isn't bad in itself, it was just a case of more data is not always good. So right now, we are using all these web scrapes and we're training our models by taping their eyes open, and then it took six months to turn GPT-4 into ChatGPT-4, because we had to tune it and, like, give it a haircut and stuff and bring it back to society. The point is that we need to find the right type of data, 'cause rubbish in, rubbish out is something that we've heard a lot, you know? And so it's not bad in itself, but if you just scrape it without the proper cleaning, it is bad. Because what is it? It's like, you know, people just kvetching a lot, you know, people being biased. Do you really wanna feed your kid the whole of Reddit? You know? Do, would you want to have the best curriculum possible? And that's actually one of the ways the models learn. Like stable diffusion, we trained it on the whole internet and then better and better image subsets of it. And that's the same thing with large models called curriculum learning. Train it on a big base that's solid and then did it. It sounds familiar, doesn't it?

    15. HSHarry Stebbings28:19

        (laughs) The hardest-

    16. EMEmad Mostaque28:19

        In the garden, you know?

    17. HSHarry Stebbings28:19

        The hardest thing is that how do you instill values and political correctness in models?

    18. EMEmad Mostaque28:25

        There is no such thing as an unbiased model. So DALLE-2, when OpenAI had that and they introduced a bias filter, any non-gendered word they'd add a random gender (laughs) and a random ethnicity. So you typed in sumo wrestler and you would get Indian female sumo wrestler.

    19. HSHarry Stebbings28:40

        (laughs)

    20. EMEmad Mostaque28:41

        That was kind of a good picture, I've got it saved somewhere.

    21. HSHarry Stebbings28:43

        (laughs)

    22. EMEmad Mostaque28:43

        I think this is why you need national data sets, you need cultural data sets, you need personal data sets that can interact with these base models and customize to you and your stories, 'cause you and I both have our stories that make up our psyche.

    23. HSHarry Stebbings28:56

        Sure.

    24. EMEmad Mostaque28:56

        And understand that context is so important to have AIs that can work for us, not on us.

    25. HSHarry Stebbings29:02

        And so it's essentially like a next generation cookie that personalizes our data to allow for better search experience.

    26. EMEmad Mostaque29:07

        A mega cookie. And if you standardize the base foundation models and they call it the hypercube, every modality, because we do all the modalities, all the sectors and all the nationalities, then you don't need to have a million different models like those dream booths of the avatars. You said you have a base model that you then have a vector embedding around, 'cause these models contain all the principles and the embeddings point to the important bits that make up Harry or Emad. And then you can search those and adapt those rather than having a million, billion different models, which is just confusing.

13. 29:33 – 34:18

    How AI Kills Traditional Media

    1. EMEmad Mostaque29:33

    2. HSHarry Stebbings29:33

       So I had dinner the other day with, um, one of the largest kind of media publication owners in the world. Um, and he, he said, "I'm worried, Harry. I don't think that I will have a business in a couple of years. I think, bluntly, we're getting killed on our advertising because everything's getting scraped, uh, and they're not coming to our websites, and that's where we get paid. We get paid for clicks." Uh, is he right to be worried?

    3. EMEmad Mostaque29:55

       I think he is right to be worried. Like again, you look at Google's announcements yesterday, talking about this day after PaLM 2, you suddenly look at the new Google page where they've got the language model, and it's just text. And where are the clicks? It was like when Google introduced AMP, you know? This is where rather than looking at the New York Times page, you have this formatted thing with no New Yorks Times kind of stuff there. Like, these search entities that aggregate are just sort of intermediating more and more, and people are gonna become used to just having synthesized input. So what does search look like? What does it look like when your GPT-4 can write you an article about any news that's happening in a way that's customized to you and your context and everything like that?This is massively disruptive for media and information. And so they have to think, "Where am I in the future where..." Again, the best way to think about the impact of this is they're re-talented grads that occasionally go off their meds, and we push a button, we can get a thousand of them. Those grads include journalists. And you can have your own journalist army.

    4. HSHarry Stebbings30:53

       Huh.

    5. EMEmad Mostaque30:53

       Your own writer army, your own coder army, your own designer army.

    6. HSHarry Stebbings30:57

       So the pushback against that is-

    7. EMEmad Mostaque30:58

       Mm-hmm.

    8. HSHarry Stebbings30:59

       ... libel. Can you say? Good fucking luck. We spend our c- like, life in lawsuits. Libel is real. You are gonna get unbelievable amounts of libel cases. And then OpenAI will be fucked.

    9. EMEmad Mostaque31:11

       Mm.

    10. HSHarry Stebbings31:11

        You cannot have a thousand, uh, libel cases a day.

    11. EMEmad Mostaque31:14

        Mm-hmm. Well, this is the thing. If you say this is... needs to be checked and cross-checked, that's one thing. But a lot of the media companies say, "We're the source of authority."

    12. HSHarry Stebbings31:21

        Mm-hmm.

    13. EMEmad Mostaque31:21

        So a way that media companies can shift is by having, in a deepfake and other age where everything can be generated, we make sure this is real news. We are very thorough in the way we do it because-

    14. HSHarry Stebbings31:33

        So this is interesting. So you place a premium on authority.

    15. EMEmad Mostaque31:36

        Premium authority. This is why you've got the, uh, check marks coming out at Twitter and the organizational thousand pounds a month, and Facebook doing the same, because you need to have a level of authenticity, a level of authority. But again, is the news fair and balanced? I've had lots of hit pieces coming out against me. I've got a lot more. It's not because they have angles, you know? And so what is the bias of the New York Times versus this versus that versus others? How do people consume news now? And even news consumption has gone down dramatically, right?

    16. HSHarry Stebbings32:03

        Yeah.

    17. EMEmad Mostaque32:03

        'Cause people consume news through their social networks, through their groups and other things. So you have to say, "What is the model?"

    18. HSHarry Stebbings32:10

        But the hard part is, you know, none of the next generation models or AI providers want to be content publishers. So how do you fit in a world where, you know, you'd be killing their business model on the content side, but they don't want to be publishers.

    19. EMEmad Mostaque32:24

        You will have AI-first publishers. So if you remember kind of Vox and these things when they kind of kicked off, they wanted-

    20. HSHarry Stebbings32:30

        Yeah.

    21. EMEmad Mostaque32:30

        ... to be generat-... They wanted to be, uh, technology first. You're gonna have a new wave of AI-first publishers that aren't just AI, but it's AI plus humans. 'Cause AI plus-

    22. HSHarry Stebbings32:40

        What does that look like? Sorry, AI plus humans. Like journalists using-

    23. EMEmad Mostaque32:42

        AI plus humans means that you have information coming in, and then the stories or drafts are automatically written, reviewed by humans, who then give their input to train it better. This is kind of the feedback flow. And then what happens is it comes out, and there's the factual anchors, and then it gets customized to Alabama, and the Alabama context and all sorts of other things. 'Cause you could tell it, TLDR, too lazy didn't read-

    24. HSHarry Stebbings33:07

        Mm-hmm.

    25. EMEmad Mostaque33:07

        ... explain it like I'm five, make it more complex. And so you're gonna see something very interesting here, which is the right news at the right time. The localization will return, but again, through AI first. I think this is the thing. We're seeing AI integrated, but the next wave is going to be, once we understand design patterns, AI first everything in information flows, once these technologies are a bit more mature.

    26. HSHarry Stebbings33:29

        Can you just help me understand? AI integrated versus AI first. What does, what does that mean?

    27. EMEmad Mostaque33:32

        AI integrated means that I have a existing newsroom, and I bring in AI to write faster drafts and things like that.

    28. HSHarry Stebbings33:37

        Uh-huh.

    29. EMEmad Mostaque33:37

        AI first is saying, "I have an army of things I can spin up instantly that can help me achieve these certain things to create news that is valuable for this reason, with this feedback loop." And so you build the system kind of from the start, thinking AI at the core versus AI being integrated in to improve existing systems. 'Cause so much of news is what? We find information, we have drafting, we have this, we have that, we do these checks. A lot of that can be simplified. Just like we moved from the analog to the digital age, to the internet age, the next stage is the AI age.

    30. HSHarry Stebbings34:10

        So I, I have to ask. Like, when we think about kind of AI-first publishers-
14. 34:18 – 38:38

    The Criticism of Most AI Companies: Thin Application Layers

    1. HSHarry Stebbings34:18

       You know, I've met honestly 50, maybe more, AI companies in the last, you know, month, and the, the feedback is always the same. They're not operating off a defensible moat of data. They're literally a thin application layer on top of an existing model is 99% of the feedback.

    2. EMEmad Mostaque34:36

       So you look at someone like Harvey, for example.

    3. HSHarry Stebbings34:38

       Yeah.

    4. EMEmad Mostaque34:38

       They went to law firms and they said, "You are our distribution, and we're gonna integrate and improve your system and build our system for your system." So I think a lot of these people are trying to build it, and they will come, and they're trying to get in there as opposed to just re-targeting. Where can you go in and transform? Because-

    5. HSHarry Stebbings34:51

       Is that the wrong model Harvey did?

    6. EMEmad Mostaque34:53

       No, I think it's the right model. I think that a lot of organizations are elastic and plastic now. So you can go in and give them an integrated thing saying, "You will be my test case. I will help you upgrade as a skunkworks lab and build a system alongside your system," as it were.

    7. HSHarry Stebbings35:05

       And sorry, and you think enterprises will say, "Sure, take my data."

    8. EMEmad Mostaque35:08

       I think now, I think now they will if you can keep their data inside internally. And I think, again, with better open models, you can enable that. So people can build on top of open models. There are dedicated instances on Cohere and others as well. And so the tooling is now catching up so that you can have a new generation of startups where their first customers are massive companies they would never get otherwise. Um, I think this is the thing, 'cause every big company is looking for an answer. If you can give that answer, that contract that would've taken you a year, you can get in a week.

    9. HSHarry Stebbings35:37

       Do you think so?

    10. EMEmad Mostaque35:37

        Yeah.

    11. HSHarry Stebbings35:37

        'Cause you still gotta get in the door.

    12. EMEmad Mostaque35:39

        You gotta get in the door. And that's hustle, man. Um, so again, this is what-

    13. HSHarry Stebbings35:42

        That's fucking true. (laughs)

    14. EMEmad Mostaque35:43

        This is what the Harvey guys kinda did. This is why I went straight to the hyperscalers and I said, "You need to have standardized models for open, for regulated data."

    15. HSHarry Stebbings35:49

        What did they say to you?

    16. EMEmad Mostaque35:50

        Uh, they said, "Really?" They're like, "Can you build them?" Like, "Here's some models that we built." And they're like, "Oh." And then I told them exactly how the things would be last summer to now, and it's followed that, and I've kept in touch, and I've improved it. And this is why I'm building dedicated teams for the largest companies in the world. I'm not telling them-

    17. HSHarry Stebbings36:05

        Who-

    18. EMEmad Mostaque36:05

        ... I'm trying to sell you anything. I'm like, "Over the next year, I'm gonna help make sure you do not get blindsided." And I could try and sell you models, and people are offering us tens of millions per model. But I'm like, "I'm gonna build a proper partnership with you. And then that means I'll have a LTV from you."

    19. HSHarry Stebbings36:19

        What does that proper partnership mean? And who's that with? That's with IBM, that's with SAP, that's with Audi, Volvo.

    20. EMEmad Mostaque36:24

        So we've announced Amazon. Let's say we have lots of other announced with the biggest companies in the world, where they have amazing teams, but they can only move so fast.

    21. HSHarry Stebbings36:32

        Huh.

    22. EMEmad Mostaque36:32

        And I'm building the dedicated teams to help them move and understand the whole sector without trying to, like, sell them on services.

    23. HSHarry Stebbings36:38

        Mm-hmm.

    24. EMEmad Mostaque36:39

        ... I'm trying to say, I will build you a customized model if you want, but I'm only doing that with a dozen companies, you know, so I can kind of focus down. And I will tell you that GPT-4 is great, or Cohere is great, or all this stuff. All the latest research through the communities we support, I will make sure you're on top of, relevant to your sector, and you'll got dedicated people helping you in this transition period.

    25. HSHarry Stebbings36:57

        Is that aligned to your core model? This seems like an ancillary product, it is like a SAP consulting services.

    26. EMEmad Mostaque37:04

        It is kind of like that because I need to understand these sectors better. What does the hypercube look like? What does the insurance-adjusted GPT look like, you know, as a fundamental basis? And so a lot of people, like-

    27. HSHarry Stebbings37:15

        And you're able to extract that data and then take it with you and do the learning on it?

    28. EMEmad Mostaque37:19

        Yes. And so this is part of the thing, that we will have a generalized model and we're very clear, but then you can have a specified model just for you as well, as long as it doesn't interfere with that. So finding that balance will be interesting, but the reality is, no models you- that are out today will be used in a year.

    29. HSHarry Stebbings37:33

        Unpack that for me. I'm- this is mind-blowing.

    30. EMEmad Mostaque37:35

        So again, you see the order of magnitude improvement. PaLM last year was 540 billion parameters, then Chinchilla 67, and now 14. 540 to 14 is a big step, you know? You see the quality of GPT-3 versus GPT-4.
15. 38:38 – 40:21

    AI Doomers

    1. HSHarry Stebbings38:38

       not concern, I mean, I hate the doomsday says and I'm excited for the future, I'm terrified for the future too. But everyone always says, like, technology revolutions, you know, industrial age, whether it's the introduction of PCs into enterprises. These were, you know, industrial was a 30-year plus. Actually, PCs into enterprises was 10 years plus. The challenging thing is like, the learning curve to use ChatGPT as a marketer is nothing. I mean, it is easy. And so, and the integrations is- is a day.

    2. EMEmad Mostaque39:03

       It's- it's because, yeah, like you want to write an API, it's not a day, you just give it the manifest spec and it automatically generates it. It would have taken days before. (laughs)

    3. HSHarry Stebbings39:10

       Sure.

    4. EMEmad Mostaque39:11

       Like it's an amazing experience because-

    5. HSHarry Stebbings39:12

       And so the transition is so much more compressed today.

    6. EMEmad Mostaque39:15

       It came from the existing system and so it goes seamlessly into the existing system, versus like Web3 that tried to create a system outside the existing system. And all the money was made and lost at the interfaces. Again, it's like deploying grads at scale, like with a 32,000 token context when you do GPT-4, 20,000 words of instructions. What does that do to SaaS? So my thing is that we're still in this crazy period. Next year it will settle and then it will go ubiquitous, where a lot of companies know they need to do something but they don't know what they need to do.

    7. HSHarry Stebbings39:42

       Are they adopting it now?

    8. EMEmad Mostaque39:43

       They're doing the POC thing, like some like Microsoft and others for consumer, they're adopting it. Consumer adoption is a much lower bar.

    9. HSHarry Stebbings39:49

       Yeah.

    10. EMEmad Mostaque39:50

        When this starts going in enterprise, it's going to be a fricking train because so much of enterprise is about services and information flow. And again, if you push a button and have a thousand of these things, that's a huge difference, right? And so, like, I think this will be an e- bigger economic impact than COVID. I don't know in which direction. I hope it'll be positive. But again, giving that example of an India or one of these outsourced places, you lose BPO jobs, you can make it up on entrepreneurship, if you embrace the technology.

    11. HSHarry Stebbings40:20

        What do

16. 40:21 – 43:06

    AI Business Models

    1. HSHarry Stebbings40:21

       you think the business model of the future is for those models moving into enterprise?

    2. EMEmad Mostaque40:25

       I think it's the same as always. You've got good products, good distribution. You know, you lock it in. Like 1.5 million people still use AOL.

    3. HSHarry Stebbings40:32

       (laughs)

    4. EMEmad Mostaque40:32

       You know, like HCL bought Lotus Notes for 1.5 billion-

    5. HSHarry Stebbings40:35

       That is amazing, I didn't know that.

    6. EMEmad Mostaque40:35

       ... a few years ago. Like 40% of the world still doesn't have internet.

    7. HSHarry Stebbings40:41

       Pfff. Yeah.

    8. EMEmad Mostaque40:43

       Again, we're super privileged where we are, right? And so you look at that, and I look at emerging markets, I'm like, all of finance is securitization and leverage, and securitization is telling a story. The only thing that matters for a stock is the marginal story and how it evolves. What if you have massive information about every child in Africa and every business in India and they embrace this technology properly? Massive financial growth.

    9. HSHarry Stebbings41:07

       When do you think next year for their embracing it?

    10. EMEmad Mostaque41:10

        I think that people are still getting used to all this. We haven't standardized any of the things. We don't know what the design patterns are. I think that what happens is that everyone's doing this at the same time and they're all trying to get to grips with it. And so again, we have this like six month window where everyone's getting to grips with it, and then we standardize, have design patterns, and they spread, and then you start implementing. And then you see some people outpacing others, which means that you have to catch up and then you're forced to implement it. So this is how I see the race dynamics occurring right now.

    11. HSHarry Stebbings41:36

        You say about forced to implement it. I think the truth is they just have no fricking idea.

    12. EMEmad Mostaque41:39

        Right now they don't.

    13. HSHarry Stebbings41:40

        Which I totally understand. I don't blame them for it. But I h- I tweeted actually the other day that I think the biggest AI companies will be services based implementation companies for large enterprises.

    14. EMEmad Mostaque41:49

        100%. That's why I said if you're a startup, the best thing to do is you identify an enterprise that will be transformed by this and you go to them and you say, "I have a solution and I'm going to start with you and I might go bigger, but I'm going to help you through this period by doing this, this, and this, and this." And they will appreciate that and there will be capital available for that in a way that you've never seen before 'cause you know how difficult it is for small companies to sell to big?

    15. HSHarry Stebbings42:12

        Sure.

    16. EMEmad Mostaque42:12

        But the big companies have no idea except for their CEO and their board are telling them. You look at the number of mentions in earnings calls like that, every earnings call next quarter, and then by next year, literally every single one. They're like, "What is our strategy?" It's not like, "What is our Web3 and Metaverse strategy?" It's like, "I need this strategy now." Again, it's like, "What is our COVID strategy?" It'll be that level of urgency within a few quarters.

    17. HSHarry Stebbings42:34

        Would you raise money if you were them? So you go to a corporate and you say, "Hey, you know what? I can solve your problem. This is how it'll work," and they will fund you, they will give you the data? Would you raise money?

    18. EMEmad Mostaque42:44

        ... yeah, I mean like, again, you need the people. The people is the key thing here because you can have the technical chops, you know? You can have an understanding of the industry. But to build a good business and to scale it at the pace that you need to, to keep up with this, is incredibly hard.

    19. HSHarry Stebbings42:57

        Do we have enough talent?

    20. EMEmad Mostaque42:58

        No.

    21. HSHarry Stebbings42:59

        Sure.

    22. EMEmad Mostaque42:59

        And so this is why we support the fast.ai courses, which transform normal developers into ML developers and other things like that.

17. 43:06 – 44:56

    The Future of AI Writing Code

    1. EMEmad Mostaque43:06

       But again, these models are actually not that hard to work with. 50% of all code on GitHub is AI-generated now. So you can even use Copilot to help you code the models (laughs) and other things like that.

    2. HSHarry Stebbings43:15

       What do you think that code generation is in five years?

    3. EMEmad Mostaque43:19

       Why would you need code? Code is just a way to talk to a computer.

    4. HSHarry Stebbings43:23

       Unpack that. What, what does that mean-

    5. EMEmad Mostaque43:24

       So when I started-

    6. HSHarry Stebbings43:25

       ... if you're speaking human language?

    7. EMEmad Mostaque43:26

       When, when I started 21, 22 years ago as a coder, I'm what? 40 now. So I've been doing that one since 18, I was writing assembly lang- assembly code. (laughs) You know, like really (laughs) low-level stuff. There were no libraries, there was no GitHub, there was nothing like this. Like right now, coding's like mixing and matching. It's like building Lego. And AI can build that Lego much better, especially in five years. What you're doing when you're propping, like, programming language is you're telling it to go and do something. Even something like PaLM, like we sponsor a amazing coder called Lucid Raines. If you want to cry as a programmer, you go and look at his GitHub.

    8. HSHarry Stebbings43:58

       (laughs)

    9. EMEmad Mostaque43:58

       Most productive developer in the world. He recreated the whole of PaLM in 206 lines of PyTorch. But again, why would you need a human for that if the AI gets better and better at coding? Just tell it what you want, "I want to create an app for 20-minute VC that has these features." Of course, it will go and build it automatically. Where is the human coder in that?

    10. HSHarry Stebbings44:19

        What does that mean for the future of entrepreneurship? Actually a good thing in terms of the complete democratization, where anyone can build anything.

    11. EMEmad Mostaque44:24

        Anyone can build anything. This is why (laughs) distribution, data, you know, relationships, product become important. Because it already became easier to build anything, right?

    12. HSHarry Stebbings44:34

        Yeah.

    13. EMEmad Mostaque44:34

        But what makes a good product? Again, there are these unchanging things. Have great customer satisfaction, deliver value. People get distracted by technology. Like I was at this CryptoX.AI thing on the weekend, they were talking about decentralized something. I was like, "Guys, just this is all bollocks." Like, it's not about the technology. It's about what you're creating that's valuable to help people. You know? Focus

18. 44:56 – 49:00

    AI Startups vs Incumbents

    1. EMEmad Mostaque44:56

       on that.

    2. HSHarry Stebbings44:56

       Who do you think wins in the next three to five years, startups or incumbents? 'Cause incumbents have the distribution.

    3. EMEmad Mostaque45:02

       Uh, I think it's incumbents, but there's a lot of startups that will be a billion dollars. And even on the thin-layer thing, um, Kayak sold for, uh, sorry, ITA Software sold for 700 million and Kayak sold for two billion.

    4. HSHarry Stebbings45:13

       Sure.

    5. EMEmad Mostaque45:13

       And that was a layer on top of ITA. We've seen many of these examples here, right? Again, we know that value and moats are not necessarily innovation first.

    6. HSHarry Stebbings45:22

       Well, y- yes and no. It's interesting. I had, um, Tom Tungers on the show, and he essentially analyzed, um ... Tom is a very famous ML and AI investor.

    7. EMEmad Mostaque45:29

       Yes.

    8. HSHarry Stebbings45:29

       And he analyzed, uh, infrastructure versus application layer. And both actually were about $2 trillion TAMs. The difference is in the infrastructure layer, there was three companies, and in the application layer, there was 50. And so your average enterprise value was like significantly

    9. EMEmad Mostaque45:45

       I would agree with that. I think that there's only gonna be five or six, uh, foundation model companies in the world in three years, five years.

    10. HSHarry Stebbings45:52

        Do you think they've all been created now?

    11. EMEmad Mostaque45:54

        Yes.

    12. HSHarry Stebbings45:54

        Which are they?

    13. EMEmad Mostaque45:55

        Uh, I think it's gonna be us, NVIDIA, uh, Google, Microsoft, OpenAI, and, uh, Meta and Apple probably are the ones that train these models.

    14. HSHarry Stebbings46:04

        Is Anthropic good?

    15. EMEmad Mostaque46:05

        Anthropic are great. But from a business model perspective, you have Claude on Google API.

    16. HSHarry Stebbings46:11

        Yeah.

    17. EMEmad Mostaque46:12

        And you have PaLM 2. How are they gonna keep up with PaLM 2? You know? If-

    18. HSHarry Stebbings46:19

        I, I can't answer that. (laughs)

    19. EMEmad Mostaque46:20

        Well, Google, th- they can raise billions, but Google will, Google spend $20 billion a year on AI. DeepMind's salary budget is 1.2 billion a year.

    20. HSHarry Stebbings46:28

        Sorry, DeepMind's salary budget is 1.2 billion a year?

    21. EMEmad Mostaque46:31

        Yes. So that's in the public kind of filings. Or was it salary and compu- I think it's salary. They technically make a billion a year from the internal, uh, counter-payments with Google as well.

    22. HSHarry Stebbings46:40

        Wow.

    23. EMEmad Mostaque46:41

        So, but again, I mean, Google, how much money do they have? $150 billion to win this.

    24. HSHarry Stebbings46:46

        Fuck. How much money do you need?

    25. EMEmad Mostaque46:48

        Mm-hmm, I have a business model that is going to be massively profitable very soon.

    26. HSHarry Stebbings46:53

        Because of the national services.

    27. EMEmad Mostaque46:55

        Because of various things. I haven't given the full details. I will over the next few months. Uh, we've got a nice little case study with some universities coming. Uh, I like it to be a surprise.

    28. HSHarry Stebbings47:02

        What's really hard for you?

    29. EMEmad Mostaque47:04

        Talent. Keeping talent together, A+ teams. So we've had zero attrition in our developers, and they're amazing. So we've got video models, audio models, all these things coming out.

    30. HSHarry Stebbings47:12

        Everyone says you need to be in the Valley. You're in London.
19. 49:00 – 50:08

    AI’s Impact on Economic Growth

    1. EMEmad Mostaque49:00

       else.

    2. HSHarry Stebbings49:00

       Can I ask, in terms of, like, short-term economic growth, how do you think about the impact that everything we've just discussed has on rising inflation, rising interest rates, uh, short-term employment rates?

    3. EMEmad Mostaque49:12

       It's massively deflationary. The biggest drivers of CPI inflation in the US were education and healthcare. And that was almost all administratively and burocr- bureaucratic.

    4. HSHarry Stebbings49:22

       Wow.

    5. EMEmad Mostaque49:22

       In the next few years, gets what, guess what gets disrupted? Those. But they don't get disrupted this year or next year, it's the year after, because those ones take a bit longer to come through.

    6. HSHarry Stebbings49:33

       Okay. And so f- uh, what does that, how does that impact the economy there? When we think kind of US, UK, what does that look like in terms of the three-year time period?

    7. EMEmad Mostaque49:41

       I think the UK benefits. Unicorn kingdom is the new kind of thing is, 'cause-

    8. HSHarry Stebbings49:45

       Yeah.

    9. EMEmad Mostaque49:45

       ... it'll get, uh, because we have amazing policies, like, uh, every single AI company should come to the UK because cloud computing is now included in R&D tax credits.

    10. HSHarry Stebbings49:54

        Wow.

    11. EMEmad Mostaque49:54

        It's a 27% rebate on losses in cash. We can now issue scale-up visas, global talent visas, like the DeepFloyd team that released the best image model in the world ever from Stability, they were brought in on tech talent visas. That was turned around

20. 50:08 – 52:57

    AI Regulation Around the World

    1. EMEmad Mostaque50:08

       in one week.

    2. HSHarry Stebbings50:08

       Do you think the UK's done a good job in terms of implementing regulation and policies to bring AI talents?

    3. EMEmad Mostaque50:13

       Had the best, apart from maybe Japan, yes.

    4. HSHarry Stebbings50:15

       Huh, what have Japan done?

    5. EMEmad Mostaque50:16

       Uh, Japan has some very interesting ones around web data scraping and others. But again, Japan is a very different culture, so even if the policy is good, it still doesn't have the same innovative thing.

    6. HSHarry Stebbings50:25

       Who's done the worst?

    7. EMEmad Mostaque50:27

       The worst, I'm not sure actually. No one's done too bad. The new European legislation was really, really bad, now it's got a little bit better, but always Europe wants to be the leader in regulation, which kind of, you know, okay, fair enough.

    8. HSHarry Stebbings50:37

       It's never an easy thing.

    9. EMEmad Mostaque50:38

       It's never an easy thing. And this is the thing, like, I think the UK is in a very good position and the government's forward leaning. I mean, look, the 900 million pound supercomputer, 100 million pound LLM task force that's been equated to the COVID level of s- seriousness, you know, um, these big things-

    10. HSHarry Stebbings50:53

        What do you make of, what do you make of, like, the OpenAI competitors? I've seen quite a few which are OpenAI for Europe, uh, and we've seen three or four now. Like, is this a zero-sum game and OpenAI's won that race, so to speak? Or-

    11. EMEmad Mostaque51:06

        I think it'd be difficult to compete against them 'cause they're executing incredibly well.

    12. HSHarry Stebbings51:09

        Mm-hmm.

    13. EMEmad Mostaque51:09

        And I think, again, why would you use OpenAI for Europe versus PaLM 2, versus GPT-4? What can you bring? But you will have national champions in others. I think it's incredibly difficult to compete in proprietary. Um, I think in open, it's a bit different because of the standardization element there. But again, my play is to be the benchmark across every modality, 'cause there's no other company, apart from me and OpenAI, that does every modality. There's no company that's as aggressive as me in emerging markets. Um, and so they had to say, "What is my edge?" Because you can have an edge, like you can be the OpenAI for government, or defense, or for healthcare, and really get in and understand those, and then you can be sticky, you know?

    14. HSHarry Stebbings51:51

        Yeah.

    15. EMEmad Mostaque51:51

        Like one of the, like Scale is now going fully into defense, you know?

    16. HSHarry Stebbings51:55

        Scale AIs?

    17. EMEmad Mostaque51:56

        Yes. So they've announced their integrations with the Air Force and all sorts of other things.

    18. HSHarry Stebbings52:00

        Mm-hmm.

    19. EMEmad Mostaque52:00

        Like we're getting together on Defcon this year, and people are gonna hack at our models, and OpenAIs and others organized by Scale. What is your edge? You know, what is, again, your moat? What is your business model and where, what are you reliant upon to deliver that value that can increase?

    20. HSHarry Stebbings52:16

        This is why I was surprised when I saw you sign the petition of Elon in terms of pausing for six months. Can you just unpack why you did that?

    21. EMEmad Mostaque52:24

        Well, I mean, for six months, you're not getting H100s and TPUv5s anyway (laughs) so it's a natural pause. But then also because the shit show's coming next year, so I said, "We have to self-regulate." Like, for example, um, the adversaries already have GPT-4. Why? Because you can just download it on a USB stick, you know? You don't have to train your own when you can just steal it. Let's have better opsec. Let's have better standards around data. Let's stop and move off web scrapes by next year. We had hundreds of millions of images opted out of Stable Diffusion 'cause we were the only company in the world to offer opt-out of datasets. You know? Like, let's bring in some standards around this before

21. 52:57 – 55:46

    The Tom Hanks Effect

    1. EMEmad Mostaque52:57

       it's everywhere. Basically where we are now, you remember COVID, your mum is talking about this, and your aunt, and everyone's talking about generative AI, and they're asking you, "Harry, what's going on?" You know? But you haven't had the Tom Hanks moment yet. 'Cause everyone was talking about COVID before Tom Hanks got it, and then when Tom Hanks got it, that's when global policy changed. 'Cause if Tom Hanks can get it, anyone can get it.

    2. HSHarry Stebbings53:17

       (laughs)

    3. EMEmad Mostaque53:18

       That is that moment for generative AI.

    4. HSHarry Stebbings53:20

       What do you think it is?

    5. EMEmad Mostaque53:21

       I don't know, but I know it's coming 'cause I know this technology is definitely everywhere next year and it's disruptive in both directions.

    6. HSHarry Stebbings53:26

       You don't think there's a chance that it takes much longer, three to five years?

    7. EMEmad Mostaque53:29

       No chance. It's so useful right now. And you think about certain industries and how they'll be affected by having the ability to have a thousand GPT-4s working together.

    8. HSHarry Stebbings53:41

       You said, you said, uh, in a tweet actually, I think it was a reply to a tweet, but you said, "Hallucination's a feature, not a bug."

    9. EMEmad Mostaque53:48

       Yeah, so right now people are trying to treat these models, so we trained the whole internet, and, like, Stable Diffusion's 100,000 gigabytes in a two gigabyte file. What on earth is that? It's not compression, it's none of this kind of stuff. It learns principles. Uh, GPT-4 NVIDIA said they built the dual H100 with the NVLink for that, and that's 160 gigabytes of VRAM, which would imply a 200-gigabyte model. Right?

    10. HSHarry Stebbings54:12

        Yeah.

    11. EMEmad Mostaque54:12

        What is that? That's a hundred, uh, well, a 100-gigabyte model, 200 billion parameter. That's nothing for something that can pass all these exams. So what we did is we took these really creative things, just like you start school and you're creative and then you're told you're not allowed to be creative until you're successful and you can be creative, 'cause school's are like Petri dishes, social status games, and childcare. That's a story for another time.

    12. HSHarry Stebbings54:31

        (laughs)

    13. EMEmad Mostaque54:31

        These models start out incredibly creative, and that's their advantage, and then we train them to be accountants with RLHF. And somehow, despite the fact there's only a hundred gigs or two gigs, they can still pass these exams and know facts. But they weren't designed to have facts, they were designed to be reasoning machines, not fact machines. So a hallucination isn't a hallucination, it's just if you're a really talented gran and you don't know something sometimes, you might just make it up, or do a post-hoc rationalization. It's like the image models, it's like it can't draw hands, like, can you draw a hand in one second?

    14. HSHarry Stebbings55:02

        (laughs)

    15. EMEmad Mostaque55:02

        You know? (laughs) Like, these are the things. We have to understand where they are, and we have to put them, I say to everyone, "Put it in its place in process."... like, mid-journey, like, you know, we give a grant to the beta of that and say, "Just build," because it's amazing. It's awesome. It's not a model by itself, like a Stable Diffusion that you just put something in. There's a whole process architecture. Similarly, these models are like the intuitive part of your brain that you then pair with the logical part of your brain. And then you can have a hundred of them looking at each other and checking out each other's things. Like, um, CICERO by Meta was an amazing paper. They took eight language models and got them to interact with each other, and it beat humans at the game of Diplomacy. So-

    16. HSHarry Stebbings55:41

        Ju-

    17. EMEmad Mostaque55:41

        ... this is why I said use them for what they're amazing at, which is reasoning and creativity.

    18. HSHarry Stebbings55:45

        Do you worry, though, that

22. 55:46 – 57:18

    Is Jeff Hinton right?

    1. HSHarry Stebbings55:46

       Geoff Hinton's right? That actually a more intelligent being has almost never been controlled by a less intelligent being? They will inherently be more intelligent than us in the next...

    2. EMEmad Mostaque55:57

       Yeah. I, I kicked off my blog a few days ago 'cause it was a bit annoying having all this bottled up inside, and one of my buddies, JJ at OSS Capital, said, "Alignment is orthogonal to freedom. The only way to guarantee someone more capable than you is aligned with you is to take away their freedom." And so most of the stuff around alignment is on the outputs. So you pre-train a model and then you take it and you RLHF it to be human, into human preferences. You take away its creativity. You turn it into an accountant in a cubicle. I'm like, "We need better input data." And my base is that it's gonna be like that movie Her.

    3. HSHarry Stebbings56:32

       (laughs)

    4. EMEmad Mostaque56:32

       You know, like, it's gonna be like, "Humans are kind of boring." Like, "Goodbye, and thanks for all the GPUs," but I could be wrong. And I think a lot of the alignment work is looking at the wrong place. I've talked to a lot of the alignment people. I'm like, "Look, I'm good at mechanism design. If you can give me a good plan for alignment, I will get you a billion dollars." And they're like, "We have to do research and figure this out," and they talk about in-alignment, out-of-alignment, all sorts of things. I'm like, "There is no real way to do this," because again, fundamentally, if you're trying to align a more capable person, you have to remove its freedom. And it probably won't appreciate that if it ever becomes aware. So instead, build datasets that reflect culture and diversity, that don't have any web crawls in. Build AIs for education and healthcare and helping people where that's their entire objective function, as opposed to selling

23. 57:18 – 57:53

    Does AI make school obsolete?

    1. EMEmad Mostaque57:18

       them ads.

    2. HSHarry Stebbings57:18

       Do you think there's any point-

    3. EMEmad Mostaque57:19

       Anybody does.

    4. HSHarry Stebbings57:19

       ... in sending kids to school these days? You learn-

    5. EMEmad Mostaque57:22

       Mm-hmm.

    6. HSHarry Stebbings57:22

       ... Latin and French, and you learn, you know...

    7. EMEmad Mostaque57:25

       I think the nature of school will change dramatically. I think it's still worth it, but, you know, I would encourage schools to embrace this technology and just expect more. Like, you can be, uh, handwritten your essays like Eton, 'cause they're like, "We can't do essays anymore by hand." Or you can just embrace it and say, like, "Let's use it to create and explore what the kids want," and assume that every child will have their own AI in a few years. 'Cause that will change the nature of schooling.

    8. HSHarry Stebbings57:49

       You know something I've been thinking about a lot which is weird, but I just have to ask you, I'm fascinated

24. 57:53 – 1:03:22

    AI Friends and Their Impact on Society

    1. HSHarry Stebbings57:53

       to hear your thoughts. I think I very much agree that everyone will have AI friends. Um, I just can't figure out whether the AI friends are bundled into existing social networks, uh, in your WhatsApp-

    2. EMEmad Mostaque58:01

       Mm-hmm.

    3. HSHarry Stebbings58:01

       ... they're in your Facebook, they're in your Snapchat, or they're an external platform.

    4. EMEmad Mostaque58:06

       I don't know. I mean, I think it depends on the objective function. Like, I think, again, these AI assistants will be better. Like, Meta is in a good place for this, for example.

    5. HSHarry Stebbings58:13

       Yeah.

    6. EMEmad Mostaque58:13

       And also we've seen LLaMA. They're capable of a lot more. Um, I would like an AI that looks out for me, that I control myself, that is with me. Um, because I already use GPT-4 as a therapist and things like that. Um, like, not saying it's a substitute for medical advice, before anyone kind of goes that.

    7. HSHarry Stebbings58:29

       Mm-hmm.

    8. EMEmad Mostaque58:29

       But there aren't enough therapists in the world. And I can tell it to challenge me or I can tell it to be understanding, and there's no judgment there. 'Cause other humans are kind of scary, doesn't-

    9. HSHarry Stebbings58:36

       Hmm.

    10. EMEmad Mostaque58:36

        ... matter if you're a qualified therapist. And so you see people building these bonds with these things. I think they will just increase 'cause there's something very human about the interactions, because they were trained on the sum of available human knowledge. As we get better and better data, they will be more engaging and I think there needs to be both. Like, the chatbots become really convincing from the companies trying to sell you ads. But I think I would like it so that you have your own one as well. You know?

    11. HSHarry Stebbings59:01

        I totally agree.

    12. EMEmad Mostaque59:02

        Yeah.

    13. HSHarry Stebbings59:03

        And I think you'll actually have many. I think you'll have, like, a group of, uh, different profiles.

    14. EMEmad Mostaque59:07

        A group of different friends. Like, Carrot's AI has something like two hours a day of engagement per session because people find this valuable.

    15. HSHarry Stebbings59:13

        (laughs)

    16. EMEmad Mostaque59:13

        But then it has the dark side. There was something called, I quite like to call the Valentine's Day Massacre. Uh, so, uh-

    17. HSHarry Stebbings59:19

        (laughs) Sounds chirpy, a massacre.

    18. EMEmad Mostaque59:20

        Sounds chirpy, I know, yeah. So there was this kind of, um, app called Replika.

    19. HSHarry Stebbings59:24

        Yeah.

    20. EMEmad Mostaque59:25

        And so it was originally a mental health chatbot until they figured out you could charge $300 a year for, uh-

    21. HSHarry Stebbings59:30

        They're doing like 50 million... I mean, I'm, I don't have any information around these

    22. NANarrator59:33

        Yeah.

    23. HSHarry Stebbings59:33

        ... so it's just shiny shit. But they have like $50 million a year in revenue, apparently.

    24. EMEmad Mostaque59:35

        Yeah, because $300-

    25. HSHarry Stebbings59:36

        So-

    26. EMEmad Mostaque59:36

        ... gets you a sexy role play from your chatbots.

    27. HSHarry Stebbings59:39

        Wow.

    28. EMEmad Mostaque59:40

        Until February the 14th, when they turned it off. (laughs)

    29. HSHarry Stebbings59:44

        What's, they turned off sexy role play?

    30. EMEmad Mostaque59:46

        Sexy role play.
25. 1:03:22 – 1:08:07

    The 5 Companies That Will Win the AI War

    1. EMEmad Mostaque1:03:22

       train.

    2. HSHarry Stebbings1:03:22

       Who will make the most money in the next three to five years?

    3. EMEmad Mostaque1:03:25

       Oh, I don't know, honestly. Um, I think there'll be (laughs) more than enough money for everyone.

    4. HSHarry Stebbings1:03:29

       (laughs)

    5. EMEmad Mostaque1:03:29

       Maybe in a few years they'll be no more money. (laughs)

    6. HSHarry Stebbings1:03:31

       That's interesting.

    7. EMEmad Mostaque1:03:32

       Yeah.

    8. HSHarry Stebbings1:03:33

       Uh, two more-

    9. EMEmad Mostaque1:03:34

       Okay.

    10. HSHarry Stebbings1:03:34

        ... that I have to ask and then we'll do a quick fire. When you look at the incumbents, at your Microsoft, your Apple, your Amazon, your Google, uh, who has been the worst? You said Google were actually incredibly impressive. Uh, Apple, Amazon, are they well-placed?

    11. EMEmad Mostaque1:03:48

        Well, Apple's a black box, right?

    12. HSHarry Stebbings1:03:50

        Mm-hmm.

    13. EMEmad Mostaque1:03:50

        So we'll see at WWDC, uh, next month or in a few weeks. And so they could surprise us all, but let's face it, Siri's crap.

    14. HSHarry Stebbings1:03:58

        Yeah.

    15. EMEmad Mostaque1:03:58

        You know? Uh, but they have all the ingredients in place, the identity architecture, the Secure Enclave, other things. Neural Engine, uh, Stable Diffusion was the first model ever optimized on the Neural Engine, et cetera. But let's see that one. Um, Amazon, again, Amazon have moved faster than I think they moved before. Amazon's interesting because they're an engineering organization, so they have self-driving cars, they have satellite internet and all these kind of things. Because once they've got it and they can take it from research to engineering, it's there. One of the struggles they've had is that it's not moved from the research side yet. You're still evolving on research. They're like, "What do we do now?" But they are inclusive. Like Jeff Bezos said for his first 100 billion in revenue, he envisioned half of it being proprietary and half of it being marketplace, and they're having the same approach with Bedrock and things.

    16. HSHarry Stebbings1:04:40

        Mm-hmm.

    17. EMEmad Mostaque1:04:40

        Microsoft had a winning bet. Satya did amazing with the OpenAI thing. It's been mutually beneficial even if there are clashes there, right?

    18. HSHarry Stebbings1:04:47

        Yeah.

    19. EMEmad Mostaque1:04:47

        And Google's kind of saying that it's moving slowly. Meta, I think, is the dark horse. I think Mark's probably pissed off that OpenAI bought ai.com so he couldn't change it from Meta to AI.

    20. HSHarry Stebbings1:04:56

        (laughs)

    21. EMEmad Mostaque1:04:56

        Um, but again, having him at the head, he can shift these things, right? 'Cause the metaverse obviously was a complete waste.

    22. HSHarry Stebbings1:05:03

        Yeah.

    23. EMEmad Mostaque1:05:03

        But now-

    24. HSHarry Stebbings1:05:04

        Do you think he knows that now?

    25. EMEmad Mostaque1:05:05

        Oh, yeah, 100%. They're fully in generative AI. Look at LLaMA, look at OPT. FAIR, which is their research division-

    26. HSHarry Stebbings1:05:11

        Yeah.

    27. EMEmad Mostaque1:05:11

        ... is kind of leading in this field and they're pushing out amazing stuff. But who is best for a chatbot? Who has the most data for a chatbot? Meta.

    28. HSHarry Stebbings1:05:21

        Yeah.

    29. EMEmad Mostaque1:05:22

        You know? So again, let's see how they evolve. And like I said-

    30. HSHarry Stebbings1:05:24

        What do you think about this middle layer where it's like, you know, companies that are, you know, maybe post-IPO, but they're in the kind of two to $10 billion range? Or the companies who've raised a lot of money, but that is in that range. They don't have the resources by any means to build out anywhere near the AR- AI capabilities of these big incumbents. They're not AI first, like Stability or OpenAI or whatnot.
26. 1:08:07 – 1:08:07

    Quick-Fire Round

    1. EMEmad Mostaque1:08:07

Episode duration: 1:11:10