Safety in Numbers: Keeping AI Open

1. 0:00 – 2:14

   Introduction to scaling laws and their impact

   1. AMAnjney Midha0:00

      Scaling laws. Now these underpin the success of large language models today. But the relationship between data sets, compute, and the number of parameters was not always clear. In fact, in 2022, a pivotal paper came out that changed the way that many people in the research community thought about this very calculus. And it demonstrated that data sets were actually more important than just the sheer size of the model. One of the key authors of this paper was Arthur Mensch, who was working at DeepMind at the time. Now, earlier this year, Arthur banded together with two other researchers, Guillaume Lample and Timothée Lacroix, two researchers at Meta who worked on Llama. And together, the three of them founded a new company, Mistral. That team has been hard at work releasing Mistral 7B in September, a state-of-the-art open-source model that quickly became the go-to for developers. And they just released, as in, in the last few days, a new mixture of experts model that naturally they're calling Mixtral. So today, you'll get to hear directly from Arthur as he sits down with a16z General Partner Anjney Midha. As the battleground for large language models heats up, to say the least, together they discuss the many misconceptions around open source and the war being waged on the industry. Plus the current performance reality of open versus closed models, and whether that gap will realistically close with time. Plus the kind of compute data and algorithmic innovations required to keep scaling LLMs efficiently. Now, it's really rare to have someone at the frontier of this kind of research be so candid about what they're building and why. So I hope that you come out of this episode as excited about the future of open source as I did. Enjoy. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax or investment advice or be used to evaluate any investment or security and is not directed at any investors or potential investors in any a16z fund. Please note that a16z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com/disclosures.

2. 2:14 – 7:57

   Arthur Mensch and the Founding of Mistral

   1. AMAnjney Midha2:15

      You've got, uh, quite the founding team story. You know, we flash back to a few years ago. Labs are building foundation models, and the consensus across the research community was that the size of these models was what mattered most. You know, how many million or billion parameters went into the model seemed to be the primary debate that people were having. But it seems like you had a hunch that data sets mattered more. Could you just give us the backstory on the Chinchilla paper you co-wrote? You know, what were the key takeaways in the paper, and how was it received?

   2. AMArthur Mensch2:45

      Yeah. So I guess the backstory is that, uh, in 2019, 2020, people were relying a lot on, um, on a paper called "Scaling Laws for Large Language Models." That was advocating for, uh, basically scaling infinitely the size of models and keeping some number of data points, uh, rather fixed. So just saying that if you had like four times the amount of compute, you should be mostly multiplying by three point five your model size and then maybe by one point two. Uh, and so a lot of work was actually done on top of that. So in particular, at DeepMind, when I joined, uh, I, I joined a project called Gopher, and, and that's, uh, uh... There was a misconception there. There was also the misconception on GPT-3. And basically in 2021, every paper, uh, made this mistake. And at the end of 2021, we started to realize there were some issues, and as it turns out, we turned back to, to the mathematical paper that was actually talking about scaling laws, and it was a bit hard to understand. And we figured out that actually, if you thought about it a bit more in a theoretical perspective, and if you looked at like empirical evidence we had, um, it didn't really make sense, uh, to actually grow the model size faster than the data size. And we did some measurements, and as it turned out, what was actually true was actually what we expect, which is in common words, if you multiply by four, uh, your compute capacity, you should multiply by two the model size and by two the data size. That's approximately what you should be doing. Which, which is good because if you move everything to infinity, everything remains consistent. So you don't have a model which is infinity big or a model which is infinity small with infinite compression or like close to zero compression. So it really makes sense. And as it turns out, it's really what you observe if you look at, uh, if you do multiple runs. And, and so that's how we, we trained Chinchilla and, and that's how we wrote the Chinchilla paper.

   3. AMAnjney Midha4:40

      At the time, you know, you were at DeepMind and your co-founders were at Meta. What's the backstory around how you three end up coming together to form Mistral after the compute optimal scaling laws work that you just described?

   4. AMArthur Mensch4:54

      So we've known each other for a while because Guillaume and I were in school together and Timothée and I were in Master together in Paris. Basically, we had like very parallel, uh, careers. Timothée and I, we actually worked together as well again, uh, when I was doing a postdoc, um, in mathematics. Uh, and then I joined DeepMind, uh, as Guillaume and Timothée, uh, went to, uh, become, uh, permanent researchers at Meta. Uh, and so we continued doing this. I was doing large language models in between 2020 and 2023. Guillaume and Timothée were working on, um, uh, solving, uh, mathematical, uh, problems, uh, with large language models. And, um, if I understand correctly, I wasn't there, but they realized they had to have stronger models, and they started to do large language models at this point. Uh, so like, I guess, uh, a year after I started. Uh, and on my side, I was mostly working in a small team at, uh, DeepMind, so we did very interesting work on, uh, with RETRO, which is a paper doing retrieval for large language models. We did the Chinchilla then, uh, we-- I, I was in the team doing Flamingo, which is actually one, one of the good way of doing a, a model that can see things. I guess when ChatGPT went out, uh, we knew, uh-- I mean we knew from before that the technology was very, very much game changing, but it was kind of, um, a signal that there was a strong opportunity for- Building a small team, uh, focusing, uh, on a different way of distributing the technology. So we're doing things in a more open source manner, uh, which was not the direction wa- that Google at least was taking. And, and so we had this opportunity, then, uh, we left the company, uh, at the beginning of last year and, and created the team that started to work on the fifth of June.

   5. AMAnjney Midha6:37

      And if I recall correctly, right before they left, Tim and Guillaume had started to work on Llama, right, over at Meta. Could you just describe that project and how it was related to the Chinchilla scaling laws work you'd done?

   6. AMArthur Mensch6:51

      So Llama was, uh, like a small team, uh, whole production of Chinchilla, at least in the, in its approach of parameterization and all of these things. Uh, it was, uh, one of the first papers that established that you could go beyond the Chinchilla scaling laws. So Chinchilla scaling laws tell you what you should be training if you want to have an optimal model, uh, for a certain compute cost at training time. But if you take into account the fact that your model should also be efficient at inference time, you probably wanna go far beyond, uh, the Chinchilla scaling laws. So it means you wanna over-train the model, so train on more tokens than would be optimal for performance. But the reason why you do that is that you actually compress models more, and then when you do inference, you end up having a model which is much more efficient, uh, for a certain performance. So by spending more time during training, you spend less time during inference, and so you save cost. And I think, uh, that was something we, well, I guess we observed at, at Google also, but, um, uh, the Llama paper was the first to establish it in the open, and it opened a lot of opportunities.

   7. AMAnjney Midha7:56

      Yep. I remember,

3. 7:57 – 13:27

   Mistral 7b and the launch of Mixtral

   1. AMAnjney Midha7:58

      you know, the, both the impact of the Chinchilla scaling laws work on the labs on, on multiple labs realizing just how unoptimal the, the compute setups were, right? Um, and then the impact of Llama being dramatic on the industry and realizing how to be much more efficient about inference time. So I can imagine that those are some of the, the top insights on your mind and the top concerns on your mind when you guys left, uh, to start Mistral. So let's fast-forward to today, you know, it's December 2023. We'll get to the role of open source in a bit, but let's just level set on what you've built so far. You know, a couple months ago, you released Mistral 7B, which was a best-in-class model. Um, and this week you're releasing a new mixture of experts model. So just tell us a little bit more about Mixtral, I believe is what you're calling it, and how it compares to other models.

   2. AMArthur Mensch8:53

      Yeah. So Mixtral is our new model that wasn't released in open source before in a, in a usable form. Uh, so it's a technology called Sparse Mixture of Experts, uh, which is quite simple. You, you take all of the dense layers of your transformer and you duplicate them. You call these layers expert layers. And then what you do is that for each ex-- for each token that you have in your sequence, uh, you have a router mechanism, so just a very simple network that decides which expert should be looking at which token. And so you send all of the tokens to their experts and then you apply the experts and you get back the, the, the output and you combine them and then you go forward in the network. You have eight experts per layer and you execute only two of them. So what it means at the end of the day is that you have a lot of parameters on your model. You have, uh, forty-six billion parameters. But the thing is that the, the number of parameters that you exe- that you execute, uh, is much lower than that because you only execute two branches out of th- eight. And so at the end of the day, you only execute twelve, uh, billion parameters per token and this is what counts for latency and throughput and for performance. So you have a model which has the performance of a twelve billion parameter network, uh, that can, uh, that have performance that are much higher than what you could get even by compressing data a lot on a twelve billion dense transformer. So Sparse Mixture of Experts is a technology, uh, that allows to be much more efficient at inference time and also much more efficient at training time. So that's the reason why we, we chose to develop it very quickly.

   3. AMAnjney Midha10:25

      Just for folks who are listening, um, who might not be familiar with sort of state-of-the-art architecture and la- language models, could you just describe the difference between, you know, dense models which have been the primary architecture to date and mixture of experts. Intuitively, what are the biggest differences between these two architectures?

   4. AMArthur Mensch10:44

      So they are very similar except on the, what you call the dense, uh, network. So the, the, you know, in a, in a dense transformer you have-- you alternate between an attention layer and a, and a dense layer generally. That's, that's the idea. A Sparse Mixture of Experts you, you take the dense layer and you duplicate it several times and so that's where you actually increase the number of parameters. So you increase the capacity of the model without increasing the cost. So that's a way of decoupling the memorization and what you can remember, the capacity of the network to its cost at inference time.

   5. AMAnjney Midha11:16

      If you had to describe the biggest benefits for developers as a result of that inference efficiency.

   6. AMArthur Mensch11:20

      It's cost and, uh, and latency. So you can have, uh, usually that's what, uh, you look at when you're a developer. You want something which is cheap and you want something which is fast. Generally speaking though, just the trade-off, uh, is strictly favorable, uh, in using Mixtral compared to using a twelve billion dense model. And the other way to think about it is that if, uh, you want to use a model which is as good as Llama 270B, you should be using Mixtral because Mixtral is actually, uh, on par with Llama 270B while being approximately six times cheaper or six times faster for the same price.

   7. AMAnjney Midha11:53

      Could you talk just a little bit about why it's been so challenging for folks to, uh, for research labs and research teams to really get the mixture of experts model right? It sounds like, you know, for a while now folks have known that the dense model architecture that all of us have been using in, in sort of the most notable products or the most well-known products, um, are slow Uh, they're expensive and they're difficult to scale. And so for a while, people have been looking for an alternative architecture that could be, like you were saying, cheaper, could be faster, could be more efficient. What were some of the biggest challenges you had to figure out to get the MoE model right?

   8. AMArthur Mensch12:35

      Well, I guess I won't disclose all trade secrets, but, uh, there's basically two challenges. The first one is you need to figure out how to train it correctly, uh, from a th-- from a mathematical perspective. The other challenge is, is to train efficiently, so how to use actually hardware, uh, as efficiently as possible. You have, like, new challenges coming from the fact that you have tokens flying around, uh, from one to-- one expert to another. Uh, that creates some communication constraints, and you need to figure it out. Uh, uh, you need to make it fast. And then on top of that, you also have new constraints that apply when you deploy the model. Uh, you do need to do inferencing, uh, efficiently. And that's also the reason why we released an open source package based on vLLM so that the community can, can take also th-this code and modify it and see how that works.

   9. AMAnjney Midha13:19

      Yeah. Obviously, we're excited to see what the community does with the MoE, uh, Mixtral release you're putting out this week.

4. 13:27 – 18:41

   Misconceptions about open source, the state of open vs. closed models, and future requirements for scaling LLMs

   1. AMAnjney Midha13:27

      Let's talk about open source, an approach and a philosophy that's, that's permeated all the work you've been doing so far. Um, why choose to tackle this increasingly competitive s-space with an open source approach, which has been-- which is quite different from the way everybody else is approaching it?

   2. AMArthur Mensch13:42

      I guess it's a good question. The, the, the answer is that it's partly ideological and, and partly pra-pragmatical. Um, we have grown with the field of AI, uh, that went fr-from 2012, we were detecting cat and do-- cats and dogs.

   3. AMAnjney Midha13:57

      Right.

   4. AMArthur Mensch13:57

      And in 2022, we were actually generating text that looked, uh, human-like. So really made a lot of progress. And if you look at the reason why we made all of this progress, uh, well, most of it is explainable by the free flow of information. So you had academic labs, you had very big, uh, industry-backed labs communicating all the time about the results and building on top of, uh, each other results. And that's the way we went from... We increased significantly, uh, the architecture and training, uh, techniques. Uh, we, we just made every-everything work as a community. And all of a sudden in 2020, uh, with GPT-3, this tide kind of reversed, and, uh, companies started to be more opaque about what they were doing because they, they, they realized there was actually a very big market. And all of a sudden in 2022, on the important aspects of AI, uh, and on LLMs we had-- we went and beyond Chinchilla, there were b-bas-basically no communication at all. And that's something that I as a researcher and, and Timothy and Guillaume and all of the people that joined us as well, deeply regretted, uh, because we think that we're definitely not at the end of the story. We need to invent new things. There's no reason why to stop now, uh, because the technology is effectively, uh, good, but not working completely well enough. And so we believe that it's still the case that we should be communicating a lot about, uh, models. We, we should be allowing the community to take the models and make it their own. And that's, that's, uh, some ideological reason why we went into that. The other reason is that we are talking to developers. Uh, developers want to modify things and, and having a deep access to a, to, to very good model is, is a good way of engaging with this community and, uh, well, and I guess addressing their needs so that the platform we're building as well is going to, to be used by them. So that's, that's also like a, a business reason. Obviously, uh, as a business, we, we do need to have, uh, a valid monetization approach at some point. Uh, but we've seen many businesses build open core, uh, approaches, uh, and have a very strong open source community and also a very good offer of services, and that's what we wanna build.

   5. AMAnjney Midha16:11

      That resonates. I, I remember a very detectable shift. You're, you're right. You know, the early days of, of deep learning were largely driven by a bunch of open collaboration between researchers from different labs who would often publish all their work and share them at conferences. You know, Transformers famously was published and, and open to the entire research community. Um, but that has, has definitely changed.

   6. AMArthur Mensch16:35

      Yes. So I think there's some level of open sourcing, uh, in, in AI. And so we offer the open, uh, the weights, and we offer the inference code. That's like the end product that is already super usable. So, uh, it's already a very big step forward compared to closed APIs because you can modify it, and you can, uh, look at what's happening under the hood, look at activations and all. So you have interpretability and the possibility of modifying the model to adapt it to, uh, some editorial tone, to adapt it to proprietary data, to adapt it to some specific instructions, which is something that is actually much harder to make if you only have access to a closed source API. Um, and that's something that also goes with our approach of the technology, which is to say pre-trained models should be neutral, and we should empower our customers to take these models and just put their editorial approaches, their instruction, their constitution, if you wanna talk like Anthropic, into the model. So that's the, that's the way we approach the technology. We don't want to pour our own biases into the, in-into the pre-trained model. On the other hand, we wanna enable the developers to control exactly how the model behaves, uh, and what kind of biases it has, what k- what kind of biases it doesn't have. So we, we really take this modular approach, and that goes very well with the fact that we release, uh, some very strong open weight models.

   7. AMAnjney Midha17:56

      Could you just ground us in the reality of where these models are today, just to give people a sense of, of where in the timeline we are? Is open source really a viable competitor to proprietary closed models, or is there a performance gap? You know, what are the trade-offs or limitations that people should be aware of, uh, with open source?

   8. AMArthur Mensch18:12

      So Mixtral is, has similar performance to GPT 3.5. So- That's the-- That's a good grounding. Internally, we have stronger models that are in between 3.5 and 4 that are basically second or third, the second or third best model in the world. So really we think that the gap is closing. Uh, the gap is approximately six months at that point. And the reason why it's, it's six months is that it actually goes faster if you do open source things, 'cause you get the community, uh, modify the model, suggest very good

5. 18:41 – 22:56

   Open Source in AI: Scaling laws, Industry Impact, data efficiency, and new model architectures

   1. AMArthur Mensch18:41

      ideas that can then be consolidated by us, for instance. And, and we just go faster because of that. So it has always been the case that open source, uh, at the end, well, ends up being going faster, and that's the reason why, uh, the entire internet runs on Linux. I don't see why it would be any different for AI. Uh, obviously, there's some constraints that are slightly different because the infrastructure cost is quite high. Uh, to train a model will cost a lot of money. But, uh, but I really think that, uh, we'll converge to a setting where we have proprietary models and the open source model are just as good. And I think eventually the field will be much more open because if you wanna go beyond the, the biggest model today, you do need to find new paradigms. And so that means that we also need to do research, uh, and that we're very excited by this perspective because we like, uh, competitive environment and research.

   2. AMAnjney Midha19:30

      Yeah. So let's talk about that a little bit more. How are you seeing people use and innovate on the open source models? And are there any use cases that diverge from proprietary closed models at all?

   3. AMArthur Mensch19:41

      I think we've seen several categories of usage. Um, you had-- There's a few companies that know how to strongly fine-tune models to their needs. So they took Mistral 7B, had a lot of human annotations, had a lot of proprietary data, just modify Mistral 7B so that it solved their task just as well as GPT 3.5, but only for a lower cost and a higher level of control. We've also seen, I think, very interesting community efforts in adding capabilities to Mistral 7B. So we saw like a context length extension to 128K that worked very well. Again, it was done in the open, so like the recipe was available and this is something that we were able to consolidate. We've seen image encoders to make it a vision-- a visual language model. A very actionable thing that we saw is, uh, I think the Hugging Face, uh, folks first, uh, did the direct preference optimization on top of Mistral 7B and made a very strong, a much stronger model than the instructed model we proposed, uh, at the early release. And it turned out it's actually a very good idea to do it. Uh, and so that's something that we've consolidated as well. Uh, so generally speaking, uh, just the community is super eager to just take the model and add new capabilities, put it on a laptop, uh, put it on a, on an iPhone. I saw Mistral 7B on an iPhone. I saw Mistral 7B on a stuffed parrot as well. So fun things, uh, useful things. Uh, but generally speaking, it's been super exciting to see, uh, the research community, uh, take a hold of, of our technology. And with Mixtral, which is a new architecture, I think we are also going to see much more interesting things because, uh, on the interpretability field, uh, also on the safety field, as it turns out. Uh, you have a lot of things to do when you have deep access to an open model. And so we're really eager to, uh, to help that, uh, and to engage with the community.

   4. AMAnjney Midha21:26

      Safety, uh, this is an important, I think, piece to talk about. The immediate reaction of a lot of folks is to deem open source less safe than closed models. How would you respond to that?

   5. AMArthur Mensch21:40

      So I think we believe that it's actually not the case for the current generation of model. Uh, models that we are using today are not that much, uh, are not much more than just a compression of whatever is available on the internet. So it does make access to knowledge more fluid. Uh, but this is the story of humanity making knowledge, uh, access more fluid. So it's no different than, uh, uh, inventing the printing machine where we had apparently similar debate. I wasn't there, but that was the debate we had. Uh, so we are not making the world any time, uh, any less safer by providing more interactive access to knowledge. So that's the first thing. Now, the other thing is that you do have immediate risk, uh, of misusage of large language models. And, uh, you do have them for open source models, but also for closed models.

   6. AMAnjney Midha22:27

      Right.

   7. AMArthur Mensch22:27

      And so the reas-- the way you do, uh, address these, uh, problems and come up with countermeasures is to know about them. Uh, so you need to know about, uh, about breaches, basically. That's the same way in which you need to know about breaches on, on operating systems and on, uh, on networks. And so it's no, no different for, for AI. Uh, putting, uh, models under the highest level of scrutiny is a way of knowing how they can be misused, and it's a way of co-coming up with countermeasures. And I think a good example

6. 22:56 – 25:17

   Safety concerns of open source models.

   1. AMArthur Mensch22:56

      of that is that it's actually super easy to exploit an API. Uh, it's super easy, especially if you have fine-tuning access to make GPT-4 behave, uh, in a very bad way. And it's, um, since it's the case and it's always going to be the case, it's super hard to be a bit adversarially robust. It means that we're only trusting the team, uh, of large companies to figure out ways of, uh, addressing these problems. Whereas if you do open sourcing, you trust the community, and the community is much larger. Um, and so if you look at the history of software in cybersecurity, uh, in operating systems, that's the way we made the system safe. And so if we wanna make the current AI system safe and then move on to a next generation that potentially will be even stronger, and then we can have this, we can have this discussion again, well, you do need to do open sourcing. So today, we think that open sourcing is a safe way.

   2. AMAnjney Midha23:49

      Yeah. I think this is, this is not understood, right, widely, that, um, when you have thousands or hundreds of thousands of people able to re-red team models because it's open source, the likelihood that you'll detect biases and, um, built-in breaches and risks are just dramatically higher. Um, and I think if you were talking to policymakers, how would you help advise them? Or how, how do you think they should be thinking about regulating open source models? Given that, you know, the safest way often to battle-harden software and tools is to put them out in the open.

   3. AMArthur Mensch24:28

      Well, we've been saying that precisely this, that, um, that the current technology is not dangerous. On the other hand, the fact that we, we are effectively making them stronger means that we need to monitor what's happening, empirically monitor our performances. Best way of mo- uh, empirically monitoring p- software performances is, is through open source. So that's what we've been saying. Um, there's been some effort to try to come up with very complex governance structure where, uh, you would have like several companies talking together, having some safe space, some safe sandbox, uh, for, uh, red teamer that would be potentially independent. So things that are super complex. Uh, but as it turns out, if you look at the history of software, the only way we did software collaboratively is through open source. So why change

7. 25:17 – 33:17

   Recommendations for policymakers in regulating AI technologies.

   1. AMArthur Mensch25:17

      the recipe today where, uh, the, the technology we're looking at is actually nothing else than the compression of the internet? So that's the-- that's what we've been saying to the regulators, uh, generally. Another thing we've added to the regulator is that if they want to enforce that AI products, uh, that needs to be safe, like, like if you want-- if you want to have a diagnosis assistant, you want it to be safe.

   2. AMAnjney Midha25:41

      Right.

   3. AMArthur Mensch25:41

      Well, in order to monitor and to evaluate whether it's actually safe, you need to have some very good tooling. And the, the tooling requires to have access to LLMs. And if you access closed source APIs LLMs, where you're, you're a bit in a, uh, in a tro- in troubled water because it's hard to be independent in that setting. So we think that independent controller of product safety should have access to very strong open source models and should own the technology.

   4. AMAnjney Midha26:08

      And if open source LLMs were to fail relative to closed source models, why would that be?

   5. AMArthur Mensch26:18

      Well, I guess the, the regulation burden is, is, uh, potentially one thing that, that could, uh, make it harder to, uh, to release open source models. It's also, generally speaking, uh, it's a very competitive, uh, market, and I think in order for open source models to be widely adopted, they need to be as strong as open sou- as a closed source model. They have a s- a little advantage because you do have more control, and so you can do, uh, heavier fine-tuning, and so you can make, uh, performance jump a lot on a specific task because you have deep access. Uh, but really at the end of the day, um, developers look at performance and, uh, and latency. And so that's why we think that as a company, we need to be very much on the frontier if we want to be relevant.

   6. AMAnjney Midha27:03

      Given the complexity of, uh, frontier models and foundation models in these systems, there are just tons of misconceptions that folks have about these models. And so if you step back, can we look at the battle that's raging between folks, um, uh, pushing for closed source systems and versus the open source system? What do you think is at stake here? What do you think the battle is really for?

   7. AMArthur Mensch27:28

      Well, I think the battle is for the neutrality of, uh, the technology. Like a technology by a sense is something neutral. You can use it for bad purposes, you can use it for good purposes.

   8. AMAnjney Midha27:38

      Right.

   9. AMArthur Mensch27:38

      If you look at what an LLM does, it's not really different, uh, from a programming language. It's actually used very much as a programming language by the application makers. There's a strong, um, confusion made between what we call a model and what we call an application. And so a model is really the programming language of AI application. So if you talk to all of the startups doing am-amazing products with generative AI, they're using LLMs just as, um, as a function. And on top of that, you have a very big systems with filters, uh, with decision making, uh, with control flow and all of these things. And what you wanna regulate, if you want to regulate something, is the system. The system is-- it's the product. So for instance, um, a healthcare diagnosis assistant is, is an application. You want it to be non-biased. You want it to take good decisions, uh, even under high pressure. So you want its statistical accuracy to be very high. And so you wanna measure that. And it doesn't matter if it uses a large language model, uh, uh, under the hood. What you wanna regulate is the application. And the issue we had, and the issue we're still having now is we hear a lot of people saying we should regulate the tech, so we should regulate the function, the mathematics behind it. But really, you never use a large language model itself. You only always use it in an application in a, in a, in a way, in-- with a user interface. And so that's the one thing you wanna regulate. And what it means is that companies like us, like foundational model companies, will obviously make the model as controllable as possible so that the applications on top of it can be compliant, can be safe. Uh, we'll also build the tools that allow to measure the compliance and the safety of the application because that's super useful for the application makers. It's actually needed. But there's no point in regulating something that is neutral in itself, that is just a mathematical tool. So I think that's the one thing that we've been hammering a lot. Uh-

   10. AMAnjney Midha29:42

       Right

   11. AMArthur Mensch29:42

       ... I think we've been heard, which is good. Uh, but, uh, there's still a lot of effort, uh, in, uh, I guess in making this strong distinction, which is super important to understand what's going on.

   12. AMAnjney Midha29:53

       So to regulate apps, not math seems like, you know, the right direction that a lot of folks who are-- who understand the inner workings of these models and how they're actually implemented in, in reality, um, are advocating for. W-what do you think, um, is the best way to clear up the, this misconception for folks who don't-- maybe don't have technical backgrounds, don't actually understand how foundation models work and how the scaling models work?

   13. AMArthur Mensch30:22

       So I've been using a lot of metaphors, I guess, [chuckles] to, to, to make it understood. Large language models are like programming languages. Uh, and so-

   14. AMAnjney Midha30:29

       Right

   15. AMArthur Mensch30:29

       ... you don't regulate, uh, programming languages. You regulate, uh, malwares. You, you ban malwares. We've also been actively, uh, vocal about the fact that pre-market conditions like FLOPS, the number of-

   16. AMAnjney Midha30:42

       [chuckles]

   17. AMArthur Mensch30:43

       ... of FLOPS that you do to create a model, is definitely not the right way of doing, uh, of measuring the performance of a model.

   18. AMAnjney Midha30:50

       Right.

   19. AMArthur Mensch30:51

       We, um, we're very much in favor of having very strong evaluations. That's... As I've said, uh, this is something that we wanna provide to our customers, the, the ability to evaluate our models in their application. Uh, and so I think this is a very strong thing, um, to... Well, that we've been stressing. We want to provide the tools for application makers to be compliant. That's the-- That's something we have, we've been saying. And so we find it a bit unfortunate that, uh, we haven't, we haven't been heard everywhere, uh, and that there's still a big focus on the tech, uh, probably because things are not completely well understood because it's a very complex field, and it's also a very fast-moving field. Uh, but eventually, I think I'm, I'm very optimistic that, uh, we'll find a way to, uh, continue innovating, uh, while having safe products, but also, uh, high level of competition on, on the foundational model, uh, layer.

   20. AMAnjney Midha31:47

       Well, let's, let's channel your optimism a little bit. You know, there's, there's very few people who have the ground level understanding of scaling laws, um, like you, Guillaume, and Tim, and your team. Y-When you step back and you look at the entire space of language modeling, in addition to open source, what are the key differentiators that you see in the next wave of cutting-edge models? Um, things like y-you know, uh, self-play. You have process reward models, um, the uses of synthetic data. Uh, if you had to conjecture, what do you think some of the most exciting or important breakthroughs will be in the field going forward?

   21. AMArthur Mensch32:23

       So I guess it, it's good to start with diagnosis. So what is, uh, what is not working that well? So reasoning is not working that well.

   22. AMAnjney Midha32:31

       Right.

   23. AMArthur Mensch32:31

       And it's super inefficient to train a model. Uh, if you compare like the training process of, uh, of a large language model to the brain, you have like, uh, a factor, uh, uh, I think one hundred thousand. So really there's some progress to be made on term of data efficiency. So I think the, the frontier is increasing data efficiency, increasing reasoning capabilities, so adaptive compute is one way. Uh, and when to increase data efficiency, you do need to work on coming up with very high-quality data, filtering things, uh, many new techniques that needs to be invented still. But that's really where the lock is. Uh, data is the one important thing, and the ability of the model to decide how much compute it wants to allocate to a certain problem, uh, is definitely

8. 33:17 – 36:36

   Predictions on how advancements in LLMs will change user interactions with technology

   1. AMArthur Mensch33:17

      on the frontier as well. So these are things that, uh, we're actively looking at.

   2. AMAnjney Midha33:22

      You know, this is a raging debate, right? We've-- And we've talked about this a few times before, which is, um, can models actually reason today? Do they actually generalize out of distribution? What's your take on it, and what do you think is required to, to exhibit... What, what would convince you that models are actually capable of multi-step complex reasoning?

   3. AMArthur Mensch33:43

      Yeah. It's very hard because you train on the entire human knowledge, and so-

   4. AMAnjney Midha33:46

      Right

   5. AMArthur Mensch33:46

      ... you have a lot of reasoning places. So it's, uh, it's hard to say whether they reason or not, or whether they do retrieval or reasoning, and it looks like reasoning.

   6. AMAnjney Midha33:55

      Right.

   7. AMArthur Mensch33:56

      Uh, I guess at the end of the day, what matters is whether it works or not.

   8. AMAnjney Midha33:59

      Right.

   9. AMArthur Mensch33:59

      And on many simple reasoning tasks, it does. So we can call it reasoning.

   10. AMAnjney Midha34:04

       Right.

   11. AMArthur Mensch34:04

       Uh, it doesn't really matter if they reason like we do. We don't even know how we reason, so-

   12. AMAnjney Midha34:08

       Right

   13. AMArthur Mensch34:08

       ... so we are not going to know about how machines reason anytime soon.

   14. AMAnjney Midha34:12

       Yeah.

   15. AMArthur Mensch34:13

       Um, so yeah, it's a, it's a raging debate. Uh, the reason, the, the way you do evaluate that is to try to be as out of distribution as possible. Uh, like working on, uh, on mathematics, uh, is not something I've ever done, but that's something that Timothé and Guillaume have-- are, uh, very, uh, sen-sensitive to because they've been doing it for a while, uh, when they were at Meta. That's probably one way of measuring whether you have a very good model or not. And actually, if you look at, um... We're starting to see some very good mathematicians, uh-

   16. AMAnjney Midha34:45

       Mm-hmm

   17. AMArthur Mensch34:45

       ... I'm thinking of Terence Tao, uh-

   18. AMAnjney Midha34:47

       Right

   19. AMArthur Mensch34:47

       ... that are using large language models for some things. Uh, obviously not the high level reasoning, but for some part of their proofs. And so I think we will move up, uh, into abstraction. Uh, and the question, where does that stop? We do need to find new, new paradigms to, uh, to go, uh, one step forward, and, and we're, we will be actively looking for them.

   20. AMAnjney Midha35:08

       We've talked a lot about developers so far. If you had to sort of channel your product view and, and sort of just conjecture on what these advances in scaling laws in, in, in representation learning, in get-- teaching the models to, to reason faster, better, cheaper. What will these advances mean for end users in terms of how they consume, how they program, and they generally work with models?

   21. AMArthur Mensch35:31

       What we think is that, uh, fast forward five years, uh, everybody will be using their specialized, uh, models within part of complex, uh, applications and systems. Developers will be very, um, looking at latency, so they will want to have for any specific task of the system, they will want to have the lowest cost and lowest latency. And the way you make that happen is that, uh, you will ask for the task, ask for user preferences, ask for what you want the model to do, and you try to make the model as small as possible and as suitable to the task as possible. And so I think that's the way we'll be evolving on the developer space. I also think that, uh, generally speaking, the fact that we have access to large language models is going to reform completely the way we interact with machines, and the internet of five years, five years from now is going to be much different, much more interactive, uh, because I think this is already unlocked. I mean, it's just about making very good applications with very fast systems, uh, with very fast models. So yeah, very exciting times ahead.

   22. AMAnjney Midha36:34

       So what would those interaction

9. 36:36 – 38:53

   Potential applications in various fields like gaming and enterprise.

   1. AMAnjney Midha36:36

      modalities look like?

   2. AMArthur Mensch36:37

      Yeah. So that's very interesting. And I think in, in games, for instance, it's going to be fascinating. Uh, we've seen some very good applications. You do need to have small models 'cause you want to have swarms of it, and it starts to be-

   3. AMAnjney Midha36:48

      Right

   4. AMArthur Mensch36:48

      ... a bit costly to-- if it's too big. Uh, but having them interact is just going to make pretty complex systems and interesting systems to observe and to use. Uh, so, uh, so we have a few, uh, friends making applications in the enterprise space, space with different persona playing different roles, uh, relying on the same language model, but with different prompts and different, uh, fine-tuning. Um, and I think that's going to be quite interesting as well to, uh, to look at. As I've said, complex applications in, in, in three years' time are just going to use different, uh, parts, different LLMs for different parts, and that's going to be quite exciting.

   5. AMAnjney Midha37:25

      Well, what's your call to action? To builders, researchers, folks who are excited about this space, what would you ask them to do?

   6. AMArthur Mensch37:33

      I, I would take, uh, Mistral models and try to build amazing applications, uh, the way many developers had. Uh, it's not that hard. Uh, it's-- the stack is starting to be pretty clear, uh, pretty efficient. Uh, you only need a couple of, uh, GPUs. You, you can even do it on your MacBook Pro if you want. Uh, it's going to, to be a bit hot, but, uh, uh, but it's, uh, good enough to, to do interesting applications. Uh, really the way we do software today is very different from the way we did it from l-last year. And so I'm really calling application makers to action because we, we are going to, to try to enable them to, to build as fast as possible.

   7. AMAnjney Midha38:11

      [upbeat music] Thank you so much for listening to the a16z podcast. What we're trying to do here is provide an informed, clear-eyed, but also optimistic view of technology and its future. And we're trying to do that by featuring some of the most inspiring people and the things they're building. And so if you believe in that and you'd like to join us on this journey, make sure to click subscribe, but also let us know in the comments below what you'd like to see us cover next. Thank you so much for listening, and we will see you next time. [upbeat music]

Episode duration: 39:02