Cohere Founder, Nick Frosst: How To Compete with OpenAI & Anthropic, and Sam Altman’s AI Disservice

Nick Frosst 0:00 I don't think Sam Altman has done a service to the world by talking about how close AGI is. I think he has made several predictions now that are wrong, and that were obviously wrong at the time he made them. I think AI will probably lead to the end of the world. You know, he's made allusions to things. He did a world tour where he spoke to every major leader, the world over, to tell them, "Hey, this technology is gonna pose as an existential threat." And I think that was academically disingenuous, and I think did a disservice to the technology he loves.

Harry Stebbings 0:25 Ready to go? Nick, I'm so excited for this, dude. When I had Aidan on the show-

Nick Frosst 0:40 Mm-hmm.

Harry Stebbings 0:40 ... he was like, "You've gotta have Nick on. He's the real star of the show." And he introduced us way back then.

Nick Frosst 0:45 Mm-hmm. Hmm.

Harry Stebbings 0:45 So, I'm so excited that we could make this happen.

Nick Frosst 0:47 Yeah, man. I'm happy to be here.

Harry Stebbings 0:48 Now, before we dive into

Harry Stebbings 0:50 Cohere, I have to ask, you were Geoff Hinton's first hire at Google Brain.

Nick Frosst 0:54 Mm-hmm.

Harry Stebbings 0:55 And so then you're put in a room with Geoff Hinton. You get to work with him every day. What was the biggest lesson from working with Geoff, a legend of the industry?

Nick Frosst 1:02 Yeah. I, I learned... Yeah. I, I loved working with Geoff. Um, I learned everything I know about research, um, from those, those... I think we were there for four years, three years? Um, I think I was very surprised at how creatively and playfully he approaches research. Um, when we would discuss, like, algorithms or, or, or, like, optimizers or, or, uh, loss functions, we would discuss them often in, like, through physical analogy. So, we'd spend a lot of time talking about, like, imagine there's like a ball here and like an elastic band to this thing, and a pulley here, and like this is what the... you know, it's on this kind of a surface. And like a lot of it was descriptions in the natural physical world. And that was very, yeah, like playful. And a lot of it was approached with like, "Oh, what would happen if..." You know, with, with curiosity. Um, and I didn't ex-... Uh, when working with him, I didn't expect that. Um, I expected hi- it to be much more like, you know, just, "Here's the equation. Let's, let's figure out what the derivative is and let's, let's go from there." Whereas instead, a lot of it's based on like intuition.

Harry Stebbings 2:14 When you look at Google Brain and you look at DeepMind, a lot think that really kind of Google were asleep at the wheel, given them not being at the forefront in what was the consumerization of it with ChatGPT.

Nick Frosst 2:24 Mm-hmm.

Harry Stebbings 2:24 Do you think that's fair?

Nick Frosst 2:26 I don't know. I mean, uh, it's certainly interesting. Look, like the transformer was invented at Google, right? Like, there was... Uh, in 2017, Aidan, uh, um, amongst with, um, many other brilliant people in Google Brain published the transformer as an architecture. Um, it wasn't, it wasn't then commercialized very quickly within Google. It wasn't scaled up very quickly within Google. Um, a lot of that work had to be done elsewhere and years later. So, that's interesting. And like w- why that is, like what, what, what systems are in place to make that be the case? I, I don't know. I, I, I will say there's still a ton of brilliant people in DeepMind, I think, now. It's, it's just cons- sub- subsumed the rest of it. Doing great work, um, and they continue to make good products. Uh, it is interesting that all the people who worked on the transformer left to continue to work on the transformer.

Harry Stebbings 3:20 If we kind of go down your tangent, for people who don't know, and just to set the scene before we dive in-

Nick Frosst 3:25 Mm-hmm.

Harry Stebbings 3:25 ... properly, what is Cohere and how does it differentiate from more generalized models that are maybe more well known, like your OpenEyes-

Nick Frosst 3:32 Mm-hmm.

Harry Stebbings 3:32 ... and your Anthropics?

Nick Frosst 3:33 Yeah. So, we're, we're a foundational model company, um, like those other two. So, we, we build foundational models, we build language models. Um, there's maybe... I don't know. Maybe there's like 10 companies in the world that are building lang- large language models.

Harry Stebbings 3:47 In the West.

Nick Frosst 3:49 In the... Yeah. Maybe the... In... Maybe there's like 15 in... I don't know. We'll have to figure out how many. There's a few that have popped up recently.

Harry Stebbings 3:56 Okay.

Nick Frosst 3:56 Um, but there's some num- some number. Less than 20 in the whole world. Most of them in America. Uh, a handful of them in China. Uh, us in Canada, and one in France.

Harry Stebbings 4:07 Yeah.

Nick Frosst 4:08 Uh, so those are really the, the companies out there. We're unique in our singular focus on bringing this technology to enterprise. So, that means we train a model that is good at enterprise tool use. So like, we train a model that you can, you know, give it a bunch of tools and APIs within your business, give it access to your business's data, and then you can ask it to help you with something in your work and it does a good job of it. So, that's what we train it for.

Harry Stebbings 4:34 How does the focus on enterprise over consumer change the way in which you train and build a model?

Nick Frosst 4:41 Hmm. So, the models themselves, like transformer architecture, which is the, the original model that... Yeah. That was introduced in 2017, uh, hasn't changed very much, right? Like every... All, the whole industry is still using transformers. We've changed the way we train them, but the model architecture itself, you know, we're approaching 10 years of, of the same model architecture. Um, when we train our model, we're not training it to be like an amazing conversationalist with you. We're not training it to like keep you interested and keep you engaged and occupied. We don't have like engagement metrics or things like that. We're just training it to augment you in the workplace. We're just training it to help you do your job. Um, and that means the type of data we train it on is very different. So recently, we started doing a bunch on like synthetic data. So, we, you know, generate a whole bunch of data, um, to create like fake companies and fake emails between people at these fake companies, and fake APIs within those fake companies. And then we train the model in that synthetic environment to help out within that fake business.

Harry Stebbings 5:41 Do you think data is a bottleneck given the ability for synthetic data to produce infinite supply?

Nick Frosst 5:47 Yeah. Data is still a bottleneck. And you, you need, you need real world data in order to start a process of synthetic data. Synthetic data has helped a lot, um, and it's made models better than they would be if they didn't have access to it. Um, but getting access to high quality data, uh, is still something people think about. We still, you know, we still make a whole bunch of data in-house with annotators, um, who are making real data and not synthetic data.

Harry Stebbings 6:09 When you think about kind of the three pillars of compute, algorithms, and data-

Nick Frosst 6:13 Mm-hmm.

Harry Stebbings 6:13 ... which one do you think is most constrained or the biggest bottleneck?

Nick Frosst 6:18 ... it's interesting. I mean, the algorithms haven't changed very much. Like that's an interes-... Like, they've changed a little bit. You know, when we started this industry, you know, originally we were just training base models, which were not called base models at the time. They were just called large language models, um, but they weren't trained from human feedback. So all they would do is, you know, take in the first part of a sentence and write the second part of a sentence. Um, but if you tried to have a conversation with them and like... it wouldn't work, 'cause that wasn't the data they were trained on. Um, since then, you know, now we, we train models in a few different steps. There's like a base modeling step, then there's a reinforcement learning step from human feedback with SFT data. After that, you know, there, there might... there's a variety of other reinforcement learning techniques you, you can do. Um, but the algorithms, I, I think, are not the bottleneck in terms of making those models more useful. I do think a, a lot of it is still getting good quality data and then making good quality synthetic data from your good quality real data.

Harry Stebbings 7:16 When we think about kind of the bottlenecks, that leads to potentially a plateauing that people are worried about, and everyone seems to now be on the train of, "Hey, more compute, scaling laws are more real than ever, and we will continue this exponential progress with more compute." Do you agree that we are seeding the benefits of scaling laws for the continuous next 12 to 24 months? Or do you think that actually more compute will not just lead to more progress?

Nick Frosst 7:43 How much better do you think GPT-5 was than GPT-4?

Harry Stebbings 7:48 I actually think it was worse.

Nick Frosst 7:49 So I think that was kind... I, I think that tells you something about the nature of just throwing more compute at the problem.

Harry Stebbings 7:55 Does it or does that show an o-... So why do I think it was worse? I think it was worse because actually the way that they now do model selection is slower and more cumbersome, and actually it's a pain. It gets it wrong sometimes, I mean, I just want a quick answer and it suddenly goes into deep research-

Nick Frosst 8:09 Mm-hmm.

Harry Stebbings 8:09 ... and I'm like, "Oh, for fuck's sake, I just want a quick answer."

Nick Frosst 8:11 (laughs)

Harry Stebbings 8:11 Right? Yeah. All right, PhD, calm down.

Nick Frosst 8:13 (laughs)

Harry Stebbings 8:13 (laughs) Uh, I do know what I mean? And so, uh, I think it's a worse product in that respect, and I think we, uh, waited for y-... a year or a year and a half-

Nick Frosst 8:23 Yeah.

Harry Stebbings 8:23 ... for model auto selection.

Nick Frosst 8:25 I, I think, like if I go back to your, your, your original question of like, you know, do I, do I think just throwing more compute? Like some people are thinking there's a plateau.

Harry Stebbings 8:32 Yeah.

Nick Frosst 8:32 Do I think sh-... there's more to compute. Uh, like under your... Like, I think we need to agree on where I th-... where we think the technology is going to establish whether or not there's a plateau. Right? Like, I think language models are incredible. I think they're super useful. I use them in my work life as often as possible. One of the reasons why we're focused on the enterprise is because that's really where I think lar- large language models are useful. Like if I look at my personal life, there's not a, there's not a ton that I want to automate. You know, like, I actually don't want to respond to text messages from my mom faster. I want to do it more often, but like io, I want to be writing those. I want to be like engaged, you know? Um, whereas in my work life, there's a ton of stuff I don't want to do. Like I... Like we need to get to a stage where I can, you know, open up North and I can say, "Hey, uh, file my expenses," and then it can figure out, okay, cool, I got to, you know, look through all your emails. I got to look through photos of receipts you've taken. I got to cross-reference that with the things you're allowed to expense via internal documentation. Then I got to figure out what the API is for how to expense things within your company, and then I got to do all of those and get approval before I do them. Like that's a super... that's a many step process, um, but that's where the technology is going. So that work, the work of making a m- model do that is not plateauing. That's more modeling work, that's more, uh, product work, that's like building better connectors, that's building, uh, safer data integration so that you can trust giving a model access to the types of stuff I just said. That stuff's still ongoing, and that's what we're, we're working on. I think when people are talking about building towards AGI, um, like, I don't think this technology gets us there.

Harry Stebbings 10:11 When you say gets us there-

Nick Frosst 10:12 Yeah.

Harry Stebbings 10:12 ... what is there?

Nick Frosst 10:14 W- uh, well, yeah. Great question. We've had many years of people discussing AGI, um, and not many definitions thereof. (laughs)

Harry Stebbings 10:22 Next to none.

Nick Frosst 10:22 Yeah. Yeah.

Harry Stebbings 10:22 I mean, my, my definition is when Sam Altman and Microsoft decide.

Nick Frosst 10:26 Yeah. They've changed their definition a few times on that. When I say AGI, what I mean is a computer that you treat like a person. I mean, when you use a computer and you expect it to behave like a person and treat it that way, um, I'll call that AGI.

Harry Stebbings 10:42 Do you not think we're already there then?

Nick Frosst 10:43 Uh, people do not treat language models like they treat people.

Harry Stebbings 10:46 Do you think OpenAI and Sam Altman then now realize that more compute does not lead to this exponential progress when they look at GPT-5?

Nick Frosst 10:53 I don't know. I think... Like, uh, you know, they're a great company, they built a really cool consumer product. Um, I, I don't know what they're thinking.

Harry Stebbings 10:59 Well, I guess my question is, why does the world still think scaling laws are so prevalent when you don't?

Nick Frosst 11:05 I think the... a lot of the world does not think scaling laws are super prevalent. I think if you go out into a university and like talk to the students there who are studying computer science, or even the students who are not, and you ask them like, "Hey, you know, is throwing more compute at this problem going to get us to AGI?" Most of them say no.

Harry Stebbings 11:22 You mentioned there are a couple of different use cases in terms of like, uh, expense management was one that you-

Nick Frosst 11:26 Yeah.

Harry Stebbings 11:26 ... clearly articulated. A question that I think I have and a lot of people have is, how far do models go in terms of value capture in the application layer? And you're seeing Anthropic now with Claude-

Nick Frosst 11:36 Mm-hmm.

Harry Stebbings 11:36 ... really challenge the cursor of the world. You're seeing OpenAI with a lot of consumer products. You're seeing you with a lot of, you know-

Nick Frosst 11:41 Mm-hmm.

Harry Stebbings 11:41 ... enterprise use cases. How do you think about whether they stay as AWS style commodity layers-

Nick Frosst 11:48 Hm.

Harry Stebbings 11:48 ... or whether they extend into value capture of application layer?

Nick Frosst 11:52 Yeah. That's a good question. Um, I don't see the two as that different. I see the two as related. And if you want to be making a good product with a large language model, you are best suited training that large language model for that use ca-... for that product. Right? Like that's, that's one of the really interesting things about LLMs, is that they're really phenomenal, they generalize really well, but they don't generalize as well as you might think.... and if you want to make the best model for a given, you know, pro- interface, it's best to be training the model on that interface. So I think the two things are more related.

Harry Stebbings 12:28 So do we see this deeply specialized, unbundled model world where you have exactly that, very, very specific use cases where models are trained for, and ElevenLabs of the world would be an-

Nick Frosst 12:40 Mm-hmm.

Harry Stebbings 12:40 ... another brilliant use case with, uh, specifically voice. Is that in the world that we live in?

Nick Frosst 12:46 Well, there's a, there's like a spectrum, right? Like, the old world of machine learning back in like, the old world, back in like 2015 or something. (laughs)

Harry Stebbings 12:53 Oh my god.

Nick Frosst 12:54 You know when, when, when, when the world was new. Uh, a- any task you wanted to do with a neural net was, uh, the best neural net you were gonna get was training a model on that task. So if you wanted to make a task that was gonna like, you know, a neural net that was gonna identify pictures of cats and tell you how many cats were in an image, you were best suited to train a model on identifying pictures of cats. That was like the best. And the, the world of machine learning in the first half of that decade was all about here's a problem, make a data set, train a new model from scratch, or maybe take like SIFT features or maybe take like, whatever, some, some base model, but pretty much fine tu- like train a model on the data self itstel- on the data set itself and go to production with that model. That's not the case with language. If you want to make a model that's the best at like summarization, you can't just train it on summarization. You have to train it on all language. And like that is, that's the in- like that is the technological reality that has brought us to where we are today, and that's true for like foundational models and not true for the neural nets of, of 2015 and, and before. That's super interesting. Um, so, but that's like a spectrum, right? Like on one end is every single task, train a single model for it. On the other end is train a model, one model to do everything. I think the reality of what we're seeing with transformers is they're not at this end of the spectrum. They're like a little over here, and they're like train a model that is generally good at all language and refine it on the type of stuff you want to do with it. So you're seeing that like Anthropics co- like code models, very good at code. But they didn't train a model specifically, like a refactoring model or a debugging model, or they didn't train a model just for writing test cases, right? It's a model that is good generically at code. Um, for us, for Cohere, with our focus on enterprise and like secure deployments and customizations for our c- and for our customers, um, that means training a model that is good at helping people in an enterprise setting. So like using internal tools, reading through massive amounts of documentation, understanding the context.

Harry Stebbings 15:01 Does that mean the model doesn't need to be as good? Um, uh, again, I, I've learned to be incredibly blunt over time. If someone wants to criticize, they say, "Oh, well, if you look at models or evals, Cohere's not as good."

Nick Frosst 15:13 Well, there's a whole long conversation to be had about evals.

Harry Stebbings 15:15 (laughs)

Nick Frosst 15:16 But effectively like, you know, what we care about, um, is not the like discord, not the hype, not the discourse. Like what we care about is if a customer uses our model, gets a copy of our model from us, and they try to do something with it, we care that it works as easy as possible. Like that's what we optimize for. None of those are really reflected in the various benchmarks that, you know, up, that cycle through every year. Uh, and so we don't like focus too much

Nick Frosst 15:47 on that stuff.

Harry Stebbings 15:47 Do you think the benchmarks are bullshit? 'Cause we place a lot of emphasis on them in, on Twittersphere, on the Reddit sphere. Are they bullshit or- or are they a accurate reflection of model progress?

Nick Frosst 15:58 Let's go back in time a little bit. Um, when we first started in this industry, the benchmark that was used the most was called LM1B. Do you remember that benchmark?

Harry Stebbings 16:05 No.

Nick Frosst 16:06 Okay, cool. Uh, that was a benchmark that was like taking in the first part of a text, like of a, of a newspaper, and then writing the second part of a, of a, of the newspaper article. Um, after that there was a benchmark called Hella Swag. Do you remember that one?

Harry Stebbings 16:19 Yeah, I do remember that one.

Nick Frosst 16:19 All right, cool. That, so that-

Harry Stebbings 16:20 Yeah.

Nick Frosst 16:20 ... okay, so that's like 2022, so that's like-

Harry Stebbings 16:23 Tha- that's my introduction. Yeah, yeah, yeah.

Nick Frosst 16:24 That's when you started. All right, cool. Uh, no one's talking about that anymore, right?

Harry Stebbings 16:27 Right.

Nick Frosst 16:27 Yeah. Um, now a lot of people talk about like AIM as like a math reasoning AMI, or actually don't, that math reasoning benchmark.

Harry Stebbings 16:36 Okay.

Nick Frosst 16:37 None of our customers ask the model to do math reasoning. That doesn't come up in the workplace that often. That comes up in a few workplaces where mathematicians work, but there aren't a ton of people out there making a living doing math reasoning. Um, stuff like the ARC AGI Challenge is a benchmark that people talk about, but that's like a pixel manipulation challenge. It's like, you know, taking in like a grid of pixels and based on rules predicting the next one. That's not a thing any of our customers have ever asked the model to do, nor do I think they will. So do I think they're all bullshit? Um, I think taken hol- like I, I think, I think it's interesting. I don't know. There's, there's good scientific work in some of them. I think it's very interesting to evaluate emergent capabilities for models.

Harry Stebbings 17:26 But they're not an accurate reflection of the utility value of models.

Nick Frosst 17:29 They're a reflection of how much the model had been trained on those benchmarks.

Harry Stebbings 17:34 So you can gamify them, essentially.

Nick Frosst 17:36 Oh, you can definitely gamify them. Yeah.

Harry Stebbings 17:37 Do the big players gamify them?

Nick Frosst 17:40 I don't know. Yeah, I don't know. I, uh, none of it's too, like none of it's too relevant to us, right? Like I don't think those leaderboards are that helpful. I think in a consumer space, it's cool. I think if you're making a consumer app and it's like exciting and fun and people like to look at it and they want to try out the most recent thing, like that's fun. That's cool. Um, I think if you're not in a consumer space, uh, people don't care about the hype as much. They care about like, "Hey, did I get to production? Hey, like did, you know, did I buy LLMs, deploy them, and then get ROI on that?" You know?

Harry Stebbings 18:13 Gi- given the pace of deployment, we are seeing model evolution so fast and so rapidly-

Nick Frosst 18:18 Mm-hmm.

Harry Stebbings 18:18 ... that you're essentially seeing this kind of decay rate on models being greater than ever because it's like next one, next one, next one. And actually-They're still being trained, though, on H100s or NVIDIA chips from 18 months ago. Is there a misalignment in terms of the progression of models versus the progression of chips?

Nick Frosst 18:36 You can cycle through new versions of models quicker. I mean, it's still, uh, it's very slow. Still, like I, when I was training neural nets in 2000 and, I don't know, 11, I mean, it would take, like, you know, hours to days. I remember being like, "This is crazy. I can't believe this takes so long to train this model." Now we spend months, months training models. So like, you know, that's, that's a timescale, I, I, I didn't anticipate when I was working on this, um, a long time... when I was working on neural nets a long time ago. Um, but that's still very different than the timescale of working on chips, right? Like, that's still slow. I think when you talk about, like, we're, we're seeing all these models iterate so quickly, like yes, on the one hand we're seeing models iterate really quickly and people are releasing new models. On the other hand, there's still the transformer that was invented in 2017, and there's still sequence models, and they still take in words and predict the next le- word. And we've changed how they're trained a bit. Uh, we've added on steps, like now there's a, you know, base modeling step, then a SFT, like supervised fine-tuning from human feedback, where like if somebody writes a sentence, someone writes the sec- the response they want, and we train on that. And then there's a reinforcement learning aspect where the model is generating and you're telling it that's good, that's bad or something. So there's like new ways of training it. But fundamentally, the tech is still the same. Um, we keep making them better, keep iterating on them. But it's not as though we've like c- anybody has, you know, trained a, a model that's fundamentally different than a transformer. So, it's, it's an interesting dichotomy. Like on the one hand, there's constantly new stuff. On the other hand, eh, we've been working on the same stuff for a while.

Harry Stebbings 20:08 We have been working on the same stuff for a while. The thing that has seemingly changed is the value of the people working on this stuff. You know, we're now seeing billion-dollar people in terms of Zuck's willingness to pay for, like, chief scientists. You recently hired, um, and I just wanna get the name right, which is Joel Pineau? From...

Nick Frosst 20:27 Yeah, Joel. Yeah.

Harry Stebbings 20:28 Joel from Facebook.

Nick Frosst 20:29 Yeah, yeah.

Harry Stebbings 20:29 Or Meta. My question to you is, how do you think about the war for talent that we're seeing today?

Nick Frosst 20:35 Yeah, I, I, I think there's a lot of crazy headlines out there. I think there's a lot of people... I, I don't, I don't know how much of it is real. I know-

Harry Stebbings 20:43 You don't think it's real that Anthropic are paying 10, 15, $20 million for great AI researchers?

Nick Frosst 20:49 Oh, I, I have no idea. I know that there are lots of people who are adding that much value, you know, and there's lots of people who are, like, bringing that much value into the industry. It's a super, it's a super impactful industry, right? Um, so I know that there are people that are bringing that much value. Um, and I know that there's lots of brilliant people, and I know that this is a really, it's really demanding work. It's really hard work. It requires a lot of experience, a lot of, uh, ingenuity, um, and a lot of dedication. So, I think it's a good place for, for people to be spending their time, and I, and I'm, think it makes sense that many of them are rewarded very well. That being said, like when I see the stories of, you know, Meta hiring people for like 100, you know, million, like I, I, I read as many stories of those as I read of people leaving the next day. So, I don't, I don't know what's going on over there. Um, I know that people like to work in a place that is like, that gives them, uh, stability, and people like to work in a place that gives them, like, purpose and is value-aligned. Um, and they, they like to work in places that make them feel good about what they're doing. And part of that is compensation,

Nick Frosst 22:02 but not all.

Harry Stebbings 22:02 Do you sit down, though, with Aidan and the team and go, "Shit, we need to step up what we pay people because we are in a war for talent, and budget is a big part of it"?

Nick Frosst 22:11 Mm-hmm. We, we definitely think, I mean, we, our, like our company is, um, you know, what, what we spend the most time thinking about and talking about. Um, and the company is the people who are there, you know. That's fun- functionally like we, we, we are only the people who we have the privilege of working with. Um, and so we do think about, hey, is this the right place for... Like, are we making sure that this is the right place for people to work? Um, are we making sure that this is getting the best in their lives, uh, in a financial perspective and in a personal...

Harry Stebbings 22:38 But would you spend five million on an AI researcher?

Nick Frosst 22:41 I mean, if they were bringing in the right value, yeah. I mean, like there are people who are... There are certainly lots of people who, through our equity, like own portions of that, you know, own, own what you're talking about. Um, and I feel great about that.

Harry Stebbings 22:56 Do you worry that the industry's kind of becoming commoditized or transactionalized with the hype around it?

Nick Frosst 23:01 I don't like... Yeah, I, I do think the hype around it is misleading sometimes.

Harry Stebbings 23:06 Mm.

Nick Frosst 23:06 Like the technology... Like I, I'm in such a strange place of being caught between the technology is the most beautiful technology I've ever seen. It's the most transformative technology, uh, I've ever worked with. I'm... Like, it, it is already fundamentally changing the way I do work, and I'm very sure it will fundamentally the change, change the way we all do work soon. Like, I think that. On the other hand, there's a lot of hype around it. There's a lot of misleading rhetoric. Uh, there's a lot of misinformation. And I don't think the hype is necessarily helpful for getting to the truth.

Harry Stebbings 23:44 Can I just dig in there?

Nick Frosst 23:45 Yeah.

Harry Stebbings 23:46 What do you think is the hype and misleading rhetoric that is most damaging or confusing?

Nick Frosst 23:52 Yeah, I think the hype around AGI is the most damaging and confusing.

Harry Stebbings 23:56 This assumption that we will all have no work to do, we'll all be in UBI and...

Nick Frosst 24:01 Yeah. Yeah, and even bef- I mean, this is kind, this isn't really in the discourse as much this year as it was last year and the year before that. Um, and that's because it's pretty clearly not true. But the idea that, oh, this technology is, like poses an imminent existential threat to humanity writ large, um, was not a, wa- was incorrect, um, and not helpful for talking about the real ways in which this technology...... could be damaging, the real ways in which this technology, uh, could, could shake up the system and, and cause, you know, rapid changes. Um, and it, it's, was not helpful for getting people to understand what the technology is, right? So I, I don't, like, I don't hear that as much anymore these days. Um, I think that's because people have realized that that's not the case. But the remnants of that discourse are still in the, in the world, are still out there, yeah.

Harry Stebbings 24:59 I think the remnants are, and I think they're most prevalent in the way the internal employees in large organizations respond to AI being introduced. People do not welcome the introduction of AI in large companies. They-

Nick Frosst 25:10 Yeah.

Harry Stebbings 25:11 Maybe a European sign, but a lot are very nervous and scared and do not embrace it wholeheartedly.

Nick Frosst 25:16 Interesting. Yeah, I think we, we, I haven't found that as much, um, when we've worked with our customers in our enterprises. Um, I, I find a lot of people are, are interested and excited about using an LLM. Mostly that's because, I think, they realize that the LLM is augmentative, um, for the most part. And it, it allows them to not do the things they don't want to do.

Narrator 25:39 Yeah.

Harry Stebbings 25:39 I mean this in a nice way. Do you actually buy that?

Nick Frosst 25:42 Do I actually buy that? (laughs)

Harry Stebbings 25:42 Yeah, like ni- nice, yeah, yeah.

Nick Frosst 25:44 I love it. Yeah.

Harry Stebbings 25:44 I mean, I had Benioff on the show from Salesforce, like-

Nick Frosst 25:46 Yeah.

Harry Stebbings 25:47 ... two days ago, and he says, like, "Oh, the same human plus agent, yeah, great."

Nick Frosst 25:50 Yeah, yeah.

Harry Stebbings 25:51 Are you serious?

Nick Frosst 25:52 (laughs)

Harry Stebbings 25:52 Most 25, 26-year-old marketing managers or SDRs, I'm sorry to say it, they're not brilliant. They do not love the craft. They are not better than a phenomenal agent will be in the next 12 months. They will be replaced.

Nick Frosst 26:09 Oh, no, yeah, I, I actually believe that. Yeah.

Harry Stebbings 26:12 You, you think?

Nick Frosst 26:12 Yeah, I believe what I said, sorry. Yeah, yeah. Yeah, yeah, no, I, yeah, I'm not, uh... Yeah, yeah, I fundamentally believe that this technology is... And, and I think this is clear if you use it f-... You know, use it for a while, um, that there are things that it's way better at you, better than you at. Um, but there are still lots of things that people are better at. Like, LL... Look, LLMs are incredible. People, people have been using them, you know, for years now. There has been no independent breakthrough that an LLM has made, right? There has been no, like, nobody has seen, "Hey," nobody asked an LLM, "Hey, solve this problem no one's solved before and, and get the, the answer." The breakthroughs are still people.

Harry Stebbings 26:58 Is that not only a matter of time?

Nick Frosst 27:00 No.

Harry Stebbings 27:00 It's not?

Nick Frosst 27:01 No. That's not a matter of time. That's, that's fundamentally the, the way that sequence models work. Like, when you're training statistical models of text, they're phenomenal, they're incredible, and they, they are capable of generalizing across unseen tasks, which is why they're so useful. But the tech... Yeah. When you're talking about, like, the 25-year-old marketer or something, th- their, some portion of their work is, like, writing text on a computer. It's like all the information's out there. They have this document and that document, this tool, that API. They just need to take that and turn it into another form and like, you know, combine it, and then put it out there. Like, that's the work. That's some portion of it. That's not the majority of it. Most of it is talking to people, like understanding the culture, understanding the zeitgeist, understanding like what's gonna hit, what's relevant, using their intuition and their human experience to understand how they can be helpful or what they can do. And that is not in the data set of text from the internet.

Harry Stebbings 28:06 (exhales) (laughs)

Nick Frosst 28:06 So, yeah, I, I, yeah, I, I, I fundamentally-

Harry Stebbings 28:09 I, I, I disagree-

Nick Frosst 28:10 Yeah.

Harry Stebbings 28:10 ... 'cause you know what they do now? They go, "Hey, I'm running a campaign for Aviana. Come up with three different storylines that would be cool for us to do. Make it relevant to new cycles today-"

Nick Frosst 28:23 Yeah.

Harry Stebbings 28:23 "... is the prompt." And then they come up with three, and they're like, "Oh, that one's pretty good."

Nick Frosst 28:28 Yeah. And I think that sounds like a good usage of it as a starting point, and then I'm sure it is worked on. And some of those are thrown out because of things they understand. And some of them are like, "Oh, that's a great insight. I'm gonna run with that. I'm gonna tinker that," like what you just described is a good use case. And I can imagine as a jumping-off point, that would be helpful. But that's not where the work ends. Like, that's the beginning of the work. You've just au- you've just augmented, you know... You're now starting with, instead of a blank page, you're starting with things to go for.

Harry Stebbings 28:56 So, you don't mind that we're gonna see this dramatic reduction in team sizes?

Nick Frosst 29:00 I think we're gonna see, um, I think we're gonna see changes to the, to the nature, like to the workforce. In the same way that we saw changes to the workforce when the computer was created, when the personal computer, when the internet was created, um, when the printing press happened, right? Like, we, we've seen, uh, like in the Industrial Revolution, right, like we've seen drastic changes to the workforce. And that's gonna keep happening.

Harry Stebbings 29:23 What will the changes be? Like, what will a company look like, do you think, in five to 10 years?

Nick Frosst 29:28 I think you will arrive to work, and you will sit in front of a computer. Um, and you will predominantly use language to interact with that computer. And anytime there's something to do that you know can be done, and you know, like, the information is out there, doesn't require, you know, creativity or insight, you know that it's there. Um, you just need to do it, and doing it's kinda boring. You will get the model to do it for you. And I think that looks like, uh, that mostly looks like sitting down, speaking to the computer, getting it to do the things you don't want to do, and then spending your time talking to other people, thinking about how it can be useful, whether or not what it did was good. And I, I think that shift is, is, is maybe chaotic. And I would like us as a world to be spending time thinking about, you know, how can we make sure that change is as easy as possible? How can we make sure that making language models allows people to do the stuff that they're good at and that they like? Um-... how can we make sure, like, the labor force is resilient? Uh, how can we make sure, you know, that income inequality doesn't go up as a result of that? Like, those are the types of things I would like us to be talking about. And those are the important things. And to go back to your earlier question, like, when talking about, you know, the a- risks of AGI, like, I think those existential threat questions, um, made it harder to talk about the real things, you know, like income inequality.

Harry Stebbings 30:56 Do you think AI does more to help or to hurt income inequality?

Nick Frosst 31:00 I think it depends on policy. You know? I think if there's good labor policy, I think it could help. I think if there's bad labor policy, it could hurt.

Harry Stebbings 31:09 Can you explain that to me?

Nick Frosst 31:11 Look, when you saw the last industrial revolution, broadly speaking, everybody looks back on that industrial revolution and says that was a good idea, right? Like, nobody's saying, "Hey, we shouldn't have automated," you know, what was before the industrial revolution, it was like, I don't know, 90-something percent people working in farms. Now it's like, I don't know, five, I don't know what the percent-

Harry Stebbings 31:31 Yeah.

Nick Frosst 31:31 Small. Five, less than that. Um, everybody thinks that was a good idea. It was a crazy time, and if you read stories about what went on during that moment, there's lots of things that people did that w- they stopped doing pretty quick, like having kids work in coal mines, right? Like, that was a crazy thing. And out of that industrial revolution came a whole bunch of really good labor policies, you know, came unions, came, like, you know, workers' rights, came, like, things that I think we also think were a good idea and resulted in not only, you know, better lives for people, but in actually more productivity. Like, actually a better economy, actually, you know, a better world. Um, a lot of those were, were from public policy. A lot of those were from things created, uh, in unison between, you know, businesses and, and governments.

Harry Stebbings 32:22 Why do we need to have such significant policy change if it only augments humans and it doesn't replace them?

Nick Frosst 32:28 Well, I think right now, um, right, right now, we've seen income inequality go up over the past, you know, several years.

Harry Stebbings 32:37 Yeah.

Nick Frosst 32:37 And a lot of that was happening before AI, before language models were popular. Um, and I'm worried that technology has the potential to exacerbate that without being deployed correctly and without having good, uh, yeah, without having good policy around,

Nick Frosst 32:59 around employment.

Harry Stebbings 33:00 Can I ask you, when we think about, like, problems to solve, I think a lot of people also get worried about the open versus closed argument.

Nick Frosst 33:07 Mm.

Harry Stebbings 33:07 Um, how do you feel about where the future of efficient AI lands in the balance between open versus closed models?

Nick Frosst 33:16 Mm. So at Cohere, we, we make our foundational models, and then we release the weights for non-commercial usage. So, we're somewhere in be- in the between the, like, open and closed, right? We're a for-profit company. Like, we, we exist to make money. Um, and so we don't, we release our weights for scientific and research and, like, you know, you can download it on your computer and run it. Um, I think that's a good sweet spot for us as a business. That allows us to, like, you know, build credibility within the community. If people want to check out our weights, like, they can go check them out, right? Like, there's lots of companies that started out as open, um, who no longer release the weights of their models, or who never did, right? So, we have our models out there. You can go look at them, you can use them, you can validate, "Hey, do they work on my problem? Yes or no." Um, but if you're using them for commercial purposes, you gotta talk to us, and then-

Harry Stebbings 34:06 (laughs)

Nick Frosst 34:06 ... and then we figure out a commercial relationship so that we can, you know, exist as a, as a business. That works for us. Um, I'm surprised to s- I'm surprised there aren't more businesses taking that tact, uh, more foundational models taking that approach.

Harry Stebbings 34:20 Do you think Meta will move to a closed model approach from an open?

Nick Frosst 34:24 They've certainly hinted at that, right? There certainly looks like... But I don't know what they're, I don't know what they're doing over there. Yeah, I don't, I don't think a lot of people know what they're doing over there, and I don't spend a lot of time thinking about it.

Harry Stebbings 34:34 Do you not think it's helpful for founders to be very aware of competitive landscapes in case they're asked about them by customers, in case customers are going, "Hey, w- why aren't you more open? Why aren't you more closed? Are we, is our data secure-"

Nick Frosst 34:47 Yeah.

Harry Stebbings 34:47 "... if you're..."

Nick Frosst 34:48 This, as in, as with most things, like, a middle ground is the right place to be, right? You could spend your whole time as a founder only looking at competitors and being like, "Oh, why are they doing that? Why are they doing this? What's going on with that?" You know, you know, um, and that will, I think, not be helpful for you. You could also spend your whole time, you know, with your head in the sand, only thinking about what's going on in your company, and I think that would not be helpful either. You have to find some middle ground. The discourse around AI is inescapable. You know, you, you, you would be hard-pressed to ignore it. It is every other headline, you know, it is, it's all over the place. I don't think there are many people who work in the industry who suffer from not enough information about what's going on in AI, right? I think there's a lot of people who suffer from way too much of it and obsessing over the minute details of like, you know, how, w- so-and-so got 0.2% better on this thing, or like, you know, is, you know, constant min- like, constant small changes in businesses out there. Um, and I think that can mislead you from staying grounded and, like, what are you actually doing? Who are you actually helping? How is this making, you know, things

Nick Frosst 36:01 better for your customers?

Harry Stebbings 36:02 Do you think we will still have prompting as the core user input guidance mechanism in five years' time?

Nick Frosst 36:09 Prompting as in, like, you write something to a model and it writes back?

Harry Stebbings 36:11 Y- yeah.

Nick Frosst 36:12 Yeah. Yeah, what else would it be?

Harry Stebbings 36:14 The way that it changes, the way that you do it changes. You wouldn't say, "Hey, make it a funny tone," um, or, "Hey, add in a light personalized style that also is sincere."

Nick Frosst 36:27 I think the idea of prompting as a skill will become less relevant. And if you look at, like, that's the trajectory. Like, when I started doing this, if you wanted to get a model to summarize something, you wrote the first paragraph, and then you wrote, "In summary," colon, new line, and then you'd generate it.And like that was, like that was the skill of prompting, was figuring out how to trick a model into getting it to do what you want it to do. And that's because they weren't trained on feedback from people, they were only trained on text from the web. And so all they were were sequence models from text on the web, and nobody on the rep, on the web wrote, like, "Please summarize this for me," and then a summary. They wrote a paragraph and then wrote, "In summary:" and so if you wanted to get the model to do that, that's what you would do. Language models are like, ff- w- we're training them more to fit how people expect them to work, and that means that getting good at prompting is less important. So I think the idea of saying, like, "Oh, yeah, you gotta learn how to prompt" is gonna go away. I think the idea of saying, "You need to learn how language models work and you need to know what they can and can't do," in the same way you had to learn how a computer works and what it can and can't do, and you had to learn how a telephone works and what it can and can't do, like I think that's going to exist. But, um, and, and that means, like, prompting is gonna exist. The idea that, you know, you write to a, you write something to a model or you say something to a model and then you get the response back, and if it's not what you like, maybe you iterate a little bit, like that's gonna exist. That's fundamentally how the technology works.

Harry Stebbings 37:53 Mm-hmm.

Nick Frosst 37:53 But the idea of it being a discipline that you have to train to do, um, we've already seen that trajectory get, uh... Yeah, it's al- it's already gotten easier. I, I look for people who know how a language model works, and who know... One of the, one of the core things that's, you know, one of the things that's been, uh, a necessary component about working at Cohere, uh, is you can't think the technology is magic. You can't think, uh, th- this is like we're doing spells. You have to know that a lang- how a language model works, how it's trained, and what that means for it. What emergent capabilities happen, uh, which ones don't. You can't think, "Oh, yeah, I just ask the digital god to do my work and then it does." Like, that's not what the technology is. And thinking that will not help, will not help you build it, and it will not help you

Nick Frosst 38:43 use it.

Harry Stebbings 38:43 If we, like, hone in a little bit more on you, you led all, w- a large part of the latest fundraisers we were chatting before. Um, when you think about the fundraising journey, how was that journey? And there, are there any big lessons from, it was 600 million, no?

Nick Frosst 38:57 Uh, yeah, so I, I, but I, uh, fundraising at Cohere is a lot, lots of people are involved in it. I by no means led the, led the, uh, the efforts. But I, yeah, I was, I was involved in it. Um, and I, uh, yeah, enjoyed talking to people about the tech and what we're building. Um, yeah, I actually quite, I quite like talking to, to VCs, um, and to pension funds and to people. I think it's f- I love Cohere, I love what we build, and I also like talking. So I like- (laughs) I like talking about both of those-

Harry Stebbings 39:25 Were their any questions very similar?

Nick Frosst 39:27 Oh, between them?

Harry Stebbings 39:28 Yeah.

Nick Frosst 39:29 Yeah, actually. That's an interesting question. Yeah, I, I do think, look, the industry's a lot more mature. Like, two years ago, you know, when we were fundraising, or three years ago, a lot of the questions were like, "What is this? Like, what, what, how are you gonna, what? How does that work? What is this?" You know, and so we'd spend more time explaining stuff. Mm, people mostly know how it works and know what it does. And now we can say, "Here's what we're doing for our customers specifically." You know, like, "Here's how RBC is using it. Here's what we're doing with Fujitsu. Here's what LG is doing with it." You know, like, we can talk about those things specifically, um, and that's, that's more interesting.

Harry Stebbings 40:00 How much of the 600 million will be spent on compute?

Nick Frosst 40:03 Oh, yeah, there's like three components that go into making language models, right? There's, there's talent, is like people, it's um, engineers and researchers. There's compute, um, and there's data. Uh, the importance of those has shifted, and they, yeah, the spend of those has shifted over time.

Harry Stebbings 40:22 Mm-hmm.

Nick Frosst 40:23 Um, we train very efficiently, right? We train models, uh, we train efficient models, so like our, our model Command-A and the Command-A reasoning model, which we just released, Command-A vision model, which we just released. Um, those, they're all trained to fit on two GPUs. That's like a really important part of our business strategy. It turns out if you go talk to a lot of companies who wanted to deploy models into production, they were bottlenecked on deploying because they don't have enough GPUs.

Harry Stebbings 40:48 Huh.

Nick Frosst 40:49 Two GPUs turns out to be like a sweet spot between performance and a- uh, performance and cost, and like ha- and actually how many GPUs they had access to. Um, so that means we train very efficiently as well, you know. We have dr- spent orders of magnitude less on creating foundational models than some of the other foundational model companies out there. Truly orders of magnitude less. Um, and I'm very proud of the efficiency of the team and like what they've done with the, you know, the resources that they have. Um, so we, we think about efficiency a lot for ourselves and for our customers, and those two things are, are related. Um, but how much of our funding goes to compute? It shifts over the years, um, but a lot. You know, it's, it's, compute is, is a, it's, it's expensive.

Harry Stebbings 41:33 Ho- how has it shifted over the years?

Nick Frosst 41:35 I mean, when we first started Cohere, one of the very first things we did, uh, because we had no funding, or, n- we did have funding, but a very small amount of it, um, was we, we spent next to nothing on compute. And we showed that you could train a model by having, like, a bit of a GPU over here and a bit of GPU over here, bit of a GPU over here, and you could link them together, and we published a few papers on that, uh, on training models with, like, the scraps of GPUs in data centers (laughs) , right? That was what we started with. Um, and we showed that you could do that. You can do that. Uh, it's very slow, and it's much easier to just rent a big data center and train the model there. Yeah.

Harry Stebbings 42:12 The question that everyone asks is, how do you compete against competitors who have billions and billions of dollars? Do you hate that question, and how do you respond to it?

Nick Frosst 42:21 No, I don't hate that question. I think, yeah, I think that's a fine question. Um, you know, uh, we've, we've announced funding rounds, um, and you can see that they are smaller than some of the other f- funding rounds out there. Um, yeah, I mean, like, w- we're pretty singularly focused in a way that the other companies who build foundational models, uh, are not, right? Like, we don't have a consumer app. We're not trying to get anybody to spend $200 a month on something for their personal lives.... were singularly focused on working with enterprises and businesses and making sure that they get to production with AI. I have my whole... I, I'm constantly telling people like, "Not AGI, ROI. ROI, not AGI." Yeah, um, and, you know, it turns out there's a lot of work that still needs to get done there. And it turns out there's a lot of companies that tried to go to production with AI by using something-

Harry Stebbings 43:17 But do you, do you think then that OpenAI and Anthropic will just cede enterprise?

Nick Frosst 43:21 I think, like, right now, you know, those, both those companies have a, are, are pretty cool. They've both made good consumer products. Um, I think where this technology, like, adds the most value is in work for, like, personal reasons. Like, that's where I see this technology being the most useful. Um, I don't know if they'll, if they'll start working on that. Um, I know that making models that work in that environment is pretty different than making a model that works in a consumer environment. In a consumer environment, you can make the biggest model possible. You can have, like, complicated switches to tell you to go to this model or that model because you're h- you're just posting it on a huge amount of GPUs. You can be, like, losing a ton of money on every inference call, but you know, you're getting, you're getting users and something, and so, like, that works. The types of models you have to build to succeed there are different. Um, I know the work you need to do on the interface, like, we know we've, we've announced North, which is our agentic framework. It's privately deployable, customizable, you know, for, uh, knowledge work- workers within an enterprise. It's pretty diff- it looks pretty different than, uh, some of the consumer applications, right? Like, a big one is, like, our model doesn't generate images. Nobody in a workforce is really wanting to generate images as part of their work, for the most part, right? It doesn't... But as a consumer, it's very fun. It's very cool to be like, "Oh, here, give me a picture of this or something." So, we... The types of models we, we train are different, and the interfaces we make are different. Um, I don't know if that's... if they'll be interested in that at some point. I think, like, we stay focused on talking to customers and adding value.

Harry Stebbings 44:53 How do you price?

Nick Frosst 44:55 Um, that's entirely dependent on what the customer wants to do with us. So, we do have some customers where we give, like make a custom model for them and give them that model.

Harry Stebbings 45:01 So, do you have forward-deployed engineers?

Nick Frosst 45:03 We do, yeah. Yeah, we have-

Harry Stebbings 45:05 Okay.

Nick Frosst 45:05 ... forward-deployed engineers. Yeah. And they're very... Yeah. So, they're a crucial component of how we, like, go get a company up and running and into production with us.

Harry Stebbings 45:13 Do you think everyone will have forward-deployed engineers in a future AI world in a way that Palantir has glamorized?

Nick Frosst 45:19 Yeah. I mean, I think forward, forward-deployed engineers are a good idea, right? Like, you're selling technology to somebody, it makes sense to have some engineers who ha- come and help them get it set up and work with them to, like, make sure it's actually delivering value. You know, I think that's a good idea. Um, I don't know if that's true for every business, you know, but...

Harry Stebbings 45:35 Does FDEs not just allow for poorer technology?

Nick Frosst 45:39 No.

Harry Stebbings 45:39 And what I mean by that is like-

Nick Frosst 45:40 No, no, I think that there's this, there's this idea, there's this idea sometimes like, "Oh, yeah. You can just make the thing, and for every business, it'll be, it'll work perfectly and require no engagement." And like, that's not the way a- that's the way some technology works. That's the way a lot of consumer technology works. That's not the way a lot of enterprise technology works, right? Like, there's... You're selling things to people, um, that are, that have to be, like, matched to the way their business is set up, right? And so having engineers go along with it and say, "Cool. Here. Here's, here's a model. Here's what we can do to make sure that that's perfect for you in your specific use case," is helpful.

Harry Stebbings 46:17 Given that you sell to enterprises, I, I'm an enterprise investor and-

Nick Frosst 46:20 Mm-hmm.

Harry Stebbings 46:20 I love enterprise. Revenue quality is much higher, much-

Nick Frosst 46:23 Mm-hmm.

Harry Stebbings 46:23 ... stickier. But growth is slower-

Nick Frosst 46:26 Mm-hmm.

Harry Stebbings 46:26 ... because you're working with large enterprises.

Nick Frosst 46:27 Mm-hmm.

Harry Stebbings 46:28 Do you think you had an, an, have an enterprise discount applied to valuation because of revenue growth being slower because of enterprise?

Nick Frosst 46:37 Hmm. It's an interesting question. Um, maybe. Yeah. I mean, our, you know, I'm, you know, I'm very proud of what we've done and what we've created.

Harry Stebbings 46:45 What was the price on the last round? It w- it was public. I think it was like 6.7, wasn't it?

Nick Frosst 46:48 Yeah.

Harry Stebbings 46:49 Yeah.

Nick Frosst 46:50 Yeah. 6.8.

Harry Stebbings 46:51 6.8?

Nick Frosst 46:51 Yeah. Yeah, it's- it's- those, these are all staggering numbers. These are all numbers that are impossible for an individual to conceive of. Like they were, were, were so far into that, you know, as a, as a single individual, this is well beyond the realm of what you can reasonably engage with in your life. Um, so yeah.

Harry Stebbings 47:06 Is it?

Nick Frosst 47:08 For a regular person who grew up working as a cook, y- like, yeah. Right? Like, my first job was burgers, you know? (laughs)

Harry Stebbings 47:19 (sighs) Okay.

Nick Frosst 47:20 But yeah.

Harry Stebbings 47:20 (laughs)

Nick Frosst 47:20 But yeah, that's a great... You know, these are all crazy numbers. Yeah.

Harry Stebbings 47:23 Do you care about money?

Nick Frosst 47:25 Yeah, I certainly. I think everybody cares about money. Um, and everybody's motivated by money.

Harry Stebbings 47:30 With being motivated by money, y- you're an incredibly acquisitive asset, and it's been a very strategically important thing for large players to do. Have you had M&A offers across the journey?

Nick Frosst 47:44 Oh, yeah. We have at times. Yeah.

Harry Stebbings 47:47 How's the decision-making gone, though? I always want to be in the room. I always picture it-

Nick Frosst 47:50 It's-

Harry Stebbings 47:50 ... kind of like thundery nights-

Nick Frosst 47:52 No.

Harry Stebbings 47:52 ... and people coming-

Nick Frosst 47:53 (laughs) .

Harry Stebbings 47:53 ... together where it's raining outside and-

Nick Frosst 47:54 No, no, no, no, no. Yeah, we have. I mean, we, look, we've been a company for five years. Um, yeah, we have. Um...

Harry Stebbings 48:02 Not tempting?

Nick Frosst 48:03 W- we're all really interested in... We, we're all being like the, the co-founders, and now, you know, the people who work here, we're all... A lot of things that you'll hear Aidan say is like, "Oh, building a generational company." You know, we're all really interested in building something that outlasts us and that goes, goes beyond, you know, our involvement of it. That's really exciting.

Harry Stebbings 48:22 Why do you want that? I know it sounds strange. Why do you want something that outlasts you?

Nick Frosst 48:26 Oh, yeah, that's a good question. (laughs) Remember earlier when I was like, oh, sometimes I can ask philosophical questions and then we deviate too far-

Harry Stebbings 48:32 Uh-huh.

Nick Frosst 48:32 ... off the thing? And this is one of those questions. (laughs)

Harry Stebbings 48:34 No, it's cool. I, I-

Nick Frosst 48:35 Yeah.

Harry Stebbings 48:35 ... love this.

Nick Frosst 48:35 Yeah. Um...

Harry Stebbings 48:37 This is why the people say like, "Oh, AI could just replace you as an interviewer, Harry." It's like, "No, it couldn't, because it doesn't have the ambiguity to go off on the like..." But why does that actually matter?

Nick Frosst 48:49 Well, then if you think what you just said, why do you not think that that exists for all jobs?

Harry Stebbings 48:54 Oh, 'cause I think what I do is a very disciplined art, honed over 10 years, uh, compared to a social media manager who's 24 coming out of university writing with thrill to publish our latest report.

Nick Frosst 49:05 That's, uh, everybody thinks that what they do is, is like honed and trained.

Harry Stebbings 49:11 Then why is mine paid millions-

Nick Frosst 49:13 And-

Harry Stebbings 49:13 ... and theirs not?

Nick Frosst 49:15 Oh, well-

Harry Stebbings 49:16 Because society places strategically more value on mine, if we're being a dick and blunt, and I'll take this out, because very few people can do it.

Nick Frosst 49:27 There's labor that's easier to work, that you can learn faster, or, like, you can get up to speed quicker, um, and there's things that take a really long time to do. And, like, the only way you're gonna be able to do that job is if you spend a really long time doing it. Um, and there are some things that are harder and some things that are more agent, like, uh, that have more agency. Um, and our economy is, you know, decent at figuring that out and compensating people based on, on the, on the investment that they had to make and the skills they had to have to get there. But I don't think it's perfect and I don't think, and I think everybody thinks and should think that the work that they do is a skill that they learned. Um, and a lot of it is. And a- and even work at, like, some of the hardest jobs, some of the hardest em- days I ever had at work was, like, working at the grill making breakfast and burgers for people when it was super chaotic and, and stressful, and there was no air conditioning 'cause it had broken. And, like, I had to run across the street to, you know, buy extra potatoes 'cause we hadn't prepared it, right? Like, that was challenging and rewarding work. Um, and I don't think just because I was getting paid minimum wage at the time means it, it w- it wasn't valuable.

Harry Stebbings 50:42 But it is.

Nick Frosst 50:42 (laughs)

Harry Stebbings 50:43 It, it, like, it is definitively less valuable.

Nick Frosst 50:46 It's, it's definitively paid less.

Harry Stebbings 50:49 No, uh, there, like, Adam Smith's invisible hand would suggest that it is just definitively less valuable that you went and got, like, more potatoes, which meant more chips-

Nick Frosst 50:58 Ah.

Harry Stebbings 50:58 ... for someone that didn't probably need more chips in a grill that was of single consumption model.

Nick Frosst 51:05 Yeah, uh, I mean- (laughs)

Harry Stebbings 51:08 (laughs) Like, I'm really sorry, dude.

Nick Frosst 51:09 No, no, no, like, I under- I understand your perspective, yeah.

Harry Stebbings 51:11 Versus the work that your team do-

Nick Frosst 51:13 Yeah.

Harry Stebbings 51:14 ... which, which has an impa- impact on thousands of employees in some of the biggest companies-

Nick Frosst 51:18 Yeah.

Harry Stebbings 51:18 ... in the world-

Nick Frosst 51:19 Look, I'm glad I do the work I do now-

Harry Stebbings 51:20 Or, w- e- exactly.

Nick Frosst 51:21 ... as opposed to doing other work before.

Harry Stebbings 51:21 Which has downstream impact on millions of Fujitsu users or-

Nick Frosst 51:24 Yeah. Yeah.

Harry Stebbings 51:26 It is legitimately, definitively less impactful and less valuable work.

Nick Frosst 51:30 Yeah, aga- again, like, as with, we're a few times over this, like, there's some extreme that says, "Hey, the invisible hand is absolutely accurate," and, like, whatever you're paid is exactly as much value you're creating and exactly as much value as you're worth. And then there's another side of it that's like, "Oh, yeah, whatever, everything's the same. Who knows? Who has any idea? It's all the same." Right? There is some middle ground.

Harry Stebbings 51:48 So you're going back, why does it matter that you have a generational company?

Nick Frosst 51:51 (laughs) After we detoured into-

Harry Stebbings 51:54 Yeah, seriously.

Nick Frosst 51:54 ... Adam Smith and the invisible hand.

Harry Stebbings 51:55 Yeah.

Nick Frosst 51:55 Yeah, um, one thing that comes to mind is, like, you know, like, look upon my works and despair, like, noth- you know, Ozymandias and the idea of, like, people obsessing over their legacy and building, you know, you know, some statue to their grander. And, like, one day, it, it will also fall, right? Like, one day, it, all that will be left are two legs in the desert, you know? Like, that's true, um, and that's true regardless of what you build. Like, that's true at, at some point. Um, but when I think about building a, like, what excites me about building Cohere, um, and when I say, like, a generational company, um, I mean timescale generations. I don't mean, like, my generations, you know? Um, I mean the idea of building something that is there for a long time. Um, it's rewarding, and I think it's inherently human. You know? Like, I think we all like to think about what are we building, how long is it gonna be there, and whether that's, like, a work of art or an actual building or a company or a philosophy or an idea, the idea of building something or participating in the construction of something that is bigger than you is rewarding.

Harry Stebbings 53:00 Totally agree.

Nick Frosst 53:00 And is, like, fundamentally human even though at some point, yes, it will be two feet in the desert. Both those things are true, you know? It's, it's rewarding and exciting and ultimately, you know, uh, as with all things.

Harry Stebbings 53:14 What was the biggest disagreement that you and Aidan have had?

Nick Frosst 53:17 We had some diff- disagreements about, like, API designs. (laughs) Like, that was a, there was a brief moment, like, before RLHF where we were talking about, like, oh, we should make an end point for summarization, an end point for entity extraction or something, and I think we disagreed about that. So we disagreed on, like, some low level stuff. But beyond that, you know, like, I've had the, the privilege of getting to work with, yeah, both, both Aidan and Ivan, the other co-founders, like, I, you know, I have a huge amount of respect for. And we definitely disagree and argue about, like, the, the mi- the little things about how to run the business or should we do this or sh- you know, should we make that policy? But there hasn't been any, like, huge-

Harry Stebbings 53:52 So I was d-

Nick Frosst 53:53 Yeah.

Harry Stebbings 53:53 I was chatting to Arvind from Perplexity, uh-

Nick Frosst 53:57 Yeah.

Harry Stebbings 53:57 ... a month or so ago.

Nick Frosst 53:58 Yeah.

Harry Stebbings 53:58 And I, I, uh, actually chatting to him, this sounds so, like, uh, behind the scenes, you know, like, our bro.

Nick Frosst 54:03 Yeah, no, no. Yeah.

Harry Stebbings 54:03 On stage in front of 4,000 people or whatever it was. (laughs)

Nick Frosst 54:06 Yeah.

Harry Stebbings 54:06 It wasn't a private conversation.

Nick Frosst 54:08 Yeah.

Harry Stebbings 54:08 And I said to him, um, "It feels to me like you, Sam, Dario, all leaders of foundational model companies, you're basically just kinda like presidents who sit on top of the machine kind of shouting views-"

Nick Frosst 54:22 Mm-hmm.

Harry Stebbings 54:23 "... because we're in a shouting views world, and then everyone in the machine does the work." And he was like, "That's exactly what we're all doing, yeah."

Nick Frosst 54:30 Wow.

Harry Stebbings 54:31 You know, "Me, Sam, Dario, uh, you, you name the leader, we just have to be, like, in front of every camera, doing every interview, basically espousing the views of the organization constantly because it is so important to be front and center and relevant today." Do you feel that Cohere is telling your story enough publicly?

Nick Frosst 54:52 But they're all consumer companies.... right? Like they all, they all fundamentally make money via subscriptions from consumers.

Harry Stebbings 54:59 Are they anthropic quasi? I mean, you'd-

Nick Frosst 55:01 Nah.

Harry Stebbings 55:01 ... say majority's enterprise.

Nick Frosst 55:03 I would say a lot of it's most, uh, most is like API calls from coders, right? Like that's-

Harry Stebbings 55:06 Mm-hmm.

Nick Frosst 55:06 ... a huge amount of their... So like, you know, maybe it's like I think a whatever, 50% Cursor or something, but then now comp-

Harry Stebbings 55:11 Mm-hmm.

Nick Frosst 55:12 Yeah. Competing with Cursor. Um, but a lot of it's still like comes down to an individual decision of a, of a-

Harry Stebbings 55:16 Sure.

Nick Frosst 55:16 ... consumer. Yeah. Um, so there, I think I understand their motivation. Like yeah, if you're, if you're selling to consumers, you wanna be telling a story. Consumers are really interested in that story right now. Like, yeah, them- that kinda makes sense for them. That's not what we're doing. We're not selling... You can't, you can't spend $200 a month on Cohere as a person. Like we, we don't have that offering. Um, so I don't... So it's not as important for us. Like could we be doing a better job, you know, telling what we're doing and, um, when you, you know, when, like, when I'm, like, asked to come on here, like, yeah, you know, I- I'm excited to come, I'm excited to talk to you. I like talking about Cohere. I think it's important. Do I think it's the most important thing? Like, no. I think building a product is the most important thing. I think solving problems for our customers is the most important thing. I think making a better model for them is the most important thing. Telling our story-

Harry Stebbings 56:03 Do you think that's more important than discussing Adam Smith's invisible-

Nick Frosst 56:06 (laughs)

Harry Stebbings 56:08 ... hand with me?

Nick Frosst 56:08 (laughs) Yeah, I do. (laughs) I do.

Harry Stebbings 56:11 How dare you?

Nick Frosst 56:12 No, I- no, I did enjoy it. Yeah, yeah.

Harry Stebbings 56:13 (laughs)

Nick Frosst 56:13 But, uh, yeah, I do. Yeah.

Harry Stebbings 56:14 Tha- that's

Harry Stebbings 56:15 hilarious. Uh, uh, one, one area we haven't covered, which is interesting, is the area of like sovereignty. We're sitting in London now.

Nick Frosst 56:21 Yeah.

Harry Stebbings 56:21 And, uh, Mistral is in Paris, and oh, I get in so much trouble these days, Nick, 'cause I just... Uh, my mother says that I have Asperger's, but I just call it kind of freedom of speech. Um, but it's just like, we all say that for Mistral, it's like the Europe play, and that's why it's funded and it continues to be funded. Um, do you think that we will see sovereign models and usage because of geography?

Nick Frosst 56:50 I think this technology is, is a lot like infrastructure, right? Like I think building a... Like having a language model that speaks the, the language of your country is like building infrastructure for the people of your country. Um, so I think that's broadly a good idea. I think the past like 20 years, longer, of technological history has been very defined by Silicon Valley. It's been very defined by, by California and America. And I think a lot of people are not very happy about that, and I think a lot of people are rightfully upset with some of the developments, right? Like I used to be a real technological optimist. Like I used to love the way technology was built and be like, "Oh, it's so exciting," or something. And I, I, I wouldn't describe myself as a technological optimist over the past 10 years.

Harry Stebbings 57:37 Wow, why? What happened to change that?

Nick Frosst 57:39 Oh. Uh, well, wait, sorry. Let me, let me answer that. First, let me get back to the, the sovereignty thing before I go off on this tangent. Um, so yeah, I think there's a lot of people who are interested in building that infrastructure within their country and having the technology for their economies. Um, and I think just using a model that is built, you know, by China or built within America might not set your com- your country and your economy up as well as having a model that is, uh, understands the context built in that language, in that dialect, in like, you know, has the, has the, the cultural fluency, um, needed to empower the people of the country. So I think that's like a good idea. What that ends up looking like, like I, I don't, I'm not exactly sure, you know.

Harry Stebbings 58:24 Geopolitics obviously influences a lot. Do you think geopolitics has influenced customer decisions around sovereignty of models in the discussions that you see?

Nick Frosst 58:34 Um, I think us being Canadian is, is an asset, right? I think, um, that's helpful for people.

Harry Stebbings 58:41 Uh, do Canadian companies wanna buy you more?

Nick Frosst 58:43 Uh, I think coun- companies around the world are interested in talking to us. And in part that's because we're Canadian. Um, look, like, you know, over the past few years, y- y- you know, America has shown that they're willing to like turn off access to tech based on political reasons, right? Like, you know, we've seen, uh, uh, uh, connections between American tech and the American government is like l- less clear as time goes on.

Harry Stebbings 59:15 What does that mean?

Nick Frosst 59:17 What does that mean?

Harry Stebbings 59:18 Yeah.

Nick Frosst 59:18 Uh-

Harry Stebbings 59:18 I'm a Brit. It means like Trump in- influences US tech companies?

Nick Frosst 59:23 Seems to be, yeah. Yeah. Seems to be. Right, I mean, I mean, even it was like last week or something they announced they're taking a 10% stake in Intel, right? Like that's an interesting development. I, I, I'm, I'm not an, I'm not an economist. And I don't know if that's good for the country or not. Like I'm, I'm not... That's... But it is an interesting, um, development, right? Um, so I think there's a lot of coun- companies in Canada and around the world that are interested in working with non-American tech companies.

Harry Stebbings 59:52 Mm-hmm.

Nick Frosst 59:52 And I would say that's been an asset for us, right? Like that's...

Harry Stebbings 59:55 Do you think governments should fund sovereign models? Like is j- is it a European imperative for us to have Mistral as an asset for Europe?

Nick Frosst 1:00:06 I think it's a good idea for countries to have infrastructure within their countries. Like I think it's a good idea for people to have power plants in the country, you know. I like that Canada has several nuclear power plants and has several water power plants. Like, you know, that- that's great. Yeah, I think language models are not that dissimilar from infrastructure.

Harry Stebbings 1:00:26 Do you think our primary, like, input device will still be a phone in five years time?

Nick Frosst 1:00:33 I do think langu- like I know language is gonna be a more important part of it, right? Like I think fundamentally the way we should be interacting with computers is using language for the majority of it. Not all of it. There are times when like language is actually not the best way of interacting with a computer. It's much better to have a graphic user interface if you're, like, doing something. I know like last year there was a f- there was like the Rabbit R1, there was those like, the Humane PIN, and like none of... They didn't really get it right. But I think there's something cool there about like, hey, how do we use...... how do we use a language model to work with a computer better? I haven't seen it done right yet, and I don't know if it will, and I don't know if that's because, um, people don't ... Like, one of the things, like, going back to the technological optimism thing, like, you know, I was really excited when Google Glass came out. I thought that was really cool.

Harry Stebbings 1:01:18 Yeah, so was I.

Nick Frosst 1:01:19 Yeah, and then I-

Harry Stebbings 1:01:20 I had it- I had it as a profile picture.

Nick Frosst 1:01:21 Yeah. Okay. Yeah, and then I got on a bus one time and somebody was wearing a Google Glass, and they were delicate, they were like... And suddenly, everybody saw it immediately. Like, poof. You know, it, it clocked it immediately. And I think, you know, I was really excited about VR for a while, and then I realized I actually don't want to strap a computer to my face. I'm not interested in being disengaged from the world more. I want to be engaged in the world more than I am. I don't, I don't want more things removing me, and I don't think many people do.

Harry Stebbings 1:01:54 Do you just worry about this is so m- messed ... Like, this is the state of the world in terms of depression, in terms of loneliness. You know, the biggest p- pandemic, epidemic, whatever we want to ... I, I never know the difference between pandemic and epidemic.

Nick Frosst 1:02:06 Wow, this is such a podcast. I haven't done many podcasts. This is the most podcast podcast ever. (laughs)

Harry Stebbings 1:02:11 I'm just, I, I, again, I can just do it.

Nick Frosst 1:02:15 No.

Harry Stebbings 1:02:15 It's too interesting where I'm like, "Fuck yeah, let's go."

Nick Frosst 1:02:17 No, no, these are interesting questions. I'm happy-

Harry Stebbings 1:02:19 But it's like, I- I-

Nick Frosst 1:02:19 I'm happy to talk about them.

Harry Stebbings 1:02:19 ... I'm just so nervous about the state of loneliness-

Nick Frosst 1:02:22 Mm.

Harry Stebbings 1:02:23 ... insecurity, eating disorders, uh, focus on materiality for young people. The number one job that any young person wants to be is an influencer.

Nick Frosst 1:02:33 Yeah, there are, there are things that you're talking about that I do worry about. You know, and I do, I do worry about, yeah, the dissolution of, of community. And I think, like, what I'm talking about earlier about saying I want to be engaged more in the world, I want the technology that I use to connect me to the world better. I don't want it to, to disconnect me from the world. Um, I think a lot of people feel that. I think a lot of people are looking for ways of connecting to technology more, right? Like, I, I play music. Uh, a lot of the reason I play music is because it's immediate, it connects you to the people who you're playing with and the people who are listening, and a lot of the people who come listen to the music are there to connect in the moment. Like, that's a really important part of it. Um, so I think a lot of people are feeling that, and I think a lot of people are feeling that because of the f- because they're experiencing what you're describing statistically in their personal lives. I also know that that worry of like, "Oh, no, the world these days is, you know, so bad and things are going in the wrong direction and the kids these days are so weird," and like, "Oh, no. Back, you know, things used to be better back then," is historically ubiquitous, and everybody has always thought that going back to, like, Greek philosophers bemoaning the prevalence of writing because it was gonna, you know, make people not use their memories anymore, and saying, "Oh, no, the kids these days don't understand honor." And like, you know, going back to people bemoaning the spread of newspapers because they were all sitting on the bus reading newspapers as opposed to sitting on the bus, like, talking to each other, like ... So, I think two mutually exclusive things. One, yes, I'm worried about all the stuff you're talking about, and I think technology and people who make technology need to think very hard about if their technology is helping with that or hurting that. I think that. Two, this, everybody's always thought that, that, that time that they're alive is the time when it's the worst, and those, you have to hold both of those two conflicting views in your, in your mind.

Harry Stebbings 1:04:30 We'll do a quick fire, but you, I'll send you off to this, a brilliant song, and it's not really a song-

Nick Frosst 1:04:34 Yeah.

Harry Stebbings 1:04:35 It's like a commencement speech by Baz Luhrmann and it's called Wear Sunscreen.

Nick Frosst 1:04:38 Okay.

Harry Stebbings 1:04:38 Um, and it basically says exactly this-

Nick Frosst 1:04:41 Yeah.

Harry Stebbings 1:04:41 ... which is every generation always looks back and goes, "Ugh, prices are so high today."

Nick Frosst 1:04:48 (laughs)

Harry Stebbings 1:04:48 And, "Kids are so rude today," and, "It was better in my time." And it's this kind of continuous pattern of, uh, s- life-

Nick Frosst 1:04:57 Yeah.

Harry Stebbings 1:04:57 ... in terms of looking back and thinking it was better than the one we have today.

Nick Frosst 1:05:01 Yeah, that, that all being, look, talking about intrinsically human things, like, that also seems to be intrinsically human. You know, wanting to be part of something bigger, wanting to build something that'll last you, human. Thinking things were better when you were

Nick Frosst 1:05:12 young.

Harry Stebbings 1:05:12 Okay, we're gonna do a quick fire.

Nick Frosst 1:05:13 Okay.

Harry Stebbings 1:05:14 Yeah? So, if you were Sam Altman today, what would you be doing that he's not doing?

Nick Frosst 1:05:18 I don't think Sam Altman has done a service to the world by talking about how close AGI is. I think he has made several predictions now that are wrong, um, and that were obviously wrong at the time he made them.

Harry Stebbings 1:05:32 Which one is most prescient?

Nick Frosst 1:05:33 Oh, that, like, AI is gonna kill the whole world in two years. Or like, he's made, you know, he's made allusions to things, like, he did a world tour where he spoke to every major leader the world over to tell them, "Hey, this technology is gonna, you know, is, poses an existential threat." And I think that was academically disingenuous, and I think did a disservice to the technology he loves, you know?

Harry Stebbings 1:05:57 Do you not see a correlation between the words that one has with regards to the future of AGI and AI, and their requirements for funding?

Nick Frosst 1:06:09 I don't, I don't know what it is.

Harry Stebbings 1:06:09 Do you see what I mean, though?

Nick Frosst 1:06:10 I, I see what you mean, yeah. Yeah, yeah.

Harry Stebbings 1:06:10 Which is like Demis and Zuck for a long time do not need funding-

Nick Frosst 1:06:14 Yeah.

Harry Stebbings 1:06:14 ... and they are much more balanced, neutral.

Nick Frosst 1:06:18 Yeah.

Harry Stebbings 1:06:18 And then other people who do need funding have to be much more provocative and out there because they need your fucking dollars.

Nick Frosst 1:06:24 Yeah. Yeah, I don't know if that was the strategy. Uh, uh, the correlation you're pointing at exists. Um, I would say that, you know, we're a, we're a venture capital funded company and we need funding, um, and we don't say that. <|agent|><|en|>

Harry Stebbings 1:06:36 What worries me, though, actually is even the rhetoric from your Demis and your Zuck-

Nick Frosst 1:06:41 Yeah.

Harry Stebbings 1:06:41 ... has changed. Even their aggression towards the changes that are coming has, has, has flipped-

Nick Frosst 1:06:48 Yeah.

Harry Stebbings 1:06:48 ... which does make me worry.

Nick Frosst 1:06:50 Makes you worry because ... For what reason?

Harry Stebbings 1:06:52 For the reason that actually we are far closer than we think-

Nick Frosst 1:06:55 Mm.

Harry Stebbings 1:06:55 ... to very material shifts in labor patterns, workforce behaviors.

Nick Frosst 1:06:59 Mm-hmm.

Harry Stebbings 1:06:59 When even Zuck, who does not need the money from anyone, or Demis, who doesn't need it from anyone, is going, "Oh, shit. The changes are real."

Nick Frosst 1:07:05 Well, yeah, well, there are some real changes, right? Like, I'm not, I don't wanna dem- You know, this technology, fundamentally transformative, the same way-... the personal computer, fundamentally transformative, the Industrial Revolution, steam engines, the printing press. Those are all big technologies. Like, I really think that. Um, so there's, there's very, there's tons of legitimate things to talk about, and I'm glad people are talking about them. Uh, there's also a whole lot of not legitimate things to talk about that those people are spending their time on.

Harry Stebbings 1:07:30 What is your founder ritual after closing each round? (laughs)

Nick Frosst 1:07:37 (laughs) Who told you about that? (laughs)

Harry Stebbings 1:07:40 Jordan. (laughs)

Nick Frosst 1:07:41 Okay. (laughs) Uh, that's funny. Uh, uh, we go to McDonald's. (laughs)

Harry Stebbings 1:07:46 You go to McDonald's?

Nick Frosst 1:07:47 We go to McDonald's, yeah.

Harry Stebbings 1:07:48 And have the same meal?

Nick Frosst 1:07:50 Yeah, I, I normally get two Junior Chickens.

Harry Stebbings 1:07:52 Wow.

Nick Frosst 1:07:53 Yeah. I don't remember why that started.

Harry Stebbings 1:07:55 But literally after every round, you go to McDonald's.

Nick Frosst 1:07:57 We go to McDonald's, yeah. After we signed it, we, the three of us go to McDonald's.

Harry Stebbings 1:08:01 How many rounds have you done?

Nick Frosst 1:08:02 Oh, five, four? Five, four? Yeah.

Harry Stebbings 1:08:07 Wow, I love that. That's awesome-

Nick Frosst 1:08:09 Yeah. (laughs)

Harry Stebbings 1:08:09 ... to go to McDonald's. What's the worst thing that could happen with regulation towards AI?

Nick Frosst 1:08:14 Out of an erroneous understanding of the technology, um, and, uh, you know, like, thinking that what we're building is, is, is digital gods, um, which is, like, large languages models are not. Uh, but if you think that that's what they're building, um, then you could think, "Okay, cool, we need to, you know, come up with benchmarks around existential threats." Um, and I think there are times when looking at those, uh, uh, at benchmarks, like, you know, fixation on particular benchmarks, which can be gamed and can be trained either to do way better on or way worse on, um, are not helpful for establishing how the technology can be used and misused. Um, so I think if you were... Like, the worst thing a regulation could do is say, "Hey, we're gonna pick this random benchmark. We think that represents AGI, and we're gonna, you know, shut down any development on it." I think that would be, that would be a misplay.

Harry Stebbings 1:09:08 Do you think China will produce leading models in the next two years that continue to beat US models?

Nick Frosst 1:09:17 No, I'm not sure. Um, they haven't yet, right? They've made good models. Um, definitely good models. But I don't think they've made models that are, like, beating, you know, the other models out there.

Harry Stebbings 1:09:29 You're not worried by China and their model capabilities?

Nick Frosst 1:09:34 Um...

Harry Stebbings 1:09:35 When I look at cadence of China, uh, it... In, like, a week, uh, about a month ago, there were, like, seven new model providers-

Nick Frosst 1:09:42 Yeah.

Harry Stebbings 1:09:42 ... who released seven models.

Nick Frosst 1:09:43 Yeah.

Harry Stebbings 1:09:44 They were pretty good.

Nick Frosst 1:09:45 Yeah.

Harry Stebbings 1:09:46 And I was like-

Nick Frosst 1:09:47 Wow.

Harry Stebbings 1:09:48 "Wow."

Nick Frosst 1:09:48 Yeah, I mean, uh, China's a very interesting coun- country. Um, but am I, like, worried about it? I would say, like, no, I don't think I'm, like, worried about it. I think they're gonna keep building models, and those models will be useful, um, and they'll be particularly good in the things that they've trained them on. But that doesn't cause me, like, a bunch of...

Harry Stebbings 1:10:06 What's your boldest prediction for LLMs in 2026?

Nick Frosst 1:10:10 In 2026, you'll be able to open up a computer, you know, log into North or whatever application you're using, uh, and say, "File my expenses." And then the model will-

Harry Stebbings 1:10:25 Yeah.

Nick Frosst 1:10:26 ... figure out, you know, what expense policy it is and what the, where the photos are, like, do all that. Like, that's, like, do all of that for you. That's my boldest prediction. And I know that's not very bold. I know in some ways, that's like, "Oh, that sounds like not too far off."

Harry Stebbings 1:10:38 (inhales deeply)

Nick Frosst 1:10:38 And yet getting it to actually work and be a thing you can rely on is still not, you know, is, is not in every company. You know, that's not... Most people don't have the experience I just described. And I think that becoming a ubiquitous way of using a computer is crazy.

Harry Stebbings 1:10:53 I think my CFO would love you. I mean, we're-

Nick Frosst 1:10:55 Yeah.

Harry Stebbings 1:10:55 ... always getting expense chasing.

Nick Frosst 1:10:57 Yeah.

Harry Stebbings 1:10:57 So, uh, totally get that. If you weren't at Cohere, which AI company would you bet your career on?

Nick Frosst 1:11:01 Google's great. Google, DeepMind is building cool stuff, you know? That's exciting.

Harry Stebbings 1:11:05 Have there-

Nick Frosst 1:11:05 Good stuff.

Harry Stebbings 1:11:05 ... been any tools that you've added to your workflow that meaningfully impr- increased your productivity? So, like, for me, like, Whisper Flow, like-

Nick Frosst 1:11:11 Oh, Whisper, yeah.

Harry Stebbings 1:11:12 Yeah.

Nick Frosst 1:11:12 Um, I mean, certainly North, our, our own product. Um, but I, I use Cursor, and Cursor's a great, a great coding application.

Harry Stebbings 1:11:18 Do you mandate Cursor across the whole Eng team?

Nick Frosst 1:11:21 No, not at all, no.

Harry Stebbings 1:11:22 Does anyone use Windsurf or Devin?

Nick Frosst 1:11:24 Um, I think so. I'm not sure. Um, I think people have... Yeah, we don't, we don't mandate one. I know lots of people rely on models. Uh-

Harry Stebbings 1:11:33 Are you price-sensitive towards Cursor costs increasing?

Nick Frosst 1:11:37 No, but I live a life of privilege.

Harry Stebbings 1:11:40 (laughs)

Nick Frosst 1:11:40 Right? So, like, so, like, n- no, I, I'm not as a person. Uh-

Harry Stebbings 1:11:44 But when pri- when cost goes up 10X and you have 100 engineers...

Nick Frosst 1:11:48 Oh, as a business?

Harry Stebbings 1:11:48 Yeah.

Nick Frosst 1:11:49 Yes. (laughs) As a business, obviously. Um, and, you know, uh-

Harry Stebbings 1:11:55 How does that shake out? Do we see this kind of, like, uh-

Nick Frosst 1:11:58 Well, like, f- for us, yeah. I mean, we, we make our own models, so if we wanted to use, uh, our own model and use, plug that into, uh, an extension of VS Code, that, that would be something we could do, if that was...

Harry Stebbings 1:12:08 Would the quality of output be similar?

Nick Frosst 1:12:12 Mm, right now, no. No, no, no. Cursor's built a good product. Like, they built, you know, they've b- done really good UX stuff, and it's, it's cool, and that... Um, but if it was to go up 10, 100-fold, like, yeah, gro- of course, then we, we'd start thinking about it.

Harry Stebbings 1:12:23 Will we see a trillion-dollar AI company outside of the US in the next decade?

Nick Frosst 1:12:28 Yeah. (laughs) Maybe north of the border. (laughs)

Harry Stebbings 1:12:33 (laughs) Uh, okay, final one. What trait do you love and has contributed to a lot of your success, but you're also quite wary of?

Nick Frosst 1:12:42 Um, I think I'm, I'm quite, um, I'm quite curious and contrarian, um, and that is both an asset and a hindrance. There are times when it's very helpful to be like, "Oh, yeah, I'm, like, super interested in something," and I learn about it, and then I'm like, "Everybody thinks this, and they're totally wrong." That's super helpful. Um, that's why when Aidan was like, "Hey, do you want found a company on language models?" back in 2019, I was like, "Yeah, absolutely." You know, that was not a view that was widespread. Not at all. Like, even... So I'm, yeah, curious and contrarian. But there's other times when, like, the whole world has been definitely right, and I've been wrong, and I've been like, "Oh, yeah, I was super excited about this."

Harry Stebbings 1:13:17 When were you most wrong?

Nick Frosst 1:13:18 Am I most wrong? When I was, when I was very young, I was very much a, like, a, a, a re- a technological optimist, as mentioned. But I thought, like, "Cool, all, all metrics of human improvement are gonna, uh, are gonna continue to go up." You know, like, life expectancy will go up. Income inequality will go down. Happiness will go up. Like, as, as...