The Twenty Minute VCJonathan Ross: DeepSeek Special - How Should OpenAI and the US Government Respond | E1253
EVERY SPOKEN WORD
105 min read · 20,747 words- 0:00 – 2:01
Intro
- HSHarry Stebbings
So everyone's seen the news about DeepSeek today. Is it as big a deal as everyone is making of it?
- JRJonathan Ross
Yes. It is Sputnik 2.0. It is true that they spent about $6 million, or whatever it was, on the training. They spent a lot more distilling or scraping the OpenAI model. I can't speak for Sam Altman or OpenAI, but if I was in that position, I would be gearing up to open source my models in response because it's pretty clear you're gonna lose that, so you might as well try and win all the users and the love from open sourcing. Open always wins. Always.
- HSHarry Stebbings
Ready to go? Jonathan, dude, I am so excited for this. So I've heard so many good things from so many different people, so thank you so much for doing this emergency show with me today.
- JRJonathan Ross
No problem. But before we start, can I, can I just say one thing?
- HSHarry Stebbings
Sure.
- JRJonathan Ross
Um, I think you have the most amazing, unique go-to-market that I've ever seen in my life for a podcast. I've never seen this before. I think your strategy is you're literally interviewing every single audience member, forcing them to watch videos and get addicted to you.
- HSHarry Stebbings
(laughs) I mean, I thought you were gonna say my accent, but, uh, I'm totally gonna take that. That's wonderful. Um, and yes, you're absolutely right. Uh, sometimes the biggest benefits of your business, you don't actually see until you do them. Um, but I wanna-
- JRJonathan Ross
And, and And ... do things at scale.
- HSHarry Stebbings
That is totally true. Um, but I do wanna start. Um, obviously everyone's just talking about DeepSeek.
- JRJonathan Ross
Mm-hmm.
- HSHarry Stebbings
Little bit of context, why are you so well placed to speak about DeepSeek? And let's just start there for some context.
- JRJonathan Ross
Well, my background, so I started at Google TPU, the AI chip that Google uses, and in 2016 started an AI chip, um, startup called Groq, with a Q, not with a K, um, that builds, uh, AI accelerator chips, which we call LPUs.
- 2:01 – 4:33
Is DeepSeek News as Big a Deal as It Seems?
- JRJonathan Ross
- HSHarry Stebbings
Fantastic. Wonderful. I wish everyone as coherent as you in terms of their introductions. Okay, so everyone's seen the news about DeepSeek today. I wanna just start off by saying, is it as big a deal as everyone is making of it?
- JRJonathan Ross
Yes. It's Sputnik. It is Sputnik 2.0. And, um, even more so, y- you know that, uh, story about how NASA spent a million dollars designing a pen that could write in space, and the Russians brought a pencil? Um, that just happened again. So it's, it's a huge deal. Yeah.
- HSHarry Stebbings
Okay. Why is it such a huge deal? Let's unpack that.
- JRJonathan Ross
All right. So up until recently, the Chinese models have been behind, um, sort of Western models. And I say Western including, like, Mistral as well and, and some other companies. And it was, um, largely focused on how much compute you could get. Most people actually, m- most don't realize this, most companies have access to roughly the same amount of data. They buy them from the same data providers, and then they just churn through that data with a GPU, and they produce a model, and then they deploy it. And they'll have some of their own data, and that'll make them subtly better at one thing or another, but they're largely all the same. And the more GPUs, the better the model 'cause you can train on more tokens. It's the scaling law. Uh, this model was, uh, supposedly trained on a smaller number of GPUs, um, and a much, much tighter budget. I think the way that it's been put is less than the salary of many of the executives at Meta, and that's not true. It's, it's actually an el- el- there's an element of marketing, uh, involved in the DeepSeek release.
- HSHarry Stebbings
What do you mean? Unpack that.
- JRJonathan Ross
Well, it is true that they trained the model on a- approximately $6 million worth of GPUs, right? They, they, they claim 2,000 GPUs for, I think it was 60 days, which by the way, also don't forget, was about the same amount of GPU time, 4,000 GPUs for 30 days, as the original, I believe, LLaMA 70. Now, more recently, Meta has been training on more GPUs. But Meta hasn't been using as much good data as DeepSeek, because DeepSeek was doing reinforcement learning using OpenAI.
- HSHarry Stebbings
Is this distillation, just so I understand?
- JRJonathan Ross
Yes, effectively. Effectively, yeah.
- 4:33 – 7:18
Distillation & DeepSeek's Use of OpenAI Data
- JRJonathan Ross
- HSHarry Stebbings
Okay. And so can you just help me and help the audience understand, what is distillation in this regard, and how have DeepSeek been using distillation to get better quality output through OpenAI data?
- JRJonathan Ross
So it, it's a little bit like speaking to someone who's smarter and getting, um, tutored by someone who's smarter. You, you actually do better than if you're speaking to someone who's not as knowledgeable about the area or giving you wrong answers. And there's this, uh, first of all, before we get into any of this, I, I need to start with the scaling laws. These are like the physics of LLMs, and there's a particular curve, and the more tokens, which are sort of the, sort of the syllables of an LLM, they don't match up exactly with human syllables but kind of, right? So, um, the more tokens that you train on, the better the model gets. But there's sort of these asymptotic returns where it starts trailing off. The thing about the scaling law that everyone forgets, and that's why everyone's talking about how it's like the end of the scaling law, we're out of data on the internet, there's nothing left, what most people don't realize is that assumes that the data quality is uniform. If the data quality is better, then you can actually get away with training on fewer tokens. So going back to my background, one of the, the fun things that I got to witness, I, I wasn't directly involved, was, um, AlphaGo, when, um, Google beat the, the world champion, Lee Sedol, in Go, um, that model was, was trained on a bunch of existing games. But later on, they created a new one called AlphaGo Zero.... which was trained on no existing games, it just played against itself. So how do you play against yourself and win? Well, (object thuds) you train a model on some terrible moves, and it does okay, and then you have it play against itself, and when it does better, you train on those better games, and then you keep leveling up like this, right? So you get better, better data. The better your model is, when it outputs something, the better the result, the better the data. So what you do is you, you train a model, you use it to generate data, and then you train a model and you use it to generate data, and you keep getting better and better and better. So you can sort of beat the scaling law problem. One quick hack to get past all of that and the stepping up is, if there's a really good model already right here, just have it generate the data, and you go, whoop, right up to where it is. And that's what they did. So it is true that they spent about six million or whatever it was on the training. They spent a lot more, um, distilling or scraping the OpenAI
- 7:18 – 11:40
Scraping OpenAI Models for Higher Quality Output
- JRJonathan Ross
model.
- HSHarry Stebbings
So they scrape the OpenAI model, they get this higher quality data from that and from refining it, and then they get greater, higher quality output. Correct?
- JRJonathan Ross
Correct. And all that said, they did a lot of really innovative things, so that's what makes it so complicated, because, uh, on the one hand, they kind of just scraped the OpenAI model. On the other hand, they came up with some unique reinforcement learning techniques, um, that are so simple you could-
- HSHarry Stebbings
What, what did they do that was so impressive? Because I think a lot of people wanna just say, "Oh, it's only the Chinese copying and duplicate as they always have done."
- JRJonathan Ross
No. No, no, no, no.
- HSHarry Stebbings
What's ...
- JRJonathan Ross
They came up with innovative stuff, but actually, the best way to describe it, have you ever taken a test before and you got an answer right and your professor marked it wrong?
- HSHarry Stebbings
Yeah.
- JRJonathan Ross
And then you go back to the professor and you have to argue with them and everything and it's a pain, right? Well, if there is only one answer, and it's a very, like, simple answer, and you say, "Write that answer in this box," then there is no arguing. You either get it right or not, right? So what they did was rather than having human beings check the output and say, "Yes," or, "No," or whatever, um, what they did was they said, "Here's the box." There's literally some code to say here's a box. I'll put the answer here, and then check it, and if it's correct, then we have the answer, if not, we don't. No need to involve a human. Completely automated.
- HSHarry Stebbings
What about... I, I, I read about reward modeling state, and they innovated on this in such a unique way. Did they, did they not? Can you explain that for me?
- JRJonathan Ross
That, that area I'm not as familiar with, so I'm probably not could... But maybe, like, you've been doing the research. Why don't you tell me what you saw and I could tell you if it tracks?
- HSHarry Stebbings
The, the, uh, essentially, they combine two different types of reward models to get higher, uh, more accurate output, and that was what I kind of didn't understand, um-
- JRJonathan Ross
Yeah, th- this is, yeah, not an area where I've dug too deeply into it, um-
- HSHarry Stebbings
Can I, can I ask you that, can OpenAI not just do distillation on DeepSeek's model then? If it gets-
- JRJonathan Ross
They don't, they don't need to because they're actually better still. Th- they're a little bit better. So, um, they could, but why would they?
- HSHarry Stebbings
Do we buy the GPU usage or is that questionable down there?
- JRJonathan Ross
I don't think you have to disbelieve it because of the quality delta. However, I will say this, why would they try and smuggle in GPUs when all they'd have to do is log into any cloud provider and rent GPUs? This is like the biggest gaping hole in the whole, um, way that export control is done. You can literally log in... You can swipe credit card, whatever, and just like, pay and get GPUs to use.
- HSHarry Stebbings
So I thought it was unnecessary then or inefficient.
- JRJonathan Ross
Well, no. They're, they're good, but the problem is, there's like, it's like, the menational line, you just go around it, so you need to like seal it up a little more. There's a little bit of room left to, to, to go here. And then the other thing is, keep in mind, OpenAI was effectively subsidizing accidentally the training of this model because they, they were using OpenAI, right? And, you know, rumors are that OpenAI may not be completely profitable yet, um, in terms of every token in the API. Like, on the subscriptions, maybe, but, but in the API. And so each one they generate, effectively they were losing a little bit of money while DeepSeek was getting training data. Now, by the way, um, OpenAI probably still has that data, in theory they could just probably train on it.
- HSHarry Stebbings
George Christian said in a tweet today though that this would likely be a violation of US export laws. Do you think that's not true?
- JRJonathan Ross
I'm not aware of where it would be an export, um, issue. I do know that many people log into cloud providers and just use them from remote. One of the problems... So we actually block IP addresses from China, um, and I believe we might be unique in doing that. It's also a little bit fruitless because, you know, someone could just, like, rent a, a server anywhere and then log into us from there, right? And then there's nothing we can check. So I, I don't know that IP addresses are really the right way to do it anyway. I think we need something more sophisticated, but, I mean, uh, it... Yeah, it's, it's, it's a big cheese, Swiss cheese wall.
- 11:40 – 13:08
Concerns About US Customer Data Going to China
- JRJonathan Ross
- HSHarry Stebbings
You said there about kind of blocking, you know, um, IP addresses from China. Um, there's a lot of concern about US customer data going back to China. Do you think that is a legitimate and justified concern?
- JRJonathan Ross
Yes, it's, it's, um, it, it, it's probably the most significant concern. There, there are other concerns, that's probably the most significant, because people don't think. They're so used to using these services. When, when you use one of these other services, y- you might be shocked to hear this. When you say delete, what they do is they write "delete" right next to your data-They don't actually delete it, they just mark it deleted. When you later come back and, and ask for your data, they give it to you with the word delete right next to it. It's still there. And these are well-meaning companies. Do you really think, like, the CCP doesn't have all your data and is gonna look it up later? And some governments are more aggressive than others, right? And if they have access to your data, not even your data, it could be your next door neighbor's data. Your next door neighbor might put something in there that, um, accidentally, um, gives information away that makes you more vulnerable, right? And then now, the CCP has something and, like, maybe you had some package delivered and, and they put a complaint somewhere and whatever. Like, you might not even do it yourself, but other people around you, like the health data of a spouse.
- 13:08 – 23:13
DeepSeek and Its Potential Use by the CCP
- JRJonathan Ross
- HSHarry Stebbings
Jonathan, I'm, uh, gonna avoid the British indirectness. Do you think DeepSeek is an instrument that will be used by the CCP to increase control on Western democracies?
- JRJonathan Ross
Yes, but I don't think it's DeepSeek that, that's doing it. So, you have to understand, any company that operates in China and Hong Kong, the, the, what was it? Um, uh, one country, two systems thing didn't quite work out as anticipated, um, or maybe as anticipated, but not as stated. Um, they have no choice, right? And so, uh, in 2016 when Grok started, we decided that we were not gonna do business in China. This was not a geopolitical decision. This was purely commercial. And what it was was we kept seeing companies like Google, Meta, just, you know, fail over and over again trying to win in China. And the formula's actually pretty simple. You're not allowed to make net money. You're allowed to spend more money in China, but the moment that you start to become profitable or anywhere near profitable, all of a sudden there's a thumb on the scale. So, companies that manufacture a lot in China and s- send more money to China can actually be successful there. They can sell things there. Yeah, it's a pretty simple formula. You must send more money to China than you take out. But at the same time, they also require that you hand over all data. And not only that, they also require that certain answers be in a form that they find acceptable. So for exam- th- one of the more common ones that you see about DeepSeek right now is when you ask about Tiananmen Square. If the temperature is low on the model, and temperature, we, we don't need to get into that, it's complicated, but it's how cre- like, low means low creativity. Then it's actually going to give you, um, an answer that basically says, "Ah, I don't wanna talk about that. It's a sensitive topic." Right? But you ask it about other things that, um, uh, are sensitive topics elsewhere in the world and it'll just answer. But what, what happens if the CCP requires that they start to say, "What about TikTok? Should it be banned?" "Absolutely not. Here's why." And it gives you a cogent reason, right? That's kind of scary.
- HSHarry Stebbings
Jonathan, what do we do from here? I c- I g- I share your concerns completely. My challenge is, TikTok you can ban and shut off. They would not sell the algo. That is a closed-end product that we can ban tomorrow if we really want to. Here, it's open source.
- JRJonathan Ross
Yeah. And, and worse. So, uh, we, we up until recently refused to run any Chinese models, and we had to make a very difficult decision on DeepSeek. We now have it on our, um, API, uh, you know, at Grok. And-
- HSHarry Stebbings
So-
- JRJonathan Ross
... o- our-
- HSHarry Stebbings
... um, h- why did you decide that you would break that rule for DeepSeek?
- JRJonathan Ross
So, um, what it came down to was when we saw DeepSeek become the number one app on, uh, the App Store, the realization was people were gonna be putting their data in there. And what we wanna make sure is that you actually have an option. So we store nothing. Like, there is no, like, delete or whatev- like, there is just, we store nothing. We don't even have hard drives, right? Like, we just, we have DRAM, and when the power goes off, everything goes away, right? So, um, we wanted to make sure that there was an alternative where when y- you use DeepSeek's model, your data is not going to the CCP. Well, right now, the CCP is probably gonna be taking the safeties off the weapons. They're gonna be like, "Why are you making this model open source? Please direct your data towards us. Go win a bunch of customers this way. But now we want the data," right? And so they're gonna change the strategy. But remember, DeepSeek is a real, I mean, it's a hedge fund. They're doing this themselves, and they're just influenced by the CCP. And the CCP, now that they've seen the success of this, might see it as yet another TikTok.
- HSHarry Stebbings
100% they will see it as another TikTok. My question to you is, how long is it before the US reacts to prevent this?
- JRJonathan Ross
I, I, I-
- HSHarry Stebbings
And we sh- we should be that, Jonathan.
- JRJonathan Ross
It's hard, right? So the, the first thing is, um, one question to ask is, are we gonna be talking about DeepSeek for the next six, or, or, or R1 for the next six months? And the answer is absolutely not. We might be talking about R2 and R3 and R4, but R1 was one shot. The question is, are they gonna keep coming up with very interesting things? Are we gonna, you know, cat and mouse it, and, uh, is everyone going to learn from this? The, the biggest problem is we've, this has just made it absolutely nakedly clear that the, the models are commoditized, right? You, you've been asking the question, right? Like, if there was any doubt before, that doubt's over. Um, so what is the moat, right? And for me, I, I love Hamilton Helmer's seven powers, right? Like-
- HSHarry Stebbings
They're one of my favorites. I do it for every single investment we do. We have to fill it out. Every single person that has invested out. So, yes.
- JRJonathan Ross
So, so marketing is the art of de-commoditizing your product.And the seven powers are seven great ways to de-commoditize your product, right? Scale economies, network effects, brand, counter positioning, cornered resource, um, uh, switching cost, uh, process power, right? So the question is, who's gonna do what? OpenAI, and y- you gotta give, l- like, Sam Altman and that team credit, like they've got amazing brand power, like n- no, no one else in this space, and that's gonna serve them for a really long time, right? But w- what, what you see Sam trying to do is scale, right? He's trying to go s- that's why we hear about Stargate and $500 billion, right? That's what he, the power he would like to have, but the power he has right now is brand, and he's trying to bridge that, right? So, but what about the others?
- HSHarry Stebbings
I'm sorry, does this news not ridicule the $500 billion announcement? At a time when we've seen increasing efficiency to a scale like never before with DeepSeek today, the $500 billion seems ridiculed.
- JRJonathan Ross
Actually, I don't think it's enough spending. And, and the reason is, so we saw this happen at Google over and over again, right? So we, we do the TPU, and the TPU, s- s- so why did we do the TPU? The speech team trained a model. It outperformed human beings at speech recognition. This was, like back in 2011, 2012, right? It was the first time. And so Jeff Dean, most famous e- engineer at Google, um, gives a presentation to the leadership team. It's two slides. Slide number one, "Good news, machine learning finally works." Slide number two, "Bad news, we can't afford it. And we're Google. We're gonna need to double or triple our global data center footprint at probably a cost of $20 to $40 billion, and then we'll get a speech recognition. Do you also want to do search and ads?" So it turns out, there's always this giant "Mission Accomplished" banner every time someone trains a model, and then they start putting it into production, and then they realize, "Oh, this is going to be expensive." This is why we've always focused on inference. And so, now think about it this way. At Google, we always ended up spending 10 to 20 times as much on the inference as the training back when I was there. Now the models are being given away for free. How much are we going to spend on inference? And I, I guaran- and now that with the test time compute, right? And, like, I've, I've asked questions of DeepSeek where it took 18,000 intermediate tokens before it gave me the answer.
- HSHarry Stebbings
I think Jensen said that now half of their revenues is from inference.
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
So what does that look like in the future then?
- JRJonathan Ross
I think 95%. I mean, it just makes sense, right? You, like, you don't train to become, um, you know, a cardiovascular surgeon, um, and then that's what you do for 95% of your life, and then you perform for 5%. It's the opposite. You train for a little, and then you do it for the rest of your life.
- HSHarry Stebbings
So can I ask, do you think the US put sanctions on DeepSeek to prevent the CCP using it for data capture on US citizens?
- JRJonathan Ross
I don't know what the solution is. Um, y- there's carrot and there's stick, right? So you can either use a stick, block it, um, not exact... I mean, that might be effective. I don't know that the US has really done that before. I'm not aware of a case. It may be possible that it's happened. There's also the carrot, right? Which is, it's, it's kind of interesting how it's being offered for free in China, um, and not just in China, but to anyone else. Um, and then others are doing that too. Is it possible the CCP is underwriting that because they want the data? In which case...
- HSHarry Stebbings
Uh, th- dude, they're doing it with the car industry, the subsidization of, of cars, for Chinese cars, with BYD and SAFA destroying the European car market, is absolutely that.
- JRJonathan Ross
The thing is, we, we have a lesson from, um, the Cold War, which was mutually assured destruction. The, the problem is, we, we, you know, do some sort of tariff and then we do a tariff back. There needs to be some sort of automated response of like, "If you do this, we will respond. If you subsidize this industry, we will automatically subsidize the equivalent industry." Just automatic. So don't do it, because there's no benefit to you.
- HSHarry Stebbings
Does the fact that it's open source, how does that change everything?
- JRJonathan Ross
I mean, it, it's the only reason people are using it. I- if it wasn't open source, it wouldn't have gotten the excitement, right? And open always wins, always. And, and r- keep in mind, Linux won back when people didn't trust open source. They thought it was less secure, they thought it, the features were worse, it was more buggy, right? And it still won. Now, people expect open to be more secure, less buggy, and have more features. So how is proprietary ever going to win?
- 23:13 – 33:07
Is DeepSeek Diminishing OpenAI's Distribution Advantage?
- JRJonathan Ross
- HSHarry Stebbings
Everyone always says that actually distribution is one of the major advantages that ChatGPT, and hence OpenAI has, especially over the other providers. Every single day that DeepSeek is out and is being used so pervasively, it is diminishing the value of OpenAI.
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
Agree or disagree?
- JRJonathan Ross
Agree. And e- especially for the pricing, because they're losing their pricing power on this. I can't speak for Sam Altman or OpenAI or anything like that. But if I was in that position, I would be gearing up to open source my models in response, because it's pretty clear you're gonna lose that, so you might as well try and win, um, all the, the users and the love from open sourcing. Otherwise, I mean, like, you're already at a point where you're gonna be using your other powers like brand and so on. I don't know why you'd try and keep that internal anymore.
- HSHarry Stebbings
Would that be possible? And would that not cannibalize one, their core main line of revenue?
- JRJonathan Ross
But how would it cannibalize in any other way? Remember, people... Like, e- distribution, right? How many people are gonna buy something because they trust Dell, right?People trust Dell. Dell has earned their reputation over the course of decades. Supermicro builds some interesting hardware, but look at what they've been going through recently, like, you know, there's a pro and con, right? Cheaper, trusted, you gotta make a decision. And OpenAI has been around for a while. Most people think of them synonymously as AI. They could just switch to DeepSeek and people would still use them. It's brand. It's one of the seven powers.
- HSHarry Stebbings
So if you were OpenAI and Sam today, you would switch to Open and offer it for free?
- JRJonathan Ross
I would. And there's probably more cleverness, they could probably strike some deals before they do it or whatever. But that would be the move that I would make. And also, it would be a position of strength and it, it would just simply say, "Look," you know. Uh, the only problem is the timing, because if it happens right after DeepSeek, it looks like a response as opposed to an intentional thing. So I don't know how you do that. But that-
- HSHarry Stebbings
It is a response. Do you not just own in, "Hey, it's a response"?
- JRJonathan Ross
Yeah, maybe. That's a good one. You just say, "Look," you know, "we had to respond, we're better. Let's see which model people choose."
- HSHarry Stebbings
What do you think is the internal discussion within OpenAI today?
- JRJonathan Ross
I would imagine it depends on where you are. If you're senior, then you're gonna have very different concerns than if, if you're, you know, at, at the sort of foot soldier level. At the foot soldier level you're gonna be worried, "Is my equity gonna be worth anything? Is there any longevity here?" Like, "How, how do I do my job? Am I gonna have a job?" If you're further up, then it's gonna be more like, "How do I keep everyone... uh, how do I keep the morale up? How do I..." Like, "What is my response?" And then you're gonna have a lot of very difficult decisions, uh, in front of you. And, and the number one, the number one driver of bad decisions is fear. And so what they have to do is they have to pick something and then they have to just like commit to it hard and be brave about it. And, you know, it, it... there's so many different decisions that work if you commit and align, it's all about the alignment, right?
- HSHarry Stebbings
How do we think about Meta? Meta share the open source values that DeepSeek has espoused. Does this help or hurt Meta?
- JRJonathan Ross
That's a good question. So, I, I think one of the ways that we've been looking at, you know, LLMs, um, is a little bit like you look at an open source, um, project, software project, like, um, Linux or something. The thing is, Linux has switching cost. And I think what we've discovered is LLMs have no switching cost whatsoever.
- HSHarry Stebbings
I see why the analogy to cloud doesn't hold up at all, 'cause everyone's like, "Oh, it's like cloud, there's gonna be a couple of cool vendors on our sheet, they're, they're gonna win."
- JRJonathan Ross
Well-
- HSHarry Stebbings
No, you don't
- JRJonathan Ross
... rip out your cloud very often. Okay, so let, let's start mapping seven powers to the top tech companies. So, so I would say, um, Microsoft's biggest strength is switching cost, right? Look, I, I love Microsoft as a company, but you go into a room full of people and you're like, "Who uses Microsoft?" Bunch of hands go up. And you're like, "Who likes using Microsoft?" Hands go down, right? It's, it's very largely switching cost. So you go into, you know, gen AI, is that a thing that gets disrupted? You look at Meta, it's network effects. They could literally give every piece of technology away for free. I am completely jealous of that, because I- if, if I had that right now, I would open source everything, right? Because then you don't have to worry about it and you get everyone helping you, right? So I think Meta is sort of... because of the network effect thing, always in a position where open source is to their advantage. It almost doesn't matter where it comes from. Now, I, I'm sure that they would prefer to have the Linux of LLMs but I think the more it goes open source, the more of an advantage they have inherently.
- HSHarry Stebbings
If you were Meta, would you do anything different?
- JRJonathan Ross
You know, Meta is an amazing competitor. I think what they would normally do, if this was some sort of proprietary social mechanism, right? Um, they would try and replicate, and then they would compete, and they would say, "Come join or not." I don't think the, the come join works here. But the beautiful thing is all of the information for this model's available, Meta's already been doing this, they have way more compute. The question is are they willing to scrape OpenAI like DeepSeek did? And I don't think they are. I, uh, th- they've been super careful on everything that they've been doing. And so that's the disadvantage.
- HSHarry Stebbings
I'm not being rude. Do you not put morals aside to win? This is the AI arms race.
- JRJonathan Ross
I, I... And I think that's gonna happen. I, I, I think people will... like, y- you cannot lose. And so what it's done is it's changed the game, right? So if, if you... Okay, so let's talk about Europe for a minute. We almost forgot about Europe. Yeah. Um-
- HSHarry Stebbings
It happens
- JRJonathan Ross
... often, don't worry. (laughs) We-
- HSHarry Stebbings
We're kinda used to that now. We just sit and watch (laughs) with an espresso.
- JRJonathan Ross
So what I... uh, for me, you know, watching everything, it feels like with Europe there's the, the, a l- a lack of a willingness to take risk, right? There's a black mark if you get it wrong. Like, everything's about downside protection. Whereas in the US it's like, "That was a great effort, you failed, but I'm gonna fund you again." Right? So there's that difference. But when you look at the US and then you look at China, China, um, practices RDT, research, development, theft. It's just part of the culture. And it's not just against Western comp- it's against each other too. The difference is if you're a Western company, then the government steals from the Western company and then provides it to the Chinese companies, which is less fair. The famous stories of, um, turning on Huawei switches and you see Cisco's logo, and all the bugs and... right? Um, so-Is that a new paradigm? I really hope not. Because, like, for Europe to compete with the US, Europe has to adopt a more, um, risk-on attitude. Does the West have to adopt a more theft-on attitude? I really hope not. Like, that's just, like, viscerally disgusting to me. Like, I'm, like, literally repulsed by the idea.
- HSHarry Stebbings
I'm not being rude. Are we not being idealistic? If you're running in a race with someone who's willing to take steroids, if you wanna win, you're gonna have to take steroids too.
- JRJonathan Ross
And then everyone is taking steroids. Whereas if no one was taking it, then everyone's healthier and you have a real competition. Yeah, it's a real problem. And the question is, can governments get involved? Like, here's the thing. I would love nothing more than to compete directly with Chinese companies on a fair footing. They have really smart people. DeepSeek has proven this, right? Really smart people. But when the government keeps putting its thumb on the scale, we're- we're gonna try and avoid that competition wherever we can, and now there's no avoiding it. So, maybe the governments just have to get involved.
- HSHarry Stebbings
But dude, I'm being blunt. Like, Xi Jinping cares about one thing, power retention, and growth is the only thing that matters to him, and AI is central to that. He will do whatever it takes to win. Having some rational discourse about some rules of play is bluntly unrealistic.
- 33:07 – 34:48
Advising the EU on Europe's Stance Today
- JRJonathan Ross
this.
- HSHarry Stebbings
If you were to advise the EU today on Europe's stance, what would you say?
- JRJonathan Ross
I would say... So, have you ever seen Station F?
- HSHarry Stebbings
Yeah, of course. I was there last week. We hosted an event.
- JRJonathan Ross
Okay.
- HSHarry Stebbings
Yeah.
- JRJonathan Ross
So, I would say, by the end of this year, you should have 100 Station Fs, and by the end of next year, you should have a thousand. Done.
- HSHarry Stebbings
(laughs)
- JRJonathan Ross
You're- you're basically telling every- So what you're doing is you're collecting up 3,000 people and surrounding them with other risk-taking entrepreneurs, and then they're supporting each other and, you know, they're- they're risk-on. And every- When you surround yourself with other people who are risk-on, you're gonna be risk-on, and you're gonna- you're gonna take the entrepreneurial leap.
- HSHarry Stebbings
What does this space look like in three years' time? How I'm obviously a venture capitalist for a living. All of my friends are going, "Oh my God, oh my God, we just lost hundreds of millions of dollars on these foundation model companies."
- JRJonathan Ross
How many companies are you aware of that have become incredibly successful that didn't pivot?
- HSHarry Stebbings
Few. Yeah, exactly. So, pivot, get over it. Like, (laughs) just pivot. So frank- I- I've been talking to a lot of the LLM companies. And frankly, they have some good ideas. In fact, I really liked- So I watched your interview, um, with the Suno founder. And, um, I- I- he w- I think he saw it from the beginning, like models are gonna be commoditized and that's why he's focused on the product. Um, he got it from the beginning, right? What is your product? Not what is the model? Model is, um, it's a piece of machinery, it's an engine. But what is the car? What is the experience?
- 34:48 – 37:22
Perplexity in 3 Years
- HSHarry Stebbings
What do you think Perplexity is in three years?
- JRJonathan Ross
A question I used to get asked when, um, when we were raising money a little while ago was, um, "Is AI the next internet?" And I'm like, "Absolutely not." Because the internet is an information age technology. It's about duplicating data with high fidelity and distributing it. It's what telephone does. It's what internet does. It's what the printing press did. They're all the same technology, just much different scale, right? And- and speed and capability. Generative AI is different. It's about coming up with something contextual, creative, unique in the moment, right? And so the LLM is just the printing press of the generative age. It's the start of it, right? And then there's gonna be all these other stages. Well, the thing we're- Like, just imagine trying to start Uber when we didn't have mobile yet. Great. I'm gonna book a trip over to here. How do I get home? Like, you- you- you can't carry a desktop with you, right? So you need to be at the right stage. So when I look at Perplexity, I look at Perplexity as being perfectly positioned for the moment that the hallucination or really confabulation rate comes down. Because the moment that these models get good enough where you don't have to check the citations anymore, that's gonna open up a w- whole set of industries. All of a sudden you'll be able to do medical diagnoses from LLMs. You'll be able to do, you'll be able to do legal work from LLMs. Until then, it's like trying to create Uber before we had smartphones. It just doesn't make any sense. However, people are willing to use Perplexity today-... even though you have to check the citations. So they have an actual business that gets to cont- so like they're getting to sort of ride the wave, and the moment that that tsunami of, um, sort of lack of confabulation or, or hallucination comes along, they're perfectly positioned.
- HSHarry Stebbings
Does Mistral survive?
- JRJonathan Ross
Each company has to find their own thing, right? And I would look at Suno as, as like a great example of how things are being done around the product as opposed to just the models. But yes, like-
- HSHarry Stebbings
Do you think it is possible to pivot when you are OpenAI, or Anthropic, or any of the very large providers who've ingested billions of dollars?
- JRJonathan Ross
Disruption happens. If you're not able to pivot now, you're not gonna be able to pivot later when you get disrupted anyway.
- 37:22 – 41:01
Commoditization of Models & Big Tech's Stock Struggles
- HSHarry Stebbings
One would think that with commoditization of models and with cheaper inference, that actually big tech wins, right? Have you seen the stock market today? They've been hit hard. How do you think about that?
- JRJonathan Ross
What you see is a bunch of people who are concerned about training and the need for it, and everyone's still thinking that most of compute is training, and that there's gonna be less of it because someone trained a model on, um, 2,000 GPUs, and the nerfed, you know, A800 version with slower memory or whatever it is. And they're like, "Oh, people aren't gonna need as many chips." But again, like Jevons Paradox, right? Which is the more you bring the cost down, the more people consume. So for the last five to six decades, like clockwork, once a decade, the cost of compute has gone down 1,000X. People buy 100,000, uh, X as much compute, spending 100 times as much. So every decade they spend 100 times as much. So you make it cheaper and they want more. And so what's really happening is every time one of these models gets cheaper, we see our developer count just skyrocket. It just like goes up, and then it comes back down a little bit but th- the slope is higher than when it started. So better models create more demand for inference, more demand for inference then has people going, "I should train a better model." And the cycle continues.
- HSHarry Stebbings
I just bought a shit load of NVIDIA because they dropped 16% on the thesis that the increase in efficiency means that obviously we wouldn't need as much NVIDIA chips. And I thought exactly that, which is like, you'll still need the NVIDIA inference and you'll just have much higher usage. So to me, it's the most screaming buy of the century. Do you share my optimism on NVIDIA given what you just said in Jevons Paradox?
- JRJonathan Ross
So I think over the long term, the, the only thing I say is, um, you know, Warren Buffett and Charlie Munger, "In the short term, the market is a popularity contest. In the long term, it's a weighing machine." Um, I can't tell you about the popularity contest, but, uh, in terms of the weighing machine part, like th- there, this is a misunderstanding. It's actually more valuable thanks to DeepSeek, not less valuable. Okay, so Jevons Paradox was actually discovered by, by Jevon in, in, you know, um, as, as recently made famous in Satya's tweet. However, I did beat him to that, um, by quite a bit and, and just as Satya likes to say that he made Google dance, I'm gonna say I made Satya dance, right?
- HSHarry Stebbings
(laughs)
- JRJonathan Ross
(laughs) But he might take exception to that. But, you know, less than a month before he posted that, I, I did a, a, a cute little tweet on it. Um, so what's really happening here was in the 1860s this guy Jevon, he actually wrote a treatise on steam engines, which I guess is what you did for fun back then in, in England. And, um, he realized every time, uh, steam engines became more efficient, people would buy more coal, which is the paradox. But if you think about it from a business point of view, when the OpEx comes down, more activities come into the money, so people do more things, right? And, and so what's happened is every time we've seen the cost of tokens for a particular level of quality of models come down, we've actually seen the demand grow significantly. Price elasticity baby, right? And so, (laughs) um, you know...
- 41:01 – 42:55
Nvidia's High Margins and the Strength of Their Moat
- JRJonathan Ross
- HSHarry Stebbings
A lot of people suggest that NVIDIA's incredible high margin status, which I'm gonna butcher, I can't remember what it was in their latest, uh, re- release, it was something 45 or whatever it was, but it was very, very high. And then relate to your margin is my opportunity. I think of it back to the seven powers and go, their margin is their defensibility, and it makes me really just consider the strength of their moat. Do you think your margin is my opportunity? Or do you think their defensibility is their margin?
- JRJonathan Ross
Today there's this wonderful business selling mainframes with a pretty juicy margin because no one seems to want to enter in that business. Um, training is a niche market with very high margins. And when I say niche, it's still gonna be worth hundreds of billions a year long term, but inference is the larger market. And I don't know that NVIDIA will ever see it this way, but I do think that those of us focusing on inference an- and building stuff specifically for that are probably the best thing that's ever happened for NVIDIA stock, because we'll take on the low margin, high volume inference so that NVIDIA can keep its margins nice and high.
- HSHarry Stebbings
Do you think the world sees this?
- JRJonathan Ross
No, and I was actually, like we raised some money, um, late 2024, and in that fundraise we still had to explain to people why inference was going to be a larger business than training. And remember, this was our thesis when we started eight years ago.So for me, I struggle on why people think that training is gonna be bigger. It just doesn't make sense.
- HSHarry Stebbings
Just for anyone who doesn't know, what's the difference between training and inference?
- JRJonathan Ross
Training is where you create the model, inference is where you use the model. You wanna become a heart surgeon, you spend years training and then you spend more years practicing, right? Practicing is inference.
- 42:55 – 49:50
The Future of Efficiency After Nvidia's Success
- JRJonathan Ross
- HSHarry Stebbings
Uh, and I'm thrilled to hear you share my optimism around NVIDIA. Um, where does efficiency go from here? Everyone was so shocked by how R1 is so much more efficient and what we've seen from it. What next?
- JRJonathan Ross
What you're gonna see is everyone else starting to use this MoE approach. Now there's another, there's another thing that happens here. So we talk-
- HSHarry Stebbings
An MoE approach, just so I understand, is like the segmentation of where information goes so it's rooted to, like, the optimal point of the model?
- JRJonathan Ross
Yeah. It's called... So MoE stands for mixture of experts. So when you use LLaMA 70 billion, you actually, um, use every single parameter in that model. When you use Mixtral's, um, uh, eight by seven B, you use two of the roughly eight B, you know, experts, although there's some shared weights on top of that. But it's much smaller and effectively, while it doesn't correlate exactly, it correlates very closely. The number of parameters effectively tells you how much compute you're performing. Now, if I have a... Like, let's take the, um, let's take the R1 model. So I, I believe it's about 671 billion parameters versus 70 billion for LLaMA. And there's a four or five billion dense model as well, right? But let's focus on 70 versus 671. I believe there's 256 experts, each of which is somewhere around two billion parameters and then it picks some small number, I'm forgetting which, maybe it's like eight of those. Um, maybe it's, um, 32. Whatever it is. Or 16 of them, whatever it is. And so it only needs to do the compute for that. So that means that you're getting to skip most of it, right? Sort of like your brain... Like, not every neuron in your brain fires when I say something to you about, um, you know, the stock market, right? Like, it... The neurons about, you know, playing, you know, football, um, like, those don't kick off, right? And so that's the, that's the, uh, intuition there. Now, previously for... It was, it was famously reported that, uh, OpenAI's GPT-4 had I bel-... It started off with something like 16 experts and they got it down to eight. I forget the numbers. But it, like, started off larger and they shrunk it a little, um, and they were smaller, whatever. Um, and then with, um, uh, what's happened with, um, the DeepSeek model is they've gone the opposite. They've gotten, they've gotten to a very large number of experts. The more parameters you have, it's like having more neurons. It's easier to retain the information that comes in. And so by having more parameters, they're able to, on a smaller amount of data, get good. However, because it's sparse, because it's a mixture of experts, they're not doing as much computation. And part of the, the, the cleverness was figuring out how they could have so many experts so it could be so sparse so they could skip so many, um, of, of the parameters.
- HSHarry Stebbings
But if we take that then back to like, that's where we are today and how they've become so efficient, what's the, the next stage of that then?
- JRJonathan Ross
Oh, here, here's-
- HSHarry Stebbings
'Cause they have all these experts, they can route it so efficiently. What now?
- JRJonathan Ross
Here's a fun one. So Meta recently released their, um, uh, LLaMA 3.3 70B and it outperformed their 3.1 405B. So their s- new 70B outperformed their 405. And what was surprising to me... I thought they retrained it from scratch. It turns out, um, you read the paper and they talk about how they just fine-tuned. Um, so they used a relatively small amount of data to make it much better. Again, this goes to the quality of the data. They have higher quality data, they took their old model, they trained it, it got much better. But that 70B, that new 70B outperforms their previous 405B. And so what you're gonna see now is now that everyone has seen this DeepSeek architecture, they're gonna go, "Great. I have hundreds of thousands of GPUs. I'm now gonna use a lot of them to create a lot of synthetic data and then I'm gonna train the bejesus out of this model." Because the, the other thing i- is, um... So while it sort of asymptotes, the, the question is on this curve, where do you stop? It depends on how many people you have doing inference. You can either make the model bigger, which makes it more expensive, and then you train it on less or you make it smaller and it's cheaper to run but you have to train it more. So DeepSeek didn't have a lot of users until recently and so for them, um, it would have never made sense to train it a lot anyway. They would much rather have a bigger model. But now what you're gonna see is all these other people either making smaller models or trying to make higher quality ones at the same size but just training it more.
- HSHarry Stebbings
Can, can I ask you, we've seen DeepSeek now say, "Hey, only now Chinese phone numbers and we can log in." Like, that is the new sign-up, I think it is. What's happened and what is the result of that?
- JRJonathan Ross
So they ran out of compute, and this is why... This is the other reason why, um, chip startups are gonna do just fine because they ran out of inference compute.You train it once, but now... So, y- you spend money to make the model, like designing a car, right? But then each car you build costs you money, right? Well, each query that you serve requires hardware, and you're gonna use a... Like, training scales with the number of ML researchers you have, inference scales with the number of end users you have.
- HSHarry Stebbings
Do you think Deep Seat are truly astonished by the response they've got from the global community? Or do you think-
- JRJonathan Ross
I-
- HSHarry Stebbings
... they knew this would happen?
- JRJonathan Ross
I think they marketed very well. Like, you look at some of the publication, and they make it sound like it's a philosophical thing, and, you know, they talk about they spent six million on the GPUs, and, and everyone just zoomed in on that, neglecting the fact that, um, LLaMa's first model was trained on, like, I think five million worth of GPU time, um, and it set the world on fire in a good way. Um, and then ignoring the fact that they spent a ton generating the data and all this. They're really good at marketing. I, I think they were probably surprised at how well it worked, but I think this is what they were going
- 49:50 – 54:01
The $500BN Stargate Project
- JRJonathan Ross
for.
- HSHarry Stebbings
Is there anything that I haven't asked or we haven't spoken about that we should?
- JRJonathan Ross
Well, yeah, maybe ask, like, "Wh- what's up with the $500 billion Stargate effort?"
- HSHarry Stebbings
(laughs) Okay, what's up with the $500 billion Stargate effort? Do you buy... Those numbers are bullshit.
- JRJonathan Ross
I've gone back and forth on that. I, I actually did... So, Gavin Baker tweeted some math. Before I saw that tweet, I came up with very similar math, like, spookily similar math. However, talking to some people in the know, (clears throat) some of the comments are, "Actually, they've got it." But then, you keep pressing, and it's like, "Well, maybe is there some cutesiness to it?" What I think it is, is an acknowledgement that the models have been commoditized, and infrastructure is what's important in terms of maintaining, uh, elite. Like, scale, it's one of the seven powers. And so, I think what you're seeing there is an attempt to move from having a cornered resource, or something like that, into a scale economy.
- HSHarry Stebbings
Do you think it will work?
- JRJonathan Ross
I don't think you get there in a short period of time with GPUs, because most of the compute is inference. And so, you know, if you're talking about, like, building out all the power, building out... Like, it's gonna take time. It's infrastructure. It's CapEx. I think the real win here is brand. That's what I would be doubling down on. I would be, like, hiring the best brand firms I could. I would do a complete makeover.
- HSHarry Stebbings
Will OpenAI have a stronger or weaker brand in three years' time?
- JRJonathan Ross
Much stronger. I think they're gonna double down on that, and they're gonna focus on it.
- HSHarry Stebbings
Wow. Who will lose?
- JRJonathan Ross
People who can't adapt to disruption. Anyone who just wants to keep going on a straight line and, and do what they were doing before is gonna lose. And the rate of disruption is probably gonna increase, because... So, think about it this way. Going back to the analogy of, um, LLMs being the printing press. Imagine if there were a couple of smartphones left over from an ancient, um, civilization. All of a sudden, the printing press is invented, and you're like, "Ooh, Uber's coming. I'm gonna position for it." Right? "I know where this is going." We are the smartphones. We know where generative, um, age technology goes, right? And now everyone's like, "Well, we know how big this gets. Let's put money into it." Right? "I can't be the one who doesn't spend money on this, 'cause I know how big of an advantage it's gonna be." It's like getting to add more workers to the workforce. And so, uh, I just think you're gonna see... Or, I think the generative age, we're gonna speed run it faster than whatever comes next, because we know what it looks like.
- HSHarry Stebbings
Is there any chance we see a plateau? We saw it in, uh, self-driving, for example, where we kind of went through this desert of lack of progression, and suddenly, all at once, it came. Will we see that, or will we just see this continuing downwards?
- JRJonathan Ross
I, I think with self-driving, the problem you had was the, the threshold h- well, it had to be way superhuman, because if you look at the number of miles driven by these self-driving vehicles, it's an enormous number, and the number of fatalities and incidents is lower per mile. But we have no tolerance whatsoever for them when it's a machine. When you're writing poetry in code, it's very different, right, versus, um, doing a surgery or, or driving a car.
- HSHarry Stebbings
If you're Elon and x.ai, how are you feeling? And do you feel better or worse post this?
- JRJonathan Ross
Well, I would probably feel... I would probably feel both better and worse. I'd feel better about my bet on building out more hardware. I would feel worse about trying to build out my own model. Like, "Why is Elon doing that?" Like, there's plenty of... Like, just pick one up off the ground. Like, why are you making your own?
- 54:01 – 57:54
Excitement or Nerves in the AI Arms Race?
- JRJonathan Ross
- HSHarry Stebbings
Are you excited when you look forward at the next few years, or are you quite nervous? You could say this is a time of heightened international warfare in terms of this new AI arms race. China's stealing everything, us forced to steal back.
- JRJonathan Ross
Long ago, I stopped having, like, good days and bad days. It's yes, it's how many good things, it's how many bad things, right, when you, when you run an organization? And so, I'm both excited and nervous, and I'm excited and nervous about different things at the same time. The thing that I am most nervous about is that, unlike nuclear war, you can use AI tools to attack each other. Google just announced recently the first zero-day exploit...Found by an LLM that was previously unknown. Yeah, that's a scary one. So now-
- HSHarry Stebbings
Why is that bit scary for anyone who does an unsound zero to exploit?
- JRJonathan Ross
So, um, how would you like me to have access to your phone?
- HSHarry Stebbings
Not ideal.
- JRJonathan Ross
How would you like the CCP to have access to your phone?
- HSHarry Stebbings
Even less than ideal. (laughs)
- JRJonathan Ross
That's a nation state, and nation states have a lot of resources. And if they stand up a bunch of compute and they start scanning for vulnerabilities and all the open source that's out there, and not even the open source, just like scanning, um, you know, ports on the internet and trying to figure out if they can break in, they can just automate that now. They don't need to hire people to do that. And now the defense has to be automated, because there's no way to keep up with automated attackers and what happens if this gets out of control? But worse, it's- it's a small enough, like it's not killing anyone. It- and it's also deniable. That's the hardest part about it, because is it r- I mean, is it really China? Is it Russia? Is it North Korea? Is it a- a friendly that's making it seem like it's one of them or vice versa? So now you have this ability. So you go from where we had a Cold War, because having a war was unconscionable, it was unthinkable, because of the consequences, to now, "Yeah, I'm just hacking you," and that could spiral out of control. So I'm- I'm worried that we're gonna have more back and forth. And- and think of it this way, if you are a nation state, and you... Le- let's say that, Harry, y- you know, you're- you're a beacon to the- the venture community and you want to rally the European entrepreneurs to be risk on, right? And I'm someone who doesn't want that because I don't want the competition, a country that doesn't want that. Maybe I sully your reputation, maybe I make you persona non grata, and how is that any worse than shooting someone? It could be worse in some ways, but you can get away with it. And so that has me nervous, really nervous, but I'm also really excited, because we are seriously going to be able to innovate as fast as we can come up with ideas. Now, you're not gonna have to implement things. You're gonna be able to prompt engineer your way through things. Just as, um, we moved from hardware engineers to software engineers and sped up productivity, right? You're now just gonna be able to have a prompt engineer who doesn't even write software. One of our engineers made this app where you can just describe what you want built and it builds it. And because, you know, we're so fast, it's like that, and you just iterate and it'll build an app for you and like crazy things. It'll just...
- 57:54 – 59:51
Where Does Value Accrue in Wrapper Apps & Foundation Models?
- JRJonathan Ross
Yeah.
- HSHarry Stebbings
I- I just don't understand where the va- and I'm sorry to just confuse, but where the value, uh, accrues then. Because you mentioned that kind of, hey, they mentioned they created this, you know, tool which allows you to prompt and it'll build the app. I'm sure you've seen Bolt.new. I'm not sure if you've seen Lovable where it's basically ChatGPT but for kind of website creation, in its bluntest terms. Is there value in that? Everyone was like, "There's no value in these wrapper apps." Everyone's like, "There's no value in these foundation models." Where the fuck is their value?
- JRJonathan Ross
And that's part of the exciting part. It's discovering that. But I think people will always prefer to use the highest quality, most polished product. I think there is an opportunity for artisanship, craftsmanship, right? And just perfecting it, right? And- and getting to a certain number of nines in the details. I mean the de- like the Eames quote, "The details aren't the details, the details are the thing." I used to be a little concerned with the quote, um, you know, "If you're not ashamed of the quality of your- your first release, then you've waited too long," because there's a subtlety and nuance there. There's soundness and then there's completeness. What you want is an incomplete product, something that doesn't do everything. That's why you should be embarrassed. But it shouldn't like blue screen of death on you, right? That- that's not a good, um, embarrassment, right? And so what you're gonna see now is because it's so easy to come up with something that just kind of works, it's a little embarrassing, but it kind of works, people are really gonna value well-crafted, high quality products.
- HSHarry Stebbings
Jonathan, I cannot thank you enough for breaking down so many different elements for me, uh, and putting up with my basic questions. You've been fantastic. And honestly, I so appreciate the short notice.
- JRJonathan Ross
No problem. And, um, good luck and, um, uh, have fun out there. I mean, this is a brand new age. It really is.
Episode duration: 1:00:01
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 4P6CUIPrrtU
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome