The Open-Source AI Reality | How Token Costs Will Fall 10X & Usage Will Explode 100X | Lin Qiao

1. 0:00 – 2:00

   Intro

   1. LQLin Qiao0:00

      What I don't want to see is there's only one company owns intelligence. That doesn't make sense to me. I think last year is the year of coding, and this year is the year of co-work.

   2. HSHarry Stebbings0:10

      And in the hot seat today, a founder who I wrote a $10 million check for after just a 15-minute meeting. Lin Qiao, founder of Fireworks. This was one of the easiest investment decisions that I've made in a 10-year investing career.

   3. LQLin Qiao0:24

      I do think the cost of token will go down drastically, 10X cost reduction in the next three years, and this 10X cost reduction will drive 100X usage. We absolutely are not gonna move into application layer. Very clear to us. Whether we will move down into data centers and so on, that could be always be on the table, but the question is-

   4. HSHarry Stebbings0:45

      Ready to go? [upbeat music] Lin, I am so excited for this. Um, I heard so many great things. I just got off the phone with your co-founder, Dima. Uh, I spoke to Alfred Lin, Sonja, Matt Miller, many more. So thank you for joining me.

   5. LQLin Qiao1:10

      Oh, thanks for having me.

   6. HSHarry Stebbings1:11

      Now, I heard that [laughs] Eric Vishria has a rule, don't invest in big tech directors, but he broke that rule with you, which is very special.

   7. LQLin Qiao1:23

      I, I think so too. So a, a funny story. Um, after we decided to handshake, he did call me and said he talked with, uh, one of his advisors, and his advisor questioned him, "Hey, how many big tech executives have you seen being successful in starting company?" Very few. And he told me that. Um, I was surprised, like are we breaking our [laughs] handshake now? No, but, uh, we, since then, we work very closely with each other.

   8. HSHarry Stebbings1:53

      Eric is one of the best. You also started the company when you were 48?

   9. LQLin Qiao1:58

      Oh, yeah.

2. 2:00 – 3:47

   Why Starting a Company at 48 Was an Advantage

   1. HSHarry Stebbings2:00

      That's quite late. Can I ask, how do you reflect on being a 48-year-old founder when we glorify starting a company when you're pretty much 15 these days? [laughs]

   2. LQLin Qiao2:12

      Um, I, I didn't think deeply about that. I always want to have a tech business myself. I actually s- want to start a business in 2015, uh, because I, I'm a first-generation immigrant. I came to US in 2000. I did my PhD in distributed system, uh, computer science, especially focused on databases. And database is very concept system to build, a lot to, lot of different objective optimize for, and pretty much touched, after I, uh, joined research lab, I pretty much touch every single aspect of processing data. And then I moved to LinkedIn to kind of further it down to build systems and products, um, to be used, drive real impact. At that time, I feel I'm ready to start a company. I know all the tech. I know what product to build. I have a business proposal. I have a list of people I want to start a company with, and I spend time think about it in that past because I don't think I have the stu- skill set on people to build a company. It's not a just about product, it's not just about tech, it's actually about people. And I decide I want to go to a place I can learn the most of, uh, of people, and the best company at that time is Facebook. Uh, it's a rising star, uh, in Silicon Valley, and secretly, I was planning to learn for one year or two and leave and go back to do my own business. I stayed there for seven years.

3. 3:47 – 9:49

   The AI Layer Everyone Is Overlooking

   1. HSHarry Stebbings3:47

      So with Fireworks, you saw something in inference that the world was not focused on. The world was focused on training. I think it's helpful for people to understand kind of the stack, 'cause beneath you, there's obviously kind of chip providers and your Nvidias of the world, and then you've got above you the model providers, and you sit in between. Why is that a valuable part of the stack and not a commodity?

   2. LQLin Qiao4:11

      That's a really good question. But why bother specialized intelligence? Why not just use generalized intelligence and, and you worry less things, right? You just kind of build on top of a, uh, API, uh, that provided by frontier labs. Wouldn't that, wouldn't that be much easier? So the argument is the following. If you think intelligence is a derivative of data, then majority of the data is actually not used for training a general intelligence model. The training data is coming from public internet and the label data. Public internet is very small corpus of data compared with world's data. Majority of world's data actually private data locked inside application, locked inside enterprise. It will never get shared with anyone else because this is company's proprietary IP. So, so then it's interesting. If you look at the space, then it becomes very interesting because majority of data is not being activated to derive any intelligence, and that's where we believe in is, is to activate that data, and we believe the future of the frontier of the intelligence a- actually private intelligence or specialized intelligence. So that's kind of where Fireworks, from the beginning, we have been focusing on driving the value.

   3. HSHarry Stebbings5:35

      I have so many questions to ask you. I totally understand you in terms of the values in private data within some of these largest companies. Is that not the premise of what Anthropic's enterprise business is, though, with Claude Co-work and with a lot of their adjacencies that they're building? Would Dario not say that that's exactly what we're going after?

   4. LQLin Qiao5:55

      That's interesting because I view Anthropic as a company fully believing AGI The definition of AGI is there's this one model that can solve all the problem in the best way. That, to me, that's the definition of AGI. To me, that means you do not need to specialize, and that one model should be able to solve all the problems. It's so intelligent, have so much knowledge of every parts of, um, of the businesses, every parts of the jobs it, it can, it can, it can fulfill. Then why do you need to bother specialize? So, so that itself is a validation that we're living in a world that's not ruled by one principle. We are living a fully diversified world. Give you, uh, one example, right? Different region will have different value systems, will have different policies, uh, will have different way of conducting business, will have different lifestyle. It's all taste, choices, judgment combined. Um, I think that's what define us as human. We are not robots. If, if our future world is gonna be ruled by one standard, a taste dictate by one company, we turn ourself into an army of robots-

   5. HSHarry Stebbings7:27

      [laughs]

   6. LQLin Qiao7:27

      ... and that's very depressing to me. And, um, I think what separate out, um, Homo sapiens from other species is the creativity, is the deep desire of pursuing, um, new things, of discovering new ways of living. That define us as a human being, and that part cannot be copied. That's my fundamental belief. Um, that's why, um, you know, in Silicon Valley, there's so much creativity. Across the world, there's so much creativity of building new businesses. What is new business? Um, I had this fun, uh, interesting conversation with Jensen after his GTC keynote. Uh, we actually recorded it, and it's interesting, uh, rec-

   7. HSHarry Stebbings8:20

      I watched it. It was great.

   8. LQLin Qiao8:21

      Yeah.

   9. HSHarry Stebbings8:21

      Yeah.

   10. LQLin Qiao8:22

       Uh, recording with Jensen is not really recording. It- he just started having conversation with me. I didn't know his crew already started recording.

   11. HSHarry Stebbings8:28

       [laughs]

   12. LQLin Qiao8:28

       And we just keep talking, you know? It's so easy. We talk about the specialized intelligence. He said one thing to me. Uh, "Lin, you're right. There's no specialized general company, as in every company is built on a special belief of doing things. Otherwise, there's no reason they should exist." It, it feels, yes, logical, but then I start to think back about what he said is profound because every single company is doing something unique that justify their existence. And this something unique is deeply baked into their product design, is deeply baked into their software design and system building, and that's deeply baked into the data, the, um, and the interaction with their user, and their deep u- understanding of their user intent interacting with their product, uh, and engagement and so on. All of that is the fundamental base of why a company should exist. That is not learnable or shared by another company sitting outside. So-

4. 9:49 – 11:40

   Is AGI Really the End Goal?

   1. HSHarry Stebbings9:49

      Can you help me understand then, uh, you know, as a podcaster, I specialize in asking basic questions, so forgive me, but why then do people like Dario, like Sam, like Larry and Sergey talk about AGI in the way that they do as a inevitable?

   2. LQLin Qiao10:04

      I think what they build is fantastic because they are basically building power line to distribute a really great source of intelligence that everyone else can build on top of. That's how I view their, their contribution. And if w- if we don't have this fundamental infrastructure, then we will not have all kinda appliances living in our home. Uh, I love my coffee machine. Um, and it's, it's special branded, right? So but without that power, then we don't get to do the things that are fun, that are unique, that are special, uh, that ingrain my- our, encode our taste. Um, so, so I do think that's very, very important. But the question is, is this power line gonna replace everything we do? I don't think so.

   3. HSHarry Stebbings11:06

      I- the question for me as an investor is, are power lines good businesses? You said about PyTorch and open and the open ecosystem. Open source in the last, I would say three months, we've all realized is actually accelerating so fast, and the capabilities have increased to such an extent that it's not comparable, quite, but it's getting 90% as efficient with, you know, 15 times, to Chamath's statement, more cost-effective. Are power lines good businesses in a world of open source?

5. 11:40 – 14:30

   Open Source vs Frontier Models: Who Wins?

   1. LQLin Qiao11:40

      So, so here's I, I view, how I view open source. So early on when we, uh, founded the company, we had pretty deep debate among the co-founders. What do we do? Do we build our own models, or we build on top of open models? At that time, open model was not, almost like at its infancy. It's a big bet. If we're gonna take that direction, it's a huge bet that, uh, it's gonna do well, right? Um, but with our PyTorch experience, we believe in the open community. We believe in openness. That's a fundamental d- uh, principle we operate with. Uh, because openness give control. Openness gave control to the user. Uh, think about open models, right? Um, once the model is released, you have the full control of the weights. You can change it however you want, it's yours. Um, and then you can build on top of it, right? So, so that is a fundamental different operating principle that we believe in because of our roots, um, in open source before. So we took that bet, and it did pay off in the sense that both open model and closed model, the quality significantly increased, improved over the past two years. Um, to the point, both of it, both of these two streams cross the threshold, cross a quality threshold, it can solve so many problems, right? So within, uh, within Fireworks, obviously, obviously with our, for our own product, we use open model to, uh, to drive our recruiting process, candidate sourcing, um, and the feedback collection. We use open model to, uh, even, uh, drive some internal finance processes. Uh, obviously, uh, for coding, we use open models to help us debug. Um, we, we have a ton of agents within Fireworks ourselves, and we are cost-conscious, so, um, so, so that is important. So both model categories cross the threshold, it solves so many variety of problems. Second is open model cross the threshold is so much easy to tune, okay? So, uh, be able to steer a model is intelligence, is part of model intelligence. Um, and the model intelligence has passed the threshold, is much easier to steer, especially with small amount of data, a small amo-amount of unique data a particular company has, and then we can hill climb towards your eval. Um, and oftentimes, the end result of hill climbing is to solve your unique problem with your data, you are better than a general purpose model, okay?

6. 14:30 – 17:48

   Are AI Giants Massively Overvalued?

   1. HSHarry Stebbings14:30

      When 90% of enterprise workflows can be done, as you said, that the incredible array of functions that you now use open source for with open models, so the usage for frontier models will not be as large as it was if it was needed for everything. So are these companies actually dramatically overvalued and overestimated if the majority can just go through open?

   2. LQLin Qiao14:54

      I think people start to realize it.

   3. HSHarry Stebbings14:56

      Yeah.

   4. LQLin Qiao14:57

      Uh, I remember from two years ago, uh, I went to different places and talk about an interesting phenomenon, um, that doesn't exist in, in the past, in the SaaS era. During SaaS time, product market fit and the durable business almost are equivalent to each other. The hardest thing is find product market fit, and then once you find it, it's just scale as fast as you can, right? Because CPU is a commodity, the infrastructure you build on top of it is almost like a commodity. You don't even worry about that as a- your Cogs. And now, product market fit and durable business are two separate concept. Um, uh, for startups, you know, we have great company that have product market fit, customer want to pay them, and they really value their product, but they cannot scale because once they scale, they could scale into bankruptcy. Have you heard about scaling to bankruptcy? [laughs] So that's a real problem. Um, it's even a bigger problem for incumbents, so the big companies for digital native, um, because they have the traffic, they have huge amount of traffic. They're the winner from a decade ago when we-- they were startups. And they have so much traffic, once they roll out, uh, those AI features, they're gonna reach to all their customer base, and they cannot afford to do it because their CFO look at their, um, cost proposal or cost forecasting as kind of there's no way you can justify this, right? So, so then it becomes a real problem to all those innovators, "Hey, we really want to plug into this new technology, new disruptive technology, but we cannot afford it. Um, and we need to find alternative to be able to afford it." And the alternative is to have the control over your open weights model and roll out your own model.

   5. HSHarry Stebbings16:48

      It is, or you see what Sam Altman's released in the last few days, which is just dramatically lower cost models. I, I can't remember the amount it is, but I think it's, like, half as expensive or maybe three times cheaper. Um, is the next step actually we just see a massive reduction in price from the frontier models?

   6. LQLin Qiao17:07

      I- it could be, but I think at the same time, it's just a very different, uh, operating principle because, um, for, for open weights model, because it's just there, basically model, uh, acquisition has no cost, right? There's obviously some company train those models, um, and willing to open it up. I, I know, uh, within US, um, there are multiple companies doing that, including NVIDIA is training Nemotron.

   7. HSHarry Stebbings17:40

      Mm.

   8. LQLin Qiao17:41

      Uh, we are working obviously very closely with them. Um, so once the model is there, who- whoever using those

7. 17:48 – 19:23

   Why Open Models Could Beat Closed AI

   1. LQLin Qiao17:48

      model, there's literally no cost. But there's fundamental cost, uh, for, for the frontier labs to, to investing those models and recruit, re- recoup the R&D cost back. So, and second is you just cannot customize those general purpose models. Um, and you use it as is, um, on top of a, a API you have no control over, versus with open model, you have full control. You can, you can tune however you want, you can use it however you want. Especially Fireworks, we, we are a special- specialized intelligence platform. We offer all sorts of tools for you to easily customize the model for one specific use case, um, and after that- Model is tuned with high quality, and then we further optimize for inference deployment. Think about Fireworks. We think about every single model deployment as one size fits one. It's unique for your workload only. Uh, it's optimized for your workload only from quality, speed, cost point of view. Um, so we believe that's, uh, that's absolutely needed because once you think about a production scale of reaching to millions of users, tens of million, billions of user, then even 5% of cost reduction means a lot. It's a massive amount. Let alone what we have seen in the past is five times to 10 times cost reduction.

8. 19:23 – 22:18

   Should We Trust Chinese AI Models?

   1. HSHarry Stebbings19:23

      The one question that I do have to ask is, the concern that enterprises have is national security concerns. When you look at OpenRouters, I think the top six models today are, uh, Chinese models, and they're incredible quality. The speed of development is incredible, but they are Chinese models. Do we have serious national security concerns when analyzing the power of Chinese open source?

   2. LQLin Qiao19:46

      I think it's a huge debate happening right now across the industry. Once the model is open, um, you can, you can put all kind of guardrails specialized to your business around, I would say to all models, doesn't matter if open or closed, you should put your own guardrail around it. The fundamental reason is the following, a model provider will infuse their own judgment, their own taste into the model training process. You cannot guarantee it matches yours. Remember, it goes back to Jensen's comment, there's no specialized general company. Every company is special. Every company will have a special design principle. Every company will have a special taste. Every company will have a special target audience to serve. Because of that specialty, it's guaranteed that the judgment, the taste, the design principle from one company would mismatch, would misalign with your company, which is spec- solve a special problem. So that is, that is a reason you need to tune those models to match yours. Um, and I really believe the future will be, will not be a few small number of AGI models dominant world. I really believe the future will be... It may be scary, but I think that's true. It will be millions of specialized model, one per application per use case.

   3. HSHarry Stebbings21:17

      We saw in the last week actually, reports that China were looking at actually restricting access to their open models because they were seeing the development being so fast and so good. What would happen in a world where China actually started restricting access to their open models, given the lack of open models we have in the US?

   4. LQLin Qiao21:37

      I think it will be a big impact in the short term, but the beauty of open ecosystem is it's not one provider. That's why it's open, right? Uh, it usually attract many, many, many interested party to participate. I do believe, um, in terms of talent density and the resources, I do believe US will be able to build that open system, um, by ourselves, and we should. Um, and, uh, I've seen this happening again, again, in many open systems, there are a thousand flower blossom, and that's the beauty of that.

9. 22:18 – 26:49

   Do AI Startups Need to Build Their Own Models?

   1. HSHarry Stebbings22:18

      When we talk about the, the specialization of intelligence within enterprises, as you have done just there, if we take a very prime example, which I don't particularly want to take [chuckles] because I'm an investor in Lagora, and I think I know which side you're gonna fall on here, but you have two companies that compete in the legal space, Harvey and Lagora, and Harvey have committed to building their own model, and then Lagora have not. Um, a year ago, it looked like companies that didn't commit to their own model were right because, you know, frontier models were increasing so fast in terms of capability. Now it looks like they're wrong. Should companies like Harvey and Lagora be building their own model? And actually, if you don't, what happens?

   2. LQLin Qiao23:05

      So here's one observation I had, and many people have, is software development, uh, s- especially SaaS space, has been significant disrupted because of the general intelligence of coding. And, um, the application development life cycle has significant collapsed in terms of the timeline and the resource needed. In the past, it requires tens of very strong product engineers and PMs to convert from idea to implementation to production scale, multiple quarters of even years of investment. That's a deep moat. And today, one person, a few weeks, can possibly launch their ideas into a product and scale quickly. Um, this is unprecedented, and that's also create interesting dynamics in redefine where the competition is, because it's really hard just to compete on the idea of application, application by itself, um, because many people have similar ideas now. Implementation is no longer such a big barrier.

   3. HSHarry Stebbings24:16

      Is that actually true, though, when you're looking at enterprise deployment, enterprise rollout? If you're working with some of the biggest law firms in the world, I mean, the enterprise sales cycle is, is at least multi-year with relationship build that's very tough, and then you have deployment that's very customized. It's not like ElevenLabs, where you pick it up and go. It's different

   4. LQLin Qiao24:37

      And also, I think legal space is particularly challenging because lawyers are usually more conservative. Legal is also not tolerant at all on errors, right? Because that's why lawyer get paid, right? Is you can, you need to build a very, like, rock solid case. Um, if, if something hallucinate and, and, and generate wrong judgment, then you're in trouble. So I, I do think, uh, the, the legal space is a very interesting space to penetrate, and these companies are both doing great job. But on the flip side, um, I do think both companies are owning proprietary knowledge and information, how to build those assistant to do case studies, to, um, go deep in, in driving, um, you know, legal research and all this, right? So and, uh, I- my understanding of legal is so shallow, but there's so many different, um, versions of flavors of, of cases. So I do think they are in unique position to convert that deep understanding, and they all have data. It's not just about how defensive their business is. It's about, hey, um, oftentime when they build those assistant, there's a harness integrating, uh, and deciding, orchestrating which AI tool to use, um, which tools calling to, and this is bespoke. This is customized. And accuracy of calling those tools, um, and calling to what kind of tools is important, and even that harness need to be co-trained with the model powering it, right? So there are, there's just kind of ample examples of driving that business to excellence by, um, by owning their own intelligence on how to do that in the workflow layer. So maybe it's a timing. Um, coding, for example, I think in coding space, Cursor probably is one of the pioneer-

   5. HSHarry Stebbings26:43

      Mm

   6. LQLin Qiao26:43

      ... starting to tune their model, and now almost all coding company tune their own models.

   7. HSHarry Stebbings26:48

      Does

10. 26:49 – 29:03

    Will AI Model Breakthroughs Ever Slow Down?

    1. HSHarry Stebbings26:49

       that pace of model development slow down? 'Cause every single day it seems like we have a new model with a new capability, and it's like, oh my gosh, Cursor's newest model is amazing. Next, we have, um, uh, someone else, uh, Mistral's newest model is amazing. Uh, Gemini's newest model is amazing. In three years' time, will the pace of model development still be so fast and model superiority be so transient, where one day it's one and the next day it's another?

    2. LQLin Qiao27:15

       So there are, there are a few layers of model advancement. There's base general IQ advancement, so that, those will take sp- step functions. So that's why n- when they release, there, there's always major release or minor releases, right? The major release are step functions. As you remember, beginning of last year, uh, there's a whole, this thinking. The thinking process is new, right? The model just don't spit out answer immediately. The model will think by solving spit out answer is much better that way. Uh, so that's one step function. And, uh, there are many step function we have seen too, but I, I see those as every year or every three quarters, there's a major leap. But at the same time, built on top of those, the best base models, and I can see the specialization start to accelerate because, as I said, it's really like a tree, right? There are so many branches and leaves that can possibly hang on, uh, on the, on the trunk, and as the base model quality start to have step function leaps, and there's so much more we can do to specialize. So I do see in the world specialization is gonna accelerate much faster, uh, than, uh, than the general intelligence part.

    3. HSHarry Stebbings28:37

       When we think about the general intelligence part, just before we move kind of further into the stack of, like, multi-model, uh, Sam proffered the 5% kind of gifting of OpenAI and others to the administration. Do you think we've reached a stage where model development is so advanced and so important to society that they will in part be government or administration owned?

    4. LQLin Qiao29:02

       That's very

11. 29:03 – 32:47

    Why One Company Should Never Control Intelligence

    1. LQLin Qiao29:03

       interesting question. I think, I think there were pres- precedents of that. If we think about the foundation tier of those, uh, general intelligence model as fundamentally a base infrastructure for, uh, for the big, big economy to operate around, there has been precedence of, like, PG&E owns electricity and gas, um, and, and, uh, and so on, right? So, um, I, I actually don't know, but I, I don't want... uh, what I don't want to see is there's only one company owns intelligence. I think that doesn't make sense to me because there are different fla- uh, as I said, there are different flavors of intelligence. There's this general common intelligence that benefits everyone, um, and then there's a specialized intelligence that actually help us ad- advance in, in history to, uh, to think differently, to, uh, create new paradigm of, of living or new paradigm of doing business and shaping the industry. I don't want that to die because there's only one company who can do that. I don't think that makes sense.

    2. HSHarry Stebbings30:23

       With the many models blooming theory, there's the idea that you will route, uh, tasks to different models dependent on what they specialize in.

    3. LQLin Qiao30:30

       I think so.

    4. HSHarry Stebbings30:32

       With that in mind, will you not build your own open router of the world- To cater to that?

    5. LQLin Qiao30:40

       Yes. You, you can argue they are the best built because they deeply understand their use case, and they have the evals. So again, uh, my thinking of what is the frontier is not just this one model. The frontier could be your special routing mechanism for your business, and, uh, you decompose that based on, hey, in order to, uh, fulfill this task, and you, uh, usually, uh, you need a highly intelligent layer, maybe the most ex- expensive open mo- uh, closed models to, to be, uh, to judge at, you know, the highest complexity and s- usually people will also build sub-agents to solve smaller problem, then those can go to smaller open models, and those can also further being customized, uh, to fit into your special design. Um, so I've seen a lot of people already doing that today. And, uh, we also think there's a space to build a automatic routing system that can learn by itself, um, and that compound with automatic tuning system eventually, we think it should all be automated. And then you can see a self-evolving system based on, uh, what flow through, uh, your product, and your co- product keeps evolving. Your product is, is live, right? So you keep, uh, deploying and launching new features and to interact with your users. And, uh, that just kind of, uh, it, it will be a totally self-e-evolving automated system.

    6. HSHarry Stebbings32:17

       Do you think then that routing layer of the stack is valuable? If it can be automated or it can be built on its own, is that a valuable layer to have?

    7. LQLin Qiao32:27

       I, I definitely think so.

    8. HSHarry Stebbings32:28

       You do think so?

    9. LQLin Qiao32:29

       I do think so.

    10. HSHarry Stebbings32:30

        If it can be automated or companies can build it themselves, why would you need a Requesty or an OpenRouter?

    11. LQLin Qiao32:38

        You probably don't. Yeah. We're not there yet. Um, but I do think this is, this could be area of, uh, of innovation.

12. 32:47 – 37:33

    The Secret Behind Cursor's Explosive Growth

    1. HSHarry Stebbings32:47

       You said Cursor being the front-runners in terms of how innovative they've been. I completely agree with you, but I heard, and, you know, I, I really stalk you before shows, but I heard that, you know, CTO Dima was embedded at Cursor for months building the RL infrastructure. Is that how it has to be done, and is that scalable?

    2. LQLin Qiao33:11

       So what's happening is usually in the early adoption curve of new technology, the early adopters are all hackers. A hacker is not in a bad way, is not, uh, does, does have a, a negative connotation. They, they have deep expertise in certain area, and they want to control a lot of things. Um, versus in the late stage of a new tech adoption curve, it start to get more accessible, um, to a much bigger cohort user, doesn't have deep expertise, and they, they need less control. So it always go into deep control first, usually, and, uh, uh, little control later. So we definitely are aiming towards the later stage as the ultimate team want to target, but it's also extremely valuable to understand, uh, what is required to get there. So, so that's why we partner deeply with Cursor. They are the pi- pioneer trying those ideas. They do have researchers from Frontier Labs, and they want to control every single thing, and th- at the same time, we're also pushing to the boundary. We're doing, we're doing things never existed before. We're building system never exist before because we push the boundary that is unique, uh, to, to this particular setting. Okay, what is unique is here, typically, if you think about training, training happens, training is very capital intense. Um, and, uh, and it usually happens in big companies. They have a lot of money. They put those money to buy very expensive training cluster interconnected with each other. Super expensive. And then once you have those ex- expensive large fleet, um, usually you don't need to think too deeply how to be efficient. You just focus on doing your work. Cursor is like us. They are startup, right? Mmm, both of us are very, uh, capital conscious, and, uh, we want to be efficient while we don't want to slow down the research innovation. So together we figure out a, a very smart way to, to drive their training process is, um, they do massive post-training, which is reinforcement learning-based. And reinforcement learning, we break that into p- two pieces. One is, uh, the trainer that, uh, is tweaking the weights of the model and, uh, basically generate a new model version constantly, and that new model will deploy to, we call the RL rollout. It basically is a deploy that new version, interact with a synthetic environment, a synthetic, like, coding environment or real coding environment, um, and then get the reward back to judge if that model is, version is good or bad, right? So that's a rough process. Um, and we decouple these two. In the past, in large hyperscaler, they run that all together. If you think about, you get 10,000, 100,000 chips all interconnected together through InfiniBand, it's extremely expensive and really hard to find. But then you go really quickly and, and we design fully distributed system. Uh, we run across, um, five, six data center regions globally, uh, and tapping to Uh, scatter GPUs, [laughs] uh, and they are able to, uh, run massive jobs, our jobs. But the challenge there is we need to sync model weights across all of these different regions, and then we think how hard can that be? It matters because the latency of delay of sending these weights over is gonna di- dictate how fresh the rewards are. And then if it's too stale, then you are too off. It- so it's a balance. But we, we, we innovate a way we can, uh, we can distribute fresh model weights quickly. It's not too off. Uh, so numerically it's still, still sound, uh, while we are not limited by our very expensive, uh, deployment of GPU fleet. So those are the innovation we work together with Cursor to push the boundary, a- and leading to their recent model launches. We're very proud

13. 37:33 – 41:36

    Is AI Coding Already Yesterday's Biggest Trend?

    1. LQLin Qiao37:33

       of them.

    2. HSHarry Stebbings37:33

       Can I, can I ask you a question bluntly, which is, a incredible customer to have, amazing progress they've had with you, um, and it's wonderful to see that partnership. It's a very large customer for you. How do you think about the concern of a Cursor churn in the wake of a SpaceX acquisition?

    3. LQLin Qiao37:52

       Yeah. Everyone's concerned. The whole entire industry in terms of, in terms of application innovation is by model, in the sense there are few companies are very successful. They escape velocity, but few of them. So that's the shape of the whole entire industry. And last year, Cursor is one of the few. Um, I would say all model companies are concentrated on Cursor. We concentrate on same group of, [laughs] um, app companies. And, um, and since then it, it does change, right? So we do have a very healthy, diversified customer base. Um, especially I think last year is the year of coding. Uh, I think all major coding companies are on us, and this year is the year of co-work. And co-work is much more diversified by itself than coding because there's general purpose co-work, for example, general purpose like, mm, co-work to help you do all kind of research. Um, you want to ask, "Hey, what will be the, um, what will be the Nvidia GPU price, uh, two years later?" [laughs] Um, what will be Anthopic's stock price after IPO? So those are deep research, general purpose deep research. Um, or there are so many different categories of special purpose co-work. Legal, we just talk about two great legal companies. Finance, customer support, recruiting, sales-

    4. HSHarry Stebbings39:26

       Mm

    5. LQLin Qiao39:26

       ... marketing, uh, healthcare. So there's very broad set of co-work space of innovation app company. They are doing really well, and we have them as our customer base. And then more interestingly, we start to see an uptick of consumer-facing company are all start to looking to, um, GenAI technology. And, uh, they are changing how they are thinking about their traditional business of doing recommendation, for example. And that's very interesting to me because, um, we have obviously worked at a huge recommendation system in the world, Meta, um, and we are very eager to see how that trans- transform into a new economic, uh, a new economy for, for us.

    6. HSHarry Stebbings40:16

       I'm s- I'm sorry for being naive here. Um, do people work with just one provider in the inference space like you, or do they work with you and with Together or anyone else in the space?

    7. LQLin Qiao40:29

       I think people are more in tuned to multi-vendor strategy in this space because they don't know what's happening. It feels safe to have multiple providers kind of, uh, to balance things out. But we don't view ourself as an inference provider. Again, we view ourself as delivering these specialized intelligence where we help companies tune their model. Uh, give you some numbers. We- today we process more than 40 trillion tokens a day. Um, so the majority of those tokens are coming from a customized model, not from off-the-shelf models, are coming from customized model. So-

    8. HSHarry Stebbings41:18

       What w- what will that token count be end of next year?

    9. LQLin Qiao41:22

       Anywhere ranging from 20 to 100X could be possible.

    10. HSHarry Stebbings41:28

        20 to 100X?

    11. LQLin Qiao41:29

        Yeah. We're at a very early stage of S-curve of explosion right now.

14. 41:36 – 46:32

    The AI Infrastructure Race Is Just Getting Started

    1. HSHarry Stebbings41:37

       20 to 100X. If, if it's 20 to 100X, the idea that we are in a CapEx bubble is ridiculous, and we are desperately needing far more CapEx than we are ever suggesting for compute. Is that right?

    2. LQLin Qiao41:52

       Um, so that is right. At the same time, I think Jensen has a five-layered cake, five-layered AI cake, uh, from top-down application model infrastructure, chips, energy. We are bottlenecked by the lower part of the AI cake in terms of supply chain. So, um-

    3. HSHarry Stebbings42:20

       Being energy

    4. LQLin Qiao42:21

       ... being energy, being chips, I think in the physical world, how fast we can manufacture, because in the history, all these industries are not designed for massive scaling. Speaking about 100X scaling, no one was designed for that. I talk with many, um, manufacturer, um, it's kind of we're bottlenecked by small parts [laughs] transistor. [laughs] Uh, the, the smallest, tiny parts that hold off the whole manufacture line of, uh, servers that can deploy to data center and be used to generate tokens.

    5. HSHarry Stebbings42:58

       Do you have to be full... Again, to Jensen's five-layered AI cake, do you have to then be full stack to win or to reduce dependencies? We've seen OpenAI come out with Jalapeno, terrible name, Anthropic, uh, talking to Samsung about building their own chips, DeepSeek are building their own chips, Zuck came out with Meta building their own chips. Do you have to be all, all of it?

    6. LQLin Qiao43:22

       It really depends on the company philosophy. To us, agility is everything, and we need to earn the rights of building anything. So focus is everything for us, and we want to focus on where we add the biggest amount of value based on our strength. And, uh, we would like to leverage other people's strength to build on top of. So in particular, um, we want to run everywhere, on all possible AI chips in the world. We don't want to limit it by how much chips we, uh, can bring into our data center, whether we construct it or we rent it. Um, but over time, uh, when the business grows b- very big, right? So I still remember when Meta was young, they, they don't build everything, and when they're big, they make sense to build. You earn the rights to, to build for your own, um, you know, giant traffic, and this, if it save, like, five times more cost, then you should go do it, right? So, um, but I think at the early stage, that's why... I give, tell you interesting story. Uh, in the coding space, we, I- I would say Cursor is the first company to have decided to work with us early on. Uh, I still remember when they work with us, they were single-digit million dollar.

    7. HSHarry Stebbings44:40

       Wow.

    8. LQLin Qiao44:40

       Very small. Uh, this is only two years ago. They grow by 100, 1,000X, [laughs] over two years, something like that. Um, but they decided to work with us early on because they recognized they only want to focus on product innovation and later on research. They do not want to focus on, um, you know, this platform innovation. They know we are putting all our R&D there, and they want to find the best partner to win big. So I do think that's the right mentality to specialize, and we want to specialize. We do not want to kind of own the whole entire stack. That's not our goal as a company.

    9. HSHarry Stebbings45:21

       I'm sorry to be harping on about it. Why does Jensen skip your layer of the cake? 'Cause he's doing Nemotron with models. Why does he not wanna cannibalize your business too?

    10. LQLin Qiao45:32

        Well, Jensen is not building a cloud either, right? You can say, "Hey, Jensen probably have all the rights to build a NVIDIA cloud." Uh, so he's not building a cloud infrastructure. Um, I think he mentioned that as well. I mean, if you ask him that question. Um, and he also mentioned he want to specialize in what they have the rights to do. Uh, why models? I think it's pure, um, a supply chain question, is if US doesn't have a US-native open model, it's a problem. It's a supply chain problem. So, so he is solely there to solve the supply chain problem, but if there's no supply chain problem because the company, uh, like us are providing this specialized intelligence platform layer, then he doesn't need to worry about it. So he just want to make sure the whole entire five layers of AI cake is flowing. There's no blockage, and if there's a blockage, you know, he's interested in solving those problems.

15. 46:32 – 54:07

    How Cheap Will AI Become?

    1. HSHarry Stebbings46:32

       Mark Benioff, one of your investors, I think, in, in your round, which obviously this will come out after the round, um, is announced, um, said that he spends, uh, about 3.8% of developer salaries at Salesforce on Anthropic, uh, and Claude Code, and I think it's a useful analogy because if you assume that that is what's spent on Claude Code and coding tools, that says one side of the market, but if it's 20%, wow, we're underestimating how big these companies can be. When you think forward a year or two, how, what percent of developer salaries do you think we'll spend? Is it less because these tools will get cheaper, or is it more because they'll get better and better?

    2. LQLin Qiao47:19

       I do think the cost of token will go down drastically because, uh, again-

    3. HSHarry Stebbings47:27

       It hasn't so far.

    4. LQLin Qiao47:29

       It hasn't so far because of supply chain constraint, but we are living in a, a free economy. So think about whenever there's shortage, price is high. Price is high, high price will invite a lot of people coming to solve the problem, and it will invite competition. Competition will bring down the cost, and then eventually will leading to a very economical solution, right? So but actually that's good for everyone because much more affordable, uh, infrastructure will invite more usage. So my prediction is with the decrease of infrastructure, um, that's where it comes to my prediction of how, like, how far next year will look like because the infrastructure cost will go down, and, uh, um, usage will explode because of that, right? So the moment you don't think about that as, um, as a problem for you, and you just, you just, i- if it's a utility, you just use it.

    5. HSHarry Stebbings48:33

       How much will token costs come down? Is this... I just, help me understand. Is it, like, a halving? Is it, like, a, oh, it'll be 100th of the cost?

    6. LQLin Qiao48:42

       So there, there's different way to think about this. It's not all tokens are equal. I think we should establish, um- Um, best practice to evaluate the token economy per task. Because different model, it, are, have different way of spit out tokens. Some are much more verbose than the other. So, uh, so you can imagine one model, um, is 2X cheaper than the other, but it's 2X more verbose to solve the same task, and then they're the same cost.

    7. HSHarry Stebbings49:16

       Huh.

    8. LQLin Qiao49:16

       Right? Uh, so, but overall I think, uh, as the model quality improve, I think being precise is gonna be part of the, uh, optimization. And, uh, so that's one level of optimization, is to solve one task we should, we should need less tokens, okay? Uh, and the second is for one token, um... And how to do that is you, you need to customize the model to solve your problem, especially better [laughs] and more precise. That goes into model tuning. Um, and second is for each token spit out from those models and processed by those models, we also specialize in making the unit economics much better, uh, through our platform. And third is underlying infrastructure, uh, like the GPUs, the surrounding, like memories and all this, today is under stark supply chain constraint, is gonna get much better. Situation will get much better. It- I don't think, uh, probably in the next one year or a year half, the situation will not change, but in the long term, two to three years, it should change, and that cost will compress. Uh, so overall I can imagine, you know, 10X, uh, cost reduction in the next three years, and this 10X cost reduction will drive 100X usage.

    9. HSHarry Stebbings50:41

       You said there about kind of, uh, token efficiency, um, and how you enable your customers to be much more efficient. With that efficiency, you do charge more. You know, when I, when I did the research, when compared to competitors, I got, like, Together is price king, and I don't mean this disparagingly, but, like, they're cheaper. If you want cheap, you go there.

    10. LQLin Qiao51:03

        Mm.

    11. HSHarry Stebbings51:03

        And respectfully, if you want better quality product, [laughs] you go to you, but it is more expensive. Do you think that's a fair assessment and a fair analogy?

    12. LQLin Qiao51:11

        Mm. I think we're probably not comparing Apple to Apple in the sense that, uh, again, goes back to our business, majority of our traffic is, um, is customized model. Um, and, uh, we optimize for quality, number one, always quality. Quality as in model quality, uh, towards your applications, your specific business, your use case and so on. The second is, um, when we deliver those model in inference, it's also quality. Um, and we care quality so much, we do extreme things. Um, for example, during training time, there's a very hard thing to achieve, it's called zero KLD. It's a little bit technical. The idea here is-

    13. HSHarry Stebbings51:57

        Zero KLD?

    14. LQLin Qiao51:59

        KLD. KLD is a measure of, uh, of quality. Um, and, uh, what it means is between the training system and the inference system, when model move over, uh, we have bit equivalence, uh, so as in the numerics are fully the same. We do not lose a bit of accuracy. Uh, that's really hard to achieve, but the reason we push that, well, we deliver that, um, and the reason we push that is because we know, um, our primary business is in model customization and inference of customized model. Um, and we want our customers' every single dollar invest in training, maximize it. Um, and then they, if cross-training inference boundary is not bitwise equivalent, they just drop the quality down, and, and then it's like you pay, you pay your training investment by, um, discounted quality. Why do you do that? Um, so, so quality first, and quality does bring additional value, and that's why we are not interested in commoditized one-size-fits-all, um, you know, this off-the-shelf model deployed in same way for everyone, that kind of business. We're always customized model deployed in a unique way, um, for your particular workload.

    15. HSHarry Stebbings53:25

        Two questions. Do you have to have an FD model to make the customized model efficient?

    16. LQLin Qiao53:31

        We, uh, as a matter of fact, we do have a FD team. It's called Applied Machine Learning Engineering team. So their primary job is to accelerate this customized deployment, um, as a matter of fact, to also be the agent to automate a lot of deployments. So-

    17. HSHarry Stebbings53:47

        Given where we are in the stack, a lot of the complexity that we have, we, we have a margin structure that's a little bit different to, like, traditional SaaS being 80%. Uh, uh, I don't, I don't know the margins precisely here, but they traditionally sit in the 30 to 40% range for where we are. Is that the new normal for where we are?

16. 54:07 – 59:30

    Hypergrowth vs Profit: Why Margins Can Wait

    1. LQLin Qiao54:07

       I don't think that's the new normal. I think that is a reflection, at least for us, I don't know other companies. Uh, for us it is a reflection of we are in a hypergrowth phase. Um, during hypergrowth phase, you have the choice, right? You either optimize... To me, margin optimization is a constraint problem, as in, "Hey, we want to go to 70% margin. We want to go to 80% margin," and, and then we are gonna go backwards and impose those constraint to guarantee those margin. And usually constraints slow down, um, innovation. So for ex- uh, giving an example, uh, during system development And, uh, in a high-velocity, uh, system expanding phase, we w- we don't want to overbuild, because we're in kind of high experimentation. We're testing, uh, you know, what will stay, will not stay. Optimization doesn't make any sense. Once we know this is system that we want to build 100%, then, and we are gonna scale this 1,000 times bigger, then we go optimize the heck out of it. I think right now you, you would think about business the same way. We're in hypergrowth, um, if our focus is only optimize growth margin, we absolutely can do that, but we are sacrificing the speed of growth as well because we want to go everywhere. [laughs] We want to go into different, uh, geo regions, uh, we want to go in to tackle different use cases, we want to create con- constantly create, um, different product lines, and, um, and those are not the time for optimization. That's my opinion.

    2. HSHarry Stebbings55:54

       So we will be able to increase margin without moving into different layers of the stack?

    3. LQLin Qiao56:00

       Not into- We absolutely are not gonna move into application layer. Very clear to us. Um, and, uh, whether we will move down into, like for example, you mentioned build data centers and so on, uh, that could be always be on the table, but the question is timing.

    4. HSHarry Stebbings56:19

       I mean, isn't the statement you, you either die or you live long enough to build your own data centers? [laughs]

    5. LQLin Qiao56:24

       [laughs]

    6. HSHarry Stebbings56:24

       As, as Elon or, or Zuck now spending, I think, 10 billion on the latest data c- data center in Canada. Would you like to build data centers?

    7. LQLin Qiao56:32

       So I build data centers at Meta, and also lots of innovation possible there. There's no one-size-fits-all as well. And building a GPU-native data center is also interesting, especially I think there is a potential direction of building, uh... So it's a trade-off, right? Um, from operation point of view, it's much better to build a heterogeneous deployment. Um, it's all the same chips, all the same SKU, as big as possible, and run multiple workloads so it's fungible, right? It's v- very easy to manage, um, um, back nodes. You just gonna- you build one principle, one process to, to maintenance operation. But, um, again, it goes to optimization, but once it's so big, then any optimization is gonna drive a lot of economical return. Um, for example, we're talking about, um, NVIDIA recently acquired a company also called Groq with Q. It's a large SRAM-based, um, ASIC accelerator. Re-

    8. HSHarry Stebbings57:36

       I spoke to Jonathan before this show. He said-

    9. LQLin Qiao57:38

       Jonathan is excellent

    10. HSHarry Stebbings57:39

        ... he, he said what a fan he is of yours.

    11. LQLin Qiao57:41

        Oh, [laughs] I'm also fan of his. Um, so but, um, it's a great combination between a FLOPS-intense, um, GPU and, uh, SRAM-intense, um, ASICs. Because the FLOP-intense is really good for first half of, uh, LM processing. It's prefill, it's called prefill, processing the prompt and so on. And SRAM-intense is really good for generation. That's just the nature of the model architecture. Uh, it's great to combine these two instead of running heterogeneously on the same chip, right? And, and but that requires a very unique system design and deployment into data center, and it is, um, it is heterogeneous, actually, um, before I, I, I, I really m- mean homogeneous design is much better for operation. Um, and this is heterogeneous. And then how to operate this heterogeneous design requires unique innovation in data center deployment and so on.

    12. HSHarry Stebbings58:46

        So data centers aren't commoditized. Like, you can specialize in data center deployment, and one data center is better than another, and data center deployment can be done well and badly.

    13. LQLin Qiao58:58

        Data centers are so complicated, right? If you think about the beginning, all the way from construction to power deployment, and you have the right power to come in, right, uh, fiber channel, um, the, uh, right cooling, uh, especially newer chips requires liquid cooling. Um, to get all this right, and the parts can fall apart, and how to replace them, it is all very deep expertise. It's no joke.

    14. HSHarry Stebbings59:26

        [laughs]

    15. LQLin Qiao59:27

        It's not tomorrow I can be a data center operator. I cannot.

17. 59:30 – 1:01:04

    Can the West Keep Up With China's Infrastructure Speed?

    1. HSHarry Stebbings59:30

       Is that not where you would bet long on China, with the greatest of respects? It, the, especially in the US, one of the biggest, uh, barriers to data center deployment is policy and is kind of local legal infrastructure that prevents it. In China, you don't have any of that, and data center deployment is much, much faster.

    2. LQLin Qiao59:51

       I think in general, infrastructure, the base, uh, the physical infrastruc- construction in China is going really fast. I literally see, um, some kind of, um, crossover bridge is being built within a week. Uh, [laughs] the velocity is very, very high there. Um, and, uh, there's a highway, uh, close to my home, after one year it's not done yet, so this is also a crossover. Um, so I, I do think there is a unique strength probably because of, uh, the population density, and, uh, um, and they are specializing those kind of construction, um, really work. So, um, but I do think, I do think here, um, we, we also have those specialty people, it's just even, I heard even electrician is under severe shortage.

    3. HSHarry Stebbings1:00:49

       Yeah.

    4. LQLin Qiao1:00:50

       We are under... Global supply chain constraint here

    5. HSHarry Stebbings1:00:53

       What change would moving into the data center layer cause to margins? Would that take it from 30 to 50? Would it be not that meaningful? Like, what would that change do to margins?

18. 1:01:04 – 1:04:45

    Why AI Hardware Depreciates Faster Than Ever

    1. LQLin Qiao1:01:04

       How we calculate gross margin is interesting these days, um, because how long does hardware dep- depreciate has significantly changed.

    2. HSHarry Stebbings1:01:13

       Yeah. [laughs]

    3. LQLin Qiao1:01:14

       In the past, it's f- six years.

    4. HSHarry Stebbings1:01:17

       Yeah.

    5. LQLin Qiao1:01:18

       Okay? Solid six years, and hardware release is usually three years. That's fast. And now, within a year from one vendor alone, we have three SKUs, and, uh, the m- newer model usually runs the best on the newest hardware. Model depreciation is also very fast. Every week we are launching a new model. Um, and then the model is kind of peak in its value before the next model comes out. [laughs] Um, and the new model likes the newest hardware. And imagine this cadence after two years, um, which model runs on the two years old hardware? It'll be two-year-old model. Um, and, uh, are those models still valuable? So I think that's kind of the real dynamics we are, we're facing right now is m- the, the hardware will last for six years still, but is-

    6. HSHarry Stebbings1:02:19

       But what you're saying, the speed of model development far outstrips the speed of chip and hardware depreciation.

    7. LQLin Qiao1:02:26

       Um, the speed of model dev definitely is the fastest, but even the hardware innovation itself is the fastest. So after three years, if every year there's three hardware SKU, after three years there are nine hardware SKU in between. Do you still want to go back to, uh, nine generation older hardware running three years old model on that? That's questionable. Maybe there's a world, uh, we still, it's still valuable, but with this pace of innovation, it's questionable. Now with a different depreciation, depreciation cycle, it change the dynamics of build versus own, uh, build versus buy. Um, and again, it goes back to my original thesis of do you optimize for growth or do you optimize for growth margin? It's all about timing.

    8. HSHarry Stebbings1:03:18

       How do you think about that question for yourself [laughs] when you, when, when you're sitting there in an armchair on a Sunday afternoon thinking, "Hmm, we're optimizing for growth now. When is that time to optimize for gross margin?"

    9. LQLin Qiao1:03:29

       Well, I would say we optimize, we want to optimize for both. [laughs]

    10. HSHarry Stebbings1:03:32

        [laughs]

    11. LQLin Qiao1:03:34

        Uh, so, so here's how I think about it. Um, optimize for growth is a l- uh, requires a lot of business planning, assuming there's product market fit. Optimize for growth margin is optimize for differentiation. Um, I, I think I want to avoid over-optimizing for gross margin, but we should optimize for gross margin continuously, as in we should optimize for product differentiation continuously. There's no question about it. And, uh, um, I think we want to continuously optimize towards a healthy growth margin which allow us to grow really fast, and it's a trade-off, and we don't want take compromises. Um, the compromise as in we over-optimize gross margin to s- resulting in very slow growth, right? And one possible way to optimize growth margin, we do not grow at all. [laughs] We just optimize the heck out of it. I know we can hill climb to a high number, but that's absolute disaster outcome.

    12. HSHarry Stebbings1:04:44

        Okay, interesting. If

19. 1:04:45 – 1:08:35

    Why AI Will Create More Jobs, Not Fewer

    1. HSHarry Stebbings1:04:45

       we just said, "Hey, so gross margin, we're gonna take it from 30% to 10%," is it a winner-take-all market where we could eat up everyone else's lunch and then optimize gross margin later?

    2. LQLin Qiao1:04:58

       I think winner eats all probably is not a snapshot in time. It's gonna be a long-term situation. We do see, um, a particular industry will oscillate and start to settle, um, with a few good ones. Um, legal, take legal for example. Uh, I was on a, a dinner table, and interesting, it seems like there were a lot of those companies around, um, two years ago, but now it's pretty much-

    3. HSHarry Stebbings1:05:30

       Two

    4. LQLin Qiao1:05:30

       ... two. Um, so, um, I, I think, yeah, I think it's a long, long game.

    5. HSHarry Stebbings1:05:37

       How do you see the more mature state of your market? Is it like a, a cloud market where you have obviously Azure, AWS, GCP, or is it an Uber and a Lyft where one takes 90% and the others kind of fight for scraps?

    6. LQLin Qiao1:05:50

       We're, we're in the adoption curve where a lot more companies, they are in the AI space, start to seriously think about moving to specialized intelligence, to start to seriously think about owning their intelligence is better than renting. Um, because going back to this optimization, when is the good timing, right? So it's the same question we are answering for ourself when build versus buy, and our customers are also thinking about build versus buy or build versus rent, own, own versus rent, right? I think AI journey or AI adoption journey has gone further along into a lot of company has meaningful traffic. A lot of company is deploying AI into production. A lot of company is at the phase of scaling, um, and that's where optimization kicks in. When o- optimization kicks in, you need to have control to optimize. If you don't have control, you have n- you just don't have the range to optimize. And for you to have the control, then you have to build on top of some, like, open model. You have to kinda turn your data into intelligence. That's pretty much the, the path we have seen so many companies, uh, across industry, they reach the same conclusion, they are moving towards disruption.

    7. HSHarry Stebbings1:07:11

       Speaking of owning your own intelligence versus renting it, that does apply to, like, a national layer, and when we've seen, you know, like, Fable be banned, in some cases by the administration briefly for 19 days, um, especially in Europe, we suddenly went, "Oh my gosh, we cannot be at the hands of OpenAI and Anthropic where we can just be banned and our health services sit on the infrastructure of something that, you know, an administration can turn off." Do we see a future of sovereign models where large nations or nation blocks own sovereign models?

    8. LQLin Qiao1:07:46

       I definitely see that possibility. Um, I also see if we think about the general intelligence model as the electricity layer, as a power line, every country should, uh, should own their own power line, right? So, um, I, I do... I, I think that is a very scary moment, is my power line's gonna be cut off and all my fundamental, um, day-to-day is gonna not working 'cause I feel fru- so frustrated whenever there's a power outage in my home alone. [laughs] I feel so frustrated when I cannot access my wifi. I feel so anxious.

    9. HSHarry Stebbings1:08:23

       [laughs]

    10. LQLin Qiao1:08:23

        I... So, uh, I mean, obviously the, you know, operating the country is, is extremely important built on top of this fundamental, uh, baseline, so, um,

20. 1:08:35 – 1:16:55

    The Biggest Mistakes AI Founders Are Making

    1. LQLin Qiao1:08:35

       and, uh, uh, for every single company, the same thing. And it's not just on whether a country should have their unique, uh, sovereign independence, but every single company should have their independence. Um, you don't want any single person to cut you off. Uh, that's extremely scary moment.

    2. HSHarry Stebbings1:08:54

       Why would you move into the data center space, but you wouldn't move into the chip space?

    3. LQLin Qiao1:09:00

       Because I know this, uh, building a chip is extremely hard.

    4. HSHarry Stebbings1:09:03

       I thought so too, okay? Again, I, I, I admit to being a moron, which is why I think this show's a little bit successful. I thought so too, but then how come everyone is seemingly doing it as if it's, like, just another product? As I said, OpenAI, Anthropic, DeepSeek, Meta, we're building our own chips now.

    5. LQLin Qiao1:09:20

       I think Meta has been building their chips for more than five years, m- way more than five years. Um, and MTIA has been project since, mm, you know, 20- 2018, uh, maybe earlier. So because Meta has been investing AI for a long time, pre-gen AI, um, and they have a huge, uh, AI workload focused on ranking recommendation, and Meta has been building other hardware as well in the past. So whenever the, the, the usage has passed certain threshold, it make economic sense for you to build it, build the underlying supply, right? So, uh, and then you can specialize towards your workload, and, uh, that's another form of specialization, is specialize, to bake your logic into hardware, and this hardware is purpose-built for your particular workload. And you better un- uh, you better make sure this workload doesn't change, because really hard, once the hardware is tapped out, it's really hard to go back and change it. It's very... It's possible, it's very costly. Um, so once your workload stabilize, once your business stabilize, it doesn't change too often, then that's the time, um, to consider building a chip. I still see the whole AI world, especially with models, customization is very dynamic. Very, very dynamic. Workload pattern is very dynamic. So think about how much energy in the application space, people are experimenting, all kind of things. You don't know which one is gonna take off, and they, they will just take off quickly, and which one, once they take off, which one gonna sustain, and a few ones will sustain, then that's the time. Oh, now we know this is a pattern, and now we should probably encode this pattern into hardware, uh, and, uh, bring this hardware into a data center, and so on. It's all cascading, and then it's gonna cascading down. To me, it's a fernal question. Where are we in the stage of fernal- maturity fernal, I mean. Um, we're still in the early stage of workload maturity fernal to warrant a chip that will be durable. Um, so, so now you go back to, oh, the, we have so many accelerators that are successful. Some are really successful, but remember those ASIC company, they started before gen AI. They start from some thesis to optimize some workload, and they all pivot to AI and trying to kind of, uh, fitting AI workload. It's almost like you bet before this AI workload emerges, and now it becomes a serendipity question. Um, are you lucky enough this just work, right? Uh, and some really worked. Um, some fundamental design of putting a lot of SRAM on the chip is great for AI model because they are, uh, memory hungry, um, and this really accelerate, uh, the execution of inference and so on. So, so those works, um, and some doesn't work.

    6. HSHarry Stebbings1:12:27

       What do you see as the greatest bottleneck today? You know, I think it was when I had Jonathan from Groq on the show, he said, like, HBM was the greatest bottleneck, and that's why you've seen the 5X increase in price. [laughs] What do you see as the greatest bottleneck that people don't talk about enough?

    7. LQLin Qiao1:12:40

       I still think we don't have a great system for very large model. I really believe the fundamental low-level infrastructure cost will go down. So t- for solving tasks, we should need less token. That will increase, so collectively, um, the cost will significantly reduce- Uh, therefore, we can run the highest intelligence model much more ubiquitously in the future, but we don't have a system designing for that. For example, we don't have a great system designed for, um, 10 trillion parameter models today, and, uh, that will require very smart engineer co-design from the model to, um, the customization serving platform layer all the way to chip layer. The chip is not the individual chip, but the system, collection of chips in system, um, and all as a total package. I think there's still a lot of innovation we can do.

    8. HSHarry Stebbings1:13:45

       I think recently you announced that you were at 800 million in ARR, um, incredible feat and scaled so fast. What is that at the end of this year?

    9. LQLin Qiao1:13:54

       We think we can at least double.

    10. HSHarry Stebbings1:13:56

        By the end of the year? Wow. You know what's so interesting for me as a, a venture investor? I've been in investing for 10 years. We used to be in the day where Slack was the golden child, where, like, 1 to 10 million in revenue in 18 months was, like, amazing, and now we have companies like Fireworks, where you scale to 800 million in revenue in a matter of years, and you mentioned Cursor scaling to, you know, billions in revenue in a matter of years. The speed of company revenue growth is just unparalleled.

    11. LQLin Qiao1:14:32

        I think it's because there's a fundamental disruption in this technology that is all-empowering, um, and, uh, all-empowering in the sense it reach out to every individual one of us to be creative, uh, and it unleash the, a lot of creativity that we just don't have access to. Um, and that's why we're seeing this phenomenon of extremely fast growth because of a demand.

    12. HSHarry Stebbings1:15:06

        Final one before we do a quick fire. You, you hired George Hu, uh, who was president of Salesforce. He's exceptional. He's one of the most direct, no BS operators I've ever met. But you met him I th- a couple of years before or a year before, and you were like, "Oh, we're not ready for you yet." Why did you say that, and why did you decide now was the time?

    13. LQLin Qiao1:15:29

        Right. So a year ago, I think we were probably just 50 people. So today, we're at 200 people. We're still not that big.

    14. HSHarry Stebbings1:15:38

        Wow, you're 4 million ahead.

    15. LQLin Qiao1:15:39

        Yeah.

    16. HSHarry Stebbings1:15:40

        Wow.

    17. LQLin Qiao1:15:41

        So, uh, at 50 people, I'm more thinking about, uh, scaling the product first, then scaling, you know, massively scale the business. Um, and then, and we, uh, we, we talked, and, uh, I have huge respect to him, but I know he's a legend. He's legendary. He's a legendary operator in Silicon Valley. Um, and, uh, I just feel like we're too small for him, and I told him that, "Hey, we're, we're probably too small for you, but I would like to work with you at some capacity." So he helped me, uh, he helped me actually build out the team, interview a lot of executives. Uh, his feedback is always well-balanced, very thought- very thoughtful, and we start to work together in that capacity until, I think, end of last year, we're growing really fast, and he knows, and we start talking seriously, and that early relationship paid off. So, um, he's really cool. He's, he's really cool in the sense that he-

    18. HSHarry Stebbings1:16:46

        He's so cool. [laughs]

    19. LQLin Qiao1:16:47

        He, he did a lot of things, um, a great accomplishment in, in the past, but I, I find a unique

21. 1:16:55 – 1:18:44

    Why Great Leaders Stay Close to the Work

    1. LQLin Qiao1:16:55

       character, uh, about him is he's extremely experienced, has high attitude of business vision, but he's also very curious. He doesn't make assumptions. I know it all. I've seen all the movies. It's the same movie, uh, and let me just kind of direct this movie as I did in the past. So he didn't come with that attitude. He knows AI goes in insanely fast pace, and, uh, he's learning along the way, but also fully embrace AI. Um, actually, his team, our GTM team, is using all kind of AI agent. They're s- they're sharing skills, um, so they maximize their productivity, and he knows we have a super linear demand curve, and, uh, he-- there are just certain pace we can build our GTM team. In order for us to catch this curve, we, we need to build our team, but the team need to also have increasing productivity to match. Uh, so that's a problem he's solving, and, uh, I feel very fortunate to work with him. And I, in general, I feel in the AI space, the unique part is people need to have very special traits, almost like contradictory characteristics. Uh, for example, very experienced, but super, um, curious in the fast learning curve, or Dima, we talked a little bit earlier, he is brilliant, high intellectual horsepower, but extremely humble. It's weird combination.

    2. HSHarry Stebbings1:18:37

       He's amazing too.

    3. LQLin Qiao1:18:37

       Um, and he's almost, like, cynical in a Eastern European way, but also at the same time, very humble.

22. 1:18:44 – 1:28:32

    Quick-Fire Round

    1. HSHarry Stebbings1:18:44

       Can I do a quick-fire round with you?

    2. LQLin Qiao1:18:45

       Okay, let's do it.

    3. HSHarry Stebbings1:18:46

       Okay. What have you changed your mind on most in the last 12 months?

    4. LQLin Qiao1:18:51

       I think how fast we grow, I changed my mind because I have been quite worried about too big a team too early. Uh, so that's why when I met George, I told him, "We're too small for you because I don't intend to grow very fast in terms of people." [laughs] Uh, I worry about- Slow down, getting slowed down, and lose agility and vel- velocity very deeply. So, um, but since then is we have been very aggressively using AI tools. We have developed our own unique way of hiring certain type of people that we know, uh, they will be charging forward with high velocity, has extreme sense of ownership, uh, very communicative, and never take no as answer. So we also learn how to, how to get those people. Uh, and now I feel much more comfortable scaling really fast.

    5. HSHarry Stebbings1:19:51

       What's your type of people? I know that sounds weird, but, like, our type of people is, is actually really specific. Pretty much only hire immigrants. British people don't work very hard. Uh, sorry. Uh, very scientific and rigorous, use data for most things. I actually think creativity often comes from data and is informed by data. Um, and unwaveringly, like, accountable and ownership. Like, nothing is anyone else's fault, it's all my fault, even if it's someone else's fault. That's a 20VC person. What would you say yours is?

    6. LQLin Qiao1:20:22

       Is not, in weird way, it's not competence. It's weird. We need- we want people with a high conf- uh, competence. Um, but more importantly, the strong indicator whether they will do well, um, in this wave, especially in Fireworks, is whether, um, they are really built for taking extreme ownership. Um, extreme ownership as in we are not putting people, anybody in any boxes, and we're just stacking the boxes together into a tower. Uh, we- people just automatically claim, "Hey, this is an end-to-end problem, I'm gonna see through the whole thing and work with a bunch of people to make it happen, and, uh, and I'm gonna deliver it no matter what." So those kind of people has the highest, longest mileage, and their growth curve is amazing also.

    7. HSHarry Stebbings1:21:18

       What's your biggest lesson from working with Jensen Huang on what makes him so special?

    8. LQLin Qiao1:21:22

       He's everywhere. I serious think he has a clone of, like, hundreds of Jensen [laughs] somehow plugged. Um, for example, I send him an email, he will reply in one minute. I, I just don't understand how he's, like, constantly, um, in details. And, but, but now I operate a company for four years, I understand why he's doing that, is that's, that defines velocity. Because what is leadership? Leadership is just judgment. It's not privilege, it's judgment. It's y- you basically have the context, and you have the right context to make the right judgment for the team. If-- and especially in a high-velocity space, if you do not know what's happening, what works, what doesn't work, what are the gaps, you make the wrong call. Um, in a slow-moving, uh, space, you, you, you can wait for the cascading information up and down and make those calls. But in a fast iteration space, you just cannot wait, um, because it's guaranteed there is information loss in tran- in, in transition. Layer after layers, people after people, it always happens. And not knowing what exactly is happening and make, having the precision of make judgment makes bad leadership. And he is demonstrate through his own example, even before this crazy AI thing, is he's operating that way. And before, I was admiring him in, in his sheer amount of volume of capability of doing that. Now I understand the wisdom behind that because I oper- also operate that way. Uh, I, I need to know what's happening on the ground to make the judgment for the company.

    9. HSHarry Stebbings1:23:30

       What did you wait on in the Fireworks journey that you wish you hadn't waited on?

    10. LQLin Qiao1:23:36

        Marketing. [laughs]

    11. HSHarry Stebbings1:23:38

        [laughs]

    12. LQLin Qiao1:23:38

        We talk about it. So we are a little bit nerdy-

    13. HSHarry Stebbings1:23:43

        [laughs]

    14. LQLin Qiao1:23:43

        ... in this way, that at the very beginning of our journey, we kind of, we didn't, we didn't discuss it, but we feel product will speak for itself. At the end, product stands, and we want to devote all our effort and focus on building product, working with customer, um, validate product market fit, and, and go from there. And we didn't spend much time marketing at all. We didn't prioritize educating our customer what's the right direction to think about the trend, um, and, uh, and the value. Um, but we do think, now I do think it's important. Marketing is not about flaws. It's more about education. Um, it's, uh, more about clarity. Um, and, uh, and we are working on that.

    15. HSHarry Stebbings1:24:39

        What area of AI is under-invested in today in your mind? Uh, you mentioned, like, cooling or servers. What area is, like, under-invested in?

    16. LQLin Qiao1:24:49

        I think AI has-- The sexy part of this is such a innovative, creative technology, and, uh, build something on top of it is the focus. But monitoring the ROI, I think the industry start to kind of pay attention to it. But eventually, that's what matters. Um, is not how much spend is, how much, what is return, and, and, uh, and what is the cost and what is the attribution. So I, I think in the next couple of years as AI is getting more and more into production, there will be a lot of focus in getting that clarity and getting that discipline out. So, uh, the token maxing is just, um, I think, I think in time, um, but we're quickly moving to RI maxing, which is about [chuckles] all about running a business.

    17. HSHarry Stebbings1:25:47

        What large customer do you not have that you would most like to have?

    18. LQLin Qiao1:25:52

        Hmm. So we haven't spent too much time in traditional enterprise segment. Um, I think that's just because we, we was very small. Um, and now as we build out our company, I do think, uh, even without our, us investing, we have customers like Geico, like Capital One, um, like, uh, Mercury Insurance and, uh, uh, RBI, all these companies. Even without us pursuing enterprise, traditional enterprise, they, they come to us and, uh, they are customer, but I do think that's a very big market.

    19. HSHarry Stebbings1:26:33

        What has to happen before the end of the year that hasn't happened for you to consider it a good year?

    20. LQLin Qiao1:26:39

        I'm confident in our capability of driving the business. Um, and to me, this is a year I want to prove we can scale quickly by keeping the same velocity, and that's very important to me. Uh, if we reach that point, reach our proof point, and next year I have a lot more confidence, just continue scale extremely aggressively. I want to make sure we do it right this year.

    21. HSHarry Stebbings1:27:04

        Final one for you. What does no one see about the next three years that you see very clearly happening or not happening?

    22. LQLin Qiao1:27:12

        I really see people will own their, every single company will own their own intelligence as a must-have. It's not optional. That's a trend I'm seeing. Um, because there's an analogy to software is there's a reason why every company build their own software stack. There's no standardized software you just use off the shelf to solve a problem because every single company is solving a unique problem, and they want to build software because they want to have full control. Um, and obviously they will pick and choose which part of the stack they want to build themselves, which part of the stack is common knowledge, there's no point of building. But every single company own their own software stack. Obviously, we're talking about, um, this in the SaaS time, right? So, uh, same. I think AI time, every single company should own their own intelligence.

    23. HSHarry Stebbings1:28:07

        Lin, you know, it was Matt that introduced us first. Uh, I've had the joy of getting to know you and obviously George. Uh, I can't thank you enough for joining me, for coming in person. It is so wonderful to do it in person, and you've been fantastic.

    24. LQLin Qiao1:28:20

        That's an amazing studio. Uh, you, uh, you did, um, you asked a lot of interesting questions. I have a lot of fun talking with you.

    25. HSHarry Stebbings1:28:27

        We, we do a lot of research before, huh? [laughs]

    26. LQLin Qiao1:28:29

        Yes, you did.

    27. HSHarry Stebbings1:28:30

        Uh, thank you so much for that, Lin. You're fantastic.

Episode duration: 1:28:42