Y CombinatorBetter AI Models, Better Startups
EVERY SPOKEN WORD
40 min read · 8,017 words- 0:00 – 1:22
Coming Up
- JFJared Friedman
Every time there's an OpenAI product release now, it feels like there's a bunch of startups waiting with bated breath to see whether OpenAI is going to kill their startup.
- GTGarry Tan
This is actually a really, uh, crazy moment for all startups. Adding more types of modalities and more capabilities, uh, per model, the, the better off every startup is.
- HTHarj Taggar
You have to be on top of these announcements and be... kind of know what you're going to build in anticipation of them before someone else does, versus being worried about OpenAI or Google being the ones to build them.
- GTGarry Tan
Welcome back to another episode of The Light Cone. I'm Gary. This is Jared, Harj, and Diana, and we're some of the group partners at YC who have funded companies that have gone on to be worth hundreds of billions of dollars in aggregate. And today, we are at an interesting moment, uh, in the innovation of large language models in that we've seen a lot of really new tech come out just in the last few weeks, whether it's GPT-4o, it's, uh, Gemini 1.5. Harj, how are you thinking about, you know, what does it mean for these models to be so much better?
- HTHarj Taggar
Anytime I see a new announcement from one of the big AI companies with
- 1:22 – 5:00
AI Challenges for Startups
- HTHarj Taggar
the release of a new model, the first thing I think about is, what does this mean for the startups, and in particular YC startups? And when I was watching the OpenAI demos, it was pretty clear to to me that they are really targeting consumer. Like, all of the demos were cool consumer use cases and applications, which makes sense. That's kind of what ChatGPT was, was a consumer app that went really viral. I just wonder what it means for the consumer companies that we're funding and, in particular, like, how will they compete with OpenAI for these users? What did you think? Like, what, wh- even if we take it back, like, how do consumer products win from, like, first principles? Like, is it more about the product or the distribution, and how do you compete with OpenAI on either of those things?
- GTGarry Tan
Yeah, that's a great question. I mean, I think ultimately it's both. And then, uh, how I want it to be is that the best product wins. Uh, how it actually is, is whoever has the best distribution and a sufficiently good product seems to win. Either way, I actually think we're at, uh, sort of, uh, in this moment where the better the model becomes, if you're already using four and suddenly four, you know, you can, uh, change one line of code and suddenly be using 4o, uh, you basically just get smarter by default every generation. And that's really, really powerful. It means that, you, I think we're entering this moment where the i- the IQ of these things is still, you know, four is arguably around 85. It's not that high. (laughs) And then if the next generation, if Cloud Three really is at 100 or, you know, the next few models end up being closer to, you know, 110, 120, 130, this is actually a really, uh, crazy moment for all startups. And, uh, the most interesting thing is, like, uh, adding new capabilities, so having the same model be great at coding, for instance. Uh, that means that, you know, you might have a breakthrough in reasoning, not through just the model reasoning itself, but you could have the model actually write code and have the code (laughs) do better. And even right now, it seems like there's, um, a lot of evidence that if, instead of trying to prompt the model to do the work itself, you have it write code and you execute the code, it can actually do things that reasoning alone could not do. So adding more types of modalities and more capabilities, uh, per model, the, the better off every startup is.
- DHDiana Hu
I mean, the cool thing about, uh, 4o is that you can get better structure output. In this particular case, they are better getting JSON, which is getting signs of getting large language models, not just outputting English, but more language for computers so that you can build even better applications on top, which is signaling that this better model can be better for startups and make it easier to integrate because one of the challenges for startups has been always coercing LMs to output the right thing. So you actually process it in regular business logic.
- HTHarj Taggar
The other thing I kind of thought about when I was looking at the demos is, as it relates to startups, if only one of these companies has the most powerful model by some distance, then that is indeed bad for startups 'cause you have to depend on them being friendly and having, like, a nice API for you to build on top of. If there are multiple equivalently powerful models, you're much safer off as a startup. It was funny, maybe coincidental, maybe not, that, like, OpenAI's announcement was, like, what, two days before?
- GTGarry Tan
(laughs) One day.
- HTHarj Taggar
One day before Google's, right? Um,
- 5:00 – 8:32
GPT-4 vs. Gemini 1.5
- HTHarj Taggar
what's the difference between the, uh, sort of under the hood, the way that GPT-4o works and then Gemini 1.5 works? And do you have any opinions on their relative strengths?
- DHDiana Hu
Yeah. So the thing about 4o, why it was so interesting, it was adding the speech modality and also video processing on top of that text. And the way they do that is still primarily a text-based transformer model underneath, basically GPT-4, and what they've done is bootstrap and added modules so that it has different code paths to handle this different type of data. OpenAI famously also implemented and launched Whisper, which is one of the state of the art for automatic speech recognition, and probably that's what they're doing. They took the architecture of, uh, Whisper and then bolted it into GPT-4, and they also bolted DALL·E, and they combined these, and that became 4o. So this is why, in terms of the reasoning capabilities, 4o isn't...... better, per se, than 4.0 by any margin. So it's how it works, is kind of adding modules, how they describe it on the white paper. The difference versus, uh, Gemini 1.5, which actually, on the technical aspects and merits, I'm actually more excited by-
- GTGarry Tan
Yeah, interesting.
- DHDiana Hu
... the Gemini one.
- GTGarry Tan
Huh.
- DHDiana Hu
I know it's counterintuitive because 4.0 and OpenAI has captured the zeitgeist of everyone, and they're so good at the demos, right? Singing Happy Birthday a bit off-key, that's like-
- GTGarry Tan
(laughs)
- DHDiana Hu
... so human.
- SPSpeaker
Happy birthday to you. Happy birthday to you. Happy birthday dear Jordan. Happy birthday to Jordan. Ba-da-ba-ba-da-ba.
- GTGarry Tan
(laughs)
- DHDiana Hu
Google I/O kind of missed the mark in terms of demo, but in terms of reading their white paper, what's interesting about Gemini 1.5 is that it's actually a true Mixture of Experts, and that is a technique that's new, where they actually train, from the ground up, a giant model with the actual data of text, image, audio, and the whole network activates a specific path for these different data types. So instead of, um, the OpenAI model that has like kind of modules, this one truly is a one-all model.
- GTGarry Tan
Very interesting.
- DHDiana Hu
And what it does is different parts of the network activate depending on this data input, so it becomes very energy-efficient. And I think the reason why, uh, Google was able to do it is because they have the engineering hammer. They have TPUs where they can really afford to put a lot of data, because it's very expensive to put not just all text, image, and video, and train this giant thing in a distributed cluster. They have TPUs, like their, I think it's their fifth generation now, and it's pretty cool what they've done.
- HTHarj Taggar
Is that the first big model release that's using Mixture Experts?
- DHDiana Hu
I think they talked a bit about it in the previous one, but everyone was a bit, uh, disillusioned after the demo of the duck was not real. (laughs)
- GTGarry Tan
Oh.
- SPSpeaker
It is a duck.
- HTHarj Taggar
Yes.
- DHDiana Hu
But this one was described better. I mean, the interesting thing is that I think this time, they learned their lesson, and I think it's actually working.
- HTHarj Taggar
Yeah.
- DHDiana Hu
And the other cool thing about Gemini is, uh, it has a context window of a million tokens, which is huge. The GPT-4o is 128,000, so imagine what you can do with that, because that's about like five books of 500 words or more. And the cool thing about the Gemini 1.5 was their white paper has this saying that, on research, they proved it to work on a 10 million token window, which brings a question for
- 8:32 – 15:19
RAG future in consumer apps
- DHDiana Hu
all of you. What does that mean for startups, especially a lot of the startups that we're funding with infrastructure that do a lot of RAG? There could be the controversial argument that, uh, all these startups building tooling around RAG, which is a whole industry right now, maybe it become obsolete. What do y'all think about that?
- HTHarj Taggar
I feel like the people who care a lot about data privacy and where the data is stored are still gonna want some sort of RAG system, right? Like, they want the data stored somewhere they control it, versus all in the context window. It's not clear that that's gonna be the biggest part of the market. Like, in general, people who care this much about any behind-the-scenes architectural thing tend to be like early adopters, but not like mass market consumer. So my guess is people just want like a massive context window, because then you can start building the kinds of consumer apps people are excited about, right? Like the assistant that just has all this context on me, that knows everything about me. Like, currently, I think the best way you can do that is you, like, run Ollama or one of these open source models, and then you, like, throw a bunch of your, like, personal emails at it. That's like a project that the hobbyists on Reddit are doing a lot of, is just try and get, like, your personal AI that's got all the information on you. But if you had like a infinite context window, you wouldn't need to do all of that.
- GTGarry Tan
I think you'd still need RAG to be able to sort of, uh, store everything, and that's like sort of the long-term permanent memory. And then what you actually want is a separate workflow to pull out the interesting things about, uh, that user and their intentions, and then you actually have a little, like, summary bullet point of things that you know about the user. You can actually kind of see some version of this even now in ChatGPT. If you go into the settings, under 4.0, it actually now has a memory. (laughs)
- DHDiana Hu
Mm-hmm.
- GTGarry Tan
And so you can actually see-
- HTHarj Taggar
(laughs)
- GTGarry Tan
... a concrete version of this inside ChatGPT. I was just using it to sort of generate some like Where's Waldo, uh, images for my son, and, uh, it wasn't quite doing what I wanted. It kept using, like, making, like, really deformed faces. So I kept, like, prompting it back-to-back. I was like, "No, no, no, I really want no deformed faces." And then for a while, it was like, uh, I said I wanted a red robot in the corner. And it kept making, uh, all of the characters, like, various forms of red. And I said, "No, no, no, I really don't want you to do it." And I, you know, sort of repeated it four or five times. And then, uh, I went and looked at my settings, and it was like, "Gary really doesn't want deformed faces in his, uh, generated images. We should also try not to use red." (laughs) And it was interesting to see that, like literally from even, like, maybe 10 or 15 different chat interactions, um, you know, I was getting frustrated, but it was definitely sort of developing some sort of memory based on, uh, my experience with it. And the most interesting thing was that, uh, you could see what the machine had like sort of pulled out from your interactions thus far, and you could like sort of delete it as necessary.
- DHDiana Hu
Maybe a infinite window doesn't necessarily mean that the retrieval is actually accurate.
- GTGarry Tan
Yeah.
- DHDiana Hu
And this is more, I mean, more anecdotal in practice from what founders have told us, versus what the actual research paper benchmark gets, which is a very kind of lab setting. So in practice, I do tend to agree that a RAG pipeline infrastructure's still very much needed, exactly for what you said. Privacy and people wanting to fine-tune models on their own data, and not getting that leaked out over the wire o- over the internet. And the other thing is, um, yeah, maybe that's still more accurate to do it on your own. When you really want that very precise information, I think you still need RAG. And I think the analogy I like to think about this is sort of like, um...... processors back in the day in the '90s as, uh, when Moore's Law was actually Moore's Law. It was the scaling. (laughs) It was not just, uh, CPU processing speed getting faster, but also memory cache levels were also getting bigger and bigger. But now more than 30 years later, we still have a very complex architecture with how we do different kinds of caching for retrieving data out of, like, databases. Out of databases you have maybe, like, a fast memory store with, like, Redis for high availability, and then you still have things stored in your browser cache. There's still very much lots of layers of how things will be cached. And I think RAG is gonna be this foundational thing that will stay, and it'll be like how we work with databases normally now, just like lots of levels.
- GTGarry Tan
Yeah. Yeah, the tricky thing about the context window... I mean, uh, Gemini may have... The team may have already fixed this by now, but certainly a lot of the founders I talked to, they said, uh, it's sort of, you know, the million-token context window sort of lacks, uh, specificity. Literally, uh, if you ask for retrieval from its own context window from... you know, or the prompt, it actually sometimes just, like, can't seem to recall it or can't seem to, you know, pick out the specific thing that you already fed into it.
- DHDiana Hu
Mm-hmm.
- GTGarry Tan
And, uh, the tricky thing there is, like, you'd rather have a 128K context window that you knew was pretty rock solid rather than a system where, you know, it's still a bit of a black box, you don't really know what's going on. And then for all you know, it's just, like, sort of randomly picking up like half a million of the tokens. And that, you know... Again, like, probably fixable. You know, I can't imagine that that's, like, a permanent situation for, you know, uh, a million or 10 million, uh, token context window, but something that we're seeing from the field for now.
- HTHarj Taggar
Also, in enterprises, like in business use cases, people care a lot about, like, what specific data is being retrieved, who's doing it, like, logging all of this stuff and commissioning around data. And so, yeah, you can imagine having some kind of... Yeah, a giant context window is not necessarily what you want in an enterprise use case. You actually probably want, in particular, sensitive data stored somewhere else and retrieve, like, when it's needed and know who's making the requests and filter it appropriately.
- DHDiana Hu
Exactly. I think that will, that will stay.
- HTHarj Taggar
I was really encouraged what you said actually about how the Google technology is maybe better than the OpenAI stuff. It feels very Googly, actually. (laughs)
- GTGarry Tan
(laughs)
- HTHarj Taggar
It's like, hey, they've got the best technology, but they just, like, don't know how to get, like, the polish around it correct. That means OpenAI does not have this, like, leap forward unassailable tech advantage. If Google has something comparable, then we should expect to see, like, Anthropic come in. We should expect to see, like, Meta come in. And what we're seeing at the batch level is just the models are pretty abstracted out, right? On a day-to-day basis. Like, our founders are already using different models to prototype versus, like, build and scale. Like, the ecosystem of model routers and observability ops software around this stuff just keeps progressing really quickly. So, it's just funny. My, my initial reaction whenever I hear, like, the model releases is not to worry for the startups actually so much-
- DHDiana Hu
(laughs)
- HTHarj Taggar
... because they're alread-... Like, we never talk about how reliant they are on any one model.
- GTGarry Tan
I worry if there's one model
- 15:19 – 19:15
Diverse AI Options
- GTGarry Tan
that's very, very good, and it'll be dominant and sort of take over the world. Uh, I'm less and less worried if there are many different, uh, alternatives, because then you have a market, and a marketplace equals-
- HTHarj Taggar
Yeah.
- GTGarry Tan
... uh, you know, non-monopoly pricing, which means that, uh, you know, a thousand flowers can actually bloom. Like, other startups can actually make choices and, uh, have gross margin of their own. And I'd much rather see, you know, thousands of companies make a billion dollars a year each rather than, you know, one or two, let alone seven companies worth a trillion dollars.
- DHDiana Hu
Mm-hmm. And I think we have a dark horse that is yet TBD. We don't know when LLaMA 3 with 400 billion parameters comes out, because that's still being trained, and that's like one that's like, "Wow, it could really turn tables as well."
- GTGarry Tan
Yeah. Uh, the interesting thing about Meta is... I mean, they have pro- probably one of the largest clusters. Uh, certainly, I think I was reading, um, you know, in terms of who has paid NVIDIA more money in the past year, uh, Meta apparently is number one by, by, um, a g- a decent bit, actually.
- DHDiana Hu
And the funny thing is, they have this giant cluster not because they necessarily have foreseen this whole shift that happened recently in the last couple years with large language models. They acquired lots of GPUs because they needed to train their, uh, recommendation models, right? Wh- that use actually similar architecture with deep neu- deep neural nets to actually compete with TikTok, because to build these, like, really good recommendations on Instagram Reels. (laughs)
- HTHarj Taggar
That's just a very classic tech innovation and disruption, right? Like, they're basically worried about competing with TikTok.
- DHDiana Hu
(laughs)
- HTHarj Taggar
They stockpile a bunch of GPUs, and it turns out the GPU is just really valuable for this, like, completely different use case that's gonna change the world. (laughs) Jared, like, on that note, if you zoom out, just like, how does this cycle of, "Hey, like, we're worried. Startups are worried about the elephant in the room," this case it's OpenAI, maybe Google competing and crushing them, how does it play out to when we first moved out here even? Like, in that, like, era where Facebook was rising, Google was starting to go from the search engine company to, like, the multi-product company. Do you see any similarities or differences?
- JFJared Friedman
Yeah, it reminds me of that a lot. Like, um, every time there's an OpenAI product release now, it feels like there's a bunch of startups waiting with bated breath to see whether OpenAI is gonna kill their startup, and then there's all this internet commentary afterwards about, like, which startups got killed by the latest OpenAI release. And it reminds me a lot of when we got YC, the, the three of us, and the, like, "Uh..."... 2005 to 2010 era. There were all these companies who were innovating in the same idea space as Google and Facebook, building, like, related products and services where the big question was always, like, "What happens if Google does this?" And when startups were pitching to investors, that was, like, the main, like... a big question that they'd always get from investors is like, "Oh," like, "but isn't Google going to do this?"
- HTHarj Taggar
The best response to that, by the way, was like, "Well, what if Google gets into VC?" Which it did.
- JFJared Friedman
(laughs)
- HTHarj Taggar
(laughs) And has a great VC arm.
- JFJared Friedman
So, a lot of the people who are building AI apps now, this is the first hype cycle they've ever been in, but we've all been through multiple hype cycles. And so I think it's interesting actually, for the people who are in the middle of this hype cycle now where all of this is new, to look back on the past hype cycles and see how the history of what happened there can inform their decisions about what to work on.
- HTHarj Taggar
If we take Google as an example, one thing that's interesting is, if you look back, there was, there was competing with Google in a very head-on way, which was, "Hey, we're going to build a better search engine," um, and YC definitely funded a lot of companies trying that. And I feel like the approach people would go after was the vertical engine. It was, "Oh, we're going to build a better Google for real estate," for example. Um-
- GTGarry Tan
Some of those made it.
- HTHarj Taggar
Did they? I'm not sure which ones.
- GTGarry Tan
Well, you could, I mean,
- 19:15 – 22:14
Specialized Services
- GTGarry Tan
argue that, um, something like a Redfin or Zillow clearly did have vertical access to data, and then-
- JFJared Friedman
Or a Kayak for travel, I guess.
- HTHarj Taggar
That's-
- GTGarry Tan
Yeah.
- DHDiana Hu
Or Algolia for, uh, company enterprise search.
- JFJared Friedman
For, for enterprise search, yeah.
- HTHarj Taggar
That's true, okay. There's... Yeah, I hadn't thought of... Yeah, I hadn't thought of Zillow as a search engine, but yeah, it's essentially that. It's exactly that.
- GTGarry Tan
It's vertical search.
- HTHarj Taggar
Yeah.
- GTGarry Tan
But you have to monetize not necessarily through the same way a search engine would.
- HTHarj Taggar
Yeah.
- GTGarry Tan
You have to have other services. You have to become, uh, a broker. You have to, you know, basically make money in all these other ways once you have a customer.
- HTHarj Taggar
Your whole UI is completely different.
- JFJared Friedman
Completely different. It doesn't look at all like Google, yeah.
- HTHarj Taggar
Yes.
- DHDiana Hu
And the data integrations is very different. Like, you have to really poke and connect to MLS, and a regular search engine would you-... wouldn't just work with that. Like PageRank wouldn't necessarily work with MLS.
- GTGarry Tan
Yeah. Redfin's very interesting because I'm very addicted to Redfin, and it has actual-
- DHDiana Hu
(laughs)
- GTGarry Tan
... absolutely caused me to buy property that I normally wouldn't buy. So (laughs) , uh, you know, in that respect, like, those are interesting consumer scenarios. Ultimately, a great consumer is actually about buying just, like, a little bit of someone's brain, such that during the course of one's day... I mean, it doesn't have to be every day, but ideally it is. You sort of think to use it.
- HTHarj Taggar
And no one of those companies would have said that they had better technology or they beat Google on technology, right? Like, anyone who went up head agai-... head-on against Google for, like, the better general purpose search engine just got crushed.
- JFJared Friedman
And in general, most of the vertical search engines didn't work. And certainly nothing that looks anything like Google worked. The, the ones that I remember the most were more ones that were in the vein of Google Apps. Like, when Google expanded beyond search and started launching Google Docs and Sheets and Slides and Maps and Photos and all, all these, all these, like, like, separate apps, there were a lot of companies that we funded-
- HTHarj Taggar
Yeah.
- JFJared Friedman
... (clears throat) that were either going to be crushed or not by the next Google product.
- HTHarj Taggar
Yeah, that's like the standard case of when you can bundle software in. I mean this...
- JFJared Friedman
Yeah.
- HTHarj Taggar
Like, this is what Microsoft did to Netscape, right? Like, once you can start bundling in software, especially in the enterprise, it's like people don't necessarily want to buy, like, 10 different solutions from 10 different vendors all the time. If you can offer a good enough product across several different use cases and bundle them together, enterprises often want that.
- DHDiana Hu
I mean, famously, uh, Dropbox was in that rogue, potential rogue tale, right?
- JFJared Friedman
Definitely.
- DHDiana Hu
Because... And Drew... Because Drew actually talks about it when he comes back and give the dinner talks, about the fear when, with Google Drive and Google had this other product carousel thing, right?
- JFJared Friedman
Yeah. In fact, there was a time when, um, Dropbox had launched. This was after The Batch, and Google was working on Google Drive but hadn't launched yet. It was called GDrive. It was like se-... the secret project inside of Google, and news of it leaked to the press. And the whole world just decided that, like, Dropbox's goose was cooked. Like, it was over. Google was going to launch GDrive, and because it was Google, they had infinite money. They were going to do the same move that they're doing now, which is throw, like, infinite money at the product and give away, like, infinite storage for free. How could a startup possibly compete with Google, you know, spending
- 22:14 – 25:15
GPT-4o and Desktop App
- JFJared Friedman
billions of dollars to give away infinite storage for free?
- DHDiana Hu
That was infinite tokens. (laughs)
- JFJared Friedman
Yeah.
- HTHarj Taggar
(laughs) Yeah.
- JFJared Friedman
And, and now it's infinite tokens.
- HTHarj Taggar
Yeah, that's interesting.
- GTGarry Tan
What are the big companies trying to do right now that maybe you should avoid doing? And, uh, the super obvious one is, well, uh, OpenAI seem to have released 4o, which is multimodal, and then it also simultaneously released the first version of the desktop app. But that version of the desktop app is merely a, sort of a skin on the web experience. But if you put two and two together, surely it's going to look a lot more like her. I mean, they've been really shading that-
- DHDiana Hu
The demo has a Scarlett Johansson voice.
- GTGarry Tan
... you know, pretty heavily, right?
- JFJared Friedman
(laughs) They just pulled that, right?
- HTHarj Taggar
(laughs)
- GTGarry Tan
Yeah. They're like, "Oh, shoot." You know, who knows? Are they getting sued? Who knows? That's, that's what Twitter says today, anyway. But I think if you look at the details of that, you know, you can sort of sketch out what's going to happen with, um, LLMs on the desktop. And the desktop is sort of... has access to all your files, has access to not just that, but all of your applications. Uh, it has access to your IDE locally. It has access to your browser. Uh, it can do transactions for you. That's starting to look like basically the true personal assistant, um, that is directly consumer. And then that sounds like a whole category. Like, you know, we're going to interface with computers and using potentially voice, and certainly, like, ex-... we will have the expectation of a lot of smarts. And, uh, that, you know... That seems like we-... that's where they're going, and that's going to be one of the fights.
- JFJared Friedman
When I was thinking back to, like, this first era of companies, I guess one thought I had is that it was fairly predictable, actually, what Google would build. Not 100% predictable. Like, Dropbox was like... It was, like, unclear if Google would win that space, but, like, a lot of them were actually pretty obvious-
- HTHarj Taggar
Yeah.
- JFJared Friedman
... in hindsight.Um, like ad tech for example, like all of ad tech just, like, never stuck around because it was like too strategic to Google and Facebook. And so they just had to own all of it and like al- almost all of vertical search just didn't really survive. It's pretty easy to imagine what the next version of OpenAI, th- like, product release is gonna be. And if you can easily imagine that what you're building is gonna be in the next OpenAI release, you know, maybe it will be. (laughs)
- HTHarj Taggar
Using that framework, it's like OpenAI really wants to capture just, like, the imagination, like the sci-fi imagination of everyone. So it's like, yeah, it's like the general purpose AI system that you just talk to and it figures out what you want and does everything. It seems hard to compete with them on that.
- JFJared Friedman
That's like competing with Google on search.
- HTHarj Taggar
Yeah. Right.
- JFJared Friedman
And that's clearly going to be like the- the core.
- DHDiana Hu
Because that was the early signs of why, what ChatGPT is being used for as well, just like a very, very rudimentary, right?
- JFJared Friedman
Yeah. Which is the same thing with Google. They always wanted to own products where billions of people would all use the same product.
- HTHarj Taggar
Yeah.
- JFJared Friedman
Anything that was like that was gonna be really tough as a startup.
- HTHarj Taggar
Yeah. When I think of it for products I use, like Perplexity, not a YC company,
- 25:15 – 30:59
Valuable Products
- HTHarj Taggar
but Perplexity is a product I use a lot because it's much better for sort of research. If I need to fix a toaster, it's way easier for me to type in, like, the model of the toaster into Perplexity and get back, like, specific links and YouTube videos and just the whole workflow. It was Diana who told me about it, actually.
- DHDiana Hu
Yeah.
- JFJared Friedman
(laughs) Like-
- DHDiana Hu
I've been using it a lot as a replacement for actually my regular search.
- HTHarj Taggar
Yeah. That's what I never... I was trying to use Perplexity for a while and I couldn't get it and I was... Because I was trying to use it in the same way I would use, like, the OpenAI, the ChatGPT app.
- DHDiana Hu
Oh. Yeah.
- HTHarj Taggar
And I was like, oh, but like ChatGPT is just so much better because I just like type in fuzzy things and it figures it out and it comes back with smart things. And Perplexity just wasn't as good for that use case but the specific, "Hey, I have this task that I want, like, source material back and links for," it works much, much, much better. It doesn't capture the imagination, right? Like OpenAI is not gonna, like, release some model that they demo that, "Oh, look, like if you search, it like gives you the links back or it like shows you the YouTube videos that it's referring to."
- DHDiana Hu
The demo is not as cool. A- Actually, Gemini 1.5 has that feature and nobody really talks about the demos from... (laughs)
- HTHarj Taggar
Yeah.
- DHDiana Hu
From Google IO. They're kind of like, "Eh."
- HTHarj Taggar
So maybe one way to figure out how not to be roadkill is to, like, if you can build the valuable but unsexy things that OpenAI aren't going to demo on stage because it doesn't, like, capture the sci-fi imagination, you might survive.
- GTGarry Tan
Yeah, that's definitely a whole line of thinking. Like Google was never going to do Instacart or DoorDash's business, so, or Ubers. So all of that was fair game and all of those turned out to be, you know, decacorn or, you know, potentially... You know, even Airbnb, like $100 billion company.
- HTHarj Taggar
Because the other thing people always underestimate is just I think the size of new markets. I remember for a long time people didn't believe LinkedIn could be a big company because it's like, well, like why? Because like Facebook won social networking.
- DHDiana Hu
Mm-hmm. Mm-hmm.
- HTHarj Taggar
LinkedIn is just a social network. It's just going to be a... Like, you have your work tab on your Facebook profile, like why would you need something else? Same thing with Twitter. I remember, um, when I first moved to San Francisco in 2007 some of the first people I met were the early Facebook employees and they were like, they saw Twitter growing and they're like, "Oh yeah, we're gonna like release status updates or something," and it's just like Twitter's gonna be done as just a feature. But yeah, it turned out like Twitter was like a whole other thing. Instacart and DoorDash I think are another great example of this because again I remember iPhone comes out, Android becomes pervasive, it's like, oh there's, it's just going to be like Apple and Google dominate mobile. But there were all these things that they would never build. Same in this AI world probably, right? There's all these things that the big companies are never gonna build and we probably have more appetite for using multiple AI agent type apps than just like the one OpenAI one.
- JFJared Friedman
And a huge, like, meta category that is basically almost anything that's B2B. Like Google basically never built anything B2B, they like basically only built mass consumer software. And so if you look at the YC unicorns, like a ton of them built, you know, some like B2B thing like Segment or something that like Google was never gonna build Segment. That's just like not interesting to them.
- HTHarj Taggar
Ironically because I think in B2B people really underestimate the human part of it. Like so much of it is actually the sales machine and it's being willing to go out and figure out who you sell to, do the sales, like listen to someone, like give you all the things they're unhappy about and note them down and take them back to your engineering team and say, "Oh yeah, we need to like tweak this, this and this and this and all these details," right? Like-
- JFJared Friedman
And build lots of like really detailed software to like handle all these obscure edge cases.
- HTHarj Taggar
Like I think of one of our AI companies at YC that's doing really well is called PermitFlow and they literally just expedite the process for applying for construction permits and not just for individuals-
- JFJared Friedman
Amazing.
- HTHarj Taggar
... but for like big construction companies now as well. And it's like, yeah, like-
- JFJared Friedman
Really hard to imagine that being the next OpenAI release, right?
- HTHarj Taggar
Yeah.
- JFJared Friedman
Like, "Hey guys, we built a feature for- for filing your construction permits."
- HTHarj Taggar
Yeah.
- JFJared Friedman
Right?
- HTHarj Taggar
Can you... Yeah, can you imagine turning up for your first day of work as an OpenAI engineer and they're like, "Okay, you're going to work on the construction permit workflow feature"?
- JFJared Friedman
(laughs)
- HTHarj Taggar
(laughs) Like, I don't think it works that way.
- GTGarry Tan
Well, I guess if you join those two ideas together something interesting happens though. It seems sort of inevitable sometime in the next two to five years, you know, assuming the OpenAI Her digital assistant comes out and then it's going to be on your desktop. It will actually know everything about you. Uh, it'll know what you're doing and know... It'll know minute to minute what task you're trying to complete and then it's conceivable, you know, if you match that with sort of a launch that I think they probably didn't invest enough into which was like the GPT store-
- 30:59 – 33:58
Better Business Models
- HTHarj Taggar
software business models are so much about, "How do I upsell? Like how do I make more money per customer next year than I did this year?" And it just, hey, like every time the model gets better, you can just pass that along as like an upsell premium feature and upgrade to the software, and your end user doesn't care, right? Like they just care about what the function that the software can do for them. And so I think there's a world where the models keep getting better, you've got your choice of which one to use, and the additional functionality, you just charge more to your customers for, and you make more money.
- GTGarry Tan
Yeah, that's definitely what we're seeing at YC. I mean, last batch of people were making $6 million a year right at the beginning of the batch, and it ended up being north of 30 million by the end of the batch. So, that's some really outrageous revenue growth in a very, very short amount of time, three or four months. And that's sort of on the back of what, uh, you know, a few people working on B2B software, you know, they can focus on a particular one that makes a lot of money. And then people are willing to fork out a lot of cash if they see ROI pretty much immediately.
- JFJared Friedman
There's not as many founders working in this area as there should be given the size of the opportunity. Like, like to your, to your point, Harsh, like people often underestimate how big these markets are. Like using LLM to automate various jobs is probably as large an opportunity as SaaS, like all of SaaS combined, right? 'Cause like SaaS is basically the tools for the workers to do the jobs. The AI like equivalent of SaaS is like, it, it just does the jobs. (laughs) Um, so like-
- DHDiana Hu
It's the tools plus the people.
- JFJared Friedman
(laughs) Yeah. So like, it should be just as large, and yeah, there should be like a lot more people working on this.
- GTGarry Tan
So there might be, you know, billions to trillions of dollars per year going into, uh, transactional labor revenue that's on someone's, uh, you know, sort of, you know, cashflow statement right now.
- JFJared Friedman
Yeah.
- GTGarry Tan
But it'll turn into software revenue at 10X, (laughs) which will be interesting for market caps over the next 10, 20 years.
- HTHarj Taggar
I was doing office hours with a startup this morning that asked me this question about, "Hey, like you probably saw the GPT-4O launch. Like should we be worried about it?" Um, yeah, my reply was, "You should be worried about it, but you should be worried about the other startups that are like competing with you." Because ultimately it's, uh, all of the stuff we're talking about, it's whoever builds the best product on top of these models with all the right nuances and details is going to win, and that's going to be one of the other startups in the space. So I just think the meta thing as a startup now is you have to be on top of these announcements and be, kind of know what you're gonna build in anticipation of them before someone else does versus being worried about OpenAI or Google being the ones to build them.
- DHDiana Hu
Let's talk a little bit about, uh, consumer because we did talk about what could be potentially roadkill for consumer startups if you're going against basically assistance, some sort of assistant type of thing, opening eyes, hinting, well, not, uh, strongly, (laughs) direct in they're going in that direction. What about opportunities for consumer AI companies? What are
- 33:58 – 37:26
Consumer AI Opportunities
- DHDiana Hu
those, those things that they could flourish?
- JFJared Friedman
Well, here's an edgy one. Anything that involves legal or PR risk is challenging for incumbents to take on.
- GTGarry Tan
Microsoft giving money to OpenAI in the first place you could argue was really about that.
- JFJared Friedman
Yes.
- GTGarry Tan
I mean, when image models and image diffusion models first came out at Google, they were not allowed to, uh, generate the human form for, uh, PR and legal risk, risk reasons.
- JFJared Friedman
This is a large part of what created the opportunity for OpenAI in the first place is Google was too scared to jeopardize their golden goose by releasing this technology to the public. The same thing could probably be true now for startups. Things that are increasingly edgy-
- GTGarry Tan
Yeah.
- JFJared Friedman
... are often the places where there's great startup opportunity.
- GTGarry Tan
Mm, I mean, things like, uh, Replika.AI, which was a AI NLP company working in this space for many years even before LLMs were a thing, still one of the top companies doing the AI boyfriend or girlfriend. And the wild thing about Replika is that they've been in touch with, uh, their sort of AI boyfriend or girlfriend for many years. And earlier we were talking about, you know, a million token context window, you can imagine that virtual entity knowing everything about you like for many, many years, like even your, you know, deepest, darkest secrets and desires. I mean, that's pretty wild stuff. But, um, you know, it's gonna look weird like that. (laughs) And, um, you know, people might not be paying attention. I mean, Character.AI has really, really deep retention and people are sort of spending hours per day sort of using things like that. So, you know, whatever happens in consumer, it might be non-obvious and, and it might be very weird like that.
- DHDiana Hu
So there's a lot of kind of more edgy stuff around, uh, deepfakes that are applied in different, different spaces. So there's a company that you work with, Jared, Infinity AI, right?
- JFJared Friedman
Yeah, Infinity AI lets you turn any script into a movie, and that movie can involve famous characters. And so it like enables you to make famous people say whatever's in your mind, which is edgy, which is part of what makes it like interesting and cool.
- DHDiana Hu
Google would never launch that.
- GTGarry Tan
No. (laughs)
- JFJared Friedman
Google would never launch that. And I think even, you know, the, the same move that OpenAI did to Google, which is being willing to release something that's really edgy, well, OpenAI is now the incumbent, guys. They now can't release super edgy stuff like that anymore.
- HTHarj Taggar
We're gonna see a lot of that during election season in particular, right? Because it's interesting when you think about it, like anything that's on the, "Hey, like I am, this is explicitly like a famous person. This is explicitly using the likeness of a famous person for profit," is, is going to get shut down. On the other end, you have like, I don't know, if I make a meme with Will Smith and some like, a caption, like no one's gonna sue me for that. And then a lot of this content is like right in the middle.
- JFJared Friedman
Mm-hmm. Yeah.
- HTHarj Taggar
Right? It's like I'm not trying to build like a video that's literally pe- I want people to believe that it's like these people saying these things, but what if it's like-
- JFJared Friedman
A joke about it.
- HTHarj Taggar
... a joke or a satire.
- JFJared Friedman
(laughs)
- HTHarj Taggar
Like where does that fit? And yeah, you can't see, you can't imagine Facebook or is gonna roll this out on Instagram anytime soon, right? Like they wanna, they wanna stay well clear of that, but yeah.
- DHDiana Hu
You're already seeing this version of memes sort of 2.0 that are basically deepfakes that-... are making the rounds and they're becoming viral tweets, right?
- HTHarj Taggar
Yeah. Hey, why don't we close out by going to a question that one of our audience asked us on Twitter? Um, so thank you, Sandeep, uh, for this question. The question is just, "What specific update from OpenAI, Google, Meta excited each of you and why?"
- 37:26 – 40:47
Emotional Depth and Translation
- JFJared Friedman
I'll give one. Um, the thing that really excited me about the OpenAI release was the emotion in the generated voice, and I didn't realize how much I was missing this from the existing text-to-speech models until I heard the OpenAI voice.
- SPSpeaker
Oh, a bedtime story about robots and love? I got you covered. Once upon a time, in a world not too different from ours, there was a robot named Byte.
- JFJared Friedman
It's amazingly better compared to the incumbent text-to-speech models just because it actually knows what it's saying. The existing ones, by contrast, sound so robotic. They, like, they're totally understandable, but they're just very boring to listen to. And the OpenAI one, it felt like you were talking to a human.
- HTHarj Taggar
My one was the translator, um, demo. The idea of basically having a live translator in your pocket. It, it's personal for me 'cause I, my wife is Brazilian, and her parents don't speak English, and so I've been learning Portuguese but it's coming along very slowly. (laughs) Like, the idea of having just, like, a translator that's always in my pocket that makes it easy for me to communicate with anyone anywhere in the world is really exciting.
- GTGarry Tan
Hey, how's it been going? Have you been up to anything interesting recently?
- SPSpeaker
Um, (Spanish) .
- JFJared Friedman
It's a massive idea. I mean, it could change the world. You could go live in a foreign country where you don't speak the language. It, it, like, it has huge consequences.
- GTGarry Tan
Yeah. Douglas Adams', uh, Hitchhiker's Guide to the Galaxy-
- HTHarj Taggar
Yeah.
- GTGarry Tan
... uh, made real is a pretty cool one. I guess for me, um, what's funny about 4.0 is it sounds like maybe it was actually just a reorg. Basically, there was a reorg at OpenAI-
- DHDiana Hu
(laughs)
- GTGarry Tan
... and they realized, uh, they want all of the teams rowing in the same direction. And then, uh, what that means is, uh, probably really good things for both their assistant desktop product, uh, but also eventually robotics, which, um, might be a really big deal down the road. There's this Chinese company called Unitree announced a $16,000 humanoid biped robot, though Twitter warns me that it's another $50,000 if you actually want open API access. (laughs)
- DHDiana Hu
(laughs)
- GTGarry Tan
Previously they made $114,000 version of that robot, um, but I think unified models means more and more likelihood that practical robotics is, you know, actually not that far away. Famous last words, of course. We've been saying that, uh, pretty consistently for many years in a row, but this time it's different.
- DHDiana Hu
I think for me, maybe a bit more of a technical one. I know it doesn't sound too, too fancy, but really the half the cost is like a huge thing. And if you extrapolate that, what that means is probably a lot of these models are hitting some kinda asymptotic growth of how much better they can get, which means also that they're becoming more stable and it can open up the space for actual custom silicon to process all of these and enable a lot more low power processing to enable robotics and build a device that you mentioned and actually have it in your pocket-
- HTHarj Taggar
Yeah.
- DHDiana Hu
... and not be tethered to the internet. So all these things that we could perhaps see, uh, excitement of new tech product releases, because I kinda miss those day when every product tech demo was, like, very exciting. Now it's just, like, kinda like a feature.
- HTHarj Taggar
True.
- DHDiana Hu
We could be excited about new things coming up.
- 40:47 – 41:06
Outro
- GTGarry Tan
Well, we're gonna be really excited to see what you guys all come up with. That's it for this week. We'll see you next time. (instrumental music)
Episode duration: 41:06
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode 4aMQPG9gPoM
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome