Parallel’s Parag Agrawal: Building a New Web for AI Agents

1. 0:00 – 3:25

   Introduction

   1. PAParag Agrawal0:00

      Our view at Parallel is that human click data is a bug. An agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information, and we can benefit from a lot of the research that have gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments, and that's the arguments we made back then, that actually now it's way more tractable as a problem because of the existence of agents, not just as in technology, but as a distinct customer.

   2. SHSonya Huang0:40

      [upbeat music] Parag, thank you so much for joining us today. Uh, we're delighted to have you on the show. For those who don't know, Parag of, uh, Twitter CEO fame was the CEO of, of Twitter, um, before selling it to Elon, uh, and is now back on the founder arc. Um, you founded a company called Parallel Web Systems, which is scaling up agentic search, uh, for the agentic web. Um, we're very excited to have you here today to talk about the future of search and the future of the internet. So let's get started. What's Parallel?

   3. PAParag Agrawal1:25

      Thank you, Sonya, for having me. Thanks, Andrew, for joining us. I'm really excited about this conversation. At Parallel, we're building, uh, a bunch of technology in order to allow agents to search and use the web. So just like humans forever have figured out how to use browsers and search engines and clicking and browsing on the web to access information, uh, agents need to do the same things. We started Parallel with the bet that agents would do it 1,000X more than humans ever have. And as a result, we need to reinvent the technology that can power search for agents, but also the business models that go alongside it, and that's what we're doing at Parallel.

   4. SHSonya Huang2:07

      Hmm. Okay, I wanna go back into what's... what makes human and agent search so different. But f-before we get there, you've told us that, you know, you're, you're unlearning a lot of the lessons that you learned from running Twitter w- as you build, build Parallel. Why is that?

   5. PAParag Agrawal2:21

      Listen, when I was at Twitter in leadership roles, Twitter was a post product market fit, extraordinarily scaled business where your feedback loops were from the hundreds of millions of customers using the product for 30-plus minutes every day, right? In that world, you operate differently than a pre-product market fit company based on the premise that in a few years, a new customer is going to show up on the internet [chuckles] and let's go build technology for the not yet here customer that we are all learning every day and every week. [chuckles]

   6. SHSonya Huang3:07

      Fantastic. Okay, let's talk about, let's talk about Parallel. Um-

   7. ARAndrew Reed3:11

      I have a question, actually.

   8. SHSonya Huang3:12

      Yeah.

   9. ARAndrew Reed3:12

      Can, can I jump in?

   10. PAParag Agrawal3:12

       Yeah.

   11. SHSonya Huang3:13

       No. No, you cannot, Andrew.

   12. ARAndrew Reed3:14

       I'm Andrew. I didn't get an introduction.

   13. SHSonya Huang3:15

       [laughs]

   14. ARAndrew Reed3:16

       But I'm also happy to be a inaugural guest on the Training Data podcast.

   15. SHSonya Huang3:19

       Andrew is making his podcast debut-

   16. ARAndrew Reed3:21

       Podcast

   17. SHSonya Huang3:21

       ... on Training Data. We are, we are honored to have you finally here.

   18. ARAndrew Reed3:24

       Thank you.

2. 3:25 – 5:17

   What Is Web Search

   1. ARAndrew Reed3:25

      Um, I feel like all of us in Silicon Valley have a very surface level idea of search. Like, what is search? Like, the job to be done is, you know, getting the answer makes sense, but we all know there's crawling and there's an index and there's ranking. But maybe just take a big step back and explain what is the problem of web search, be it for a human or an agent, and then we can dive into the differences.

   2. PAParag Agrawal3:49

      Um, the problem of web search, and we all know it and experience it, when we want to find something and we do not know where it is on the web, we go to a search engine like Google, and the search engine then hopefully surfaces the answer to us in the most convenient of locations. That's the base problem. Now, in order to do this, what is the search engine doing? The search engine is going and crawling the web, which is finding every URL that's out there, trying to read it, trying to organize that information in what might be called an index, so that by having all of this stuff in memory in one location, you don't have to... Once the query comes in, you don't have to spend time loading up pages because you already have them. You've already done pre-processing to organize them. And then when the query does come in, you quickly understand the query. You find the most relevant results. And then there are many, many... Because you're essentially taking hundreds of billions of pages and narrowing it down to, what, five or 10 or ideally one [chuckles] in terms of what someone is looking for right now. So you go through these multiple stages of retrieval and ranking in order to surface the most relevant result. So that's the broad problem. One way to think about it is it's a billion-to-billion matching problem, right? So you have hundreds of billions of pages and hundreds of billions of queries over time, and you need to figure out how to match make across

3. 5:17 – 7:52

   Why Start a New Index

   1. PAParag Agrawal5:17

      these two.

   2. SHSonya Huang5:17

      That sounds like an enormously expensive infrastructure challenge, and I think for the longest time, basically only Google and Bing had, you know, done full, full web scale crawls and indices. Why did you think about, you know, you as a young company could go off and tackle that problem? It seems like a problem of the giants.

   3. PAParag Agrawal5:37

      So it is expensive in the long term. So what's actually interesting is that when I was starting the company three years ago, you could imagine a world where, one, some of the reasons it was difficult for others to compete, which was not having access to great feedback in terms of, is this a better search result than this? To collect that feedback at scale, there was a problem around human ratings There's a problem around click data. You need to access those. Now it is, of course, expensive to crawl the entire web and index it. Uh, but as you think about what agents and the large models that we now have access to have enabled is the ratings data can be now created by experts way more cheaply. Our view at Parallel is that human click data is a bug, and agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information, and we can benefit from a lot of the research that have gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments, and that's the arguments we made back then, that actually now it's way more tractable as a problem because of the existence of agents, not just as in technology, but as a distinct customer. And then we figured out a way to go about building this business, which did not require us to go spend all of the money on infrastructure upfront before we could service a customer, right? So if you can paint a path of incrementally being able to build an increasingly larger and more sophisticated index over time as you solve problems for more and more customers, that's the insight that actually convinced me that this was a viable problem for us to take on.

   4. SHSonya Huang7:44

      And how do you make that happen? 'Cause I imagine this, this is one of those things where customers want full coverage day one. So how do you go about making that happen?

4. 7:52 – 10:17

   Search Agents First

   1. PAParag Agrawal7:52

      When, when we first launched the product, right, we launched in... We did not launch a search product first. We launched a search agent product first. Our search agent could go essentially crawl the web after a query arrived. So if you're doing deep research, you have patience to the extent of a minute, and w- we've had products which sometimes take... do ten minutes of research. That's a lot of time [chuckles] to be able to crawl a lot of pages if only you have enough of a map to know what to prioritize crawling, right? So you can make up for shortcomings. Like, a index is oftentimes you can think of it as a latency optimization. So if you give up on that dimension, if you're competing with humans... That's why our search agents were competing with the alternative being outsourcing to humans to curate amazing data, right? So we said it seems like humans sitting on search engines are far... way easier to compete with [chuckles] than a search engine on day zero. So by building a product that was a search agent to do real work on top of web data, we were able to incrementally go build our index.

   2. SHSonya Huang9:03

      Hmm. What are some examples of what people built with your search agents?

   3. PAParag Agrawal9:07

      In the very early days, uh, there were search agents being built with us for doing some kinds of insurance underwriting workflows and claims processing workflows. People in, uh, sales were doing all kinds of sales data enrichment. People in finance, uh, who would previously and historically go to build a model, collect data by sending it overnight to a set of humans who would curate data that would feed into a modeling exercise they would do, would start using us to do that instead. And so we were going initially after replacing where there was outsourced human work on top of web data in order essentially to collect evals to run agents to figure out what search for agents should look like in the first place with empirical use cases instead of theoretical evals.

   4. SHSonya Huang10:11

      I see. You're trading off the crawl for inference time compute effectively.

   5. PAParag Agrawal10:16

      Yeah.

5. 10:17 – 13:14

   Not a Neolab

   1. ARAndrew Reed10:17

      You guys were started before the term, like, neo lab came into exis- existence and you have, you know, a web systems product, but you also have, like, a bunch of AI people. From the outside, before we met for the first time, like, it wasn't obvious to me how much of a web systems company or an AI company that you've built. The more we spent together... the more time we spent together, the more clear it's, like, by some definitions, this is a neo lab. Do you wanna talk about, like, where the research side of what you're doing comes in, not just the, like, infrastructure and system side of what you're doing?

   2. PAParag Agrawal10:49

      So we don't categorize ourselves as a neo lab.

   3. ARAndrew Reed10:53

      Well, of, of course you would not.

   4. PAParag Agrawal10:54

      We... No, because I think... I don't think our output is a model. Like, I somehow... maybe my definition is broken. My definition of a neo lab is an output is a model. I think our output is a complement to a model.

   5. ARAndrew Reed11:09

      Mm.

   6. PAParag Agrawal11:09

      What we build is something that multiplies on top of a model in order to give either you can call it the model m- gets better or the agent built with the model gets better and has superpowers, right? So we've always wanna be in a place where whenever someone sh- ships a better model... A neo lab, for example, if somebody else ships a better model, they have a higher hill to climb. For us, somebody ships a better model, they have now unlocked four more use cases where we can be valuable. Now, whether we need to do work that might be framed as research, that remains. So in that sense, we do have to do research. But we're also not, uh, pre-training extraordinarily large models. In fact, our job is to compress things down into tiny ranking models, right? Like, I think if you go back to my framing around, like, this hundred billion to hundred billion matching problem, every query is essentially give me 1,000 tokens from A trillion web pages on the web and make sure they're the right thousand tokens. Like, that's the query that we are getting effectively from... to our search engine. And so now what you have to do, and until last week we would give ourselves three seconds to throw a bunch of compute at read time to do it. This week, last week we shipped a product which now does it in 200 milliseconds, right? So you have now that much time to go figure out how to pick those, right? So you will do... We're trying to organize information in memory across the memory hierarchy in a way that we can access it fast. We are trying to figure out how to train what model, uh, to select the best 1,000 tokens. And so there's a large amount of research that goes into figuring out how to take a pre-trained model of some kind, adapt its architecture for this new problem, and allocate effectively compute in a way that produces the best output with a limited compute/latency budget.

6. 13:14 – 19:38

   Agents vs Google Search

   1. SHSonya Huang13:15

      So a lot of these use cases seem like the deep research kind of shaped use case. And when I think deep research, at least in the earlier incarnations, it was effectively like an agentic loop with the model reasoning and then basically just calling a search engine, calling, calling Google or some proxy to Google.

   2. PAParag Agrawal13:30

      Yep.

   3. SHSonya Huang13:31

      Right? Why is that insufficient in your eyes versus what I'm hearing from you is agentic search is kind of a, a net new capability?

   4. PAParag Agrawal13:39

      So calling Google for every query-

   5. SHSonya Huang13:45

      Yeah

   6. PAParag Agrawal13:45

      ... in a deep research thing, it gets you somewhere, right?

   7. SHSonya Huang13:49

      Okay.

   8. PAParag Agrawal13:49

      If you use parallel search, you will, for the most part, use under half the tokens in your agent. It will become more accurate and be faster end to end. And so you can think of it as, but every time... If you use only half the tokens, if your model is context limited or memory limited, you can now do more problems, you can do the same problems cheaper or faster. So everything to me, when you have a infinite appetite for information and relevant information for all kinds of work, it's at its core an optimization problem around quality, cost, and latency. Every model advancement is about how do you squeeze out more intelligence, and then how do you distill it down into keep most of it at a 10th of the cost, right? And that applies to search as well. And every time you can produce all of the signal with less noise in your search results to give to a model, you now let... give the model the ability to do more. And so a lot of the bet here is intelligent compute allocation across the model layer and the agent layer down to the search layer. The other interesting thing to observe is the interface actually changes, uh, when you think about serving agents versus humans. Humans rely on keyword search. With agents, we've had to innovate quite a lot on the what does an agent tell our search engine? The more we know, the better we can do.

   9. SHSonya Huang15:33

      Interesting. So like humans can only hold a few words in, in memory basically, versus an agent.

   10. PAParag Agrawal15:38

       No, I think we can, we're just lazy.

   11. SHSonya Huang15:40

       Yeah.

   12. PAParag Agrawal15:41

       Right? [laughs] Like we, we can, we can have a conversation, we can hold a lot of words, but when you start typing, we, we want to tell Google like two incomplete words with a typo in there and hope for the best, or rely on some form of a drop-down autocomplete thing to avoid typing those three words- [laughs]

   13. SHSonya Huang16:01

       Yeah

   14. PAParag Agrawal16:01

       ... in the first place. So we're fundamentally lazy. Turns out like models less so. Fewer typos, better specified queries, perhaps longer queries, less for the search engine to guess what the agent might want. Uh, so you get to solve a different class of problems.

   15. ARAndrew Reed16:20

       One other thing about humans is I feel like we know how to decipher the, you know, pre-AI slop that populates many of the top-ranking Google results, you know, for best product for X, Y, and Z, and you get all the affiliate advertising things. Um, and yet obviously they, they rank highly.

   16. SHSonya Huang16:41

       I'm very susceptible to it.

   17. ARAndrew Reed16:42

       Are you?

   18. SHSonya Huang16:43

       Yeah.

   19. ARAndrew Reed16:43

       Yeah, I'm like, I feel like I have this in- incredible f- I can just see it coming a mile away. How do agents deal with things like that, right? Where these are, you know, well-trafficked pages, they have seemingly good answers, but you just know they're not real.

   20. PAParag Agrawal16:56

       Perhaps with Parallel agents have to deal a little bit less with that.

   21. ARAndrew Reed16:59

       Mm.

   22. PAParag Agrawal17:00

       Let me tell you why those pages exist in the first place, right? Let's work through a simple example. If you ask for a public company's most late-- most recent financials, like just the headline revenue number. We can sit here and know that there exists an authoritative filing with the SEC which has that number, perhaps on page 73 of a PDF, but that is the authoritative number, right? Now imagine Google decided like, "I care about authority," and whenever you ask that query, that's the first result you see, right? And then you click that. This PDF takes, what, like three and a half seconds to load instead of the second that the another page does. You're already frustrated. And then you see a earnings page-

   23. ARAndrew Reed17:50

       Seeking out a-

   24. PAParag Agrawal17:52

       That's the alternative. But here you have to now grab your way to page 77 to find your answer. We're lazy. We're not gonna do the work, right? So now there exists an entire class of content on the web which is like, okay, this information is needed by a lot of people. It is worth- Putting it on a page that loads fast, where this information is above the fold. All of you can go there. It's 99.99% right, so you're not like so skeptical when you go there that this is 100% wrong, and it's added real value in the process because it identified out of the 300-page earnings report, 20 bits of information that should be above the fold.

   25. SHSonya Huang18:41

       Yeah.

   26. PAParag Agrawal18:41

       Right? And so you can call it slop, pre-AI human slop, [chuckles] or you can call it catering to a lazy human and being successful at SEO, right? The good news is with agents, you're not making the agent click around and fumble around and crap a PDF, right? We're taking a excerpt from the most authoritative place on the web and trying to bring it to the agent's context window. And so we aren't forced into this weird trade-off, and this trade-off existed with humans in the first place because like with, with standard browsers and protocols and everything, we just didn't figure out how to have us point exactly consistently across everywhere on the web to the exact right highlighted tiny paragraph, right? With agents, we get to bring that to the agent's context, and then let it figure out

7. 19:38 – 28:59

   Inside the Search Stack

   1. PAParag Agrawal19:38

      what's next.

   2. SHSonya Huang19:38

      Can you walk us through what actually happens when, when one of your developers sends a query off to your agentic search API?

   3. PAParag Agrawal19:45

      It's in some sense pretty standard. So we run some models to figure out what this query is and enrich it to figure out how it will flow into the rest of our system. We have a bunch of indexes which organize different subsets of the inf- of the web in different ways. And so the first layer will essentially c-craft queries for each of these different systems. Each one of these systems perhaps is our sort of big index. One of these is perhaps our fresh index. One of these is perhaps like you can... Some people will describe it like a knowledge graph. Some people will describe it like a structured index. Like, there are a bunch of these, right? So you're now deciding which ones this query needs to go to. You're figuring out what is the query rewrite for each of these. Then each of these has like a big retrieval layer, and then a ranking layer, and then more ranking layers. So you're gonna try to boil down tens, hundreds of billions of URLs or documents, uh, down into thousands, tens of thousands, down into specific excerpts and paragraphs in those tens of thousands with more and more bigger models running at each stage with different architectures, pulling more features to ultimately get down to here are the 1,000 tokens I want to bring back to this AI which has the highest signal.

   4. SHSonya Huang21:09

      Hmm.

   5. PAParag Agrawal21:09

      Right? And now if you look at our various versions of our search API, they just throw for different latency and cost constraints, different amounts of compute at various points in this journey to hit those limits. So in the abstract, it's simple, right? What's interesting is what each of these models that I described, how you curate and collect the training data for them, right? How you build those models and optimize them. The index itself, how you use the, the memory hierarchy to store it to be able to hit certain cost, quality, latency thresholds.

   6. SHSonya Huang21:54

      What's the North Star from a quality perspective? Like, you know in Google there's that, you know, did you get the result you wanted in the first three answers or something. What is the equivalent North Star for you?

   7. PAParag Agrawal22:03

      So I think of... I don't know if I'm right on this, but like my take is that a billion-to-billion matching problem is a forever problem. And so the real question is, at what point incremental optimization isn't worth the squeeze, right? [chuckles] So I don't think there is a thing as like, okay, we're done on improving this thing. The question is at some point it's going to get harder and harder to improve this thing, and it just won't be worth it, right? But I'm hoping that we don't get there actually because like if you think of what we're doing with AI, we will have more intelligence that gets cheaper every few months. And as a result of it, it'll come down to having great models which are cheap, having great information, matchmaking across a need and all of the information available to you, whether it's your own or on the web, and doing something unique and differentiated with it to produce more knowledge, right? And anytime you can do something 20% better than somebody else, that might give you an edge. So why wouldn't you? Right? [chuckles] So in the super AGI-built worldview, it feels like if you can push on quality across web search and in the model layer, why wouldn't you?

   8. SHSonya Huang23:34

      I've heard a point of view that this is so fundamental to the model companies that they're just gonna own it, and, and in part because as they're collecting data for pre-training, that's a very, very expensive infrastructure exercise. That kind of is your source of truth for the, for the crawl. Um, what do you think of that?

   9. PAParag Agrawal23:52

      I don't see empiric data on the ground to support that view. To build a fresh web index, I don't think that crawl is particularly useful. And let me frame why. So if you think of what we are building, we're building a complement to models. We like to crawl things that- People don't like to crawl for pre-training. Because like if the model already was trained on it, like it's not useful. Model companies for the training aren't patient enough to go in a completionist way, try to wait for really slow random JavaScript to load, because the number of tokens you get per amount of compute you throw at it is like one order of magnitude to two order of magnitude too bad. And so like is it worth the extra effort to get these tokens? For us it is because we're completionist, right? For them, it's like, "I'll take X trillion tokens." So that's one. Um, now I do think this is a core part of every agent, right? I... My worldview is that if you're buying LLMs for doing work, for nine out of 10 use cases, you will want them to have access to the web and create search infra optimizing for agents. So it is a real adjacency for all kinds of LLM inference, and that supports your view that model companies could, should have the best-in-class infra for it. So now they can build it or they can buy it, and that's the conversation and the... W- we will see who builds and who buys and who partners and, uh, how things evolve.

   10. SHSonya Huang25:49

       Are you partnering with any of the model companies that you can share?

   11. PAParag Agrawal25:51

       We want to. We will. I can't share anything on that. We did announce, and I don't know how your definitions of model companies. Uh, we announced today actually that we are working with, uh, Google Cloud to be a search and grounding provider for their enterprise agent APIs. So if you think of grounding Gemini models or other models available on GCP, when you build agents on GCP or chat apps on GCP or do any other inference with LLMs on GCP, uh, when you attach web search to it, uh, your options are Google Search or Parallel Search. And Parallel Search is product integrated. The integration is optimized. We've spent time with technical teams and training teams and product teams and commercial teams to make sure that when people use Gemini models with Parallel, like they get exceptional and great results. So yes, there are these partnerships now emerging. Uh, I bet that there'll be several of these. They will all look somewhat unique.

   12. SHSonya Huang27:08

       That's a big deal from the search king. Congratulations.

   13. ARAndrew Reed27:10

       Google's the original Neo lab.

   14. PAParag Agrawal27:12

       I, I don't know how to frame Google as a model lab versus a hyperscaler, and I don't know what precise lessons to learn from this one, whether it applies to other labs or to hyperscalers, and so we will see.

   15. ARAndrew Reed27:28

       The way that I use voice agents and, you know to get reservation for dinner at night, what restaurants should I go to? And then that request gets fulfilled by the agent. When we talk to a lot of the Parallel customers, there's this, you know, background agent, whether the monitor product, these agents that sort of are always watching the world or watching the web, and when something happens, then they go off and take actions and do something with it. Um, it might be worth because if we think about what does 1,000X more mean, there's like the depth of research and then there's just like the, the, you know, what is actually initiating the tasks. Is it a human initiating the search or is it the agent itself? You wanna talk about that dynamic a little bit?

   16. PAParag Agrawal28:07

       Yeah. So there's a bunch of dimensions here. Let's go back to for a moment on search agents, right? If you run a typical search agent and even without doing deep research, it'll, it'll do somewhere between five to 20 searches, even if it answers within a few seconds. Because why not, right? So already if you transition from using a ChatGPT medium or, and all the categories, but like somewhere not on the high tier, like instant, low, medium. Every time you write a prompt to it in ChatGPT, it will do five to 10 searches. As you dial it up, it'll do hundreds and thousands of searches. So one interesting thing to observe is like a human action to a multiplier on number of searches that happened,

8. 28:59 – 30:21

   Search Multipliers With Agents

   1. PAParag Agrawal28:59

      right? So just by using a AI app, you're kind of multiplying your way to perhaps one order of magnitude more searches. Now, a lot of our initial takes on the product and the market were to go after bigger multipliers than even that, right? So we were much more interested when you said, "I have a portfolio of 10,000 small businesses where I have given out credit to. For all of them, every month, I have this human process that runs to feel out how my risk is going up or down." Now can we... And it relies on a bunch of web data. We're trying to use agents for doing this. So here a developer is effectively the multiplier there is hundreds of thousands or a million in terms of the number of web searches that happen because a human goes and programs that instead of now doing this process every month, we can do it every week, right? So it's doing a lot of searches when this agent runs every week to create a dashboard on the portfolio and a collection of action items that somebody needs to look at, right? Um- Now you can go to another example which is even more interesting.

9. 30:21 – 31:46

   Meeting Prep Agent Workflows

   1. PAParag Agrawal30:21

      Uh, I don't know, do you use something to do, uh, meeting prep documents for you all that's an agent?

   2. SHSonya Huang30:28

      There's a Sequoia one. There's a Sequoia agent.

   3. ARAndrew Reed30:30

      Yeah, yeah. I was, I was gonna say James Flynn, but [laughs] He's one of our great young guys.

   4. PAParag Agrawal30:36

      So I use, um, Notion's, uh, agent, and you can build custom agents which look into all the internal data that y- we have at Parallel, plus all of the web data using Parallel's APIs to create meeting prep docs. One time I went and created one prompt to build this custom agent. Now it does tens and hundreds of web searches for every meeting I have. Every time I build a new agent for a new use case, that keeps multiplying. So I think the path to these background agents doing more and more and more work all the time for us, it's only going to be bounded by value versus spend.

   5. ARAndrew Reed31:22

      Mm-hmm.

   6. PAParag Agrawal31:22

      You know? Like, I don't think it's rational right now. Like, I'm probably spending more on it than I should be, but it's not too much, so I don't care yet. So there'll be some rationalization in all of these agents. Um, but I think we will deploy background agents to the extent that there is incremental value in doing that compute. And the same thing applies to searches. All of these background agents will do a bunch of searches.

10. 31:46 – 32:42

    Quality Cost Latency And Turbo

    1. ARAndrew Reed31:46

       Mm-hmm.

    2. PAParag Agrawal31:46

       And so our first set of products were really obsessively focused on these, in part because we were building, growing the index, and we decided that our company was based on three dimensions: quality, cost, latency. And for the first couple of years we said, "Let's focus, let's ignore latency, and let's just nail the other two." Because optimizing systems, distilling to smaller models is much more known art than unknown research-

    3. ARAndrew Reed32:20

       Mm-hmm

    4. PAParag Agrawal32:20

       ... right? So n- once we achieved the best quality search and search agent products at every price point, we started working on latency, and that's what we shipped with a product we call Turbo now. It is the fastest, highest quality agentic web search on the market by a lot.

11. 32:42 – 34:28

    Are Agents Overtaking Humans

    1. SHSonya Huang32:42

       Do you think there are more agentic queries than human queries on the web now?

    2. PAParag Agrawal32:47

       I don't think yet. I don't think we've hit that yet.

    3. SHSonya Huang32:49

       Just given some of the multipliers you mentioned, and given the background agents-

    4. PAParag Agrawal32:52

       Yeah

    5. SHSonya Huang32:52

       ... it seems...

    6. PAParag Agrawal32:52

       Yeah. I, I think I don't have to remind you, but we are early. Like, we are very early in agent adoption. Like, you go step outside of our bubble, people haven't heard the word Fable. Like [laughs] so we are very, very early. I think there are now people like me who are probably operating at the 1,000X. I don't know. What do you think? How many Google searches a day did you do three years ago before ChatGPT?

    7. ARAndrew Reed33:23

       20, 30?

    8. PAParag Agrawal33:24

       Yeah. I would've guessed something like 20, 30. I think today if you just look across at all of my agents, like I, I bet they're doing 1,000X more than that.

    9. ARAndrew Reed33:36

       Mm.

    10. PAParag Agrawal33:38

        Maybe 100 to 1,000. W- um, if you count some of the things that happen at my company which isn't, like, assigned to a human-

    11. ARAndrew Reed33:48

        Yeah

    12. PAParag Agrawal33:48

        ... it might easily be more than 1,000X. But I think we are the outliers rather than the norm. So I think we're very, very early on this journey. I, I do think I recently saw, I think it was CloudFlare that said that in their monitoring of web traffic-

    13. SHSonya Huang34:07

        Yeah, it was

    14. PAParag Agrawal34:08

        ... the AI traffic is about the same as human traffic in terms of page reads, which is slightly different from searches because it includes perhaps all crawlers that are out there, and a bunch of other stuff. But I think it's, it's going to happen.

12. 34:28 – 37:20

    Ads Model Meets Agent Web

    1. SHSonya Huang34:28

       So maybe this is a good segue to talk about a topic that I know you are passionate about, which is the economics of the internet as we know them. Some of the kind of fundamental assumptions there, you know, human eyeballs, scarcity of attention seem to be falling right in front of us right now. Are the economics of the internet broken now, and what's gonna happen?

    2. PAParag Agrawal34:48

       Yeah, no, I think this was perhaps part of the thing that totally convinced me that this needed to be done by me and build Parallel. Uh, I know most people hate ads. I used to do ads and build systems for ads.

    3. SHSonya Huang35:02

       I love ads.

    4. PAParag Agrawal35:03

       Oh, wow. How?

    5. SHSonya Huang35:04

       I love shopping.

    6. PAParag Agrawal35:05

       Okay. Yeah. So you get good ads. I don't know if I love ads, uh, but I intellectually love ads because ads is the reason that so much amazing content and technology is available for free to all of us, right? Like Google Search wouldn't be free without ads. Twitter wouldn't be free without ads. And these are truly useful pieces of technology. A lot of content on the out- on the web out there that we can access for free wouldn't be free if not for ads. Um, and so ads is a very efficient monetization scheme. It's a very efficient monetization scheme because it is exceptional at differential pricing, right? So most queries Google loses money on. Some of them make it up and it's an extraordinary business with extraordinary margins. Same with Twitter. Like most users-

    7. SHSonya Huang36:02

       Just from your shopping. [laughs]

    8. PAParag Agrawal36:03

       Exactly. You-

    9. SHSonya Huang36:04

       You're welcome. I'm subsidizing the free-

    10. PAParag Agrawal36:05

        You-

    11. SHSonya Huang36:06

        ... the free information you guys are getting

    12. PAParag Agrawal36:07

        ... you are subsidizing all of us. Thank you. Um, but I think ads is extraordinarily efficient-

    13. SHSonya Huang36:14

        Yeah

    14. PAParag Agrawal36:14

        ... at differential pricing and monetization on the web, which is why it has been a dominant business model. Now, the core assumptions, as you noted, going into it around limited human attention To translate, uh, into outcomes. If humans don't show up and their agents show up on the web, like, what does this mean? How does the business work? Um, and yeah, I think this was the... If we don't figure out a new business model, we're seeing it already, right? Like, people are gonna say, "Okay, I don't want my content to be accessed by an agent because I have a business model for humans, so I actually wanna go pay someone to SEO out, optimize myself, so more humans show up. But then if their agent shows up, I'm gonna cut it off." Right? Which seems confusing and discordant. Like, okay, ultimately, this agent is acting on behalf of a human, but we haven't found business model alignment-

    15. SHSonya Huang37:15

        Yeah

    16. PAParag Agrawal37:15

        ... in a way which is leading to this behavior.

    17. SHSonya Huang37:18

        You can't monetize that visit.

    18. PAParag Agrawal37:18

        You can't monetize that visit.

    19. SHSonya Huang37:19

        Yeah.

13. 37:20 – 40:48

    New Incentives For Content

    1. PAParag Agrawal37:20

       So let's say you are in the business of subscriptions, right? So you get a bunch of... You get 1,000 humans, and you convert 20 of them into monthly subscriptions, right? You don't know how that loop works yet with agents. You can't distinguish. You don't have statistics. You don't know if these will lead to subscriptions, or they'll just keep stealing your content as nameless agents. And so there are real challenges around the old business models breaking and us not figuring out real scalable new business models, right? So if you own high quality content and you currently put it on the web, your one available business model pre-Parallel was to be in the head and be able to transact with a lab on some fixed fee contract. There's little, like, which includes some amount of training and liability and then, uh, inference time access, right? That was your one option. That option's not available to most content on the web. It's a very head phenomena. And even for the head, it is a broken business model because when AIs or inference grows, let's say 7X this year and another, another 7X the next year, on this 50X, their deal size is not growing 50X. Like, none of them after signing a two-year deal believes that their share isn't going to decline materially [chuckles] at renewal. And so these are like fixed price constructs in the world of AI inference, and so which doesn't drive for sustainability for, for all of these businesses. Now, our solution is trying to learn all the lessons from my work at Ads on having transacted, like I sold Twitter data to OpenAI, having transacted on that side, uh, to figure out what actually might work and be incentive aligned.

    2. SHSonya Huang39:25

       So what might work?

    3. PAParag Agrawal39:26

       Efficient differential pricing. Paying for differentially a lot for extraordinarily high value content. Paying a lot for high value work accessing the same content, so it's differential on both dimensions, quality and value of work being done with it. Uh, and a way of doing this scalably and not just with bespoke deals. So those are the properties needed for any reasonable solution. And the biggest property of it all is incentive alignment, right? Like, at what point do people wanna collaborate into this enterprise?

    4. SHSonya Huang40:07

       Why do the model companies need to pay anything at all?

    5. PAParag Agrawal40:10

       Um, you should ask them. Um, my understanding is, one, you want fresh data during inference time to be able to display it in products like ChatGPT or Claude. Two, you want training data. And three, you want some liability protection for training that you already did. [chuckles] And so the payments are some combination of these three things, and I can't be sure of, like, how they value each of these three.

14. 40:48 – 47:46

    Shapley Values Attribution

    1. SHSonya Huang40:48

       You said, you know, even a very simple query will go do 10 searches, and so how do you do attribution between all the different sources that, you know, boil down to one paragraph response?

    2. PAParag Agrawal40:58

       Yeah. At Parallel, we like to build models. Um [laughs]

    3. SHSonya Huang41:01

       I heard.

    4. PAParag Agrawal41:02

       So, uh, no, I think the, the... It's a, it's a... It goes back to my point around incentive alignment. So what are you... Let's, before we try to build a model, let's try to figure out, like, if you were going to go try to do this intellectually, theoretically, how would you go do it? So you have to ask the question of, okay, how much incremental value did somebody's content add, right? So you can, you can run all of these simulation exercises. You take one piece of content out of the corpus, and then you say, "Let's run the agent. Let's see if the quality of our outputs declined. How much? To claw back that quality, perhaps if I threw a little bit more compute or a better model in some way, could I claw back that qua- quality? Oh, I could. It cost me a cent. Huh. I could get the quality I lost by not having this source." My alternative was to throw more compute, a cent worth of compute, to get that same quality. It might be a different result, but as far as the end customer value is concerned, it's the same quality. And so you're like, "Okay, this source is worth close to a cent," feels like, right? That's intuitive. Now, the formalization of this kind of an intuition is the, the, the core framework we use. It's called Shapley values. Uh, it's a-

    5. SHSonya Huang42:23

       Yes. Let's go. What's a Shapley value?

    6. PAParag Agrawal42:26

       It's a-

    7. SHSonya Huang42:26

       Music to my ears

    8. PAParag Agrawal42:27

       ... it's a game theoretic... You're, you're a game theorist?

    9. SHSonya Huang42:30

       Did a bunch of game theory, yeah. [chuckles]

    10. PAParag Agrawal42:31

        Yes. Shapley value is this- Very theoretical mathematical concept. It is used in SHAP values in feature importances for those who are ML folks here. Let's say the three of us collaborate on something, and the whole is bigger than sum of parts, uh, in that moment. The theoretical question is, okay, how do I divide up this sort of bigger pie that we created by collaboration so all three of us have incentive to collaborate, right? And Shapley values is a mathematical way of effectively answering this question, right? Now, that sounds amazing, right? Like you could... If you're co-founders, you could figure out how to divide equity. Unfortunately, it's not that useful. It's not that useful because in order to compute Shapley value, you need to simulate all worlds where some subset of the two of us collaborated, but the third one doesn't, [laughs] and play out those realities to then impute back to today in terms of how we should divide the pie, which in practice you can't do most places, right? In ML models, when you do feature importances, you kind of can. You can hold a feature back, run your model and see how well it did. In web search, we can run simulations of if we did not have access to this URL or this domain or this collection of them, how would the agent perform? We can, if you're, if you're good at evals, if you're good at assessing quality, you can build that data by running s- various scenarios. Collect a bunch of this data, and then you can train models to-

    11. SHSonya Huang44:07

        Your favorite thing

    12. PAParag Agrawal44:08

        ... favorite thing. So the challenge with Shapley values is computing Shapley values in our context is way more expensive than the amount of dollars we spend on the agent, forget the amount of dollars we want to pay a publisher, right? So to compute a content owner, to compute that a content owner gets a dollar, if I decide to do the sh- full Shapley value computation, that might take several dollars, right? So it doesn't make any sense, except you can really well estimate them if you build the right kind of data and models around it. But we have confidence that our estimations are good and that this is sound theoretically that if there was perfect information symmetry, people would want to collaborate. So the same way in the old ads days, people did second prize auctions and believed, for better or worse, that people would reveal their true bids in, in, in an auction and end up paying less than that. Uh, once feedback loops get established in a market, like today, everything is like an auto bid, right? Like most people are measuring ROI when they're doing advertising and running on auto bid instead of making up bids. Uh, I think a solid foundation based on incentive alignment that Shapley math drives will ultimately maximize participation of content owners, uh, into this as well as content seekers via AI in an optimized system. And we have some positive evidence to support it. Like we've had some interesting partnerships that we've been able to do and announce. Uh, and yeah, like imagine sitting with content owners and explaining Shapley math. Takes a moment. Uh, but ultimately once, once you pull out the properties, right, that you participate in the value, if you have unique differentiated data, you get paid more. If a banker in an expensive job reads your data versus my retired dad reads your data, the banker ends up paying more for that read, right? Because it's a part of high value work. And the, the macro math also seems to work. Like if you're going to spend a lot of money on inference on LLMs for knowledge work, if we allocated like what, two to 10% of it to data on the web, that's way bigger than all web data business models today outside of like walled gardens like Facebook and LinkedIn. [laughs] Like, so the macro math supports it. It is a scalable approach. And as agents on the web grow in order of magnitude year on year, it's like by my calculations, like we're 12 to 24 months from this math being able to give meaningful dollars for a very wide range of content owners on the web.

    13. SHSonya Huang47:27

        That's very exciting, especially at a time when, you know, you see the traffic data, the Stack Overflow plummeting. You see a lot of the human internet as we know it going away because of incentives. That's very exciting to see how you're thinking about incentive alignment for people to keep publishing.

    14. PAParag Agrawal47:44

        Yeah. It's, it's, it's why we've started

15. 47:46 – 55:18

    Parallel Web And Future Vision

    1. PAParag Agrawal47:46

       the company.

    2. SHSonya Huang47:46

       Why is the company called Parallel?

    3. PAParag Agrawal47:47

       [laughs] The company's original name was Shapley Inc.

    4. SHSonya Huang47:55

       Really?

    5. PAParag Agrawal47:56

       It was. When-

    6. SHSonya Huang47:57

       Do you know this?

    7. ARAndrew Reed47:58

       I knew that, yeah.

    8. SHSonya Huang47:59

       Oh.

    9. PAParag Agrawal47:59

       When I incorporated, I- I- I'm telling you, I was obsessing about everything to do with the problem space. So while knowing that the first set of... Also, one, Shapley Inc. is a terrible name for a B2B product. It was not going to be the long-term name. So we, we inc- But I incorporated as Shapley Inc. for lack of a better word, and Shapley.ai happened to be available. Shapley.com is a parked pawn domain, so it was not going to be our name. I went around not talking about my company as Shapley, and my badges at all events used to call it NewCo Or stealth company. So, and it took us almost six, eight months to figure out what the real name of the company will be. We ended up at Parallel in part because it- it's like, one, at the time we were doing a lot more in parallel, and two, we started visualizing this sort of a parallel web built for AIs and how its properties are different. And this metaphor that when you publish, you're now thinking of, as we all are now, of two audiences. Okay, I'm gonna create a page. I know humans will read it. What should it look like to them? And then how should I make sure that agents can read it too? So it feels like you're dual publishing to two audiences. And so we had this like, this parallel web for agents will emerge. And so that's why we started liking Parallel.

    10. SHSonya Huang49:35

        Love it.

    11. ARAndrew Reed49:36

        I was just thinking about this recently with the, um, you know, things like earning transcripts, right? Like, I feel very confident that more people are consuming earnings transcripts through agents than are actually certainly listening to the audio, and honestly probably reading the transcript itself. So there are these like very high value portions of the web that have flipped already to being agent first, and obviously the communication has not yet flipped. Um, but I think if I were a public company CEO today and I was doing an earnings report, I would make v- very clear that the thing that I'm saying will be transcribed and interpreted correctly by the agents, not just by the people listening, theoretically.

    12. PAParag Agrawal50:17

        Yeah. Same for us when we're publishing docs for our APIs. Like-

    13. ARAndrew Reed50:22

        Yeah

    14. PAParag Agrawal50:22

        ... it's like our customers are building AI solutions. They are using AIs to do it. It's their agents reading our docs and code in our FD case. It's not humans fumbling around docs pages-

    15. ARAndrew Reed50:40

        Yeah

    16. PAParag Agrawal50:41

        ... for the most part. So in fact, for us, the primary audience is a agent, and that's how we test our docs.

    17. SHSonya Huang50:47

        A parallel web for agents. That's very cool. Um, maybe close us out. Tell us, just give us a snapshot of where, where Parallel is today, and if everything that you hope to build comes true, what does Parallel look like? What does the world look like? What's your role in it?

    18. PAParag Agrawal51:03

        I think of three levels in this journey. Level one is people are building simple agents that use the web more as like a tool, like a web search tool, because it's familiar. Because the first set of agents we built, to your point, was just like model and give it the same tools, because these models have been trained to use all the tools human have been u- humans have been used to, and let them do work. By and large, if you think of like most work, it's there today, of our influence, it's there today. We have now started seeing some subset of customers who are in this world where they're building more sophisticated multi-agent systems which use sub-agents and have agents wake each other up or orchestrate, uh, in interesting ways, right? People are seeing that as sub-agents. Most familiar one is sub-agents within coding agent harnesses. But we see a lot more of that. Like if you're building an AI scientist, uh, some of those systems are very interesting, very sophisticated, very long-running and like just throw large amounts of compute and data at really hard problems. Uh, and I think the third layer for the web specifically is the web goes from pull to push. So today by and large across the first two modalities I described, either an agent is calling a tool or a sub-agent, but it is telling it a request saying, "Go find this for me right now." And I think where we will end up in a couple of years is a variety of use cases will be the web Parallel call me if this happens so that my agent can do some work or a human can do some work.

    19. SHSonya Huang52:58

        Mm.

    20. PAParag Agrawal52:58

        So I use this line with my team all the time, which is like imagine like agents are everywhere and they can do a lot of things, right? If there is something you can do today and it's worth doing, and we still have GPUs available, we'll just go do it, right? Like we won't like, we won't be like, "Oh, let's just do this tomorrow for the sake of it." If it can be done today, do it. So what will we do tomorrow? We will do tomorrow work in response to something that circles through either another agent's work or something changes in the world as visible in satellite imagery or some customer commentary that happens, some agent finishing some compute, a human having a new insight to trigger work. But there are going to be a few feeds like this which will drive new agentic work tomorrow. And one of those feeds is going to be everything that changed on the web, which is really exciting for me because then you are framing not a point in time need, but a long term, here is what is actionable for me. If something like this happens as evidenced by all of the information on the web that is changing all the time, call me, right? And then I'll run my agent on it. And so we get to then allocate compute onto the entire web all the time on behalf of all the customers, and that's really exciting.

    21. SHSonya Huang54:32

        That's awesome. Really ambitious vision. You're clearly extremely passionate about, about this and exciting to see you building. Thank you so much for joining us today, Parag. And thank you for joining us, Andrew.

    22. ARAndrew Reed54:43

        You're very welcome, Sonya. Thanks, Parag.

    23. PAParag Agrawal54:45

        Thank you, Andrew, for your debut. [outro music]

Episode duration: 55:18