Tavus: The AI Human Platform

1. 0:00 – 0:24

   Intro

   1. DHDiana Hu0:00

      [upbeat music] I'm excited today to welcome the founders of Tavus, to Quinn and Hassaan, who just raised a 40 million Series B led by CRV. Tell us what Tavus does.

   2. HRHassaan Raza0:18

      We build AI humans. We're an AI research lab that focuses on teaching machines the art of how to be human, so teaching them to see,

2. 0:24 – 1:18

   From Personalized Video to AI Humans

   1. HRHassaan Raza0:24

      hear, respond, act, even look like humans do.

   2. DHDiana Hu0:27

      What does that look like? Can we take a look at a demo?

   3. HRHassaan Raza0:29

      Yeah, that'd be great. Here, we're about to talk with Charlie, one of our awesome Tavus pals, and it's really cool because, you know, you can video call them, you can audio call them, you can text them even. They're always on. They're thinking about you. They're agentic. You can-

   4. DHDiana Hu0:41

      They look real, like a human.

   5. HRHassaan Raza0:42

      Yeah. Yeah, absolutely. And they can see you. They can react to all your expression and your emotions. They really feel like you're talking to a coworker or friend rather than a machine.

   6. DHDiana Hu0:50

      And I think the impressive stuff is that this is all running in real time.

   7. HRHassaan Raza0:53

      Yeah, absolutely. It's all running in real time. Yeah.

   8. DHDiana Hu0:55

      Let's actually see a very short clip of it interacting.

   9. HRHassaan Raza0:58

      Hey, what's up, Dom?

   10. SPSpeaker1:00

       Good morning, Hassaan. How may I assist you today?

   11. HRHassaan Raza1:06

       Yeah, can you tell me what my schedule's like for today?

   12. SPSpeaker1:09

       Certainly, sir. You are currently scheduled for a YC recording of some sort, followed by a 2:00 PM product meeting and a 3:30 investor call.

3. 1:18 – 2:36

   Why Real-Time Matters

   1. HRHassaan Raza1:19

      Great. My recording's running over. Can you draft a quick email saying I'll be 15 minutes late to the product meeting?

   2. SPSpeaker1:25

      Right away, sir. If you check your email, I've already prepared a message informing them of your delay.

   3. HRHassaan Raza1:33

      Beautiful. Thanks.

   4. SPSpeaker1:36

      My pleasure, sir. Can I be of any other assistance today?

   5. DHDiana Hu1:39

      Very cool, thank you.

   6. HRHassaan Raza1:40

      Yeah.

   7. DHDiana Hu1:40

      That's, that was a cool demo. So tell us a bit about who are some of your top customers.

   8. HRHassaan Raza1:44

      We have a pretty awesome diverse set of customers, everything from, like, small startups to, like, Fortune 10 companies, uh, some of which are, like, Amazon, um, Better.com, Alibaba, that are using our interfaces and our models to build these really cool AI employees.

   9. DHDiana Hu1:57

      What do these AI employees do? I think you mentioned you have three categories of types of applications?

   10. QFQuinn Favret2:04

       Yeah, so there's three big buckets that we really do a lot of work with. The first one is learning and development, so a lot of training, education, things like that. The second one is healthcare, so think patient intake, nutrition coaches, elderly companionship. And then the third bucket is go-to-market, so anything from an AI SDR to a AI solutions engineer to a customer support manager. So it's really across the board, and I think that's been part of the fun of building it, too.

   11. HRHassaan Raza2:27

       Yeah. It's, it's amazing to see all of the really awesome AI human use cases that we're seeing, um, people create. Like, just the craziest thing that we would never have imagined.

4. 2:36 – 4:05

   How AI Humans See, Hear, and Respond

   1. DHDiana Hu2:36

      So it's basically a virtual human that's doing all these tasks over, uh, video, right?

   2. HRHassaan Raza2:41

      Yeah. I, I mean, it's, you know, it's an AI human on video, um, but with, with the new updates, also we can, we can, it can video call you, it can audio call you, it can text you. It's, it feels like a coworker or friend.

   3. QFQuinn Favret2:52

      And for that to be the case, it needs to actually feel human, right? It needs to look like you, sound like you, understand, interact just like a human would and, and that's where, you know, it really, really gets its, its special touch.

   4. DHDiana Hu3:02

      And there's a lot of custom use cases, so the way you guys work right now, you got to here with being a SDK where customers like Amazon, Alibaba, take your SDK and build-

   5. HRHassaan Raza3:12

      Mm-hmm

   6. DHDiana Hu3:13

      ... the custom AI employee, right?

   7. HRHassaan Raza3:14

      Yeah.

   8. DHDiana Hu3:15

      But that was not the case all the time, right?

   9. HRHassaan Raza3:17

      Yeah.

   10. DHDiana Hu3:17

       I think when you guys went through the batch back in summer '21, this is, uh, way before the ChatGPT era. You were one of the OGs in AI before AI was a thing.

   11. HRHassaan Raza3:29

       Yeah.

   12. DHDiana Hu3:29

       Tell us about that era.

   13. HRHassaan Raza3:31

       Well, we like to say that we were doing AI before it was a cool thing to do back in 2020 and 2021. Yeah, I mean, when we went through YC, it was, um... There was, there was not that much that you could do with the models of the time, right? And so we were, we were sort of limited by what we could create. And so at the time, the best that we could do with these generative models was do what we call, like, um, lip sync infill, which is like, you could, uh... And we used it for AI personalized video. So basically you could record one video saying, "Hey, Diana, saw you worked at YC," and then you could send out thousands that were all personalized, lip synced with your, with your voice that would say, "Hey there,

5. 4:05 – 5:42

   Introducing PALs: Agentic AI Humans

   1. HRHassaan Raza4:05

      Alex, saw you worked at XYABC," right? That was what you could do at the time. You could create personalized video 'cause you could take a segment and, and really scale it up. Um, over time, though, of course, the models have evolved and our research has evolved to be able to do a lot more.

   2. DHDiana Hu4:19

      You guys stayed the course in this direction because before you were selling this tool to sales team-

   3. HRHassaan Raza4:25

      Mm-hmm

   4. DHDiana Hu4:25

      ... and then evolved into the SDK.

   5. HRHassaan Raza4:27

      Yeah.

   6. DHDiana Hu4:27

      So tell us what was that pivotal moment.

   7. HRHassaan Raza4:30

      I mean, I think it was a few things. I think it was, um, really understanding who we wanted to be as a company, right? I think that, like, after we did our Series A, there was, like, this, uh, crucible moment where we're like, "Hey, are we going to be, like, an AI sales company, or are we going to be focusing on developing these models for, like, you know, like, human computing, essentially? Like, to develop the next generation models for, like, human behavioral analysis and simulation." And we decided that, hey, like, we don't wanna go down the path of being an AI sales company. That's not, just not the DNA of our team. It's not who we are. And ultimately, we didn't really think that we c- we personally could make a massive business with that. And so we decided to take the path of saying, "Hey, we're actually gonna take a step back, and we're gonna go and focus on building the models and serving these as a next generation interface and as an API that you can consume and build on."

   8. QFQuinn Favret5:19

      I think a lot of it came back to our core, too. I think, you know, we went through the YC batch and it was, you know, we were pushing, we were trying to get as much momentum as possible and, and build, build, build. And we got a ton of customers that came to us looking for very specific things, which was awesome at the time. And then at the end of the summer, we took that step back and said, "Wait a sec. Like, okay, these are, these are really cool customers to work with, but, like, is this really who we are and what we want to be building?" And

6. 5:42 – 7:28

   The Foundational Models Behind Tavus

   1. QFQuinn Favret5:42

      took that, took that really critical reflection and ultimately, you know, made that jump into the API and SDK side.

   2. DHDiana Hu5:47

      The impressive thing is having the courage to take that leap because you essentially... I remember having this conversation. You churn all those-

   3. HRHassaan Raza5:54

      Yeah

   4. DHDiana Hu5:55

      ... customers, and you were on the track to get Series A traction before all that.

   5. HRHassaan Raza5:59

      Yeah.

   6. DHDiana Hu5:59

      And you were growing quite quickly. And to say, "That is not the path, and our DNA are more extremely technical." You had some really impressive research work that you'd done. It was not to be a sales company, and to be now what you are becoming, a AI research lab, right, took a lot. And it was the right choice.

   7. HRHassaan Raza6:17

      Yeah.

   8. DHDiana Hu6:17

      Evidently.

   9. HRHassaan Raza6:18

      Yeah, yeah.

   10. DHDiana Hu6:18

       Because now-You are building foundation models not just for rendering, which is what it was before, and now also perception. Can you tell us about these two sides of the yin and yang?

   11. HRHassaan Raza6:29

       I mean, ultimately it came down to, like, we saw a larger vision that we could really... We wanted to pursue. And I think a lot of the team has had this vision for a long time around, you know, machines that, like, meet us where we are. Uh, instead of us having to learn how a machine talks, like, they can meet us so they can video call us, they phone us, text us. And in order to do that, in order to teach machines, like, the art of, of, like, what this is, like, this is, like, a dance, um, then just giving them the face isn't good enough. And also the f- the face won't be good enough ever unless you teach a machine to see the way that we see and teach them, like, perception, contextual perception. So we, we both... It was both necessity to make the rendering models better, but also part of, like, if we wanna build these AI humans, then they have to be able to see our expressions, our gestures, because we speak as much through our face and what we don't say as we do through our words. And so we really focused a lot on trying to collect all these signals that otherwise weren't collected before, um, and then make meaning out of them. So teach the machines the relationship

7. 7:28 – 9:10

   Building Emotion, Expression, and Context

   1. HRHassaan Raza7:28

      between what you said and how you said it and the expression on your face and, like, how that was a reaction to what I had said, right? Like, all those things became es- you know, essential to building these AI humans.

   2. DHDiana Hu7:40

      And I think a key thing is that this is only possible to be built now because for this to work, it needs the real low latency-

   3. HRHassaan Raza7:48

      Yes

   4. DHDiana Hu7:48

      ... to have a fast response time. I mean, the human perception is more than, what is it, 10 milliseconds?

   5. HRHassaan Raza7:53

      Yeah. Yeah. At like, you know, an, a great response back and forth happens in, like, less than 200 milliseconds.

   6. DHDiana Hu7:59

      So you guys have some new product launches that are coming up. Can you tell us more about it?

   7. HRHassaan Raza8:04

      I mean, yeah. So for the past couple of years we've been working on the foundational models to teach machines to see, hear, respond, even look like humans do. Um, and with this new product launch, we're actually, um, essentially bringing, like, AI humans to life for regular consumers and prosumers. And we really think it's, like, a transformative moment where, like, we feel like we've been in the command line era of AI, or just like in early computing where we went from these, like, command line computers to GUIs that allowed millions of people to use m- uh, machines. I think with what we're calling the Tavus PALs, it's going to be the same thing, where you'll have this AI human, um, that meets you where you are. It can video call you, it can text you, um, it can... You know, you can, you can, you can call it. It's proactive, it's multimodal, it's agentic, so it can go and do things for you, and it feels really, really natural because it has a high degree of emotional intelligence powered by these really amazing state-of-the-art perception models.

   8. DHDiana Hu8:56

      That's pretty cool. This is a big evolution. You're going from serving very technical users with the SDK that need to be good software engineers to ship an AI human, now to someone who's not technical to be able to just launch it,

8. 9:10 – 11:00

   Use Cases From Startups to the Fortune 10

   1. DHDiana Hu9:10

      and that's gonna unlock a lot more interactions. I mean, if you guys, if you guys succeed with this, you're gonna probably be doing at some point 100 billion-

   2. HRHassaan Raza9:18

      Yeah

   3. DHDiana Hu9:18

      ... Tavus interactions, right?

   4. HRHassaan Raza9:20

      Yeah. Yeah.

   5. DHDiana Hu9:20

      How will the world change when you get to... When you get there?

   6. HRHassaan Raza9:23

      Yeah, I mean, I think our team will cry happy and sad tears. [laughs] Um, but I think, I think what we're really excited about is solving the human computing problem, which if you solve, then essentially, like, using a computer becomes something that's like second nature. You don't have to learn how to do it. Uh, it just feels like talking to a friend or coworker. So you'll talk to your AI doctor or your AI therapist or your AI assistant sidekick. Everyone will have, like, Cortana and Jarvis, like, you know, like sci-fi dreamed of, in their hands. It'll understand you immensely and that's, like, a future we're really excited about.

   7. QFQuinn Favret9:55

      It's the start, right? It's the tip of the iceberg. These are the five first class citizens showing what AI humans can and, and should be, and then from there it will, will continue to expand where people can create their own and build on top of those.

   8. DHDiana Hu10:05

      That's a very cool future where Tavus is going, and this is the other side of it where there might be a bit of, um, concerns from people-

   9. HRHassaan Raza10:14

      Mm-hmm

   10. DHDiana Hu10:14

       ... in terms of alignment and a lot of these jobs that AI-

   11. HRHassaan Raza10:19

       Yeah

   12. DHDiana Hu10:19

       ... are going to start taking. How do you guys think about that?

   13. HRHassaan Raza10:22

       I would say that, like, certainly some of that will happen, uh, I think at Tavus, like, we, we don't try to shy away from that and say, "Oh, we definitely, definitely won't be replacing jobs." But I think that the, the key goal is not to replace humans, but to replace bad machines that have been already put in place and are causing a regression. Like, telehealth already exists. You already don't get to talk to a front desk receptionist or a nurse, and it's just a worse experience for everyone involved. If you have an AI intake assistant that really understands you, it sees you, it sees how you're feeling, and it spends time with you in your language, like, that's a better experience. I also think that it's

9. 11:00 – 12:52

   Raising the $40M Series B

   1. HRHassaan Raza11:00

      a little bit of a privilege approach to be able to say, hey, like, if you had asked me, like, two years ago about AI therapy, I would've been like, "That's crazy. That's a bad idea." But the reality is, like, most people in the world can't afford therapy. They don't have access to it, and so the alternative isn't a human, it's nothing. Uh, and so if we can deliver a 80 to 90% good experience, it's not even replacing a human, it's putting this AI human somewhere where there was no alternative. Um, and that's something that we find to be really special.

   2. DHDiana Hu11:26

      So how do you actually build these AI humans that have empathy?

   3. HRHassaan Raza11:31

      That's a good question. It's interesting because, like, human conversation is incredibly nuanced. Um, we like to say at Tavus that human conversation is an art, it's a dance. Like, this is a waltz that we're doing right now, and machines are in the corner doing the robot, and we're trying to do a waltz. And so to m- to bring machines, uh, to be able to do the waltz, the first thing is you have to give them the right signal and be able to collect the right data. And so that's why we spent some time on perception, being able to collect the most nuanced data on that eyebrow twitch you had or the slight smile, uh, to be able to collect that data and then to be able to actually form understanding around it. Um, and the understanding piece and the relationship piece is actually really important. So teaching them the relationship between the eyebr- the eyebrow twitch and what was said before that, like, what was that in response to? We spend a lot of time modeling those things, and essentially you can summarize what we're doing as creating human simulation models.And we're doing, we're simulating human conversations and reactions and expressions. And it's a lot of fun, and there's a- it's a very human thing to do, uh, which is, like, always a little tough for research teams is, like, there's like the research angle is, like, very, very human rooted.

   4. QFQuinn Favret12:40

      And a big part of what's important here, too, is how we bring it to the world. I think one of the lessons we learned really, really early on was we need an amazing demo that shows someone the magic, the experience.

   5. HRHassaan Raza12:49

      Yeah.

   6. QFQuinn Favret12:49

      Because we've, you know, adjusted to how

10. 12:52 – 15:18

    The Future: AI Humans as the Next Interface

    1. QFQuinn Favret12:52

       we talk to machines for so long, right? We type instead of talk, right? We have all of these lossy mechanisms for how we change information, and we're almost having to reteach humans how they can communicate with machines, how they can-

    2. HRHassaan Raza13:02

       Yeah

    3. QFQuinn Favret13:02

       ... actually talk naturally and normally. So it's not just the models, but it's actually creating the experience around it, where people are able to have this, you know, really refreshing, this new experience of, of talking to a machine.

    4. HRHassaan Raza13:13

       Yeah, 100%.

    5. DHDiana Hu13:14

       What kind of advice would you have for founders that are just getting started?

    6. HRHassaan Raza13:18

       Have more conviction in, uh, in yourself and in the vision. Whenever you're a young founder, you can be more easily swayed [laughs] by, by, uh, by opinions and in, and feeling like you need to do, you know, you need to follow a certain path. And I think it took us a bit to figure out what we believed in is what we should be working on.

    7. DHDiana Hu13:35

       In particular, was that crucible moment.

    8. HRHassaan Raza13:36

       Yeah.

    9. DHDiana Hu13:37

       Because I remember that was a pretty big pivot for your company, and it took a bit of time for you to come into your own belief.

    10. HRHassaan Raza13:44

        Yeah.

    11. DHDiana Hu13:45

        Because you had a bunch of other people telling you otherwise, right?

    12. HRHassaan Raza13:47

        Yes, absolutely.

    13. DHDiana Hu13:48

        And it was a struggle.

    14. HRHassaan Raza13:49

        Yeah, it was. It, it felt like, it felt like you might let someone else down, and ultimately, like, this is a company that you're building, and you have to have full belief that the work that you're doing is a form of love. If you don't have that, then it's not gonna be successful no matter what.

    15. QFQuinn Favret14:02

        It, it's a craft. I, I think that was probably actually one of the most difficult times at Tavus, looking back at it-

    16. HRHassaan Raza14:07

        Yeah

    17. QFQuinn Favret14:07

        ... when we let those opinions influence us, right?

    18. HRHassaan Raza14:09

        Yeah.

    19. QFQuinn Favret14:09

        I think it impacted who we hired, what we were building, who we worked with. Like, it, it went throughout the business. It wasn't even just about the direction.

    20. HRHassaan Raza14:16

        Yeah.

    21. QFQuinn Favret14:16

        One of the learnings was definitely about not even believe in ourselves, but just have a deep conviction in what we're doing and let that flow everywhere throughout the business. But the other one is the only thing that matters is momentum.

    22. DHDiana Hu14:25

        Hmm.

    23. QFQuinn Favret14:26

        Like [laughs] if, if things are not moving, then the business isn't moving. And, and it's in the small things, it's in the big things. But the only thing that matters is every day something needs to happen, right? And I think driving that through and pushing that through is, is one of the few things that has-

    24. HRHassaan Raza14:39

        Yeah

    25. QFQuinn Favret14:40

        ... kept things moving forward day over day.

    26. HRHassaan Raza14:41

        100%.

    27. DHDiana Hu14:42

        I think that's a good way to end. I think we talked about in one of the videos recently that the first moat startups really only have is just speed.

    28. HRHassaan Raza14:49

        Yeah.

    29. DHDiana Hu14:50

        And you guys done it.

    30. QFQuinn Favret14:51

        Our take on it is we are six months ahead, and we need to keep moving as, as fast as we can to keep that up.

Episode duration: 15:19