By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
YC Root AccessThis Startup Built the Infrastructure Powering Voice AI
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
Transcript
50 min read · 9,658 wordsIntro
Jared Friedman 0:00 [upbeat music] It's super cool to be here today with Dylan from AssemblyAI. Dylan, thank you so much for joining us.
Dylan Fox 0:10 Yeah, I'm excited to be here.
Jared Friedman 0:11 So AssemblyAI is one of the first AI companies YC ever funded in it, the, in our first show, we called at the time, AI Cohort [laughs] back in the summer of 2017, and today it's actually one of the most successful companies YC has ever funded. For folks who aren't familiar, how about you just tell everybody what AssemblyAI is and, uh, where you're at now?
Dylan Fox 0:32 So we help other companies build voice AI features and applications. Everything from AI note takers, to AI capabilities in contact centers, to real-time voice agents. Healthcare companies are building ambient medical scribes on top of our voice AI infrastructure platform. So we have about a million developers that have signed up to the platform. Um, we have about 10,000 customers. Last year, o- around 250 million voice hours were sent through our platform, and we're now doing-
Jared Friedman 1:05 250 million voice hours. Okay. [laughs]
Dylan Fox 1:08 Yeah. We're now doing, um, almost two million hours per day. Uh, so the run rate-
Jared Friedman 1:12 Wow
Dylan Fox 1:12 ... on that would be, you know, like 700 million [laughs] voice hours for this year.
Jared Friedman 1:17 Voice hours. Wow.
Dylan Fox 1:17 And it's continuing to grow week over week. Um, so we really are focused on helping other companies, uh, have the voice AI infrastructure and primitives they need to just innovate around voice and to build innovative products and features, uh, either like standalone or in their, in their products.
Jared Friedman 1:37 Do you have an example of maybe one or two customers that people would know and how, uh-
Dylan Fox 1:42 Yeah
Jared Friedman 1:42 ... how, how they're built on Assembly?
Dylan Fox 1:43 Yeah. For sure. So note taking is a really popular use case that companies are, are, uh, shipping with our platform. So if you've used Granola or Fireflies note takers, those use AssemblyAI's, uh, platform.
Jared Friedman 1:55 Interesting. So, so all, all my Granola notes that I, you know, I had said to where w- it was actually all going through Assembly.
Dylan Fox 2:03 Yeah. Uh-
Jared Friedman 2:03 Okay
Dylan Fox 2:04 ... that's right. Um, also, um, in, in hiring, uh, hiring
What AssemblyAI actually does
Dylan Fox 2:08 segment, so, um, if you've used MetaView or, um, Ashby's note taker-
Jared Friedman 2:14 Okay. Yeah. Ashby. Yeah. Excellent
Dylan Fox 2:15 ... yeah. Um, yeah. So note takers are a really big segment. We're also, um, pretty widely deployed in contact centers, so there's a big contact center company, they're called Collaboro. Um, they provide contact center, uh, software to big, uh, brands like Delta Airlines.
Jared Friedman 2:31 Okay. So if I called 1-800-Delta or, or, or whatever to like change my Delta flight, there's a good chance that like under the hood that audio was actually being processed by Assembly through this like chain of customer relationships.
Dylan Fox 2:43 Most likely, yeah.
Jared Friedman 2:44 Okay.
Dylan Fox 2:44 So right now, um, product teams are shipping, uh, products and cap- like capabilities around voice with our platform, but then we're also seeing enterprises. So right now we have a, a lot of, um, Fortune 500 enterprises that are building AI capabilities themselves within their contact center operations or within their trust-
Jared Friedman 3:06 Hmm
Dylan Fox 3:06 ... and safety operations around voice to automate workflows for their team. So it's a mix of product teams, but then also enterprise companies that are, are, uh, looking to build with voice AI. So Zoom, for example, um, has used our, our infrastructure for-
Jared Friedman 3:21 Oh, wow
Dylan Fox 3:21 ... a, a number of different capabilities. Yeah.
Jared Friedman 3:23 You guys were super early to AI. 2017, this was like way, way before ChatGPT. I remember your batch. There was what? A handful of companies working on AI?
Dylan Fox 3:32 Yeah. Yeah, there were like 10 other companies in the Y, in the AI cohort.
Jared Friedman 3:36 Tell us the origin story. How did you end up starting this company? How did you end up getting into AI so early?
Dylan Fox 3:41 Yeah. So I, um, I started a c- I mean, the, the really, you know, ear- early story is I started a company in college, um, and, and it was not an AI company, but through that, uh, that experience, I taught myself how to code. So, um, you know, you wouldn't need to do this now with AI, but back then I bought a bunch of programming books on PHP and Python and Django, and I just read them all and taught myself how to code and started-
Jared Friedman 4:10 Hmm
Dylan Fox 4:10 ... building, um, like SAS software products and, and, and launching them as, as little startups. Um-
Jared Friedman 4:17 Like what?
Dylan Fox 4:18 So one-
Jared Friedman 4:18 What did you launch?
Dylan Fox 4:18 [laughs] One was like a fundraising tool for college organizations.
Jared Friedman 4:23 Okay.
Dylan Fox 4:23 Um, one was a tool where, uh, small businesses could like put up a QR code and then customers could text them, uh, f- uh, like a, a number that they could text feedback to, so it was like anonymous feedback capture through text message, a bunch of random stuff. Um, and I, I learned from that, that I really loved programming and that I really loved the, the creative, um, like journey of building something and, and like thinking of this, you have this like fuzzy picture of a product or something in your mind, and then you could just, you know, through programming, build it and see it come to life and get feedback. It was this very like addictive kind of, um, feedback loop that I, I really enjoyed. So after college, I just, I kept programming and kept, you know, learning about, um, uh, everything from like, uh, uh, security to machine learning, and I really got interested in machine learning. So this is like support vector machine,
Dylan learns to code and discovers ML
Dylan Fox 5:23 uh, time. This is not like deep neural net net time.
Jared Friedman 5:26 Yeah.
Dylan Fox 5:26 Um, eventually I joined a machine learning team out in San Francisco. So I, I moved to San Francisco, joined a team at Cisco, um, the company Cisco, that's based here. It's actually like a few blocks from where we're filming right now.
Jared Friedman 5:41 Oh, cool.
Dylan Fox 5:41 That's where my job was in Dogpatch. Yeah. And that was where I got, I, I started getting, learning about neural networks and getting into neural networks.
Jared Friedman 5:49 Hmm.
Dylan Fox 5:49 So to give you a sense of time, this is like 2015. Um, so, uh, like very early into, to, you know, the neural network boom.
Jared Friedman 5:57 It's just like right when, when we ... Wait. W- w-When did we crack ImageNet?
Dylan Fox 6:04 I think it was like 2013 or '14.
Jared Friedman 6:06 '14. Okay. Yeah.
Dylan Fox 6:06 It was around that time.
Jared Friedman 6:07 Okay.
Dylan Fox 6:08 Um, but I do remember going to the very first TensorFlow meetup down at Google's headquarters in like 2014 or '15. So it was, you know, that, that was the, the, um, the state of things at that time. A- so a couple things happened around that time. Um, one is I saw that the AI and, and neural networks and deep learning were going to, um, take off over the next decade. That it was such early days and data compute algorithms were just gonna continue to get stronger and stronger, and they were going to bring, you know, really disruptive new capabilities to everything from self-driving cars to natural language processing and, you know, ev- everything that we've seen over the last 10 years. Um, and so I just really felt like, wow, that's an area I wanna... I, I was drawn to and wanted to keep going, uh, deeper in. And then I, [clears throat] I bought an Amazon Echo when that came out in 2015. It was crazy because my experience with voice recognition, uh, prior to the Amazon Echo was that it
The Amazon Echo moment
Dylan Fox 7:11 sucked, right?
Jared Friedman 7:11 It sucked. Yeah, yeah.
Dylan Fox 7:11 Siri, Siri didn't work.
Jared Friedman 7:13 Yeah.
Dylan Fox 7:13 Everything sucked.
Jared Friedman 7:14 Yeah.
Dylan Fox 7:14 You didn't use it. But the Amazon Echo, you, you could use it across the room, so far field it worked, like, you know, great distance from the microphone. Your TV could be on, you could be, you know, shouting to it over a bunch of background noise, and it would still work. And what was so incredible was that it worked so well that I found I was building new habits around the product because it was reliable.
Jared Friedman 7:37 Mm.
Dylan Fox 7:38 So setting timers, asking for the weather, playing music. It was just like, I was blown away by how well those things worked over voice and that it was reliable. Um, and so I started looking into voice recognition technology to just build stuff on my own and to innovate and experiment on my own. Um, and there was nothing in the market at the time. It was either, like, bad or, um... So the, the company Nuance at the time, which was like a big-
Jared Friedman 8:06 They were the big incumbent.
Dylan Fox 8:07 Yeah, yeah, exactly. Um, I got in touch with them, and I, you know, was trying to find their developer SDKs, and it, it... You had to pay, I think it was like, you know, thousands of dollars up front.
Jared Friedman 8:17 I, I, I remember. I, I purchased it, yeah. [laughs]
Dylan Fox 8:20 And then they would mail you a CD-ROM-
Jared Friedman 8:22 Ooh
Dylan Fox 8:22 ... with a developer SDK on it, and I didn't even have a CD-ROM drive-
Jared Friedman 8:25 [laughs]
Dylan Fox 8:25 ... on my laptop. So, you know, for me, I was used to the Twilio or Stripe-
Jared Friedman 8:30 Mm
Dylan Fox 8:31 ... style developer experience and, and [laughs] this was, like, the opposite of that. Um, so all these, like, ideas kinda came together for me and I got really excited about the idea of what if you could, um, you know, use like these new deep learning algorithms, a- all this innovation that was happening around AI and deep learning at the time and to, to build way better, uh, voice AI capabilities and technology, and then make it super easy for any developer to build with those things. 'Cause for me, I remember getting access to Twilio, you know, back in the day-
Jared Friedman 9:12 Yeah
Dylan Fox 9:12 ... so like 2012 or 2013, getting access to Twilio, and I had s- I, I was just overwhelmed with creativity-
Jared Friedman 9:20 Yeah
Dylan Fox 9:20 ... because I was like, wow, now I ha- like, I have access to this cool technology-
Jared Friedman 9:26 Yeah
Dylan Fox 9:26 ... me as a, as a college student developer, and, like, I can e- experiment with all these really cool
Why Dylan built voice AI infrastructure
Dylan Fox 9:32 ideas. And, and that has been the whole spirit about Assembly. You know, we're not an application company. We're really focused on being a developer infrastructure tool that just makes it really easy for a developer, whether you're, you know, a, a college student or a high school student or a development team at a Fortune 500, to have access to this amazingly powerful technology, um, [clears throat] effortlessly and with a really good experience around it. So that, that was the, [clears throat] kinda, um, the, the mix of, of ideas that came together and then, you know, is still what we're, what we're focused on, you know, delivering, uh, many years later.
Jared Friedman 10:13 I don't know if you've read Paul Graham's essays, but, um-
Dylan Fox 10:16 Some of them
Jared Friedman 10:17 ... a lot of his, his early essays talk about how, um, most of the best companies of the past, like Microsoft and Apple, didn't start as companies. They started as just an engineer building technology that they found intellectually interesting, without any clear idea of how it would turn into-
Dylan Fox 10:33 Yeah
Jared Friedman 10:33 ... a business and scratching their own itch, right? Like, building the thing that th- that they saw missing in, in the world-
Dylan Fox 10:40 Yeah
Jared Friedman 10:40 ... that they personally wanted to use. And, like, you know, at the time that you started this, like, AI was not the hot thing. It was not, it was not the thing raising huge venture rounds. It-
Dylan Fox 10:50 You couldn't use AI.
Jared Friedman 10:51 Yeah. [laughs]
Dylan Fox 10:52 If you said AI in your... It, it was deep learning was the, the-
Jared Friedman 10:55 Okay
Dylan Fox 10:56 ... the thing to say. If you used AI-
Jared Friedman 10:57 AI
Dylan Fox 10:57 ... it was, it was, um, it was like a scam. [laughs]
Jared Friedman 11:00 Yes. [laughs] Exactly.
Dylan Fox 11:01 That was the state of things. Yeah.
Jared Friedman 11:02 Totally. Yeah. Really 'cause there had been like decades of, like, failed promises-
Dylan Fox 11:04 Yeah, yeah, yeah
Jared Friedman 11:05 ... from AI startups-
Dylan Fox 11:06 Yeah
Jared Friedman 11:06 ... so VCs were allergic to anything with AI in it. Yeah. [laughs]
Dylan Fox 11:08 Yeah, exactly. Exactly. My experience as a founder has, has, has been one that I, I think you have to be obsessed with the problem that you're solving.
Jared Friedman 11:16 Yeah.
Dylan Fox 11:17 Because what keeps me going is, like, I really want this product to exist that, that we have not, you know, finished creating yet, you know? And, and I, I'm, I'm obsessed with, like, that problem and wanting that to, to be realized. Whereas I think if you're just focused on like a business opportunity, you know, um, then, and you're not obsessed with the product or the, the subject matter, um, you can find yourself losing interest. But I think if you're... You know, you should pick... M- my lesson has been, like, and what I share with other founders, like, you should pick a problem that you're, you're obsessed about. Because when I think about, like, what would I be doing if I wasn't building Assembly, you know, I, I don't know. Like [laughs] I'm having a lot of fun-
Jared Friedman 12:03 [laughs]
Dylan Fox 12:03 ... building Assembly. Um, and I use our product all the time. Like, I'm constantly-Dogfooding it and playing with it and, and showing people demos, and it's fun. So I think that, you know, picking something, working on something that, like, y- you, you want to exist because you wanna be a user of it, um, ha- has been really fun.
Jared Friedman 12:24 To, to me, probably the craziest part of the Assembly story is that you started working on this in 2017 when very few people were interested in AI.
Dylan Fox 12:32 Yeah.
Jared Friedman 12:33 And it basically didn't start taking off until 2021, 2022. You, you raised your Series A in 2022, five years after starting the company.
Dylan Fox 12:41 Yeah.
Jared Friedman 12:41 And the first years were just this, like, long slog where it wasn't clear that it was ever gonna work. I'd love to hear about those early years-
Dylan Fox 12:52 [laughs]
Jared Friedman 12:52 ... and, like, what kept-
Dylan Fox 12:53 A lot of-
Jared Friedman 12:53 Yeah, what-
Dylan Fox 12:53 Yeah
Jared Friedman 12:54 ... w- w- what was it like? What kept you going? Especially as a s- solo founder, there weren't that many, there weren't that many solo founders back in 2017.
Building AI before anyone cared
Dylan Fox 13:02 Yeah. Yeah, so I got into YC summer to- summer 2017. Um, solo founder. I had pretty much just started working on the company, and I got in la- I applied late, got in a week before the batch started, and I was just so stressed out. [laughs]
Jared Friedman 13:18 [laughs]
Dylan Fox 13:19 Because, uh, you had all these other companies that, you know, had, like, real products, and they were further along, and, um, they could iterate so much faster. But, like, it... I picked a very hard product to work on back then, and I just remember being, you know, s- [laughs] so stressed out and so overwhelmed ba- back in 2017. Um-
Jared Friedman 13:39 Yeah, everybody else was just, like, building database-backed websites.
Dylan Fox 13:42 Right.
Jared Friedman 13:43 They could, like, push things every day.
Dylan Fox 13:44 It's like, oh, you get feedback-
Jared Friedman 13:45 Yeah. [laughs]
Dylan Fox 13:45 ... you can iterate on it that night, right? Whereas, like, our early users would give us feedback, and it's like, "All right. We'll come back to you in a month [laughs] when we have a, a better model." And YC is three months, so we had three-
Jared Friedman 13:57 [laughs]
Dylan Fox 13:57 ... you know, iteration cycles.
Jared Friedman 13:58 [laughs]
Dylan Fox 13:58 It's not, it's not a lot of iteration. Um, and no one was building with voice because, like, what I have now realized is that, um, this whole ecosystem of technology needed to be created l- so that you can build voice AI applications. What do I mean by that? I mean, yeah, you need really good voice AI models, but you also need LMS. You also need vector databases. You also need WebRTC. You need mobile 5G. Like, you need this whole ecosystem-
Jared Friedman 14:28 Yeah
Dylan Fox 14:28 ... um, to, to build what we're seeing developers build. You know, w- I remember, like, 2021, we started to get transcription models to work, like, decently well. Um, but the interesting applications were not just, like, transcribing something. It was sentiment analysis of a phone call. It was summarizing a meeting. But if you wanted to do those types of tasks back then, you had to train sentiment analysis models or train summarization models, right? So whereas now, you can use Assembly and, uh, you know, an LM, and, like, what you can do is... It's, it's insane.
Jared Friedman 15:06 Yeah.
Dylan Fox 15:06 And it's super easy to build-
Jared Friedman 15:08 Yeah
Dylan Fox 15:08 ... and innovate around voice data. So it took a while for that ecosystem of technology to come together, and I think it goes back to probably what I was saying earlier, where if in 2017 I, if I had put together, like, a business plan-
Jared Friedman 15:24 [laughs]
Dylan Fox 15:24 ... I'd probably been like, "I shouldn't work on this," because [laughs] you know-
Jared Friedman 15:28 Like, like-
Dylan Fox 15:28 ... the market is so small.
Jared Friedman 15:29 Yeah, yeah. The tab is, like, $10 million.
Dylan Fox 15:30 Yeah, exactly.
Jared Friedman 15:31 [laughs]
Dylan Fox 15:31 The market's so small, and all the technology sucks, and, like, why am I working on this?
Jared Friedman 15:35 [laughs]
Dylan Fox 15:35 But I, you know, I, I thought it was a fun problem to work on. Um, and I was really fortunate that back then... So Daniel Gross was our YC partner. Um, he had worked at Apple. He saw the state of voice recognition technology, and he was a really early believer. And so he invested personally in the company, um, and really supported the company early on post-YC. And we just had these, like, early believers in the company that knew it was very early. And I think for me, I always had conviction that it was kind of like self-driving cars, where, okay, once you... It wasn't a question of, like, product-market fit. Like, will, will people want voice AI technology? It was a question of, um, when will it be... When will it have product-market fit? When will it get good enough, like self-driving cars?
Jared Friedman 16:26 Yeah.
Dylan Fox 16:27 Um, and I think that in hindsight, we probably could have gone a bit faster, but, you know, I was learning and... [lips smack] Um, yeah, it took, it took a while. But to give you, you know, a sense, it was like the first couple of years, not much progress. We were, like, three, four people. [clears throat] And then I think we landed our first, you know, real customer in, um, 2021.
The 2021 inflection point
Jared Friedman 16:50 Who was that? W- The first real customer was-
Dylan Fox 16:52 Like, real legit customer-
Jared Friedman 16:53 Real legit customer
Dylan Fox 16:54 ... in 2021. Still a customer of ours. Um, a contact center company.
Jared Friedman 16:57 Okay.
Dylan Fox 16:57 And raised our Series A in January of 2022. Um, and then to date, we've raised about-
Jared Friedman 17:04 Which was still a little bit before LLM. So, like-
Dylan Fox 17:07 It was, yeah
Jared Friedman 17:07 ... what, what happened in 2021, 2022-
Dylan Fox 17:10 Yeah
Jared Friedman 17:10 ... that caused the, like, fir- the, the beginning of the inflection point?
Dylan Fox 17:13 Yeah. So, um-
Jared Friedman 17:14 Okay
Dylan Fox 17:15 ... COVID was a big part of it.
Jared Friedman 17:16 Mm.
Dylan Fox 17:16 So during COVID, all of a sudden you had way more voice data created and captured over the internet than ever before with remote work. Podcasts started to get popular around that time. Our models started to get better. We started to use, you know, modern transformer architectures, train on more data. So, like, transcription was getting better, it was getting cheaper. Then you had other NLP models like BERT that made it easier to do summarization-
Jared Friedman 17:39 Mm
Dylan Fox 17:39 ... or sentiment analysis-
Jared Friedman 17:40 Mm
Dylan Fox 17:40 ... on top of transcription. So you had, like, the ecosystem of technology starting to come together, where it was easier to build a voice AI application. And so there was acceleration in the TAM around that time. Um, and Accel led our Series A, and they saw that too. Um, and then between then and now, obviously it's just continued to accelerate, you know, pretty dramatically. We raised about $160 million in the course of, like, uh-... I don't know, three years or something, um, last. So or, or yeah, like three years. So it was, uh, it, it took a while to get to that point, but then once it did, it really started to accelerate.
Jared Friedman 18:19 All the use cases in those early years, were they all non-real-time use cases?
Dylan Fox 18:23 They're all not real-time.
Jared Friedman 18:24 On-
Dylan Fox 18:25 Yeah
Jared Friedman 18:25 ... all so, like, analyze all the calls that went through this call center o- for the past week-
Dylan Fox 18:30 Yeah
Jared Friedman 18:30 ... and, like, find the ones with bad, bad, like, NPS scores-
Dylan Fox 18:34 Yeah. It was-
Jared Friedman 18:34 ... or things like that
Dylan Fox 18:35 ... pretty much all, you know, use cases on top of pre-recorded audio. And then now, um, there's still a ton of growth in those use cases. Um, like usage to our non-real-time APIs is still growing, you know, 200% year over year. But real-time use cases are exploding because the real-time capabilities are now way better, uh, way lower cost. You have, you know, ecosystem of real-time technology to build real-time note-takers, real-time voice agents. That market is really taking off, that use case. Um, but, but back then it was all non-real time.
Jared Friedman 19:13 And, and that was your intuition with the Amazon Echo. I mean, that that was a real-time product.
Dylan Fox 19:17 Right.
Jared Friedman 19:17 That was, that was the-
Dylan Fox 19:19 Right. We started the-
Jared Friedman 19:20 ... magical experience-
Dylan Fox 19:21 Yeah
Jared Friedman 19:21 ... that got you started. Yeah
Dylan Fox 19:21 The very first API I built was a real-time API. But the real-time models back then, I mean, in general, all [laughs] all the models were bad.
Jared Friedman 19:30 [laughs]
Dylan Fox 19:30 Um, real time, which is a lot harder, and it's only been, I would say w- really within the last probably 18 months that real-time models have started to get like, like cross ID threshold. And, and that's why you're seeing real-time voice AI use cases take off because that threshold has, has been crossed in the last 18 months across accuracy, latency, and it's still not perfect, right? Like, um, there's, there's real-time speaker identification that, you know, doesn't work that well today. So there's still a ton of, of room to go.
Jared Friedman 20:07 Mm-hmm.
Dylan Fox 20:07 But, but what I have found is that all of a sudden, these technologies cross some threshold where the... and it's so hard to pinpoint what that threshold is, but all of a sudden they cross some threshold, and there's product market fit, and it just takes off. And so, you know, for the first time we experienced that was around 2021. Um, and then now we're experiencing that again.
Jared Friedman 20:27 Mm.
Dylan Fox 20:28 Um, and I would say that's across both real-time and non-real time again. So some of the newer non-real-time models that we have, they're more like, uh, uh, general purpose, like audio understanding models so they can, you know, identify speaker genders, and they can, uh, uh, capture like, uh, background, uh, sound effects, and they can, they can just understand and capture audio with new capabilities, and that's unlocking new use cases. So, um, what, what innovative teams can do i- it just keeps kind of, it, what they can do just keeps kind of increasing.
Jared Friedman 21:07 So that's sort of the business side. I'm, I'm kind of curious to hear about more like the personal side, the, like the, the, the company side. Like you were building this thing very quietly. It's really just a few people for like five years.
Dylan Fox 21:20 Yeah.
Jared Friedman 21:20 And then all of a sudden, I guess, I guess it wasn't like actually overnight, but like in a relatively short period over the course of like three years or something-
Dylan Fox 21:27 Yeah
Jared Friedman 21:28 ... it just started like really working, and it went on fire and like what, what was that like as a, as a, as a founder?
Dylan Fox 21:35 [laughs] Yeah, definitely like lear- a lot of growing pains. Um, w- and I, I, I think I learned a lot through that period where [clears throat] in the early days, um, in the early days it was kind of just like, you know, no one's paying attention, uh, stressful in a different way. And then now it's like, okay, there's a huge, you know, opportunity. We really wanna be a leader in the space and, and, and really like executing against that, that opportunity. So I would say, um, we, you know, we, w- when you, when you raise a lot of capital quickly, I think that, that puts a lot of pressure on a company, um, and on a team. And I think that, uh, like the, the biggest lesson I've learned is like you just have to kind of trust your, trust your instincts as a founder, um, 'cause you really, you know, understand, like you should, like the market you're building in and the right like, company market fit you need and stuff. And so, um, I think like it w- it was, it was hard to know, um... I, I think it was hard to know, like what were the right instincts to follow, um, when we started to see a lot of acceleration in the company and, um, and, and that's just like took a bit of time, but now I think our instincts have, have been more, have become like more refined as a company and then me too, as a founder.
Jared Friedman 23:09 You have sort of incredible insight into what's happening in the voice AI market. Like what's happening now, especially over the last few months, what are, what are the big things you guys are working on? Where do you think it's gonna go in 2026?
Dylan Fox 23:20 Yeah. So there's a couple of very exciting things we're seeing. So, um, real-time voice agents are, are, like, they work really well. Now, it's kind of wild. Um, my, uh, yeah, they, real-time voice agents, they, they work really well now. So I think those are being rapidly deployed because the success rate of a real-time voice agent is high enough now where they can deliver a good customer experience over customer support, um, you know, frontline, like reception tasks. And I think as a consumer, I, I don't know. Have you, have you interacted with one yet?
Jared Friedman 23:59 Of course. All the time.
Dylan Fox 24:00 Yeah. Yeah. So like when, when you... I'm starting to now, when I call a plumber or service tech, realize like, "Oh, I'm talking to a voice AI." [laughs]
Jared Friedman 24:07 Yeah. [laughs]
Dylan Fox 24:07 'Cause I, 'cause I know like the-
Jared Friedman 24:09 The-
Dylan Fox 24:09 Yeah
Jared Friedman 24:09 ... the, the, the vast majority of people can't tell, I think.
Dylan Fox 24:12 Right. Exactly.
Jared Friedman 24:12 Yeah.
Real-time voice agents are here
Dylan Fox 24:13 Yeah.
Jared Friedman 24:13 Okay.
Dylan Fox 24:13 And it's, we're still in this phase where-You can't tell, but then like some, some, some people will midway through be able to tell, and it just gets weird. It's like, wait, I'm- I, I just realized I'm talking to an AI.
Jared Friedman 24:23 Yeah.
Dylan Fox 24:24 Um, so I think real-time voice agents are gonna continue to be widely deployed, and, um, and the ROI there is like crazy, and the capabilities around real time are getting really amazing. We're also seeing a lot of demand around, [sighs] um, robotics and consumer hardware. So a lot of like pretty popular robotics companies, I don't know if I can name them or not, [laughs] but a lot of really popular robotics companies are, um, putting our models on their robots, so you know, humanoid robots that are walking around.
Jared Friedman 24:56 Yeah.
Dylan Fox 24:56 'Cause-
Jared Friedman 24:56 Oh, that's cool.
Dylan Fox 24:56 Yeah. I think robotics, but also consumer hardware, you're gonna start interacting with that more and more through your voice. So even, you know, the coffee machine downstairs, right? Like you have to kind of-
Jared Friedman 25:06 Right
Dylan Fox 25:06 ... flip through a touch screen. But if you could walk up to that coffee machine and just ask for the-
Jared Friedman 25:11 Yeah
Dylan Fox 25:11 ... exact type of coffee that you want, you know, there's no reason why that technology shouldn't also exist.
Jared Friedman 25:16 Totally.
Dylan Fox 25:17 It took a while for consumer hardware to all have a touch screen on it. Um, but now I think, you know, voice will also be a, a another modality into these, um, uh, hardware devices that we, that we buy. So we're seeing a lot of demand there. But then also a- these ambient devices. So, uh, healthcare. We have a ton of healthcare companies-
Jared Friedman 25:37 Hmm
Dylan Fox 25:38 ... building ambient, uh, scribes for doctors.
Jared Friedman 25:42 And this is a physical device?
Dylan Fox 25:43 This is, uh, software-
Jared Friedman 25:45 Okay
Dylan Fox 25:46 ... that they're just running on a laptop-
Jared Friedman 25:48 Got it
Dylan Fox 25:48 ... or running on a phone.
Jared Friedman 25:49 Sure.
Dylan Fox 25:50 And the doctor/patient visit is captured.
Jared Friedman 25:53 Yeah.
Dylan Fox 25:53 'Cause if you listen to some of this audio, it's actually like cra- [laughs] crazy hard. Uh, it's someone's laptop is recording a conversation that's like 10, 15 feet away. Um, tons of background noise, low speech. But the models can do a pretty good job now-
Jared Friedman 26:08 Wow
Dylan Fox 26:08 ... at capturing all that at- with like, you know, accuracy rates in the 90 percentile. And so, um, now you can actually automate chart notes, and you can automate insurance submissions post-doctor visit. So we're seeing a ton of healthcare companies build ambient scribes, but also sales companies build ambient scribes. So-
Jared Friedman 26:27 Mm-hmm
Dylan Fox 26:27 ... um, there's a company we work with called Zero, if you've heard of them. They, they have a ambient, um-
Jared Friedman 26:33 Oh, yeah
Dylan Fox 26:33 ... scribe for, uh, in-person sales teams. So you're going, you know, door to door or you're, you're doing-
Jared Friedman 26:39 Mm
Dylan Fox 26:39 ... field sales. You have a app that... They, they build an app you can run. It will give you advice on your, your conversation that you're having, um, in real time in, in person. And-
Jared Friedman 26:51 That's cool
Dylan Fox 26:51 ... those use cases are now working well enough where, um, where the sales people that are using that tool are... I think the quote I heard from Jake, their founder, is they're taking like 10 to 20K more per quarter home in take home pay-
Jared Friedman 27:06 Wow
Dylan Fox 27:06 ... because they're, they're getting this advice that they weren't getting before. So, um, real-time voice agents, um, uh, consumer hardware, and then ambient capabilities that are just kind of listening, healthcare, um, sales, those are where we're seeing a ton of new applications being built and a lot of, lot of like greenfield.
Jared Friedman 27:28 What's the team working on now? What's on, what's, what's on the roadmap for 2026?
Dylan Fox 27:32 We're really focused on building smarter voice AI models, and so what I mean by that is models that can do more than just transcribe your speech, that you can give instructions to, that you can control. Um, they're just a lot more intelligent. I think, uh, c- current voice AI models have gotten really good, but they're still kind of dumb. They don't have a lot of intelligence in them, um, to understand context, to, uh, you know, for example, understand this person's talking to me, but they're in a noisy room, and there's a, you know, baby crying.
Jared Friedman 28:09 Hmm.
Dylan Fox 28:09 And they're, they're, they're s- they're, they sound stressed, right?
Jared Friedman 28:11 Hmm.
Dylan Fox 28:12 Um, to, to understand speakers that have, uh, that speak different languages really well, to know who's the primary speaker, who's the background speaker, a common issue with, v- you know, if you're building a robot, for example,
Inside AssemblyAI’s new voice models
Dylan Fox 28:26 a common issue is like which voice do I listen to and take commands from?
Jared Friedman 28:30 Ah.
Dylan Fox 28:30 And which voice is the voice that's just like chatting with the human?
Jared Friedman 28:33 Interesting.
Dylan Fox 28:33 Right? So-
Jared Friedman 28:34 Kind of the like noisy restaurant problem of-
Dylan Fox 28:36 Yeah.
Jared Friedman 28:36 Yeah.
Dylan Fox 28:36 Yeah. And, and knowing like who, who is saying what, what's the role of the person. So all this we think about as, you know, more intelligent, you know, voice AI capabilities that we're focused on building that will operate in real time. Um, I, we think that's like a, a, a big capability that we're seeing a lot of demand for. And so that's... We just released a model last week, um, called Universal 3 Pro. It's our latest model, and it's our most intelligent model 'cause you can give it instructions on what you want it to do around the audio that it's listening to, and it can follow those instructions pretty well, and this is just the first release. Um, and I, I can show you some of that.
Jared Friedman 29:17 In, in, in, in instructions like a, like an LLM prompt.
Dylan Fox 29:19 Correct. Yeah. So it's, it's... We, we, we think about it somewhere in, in the middle between a multimodal LLM and a traditional speech-to-text model, where what we really did was focus on creating a reliable, you know, transcription model, um, that can, can stay on the rails on transcription and transcription-like tasks, but that c- you can still give instructions to. Whereas if you were to use a multimodal LLM today and give it instructions, it goes off the rails too much. It's not reliable enough in real world use cases. So, um, we focused a lot on the post-training of this model so that it's like a reliable transcription model, but you can give it instructions and really guide it on what you want it to do.
Jared Friedman 30:03 Okay, so let me see if I, if I, if I understand this right. Like historically, the trans- transcription models, like the ones that you guys built, d- only do transcription. They don't... They, they lack the general in- like the, the general intelligence of a, of a, of an LLM. And so like the magical combination came when you paired a transcription model with an LLM. What this new model, it's kind of like the fusion of the two of them. It is a transcription model, but you've actually injected the like generalized intelligence of an LLM into it.
Dylan Fox 30:30 Yeah. Mor- more or less, exactly. Yeah.
Jared Friedman 30:33 Okay.
Dylan Fox 30:33 Yeah. 'Cause we-If you think about a multimodal general purpose LLM, it can do transcription tasks, but that's maybe 10% of its total training data, right?
Jared Friedman 30:43 Mm.
Dylan Fox 30:43 So it also can do math tasks, and for, um, our customers, they need a reliable voice AI model that can understand speech and speakers. And so we really focus the model on those types of tasks.
Jared Friedman 30:57 Yeah.
Dylan Fox 30:57 And, um, it doesn't get confused and think it's like a- an assistant. [laughs]
Jared Friedman 31:03 [laughs]
Dylan Fox 31:03 It, it will, it will kinda operate as this like... as this kind of i- i- in this like narrow space.
Jared Friedman 31:09 And I assume it's also much smaller and faster and cheaper-
Dylan Fox 31:12 Yeah, correct
Jared Friedman 31:12 ... to run than a full LLM, where you're, like, taking this giant brain and having it only do this one little thing, right?
Dylan Fox 31:17 Yeah.
Jared Friedman 31:17 Yeah.
Dylan Fox 31:17 You can run it in real time. Um, you can actually deploy it on your own server, so we support self hosting these models-
Jared Friedman 31:25 Mm-hmm
Dylan Fox 31:25 ... uh, to get latency down. And where we're really excited is for customers to use this in, um, their real-time voice agents, because you can just capture a lot more, uh, a lot more information from the speaker.
Jared Friedman 31:37 Can we see a demo?
Dylan Fox 31:38 Yeah, definitely. So here's, here's a demo of Universal 3 Pro. Mm-hmm. Mm-hmm. Uh-huh. Mm-hmm. And you can see it gets pretty much all the, the, like, stutters that I, I, I, um... and the, the noises that I made with really low latency. And [laughs] you can even see that here as I was stuttering and talking here.
Jared Friedman 32:03 [laughs]
Dylan Fox 32:03 It's super verbatim at what it captures. Um, another example that a lot of real-time models struggle with are complex terms like emails, uh, long strings of digits, and so I'll give you an example of how well this model can do there.
Jared Friedman 32:16 Which makes sense. Like, if I'm calling a call center, one of the most common things that they're gonna do is ask me for my email address-
Dylan Fox 32:21 What's your email?
Jared Friedman 32:22 ... my address-
Dylan Fox 32:22 Yeah
Jared Friedman 32:22 ... my name. Yeah.
Dylan Fox 32:24 And so I'll give you an example of how well this can capture that. Yeah, I'll read you my email. It's 726_dylan@hotmail.com. Sorry, that was the wrong one. It's xyz14_x28@gmail.com.
Jared Friedman 32:46 Oh, wow.
Dylan Fox 32:47 Yeah.
Jared Friedman 32:47 It, it, like, completely nailed it. Like, actually much better than a human.
Dylan Fox 32:51 Yeah. It gets-
Jared Friedman 32:52 [clears throat]
Dylan Fox 32:52 ... you know, complex strings of alphanumerics really well, even if I do things like... I'll do another one. AAABBBCC1112X56.
Jared Friedman 33:03 Okay, so if I'm calling in and I'm reading, like, a, like a, like an airplane, like, a ticket number or something like that-
Dylan Fox 33:11 Yeah
Jared Friedman 33:11 ... it's just gonna be able to, like, totally nail a string of alphanumerics like that.
Dylan Fox 33:14 Yeah. And you can... It's, it's even, like, super robust the, um, to, to really challenging audio conditions. So I'll show you something that's cool. If I whisper to it, it still will get it pretty well.
Jared Friedman 33:23 Okay.
Dylan Fox 33:25 Yeah, my email address is Dylan, D-Y-L-A-N, @assemblyai.com. Uh, almost. [laughs] I was whispering pretty low.
Jared Friedman 33:36 [laughs]
Dylan Fox 33:36 Uh, it got assemblyai.com.
Jared Friedman 33:37 Wow.
Dylan Fox 33:37 Now what-
Jared Friedman 33:38 That's impressive
Dylan Fox 33:38 ... is cool, um, is, you know, a lot of times you see errors like this and y- you know, it's like you have no control. Uh, but you can actually prompt this model to f- to, to be more accurate in these types of conditions. And so, um, I'll switch over here and give you an example. Um, so I can tell the model, let's say I, I want it to convert all the speech it's hearing into Spanish. I can prompt the model. I can say-
Jared Friedman 34:00 Ooh
Dylan Fox 34:00 ... "Transcribe this audio and translate everything you hear into Spanish." And I'll record some audio for this example. Hi, my name is Dylan and I live in New York City. So you can see, um-
Jared Friedman 34:28 Yeah
Dylan Fox 34:28 ... it transcribed me, but it transcribed me in Spanish because I, I told it, "Here, I want you to transcribe everything in Spanish." Um, so yeah.
Jared Friedman 34:35 So, so, so this model, it, it's, it's effectively like speaks many languages. How many languages?
Dylan Fox 34:42 Yeah. So this is a multilingual model. Um, right now it supports seven or eight languages-
Jared Friedman 34:46 Okay
Dylan Fox 34:46 ... and then we're gonna add an, a couple dozen, um, over the next couple of, of weeks. Yeah.
Jared Friedman 34:52 How have people been using this promptable configuration since you launched it? Or is it too early to say? It's only, like, a week out.
Dylan Fox 34:59 You know, I think that what we found over the last couple years is that every application has different requirements. And so what this model gives developers the ability to do is, um, explain those requirements to the model and then it will follow them. So I'll... Let me just show you an example of what I mean by that. Um, so this is an audio. Um, I'll play this here. I hope you got our card. We got quite a pull strap here. Okay, nobody talk. We'll just wait for her to talk. Well, we just wanted to- I can feel a bear here. Damn it. So, um-
Jared Friedman 35:36 [laughs]
Dylan Fox 35:36 Let me, let me throw this audio in and just show you out of the box what, what the behavior will be.
Jared Friedman 35:41 Okay.
Dylan Fox 35:46 So you can see, um- I hope you got our card. We got quite a pull strap- It, it, it's missing the background speaker.
Jared Friedman 35:52 Mm.
Dylan Fox 35:52 Right? Um, if you, if you want the model... So, so some developers don't want the background speaker to be captured.
Jared Friedman 35:59 Right.
Dylan Fox 35:59 Um, for example, if you're a robot and you just wanna focus on the primary speaker, you can have the model do that. But in this example-
Jared Friedman 36:06 Interesting
Dylan Fox 36:06 ... let's say you do wanna pick up the background speaker, um, you can do cool things like what I'll show you here. Um, I need to actually just copy this prompt out. S- so... So, um, what... Let's say you wanna pick up the background speech, but you don't actually wanna transcribe it. You just wanna know that there's some background speechUm, here's a prompt we'll give. The model will say when multiple speakers talk simultaneously, mark the cross-talk segments. And then we'll run that, that file through again. And you can see now the model's identifying the cross-talk from the background speaker.
Jared Friedman 37:03 Mm-hmm.
Dylan Fox 37:03 And, um, it's not transcribing it because we told it to just mark those cross-talk segments.
Jared Friedman 37:08 Okay.
Dylan Fox 37:09 But if you wanted to have it actually transcribe that background speech, you could tell it and transcribe that background speech. So you have this level of controllability over-
Jared Friedman 37:17 Yeah
Dylan Fox 37:17 ... how the model works. And, um, again, because it's not this giant multimodal LLM, it's something in the middle, um, it's, it's gonna behave like a STT or SDT-like model. Um, so it can do really good multilingual code switching, it can do speaker identification tasks, and this is just the first version. We're, we're already training new ones that are gonna come out and be a little bit better at instruction following, new capabilities in just a couple weeks even.
Jared Friedman 37:46 This is super cool.
Dylan Fox 37:47 Yeah.
Jared Friedman 37:48 How did, how did you train it? And how, how did you get it to be so accurate at phone numbers [laughs] and email addresses?
Dylan Fox 37:53 [laughs] Yeah. The, the, the phone numbers, the email addresses are really cool, um, the accuracy there. So we have spent years working with customers building these types of applications and building up this like, deep subject matter expertise into, um, what are the failure modes, what are the things that really matter? And really, most of our time has been spent on, um, coming up with really good post-training techniques to make the model, uh, perform really well in these, in these traditionally challenging environments. So for example, when you're saying mm-hmm or no or yes, phone numbers, a lot of times that's how you're talking to a, a voice agent today-
Jared Friedman 38:37 Mm
Dylan Fox 38:37 ... when you're trying to complete some action. Um, and we spend a lot of time on the post-training on, on how we're training the model to make it really good on, on those types of tasks.
Jared Friedman 38:47 So there's another interesting part of the AssemblyAI story to me, which is back in 2017 when you started this, in addition to voice AI being a bad market where it was like, really small and nobody really had anything working, it was also like, a hyper-competitive market where you were competing with Google and Nuance, which people probably don't remember, but was like a giant in the space. And I, I forget who else was around in 2017, but I'm curious what that was like.
Dylan Fox 39:11 Yeah.
Jared Friedman 39:11 And like, were you scared of those competitors? Did you like... Yeah, were, were, were investors like, skeptical of investing because you had all these like, giant competitors?
Dylan Fox 39:20 Yeah. [laughs]
Jared Friedman 39:21 [laughs]
Dylan Fox 39:21 It... They were definitely skeptical of investing. Still, I would say skeptical. [laughs]
Jared Friedman 39:25 [laughs]
Dylan Fox 39:25 Uh, but yeah. Yeah, definitely. I think it, like i- it goes back to like, I was a user of those products, and I saw they weren't good enough. Um, and I think that, you know, that to me, back then and still gives me conviction over our ability to win in this space because we have, um, we have the subject matter expertise over the problem space, what customers want, that gives us the ability to build the best product. And I think that in general, that's kind of been something I've learned. Like, um, like, yes, you need capital and you need team and resources, like, for sure, but you also need to have subject matter expertise, which comes from like, being very close to the problem. And there's this example, um, that I, I think about, which is like, the Wright brothers. The Wright brothers, in modern day time, uh, self-funded their, their, um, their, the plane development for like, with like $40,000 out of their own pocket, and they were competing with Samuel Langley, which, um, was funded by the Smithsonian, like, you know, I think 20X the amount of funding or something. Um, and the Wright brothers won, and I, a big reason for, for my takeaway is like, they had such better subject matter expertise over the problem because they in o- in one year had, um, 1,000 test flights-
Jared Friedman 40:51 Hmm
Dylan Fox 40:52 ... where they personally would do the [laughs] test flights.
Jared Friedman 40:54 Hmm.
Dylan Fox 40:54 And they would understand-
Jared Friedman 40:55 Hmm
Dylan Fox 40:55 ... what's working, what's not, what do they need to go iterate on. Whereas Samuel Langley, I think he had like, seven, and he didn't even do them himself. He-
Jared Friedman 41:04 He had his minions do those. [laughs]
Dylan Fox 41:05 He had his team do it. And, and not a, a, a dumb person, right? Like, smart guy, but-
Jared Friedman 41:10 Yeah
Dylan Fox 41:10 ... missed the subject matter expertise that the Wright brothers had.
Jared Friedman 41:13 Subject matter.
Dylan Fox 41:13 And so for me, I think any company to win, you need to have like, deep subject matter expertise over, like, what are you trying to, to, um, to go build and do and, and what kyp- type of product you're trying to make. Um, and so back then and now, I think that, uh, what gives us the ability to build the best product and to win is that we're a, a really dense team of subject matter experts. We're so close with customers. We have thousands of customers in Slack [laughs] with us.
Jared Friedman 41:46 Yeah.
Dylan Fox 41:46 We, uh, like, our, our Slack is a fire hose of customer feedback, and, and we don't filter it to just the good stuff. Like-
Jared Friedman 41:56 Hmm
Dylan Fox 41:56 ... we have a, a, a big product team channel with like 80 people in it. We got some pretty bad feedback, you know, a week or two ago, and it's just pasted in there.
Jared Friedman 42:05 [laughs]
Dylan Fox 42:06 [laughs] And, and everyone's like, "Yeah, that's good feedback."
Jared Friedman 42:08 Hmm.
Dylan Fox 42:08 "Let's go fix that thing."
Jared Friedman 42:09 Hmm.
Dylan Fox 42:09 And so that's part of our culture of like, we're just trying to build the best thing. Um, and I, I think that is, back then, what gave us, you know, an advantage and now, um, what still gives us the ability to, to, to build a better product.
Jared Friedman 42:24 Did, did that also play out on the technology side where like, Google had unlimited money to like, throw compute at this problem, but did they end up basically like, squandering it like how Langley did? And have you guys been able to be smarter about how you do it because you're so much closer to the problem and the customers?
Dylan Fox 42:39 I don't know what they're, like Google or others, are doing internally, right? I know it's like super smart teams. I think that we-Just care about this problem space much more
Jared Friedman 42:47 Mm-hmm
Dylan Fox 42:48 And I would say the people on our team are probably, like, the most experienced people at developing and building real world voice AI systems in the world [laughs]
Jared Friedman 43:04 [laughs]
Dylan Fox 43:04 Because we've been so deep in this. We have some of the- the most scale coming through our- our API platform, like, in the market, and we get to learn from all of that, from customer feedback, good and bad, and from knowing where are people trying to take the technology. And so, um, we have a brilliant team, but we have deep subject matter expertise within that team, um, not just, like, functional expertise. And I- I think that's usually the missing ingredient in a big company. You know, the- the people leading the projects are so far removed from the customer feedback, and- and s- a lot of times haven't even tried the product themselves.
Jared Friedman 43:45 Yeah. [laughs]
Dylan Fox 43:45 You know? Like, it's remarkable.
Jared Friedman 43:47 [laughs]
Dylan Fox 43:47 Um, [laughs] and as soon as you get hands-on with your product, you realize, like, "Oh, it's bad there. It's bad there," right? And so, um, a big thing we have, a big part of our culture at the company is, like, everyone's hands-on with the product. We're all testing it. We're all trying it. We're all building stuff with it. Um, and so you really quickly see, like, where it's good and where it's not. So, like, the email demo I gave you, like, we- we- we really, um, um, understand, like, that as a problem and are- are making sure that that works really well. So we're operating really close to the metal and close to customers, and I think that's- that's what helps us, um, out execute bigger, you know, more well-funded teams.
Jared Friedman 44:27 So the team was, like, really small through 2021. Was it just, like, a handful of people?
Dylan Fox 44:31 A handful of people, yeah.
Jared Friedman 44:32 A handful of people. And then in 2021, you kind of entered this, like, hypergrowth mode.
Dylan Fox 44:35 Yeah.
Jared Friedman 44:36 Um-
Dylan Fox 44:36 Raised a bunch of capital. Went into this hypergrowth mode, yeah.
Jared Friedman 44:39 Yeah. What's- what's the team like today? And then, like, what was that journey like going from, like, basically not growing to, like, hypergrowth-
Dylan Fox 44:48 Yeah
Jared Friedman 44:48 ... all of a sudden? Were there, uh, some, uh, uh, lessons learned along the way?
Dylan Fox 44:56 Yeah. Yeah, totally. I think that we- we raised a bunch of capital and then, um, we started hyperscaling and growing the team and building the team really aggressively. And I think probably, um, one- one of the- the bigger mistakes we made at that time was, like, just not having clear enough conviction over, like, where do we wanna invest versus where are we just exploring. Um, and so vers- for example, like, now I- I try to really avoid hiring someone to explore an area versus, like, let's, with our existing team, go explore it, and if it's working well, then we can go invest in that area
Lessons from hypergrowth
Dylan Fox 45:33 and hire more people into that. But I think a lot of companies y- you know, will follow the traditional playbook of, like, "Okay, this company, which is similar to mine, did X and Y and Z, therefore, like, I should do X and Y and Z, and I should go build out a, you know, 50-person SDR team. And I should go, you know, I need this and that," as if it's, like, a franchise business.
Jared Friedman 45:54 Mm-hmm.
Dylan Fox 45:54 Um, but I think every company's different. And so knowing what to invest in, what teams to build, um, especially in an emerging market like the one we're in, is- is- is difficult. And, um, now we're more conservative over, like, hey, where do we really wanna invest versus where is this just an, you know, something- s- an area we wanna explore? And if it's exploration, let's do that with the existing team. Um, and then also just, like, really just strongly filtering for, um, non-negotiables and- and culture fits. I think-
Jared Friedman 46:29 Like what?
Dylan Fox 46:30 So every role, I think, has different non-negotiables. Um, but then you start talking to people and you're like, "Oh, I like this person," and, you know, we're- we're vibing and so, "Yeah, let's hire them." But then you realize, oh, I really, you know... This thing was really important in the role, and I should've been firmer in that- on that non-negotiable. So, like, now we're very clear on, all right, for these, for this role, these are the non-negotiables. For this role, these are the non-negotiables. For me as a founder, one thing that's really important when hiring is, um, why do you wanna work at a voice AI company? Like, why here? Um, not just an AI company, not just a VC-backed company, not just a company that's growing fast, but, like, what about our product? What about our space, our market, our customers are y- are you passionate about? Um, because I think that, um, that- that's a really important ingredient, right? Like, there's- there's challenging problems-
Jared Friedman 47:26 Mm-hmm
Dylan Fox 47:26 ... here that, you know, in any company there's a lot of challenging problems. But I think that, you know, what I care a lot about is- is- is finding people that are really excited about our product and what our customers are building, um, and- and will be a lot more, uh, you know, obsessed and passionate as a result. And so that's something that I think I've, I also, um, am a lot more, like, strict on when we're hiring people now. So there's all these, like, things that I've- I've- I've learned, and I think that- that was probably, um, one of the harder lessons to learn. But now we're- we're a- about 80 people at the company, um, super dense team.
Jared Friedman 48:09 Only 80 people?
Dylan Fox 48:10 Yeah.
Jared Friedman 48:10 That's actually a lot smaller than I would've expected given the- the 700 billion hours of audio [laughs] that you're processing every year.
Dylan Fox 48:16 Yeah, yeah. Super, super small, dense team. I mean, we're, like, you know, using AI everywhere across the company-
Jared Friedman 48:23 Mm-hmm
Dylan Fox 48:24 ... um, to just be, like, to move as fast as possible. And I- I think, you know, one thing we talk about at the company is we're gonna optimize for speed and innovation. Like, that's gonna be an explicit decision. So we don't do, like, OKR cascades, and we don't do planning exercises. Like, we have m- clear metrics that we wanna hit, and those are very transparent across the whole company. And then we're all just running towards those, and we can move quickly and innovate and be dynamic and responsive because we're this small, transparent team. Um, and I- I think that my goal is, like...Having as little operational overhead as possible at all times, like teetering like just on the, the, the line of like complete chaos. Like, [laughs] like I wanna be like right there.
Jared Friedman 49:15 [laughs]
Dylan Fox 49:16 Um, 'cause we've, you know, we've like oscillated, right? Like, it feels good to have like, here's every team's OKRs, and here's every team's six-month roadmap, and here's every team's-
Jared Friedman 49:25 Yeah
Dylan Fox 49:25 ... like strategy doc. And-
Jared Friedman 49:26 Like Microsoft style.
Dylan Fox 49:27 Yeah. It's like-
Jared Friedman 49:27 Yeah
Dylan Fox 49:27 ... here everything is like planned and organized, and it's like that, that feels good.
Jared Friedman 49:31 Yeah.
Dylan Fox 49:31 But like that's not a replacement for like you actually gotta go build the thing and sh- and do what's in that document.
Jared Friedman 49:37 Yeah.
Dylan Fox 49:37 And then what if two months in you get new information that you should adapt to?
Jared Friedman 49:43 Right.
Dylan Fox 49:43 If you're committed to some plan, you're like, "Oh, we'll do that next quarter."
Jared Friedman 49:46 Yeah.
Dylan Fox 49:46 It's like, well, we should be doing that right now if that's-
Jared Friedman 49:48 Yeah
Dylan Fox 49:48 ... the most important thing.
Jared Friedman 49:49 Yeah.
Dylan Fox 49:49 And so we really now operate the company, um, as small as we can, as lean as we can 'cause I think that gives us the ability to innovate and move quickly. Um, and that's really like what's it, what it, it is in, in service of. So now we are, yeah, we're, what, you know, we have about like 10 open roles at the company right now, um-
Jared Friedman 50:10 Who are you hiring?
Dylan Fox 50:11 Across, uh, forward deployed engineers, software engineers, research engineers. Um, um, uh, uh, on our marketing team we have open roles. So across like pretty much all departments we have open roles, but we really look for people that, you know, wanna work at a company like how we operate, um, are excited about our product, our space. And, and that's not for everyone, and that's totally fine. But for the people that wanna operate in like a, you know, an environment like the one we're building, um, you know, it's amazing, and we wanna, we wanna go find those people.
Jared Friedman 50:47 Okay, this is random, but I'm curious. Are you guys living in the future that you yourselves are creating? Like, does everybody talk to their computers all day long instead of typing? Do you record all your meetings and transcribe all of them?
Dylan Fox 50:59 Yeah.
Jared Friedman 50:59 Do your coffee makers talk?
Dylan Fox 51:00 [laughs] So, so yeah, we are. I mean, pretty much every meeting we have an AI note-taker in there, and it's not just to take notes, right? It's we're building this, this knowledge base that's public at the company around customer feedback, sales conversations. I could go in right now and ask this knowledge base, "What should our product roadmap be-
Jared Friedman 51:23 Hmm
Dylan Fox 51:24 ... based on customer feedback?"
Jared Friedman 51:24 And that's, this has access to all the transcripts of all the meetings.
Dylan Fox 51:27 Mm-hmm. And, and it's gonna tell you something that's perfect.
Jared Friedman 51:30 Wow.
Dylan Fox 51:30 That's like, "Hey, uh, this is the number one thing. This is the number two thing." Um, and everyone at the company has access to, to that information, which is amazing because now if you're an engineer on this product, there's not like layers between the customer and you. It's like you actually have this fire hose of customer feedback, everything from what are people saying on Reddit and, you know, X, to customer conversations that we're having, meetings to, you know, support tickets we're
The future of voice AI
Dylan Fox 52:00 getting. Like-
Jared Friedman 52:00 Hmm
Dylan Fox 52:00 ... all that is brought in-
Jared Friedman 52:01 Hmm
Dylan Fox 52:02 ... to the company. Th- there's like all the like truth out there, right? Of like your product and feedback in the market and stuff, right? Um, and I think it's easy to like put a subjective point of view on that, right? And usually it's like the people in charge that get to decide what, what the subjective point of view is.
Jared Friedman 52:21 Yeah. That's fair.
Dylan Fox 52:21 But if, if you have all that, all that truth just like publicly available, it's like e- everyone is looking at this information objectively, and there is no subjective layer. Um, and I think that's been so powerful for us as a company, and I think that that's how like most companies should and will... The companies that are g- successful I think will operate in the same way.
Jared Friedman 52:43 Yeah, I mean, it's, it's just essential. Like the, the, the, the companies that operate without AI augmentation of human intelligence will lose to the ones that have augmented their intelligence.
Dylan Fox 52:51 Totally. Yeah. [upbeat music]
Episode duration: 53:00
In the uListen app, the transcript lights up word by word as you listen. Get uListen
Transcript of episode Elm2p_TRPwk