By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.
No PriorsNo Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava
Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.
At a glance
WHAT IT’S REALLY ABOUT
Baseten CEO Explains Fast AI Inference, Infrastructure, And Enterprise Adoption
- Baseten CEO Tuhin Srivastava discusses how his company provides fast, scalable AI inference infrastructure for teams deploying large models, emphasizing "efficient code" over no-code abstractions. He contrasts training vs. inference workloads, and explains why inference is more repeatable, SLA-driven, and reliability-sensitive. The conversation covers performance optimization (e.g., speculative decoding, TRT-LLM, continuous batching), GPU supply dynamics, and how customers move from shared endpoints to dedicated and self-hosted deployments. They also explore how AI will change enterprise software economics, build-vs-buy decisions, and what defensibility looks like in rapidly growing AI markets.
IDEAS WORTH REMEMBERING
5 ideasSpeed is the primary competitive advantage in early AI markets.
Teams that move fastest from idea to reliable deployment gain a significant edge, which pushes many to buy infrastructure instead of building it, so they can focus on proprietary models, data, and workflows.
Inference has distinct requirements from training and is more repeatable across customers.
Inference workloads demand low latency, high reliability, versioning, and CI/CD integration; these patterns recur across teams, making them well-suited for specialized platforms like Baseten.
Performance optimization is now existential for inference providers.
Staying near the state of the art in throughput and latency (via speculative decoding, batching, and engines like NVIDIA’s TRT-LLM) is critical, as users are highly sensitive to speed and open-source models can be made very fast.
Customers typically progress from shared endpoints to dedicated and then self-hosted deployments.
As scale, privacy needs, and SLA expectations grow, teams outgrow shared inference endpoints and either move to dedicated clusters or host models inside their own cloud accounts to gain control and reduce costs.
AI-native businesses may trade headcount for high ongoing compute spend.
For many Baseten customers, inference is the second-largest expense after payroll, yet the businesses can still be highly valuable due to leverage, markups, and software optimization that preserve healthy margins.
WORDS WORTH SAVING
5 quotesWe’re not no-code, we’re efficient code — strong abstractions that make the easy things super easy, but still make the hard things possible.
— Tuhin Srivastava
Speed is actually your number one advantage in AI right now. If you’re not competing on speed, you’re going to be left behind.
— Tuhin Srivastava
What is proprietary to them is models, data, and workflow. What is repeated for them is infrastructure.
— Tuhin Srivastava
We had a four-person AI infra team that had been building for two years migrate all their workloads to Baseten in 36 hours.
— Tuhin Srivastava
I think we’re overestimating how big enterprise will get in the next 12 to 18 months, but underestimating where it will be in three to five years.
— Tuhin Srivastava
High quality AI-generated summary created from speaker-labeled transcript.
