Baseten CEO Tuhin Srivastava on Custom Models, and Building the Inference Cloud

Baseten CEO on custom models, scaling inference, and compute constraints

1. A.Baseten’s growth is driven by the rapid expansion of the application layer and the mainstreaming of post-training/RL techniques that let companies “own” and specialize inference.
2. B.Most production inference on Baseten is custom (about 90–95% of tokens), with customers modifying, compiling, and optimizing open-source weights for both quality and performance rather than running vanilla models.
3. C.GPU supply is extremely tight with minimal slack, pushing providers toward multi-year contracts with significant prepay and making operational reliability and access to quality suppliers as critical as software.
4. D.Customers prioritize model capability first and cost second, using a mix of frontier closed models and increasingly strong open-source models (including Chinese-origin models), while navigating security and geopolitical considerations.
5. E.Baseten’s product direction emphasizes an end-to-end loop: inference produces data and eval signals, which feeds post-training, which in turn drives more inference—plus runtime innovations like cache-aware routing, prefill/decode separation, and agent sandboxes.