Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google & Amazon

We worked on a guest post together. They had this really key insight that building AI products is very different from building non-AI products.

Most people tend to ignore the non-determinism. You don't know how the user might behave with your product, and you also don't know how the LLM might respond to that. The second difference is the agency control trade-off. Every time you hand over decision-making capabilities to agentic systems, you're kind of relinquishing some amount of control on your end.

This significantly changes the way you should be building product.

So we recommend building step by step. When you start small, it forces you to think about, what is the problem that I'm gonna solve? In all this advancements of the AI, one easy slippery slope is to keep thinking about complexities of the solution and forget the problem that you're trying to solve.

It's not about being the first company to have an agent among your competitors. It's about, have you built the right flywheels in place so that you can improve over time?

What kind of ways of working do you see in companies that build AI products successfully?

I used to work with the CEO of now Rackspace. He would have this block every day in the morning, which would say, "Catching up with AI, four to six AM." Leaders have to get back to being hands-on. You must be comfortable with the fact that your intuitions might not be right, and you probably are the dumbest person in the room, and you want to learn from everyone.

What do you think the next year of AI is gonna look like?

Persistence is extremely valuable. Successful companies right now building in any new area, they are going through the pain of learning this, implementing this, and understanding what works and what doesn't work. Pain is the new moat.

[upbeat music] Today, my guests are Aishwarya Reganti and Kiriti Badam. Kiriti works on Codex at OpenAI and has spent the last decade building AI and ML infrastructure at Google and at Kumo. Ash was an early AI researcher at Alexa and Microsoft and has published over thirty-five research papers. Together, they've led and supported over fifty AI product deployments across companies like Amazon, Databricks, OpenAI, Google, and both startups and large enterprises. Together, they also teach the number one rated AI course on Maven, where they teach product leaders all of the key lessons they've learned about building successful AI products. The goal of this episode is to save you and your team a lot of pain and suffering and wasted time trying to build your AI product. Whether you are already struggling to make your product work or want to avoid that struggle, this episode is for you. If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube. It helps tremendously. And if you become an annual subscriber of my newsletter, you get a year free of a ton of incredible products, including a year free of Lovable, Replit, Bold, Gamma, n8n, Linear, Devon, PostHog, Superhuman, Descript, WhisperFlow, Perplexity, Warp, Granola, MagicPad, AndroidCast, Chapter, dmobit, and Stripe Atlas. Head on over to lennysnewsletter.com and click Product Pass. With that, I bring you Aishwarya Reganti and Kiriti Badam, after a short word from our sponsors. This episode is brought to you by Merge. Product leaders hate building integrations. They're messy, they're slow to build, they're a huge drain on your roadmap, and they're definitely not why you got into product in the first place. Lucky for you, Merge is obsessed with integrations. With a single API, B2B SaaS companies embed Merge into their product and ship two hundred and twenty-plus customer-facing integrations in weeks, not quarters. Think of Merge like Plat, but for everything B2B SaaS. Companies like Mistral AI, Ramp, and Drata use Merge to connect their customers' accounting, HR, ticketing, CRM, and file storage systems to power everything from automatic onboarding to AI-ready data pipelines. Even better, Merge now supports the secure deployment of connectors to AI agents with a new product, so that you can safely power AI workflows with real customer data. If your product needs customer data from dozens of systems, Merge is the fastest, safest way to get it. Book and attend a meeting at merge.dev/lenny, and they'll send you a fifty-dollar Amazon gift card. That's merge.dev/lenny. This episode is brought to you by Strella, the customer research platform built for the AI era. Here's the truth about user research: It's never been more important or more painful. Teams want to understand why customers do what they do, but recruiting users, running interviews, and analyzing insights takes weeks. By the time the results are in, the moment to act has passed. Strella changes that. It's the first platform that uses AI to run and analyze in-depth interviews automatically, bringing fast and continuous user research to every team. Strella's AI moderator asks real follow-up questions, probing deeper when answers are vague, and surfaces patterns across hundreds of conversations, all in a few hours, not weeks. Product, design, and research teams at companies like Amazon and Duolingo are already using Strella for Figma prototype testing, concept validation, and customer journey research, getting insights overnight instead of waiting for the next sprint. If your team wants to understand customers at the speed you ship products, try Strella. Run your next study at strella.io/lenny. That's S-T-R-E-L-L-A.io/lenny. [upbeat music] Ash and Kiriti, thank you so much for being here, and welcome to the podcast.