Lenny's Newsletter
Lenny's Podcast: Product | Career | Growth
Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google & Amazon
Preview
0:00
-1:26:22

Why most AI products fail: Lessons from 50+ AI deployments at OpenAI, Google & Amazon

Aishwarya Naresh Reganti and Kiriti Badam have helped build and launch more than 50 enterprise AI products across companies like OpenAI, Google, Amazon, and Databricks. Based on these experiences, they’ve developed a small set of best practices for building and scaling successful AI products. The goal of this conversation is to save you and your team a lot of pain and suffering.

We discuss:

  1. Two key ways AI products differ from traditional software, and why that fundamentally changes how they should be built

  2. Common patterns and anti-patterns in companies that build strong AI products versus those that struggle

  3. A framework they developed from real-world experience to iteratively build AI products that create a flywheel of improvement

  4. Why obsessing about customer trust and reliability is an underrated driver of successful AI products

  5. Why evals aren’t a cure-all, and the most common misconceptions people have about them

  6. The skills that matter most for builders in the AI era


Brought to you by:

Merge—The fastest way to ship 220+ integrations

Strella—The AI-powered customer research platform

Brex—The banking solution for startups

Get 15% off Aishwarya and Kiriti’s Maven course, Building Agentic AI Applications with a Problem-First Approach, using this link: https://bit.ly/3V5XJFp

Where to find Aishwarya Naresh Reganti:

• LinkedIn: https://www.linkedin.com/in/areganti

• GitHub: https://github.com/aishwaryanr/awesome-generative-ai-guide

• X: https://x.com/aish_reganti

Where to find Kiriti Badam:

• LinkedIn: https://www.linkedin.com/in/sai-kiriti-badam

• X: https://x.com/kiritibadam

Referenced:

• LevelUp Labs: https://levelup-labs.ai/

• Why your AI product needs a different development lifecycle: https://www.lennysnewsletter.com/p/why-your-ai-product-needs-a-different

• Booking.com: https://www.booking.com

• Research paper on agents in production (by Matei Zaharia’s lab): https://arxiv.org/pdf/2512.04123

• Matei Zaharia’s research on Google Scholar: https://scholar.google.com/citations?user=I1EvjZsAAAAJ&hl=en

• The coming AI security crisis (and what to do about it) | Sander Schulhoff: https://www.lennysnewsletter.com/p/the-coming-ai-security-crisis

• Gajen Kandiah on LinkedIn: https://www.linkedin.com/in/gajenkandiah

• Rackspace: https://www.rackspace.com

• The AI-native startup: 5 products, 7-figure revenue, 100% AI-written code | Dan Shipper (co-founder/CEO of Every): https://www.lennysnewsletter.com/p/inside-every-dan-shipper

• Semantic Diffusion: https://martinfowler.com/bliki/SemanticDiffusion.html

• LMArena: https://lmarena.ai

• Artificial Analysis: https://artificialanalysis.ai/leaderboards/providers

• Why humans are AI’s biggest bottleneck (and what’s coming in 2026) | Alexander Embiricos (OpenAI Codex Product Lead): https://www.lennysnewsletter.com/p/why-humans-are-ais-biggest-bottleneck

• Airline held liable for its chatbot giving passenger bad advice—what this means for travellers: https://www.bbc.com/travel/article/20240222-air-canada-chatbot-misinformation-what-travellers-should-know

• Demis Hassabis on LinkedIn: https://www.linkedin.com/in/demishassabis

• We replaced our sales team with 20 AI agents—here’s what happened | Jason Lemkin (SaaStr): https://www.lennysnewsletter.com/p/we-replaced-our-sales-team-with-20-ai-agents

• Socrates’s quote: https://en.wikipedia.org/wiki/The_unexamined_life_is_not_worth_living

• Noah Smith’s newsletter: https://www.noahpinion.blog

Silicon Valley on HBO Max: https://www.hbomax.com/shows/silicon-valley/b4583939-e39f-4b5c-822d-5b6cc186172d

• Clair Obscur: Expedition 33: https://store.steampowered.com/app/1903340/Clair_Obscur_Expedition_33/

• Wisprflow: https://wisprflow.ai

• Raycast: https://www.raycast.com

• Steve Jobs’s quote: https://www.goodreads.com/quotes/463176-you-can-t-connect-the-dots-looking-forward-you-can-only

Recommended books:

 When Breath Becomes Air: https://www.amazon.com/When-Breath-Becomes-Paul-Kalanithi/dp/081298840X

The Three-Body Problem: https://www.amazon.com/Three-Body-Problem-Cixin-Liu/dp/0765382032

A Fire Upon the Deep: https://www.amazon.com/Fire-Upon-Deep-Zones-Thought/dp/0812515285


Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

Lenny may be an investor in the companies discussed.


My biggest takeaways from this conversation:

This post is for paid subscribers