Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future | Dianne Penn (1h 34m)

← Back to week overview

• Release date: 2026-07-26
• Listen on Spotify: Open episode
• Episode description:

  Dianne Penn is Head of Product for Anthropic’s AI Research and Labs teams. She joined in 2023 as Anthropic’s first technical product manager, when the entire product team was five engineers, and has since helped ship every model from Claude 2 through Fable, and helped incubate Claude Code, MCP, Skills, computer use, tool use, and reasoning. Before Anthropic, she helped build Alexa’s AI at Amazon and, before that, traded high-yield bonds at JP Morgan Chase.In our in-depth conversation, we discuss:What Anthropic’s early days were likeThe inflection points that turned Anthropic from an underdog into the fastest-growing company in historyHow exactly Claude got so good at codingThe eval-driven development loop her team is pioneeringHow to find joy in AI when everything is moving this fastWhy Claude’s willingness to push back is key to its successWhere human judgment remains irreplaceable—Brought to you by:WorkOS—Make your app enterprise-ready, with SSO, SCIM, RBAC, and moreMercury—Radically different banking, now with Command—Episode transcript: https://www.lennysnewsletter.com/p/anthropics-first-technical-pm-on—Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0—Where to find Dianne Penn:• LinkedIn: linkedin.com/in/dianne-na-penn—Where to find Lenny:• Newsletter: https://www.lennysnewsletter.com• X: https://twitter.com/lennysan• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/—In this episode, we cover:(00:00) Introduction(02:31) Early Anthropic days(08:55) Big milestones(13:50) Inside the exponential(20:02) Token maxing(23:30) Anthropic Labs and the incubation model(27:30) How the research role works(31:35) How to become a top researcher(35:18) Frontier model safeguards(39:38) Hiring in the AI era(44:16) Building an eval set(47:48) Evals vs PRDs(49:55) The importance of hands-on leadership(52:46) Finding joy in AI(58:10) How Dianne uses Claude(01:01:05) Avoiding overreliance on AI(01:03:50) The constitution that makes Claude better(01:07:11) AI writing and verification(01:11:40) Where human brains will continue to be valuable(01:14:10) Navigating AI with kids(01:16:26) Alignment, the future of the PM role, and burnout(01:21:54) Lightning round and final thoughts—Referenced:• Anthropic: https://www.anthropic.com• Golden Gate Claude: https://www.anthropic.com/news/golden-gate-claude• Dario Amodei’s website: https://darioamodei.com• Scaling Laws and Interpretability of Learning from Repeated Data: https://www.anthropic.com/research/scaling-laws-and-interpretability-of-learning-from-repeated-data• Tokenmaxxing: How Top Builders Use AI To Do The Work Of 400 Engineers: https://www.ycombinator.com/library/Pa-tokenmaxxing-how-top-builders-use-ai-to-do-the-work-of-400-engineers• Garry Tan on X: https://x.com/garrytan• Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann: https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann• Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next• Introducing Labs: https://www.anthropic.com/news/introducing-anthropic-labs• Louis CK | about airplane Wi Fi: https://www.youtube.com/watch?v=me4BZBsHwZs• What happens after coding is solved? | Fiona Fung (Manager of the Claude Code and Cowork Teams): https://www.lennysnewsletter.com/p/building-the-most-ai-pilled-engineering...References continued at: https://www.lennysnewsletter.com/p/anthropics-first-technical-pm-on⁠—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.—Lenny may be an investor in the companies discussed.

Summary

• 🧪 Evals Are the New PRDs: Product teams at Anthropic now define user value through rigorous evals rather than traditional product requirement documents, turning user feedback into measurable model improvements.
• 💻 Coding Became the Killer App: Focusing training on long-form coding rather than autocomplete helped Anthropic differentiate early and created a powerful flywheel with Claude Code.
• 🚀 Labs Incubates Discontinuous Bets: Small, autonomous pods inside Anthropic pursue high-ambition prototypes that later become core products, enabled by bottoms-up culture and rapid model iteration.
• 🧠 Hands-On Model Fluency Required: Even senior leaders must personally experiment and ship with frontier models to develop the taste and judgment needed for fast-moving AI product decisions.
• 🔄 Exponential Pace Demands Adaptability: Because capability jumps are unpredictable, teams succeed by staying adaptable, thinking from first principles, and constantly asking what changes when the next model arrives.

Insights

• What does it take for a small AI lab to overtake early leaders by obsessively focusing on a single high-value use case like long-form coding?
  • Time: 0:14 – 0:47
  • Answer: Anthropic went from being seen as having ‘no chance’ against OpenAI to shipping frontier models by identifying that users wanted full coding agents, not just autocomplete, then training Opus specifically for that. The combination of model improvements and Claude Code created a flywheel that accelerated adoption.
• How are product managers redefining their craft when evals replace traditional PRDs as the core way to capture and deliver user value in AI products?
  • Time: 0:53 – 1:18
  • Answer: Dianne Penn describes how her team at Anthropic shifted from writing PRDs to building evals that translate user feedback into measurable model improvements, especially for capabilities like tool use and coding. This reflects a broader change in how PMs create impact when working directly with frontier models.
• How can internal ‘Labs’ teams within AI companies systematically surface 10x–1000x opportunities that the core product roadmap would miss?
• Why must product leaders stay deeply hands-on with the models themselves rather than delegating experimentation to their teams?

Want the full treatment? Download expanded insights—timestamps, complete answers, and course materials from AI Weekly Insights.