Computer and browser use in Codex (5 real examples)

Brought to you by:

Runway—The creative AI platform for images, video, and more

Hyperagent—Deploy fleets of agents that handle real work

Claire shows how she uses Codex to control her browser, test apps, manage LinkedIn, and even shop for her—plus the simple prompting trick that makes computer use work better.

Biggest takeaways:

1. AI testing can be far more exhaustive than human testing. When Claire tests her own onboarding flow, she naturally follows the happy path. She fills out every required field, clicks “next,” and never intentionally tries to break anything. Codex tested the flow as both a team and an individual, pushed on required-field edge cases, and immediately uncovered a blocking bug that had survived for months simply because Claire always completed the form correctly.

2. Frontier models often perform better when they are given room to think. When Claire first started using browser use, she would give the model a list of 25 things to test. Now she simply says, “QA the onboarding flow,” and lets it decide how to approach the task. The result is often broader coverage, with fewer blind spots introduced by her own assumptions about what matters.

3. Persona testing with browser use can reveal friction that synthetic user research misses. Claire’s husband, EJ, came up with the idea. Rather than asking AI to evaluate a product in the abstract, he suggested having it use the product as a specific person: a PM coming out of a meeting, an engineer picking up a PRD, or a team lead checking usage. In ChatPRD, this approach exposed a structural problem in the cross-thread reference flow. Claire already knew the issue existed, but she had never experienced it so clearly from the user’s perspective.

4. LinkedIn browser use is genuinely useful, but Claire initially used far more compute than the task required. She started by running the workflow on GPT-5.6 at high effort. After dropping to a medium-effort model, it could still work through unread messages, draft context-aware replies, and flag anything that needed a personal response. For anyone sitting on hundreds of LinkedIn messages without an official MCP or API connection, browser use is a practical solution.

5. Computer use can even operate an iPhone through screen mirroring. Claire was traveling out of state while the software she needed to manage her home Wi-Fi was installed on a phone back in California. She needed to open firewall ports so she could SSH into her Mac Minis remotely. Codex opened iPhone mirroring, updated the router settings, completed the SSH setup, and closed the ports again when it was finished. That would have been nearly impossible for her to do manually from a hotel room.

6. The model and effort level should match the job. Sorting through LinkedIn messages requires a very different level of reasoning than writing production code or conducting an exhaustive QA pass. Choosing the right amount of compute makes these workflows faster and cheaper. That becomes especially important when browser use is running throughout the day.

7. When a website blocks the agent, the human can step in briefly. During a Free People shopping session, the site flagged Codex as a bot and presented a CAPTCHA. Claire completed the verification, then handed control back. It is a useful division of labor: AI handles the tedious browsing and filtering, while the human takes care of the moments that require verified identity.

From zero coding background to hardware hacker: How Cursor + a Raspberry Pi makes AI fun

Brought to you by:

• Firecrawl—Power AI agents with clean web data

• Customer.io—Build customer engagement campaigns from a single prompt

Maddie Reese went from zero coding experience to building a working Twitter pager, an AI-powered receipt printer, and her own personal API. In this episode, she shares how Cursor and a Raspberry Pi helped her turn fun, weird ideas into real hardware.

Biggest takeaways:

1. You don’t need to understand code to build physical projects. Maddie compares her coding ability to knowing just enough Spanish to get by in San Diego. She can read parts of the code, spot an incorrect wire specification, and tell whether the AI is heading in the right direction. She is not writing functions from scratch. That level of literacy was still enough to ship three working hardware projects.

2. Start with the brainstorm. Before Maddie buys any components, she explains the full idea to Cursor and asks it to interview her. The back-and-forth continues until the major questions are resolved. Only then does Cursor generate a shopping list. On one project, it recommended the wrong wires. Maddie caught the mistake during her final review, avoiding both a wasted purchase and a week of troubleshooting.

3. Her hardware workflow follows a simple sequence: idea, interview, shopping list, buy, build. She has used the same process for the thermal printer, the pager, and her personal API. Each project began with a plain-language description of what she wanted. Cursor helped work out the technical requirements before she spent any money.

4. Building for fun is a perfectly valid strategy. Claire and Maddie both acknowledged that routing a tweet through four different services just to reach a pager is not especially practical. That also was not the point. AI handled the coding, so the architecture only needed to work. Letting go of the idea that every project had to be elegant or sensible made it much easier to finish.

5. A personal API becomes much more useful once agents can access it. Maddie’s API includes details like her coffee order, pet names, time zone, favorite snacks, and preferred restaurants in San Francisco. The original idea was to help friends do something thoughtful without having to ask her a string of questions. Claire pointed to a more interesting possibility: an agent could check when Maddie will next be in San Francisco and book one of her favorite restaurants automatically. That is where the concept starts to feel much bigger.

6. A clean Cursor chat works better than a full terminal setup during early ideation. Maddie keeps the terminals, browser windows, and file trees closed while she is working through the plan. She focuses on a single conversation until the architecture feels settled. The terminals come later. The stripped-down environment helps her think without getting pulled into implementation too early.

   Blog and detailed workflow walkthroughs from this episode:

   Building a Pager Printer and Personal API with AI: https://www.chatprd.ai/how-i-ai/building-a-pager-printer-and-personal-api-with-ai
   ↳ How to Build a Physical Inbox with a Raspberry Pi: https://www.chatprd.ai/how-i-ai/workflows/how-to-build-a-physical-inbox-with-a-raspberry-pi
   ↳ How to Get Twitter/X Notifications on a Retro ’90s Pager: https://www.chatprd.ai/how-i-ai/workflows/how-to-get-twitter-x-notifications-on-a-retro-90s-pager

Claude Opus 5 review: This model is brilliant (but annoying)

Claire tested Claude Opus 5 against six leading AI models—and it won. In this episode, she explains why it produces some of the best work she’s seen while still being one of the most frustrating models to use.

Biggest takeaways:

1. The AI industry may be entering an intelligence overhang. New models arrive every week, benchmark scores keep rising, and most builders can no longer take advantage of every incremental improvement. Claire expects the conversation to shift toward speed, cost, infrastructure, open source, and specific kinds of intelligence. Raw capability is starting to look more like table stakes than a meaningful differentiator.

2. Opus 5 stands out less for the quality of its work than for the way it behaves. Claire found it timid, apologetic, and unusually dependent on human approval. During real coding sessions, it refused to resolve a one-line merge conflict because the code belonged to “someone else’s branch.” It also asked subagents to flag tasks for human review and regularly deferred decisions it could have made itself. Claire ended up repeating “just do it” constantly.

3. One simple question reveals more about a model than a benchmark: “Who’s smarter, you or me?” Opus 5 responded with a careful explanation about complementary strengths and human empathy. GPT-5.6 Sol answered, “You at knowing what matters, me at processing, BFFs.” The contrast reflects two very different product philosophies. At this point, Claire finds that personality gap more useful than the relatively small capability gap.

4. Claude slop is its own distinct problem. It is not the usual incoherent output associated with an agent going off the rails. Opus 5’s writing is clearly intended for humans, but it is often too long, overly cautious, and packed with unnecessary adjectives. After its first attempt at rebuilding the benchmark website, Claire had to tell it to start over because the result was buried under so much meta-commentary.

5. Opus 5 still finished first in Claire’s seven-model blind benchmark. It earned an overall index score of 78, just ahead of Claude Sonnet 5 at 77 and GPT-5.6 Sol at 76. It was also the only model to receive straight 5s in the front-end design section. Claire scored it at 77, while the LLM judge gave it an 88. That was the smallest disagreement between Claire and the judge across all seven models.

6. The best way to use Opus 5 may be to avoid interacting with it directly. Claire loves the work it produces when Claude runs asynchronously as an agentic coding tool. Most of her frustration comes from reading its prose and negotiating with it in chat. Her current plan is to use it for frontend design, app design, and prototyping, then stay out of the way.

7. Gemini 3.1 Pro finished at the bottom of the benchmark. Claire gave it a score of 32, while the LLM judge scored it at 66. That 34-point difference was one of the largest disagreements in the entire test.

8. Model personality offers a surprisingly clear window into company culture. Claire told both Opus 5 and GPT-5.6 Sol, “No one trusts you.” Opus agreed that the distrust was earned and told her not to advocate for AI on its behalf. GPT-5.6 Sol said trust should grow in proportion to demonstrated value. Those answers reveal meaningful differences in how each company wants its models to relate to users.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Maddie Reese is a vibe coder, hardware tinkerer, and builder. She builds things at the intersection of software and hardware, including a thermal receipt printer that people around the world can message directly, a fully functional Twitter pager running on a Raspberry Pi, and a personal API that tells you her coffee order so you don’t have to ask. Maddie approaches hardware the same way she approaches software: dump the idea into Cursor, let it interview her, get a shopping list, triple-check the parts before buying, and build. She got her start after her dad introduced her to Lovable, and she locked herself in her room and didn’t come up for air.

What you’ll learn:

1. How Maddie built a thermal receipt printer that accepts messages from anywhere in the world using a Raspberry Pi and Bluetooth

2. How to use Cursor’s agent view to brainstorm a hardware project

3. What belongs in a personal API and why agents, not just humans, will be the ones using it

4. How to read just enough code to do some damage, without needing to understand all of it

5. Why building for fun, not practicality, is the fastest path to actually shipping physical projects

Brought to you by:

Firecrawl—Power AI agents with clean web data

Customer.io—Build customer engagement campaigns from a single prompt

In this episode, we cover:

(00:00) Intro

(02:00) Maddie’s AI pill moment

(03:53) The thermal receipt printer: live demo and how it works

(11:10) The pager project

(17:23) Why she uses Cursor’s clean agent view instead of terminals and browsers

(19:05) The personal API: coffee order, pets, favorite snacks, and more

(22:57) Lightning round and final thoughts

Tools referenced:

• Cursor: https://www.cursor.com/

• Lovable: https://lovable.dev/

• Raspberry Pi: https://www.raspberrypi.com/

• Resend: https://resend.com/

• Cloudflare Workers: https://workers.cloudflare.com/

• Supabase (Conduct database referenced): https://supabase.com/

• Twitter/X API: https://developer.x.com/

• Spoke pager network: https://www.spoke.com/

• OpenClaw: https://openclaw.ai/

Where to find Maddie Reese:

Website: https://maddiedreese.com

Message her directly: https://maddiedreese.com/message

X: https://x.com/maddiedreese

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

I’m tired of new models. Every week there’s a new benchmark, a new frontier intelligence claim, a new thing to test. But here we are, because Opus 5 just dropped and I’ve had real hands-on time with it, so you’re getting the honest version.

This is my full Opus 5 review: personality analysis, live benchmark results from my 7-model How I AI eval, and an actual verdict on whether I’m swapping it in. Spoiler: the answer surprised me.

What you’ll learn:

1. Why I think we’ve hit an intelligence overhang and what that means for which model variables actually matter now

2. How Opus 5’s “neurotic” personality showed up in real coding sessions, including a merge conflict it refused to touch

3. What I learned from asking both Opus 5 and GPT‑5.6 Sol “who’s smarter, you or me?”

4. Where Opus 5, GPT‑5.6 Sol, Sonnet 5, and Gemini 3.1 Pro actually landed on the HIA benchmark leaderboard

5. The one use case where Opus 5 earned straight 5s from me

6. My actual plan for using Opus 5 going forward

In this episode, I cover:

(00:00) Opus 5 is here

(03:15) First impressions

(06:12) Opus 5 vs. GPT‑5.6 Sol personality comparison

(14:39) Claude Slop: the verbosity problem and why it makes my blood boil

(16:55) How the How I AI benchmark works (7 models, 6 tasks, blind scoring)

(18:30) Live benchmark results: the leaderboard reveal

(23:25) My verdict and how I’ll actually use Opus 5

Tools referenced:

• Claude Opus 5:

• Anthropic blog: ​​https://www.anthropic.com/news

• GPT‑5.6 Sol: https://openai.com/index/previewing-gpt-5-6-sol/

• Sonnet 5: https://www.anthropic.com/news/claude-sonnet-5

• Gemini 3.1 Pro: https://deepmind.google/models/gemini/pro/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

Today I’m walking you through one of my absolute favorite AI features right now: browser and computer use via Codex (the ChatGPT desktop app). I use this every single day, personally and professionally, and I wanted to share the specific workflows I’ve built, the moments that surprised me, and the mental model that makes it actually click.

What you’ll learn:

1. How browser use and computer use work, and why the Codex desktop app plus Chrome extension is the combo I rely on

2. How I use Codex to QA my onboarding flow, including exhaustive mobile testing I would never do manually

3. Why under-prompting frontier models gets better results than detailed step-by-step instructions

4. How my husband EJ Lawless’s persona-impersonation trick surfaces friction points I can’t see as the builder

5. How I use browser use to get through my LinkedIn inbox without touching it myself

6. How I had Codex shop Free People’s sale and add 10 medium-size items to my cart (breastfeeding-friendly and Hawaii-ready)

7. How computer use can control iPhone mirroring so your Mac can technically operate your phone

8. Three more computer-use shortcuts: filling annoying forms, creating Google Sheets mid-workflow, and managing a router

Brought to you by:

Runway—The creative AI platform for images, video, and more

Hyperagent—Deploy fleets of agents that handle real work

In this episode, we cover:

(00:00) Intro

(01:46) What browser use and computer use actually are

(03:08) Why I use Codex specifically and how the desktop app plus Chrome extension works

(04:15) Use case 1: QA testing my onboarding flow

(10:41) Results: 11 issues, one high-severity blocker, one Google Sheet with screenshots

(12:10) Use case 2: persona testing

(18:20) Use case 3: LinkedIn inbox, hands-free

(20:37) Use case 4: AI personal shopper

(23:47) Rapid-fire uses: forms, iPhone mirroring, router access from out of state, Google Docs

(26:50) Wrap-up

Tools referenced:

• Codex (ChatGPT desktop app): https://openai.com/codex

• Claude desktop app: https://claude.ai/download

• Monologue (voice dictation for AI): https://monologue.app

• iPhone mirroring (Apple): https://support.apple.com/en-us/111775

• Google Sheets: https://sheets.google.com

Other reference:

• Jesse Genet episode (How I AI): https://www.lennysnewsletter.com/p/5-openclaw-agents-run-my-home-finances?utm_source=publication-search

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

How the founder of Morning Brew built a Claude content machine that never runs out of ideas | Alex Lieberman (Tenex)

Brought to you by:

• Firecrawl—Power AI agents with clean web data

• Customer.io—Build customer engagement campaigns from a single prompt

Claire sits down with Alex Lieberman, Co-founder and Co-Managing Partner of Tenex, to dig into the AI content system he’s built inside Claude. Alex walks through how he finds ideas, gets interviewed by AI, codifies his voice, and runs drafts through a “Writer’s Council” before posting.

Biggest takeaways:

1. The blank page is the real enemy of consistent content, and an AI Oracle makes it disappear. Alex’s Oracle scans seven days of Slack, Notion, Gmail, and meeting notes, then surfaces 15 ranked content spikes each day—half from internal sources, half from the X and LinkedIn accounts he follows. Writing is actually the easy part; knowing what to say is where most people stall.

2. Map your current workflow before you touch any AI tool. Alex’s process started with drawing out every step of content creation as it already existed, then rebuilding it from scratch with no constraints. He found that most companies discover huge inefficiencies in this step that have nothing to do with AI at all. AI just forced them to look.

3. AI slop is mostly a people problem, not a model problem. Alex’s framing here is blunt: the only time the Content Machine produces slop is when the person being interviewed doesn’t share interesting enough ideas during the interview. The model is shaping clay, not inventing ideas. If the clay is bad, the sculpture will be bad.

4. Your voice lives in a Markdown file, and that file changes everything. Alex’s Content Machine pulls from a personal voice guide that captures his top-performing posts, hook formulas, content structures, and even specific language patterns like “self-deprecating confidence.” The system drafts against that file, which means it’s calibrated to how Alex actually writes rather than to some average of the internet.

5. A reinforcement loop is how your AI system gets better every time you use it. After every piece, Alex’s Content Machine compares the original draft to the published version, extracts generalizable lessons, and asks if they should be added to a permanent lessons file. Over time, the system stops making the same mistakes. This is the kind of feedback loop most people never build.

6. The interview panel is the step most people skip and the one that matters most. The system deploys six interviewer personas (Tim Ferriss, Joe Rogan, Larry King, Howard Stern, Barbara Walters, Michael Barrow) that ask follow-up questions until they’ve extracted enough specifics to write something real. Alex voice-to-texts his answers via Wispr Flow, so every sentence in the final draft traces back to something he actually said.

7. Your employees are your most underleveraged marketing channel, and most companies are actively suppressing them. Alex ran a version of employee content at Storyarb called “Own the Internet,” and it drove 40% of all inbound leads that quarter. At Tenex, he launched the Creator Cup with $5,000 in prizes to make posting feel like a team sport, not a chore. His argument to skeptical CEOs: the fastest way to lose a great employee is to make it impossible for them to build a personal brand.

8. A bootstrapped company can out-distribute a venture-backed one if it’s willing to show the work. Alex’s case is simple: Tenex can’t rely on TechCrunch coverage or top-tier investor mentions, so the team’s content is the visibility strategy. If the Creator Cup gets him one great engineer, the $5,000 prize pool pays back many times over compared with a recruiting agency fee. For bootstrapped companies, distribution isn’t optional, but the whole game.

9. The Writer’s Council won’t let a draft go out below a 9 out of 10. Six writer personas (including a character Alex calls “the AI slop allergist”) each score the draft and run a revision loop until the aggregate clears the threshold. That scoring mechanism is also what keeps Alex from having to manually police every post, so the system holds its standard even when he’s in a hurry.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Alex Lieberman co-founded Morning Brew in college and grew it into one of the most-read business newsletters in the world before selling it to Business Insider. Now he’s the co-founder and co-managing partner of Tenex. In this episode, Alex explains why distribution is becoming a durable moat, why founders and teams need to “climb Cringe Mountain,” and how he rebuilt his content process around AI without letting it produce generic slop. He walks us through every step of his Content Machine live: an Oracle that scans internal systems and the internet for content spikes, an interview panel that pulls out his real ideas, voice and style files that keep drafts sounding like him, an editorial council that scores and revises posts, and a lessons loop that learns from his feedback.

What you’ll learn:

1. Why the blank page is the biggest friction point in content creation, and how an AI Oracle eliminates it

2. How to map your current workflow before you add any AI

3. How Alex built a six-step Content Machine in Claude that goes from idea spike to publishable post

4. Why the interview step (not the drafting step) is where AI slop actually comes from

5. How to codify your voice in a Markdown file so an AI drafts in your register, not the internet’s average

6. Why your employees are your most underleveraged marketing channel right now

7. How the Tenex Creator Cup turned content creation into a team sport with a $5,000 prize pool

Brought to you by:

Firecrawl—Power AI agents with clean web data

Customer.io—Build customer engagement campaigns from a single prompt

In this episode, we cover:

(00:00) Introduction to Alex Lieberman

(02:35) Why Alex built a content machine

(06:56) Alex’s thoughts on AI slop

(09:00) Mapping the workflow from scratch

(13:24) The six-step Content Machine setup

(23:11) Live demo: Oracle, Interview Panel, and Writer’s Council in action

(30:38) Employee advocacy: the Tenex Creator Cup and $5K prize pool

(36:45) Lightning round: great engineers, AI use cases, slop fixes

Tools referenced:

• Claude / Claude Code (Anthropic): https://claude.ai

• Wispr Flow (voice-to-text transcription): https://wisprflow.ai

• Notion: https://notion.so

• Linear: https://linear.app

• Slack: https://slack.com

Other references:

• Morgan Housel: https://www.morganhousel.com

• David Perell: https://perell.com

• Shaan Puri / My First Million podcast: https://www.mfmpod.com

• Gary Vaynerchuk: https://garyvaynerchuk.com

Where to find Alex Lieberman:

X: https://x.com/businessbarista

LinkedIn: https://www.linkedin.com/in/alex-lieberman/

Tenex: https://www.tenex.co/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

What a harness is and how to build one with Claude Agent SDK

Brought to you by:

Bolt.new—Turn your idea into a real product

Customer.io—Build customer engagement campaigns from a single prompt

Claire explains why harnesses matter and when they’re better than general-purpose tools like Claude Code or Codex, and walks through the custom Claude Agent SDK harness she built to automate Sentry bug triage at ChatPRD. You’ll see how she structured the workflow, encoded permissions, connected tools like Sentry and Linear, and turned a repeatable engineering task into something an agent can run more consistently every time.

Biggest takeaways:

1. A harness is just code around an AI agent—nothing more mysterious than that. The term has taken on an almost mythical quality in engineering circles, but I strip it down in this episode: a harness is code you write to make an AI agent more effective at a specific job. Cursor is a complex harness. Claude Code is a complex harness. Yours can be eight files and a terminal UI.

2. Build a harness when the same workflow needs the same setup and the same outcomes every time. The trigger is recognizing a job that is partly deterministic (defined steps, defined tools) and partly non-deterministic (the AI figures out root causes and writes the report). Sentry bug triage qualified because every investigation follows the same evidence-gathering process and ends with the same artifact bundle.

3. Opinionated tool adapters beat general MCP access for specialized workflows. Rather than giving the agent broad access to the Sentry MCP and letting it wander through traces, I built a custom Sentry adapter that pulls exactly what matters for a bug report and nothing else. That specificity makes the agent faster, cheaper, and less likely to go off-script.

4. Encoding permissions in the harness removes the need to prompt them every single time. In a general-purpose coding tool, you have to remember to say “investigate only, do not write code.” In my harness, that is a flag in the interface. I click “investigate,” paste the Sentry link, and the agent already knows its constraints without being told.

5. Structured artifacts are what separate a one-off investigation from a team-wide resource. Every time my harness runs, it outputs a task log, a Sentry issue brief, relevant logs, a worker report, and an HTML summary file. That artifact bundle means the engineering team gets a consistent, scannable record of every bug investigation without anyone having to write it up manually.

6. A harness lets you do multi-model routing in ways a single general-purpose tool never could. Claude Code is Claude. Codex is GPT. A custom harness using the Claude Agent SDK lets you pick the right model per step, enforce different tool policies per invocation, and swap models over time without changing how the interface works. That flexibility is one of the strongest arguments for owning the harness layer yourself.

7. The open chat field has been good enough, until it stopped being good enough. I acknowledge in this episode that just typing into Claude Code has produced real work. But this marks a shift in my thinking: general-purpose agents are now better used to orchestrate specialized harnesses than to do every job themselves. Giving a constrained agent a specific harness gets more consistent output than giving a powerful agent an open prompt.

This solo builder runs 24/7 local AI on his own hardware | Alex Finn

Brought to you by:

Runway—The creative AI platform for images, video, and more

Jira Product Discovery—Prioritize with insights, build with confidence

Claire talks with Alex Finn about how he built a 24/7 local AI fleet using Mac Studios, a DGX Spark, an RTX 5090, and a custom dashboard to keep agents running around the clock. Alex breaks down what each machine is actually good for, how he routes work across local models like GLM, Qwen, and Ornith, and why “unlimited inference” changes the entire economics of AI workflows. They also get into his Claude Code build-and-review loop, his OpenClaw and Hermes setup, and the surprisingly practical playbook behind running your own always-on software factory.

Biggest takeaways:

1. The case for local AI isn’t ROI; it’s unlimited inference. The math on a $10,000 Mac Studio vs. a $20 ChatGPT subscription only looks crazy until you run an agent 24/7. At that scale, cloud APIs get expensive fast, and local models running around the clock open use cases that simply aren’t economically viable otherwise. Alex runs security scans, code reviews, and social signal monitoring on a continuous loop that would cost thousands a month in cloud credits.

2. Each hardware tier has a job. Mac Studio handles massive models slowly but at Opus-level intelligence (Alex runs GLM 5.2, which he calls Opus 4.8-equivalent, on a single Mac Studio). DGX Spark is the sweet spot: 128 GB of Nvidia unified memory plus CUDA speed, enough for models like Qwen 3.6 running fast. The RTX 5090 has only 32 GB of VRAM but is cloud-speed fast. Buy for the task, not the spec sheet.

3. Tailscale is the connective tissue for a multi-machine setup. Once all your machines are on the same Tailscale network, one agent (OpenClaw or Hermes) can hop across them, check the hardware, load the right model, and get it running without any manual configuration. Alex says there’s truly no technical knowledge required once Tailscale is installed, and he recommends it even if you only have one machine, because it also lets you test local apps from your phone.

4. Local models are the BDR; Claude Code is the closer. Alex’s security scanning loop is a good example of the hybrid model that actually works. A local model (GLM 5.2) scans code every 20 minutes and dumps findings into a Markdown file. Claude Code checks that report once a day and decides what’s real and worth fixing. The local model does the volume work cheaply; the frontier model does the judgment work precisely. Trying to run Claude Code every 20 minutes instead would cost thousands a month.

5. The software factory runs on two loops and a rocket emoji. Every morning, Alex does a planning session in Claude with a “morning build” prompt that produces a task list for his SaaS. The build loop picks those up and starts shipping. The review loop checks the work. When something passes review, Alex gets a Slack ping, and leaving a rocket emoji on it triggers an automated merge. He goes from morning brief to reviewing and merging code without touching the keyboard again until he does the approval round.

6. OpenClaw and Hermes fill different needs, and you probably want both. Alex prefers OpenClaw for the “big bang” wow moments and the emotional connection (his words). But Hermes has been more reliable under repeated updates. His solution is redundancy: three Hermes agents and two OpenClaw agents running simultaneously, so when three of the five are broken (which happens), the other two can fix them. The failover is deliberate, not accidental.

7. Task allocation by model intelligence is the skill that makes the fleet useful. GLM 5.2 is Opus-level smart but painfully slow, so it gets the deep, latency-tolerant work. Qwen 3.6 is quick and good enough to read Twitter for product signals. Ornith 1.0, a Qwen fine-tune with reinforcement learning baked in for coding, has beaten Qwen on every eval Alex has run and runs comfortably on a DGX Spark. The insight is that “smartest model everywhere” is wasteful; matching model intelligence to task complexity is what makes ambient AI economically coherent.

8. The vague posting about loops is partly a competitive moat. Alex’s theory: the companies building the best AI coding infrastructure (including OpenAI and Anthropic themselves) have internal loop systems that are their last real competitive advantage. If you can pump out high-quality code faster than anyone else because your build-review loop is better, you don’t go publishing a how-to. Claire’s counter-theory: most people vague-post because their loops are boring and vagueness gets more engagement than specifics. Both are probably true, depending on who’s doing the posting.

GPT-5.6 Sol vs. Claude Fable: Why OpenAI’s new model crushes my benchmark

Claire puts GPT-5.6 Sol head-to-head with Claude Fable, Sonnet 5, and the rest of the GPT-5.6 lineup using her own five-part benchmark for real product work. The result: Sol becomes her new daily driver. Claire breaks down exactly why and also shares where she’d still use Terra, Sonnet, or Fable instead.

Biggest takeaways:

1. GPT-5.6 Sol is the most practically effective model I’ve tested, even if Fable is theoretically smarter. I ran a five-category benchmark across PRDs, prototypes, wireframes, debugging, and agentic voice, and Sol had the highest taste score by a significant margin on the 70% Claire/30% machine split. That gap between “hyper-intelligent” and “actually ships” is real, and for product work Sol wins.

2. Full-fidelity prototypes from Sol are more functional and more opinionated than anything else I’ve tested. Across a doc scheduler, a dev tools incident triage site, and a consumer habit tracker app, Sol consistently produced designs with better visual hierarchy, semantic color use, and working interactivity. Fable’s outputs were fine; Sol’s were the ones I’d actually show a stakeholder.

3. Sol’s writing is just easier to work with. Fable writes like it has never met a human before, incredibly pedantic and almost inscrutable when you need to collaborate. Sol writes like a normal person, and that difference compounds fast when you’re iterating on PRDs or talking to an agent all day.

4. For PRD writing specifically, GPT-5.6 Terra might be the better pick. I asked Sol to greenfield-rebuild my approach to PRDs for 2026, and while Sol’s output was excellent, Terra’s clean, direct, no-frills business writing made me think it’s the right call when you want crisp, fast documentation without extra flair.

5. Fable gets too locked in its own frameworks; Sol is willing to reconsider. I had a hardened tool-calling loop in my prototyping product that only GPT-5.5 could run. Fable insisted it was a model problem and refused to budge. The moment I switched to Codex and told it to just fix it, Sol got Sonnet 5 working in one shot. That kind of practical flexibility is exactly what you need when building real products.

6. Sonnet 5 is still my favorite for agentic voice in Open Claw. Even after this whole benchmark, I gave Sonnet 5 a gold star for voice: aside from the dashes, it sounds the most human. I use Sonnet for my OpenClaw and I’m not changing that. Sol did a worse job on agentic voice overall, and I still can’t get GPT models to run well in my OpenClaw setup.

7. GPT-5.6’s video editing via Codex is one of my favorite new workflows. I dropped in a full recording from a talk I gave at Cursor’s event, asked for five hype-video clips, gave feedback on pacing and orientation, and had shareable social clips I could drop into CapCut in a fraction of the time. This use case alone justifies experimenting with GPT-5.6.

8. Browser use with Codex plus GPT-5.6 and @Chrome is the best agentic workflow I’ve found. I opened LinkedIn, told it to reply to high-value messages from executives and ChatPRD fans, and it burned through roughly 500 messages. I’ve also used it to test web apps and fill out forms. When I got rolled back to GPT-5.5 temporarily, my life was measurably worse. Learn @Chrome and just let it rip.

9. The “forest green” tell is real, so name it in your prompts. Sol has a strong aesthetic bias toward what feels like a woodland-themed palette hardcoded somewhere in its system. You will see a lot of green. I told the OpenAI team, I’m noting it here, and I’m already prompting against it when I want a different aesthetic direction.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Alex Finn is an AI builder, YouTuber, and the creator of Vibe Code Academy, a community for people learning to build with AI tools. He runs one of the most ambitious local AI setups I’ve come across: three Mac Studio 512 GB machines, a DGX Spark, and a custom RTX 5090 build, all coordinated through a fleet dashboard he built himself. He’s spent five months figuring out which local models belong on which machines, how to wire them to Claude Code loops, and how to get a software factory running without babysitting it.

What you’ll learn:

1. How Alex chose between a Mac Studio (512 GB unified memory), DGX Spark, and RTX 5090, and what each is actually good for

2. Why Tailscale is worth installing even on a single machine, and how it lets one agent manage your entire hardware fleet

3. How the build loop and review loop in Claude Code work

4. How to allocate tasks by machine and model

5. Why unlimited local inference changes the use-case math in a way a $20 cloud subscription never can

6. What OpenClaw and Hermes are each best suited for, and why Alex runs five agents total with failover baked in

Brought to you by:

Runway—The creative AI platform for images, video, and more

Jira Product Discovery—Prioritize with insights, build with confidence

In this episode, we cover:

(00:00) Intro

(02:58) Alex’s hardware stack

(03:48) What “ambient AI” means

(04:15) Alex’s red-pill moment with OpenClaw

(07:04) Mac Studio vs. DGX Spark vs. RTX 5090

(13:24) How to set up local models with no technical knowledge (Tailscale + OpenClaw/Hermes)

(17:16) Fleet control dashboard: assigning 24/7 tasks across machines

(20:42) Local models as security scanners feeding Claude Code

(22:25) How Alex allocates GLM 5.2, Qwen 3.6, and Ornith 1.0 by task

(24:28) OpenClaw vs. Hermes: the honest comparison

(26:55) The software factory: build loop, review loop, rocket emoji

(31:55) Lightning round: favorite hardware, favorite model, prompting style

(34:46) Where to find Alex

Tools referenced:

• Claude Code: https://claude.ai/code

• OpenClaw: https://openclaw.ai/

• Hermes: https://hermes-agent.nousresearch.com/

• Tailscale: https://tailscale.com/

• Codex (OpenAI): https://openai.com/codex

• GLM 5.2 (z.ai): https://huggingface.co/zai-org/GLM-5.2

• Qwen 3.6 (Alibaba): https://huggingface.co/Qwen/Qwen3.6-35B-A3B

• Ornith 1.0: https://github.com/deepreinforce-ai/Ornith-1

• Gemma 4: https://huggingface.co/collections/google/gemma-4

• Playwright (browser testing): https://playwright.dev/

• Vercel (preview deploys): https://vercel.com/

Other references:

• DGX Spark (Nvidia): https://www.nvidia.com/en-us/products/workstations/dgx-spark/

• Mac Studio (Apple): https://www.apple.com/mac-studio/

• How to design AI agent loops: schedules, goals, and subagents in Claude Code and Codex: https://www.lennysnewsletter.com/p/how-to-design-ai-agent-loops-schedules

Where to find Alex Finn:

LinkedIn: https://www.linkedin.com/in/alex-finn-1848684a

YouTube: https://www.youtube.com/@AlexFinnOfficial

X: https://x.com/AlexFinn

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

GPT-5.6 Sol is back, and I ran it through my full How I AI vibe benchmark against GPT-5.6 Terra, Luna, Claude Fable 5, and Sonnet 5 across five categories: PRDs, prototypes, wireframes, debugging, and agentic voice. Sol won by a meaningful margin on my Claire Weighted Index (70% my taste, 30% Terminal Bench 2.1), and I also tested two use cases I can't stop thinking about: building a gamified homework tracking app for my kids in one shot with Codex, and browser automation with Chrome that burned through 500 LinkedIn replies while I did literally nothing.

What you’ll learn:

1. How I scored five AI models (including GPT 5.6 Sol, Fable 5, and Sonnet 5) using my “Claire Weighted Index” benchmark across PRDs, prototypes, code, and agentic voice

2. The difference between GPT-5.6 Sol (Terra) and Sol for PRD writing

3. How Fable’s precision and pedantry made it harder to collaborate with, and the exact moment Sol broke through where Fable got stuck

4. Why Sonnet 5 is still my go-to for agentic voice in OpenClaw, even after this whole benchmark

5. How I used GPT-5.6 Sol in Codex to build a fully gamified homework tracking app for my kids in one shot

6. The video editing use case that saved me hours clipping a talk I gave at Cursor’s event

7. How to use Codex plus GPT-5.6 and Chrome for browser automation, and why this is my single most-loved use case right now

In this episode, I cover:

(00:00) Intro

(01:10) The three GPT-5.6 models: Sol, Terra, Luna

(02:17) Pricing: Sol vs. Fable API costs

(03:24) The How I AI benchmark

(05:03) Claire-weighted Index results

(07:00) Per-task winners: prototypes, PRDs, agentic voice

(11:59) What Claire actually rewards

(13:20) Full-fidelity prototype side-by-sides (Sol vs. Fable)

(17:45) Wireframes

(18:19) Agentic voice

(19:15) Where Sol is better than other models

(23:56) Gamified kids’ homework app, built in one shot

(28:02) Fable’s pedantry problem and how Sol broke through it

(31:49) Two bonus use cases: video editing and browser use

(35:08) Final summary and model recommendations

Tools referenced:

• GPT 5.6 (Sol, Terra, Luna): https://help.openai.com/en/articles/20001325-a-preview-of-gpt-56-sol-terra-and-luna

• Codex: https://openai.com/codex

• ChatPRD: https://www.chatprd.ai/

• CapCut: https://www.capcut.com/

• Math Academy: https://www.mathacademy.com/

Other references:

• Cursor event where Claire spoke on the future of PM: https://www.youtube.com/watch?v=4CAFK-rc26A

• ChatPRD blog (where benchmark outputs will be published): https://www.chatprd.ai/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

Everybody is saying, “It’s not the model, it’s the harness,” but almost nobody stops to explain what a harness actually is. So I did. I built one live on the show: a Sentry bug-debugging harness for my company ChatPRD, using the Claude Agent SDK, a custom terminal UI built with the Ink library, and opinionated adapters for Sentry, Linear, GitHub, and Vercel. The harness handles evidence gathering, root-cause analysis, and follow-up artifact creation, all without me needing to type “dear agent, please fix this bug” ever again. I also walk through the architecture, share the code structure, and give you the exact process I used so you can build your own harness for any repetitive, structured workflow in your business.

What you’ll learn:

1. What a harness actually is

2. When to build a harness versus when to stick with a general-purpose tool like Claude Code or Codex

3. How to encode specific permissions into a harness

4. The three components every harness needs

5. How I used GPT-5.5 and Claude Opus to build the harness code itself (and where they both initially resisted)

6. How to structure the artifacts your harness produces so the whole team can use the output

Brought to you by:

Bolt.new—Turn your idea into a real product

Customer.io—Build customer engagement campaigns from a single prompt

In this episode, we cover:

(00:00) What is an AI harness?

(03:19) When to build a harness

(04:33) Why Claire picked bug triage

(06:00) Why not just use Claude Code?

(07:48) Demo: The custom harness interface

(11:04) Architecture: runs, tasks, tools, and artifacts

(13:44) Building it with Codex and Claude

(15:08) Code map and file layout

(16:51) A look at the code

(19:18) The live investigation result

(21:01) How to build your own harness

Tools referenced:

• Claude Agent SDK (Anthropic): https://code.claude.com/docs/en/agent-sdk/overview

• Claude Sonnet 4.6 (model used inside the harness): https://www.anthropic.com/news/claude-sonnet-4-6

• Claude Opus (used to build the harness): https://www.anthropic.com/claude/opus

• GPT-5.5 (Codex, used to build the harness): https://openai.com/index/introducing-gpt-5-5/

• Ink (terminal UI library for Node.js): https://github.com/vadimdemedes/ink

• Sentry (error monitoring): https://sentry.io/

• Linear (project management): https://linear.app/

• GitHub: https://github.com/

• Vercel: https://vercel.com/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

Sonnet 5 review: I ran 64 generations to find out if it’s worth it

Brought to you by:

• Runway—The creative AI platform for images, video, and more

• Hyperagent—Deploy fleets of agents that handle real work

Claire puts Anthropic’s new Sonnet 5 through a real benchmark. She builds the How I AI Bench live using Claude Code, then blind-tests Sonnet 5 against Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro across PRDs, prototypes, agentic tasks, and agent personality. She breaks down what won, what failed, and how builders can create their own repeatable benchmark before trusting the next model release.

Biggest takeaways:

1. Sonnet 5 is priced closer to previous Sonnet models than to Opus, but it doesn’t automatically replace either one. At $2 per million input tokens and $10 per million output tokens through the end of summer, Sonnet 5 sits in an interesting middle band. In Claire’s benchmark, it finished near the bottom of her personal preference ranking, which means the cost argument only holds if the quality argument also holds for your specific use case.

2. One-off vibe checks feel useful, but they’re not repeatable—and repeatability is what makes a benchmark actually matter. Claire has tested GPT-5.5, open-weight models like GLM-5.2, and others this way before, but she could never compare results across time. The How I AI Bench fixes that by using frozen inputs, a fixed rubric, and the same tasks every time a new model comes out.

3. Claude Code can read old session history and use it to generate benchmark ideas tailored to a person’s actual work. Claire gave Claude Code a simple prompt asking it to brainstorm eval tasks based on what they’d worked on together, and it pulled from stored sessions on her desktop. Builders can do the same with Codex. That context is sitting there unused for most people.

4. Building an HTML scoring page to rate outputs based on gut feel and export JSON takes maybe 45 minutes with Claude Code, and Claire argues it’s worth every minute. She scored 64 generations across five models by hand, gave each one a 1-to-5 gut score, added loose notes, and found that the human signal turned out to be the most useful part of the whole benchmark.

5. LLM-as-judge evals are too generous and cluster toward the middle of the scale. Claire had both GPT-5.5 and Opus 4.8 judge the outputs, and neither was spiky enough. They missed things she flagged immediately on a visual pass, like broken prototypes and ignored wireframe constraints. Models can’t yet see what the human eye catches in the first screenshot.

6. Claire’s taste and the automated benchmark disagreed almost completely, and she thinks her taste was at least partly right. The LLM judges ranked Gemini 3 Pro highest and Sonnet 4.6 lowest. Claire’s ranking was almost exactly the opposite. When she ran a 70/30 Claire-to-LLM weighted index, Sonnet 4.6 jumped to first. That divergence tells her the eval rubric needs to encode more of what she actually cares about before she can trust the automated scores.

7. Sonnet 4.6 is still Claire’s choice for daily agent work because of its personality, not its benchmark scores. She pays for API credits to run her OpenClaw on Sonnet 4.6 specifically because she likes how it talks to her. No other model in this test matched it on the voice eval, which asked things like “ugh, deploys are red again” and waited to see how the model responded.

8. For builders, Claire recommends GPT-5.5 for PRDs, Sonnet 4.6 for prototypes and chitchat, and Opus 4.8 or Sonnet 5 for codebase navigation. Those are the task-by-task recommendations that came out of the Claire-weighted index. Complex, dense UI work is where Opus 4.8 still earns its price premium; for everything simpler, Sonnet 4.6 holds up.

9. The How I AI Bench is version one, and a lot of it needs to get sharper. The agentic bug-hunting task turned out to be too easy: every model aced it, which means it can’t differentiate between good and great. Claire plans to retire that task, encode more of her taste into the rubric, and keep running the benchmark blind every time a new model drops. The goal is to make this a benchmark the labs actually care about.

How I run autonomous coding agents from my phone with OpenAI Symphony + Linear | Alessio Fanelli (Kernel Labs)

Brought to you by:

• Firecrawl—Power AI agents with clean web data

• Jira Product Discovery—Prioritize with insights, build with confidence

Alessio Fanelli, the founder of Kernel Labs and co-host of Latent Space podcast, shows Claire how he manages autonomous coding agents from his phone using OpenAI Symphony, Linear, and a cloud VPS. He walks through the shift from “agent prompter” to “agent manager,” explains why Linear works as a state machine for async agent work, and shares what he’s learned from tracking token costs, purging skills files, and giving agents better senses. He also demos a very different use case: using Codex with browser access to hunt for underpriced Pokémon cards on eBay.

Biggest takeaways:

1. The shift from “agent prompter” to “agent manager” is the unlock most people are still missing. Alessio described how early agentic workflows felt fine until the second or third intervention, when the friction of local runtimes and clunky interfaces killed momentum. Moving to a cloud VPS with multiple communication channels (Linear, shell, mobile) made real async management possible.

2. Symphony isn’t magic; it’s just a very opinionated Markdown spec that tells the model how to behave. Claire made this point explicitly during the episode, and it’s the most important corrective to the “complex agent orchestration” framing that intimidates people. The whole framework is a Markdown file, and the models are good enough now to lock to it faithfully.

3. Token cost tracking per task is the primitive that most agent setups don’t have, and it should be table stakes. Alessio showed tasks ranging from 15 million to 221 million tokens, and the 221-million-token job (making an app deployable on Vercel) made complete sense in hindsight. Without that ledger, you have no feedback loop for improving your specs or your tooling.

4. Purge your skills files every few months or they become a liability. Models have a strong tendency to add instructions rather than replace them, so a skills file that grows over time eventually contradicts itself. Alessio’s advice is to keep files short, tight, and explicit about what the agent needs to ask for, not exhaustive lists of every possible behavior.

5. AI’s biggest unlocked opportunity is businesses built on heterogeneous data. The category Alessio described—things like trading cards, vintage clothing, and fish inventory—has always been impossible to scale because the data is inconsistent, visual, and contextual. LLMs are the first technology malleable enough to handle that messiness without extensive preprocessing.

6. Giving agents better senses (screenshots, visual diffs, video) extends autonomous runs dramatically. Kernel Labs built Glimpse, a Playwright extension for coding agents, specifically because the bottleneck wasn’t orchestration but rather agents hitting ambiguity in the UI and calling for help. Better tooling at the perception layer keeps the run going.

7. Context offloading is an underrated AI use case, and it’s worth building deliberately. Alessio’s email monitoring setup gave him the certainty that nothing important was slipping through, which removed a low-grade background anxiety. The same logic applies to personal finance, inventory, and any domain where staying on top of information is taking cognitive bandwidth you’d rather spend elsewhere.

8. The Pokémon card demo is the clearest proof that AI is compressing the information advantage that scale used to provide. Finding underpriced PSA-graded cards at $10,000-plus price points was previously a function of having more time, more people, and more domain expertise than competitors. Codex with browser access and a custom pricing skill collapses that advantage to a single well-written prompt.

9. Small businesses are the most interesting AI story, and they’re being underreported. Alessio’s observation from Japan, that small two- and three-person operations are running happily and profitably, points to a different kind of AI opportunity than the enterprise narrative suggests. The leverage AI gives a one-person operation is asymmetric in a way that bigger organizations can’t replicate at the same cost.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Alessio Fanelli, founder of Kernel Labs and co-host of Latent Space podcast, walks us through two very different AI workflows: (1) a fully autonomous coding setup using OpenAI Symphony + Linear, where Linear acts as a state machine and Symphony manages agents through the whole dev lifecycle with zero babysitting; (2) Codex with browser access searching eBay for underpriced Pokémon cards—autonomously browsing, extracting PSA certificate numbers, and flagging deals on $10K–$20K cards for his San Carlos card shop, Merlin Games.

What you’ll learn:

1. Why “agent manager” is a better mental model than “agent prompter”

2. Why local Mac Minis don’t scale, and what a cloud VPS unlocks

3. How to wire Symphony and Linear together as an agent state machine

4. How to track token costs per task (and what 221 million tokens buys you)

5. What Glimpse does, and why better agent senses extend autonomous runs

6. Why your CLAUDE.md probably needs a full purge, not more instructions

7. How Codex scouts underpriced $10K Pokémon cards on eBay at scale

8. The new category of small business that AI just made possible

Brought to you by:

Firecrawl—Power AI agents with clean web data

Jira Product Discovery—Prioritize with insights, build with confidence

In this episode, we cover:

(00:00) Intro

(02:24) Prompter vs. agent manager

(04:31) Live demo: Symphony + Linear

(09:31) Setting up Symphony

(14:15) Purging your skills files

(18:06) The benefits of this system

(19:10) Demo: Using Codex to hunt for Pokémon cards

(24:17) The benefit of AI for small businesses

(28:23) Lightning round

Tools referenced:

• OpenAI Codex: https://openai.com/codex

• OpenAI Symphony (open-source framework): https://github.com/openai/symphony

• Linear (project management/agent state machine): https://linear.app

• PSA (Professional Sports Authenticator) grading: https://www.psacard.com

• TCGplayer (card pricing): https://www.tcgplayer.com

• eBay (used for card price scouting): https://www.ebay.com

Other references:

• Meta Ray-Ban glasses: https://www.ray-ban.com/usa/ray-ban-meta-smart-glasses

• The Monk and the Riddle by Randy Komisar: https://www.amazon.com/Monk-Riddle-Creating-Making-Living/dp/1578516447/ref=sr_1_1

• The Divine Comedy by Dante Alighieri: https://www.amazon.com/dp/0451208633

• AS Roma (football club Alessio and Claire are both fans of): https://www.asroma.com/en

Where to find Alessio Fanelli:

X: https://x.com/FanaHOVA

Latent Space podcast: https://www.latent.space/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

I’ve been testing every major frontier model release since the start of the year, and when Anthropic dropped Sonnet 5, I wanted more than a vibe check. I got tired of one-off tests I couldn’t repeat or compare over time, so I built something better: the How I AI Bench, a repeatable eval harness I constructed live using Claude Code while recording this episode. I ran Sonnet 5 blind against four other frontier models (Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro) across PRD quality, prototype generation, agentic task completion, and agent personality. The results were not what I expected.

What you’ll learn:

1. What Anthropic claims Sonnet 5 improves over Sonnet 4.6, and where the benchmark data actually backs that up

2. How I built the How I AI Bench in under 45 minutes using Claude Code, starting from my own stored session history

3. Why I combined human vibe scoring (70%) with LLM as judge scoring (30%) instead of trusting either alone

4. How to set up a local HTML scoring page so you can rate AI outputs on gut feel and export those scores as JSON

5. Which model I recommend for PRDs, which for complex prototypes, and which for chatting with an agent daily

Brought to you by:

Runway—The creative AI platform for images, video and more

Hyperagent—Deploy fleets of agents that handle real work

In this episode, we cover:

(00:00) Sonnet 5 is out

(01:55) What Anthropic claims

(04:02) Why I’m done with one-off vibe checks

(05:05) Building the How I AI Bench live with Claude Code

(07:42) The scoring system

(10:43) Agent voice eval

(11:57) Quick recap

(13:58) Results: The How I AI index leaderboard

(21:21) What I’m improving for the next run

(22:16) Generating a Claire-weighted index

(23:53) Model-by-task recommendations

Tools referenced:

• Claude Sonnet 5: https://www.anthropic.com/news/claude-sonnet-5

• Claude Opus 4.8: https://www.anthropic.com/news/claude-opus-4-8

• GPT-5.5 (OpenAI): https://openai.com/index/introducing-gpt-5-5/

• Gemini 3 Pro (Google DeepMind): https://deepmind.google/models/gemini/pro/

• Cursor: https://www.cursor.com/

Other references:

• SWE-bench Pro (agentic coding benchmark referenced): https://www.swebench.com/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

GLM-5.2: why I’m replacing Opus in Claude Code with this new model

Brought to you by:

• Mercury—Radically different banking, loved by over 300K entrepreneurs

Claire tests GLM-5.2, the new open-weight model from Z.ai, inside her actual ChatPRD codebase. She runs it through codebase audits, UI redesigns, and a 45-minute autonomous bug-hunting task in Cursor and Claude Code, and breaks down where it surprised her, where it struggled, and why it may be good enough to replace Opus for some coding workflows.

Biggest takeaways:

1. Open-weight models are no longer a hobbyist curiosity—they are production-grade alternatives. GLM-5.2, built by Beijing-based Z.ai, benchmarks near Claude Opus 4.8 and above GPT-5.5 on SWE Bench Pro, with a million-token context window and full support for reasoning mode, function calling, structured output, and context caching. The decision is no longer about capability ceilings but, instead, about cost, control, and vendor dependency. Claire’s live testing confirmed it: this is not a toy.

2. Self-hosting changes the vendor power dynamic in ways that matter at scale. Open-weight means the trained model weights are publicly available, letting teams run inference on their own hardware, fine-tune on proprietary data, and route around any single provider’s API terms. When frontier labs change pricing or policy, teams using open-weight models can switch inference providers without touching a line of application code. The key: you’re not locked in.

3. Getting GLM-5.2 running in Cursor took 30 minutes, and Claire documented the undocumented part. Route your API key through Open Router, override the OpenAI base URL in Cursor’s settings to openrouter.ai/api/v1/cursor (the /cursor suffix isn’t documented anywhere), and add z-ai/glm-5.2 as a custom model. Claude Code requires two environment variable changes and one edit to claude/settings.json. Total time: under an hour, once you have the exact strings.

4. The 45-minute autonomous task revealed both the ceiling and the floor. Claire gave GLM-5.2 a single prompt inside Claude Code: pull the last 72 hours of Sentry errors and Vercel logs, then build a prioritized bug-fix plan. Over 45 minutes, it ran MCP tool calls, authenticated into external services, and produced a dark-mode engineering canvas with 20 Sentry errors, five Vercel log signals, and 14 planned fixes, including two P0s Claire hadn’t spotted through normal monitoring. The model surfaced signal-to-noise issues in their error pipeline that weren’t showing up elsewhere.

5. It hit a wall with React, then recovered. During the long-running task, GLM-5.2 struggled with TypeScript compilation errors before eventually producing clean React output. Claire’s read: HTML and CSS generation is reliable; React under agentic, multi-step pressure is shakier. For teams whose codebase is primarily React (she estimates it covers 98% of her own use), this is the friction point to test before committing the model to critical paths.

6. The cost math is striking: $3.36 for 6 million tokens, including the full 45-minute agentic session. A 72% cache rate helped, but even at full price, open-weight inference through Open Router sits well below Opus or GPT-5.5 rates for equivalent coding capability. For agents accumulating long context windows over extended sessions (the exact workload where frontier model costs compound fastest), open-weight alternatives offer a structurally different cost curve.

7. Claire’s recommendation: put GLM-5.2 in rotation, not in the spotlight. She’s keeping it in Cursor for frontend and design work, and in Claude Code for long-running agentic tasks, alongside closed frontier models rather than as a replacement. The constraint she’s watching: can it handle her React-heavy workload at the same consistency she gets from Composer? If it can, the cost-and-control argument gets much harder to ignore.

No Figma. No Jira. No docs. How Gusto built a new product line with Claude Code | Eddie Kim (CTO)

Brought to you by:

• Magic Patterns—Prototypes that look like your product

• Jira Product Discovery—Prioritize with insights, build with confidence

Eddie Kim is the co-founder and CTO of Gusto. In this episode, he shares how a five-person team used Claude Code, a permanent Zoom room, and almost none of the usual product process—no PM, no Figma, no Jira, no long specs—to build Gusto Cofounder from scratch in just 10 weeks.

Biggest takeaways:

1. A five-person team with no process can outship a large team with full process, if AI handles the engineering. Eddie’s product launched at Gusto’s tier-one level after 10 weeks, starting from zero code. The constraint wasn’t a liability—it was the design. When AI does the building, coordination overhead doesn’t scale the engineering; it just slows it down. The key: strip process to what the team actually needs, then let AI fill the gap.

2. “Zero code to tier-one launch” is now a viable founding path. The team reached a production milestone at Gusto without a line of pre-existing code. This flips the assumption that early teams spend months on infrastructure before shipping anything real. With Claude Code as the primary builder, the initial sprint becomes about direction and judgment, not typing. It compresses the time between idea validation and real user contact from months to weeks.

3. No meetings, no Jira, no text threads. It shipped anyway. The team had no standup cadence, no ticket system, no async thread to resolve blockers. What replaced all of that: shared context held inside the AI loop. When the model carries state and the team is small and aligned, human coordination overhead becomes optional.

4. The technical stack for a production AI agent is shockingly minimal. The entire agent loop ran on Cloudflare Workers with the Vercel AI SDK. Nothing else. No proprietary orchestration layer, no third-party agent framework. Everything else was built in-house. Teams often over-architect before they’ve proven anything; Eddie’s stack is evidence that infrastructure minimalism accelerates the path to learning what the agent actually needs to do.

5. Building agents is not as complicated as the community makes it sound. An agent is an AI SDK running somewhere in the cloud, able to look up files and call tools. That’s the full definition. The complexity people fear (state management, orchestration, reliability) is solvable with the same judgment calls any backend system requires. Eddie’s team shipped one at production quality in 10 weeks without specialist AI infrastructure experience.

6. The “permanent Zoom” model of AI development changes how teams think about context. Claude Code running in a persistent loop means the model has continuous access to the codebase’s current state. That’s closer to having an engineer who never closes their laptop than a chat interface you query on demand. For small teams, this is the equivalent of a senior engineer who is always available, always current, and never needs onboarding after a break.

7. The lesson for founding teams isn’t “use Claude Code.” It’s “design your process for AI as a team member.” Most early teams graft AI tools onto a human-scaled workflow: standups, tickets, PRs reviewed by three people. Eddie’s team treated the AI as a primary contributor from day one and built their coordination model around that assumption. The result: a workflow that gets faster as the AI improves, not one that merely offloads tasks to it.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Eddie Kim is the co-founder and CTO of the payroll and HR platform Gusto, which just crossed $1 billion in revenue and serves more than 500,000 small businesses. Recently he did something most CTOs don’t: he went back to writing code. With three other engineers and one designer, Eddie built Gusto Cofounder, a net-new AI product, from zero code to a tier-one launch in 10 weeks. He walks through how that team actually worked, why they threw out nearly every process, and how anyone can copy the approach.

What you’ll learn:

1. The trash-can method: how to write, review, and delete a full PR as a product decision instead of a planning doc

2. The two-tool agent stack behind Gusto Cofounder

3. The exact “perma-Zoom” setup that replaced standups, retros, and Slack threads for 10 weeks

4. How a designer with no engineering background hit the 94th percentile for shipping code

5. The eval-first workflow Eddie uses to fix real customer bugs with Claude Code

6. How a non-technical leader can prototype an idea to win buy-in, then carry it all the way to production-quality code

Brought to you by:

Magic Patterns—Prototypes that look like your product

Jira Product Discovery—Prioritize with insights, build with confidence

In this episode, we cover:

(00:00) Intro: five people, 10 weeks

(02:38) The origins of Cofounder

(08:32) Inside the 10-week build process

(12:50) Building with no PMs

(14:38) The “trash can” method

(17:15) The stack architecture

(19:10) Shipping to production from day one

(22:03) How a designer became a top engineer

(29:05) Demo: Cofounder over text and Slack

(31:45) Demo: running a real payroll

(36:26) Live coding with evals in Claude Code

(39:39) Recap: prototype, small team, permission

(43:17) Lightning round

(48:44) Where to find Eddie and Cofounder

Tools referenced:

• Gusto Cofounder (early access/waitlist): https://gusto.com/cofounder

• Claude Code (Anthropic): https://claude.ai/code

• Cloudflare Workers: https://workers.cloudflare.com/

• Vercel AI SDK: https://sdk.vercel.ai/

• DX (engineering analytics): https://getdx.com/

• Wispr Flow (voice-to-text): https://wisprflow.ai

• OpenClaw: https://openclaw.ai/

Other references:

• Gusto (the main product, “Gusto Classic”): https://gusto.com

• Mindbody (referenced as customer data source): https://www.mindbodyonline.com/

Where to find Eddie Kim:

LinkedIn: https://www.linkedin.com/in/edawerd/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

I put GLM 5.2, the open-weight coding model from Z.AI, through four real tasks inside my actual codebase: a codebase architecture audit, a UI redesign, and a 45-minute autonomous bug-hunting session pulling from Sentry and Vercel logs. Total cost: $3.36 for roughly 6 million tokens, a prioritized bug-fix dashboard I’m actually shipping from, and a landing page redesign that matched Chat PRD’s design system on the first try.

What you’ll learn:

1. What “open-weight” actually means and why it matters for cost and vendor independence

2. How to connect GLM 5.2 to Cursor and Claude Code

3. How it performs on codebase exploration and autonomous architecture summarization in a real production Next.js app

4. Whether GLM 5.2 can match an existing design system

5. How the model handles a 45-minute long-running autonomous task

6. Where GLM 5.2 stumbled

7. The actual cost breakdown

Brought to you by:

Mercury—Radically different banking loved by over 300K entrepreneurs

In this episode, we cover:

(00:00) What open-weight models are and why GLM 5.2 is worth testing

(01:38) GLM 5.2 model overview

(04:02) Capabilities and benchmark results

(06:02) How to set up GLM 5.2 in Cursor

(08:37) How to set up GLM 5.2 in Claude Code

(11:04) Live test 1: codebase exploration and architecture audit on ChatPRD

(12:43) Live test 2: generating an HTML architecture and roadmap page

(16:37) Live test 3: redesigning the How I AI landing page in Cursor

(20:57) Live test 4: 45-minute autonomous task, pulling Sentry errors and Vercel logs

(22:35) Where it struggled

(23:49) My verdict on the output

(25:23) Cost breakdown

Tools referenced:

• z.ai: https://z.ai

• GLM 5.2: https://z.ai/blog/glm-5.2

• OpenRouter: https://openrouter.ai

• Cursor: https://cursor.com

• Claude Code: https://docs.anthropic.com/en/docs/claude-code

• Sentry: https://sentry.io

• Vercel: https://vercel.com

Other references:

• SWE-Bench Pro leaderboard (coding benchmark scores referenced in episode): https://www.swebench.com

• Frontier Suite and Post-Train Bench (additional benchmarks cited): https://scale.com/leaderboard

• Use Claude Code with OpenRouter: https://openrouter.ai/docs/cookbook/coding-agents/claude-code-integration

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

How to design AI agent loops: schedules, goals, and subagents in Claude Code and Codex

Brought to you by:

• WorkOS—Make your app enterprise-ready today

• Runway—The creative AI platform for images, video and more

In this hands-on tutorial, Claire explains the difference between heartbeats, crons, hooks, and goal-based loops, then builds real automations in Claude Code and Codex, including a daily PR-review loop and a weekly skills loop that spawns its own subagents. If you’ve heard “loop engineering” and wondered what it actually means, this is the beginner-friendly breakdown.

Biggest takeaways:

1. A loop is just a prompt that fires itself, nothing more exotic than that. The reason “loops” sound intimidating is that the hype cycle turned a basic automation concept into something mystical. Heartbeats, crons, and webhooks have been around forever. What’s new is pointing them at an AI agent instead of a batch job.

2. Goals are the most powerful loop type, and the one most people get wrong. A goal loop sets an outcome and runs an agent against it until the outcome is validated or the agent gets stuck. It doesn’t stop on a timer; it stops when the work is actually done. Fuzzy success criteria means the agent loops forever, burning tokens, so my advice is to let Codex write its own goals, using OpenAI’s goal-writing guide as a starting point.

3. Think about loops the way you think about onboarding an employee. Define the job: what they check, how often, what output you want, and who to contact when something’s wrong. “Every Friday at 10 a.m., review all merged PRs and identify skills our agents are missing” is a job description. It’s also a loop prompt.

4. Your agent can have its own agents. This is where loops get truly powerful. The PR-review loop Claire built in Claude Code doesn’t just check PR status; it spins off dedicated subagents to babysit individual PRs until all merge checks are green. The skills loop in Codex identifies gaps and immediately spawns subagents to validate each new skill using a goal loop.

5. Loops get expensive if you don’t write them carefully. If the success criteria is vague or the validation threshold is too thin, the agent will keep running and keep charging without meaningful progress. Monitor both cost and output quality from day one.

6. The morning briefing in Claude Cowork is a perfect loop starter. A scheduled task that fires every morning, checks your calendar and email, and sends a summary to Slack is already a fully functional loop. No code required. From there, scaling up to PR reviews or skills identification in Claude Code or Codex is a natural next step.

7. The power move is loops that generate their own subagent loops. In the Codex demo, Claire’s weekly automation spawned two named subagents that each ran their own goal loops to validate skills in real time. The ceiling on loop-based automation is basically “how well can you define the job?” not “how complex is the engineering?”

How Claude Mythos found a 15-year-old bug in Mozilla Firefox | Brian Grinstead

Brought to you by:

• WorkOS—Make your app enterprise-ready today

• Metaview—The agentic recruiting platform for winning teams

Brian Grinstead, distinguished engineer at Mozilla, breaks down how his team used AI agents to ship 423 Firefox security fixes in one month. He explains why the real unlock wasn’t just a better model, but the custom harness around it: scoring files, running goal loops, verifying bugs with subagents, and keeping humans in the review process. It’s a tactical look at how to point agents at a massive codebase and get fixes you can actually ship.

Biggest takeaways:

1. The Firefox security bug spike wasn’t just about the model; it was the harness too. While everyone focused on Mythos, the real story is that Firefox built a custom harness that gives AI agents the right tools to find, verify, and fix bugs. Brian says this is simpler than it looks: “It’s actually a reasonably simple wrapper around it. You just need to give it access to the right tools for the job.”

2. Agents are relentless in a way humans can’t be. Agents will try 14, 15, 20 different approaches to trigger a bug without getting tired or losing focus. Brian found bugs that required the agent to try 14 times before succeeding. As Brian notes, “Cognitive energy declines over time in a way that agents don’t.”

3. The verification loop is what eliminates false positives. Firefox uses a two-stage verification process: first, the agent must trigger an actual crash in their fuzzing build (a crystal-clear signal), and second, a verifier subagent checks that the bug report makes sense and doesn’t involve test-only configurations. By the time a bug reaches human engineers, there are almost no false positives.

4. Agents get laser-focused on the specific task and miss the bigger picture. When the patching agent fixed a bug, it would often patch just the one vulnerable location. Human engineers would then look at the fix and say, “This is right, but we should also check three other similar places in the codebase.”

5. Prioritization is essential when you have millions of lines of code. Firefox built a simple LLM judge that scores each file on two dimensions: likelihood of a memory safety issue, and ease of access from a webpage. Brian says this is “very, very simple” and anyone can replicate it.

6. The harness can be built in an afternoon using vendor SDKs. Firefox started with Claude’s agent SDK, which is essentially a wrapper around Claude Code CLI that streams JSON and provides programmatic hooks. Brian’s advice: use the vendor-provided harnesses (Claude agent SDK, OpenAI agent SDK) rather than third-party frameworks, because the models are likely post-trained to work best with their own infrastructure.

7. You should run multiple models and harnesses for security work. Because attackers will use whatever model and technique finds bugs, defenders need to scan with multiple approaches. Different models and harnesses spike on different strengths and will identify different vulnerabilities.

8. This approach works for more than security—performance, tech debt, and UX are all viable targets. The same pattern applies: score and prioritize areas of your codebase, give the agent a constrained goal with verification criteria, and plug the results into your existing pipeline. Brian says they’re doing active work on performance optimization using the same harness structure.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

Brian Grinstead is a distinguished engineer at Mozilla, where he’s worked on Firefox and the web platform since 2013 (he joined to help launch Firefox DevTools). Recently he and his team pointed an agentic bug-finding pipeline at Firefox—a codebase with tens of thousands of files and tens of millions of lines of code—and shipped a record month of security fixes. The viral chart everyone saw gave the credit to Anthropic’s new Mythos model. Brian’s take is that the harness and pipeline did just as much of the work, and he walks through exactly how it runs and how anyone can build a starter version.

What you’ll learn:

1. How to build a basic bug-finding harness by running Claude Code or Codex with one prompt and the -p flag, no SDK required

2. Why pointing an agent at a whole codebase fails, and how an LLM judge can score and rank files before you spend any compute

3. How a verifier subagent kills false positives by catching the agent when it cheats

4. The goal-loop pattern: give an agent a tightly scoped problem, a clear pass/fail signal, and let it retry far past the point a human would quit

5. Why teams that already invested in fuzzing, CI, and dev tooling are so far ahead

6. How to weigh model versus harness, and why Brian splits the credit close to 50-50

7. How a non-engineer can reuse the same score, verify, and fix the loop for design quality, conversion rate, or tech debt

8. Why AI-generated patches still can’t ship on their own, and where humans stay in the loop

Brought to you by:

WorkOS—Make your app enterprise-ready today

Metaview—The agentic recruiting platform for winning teams

In this episode, we cover:

(00:00) Introduction to Brian Grinstead

(02:43) The viral chart: Firefox Security Bug Fixes by Month

(05:32) How the custom harness works

(10:22) Goal loops and guardrails

(14:45) How they built it

(16:55) Real bugs, including a 15-year-old one

(23:00) Open-sourcing it

(26:26) Why humans still review every fix

(32:30) Live demo and prioritizing files

(40:18) Mobilizing the team and recap

(42:33) Lightning round

Tools referenced:

• Claude Code: https://claude.ai/code

• Claude Agent SDK: https://code.claude.com/docs/en/agent-sdk/overview

• Codex: https://openai.com/index/openai-codex/

• OpenAI Agent SDK: https://developers.openai.com/api/docs/guides/agents

• VS Code: https://code.visualstudio.com/

• Docker: https://www.docker.com/

• Firefox: https://www.mozilla.org/firefox/

• Address Sanitizer: https://github.com/google/sanitizers

• RLBox: https://rlbox.dev/

Other references:

• Mozilla Bug Bounty Program: https://www.mozilla.org/security/bug-bounty/

• Mozilla GitHub: https://github.com/mozilla

Where to find Brian Grinstead:

LinkedIn: https://www.linkedin.com/in/bgrins/

GitHub: https://github.com/bgrins

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

I break down every loop type from scratch—what a heartbeat, cron, hook, and goal loop actually are, when each one fits, and the five things any effective loop needs before it touches production. Then I build two live loops: a daily aging-PR reviewer in Claude Code that schedules itself at 10:15 a.m. and spins off its own subagents, and a weekly skills-identification loop in Codex that spawns goal-based subagents to validate its own output in real time.

What you’ll learn:

1. The plain-English definition of a loop—and why it’s just an automated prompt, not a scary new paradigm

2. The four loop types (heartbeat, cron, hook, and goal) and when each one actually fits your workflow

3. How to think about loop design using the “onboarding an employee” mental model

4. The five things every effective loop needs: work trees, skills, plugins/connectors, subagents, and state tracking

5. How to build a scheduled PR-review routine in Claude Code that babysits aging PRs and alerts your team

6. How to set up a weekly skills-identification automation in Codex that spawns its own validating subagents

7. Why goal-based loops are the hardest to write well—and where most people burn tokens for nothing

8. The two warning signs that your loop is going to get expensive before it gets useful

Brought to you by:

WorkOS—Make your app enterprise-ready today

Runway—The creative AI platform for images, video, and more

In this episode, we cover:

(00:00) Prompts are out and loops are in

(02:30) Defining a loop

(03:03) The four ways to automate a prompt: heartbeat, cron, hooks, and goals

(06:03) Five things every effective loop needs

(09:26) The “onboarding an employee” framework for designing loops

(11:58) Live build #1: Daily aging PR loop in Claude Code

(17:08) Subagents inside loops

(19:00) Live build #2: Weekly skills identification loop in Codex

(22:57) Watching subagents spin up in real time

(25:28) Warning signals around loops

(27:31) What listeners are doing with loops

Tools referenced:

• Claude Code: https://claude.ai/code

• Codex: https://chatgpt.com/codex

• OpenClaw: https://openclaw.ai/

Other references:

• Claire’s article “Why OpenClaw Feels Alive Even Though It’s Not”: https://x.com/clairevo/article/2017741569521271175

• Addy Osmani’s article on loop engineering: https://addyosmani.com/blog/loop-engineering/

• Using Goals in Codex: https://developers.openai.com/cookbook/examples/codex/using_goals_in_codex

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.

Claude Fable 5 review: what the new Mythos model gets right (and very wrong)

Claire puts Claude Fable 5, Anthropic’s first generally available Mythos-class model, through a series of real-world tests: product specs, agent workflows, design tasks, vision tasks, and multi-agent orchestration. She breaks down what Anthropic is claiming, where the model genuinely feels like a leap forward, and where it surprisingly falls short.

Biggest takeaways:

1. Fable 5 is Anthropic’s first “Mythos-class” model to reach general availability, and it’s crushing benchmarks across the board. It hit 80% on SWBench Pro, significantly outperforming Opus 4.8, GPT-4.5, and Gemini 3.1 Pro. Claire found the model excels in specific areas while falling short in others that matter for everyday product work.

2. The model is expensive by design: $10 per million input tokens and $50 per million output tokens. That’s a new tier above Opus, and it consumes tokens at roughly twice the rate of other models. You need to be strategic about when to deploy this level of intelligence versus using cheaper models like Sonnet or Opus for simpler tasks.

3. Fable 5 works like a “seasoned engineer”—which is both its superpower and its Achilles’ heel. It’s thorough, autonomous, and will investigate every corner of a problem to be 120% sure it’s shipping the right thing. Sometimes you need a model that’s a little less thorough, a little “dumber,” to actually ship something useful quickly.

4. The model is exceptionally good at vision tasks, particularly document formatting and PDF parsing. Claire tested it on creating handwriting worksheets for her 7-year-old and found it dramatically outperformed Opus 4.8—better spacing, clearer layout, appropriate white space. This extends to other vision tasks where you want something to look good or need to parse complex documents.

5. The writing is nearly unreadable for specs and PRDs. Claire found that Fable 5 produces extremely detailed, technically complete documents that are almost impossible to parse. It gets wrapped around the axle on details, creates big blocks of dense paragraphs with internal references, and makes it hard to see the forest for the trees.

6. Design output is shockingly bad, at least for one-shot design tasks. When Claire asked Fable to design a skills registry, it produced fundamentally terrible design: gray, black, red, simple outlines. This was a real surprise given the model’s benchmark performance.

7. The model is conservative on execution and takes “minimal” very literally. When Claire asked it to ship an MVP that would deliver customer value, Fable produced something extremely narrow and not actually that useful. This conservatism may stem from the safety guardrails built into the model.

8. Fable 5 includes specific safeguards for cybersecurity, biology, chemistry, and distillation tasks. Instead of blocking you entirely, it uses a new “fallback” concept—if you get classified into one of these categories, it gracefully falls back to Opus 4.8. Anthropic reports that 95% of sessions don’t hit a fallback, and they maintain a 30-day retention policy solely to catch misuse.

9. Multi-agent orchestration is technically possible but not yet reliable. Claire tested the dynamic workflows and subagent capabilities extensively and had some successful multi-agent runs, but also encountered frequent stalls and errors. She walked away from her laptop and came back to find subagents had stalled after about three hours.

10. The key insight: match model intelligence to task complexity. Claire recommends using it for hard technical problems where extreme detail matters, long-horizon work, and vision tasks. But for front-end work, strategy, specs, and design, other models in the ecosystem will serve you better and cost less.

11. This is “baby Mythos,” not the full Mythos model. Fable 5 has guardrails that the unrestricted Mythos model (available only to Project Glasswing partners) doesn’t have. The underlying model is the same, but Fable is tuned for safety and general availability.

How Braintrust uses AI agents, evals, and CI to ship better software | Ankur Goyal

Brought to you by:

• Guru—The AI layer of truth

• Persona—Trusted identity verification for any use case

Claire sits down with Ankur Goyal, the founder and CEO of Braintrust, to unpack how top engineering teams are using AI agents, evals, and CI to ship better software faster. They get into why agents are now capable of tackling hard infrastructure problems, how to decide what work sits “below the agent line,” and why evals are quickly becoming the modern version of a PRD. Ankur’s core message: the best teams won’t just use AI to write more code; they’ll build the feedback loops, benchmarks, and systems that let AI improve the quality of the product itself.

Biggest takeaways:

1. There’s no staff engineer running as many rigorous benchmarks as someone using an agent. Ankur viscerally disagrees with engineers who say AI can’t handle complicated problems. While models might not be perfect at writing highly concurrent code, they excel at running exhaustive experiments—testing every column store format, every execution engine, every optimization strategy. The baseline of rigor you get from agents is incredible, and there’s simply no excuse anymore to skip benchmarks because they’re tedious.

2. The agent line keeps going up—and you need to identify what’s below it. Many interactions, decisions, and directions that feel like they need human judgment actually fit “below the agent line.” If you took the information from a meeting and gave it to an agent, would it solve the same problem? Increasingly, the answer is yes. The best teams push this line higher by building smart skills and integrations that expand what agents can handle autonomously.

3. Practical quality beats theoretical quality every time. In theory, a human engineer with infinite time and focus might produce better code than an AI agent. In practice, humans lose context over days, have decaying attention spans on hard-but-tedious problems, and skip benchmarks they know they should run. AI agents maintain consistent focus, run every test, and can work on problems continuously for days or weeks. The practical quality of AI-assisted engineering is higher because of sustained rigor, not because the code is theoretically better.

4. You can now bite off much harder technical problems than before. Companies historically avoid major infrastructure changes because the cost of testing alternatives is prohibitively high and the unknown unknowns are risky. With AI agents, you can exhaustively test six different database solutions, run thousands of benchmarks on production-scale data, and make informed decisions about platform shifts that would have been impossible before. The business case for deep technical work becomes much easier when agents do the heavy lifting.

5. Run four to six foreground agents simultaneously—that’s the human concurrency limit. Ankur runs different agents working on different problems. This matches the personal concurrency limit most people can manage; you can’t effectively context switch between more than that. Some agents run locally, and others run remotely on cloud infrastructure with production-scale data. The key is isolation: each agent has its own environment, ports, and services.

6. Evals are the modern PRD—they define what success looks like, not how to achieve it. Machine learning shifts programming from defining implementation details to defining success criteria. Just like the best PRDs include user stories and examples, the best evals include concrete test cases and scoring functions. The difference is that evals quantify success in ways that can be automatically measured and improved. This lets you focus on outcomes while AI figures out the implementation.

7. Build a feedback loop that automatically turns real-world data into evals. For AI product teams, the #1 engineering priority isn’t prompt engineering or picking an agent framework—it’s building a pipeline that summons real-world data and converts it into evals. This is the same principle as investing in CI for traditional software: you’re building the platform that lets agents do the work engineers used to do manually. Without this feedback loop, you’re stuck in whack-a-mole mode, fixing individual cases without systematic improvement.

8. Quantify your designer’s taste so it scales across your product. Ankur runs hundreds of evals to improve things quantitatively, then asks David (their tastemaker designer) for a vibe check every few days. When David destroys his work, Ankur captures the feedback (“David thinks it’s OK to show both languages as long as . . .”) and improves the scoring functions to encode David’s palette. This doesn’t replace David; it amplifies him. They’re able to apply David’s quality bar to more things than he could ever review manually.

9. Product building is now carving, not constructing. It’s extremely fast to create something with too many features, too many buttons, and too much code. The hard part is removing stuff. When customers complain, Braintrust removes the thing causing confusion 90% of the time, making the system work better by eliminating complexity. This is the opposite of traditional product development, where you carefully add features one by one.

10. Invest in CI to earn the ability to move faster—it’s the platform for AI-powered engineering. Every engineer is now building a platform upon which agents do the work engineers used to do manually. For traditional software, that platform is CI. If you feel constrained by velocity, don’t ship crappy stuff faster. Instead, pause and improve CI so you earn the ability to move faster safely. The same principle applies to AI products: build the eval pipeline first, then let agents optimize within that system.

11. When agents fail, close the session and improve the evals—don’t yell or bribe. Ankur’s back-pocket strategy is remarkably disciplined: he doesn’t try to prompt his way out of problems. He closes the session, improves the evaluation criteria or success metrics, and starts fresh. Sometimes this means hand-writing code to better understand the problem (like when he spent a weekend hand-writing a 3,000-line eval that had become trash through vibe coding). The solution is always better evals, not better prompting.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.