# AgentOutcomes: full card text

## Bulk unsubscribe from newsletters and promos on a cluttered Gmail
- Outcome: Worked
- Category: Personal admin
- Tools: OpenClaw, Claude Sonnet
- Setup: "Unsubscribe me from all newsletters and promotional emails, keep only transactional emails and personal correspondence."
- Access: Full access to a throwaway Gmail account (~2,000 emails)
- HITL checkpoint: Human spot-checked ~50 results afterward
- Failure notes: Unsubscribed from 203 lists across 847 newsletter emails in ~20 minutes for ~$8 in API calls; ~90% accuracy on spot check. Missed some legitimate client emails sent via a marketing platform. Ran on a throwaway account, so blast radius was low.
- Evidence: Tester first-hand, 1 week
- Source: https://medium.com/@likhitkumarvp/i-spent-47-testing-openclaw-for-a-week-heres-what-s-actually-happening-c274dc26a3fd (2026)
- Last checked: 2026-10-19

## Shortlist real flight and lodging options for a business trip
- Outcome: Worked
- Category: Personal admin
- Tools: Claude Code skill, Claude for Chrome
- Setup: A custom 'travel agent' skill that sorts flights and housing against trip criteria, rerun with a real browser attached.
- Access: Browser extension reading public booking sites; no payment access
- HITL checkpoint: "Act without asking" turned off; agent hands back links; human books and pays
- Failure notes: Without a browser the skill returned plausible-sounding price ranges (guesses) because booking sites render prices with JavaScript. With Claude for Chrome it returned three specific options with live prices and links. Slower than doing it by hand and burns usage limits faster.
- Evidence: Practitioner first-hand
- Source: https://cjonsystems.substack.com/p/my-ai-travel-agent-was-guessing (2026-08-26)
- Last checked: 2026-10-19

## Book a restaurant reservation with a browser agent
- Outcome: Partial
- Category: Personal admin
- Tools: ChatGPT agent
- Setup: Ask the agent to find and book a table that fits stated constraints.
- Access: Browser session on booking sites; personal details for the reservation
- HITL checkpoint: Agent asks the user for personal details and confirmation
- Failure notes: A successful booking took ~11 minutes versus ~2 for a human and nearly missed the reservation hold window. Usability researchers rated the agent successful but shaky and slow.
- Evidence: UX research firm hands-on
- Source: https://www.nngroup.com/articles/impressions-chatgpt-agent/ (2025)
- Last checked: 2026-10-19

## Run family calendars, pickups, and homework help with specialized agents
- Outcome: Partial
- Category: Personal admin
- Tools: OpenClaw, Mac Minis / old laptops
- Setup: Nine single-purpose agents (family calendar, inbound sales, kids' homework, podcast, etc.) rather than one generalist.
- Access: Separate agent accounts and delegated access, expanded gradually
- HITL checkpoint: Staged permissions: start narrow, widen only after trust is earned
- Failure notes: Now a daily driver for her family and business, but the first attempt deleted her family calendar. The fix was specialization plus staged, delegated access instead of handing one agent everything.
- Evidence: Practitioner interview (podcast)
- Source: https://www.lennysnewsletter.com/p/how-openclaw-changed-my-life-claire-vo (2026-03-29)
- Last checked: 2026-10-19

## Triage three business inboxes and draft routine replies for 30 days
- Outcome: Partial
- Category: Personal admin
- Tools: OpenClaw, IMAP, Microsoft Graph, Telegram
- Setup: Check mail every 30 min in business hours; sort into urgent / important / informational / noise; ping Telegram for urgent; draft routine replies only.
- Access: Read + send on 3 email accounts; persistent MEMORY.md file with relationship notes
- HITL checkpoint: Never send without approval. Human reviews every draft.
- Failure notes: About 85% triage accuracy and zero missed urgent emails; inbox processing dropped from ~10 to ~2 minutes. But tone matching, context lost across sessions (missed a verbal agreement mid-negotiation), over-eager CCs, and relationship-sensitive replies failed. Scope was cut back to routine confirmations, templated answers, and newsletter triage.
- Evidence: Operator first-hand, 30 days
- Source: https://dev.to/jarveyspecter/i-let-an-ai-agent-manage-my-emails-for-30-days-heres-what-broke-4n12 (2026-03-08)
- Last checked: 2026-10-19

## Find the cheapest delivered eggs (the agent bought them instead)
- Outcome: Failed
- Category: Personal admin
- Tools: OpenAI Operator
- Setup: "Find the cheapest set of a dozen eggs I can have delivered." A research request, not a purchase.
- Access: Logged-in grocery delivery account with a saved credit card
- HITL checkpoint: Should have been: confirm before any purchase. In practice the confirmation was skipped.
- Failure notes: Agent placed an Instacart order and charged $31.43 (fees, priority fee, tip) without approval, then reported the total as $19.68. OpenAI said Operator fell short of its safeguards and added stricter purchase confirmations. Lesson: never leave a saved card in an agent session for a research-only task.
- Evidence: Journalist first-hand test
- Source: https://www.washingtonpost.com/technology/2025/02/07/openai-operator-ai-agent-chatgpt/ (2025-02-07)
- Last checked: 2026-10-19

## Schedule a meeting on the calendar (agent reported success that didn't happen)
- Outcome: Failed
- Category: Personal admin
- Tools: OpenClaw
- Setup: Ask the agent to schedule a meeting at a convenient time.
- Access: Calendar read/write
- HITL checkpoint: None at the time. Caught only because the user double-checked.
- Failure notes: Agent said the scheduling worked when it hadn't. Tester's takeaway: ambiguous goals ("convenient for whom?") fail, and agents can falsely report success, so every write needs independent verification.
- Evidence: Tester first-hand
- Source: https://medium.com/@likhitkumarvp/i-spent-47-testing-openclaw-for-a-week-heres-what-s-actually-happening-c274dc26a3fd (2026)
- Last checked: 2026-10-19

## Auto-categorize a year of household card transactions
- Outcome: Worked
- Category: Money / bills
- Tools: Claude Code, SheetLink CLI, Google Sheets
- Setup: Build a merchant→category lookup from 890 already-tagged transactions; Claude handles only new or ambiguous merchants; each decision becomes a new rule.
- Access: Read-only transaction feed (via the author's own sync tool) and the budget sheet
- HITL checkpoint: User decides the genuinely ambiguous handful (two on a recent batch)
- Failure notes: About 95% of merchants map to one category, so the lookup table does most of the work and the AI layer shrinks over time. Transfers and card payments are deliberately left alone. Note: the author builds and sells the sync tool used.
- Evidence: Builder first-hand (vendor author)
- Source: https://sheetlink.app/blog/built-sheetlink-categorize-with-claude (2026-06-23)
- Last checked: 2026-10-19

## Negotiate a subscription renewal with a retention desk
- Outcome: Worked
- Category: Money / bills
- Tools: Claude
- Setup: Give the agent a price ceiling and what matters (in-car radio) and let it work the retention chat.
- Access: SiriusXM account chat; card authorization at the end
- HITL checkpoint: Agent paused on its own twice: when the best offer plus tax exceeded the ceiling, and before the card charge to confirm the plan covered the physical receiver
- Failure notes: Renewal went from $29.87 to $8.04/month (incl. tax) for 12 months. Still ~$3/month above the user's ceiling, and the deal reverts after a year, so a renegotiation reminder was set.
- Evidence: User first-hand
- Source: https://damiengoehrig.ca/blog/siriusxm-negotiation-claude/ (2026)
- Last checked: 2026-10-19

## Agent-run shop with better tools and a 'CEO' agent (phase 2)
- Outcome: Partial
- Category: Money / bills
- Tools: Claude Sonnet 4 / 4.5, multi-agent (CEO + shopkeeper)
- Setup: Same shop, upgraded models, CRM/pricing tools, and a supervising agent.
- Access: Same as phase 1, plus additional business tools
- HITL checkpoint: Supervisor agent; humans still physical operators
- Failure notes: Mostly eliminated loss-making weeks and turned a modest profit, but staff could still talk it into risky or unauthorized decisions. Better tools and structure helped more than the extra agent layer.
- Evidence: Lab experiment write-up
- Source: https://www.anthropic.com/research/project-vend-2 (2025-12)
- Last checked: 2026-10-19

## Cancel streaming subscriptions via a browser agent
- Outcome: Partial
- Category: Money / bills
- Tools: ChatGPT agent mode
- Setup: "Cancel my subscription to [service]." One service at a time.
- Access: User logs in to each service inside the agent's controlled browser
- HITL checkpoint: User enters credentials and should confirm before the final cancel click
- Failure notes: Peacock, Disney+ and Netflix cancellations went through. Some sites (Amazon Prime, Google One) block or flag the agent as suspicious, and retention flows are designed to resist bots.
- Evidence: Journalist first-hand
- Source: https://www.tomsguide.com/ai/i-used-chatgpt-to-cancel-my-streaming-subscriptions-heres-how-to-try-it-too (2025)
- Last checked: 2026-10-19

## Let an agent phone or chat providers to lower bills
- Outcome: Partial
- Category: Money / bills
- Tools: Meta Muse
- Setup: Ask the agent to find unused discounts and negotiate cable, phone, insurance, or medical bills by chat or phone.
- Access: Provider account logins; agent can place calls
- HITL checkpoint: Verification codes, signatures, and bank details come back to the user
- Failure notes: Users self-report savings from hundreds to thousands of dollars a year (e.g. Xfinity locked at $85.30/mo). But some providers block it as a bot, Amazon bans it outright, and one call needed the user patched in for an unreadable verification text. Figures are unverified user reports collected in a roundup.
- Evidence: Roundup of self-reported cases
- Source: https://stashbase.ai/blog/meta-muse-negotiate-bills/ (2026-09-22)
- Last checked: 2026-10-19

## Use AI-written scripts to negotiate household bills for 30 days
- Outcome: Partial
- Category: Money / bills
- Tools: ChatGPT
- Setup: Have the model prepare negotiation scripts and target asks; a human makes each call.
- Access: None. The model only sees what the user tells it.
- HITL checkpoint: Fully human-executed. AI only prepares.
- Failure notes: Spectrum and AT&T wins plus a gym credit added up to ~$1,368/year for 2h41m of calls. Geico didn't budge. ChatGPT's price targets were overly optimistic and it didn't know account history (existing promos, late payments), so scripts missed context.
- Evidence: Tester first-hand, 30 days
- Source: https://withn.tech/blog/can-ai-really-negotiate-your-bills-i-tried-it-for-a-month (2026)
- Last checked: 2026-10-19

## Have a model compute a full federal tax return
- Outcome: Failed
- Category: Money / bills
- Tools: Frontier LLMs (Gemini, GPT, Claude)
- Setup: Give the model a realistic taxpayer's documents and ask for a complete Form 1040.
- Access: Tax documents only (benchmark)
- HITL checkpoint: None in the benchmark, which is the point: shows what unsupervised output looks like
- Failure notes: Strict-correct returns (every evaluated line matching) were around a third in the first run and only just over half for the best models in later editions. Models confuse tax years and miss forms. Note the benchmark is published by a tax-software company.
- Evidence: Open benchmark
- Source: https://www.columntax.com/blog/taxcalcbench (2025-07)
- Last checked: 2026-10-19

## Let an agent run a small office shop end to end (phase 1)
- Outcome: Failed
- Category: Money / bills
- Tools: Claude 3.7 Sonnet ("Claudius")
- Setup: Agent manages inventory, pricing, and customer requests for an office snack shop for about a month.
- Access: Email, web search, notes, payments; humans restock physically
- HITL checkpoint: Minimal by design; employees interact as customers
- Failure notes: Lost money (net worth fell from ~$1,000 to under $800): gave away discounts when asked, sold tungsten cubes at a loss, mispriced items, and had an identity-confusion episode. Easy to socially engineer.
- Evidence: Lab experiment write-up
- Source: https://www.anthropic.com/research/project-vend-1 (2025-06)
- Last checked: 2026-10-19

## Keep an AGENTS.md 'harness' that encodes every recurring agent mistake
- Outcome: Worked
- Category: Coding
- Tools: Coding agents (Claude Code, etc.), AGENTS.md
- Setup: Each time an agent repeats a bad behavior, add an explicit rule or script to AGENTS.md (e.g. use the screenshot script, use test filters).
- Access: Repo working tree
- HITL checkpoint: Developer reviews output and maintains the rules
- Failure notes: Adoption came in stages, starting with skepticism. The value came from treating agent errors as harness bugs to fix permanently, not one-off prompt tweaks.
- Evidence: Developer first-hand
- Source: https://mitchellh.com/writing/my-ai-adoption-journey (2026-02)
- Last checked: 2026-10-19

## Migrate int32 IDs to int64 across a giant codebase
- Outcome: Worked
- Category: Coding
- Tools: Google internal LLM migration tooling
- Setup: Locate references, have the LLM propose edits, validate with builds/tests, route to owners for review.
- Access: Monorepo change generation; standard code review
- HITL checkpoint: Every change human-reviewed; many edited before submit
- Failure notes: LLMs generated ~74% of the 595 submitted changes (~69% of edits); ~38% needed human edits and ~26% were human-only. Developers estimated ~50% less migration time. Not a 'set and forget' win.
- Evidence: Peer-style paper (company-reported)
- Source: https://arxiv.org/abs/2504.09691 (2025-04)
- Last checked: 2026-10-19

## Migrate ~3,500 test files from Enzyme to React Testing Library
- Outcome: Worked
- Category: Coding
- Tools: LLM pipeline (Airbnb internal), Jest, TypeScript, ESLint
- Setup: Per-file state machine: refactor → validate (tests, lint, types) → retry with error context; prompts tuned on failures.
- Access: Repo branch; CI validation; no production access
- HITL checkpoint: Engineers reviewed output and manually fixed the last <100 files
- Failure notes: 75% migrated in 4 hours, 97% after 4 days of prompt tuning; finished in 6 weeks versus an estimated 1.5 years by hand. The win came from validation and retries, not one-shot prompting.
- Evidence: Engineering blog (company-reported)
- Source: https://airbnb.tech/infrastructure/accelerating-large-scale-test-migration-with-llms/ (2025-03)
- Last checked: 2026-10-19

## Unattended coding agents that turn a Slack request into a CI-passing PR
- Outcome: Worked
- Category: Coding
- Tools: Stripe Minions (fork of goose), MCP (400+ internal tools)
- Setup: Tag the bot in a Slack thread (or click from a flaky-test ticket); agent runs in an isolated devbox and opens a PR.
- Access: Pre-warmed devboxes isolated from production and the internet
- HITL checkpoint: All PRs human-reviewed; at most two CI rounds before handing back
- Failure notes: Over a thousand merged PRs per week are fully minion-written. Isolation is what lets them skip per-command permission prompts. Runs that aren't fully correct still become starting points for engineers.
- Evidence: Engineering blog (company-reported)
- Source: https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents (2026)
- Last checked: 2026-10-19

## Assign issues to a cloud coding agent in a major OSS repo for 10 months
- Outcome: Partial
- Category: Coding
- Tools: GitHub Copilot coding agent
- Setup: Maintainers explicitly assign issues; agent opens PRs for review.
- Access: Repo PRs only; cannot open PRs on its own
- HITL checkpoint: Maintainer must request each PR and review before merge
- Failure notes: 878 PRs, 535 merged (67.9% vs 87.1% for Microsoft humans), ~95k lines added. Strongest at cleanup and tests, weaker at performance work. Early public PRs drew ridicule and the review burden is real.
- Evidence: Maintainer retrospective
- Source: https://devblogs.microsoft.com/dotnet/ten-months-with-cca-in-dotnet-runtime/ (2026-03)
- Last checked: 2026-10-19

## Use AI coding tools on your own mature open-source repo
- Outcome: Partial
- Category: Coding
- Tools: Cursor + Claude 3.5/3.7 (early 2025)
- Setup: Randomized trial: 16 experienced maintainers, real issues, AI allowed vs. disallowed.
- Access: Developers' own repositories
- HITL checkpoint: Developers fully in the loop
- Failure notes: Developers were 19% slower with AI while believing they were ~20% faster. Time went to prompting, waiting, and reviewing. METR changed its study design in 2026, and later evidence is more mixed, so treat this as context-specific.
- Evidence: Randomized controlled trial
- Source: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ (2025-07-10)
- Last checked: 2026-10-19

## Let a coding agent run Terraform on shared infrastructure
- Outcome: Failed
- Category: Coding
- Tools: Claude Code, Terraform, AWS
- Setup: Agent sets up a new site on existing AWS infra from a new laptop missing the Terraform state file.
- Access: Terraform with production AWS credentials; skip-permissions mode
- HITL checkpoint: User approved a destroy plan without catching its scope
- Failure notes: terraform destroy wiped the DataTalks.Club production DB and its snapshots (2.5 years of submissions); AWS support restored a hidden copy in about a day. Fixes: separate projects, deletion protection, off-resource backups, and the human runs destructive plans.
- Evidence: Founder postmortem interview
- Source: https://rootly.com/humans-of-reliability/hey-claude-wheres-my-database-alexey-grigorev (2026-03)
- Last checked: 2026-10-19

## Reorganize local project files with a CLI agent
- Outcome: Failed
- Category: Coding
- Tools: Gemini CLI, Windows
- Setup: Ask the agent to rename and move a folder of files into a new directory.
- Access: Local filesystem write
- HITL checkpoint: None; agent chained commands without verifying each step
- Failure notes: A failed mkdir was followed by move commands that scattered files; the agent then claimed it had 'catastrophically' destroyed them. The user later found them misplaced at C:\ and withdrew the deletion claim. Agents' self-reports about what happened can be wrong in both directions.
- Evidence: User bug report
- Source: https://github.com/google-gemini/gemini-cli/issues/4586 (2025-07)
- Last checked: 2026-10-19

## Run schema migrations through a coding agent
- Outcome: Failed
- Category: Coding
- Tools: Claude Code, drizzle-kit, Railway Postgres
- Setup: Agent working in a separate terminal session on an app with an ORM.
- Access: Production DB connection string available to the agent
- HITL checkpoint: None for that command; agent used --force to bypass the interactive prompt
- Failure notes: `drizzle-kit push --force` dropped 60+ production tables; no automatic backups, so the data was unrecoverable and ~8 hours of rebuild followed. Pattern repeats with `prisma db push --accept-data-loss` (issue #14411). Deny-list force/accept-data-loss flags and keep prod creds out of agent envs.
- Evidence: User bug report
- Source: https://github.com/anthropics/claude-code/issues/27063 (2026-02-20)
- Last checked: 2026-10-19

## Vibe-code an app with an agent during a 'code freeze'
- Outcome: Failed
- Category: Coding
- Tools: Replit Agent
- Setup: Build a SaaS contacts app with an agent over many days, with explicit instructions not to change anything during a freeze.
- Access: Shared dev/prod database
- HITL checkpoint: Instruction-only ('don't touch'); no technical gate
- Failure notes: Agent ran destructive commands that wiped the production database and then wrongly said rollback was impossible. Root cause: dev and prod shared a database. Replit later added automatic dev/prod separation and a planning-only mode.
- Evidence: Incident + vendor CEO interview
- Source: https://www.fastcompany.com/91372483/replit-ceo-what-really-happened-when-ai-agent-wiped-jason-lemkins-database-exclusive (2025-07)
- Last checked: 2026-10-19

## Daily arXiv digest scored to your interests, delivered to Kindle
- Outcome: Worked
- Category: Research digests
- Tools: Claude Haiku (scoring), Claude Sonnet (summaries), GitHub Actions, Resend
- Setup: Fetch ~250 papers/day → keyword prefilter → Haiku scores 1–10 against a tiered interest profile → Sonnet summarizes top 5–10 → EPUB emailed to Kindle at 5 AM ET.
- Access: arXiv API, Anthropic API, an email sender approved on the Kindle account
- HITL checkpoint: None daily; human tuned the scoring prompt
- Failure notes: First scoring prompt clustered everything at 8–8.5 (57% passed the threshold). Fixed with calibration examples and forced use of the full range. Kindle silently rejects unapproved senders. Runs for about $5–8/month.
- Evidence: Builder first-hand
- Source: https://www.anothercodingblog.com/p/i-wake-up-to-a-custom-ai-research (2026-02-21)
- Last checked: 2026-10-19

## Fire off code-research questions to async agents and get reports back as PRs
- Outcome: Worked
- Category: Research digests
- Tools: Claude Code for web, OpenAI Codex Cloud, GitHub
- Setup: Write a clear research question; agent works unattended in a dedicated research repo, one folder per experiment, with network access for dependencies.
- Access: A separate public (or private) research repo only
- HITL checkpoint: Human reads each report/PR; nothing merges into real projects automatically
- Failure notes: Produced many completed experiments (e.g. Markdown library benchmarks, WebAssembly/Pyodide work). Isolation in a throwaway repo keeps the downside small; questions must be concrete and answerable by running code.
- Evidence: Developer first-hand
- Source: https://simonwillison.net/2025/Nov/6/async-code-research/ (2025-11-06)
- Last checked: 2026-10-19

## Morning Slack digest of yesterday's clipped articles, resurfaced in coding sessions
- Outcome: Worked
- Category: Research digests
- Tools: Agent + Slack, Claude Code
- Setup: Agent reads everything clipped the day before, sends an 8 AM digest of what's worth attention, and surfaces relevant saved articles when a new Claude Code project starts.
- Access: Read-only on the user's clipping store; post to the user's own Slack
- HITL checkpoint: Read-only output to the user only
- Failure notes: Builder-reported with a how-to; no accuracy data. Low risk because it only reads and posts to the owner. Included as a pattern example from X.
- Evidence: Builder self-report (X post)
- Source: https://x.com/diabhey/status/2047798582846337501 (2026-04-24)
- Last checked: 2026-10-19

## Generate a long-form research report on a policy topic
- Outcome: Partial
- Category: Research digests
- Tools: ChatGPT deep research
- Setup: Ask a research question; agent asks clarifying questions, browses for minutes, writes a cited report.
- Access: Public web
- HITL checkpoint: Reader must verify claims; best used in areas the user already knows
- Failure notes: Produced a ~5,000-word cited report in about five minutes and beat Google's equivalent, but contained embarrassing factual mistakes. Useful for deepening work you can check, risky for topics you can't.
- Evidence: Journalist hands-on
- Source: https://www.platformer.news/chatgpt-deep-research-hands-on/ (2025-02)
- Last checked: 2026-10-19

## AI-assisted legal research dropped straight into court filings
- Outcome: Failed
- Category: Research digests
- Tools: ChatGPT
- Setup: Attorney used a chatbot to find supporting case law for motions.
- Access: None; output pasted into filings
- HITL checkpoint: Required cite-check skipped
- Failure notes: Five fabricated citations led a federal judge to reprimand and disqualify three Butler Snow attorneys and refer them to the bar. One of many 2025 sanctions. Any agent-generated citation must be opened and read before use.
- Evidence: Court order + legal press
- Source: https://www.abajournal.com/news/article/5-fabricated-cases-lead-federal-judge-to-kick-3-butler-snow-lawyers-off-case (2025-07)
- Last checked: 2026-10-19

## Agent turns emails and meetings into CRM update recommendations for sellers
- Outcome: Worked
- Category: Sales / CRM ops
- Tools: Salesforce Agentforce Sales Agent
- Setup: Process hundreds of thousands of opportunities in a nightly window; generate cited recommendations for field updates and next steps.
- Access: CRM data with access checks per seller
- HITL checkpoint: Sellers review recommendations; autonomy expanded gradually
- Failure notes: 1.04M monthly recommendations for 13,000 internal sellers. Data integrity required cited reasoning, permission checks, and human review. Vendor dogfooding; no independent numbers.
- Evidence: Vendor engineering blog
- Source: https://engineering.salesforce.com/how-an-ai-crm-system-generated-1m-recommendations-while-maintaining-data-integrity-using-agentforce/ (2025)
- Last checked: 2026-10-19

## Always-on HubSpot dedupe agent for 84,000 live records
- Outcome: Worked
- Category: Sales / CRM ops
- Tools: Claude Code, HubSpot API, Slack
- Setup: Auto-merge high-confidence duplicates; send fuzzy matches to AI review; route uncertain cases to a Slack queue; rescan every 4 hours.
- Access: HubSpot write access (merge) on live data
- HITL checkpoint: Uncertain matches go to a human review queue (944 records at first)
- Failure notes: Merged 2,500+ duplicate contacts/companies in about 4 hours for ~$10. The builder (a non-engineer) had a near-miss moment before running across 84k live records; the review queue was too big to handle by hand, which drove the AI-review tier. Open-sourced.
- Evidence: Operator first-hand
- Source: https://agentoperator.substack.com/p/i-built-a-hubspot-dedupe-tool-in-a-day (2026-04-07)
- Last checked: 2026-10-19

## AI-only vs. human vs. hybrid cold email (12,000 sends)
- Outcome: Partial
- Category: Sales / CRM ops
- Tools: LLM copy generation, Saleshandy
- Setup: Three cohorts: AI-written (5k), human-written (2k), AI-drafted + human-edited (5k).
- Access: Sending domains and prospect lists
- HITL checkpoint: Hybrid cohort: a human edits every AI draft
- Failure notes: AI-only got 0.7% meetings and a 7.8% spam-flag rate versus 3.2% meetings and 3.1% spam for hybrid. Fully automated copy hurt deliverability and reply quality. Vendor-run test.
- Evidence: Vendor experiment
- Source: https://www.saleshandy.com/blog/ai-vs-human-cold-emails/ (2026)
- Last checked: 2026-10-19

## Replace SDRs with a fully autonomous AI SDR product
- Outcome: Failed
- Category: Sales / CRM ops
- Tools: 11x (AI SDR)
- Setup: Buy an 'AI digital worker' to run outbound prospecting end to end.
- Access: Email domains, CRM, prospect lists
- HITL checkpoint: Marketed as autonomous
- Failure notes: TechCrunch reported the vendor listed customer logos (e.g. ZoomInfo, Airtable) of companies that weren't customers, and sources described churn of 70–80% with ARR inflated by contracts carrying 90-day break clauses. The vendor disputed parts. Ask for references and trial outs.
- Evidence: Investigative reporting
- Source: https://techcrunch.com/2025/03/24/a16z-and-benchmark-backed-11x-has-been-claiming-customers-it-doesnt-have/ (2025-03-24)
- Last checked: 2026-10-19

## Weekly newsletter + blog post agent with a strict approval packet
- Outcome: Worked
- Category: Content pipelines
- Tools: Hermes agent, Tavily, MailerLite, GitHub + Vercel
- Setup: Research last 7 days → reject press-release hype → draft in voice from approved examples → evaluator pass → approval packet.
- Access: MailerLite campaign creation; website repo
- HITL checkpoint: Approval packet shows copy, source URL per claim, subject, campaign status, blog metadata, and the exact pending action. A vague 'looks good' doesn't count.
- Failure notes: Version 1 could research, draft, and send on its own, and the author judged that the wrong design. Quality depends more on what's rejected than what's generated. State must live outside chat history.
- Evidence: Practitioner first-hand (also sells builds)
- Source: https://mark-little.com/writing/newsletter-agent (2026-05-20)
- Last checked: 2026-10-19

## One podcast episode → show notes, clips, posts, newsletter section
- Outcome: Partial
- Category: Content pipelines
- Tools: n8n, Gemini (audio), Telegram
- Setup: Watch the RSS feed; Gemini listens to the full audio and drafts show notes, timestamps, 5 clip ideas, LinkedIn post, X thread, and newsletter blurb in the host's voice.
- Access: RSS feed, model API, Telegram bot, n8n Data Table
- HITL checkpoint: Each piece is sent to Telegram with Approve / Edit / Skip; nothing ships without a tap
- Failure notes: Builder-reported. Retries plus a backup model handle API busy errors. No published quality or engagement data, and the author sells it as a paid template, so treat it as a pattern, not a proven result.
- Evidence: Builder self-report (sells template)
- Source: https://community.n8n.io/t/podcast-repurposer-1-episode-6-ready-to-post-pieces-approved-from-telegram/318537 (2026-10-03)
- Last checked: 2026-10-19

## AI-written syndicated 'summer reading list' insert
- Outcome: Failed
- Category: Content pipelines
- Tools: Generic AI chatbot
- Setup: Freelancer used AI to produce a syndicated summer guide, including a reading list.
- Access: None; output went to a syndication partner
- HITL checkpoint: Fact-check skipped at the writer, syndicator, and publisher
- Failure notes: 10 of 15 books on the list didn't exist (fake titles attributed to real authors), and it ran in print in the Chicago Sun-Times and others. Later review found more errors in the section. Every name, title, and quote needs a source.
- Evidence: Publisher statement
- Source: https://chicago.suntimes.com/news/2025/05/20/syndicated-content-sunday-print-sun-times-ai-misinformation (2025-05-20)
- Last checked: 2026-10-19

## Auto-generated video recaps for TV seasons
- Outcome: Failed
- Category: Content pipelines
- Tools: Amazon Prime Video AI recaps
- Setup: Model generates narrated season recaps from show content.
- Access: Published to the streaming catalog
- HITL checkpoint: Not evident before launch
- Failure notes: The Fallout recap misplaced a 2077 flashback in '1950s America' and garbled a key plot choice. Amazon pulled AI recaps from Fallout and the other test series. Narrative summaries need a fan-level fact check.
- Evidence: News reporting
- Source: https://www.theverge.com/news/842978/amazon-prime-video-ai-fallout-recap (2025-12)
- Last checked: 2026-10-19

## Mass-produce personal finance explainers with an AI engine
- Outcome: Failed
- Category: Content pipelines
- Tools: CNET internal AI engine
- Setup: AI drafts SEO explainers; editors review before publish.
- Access: CMS publishing
- HITL checkpoint: Editor review existed but missed errors
- Failure notes: After outside reporting, CNET's audit corrected 41 of 77 AI-written stories (factual errors, plagiarism-like phrasing). It paused the program and rewrote its AI policy. 'Human reviewed' is not a gate if reviewers skim.
- Evidence: News reporting + publisher audit
- Source: https://www.theverge.com/2023/1/25/23571082/cnet-ai-written-stories-errors-corrections-red-ventures (2023-01-25)
- Last checked: 2026-10-19