Home / Work / From Customer Calls to Drafted Roadmap Ideas

From Customer Calls to Drafted Roadmap Ideas

Internal Tooling · 2026

Thousands of customer calls, a roadmap of one-liners. Now 89 roadmap ideas carry an AI-drafted problem, evidence, assumption and experiment with a certainty level, and new opportunities surface weekly.

  • 89Roadmap ideas with a drafted block
  • 2,200+Calls analysed, 1 to 12 Nov
  • 715Insights, milestone 1
  • 48Themes, milestone 1
  • Product Discovery
  • Jira Product Discovery
  • Python
  • Claude
  • Agentic Automation
  • RICE
  • Customer Insights
From Customer Calls to Drafted Roadmap Ideas

What happened

Product discovery at a Dutch home-services platform ran on gut feel and whoever spoke last. Ideas landed in Jira Product Discovery as one-liners. The evidence was there (thousands of recorded customer and partner calls, KPIs in BigQuery, CRM data), but nobody had time to connect it to the backlog.

I made automated discovery my first priority for the quarter. The goal: every idea on the roadmap automatically gets an AI-drafted first version, written into the idea itself. That means a problem statement, the evidence for it, the key assumption and a suggested experiment, each with a source, a date and a certainty level.

A second loop runs in the other direction. Recurring pain in the calls that matches no idea yet is proposed to me as a new opportunity each week. I approve or reject it with a one-line reply.

The bar for "done" is strict: four weeks in a row unattended, with at least five discoveries processed.

How I built it

The loop has four parts, each with one job:

  1. Call Insights (evidence layer). A Python pipeline reads call transcripts and filters them to the analysable ones, accounting for every excluded call. It extracts insights per chunk and groups them into themes against a versioned taxonomy. Severity is sampled three times and decided by majority vote. Ranking is deterministic: Pain Score = Reach × mean Severity. No model ranks anything; that is a written architecture rule. Tests prove that reports never contain a quote or a contact identifier, and data is written only through database functions.

  2. Insight Run. The scheduled run of that pipeline. Since 31 October it is a Windows task on my laptop (weekdays 09:00) instead of a CI job. It works through calls in batches of 150, up to eight batches per start, and wipes its model cache and reports afterwards, so no customer text stays on the laptop. Only the high-water mark is kept.

  3. Discovery Enricher. A Claude cloud routine, every weekday morning. It appends a fixed six-line block to each open roadmap idea: problem, evidence, assumption, experiment, source and date, certainty. Its only allowed write is the idea's description. The evidence rules are explicit:

    • never invent evidence
    • say when data is old
    • no names, quotes or euro amounts
    • "high" certainty needs two or more sources no older than four weeks
    • competitor-only evidence caps certainty at "medium"

    Ideas already in delivery get a delivery check instead of an experiment.

  4. Discovery Scout. A weekly Claude cloud routine. It matches the week's call themes to the roadmap (supports an idea, came back after delivery, was rejected before, or is new) and mails me at most three unmatched opportunities. My "1 yes, 2 no" reply is recorded back in the database. It has no write access to Jira at all.

Alongside it, a RICE review skill scores ideas with one reviewer agent per idea plus a controller agent. The controller checks consistency across ideas: the same population must mean the same reach, and nothing may be counted twice. It runs only when I ask, and it writes scores to Jira only after my OK.

The design choices behind the loop (one loop, unattended writes for enrichment, human approval for new ideas) are recorded in my decision log, so every routine can be traced back to a decision.

Privacy by design

The pipeline builds on an earlier one-off. A colleague needed the customer-service call history as an analysable dataset, so I exported 71,899 calls with 50,569 Dutch transcripts from JustCall. Before handoff, a scrubbing pass over every record removed about 660,000 phone numbers and about 240,000 person names, including names found inside the spoken text, and the output was checked for record counts and validity.

A scrub cleans one dataset. The discovery pipeline decides up front instead: transcripts are kept for 12 months (plus at most 7 days in backups), verbatim quotes are deleted on the same schedule, only an allowlisted role can read quotes, model-written text is redacted when it is rendered, and tests assert that no report contains a quote or a contact identifier.

Outcomes

Status (13 November): the four-week unattended clock ends at the earliest around 30 November. CRM, BigQuery and competitor data are designed in but not yet connected. For now the evidence is call data plus roadmap history. On 12 November the Insight Run stopped early: it started 70 seconds after a reboot, before the network was up. The script now waits for the network, keeps the laptop awake during batches and retries a failed batch once.

What I learned

Pay-per-minute CI and volume-driven work don't mix. The Insight Run alone needed about 2,090 CI minutes a month against a 2,000-minute free tier. Minutes follow call volume, not how often it runs. The account ran out on 22 October, and the run moved to a scheduled task on my laptop. The catch: the laptop has to be on, weekend calls arrive on Monday, and a laptop restarts at awkward moments. A watchdog now flags a failed run or four days without one.

Measure on real data before you budget. The first real runs gave far more insights per call than I had assumed (about 3.7 per call at the 26 August milestone), so the cost estimate had to be redone.

The deployed prompt must equal the repo file, byte for byte. A leftover test override made the Scout report the same week twice.

Automation needs to see what's already shipped. The Enricher proposed an experiment for something that was already live. That is why ideas in delivery now get a delivery check.

Be sceptical about euro values. One idea's value was halved once its mechanism turned out to be wrong. The rule now: euros only when every factor is measured, otherwise the KPI it moves plus an order of magnitude.

Honest certainty is a feature. Every block still reads "low" or "medium" because call data lags. That is the correct answer, and it shows where better data would pay off.

How I did it

Solo with Claude Code, as Product & AI Lead: I designed the loop, built the pipeline and routines, and own the roadmap it feeds. The pipeline was built ticket by ticket from 49 MVP tickets (47 built, 2 deliberately not). The Enricher got three dry runs before it went live, and every routine prompt is versioned in git with a kill switch. A watchdog flags any scheduled run that didn't happen. I keep the decisions; the agents do the reading, matching and drafting.