What happened
The product roadmap at a Dutch home-services platform lived in Jira Product Discovery: 145 ideas collected since October 2023, spread over six products. Many were one-liners. Some were done but still marked open, some described work that had moved to another system, and almost none said what evidence backed them. Ahead of a management roadmap session on 22 October 2026 I had promised one overview of all items and the value each might represent.
In four days I turned that list into something a management team can decide on. AI agents reviewed every idea for clarity, mapped it to a product and sized it. The open ideas then got an evidence check and a RICE score, from one reviewer agent per idea plus a controller agent that kept the scores consistent. The scores went into Jira Product Discovery as fields and a board view, and that view, not a slide deck, was what the management team worked from in the session.
How I built it
1. Map and clarity check. I exported all 145 ideas and had subagents review them in batches. Each idea got one main product (plus secondary ones), inferred dependencies on other ideas and on system milestones, a t-shirt size taken from the development team's own sizing table with a similar past epic as reference, and a clarity verdict. An idea that was not clear enough to rank got the note "not clear enough to rank, needs more detail" and no size or value, instead of a guess. The first version was a network map. At 145 nodes it was unreadable, so it became a board with one column per product and cards grouped by status.
2. Evidence check. One agent per five open ideas, with one question: is there evidence about the effect of this idea, not just about the problem? Each idea got a verdict (supported, partly supported or none), plus what is missing and how old the evidence is.
3. RICE review with a controller. A scripted workflow ran one reviewer agent per open idea. Each agent read the idea and its comments in Jira, worked out which behaviour would change and which business lever it moves (supply, sales, quality, retention, new business, efficiency, risk), and scored Reach, Impact, Confidence and Effort with a short explanation. Then a controller agent read all outputs together and corrected them: the same population must mean the same reach, the same effect the same impact, confidence is about the effect, and related ideas must not be counted twice. It wrote its corrections and the overlaps between ideas as notes for me. I reviewed the changes per idea and decided on the doubtful ones.
The scoring rules that came out of this:
- Confidence: 100 tested, 80 data about the effect, 50 partial, 20 none. Data that only proves a problem is not evidence of an effect, and a vendor or competitor claim counts as 50 at most.
- An impact of 0 when an idea was tested without effect.
- A strategic factor for bets RICE cannot measure, and labels for maintenance and temporary work instead of forcing them into the ranking.
- No money amounts unless every factor is measured. The cards show the expected KPI effect and an order of magnitude instead.
4. Into Jira Product Discovery. I cleaned up the product field to the six products and reused existing fields where possible. New fields hold the lever and the RICE inputs, a formula field holds the score, and there are two views: a board (columns per product, rows per status, sorted by score) and a RICE matrix. One subagent wrote all fields from a plan file. Jira is the source of truth; the page in my own workspace is a readable mirror.
5. Preparing the session. A short, neutral email to the management team with a link to the board view, and no recommendations. On the morning of the session I answered a commercial lead's eight written questions on the roadmap in the thread, and turned a department head's written input into four new scored ideas.
The procedure is now a reusable skill for the next scoring round, and the product discovery routines feed new evidence into the same ideas.
Outcomes delivered
- 145 ideas mapped to six products, with 108 inferred dependency links and t-shirt sizes, on 19 October 2026.
- 60 open ideas reviewed for clarity, 17 flagged as not clear enough to rank instead of being guessed.
- 49 ideas evidence-checked: 1 supported, 14 partly supported, 34 with no evidence about the effect.
- 43 ideas RICE-scored by one reviewer agent each plus a controller, then written into Jira Product Discovery with a board view and a RICE matrix.
- One board view for the management session on 22 October 2026, plus four new scored ideas from written input on the same morning.
- A backlog hygiene list: stale statuses, empty ideas, Jira example ideas and naming issues reported, not silently changed.
What I learned
Evidence about the problem is not evidence about the solution. 34 of 49 ideas had no evidence about their effect. Many did have evidence that the problem exists, for example from customer calls. Separating the two changed the ranking more than any other rule.
A test beats a claim. One idea, a tracking change, came with a vendor's own conversion claim. A test showed no effect. That example did more for the "test first" rule than any slide.
One agent per idea is not enough on its own. Independent reviewers score the same population differently and count the same effect twice. The controller pass, which only compares, fixed more than any single review.
Money values invite false precision. The first version showed euro ranges. They rested on unmeasured assumptions, and one turned out to be current revenue, not extra value. KPI effects with an order of magnitude are less impressive and more honest.
"Not clear enough to rank" is a valid answer. Flagging 17 ideas as unclear kept them out of a ranking they would have distorted, and gave their owners a concrete question.
Neutral prep lets the room decide. The board showed scores, levers and evidence, no recommendations. The decisions were the management team's.
Network maps do not scale to 145 nodes. A board per product with dependency chips was readable; the graph was not.
How I did it
Solo with Claude Code over four days before the session (19 to 22 October 2026): subagents for the batch reviews, a scripted workflow for the per-idea RICE review and controller, Node scripts to rebuild the overview page from JSON, and one subagent to write the scores into Jira from a plan file. Every agent output was a draft; I checked each change before it went into Jira. Decisions were logged with their assumptions in a decision log.