Explainer Randy's PM Frameworks

Measurement Plan Rubric — Measuring One Feature After Release

Your three-part rubric for planning, and grading, how one feature is measured after release: the parts and points, your testing-language forms, the signal table, how it grew over six versions, your practice scores, and its job inside AI Agent for PM.

Generated
November 18, 2026, 3:02 PM MT
Built from
  • wiki/concepts/measurement-plan-rubric.md
  • wiki/sources/measurement-plan-rubric.md
  • wiki/entities/ai-agent-for-pm.md

A rubric for planning how to measure one feature after release, and for grading practice answers. It's the detailed version of step 5 ("Measure + Exit") in your PM decision framework, and the spec for the Measurement Plan Agent in AI Agent for PM. Built over two days of practice (2026-11-16/17) and rewritten into your testing language.

Here's why it should work, here's what I'll watch and what happens if it misses, and here's how I'd know I wasn't fooled.

Scope: one feature, not a group (v5.1)

One plan per bet with one hypothesis. Small fixes shipped together to one flow under one hypothesis count as one feature. Themes, OKRs and KPIs are measured directly at their own level, never by adding up feature results. Each plan names its link upward: "Contributes to: [key result or KPI]."

v5.1current version
3parts: A · B · C
20 / 50 / 30points per part
8signal table columns
6versions in two days
3effort levels

Three parts, 100 points

v1's six areas (top) regrouped in v4 into three parts (bottom). Same checks, same total. Parts are scored as sub-items so individual gaps stay visible.

A · Hypothesis 20

Why should it work? A Gherkin line from pain and workaround to new behavior and signal, plus "Contributes to."

B · Signal table 50

What do I watch, from where, to what bar, by when, and what if it fails?

C · How could I be fooled? 30

C1 by something else: with/without, rule out a cause, fingerprint. C2 by the feature: negative scenario, coverage gap, AI default.

Your forms, in testing language

Each original area has a form you already know from QA. Ask "how could this fool me?" twice: once about the feature (guardrails) and once about something else (attribution).

Given [pain + workaround]
When [feature]
Then [new behavior]
… which means we should see [signal]
#AreaWeightYour form
1"Because" and chain20%Given / When / Then … which means we should see [signal]
2Baseline and data sources20%Background: today [value] from [source] over [window]; if missing, logging, a task test or a hand sample
3Prediction and exit15%Then within [time], [metric] from [baseline] to [target]. Pass → expand or next stage; Fail → rework, pause, roll back or stay
4Leading vs. lagging, timing15%Signal table: Early / Outcome / Long-term rows, each with baseline, expected change and check date
5Attribution15%Scenario with vs. without (same weeks), rule out one cause, fingerprint
6Guardrails and blind spots15%Negative scenario counted with a threshold and action; coverage gap; AI default

Part B: the signal table

Read across: "For [who/what], I count [signal], from [source]. Today it's [baseline]. It's an [Early/Outcome/Long-term] signal, checked by [date]. It passes if [bar]. If it fails, we [action]."

Who / what vs. From

  • Who / what = whose number: a person, a group, the product or system, or money.
  • From = where you look it up. Artifacts (tickets, transcripts, logs) are sources, not subjects.

Baseline and Pass if

  • Baseline: the value before the feature, measured the same way and over the same window as afterward. "New" if the signal only exists with the feature, judged against Pass if.
  • Pass if: an acceptance threshold set by ceiling, break-even or worth keeping, locked before outcome data. A wrong bar is fine.

Counting rules

  • Count per unit of work.
  • User-facing features get the adoption funnel.
  • Small numbers give direction, not precision; track per account in B2B.
  • Doubt the feature? Add the doubt as a diagnostic row.
The adoption funnel as a template, with no numbers yet: each step becomes an Early row for a user-facing feature.

Match the effort to the stakes

LevelWhat you do
LightPart A plus one Outcome row
MediumA, a full B, and C1 with a staged rollout
HeavyAll parts, a holdout, a 30–90 day review

The level depends on cost, reversibility, uncertainty and whether the result will be repeated as a claim.

Filling it in

Skeleton first

Write a rough line for every part before going deep. An empty area scores 0.

Defaults, not invention

  • Exit rule: "if [metric] < [X] by [date], we [rework / pause / roll back / stay]."
  • Comparison: holdout → staged rollout → own before plus steady total volume.
  • "From ___" after every metric.

AI default, always

A pass doesn't mean correct. Grade a sample for correctness, and track reverts and cost per successful output.

In AI Agent for PM

The Measurement Plan Agent runs after Review and before build, because the baseline has to exist before launch. The verdict goes to you only; it never feeds into building or scoring other features.

Light check: every feature

The Gherkin line, the feature's tags and "Contributes to"; one Outcome row, preferably a tag-level signal the pipeline already tracks; one negative scenario. Mostly template plus code; the LLM drafts only part A.

Full plan: exceptions only

Strategic bets, AI features, or results Sales or Marketing will repeat as a claim. The Measurement Plan Checker Agent scores it against the rubric and confirms, flags or rejects, never rewrites.

Data is a boundary

Missing or unvalidated data means "not measured." The agent can only suggest how to measure it, filed into intake as a new candidate feature. No invented numbers.

How it got here

VersionDateWhat changed
v5.1Nov 17Scope rule: one feature, not a group; "Contributes to" line
v5Nov 17Part B reworded (Who / what, Baseline, From, Pass if); restored effort levels, funnel, per-unit and small-number rules
v4Nov 17Six areas regrouped into three parts with sub-item scoring
v3Nov 17Rewritten in your testing language
v2Nov 17"Filling it in": skeleton, six questions, defaults, practice ladder
v1Nov 16Six areas and weights, templates and full example lines

Dropped by v5's value rating: the six-question list and the practice ladder.

Your practice log from the wiki. The fifth round was the first on the three-part rubric; next up are scaffolded rounds on v5.1.

Key principles