The point is not the quarter's number. It is that the organisation stops re-arguing the definition and starts arguing about the system.
The handover test
By the second review the client presents and we challenge. If we are still presenting at the third, the capability has not transferred — and that is our failure, not theirs.
Standing agenda — 90 minutes
| Time | Item | Note |
|---|---|---|
| 0:00 | Scorecard walk | Numbers first, interpretation second |
| 0:20 | Movement explained | Each material movement mapped to the intervention register or to a confounder |
| 0:35 | Attribution update | Provenance coverage and audited error rate; repeat the self-selection caveat every quarter |
| 0:45 | Counter-metric review | If any degraded while speed improved, this item takes the rest of the meeting |
| 0:55 | Maturity re-score | Same rubric version only |
| 1:05 | Value model update | Range before point estimate |
| 1:15 | Next quarter's interventions | Pre-registered with expected movement |
| 1:25 | Risks and asks |
The four questions
- Did delivery get better against our own baseline — and what did it cost?Speed and counter-metrics together, never separately.
- Where is AI helping, where is it not, and how confident are we?With the self-selection caveat and the provenance error rate attached.
- Where are we on the J-curve, and what is shortening the trough?Position on the curve is a legitimate quarterly answer.
- What is the constraint now?The constraint moves. A quarter that does not name a new one has probably stopped measuring.
Quarterly maintenance
- Re-run all queries; reconcile one known deployment end to end
- Provenance hand audit of 30–50 PRs; update the error rate
- Confounders register updated with the quarter's named events
- Developer-experience survey re-run (half-yearly is usually right)
- Intervention register closed out: expected versus observed
- Definitions unchanged — or history restated and logged
- Evidence register reviewed: has a cited source published a new edition?
Why that last item matters
DORA's finding on AI and delivery throughput reversed between its 2024 and 2025 reports. A deck quoting the older position is not dated — it is wrong.
Anti-patterns
| Anti-pattern | Why it is fatal | Fix |
|---|---|---|
| The consultant presents | The cadence dies when we leave | Client chairs and presents from the second review |
| Only improvements shown | Credibility spends itself once | Standing counter-metric slot |
| A new metric appears to show progress | Silent redefinition | Change control; restate history |
| Movement attributed to AI by default | Ignores the register and the confounders | Mapping step before attribution |
| Usage numbers presented as value | The exact failure the engagement existed to correct | Keep adoption in a diagnostic annex |