If a metric is not defined here, it does not go in a client scorecard. Definitions are frozen by the telemetry contract: changing one requires an amendment, both signatures, and a restatement of affected history.
Conventions
Median and p85 for every duration metric, because delivery durations are right-skewed and a mean describes outliers. Every figure carries n, the window and a query ID. Minimum reporting unit is the team; minimum survey cell size is five.
Delivery outcomes — the four keys
| ID | Metric | Direction | Counter-metric | Why it is in the set |
|---|---|---|---|---|
| M-01 | Deployment frequency | Higher | M-03 | Throughput and stability improve together in high performers; one without the other is not a finding |
| M-02 | Lead time for changes | Lower | M-12 | The outer-loop metric where inner-loop AI gains most often fail to appear |
| M-03 | Change failure rate | Lower | M-01 | The early warning for the instability tax — the metric DORA finds moving the wrong way under AI adoption |
| M-04 | Failed-deployment recovery time | Lower | M-03 | Resilience under increased change volume |
Flow — where the constraint lives
| ID | Metric | Direction | Note |
|---|---|---|---|
| M-05 | Time to first review | Lower | AI-assisted pull requests are reported to wait several times longer for first review |
| M-06 | Review wait share of lead time | Lower | Converts review latency into a share of a metric executives already track — which is how the fix gets funded |
| M-07 | Review depth | Watch | Counter-metric, never a target. Latency falling while zero-comment approvals rise is rubber-stamping |
| M-09 | WIP (open PRs per engineer) | Lower | Queueing effects dominate delivery time; AI raises arrival rate |
| M-10 | CI queue wait | Lower | Platform quality decides whether AI amplifies organisational performance |
| M-12 | PR size | Lower | The main driver of review latency and shallow review |
Quality and maintainability counter-metrics
| ID | Metric | Direction | Note |
|---|---|---|---|
| M-13 | Code churn rate (21-day) | Lower | Churn roughly doubled across the AI adoption period in a 211-million-line analysis |
| M-14 | Duplicated block rate | Lower | Duplication is deferred cost that no velocity metric will show you |
| M-16 | Defect escape rate | Lower | Production defects per 100 merged changes |
| M-18 | Test suite flake rate | Lower | The control system that keeps AI volume from becoming instability |
AI attribution
| ID | Metric | Direction | Note |
|---|---|---|---|
| M-25 | AI-assisted share of merged changes | Reported, never targeted | Adoption is an input; rewarding consumption produces consumption |
| M-26 | Lead time, AI-assisted vs not | Lower | The central question — and observational, because engineers choose which work to use AI on |
| M-27 | Change failure rate, AI-assisted vs not | Lower | Localises the instability finding |
| M-28 | Verification load per AI-assisted PR | Watch | The verification tax, costed rather than absorbed silently |
| M-29 | Provenance coverage and audited error rate | Higher / lower | Publishing the error rate is what separates measurement from theatre |
Metrics we deliberately exclude
| Excluded | Why |
|---|---|
| Lines of code, commits per engineer | Activity metric used in isolation; AI makes it actively misleading |
| Token consumption, prompt counts, AI leaderboards | Rewarding raw consumption is a documented trap |
| Suggestion acceptance rate as a success measure | Measures tool behaviour, not delivered value |
| Story points and velocity | Not comparable across teams or time; trivially gamed |
| Anything at individual level | Destroys data quality and trust, permanently |