A single undifferentiated "use AI more" message is not merely inefficient: for part of your population it is a measurable instruction to go slower. This is the part clients most often say they had never considered.
| Study | Population and task | Result |
|---|---|---|
| Peng et al. (2023) | ~95 developers, greenfield HTTP server | 55.8% faster (95% CI 21–89%) |
| Paradis et al. (2025) | ~96 Google developers, realistic C++ task | ~21% faster, not significant once proficiency and familiarity were controlled |
| METR (2025) | 16 experts, 246 tasks, their own mature repositories | 19% slower, while believing they were 20% faster |
Junior and early-career (0–3 years)
Where the gain is: unfamiliar codebases, boilerplate, test scaffolding, explanation and onboarding. Where the risk is: accepting output they cannot explain, losing the struggle that builds judgement, and duplication because they cannot see the existing abstraction.
- Explain before accept — be able to say what it does, why it belongs there, and what breaks if it is wrong.
- Search before generate — the main defence against duplication.
- Tests first on AI-assisted work.
- Declare provenance honestly; it is never traced to an individual.
- One deliberate unassisted exercise per sprint in a core area, reviewed by a senior. This is the skill-atrophy control.
Senior (4–10 years)
This is the METR population. On code they know deeply, assistance may slow them — and they will feel faster regardless. The gain is at the boundaries: unfamiliar modules, migrations, legacy comprehension, breadth work, and above all review leverage.
- Deliberate opt-out on deep work in their strongest area — and make opting out explicitly legitimate, because most organisations have accidentally made it career-risky.
- Assist at the boundaries, where the same person gets genuine speed.
- Own the verification system: push mechanical checks into CI so human review buys architecture and edge cases.
- Architecture fit as the first review question on generated code.
- Write down what only they know — ADRs, interface docs, examples — so assistants can ground on it.
Staff and principal
- Context engineering as a product: accessibility, freshness and quality of what assistants ground on.
- Guardrail design — decide what is enforced in CI versus expected in culture. Enforced beats expected.
- Use-case selection pointed at user-relevant problems.
- Architecture for verifiability: generated code is cheap to produce and expensive to verify.
- Agentic blast radius: scope, permissions, review gates, rollback, audit trail.
Engineering managers
- Never set an AI usage target.
- Watch the review queue weekly — it is the first place gains disappear.
- Protect the J-curve: tell your leadership in advance that the first weeks may look worse, and why.
- Read the counter-metrics before the speed metrics.
- Make opting out safe, and say so out loud.
- Run the monthly provenance audit, or make sure someone does.
Use-case sequencing by verification cost
| Tier | Use cases | Prerequisites | Typical first value |
|---|---|---|---|
| 1 · Start here | Test generation, documentation, code explanation, PR summaries, boilerplate | Acceptable-use policy; provenance declaration | Fast, safe, builds habit and trust |
| 2 | Feature work in well-tested modules, bug fixes with reproductions, migration mechanics | Trusted test suite; review SLO; PR size discipline | The main delivery gain |
| 3 | Legacy comprehension and grounded retrieval over large codebases; cross-service change | Context accessibility; senior review capacity | High value on old monoliths — the case where evidence is most favourable |
| 4 | Autonomous agents raising changes; AI-assisted review as a gate; ops automation | Governed delivery: enforced guardrails, audit trail, provenance, rollback | Only after tiers 1–3 are measured and stable |
Do not promote a use case to the next tier until the current tier shows no degradation in change failure rate, churn or review depth — and a measured benefit. Written as a gate, this is the single most effective thing we do to stop enthusiasm outrunning the verification system.
Printable role cards
Junior
Explain before accept · Search before generate · Test first · Declare provenance · One unassisted exercise per sprint
Senior
Opt out on deep work, assist at the boundaries · Review for architecture fit · Push mechanical checks into CI · Write down what only you know
Staff
Make context accessible · Enforce guardrails in CI · Choose user-relevant use cases · Architect for verifiability
Manager
No usage targets · Watch the review queue · Protect the dip · Counter-metrics first · Make opting out safe