Engineering Practices That Make AI Work
Governance sets the frame: what an agent is allowed to do. This page is the engineering: the eleven practices that decide whether an agent's output is actually good once it is allowed to touch the codebase. DORA's seven-capability model says four of them are ordinary discipline that predates AI - and AI amplifies whichever you already have.
Governance Is the Frame. This Is the Engineering.
Everything upstream of this page - policy, ownership, logging, data classification, vendor evaluation - decides what an agent may do. It says nothing about whether the work is good. That is an engineering question, and it is answered the same way it always was: with discipline, just applied to a faster, less predictable collaborator.
DORA's 2025 AI Capabilities Model names seven organizational capabilities that turn AI adoption into measured performance: a clear and communicated AI stance, healthy data ecosystems, AI-accessible internal data, strong version control practices, working in small batches, a user-centric focus, and quality internal platforms. Four of the seven are ordinary engineering discipline that has nothing to do with AI. AI does not create good engineering - it amplifies whichever you already have.
Jump to a Practice
Context Engineering, In Depth
An agent does not know your codebase, your conventions, or your risk tiers unless you tell it, in a file it reads every session. Context engineering covers the wider practice; this section goes deeper on the one artifact that carries most of it - the repo-root instruction file.
One File, Many Names
Every repo an agent touches needs an instruction file at its root: CLAUDE.md for Claude Code, AGENTS.md for Codex, Cursor, Copilot, Devin, Jules, Amp, and most others. The convention is stewarded by the Linux Foundation's Agentic AI Foundation and read by more than 60,000 open-source projects, per AGENTS.md. Claude Code has read AGENTS.md natively since September 2026 when no CLAUDE.md exists, so a single canonical file is now practical even in a mixed-tool repo.
What Belongs In It
How to build and test, the conventions that are not obvious from the code, the architectural rules, what never to touch, where the risk tiers live, and how to write a commit and a PR. Keep it short: Anthropic's best-practices guide tests every line the same way - would removing it cause a mistake? If not, cut it. A 2,000-line instruction file gets ignored the same way a 2,000-line README does.
The Security Baseline
Include the security rules a model gets wrong most: input validation, output encoding, parameterized queries, no secrets in code, dependency pinning. OpenSSF's Security-Focused Guide for AI Code Assistant Instructions (08-2025) is a ready-made starting block. In a monorepo, keep a root file for the universal rules and a package-level file for the local ones.
Instruction Files Are Code. Hooks Are Enforcement.
Instruction files are reviewed, versioned, CODEOWNED, and owned like any other source file. Rot here is prompt drift at the organization level, and a poisoned instruction file sits on OWASP's Top 10 for Agentic Applications.
But instruction files are only advisory. Hooks and permission settings are enforcement: pre-commit hooks that run the linter, pre-tool hooks that block destructive commands, deny lists, and allow lists for the tools an agent may call without asking. Put every never-event in a hook, not just in prose that an agent can talk itself past.
Give agents access to internal data - ADRs, runbooks, API docs, ticket history - through governed connectors, not by pasting it into a prompt. This is DORA's "AI-accessible internal data" capability in practice.
The Ticket as a Contract
A ticket a machine can test against is the entire unlock. Tests-first generation, agent implementation, and compliance evidence all depend on it existing. Upgrade ticket output from prose to a contract with eight required fields.
An ambiguity check against the source requirements runs before the ticket is accepted. A full template lives on Templates and Checklists.
Plan First, Then Diff
Approve the plan, not the diff. An agent reads the repo and produces an implementation plan - files, interfaces, migrations, flags, test surface, blast radius, and rollback - tagged with a risk tier. A senior human approves or revises that plan. Only then does implementation start.
Reviewing a plan takes about five minutes and catches architectural mistakes. Reviewing a fifteen-hundred-line diff takes about an hour and mostly catches typos.
Plan approval is where senior judgment scales - diff review by itself does not.
Tests First, Mutation Scored
Generate the tests from the acceptance criteria before the implementation exists. File them where QA can see them. Then implement until they pass.
Coverage will not catch the actual AI failure mode: tests that look thorough and assert nothing. Mutation testing will - it mutates the code and checks whether the tests notice. Tools: Stryker for JavaScript, TypeScript, and .NET; PIT for Java; mutmut for Python; cargo-mutants for Rust. Put a mutation-score gate on the modules that matter - money math, auth, rule engines, adapters. It doubles as a quality proof for auditors.
The Evidence
TestGenEval, ICLR 2025
The leading model at the time reached 35.2% line coverage but only an 18.8% mutation score; other frontier models scored similarly. arxiv.org/abs/2410.00752
MUTGEN, IEEE TSE
LLM-generated suites reached "100% coverage but only 4% mutation score." Feeding the surviving mutants back to the model as a prompt fixed most of it. arxiv.org/abs/2506.02954
Meta's ACH, 2025
Mutation-guided hardening generated 571 tests; engineers accepted 73% of them, because each one was proven to kill a specific mutant. arxiv.org/abs/2501.12862
ISSTA 2026 Replication
Across 11 models and 101,000 tests, mutation score correlated strongly with real bug detection (r = 0.86) for regression tests, but only weakly when the code under test was already wrong. arxiv.org/abs/2607.22880
Generate tests from the acceptance criteria, not from the implementation. The pattern that works: generate, mutate, feed the survivors back, regenerate. Most mutation tools now have incremental modes that make this affordable per pull request.
Small Batches and PR Size Caps
PR size rose 154% in Faros AI's 2025 data, and review time rose with it (Faros AI, "The AI Productivity Paradox," July 2025). Cap PR size in the pipeline - a soft warning at 400 lines and a hard split request at 800 lines is a reasonable start. Stacked PRs and a merge queue make small batches cheap to maintain.
DORA lists working in small batches among its seven AI capabilities for a reason: it is the single practice that most directly counters the review-queue failure mode that shows up everywhere else on this page.
Risk-Tiered Review
Uniform review collapses once PR volume rises. Classify every diff against the approved plan plus the CODEOWNERS map, and route it by tier.
| Tier | Scope | Review | Agent-authored allowed |
|---|---|---|---|
| 1 | Docs, copy, tests, flag-off scaffolding, dependency bumps in the safe class | Auto-merge on green, human audit sample weekly | Yes |
| 2 | Standard feature work behind a flag | One human; AI pre-review resolves the mechanical comments first | Yes, labeled |
| 3 | Auth, money and eligibility math, PII paths, migrations, rule engines, external adapters, infrastructure | Two humans plus an architect, no AI auto-approve | No |
Tier 1 - Auto-Merge
Docs, copy, tests, flag-off scaffolding, safe-class dependency bumps. Auto-merges on green with a weekly human audit sample.
Tier 2 - One Human
Standard feature work behind a flag. One human reviewer; AI pre-review clears the mechanical comments first.
Tier 3 - Two Humans Plus Architect
Auth, money and eligibility math, PII paths, migrations, rule engines, external adapters, infrastructure. No AI auto-approve.
Merge Queue
A merge queue batches verification, keeps the main branch green, and gives agents a safe place to repair rebase and lockfile conflicts without ever touching a protected branch directly. It is the mechanical answer to a problem every practice on this page eventually creates: more pull requests than the pipeline can verify one at a time.
Model and Tool Pinning
Pin model versions in CI, and log the version used on every agent action. A model upgrade is a change like any other - it goes through the same review as a dependency bump: run the eval suite, compare results, then promote. The same discipline applies to the review agent's rules and to the instruction files themselves. Unpinned means unreproducible.
The Learn Loop: Rules, Skills, and Evidence
Every escaped defect and every incident becomes a new rule in the review agent's library. Every repeated task becomes a skill: a versioned, testable procedure the agent runs the same way every time. Every release generates its evidence pack straight from the ticket contract.
This is the compounding asset. A review agent gets sharper on your codebase every quarter; a competitor's stays generic. The underlying models are the same. The context is not.
From Defect to Rule
Still escaping feeds back into the rule
From Task to Skill
A regression feeds back into the skill
Evals for Prompts, Agents, and Review Rules
A review rule that fires on the wrong things trains reviewers to ignore all automated feedback. A skill that regressed after a model upgrade fails silently. Treat prompts, rules, and skills as code, with tests:
- A fixture set of real, sanitized pull requests with known issues. Run the review agent against it on every rule change and every model change, and track precision and recall per rule.
- A fixture set of tickets with known-good plans. Run the planning agent and diff its output against the reference plan.
- Failure cases from production go into the fixture set the same week they happen.
Tooling: Promptfoo (open source, acquired by OpenAI in 03-2026 and still open), Braintrust, LangSmith, Langfuse, DeepEval, Arize Phoenix, and the UK AI Safety Institute's Inspect all work. OpenAI's hosted Evals API is deprecated, with a shutdown date of 11-30-2026 - itself an argument for keeping the fixture set in your own repo and the runner interchangeable. Anthropic's "Demystifying evals for AI agents" (01-2026) is the best short primer on grading a full agent transcript rather than a single output.
The fixture set matters more than the tool.
Attribution and Provenance
- Agent commits go under a bot identity that carries the ticket ID and model version.
- Human commits that used AI materially carry a co-authorship trailer - this is for the postmortem, not for blame.
- The PR template asks whether the change was AI-assisted, which tool, and what a human verified and how. See Templates and Checklists.
- Build provenance (SLSA, Sigstore) does not change because AI wrote the code, but the attestations now need to include the agent identity that opened the PR. There is no standard for this yet: SLSA v1.2 (11-2025) has no AI or agent language, and a proposal to add one was closed without change in 04-2026. Record it in the commit trailer and the PR template until one exists.
- SLSA's Source track L4 requires two-party review. An agent author plus one human reviewer does not satisfy it if the agent is counted as a party - treat the agent as zero parties.
Where It Fits in AIDLC
These practices are not a sixth phase - they run through the ones that already exist. The ticket contract is written in Analyze and Ideate. Plan-first, tests-first, small batches, risk-tiered review, the merge queue, and model pinning all execute inside Develop and Launch. The learn loop and evals close the circle in Curate, feeding what shipped back into the instruction files and rules the next cycle starts from.
Frequently Asked Questions
Because reviewing a plan takes about five minutes and catches architectural mistakes, while reviewing a fifteen-hundred-line diff takes about an hour and mostly catches typos. An agent produces a plan naming files, interfaces, migrations, flags, test surface, blast radius, and rollback; a senior human approves or revises it before any code is written, so judgment applies where it is cheapest to change.
Meta's TestGenEval found tests that reached 35.2% line coverage while catching only 18.8% of injected mutants, and a separate study found suites with 100% coverage but a 4% mutation score. Coverage measures how much code a test touches, not whether it would catch a real bug. Mutation testing checks that instead, by changing the code and confirming the test notices.
The learn loop turns every escaped defect or incident into a new versioned review rule, and every repeated task into a skill: a testable procedure the agent runs the same way each time. Rules and skills are then checked against real fixtures so regressions surface immediately. The payoff compounds - the review agent gets sharper on your codebase every quarter, on the same underlying model everyone else has.
Turn These Practices Into Checklists
The ticket contract, the PR template, and the reviewer checklist are ready to copy - and every practice on this page runs inside the Develop phase.