Agents in the CI Pipeline
An agent running in continuous integration is a service, not a teammate with judgment. It needs an identity, a permission scope, a sandbox, a budget, and hard rules enforced in the workflow - the same controls you would put around any other automated workload with write access to your repository.
Why Treat an Agent Like a Service
Most incidents in agentic pipelines are not model failures - they are access-control failures. An agent that authenticates as a person inherits that person's permissions and disappears from the audit log the day the person leaves. An agent that can push to a protected branch can push a broken build to that branch. An agent under pressure to turn a build green can delete the failing test instead of fixing the bug behind it.
None of that requires distrusting the model. It requires the same discipline already applied to any other credentialed service: least privilege, short-lived tokens, a sandbox that limits blast radius, a budget that caps what a runaway loop can cost, and rules enforced by the harness rather than requested in an instruction file the model can talk itself past under pressure.
Identity
Seats authenticate humans. Pipelines get their own identity: a service account or workload identity on the vendor's API platform - never a personal OAuth token, never a shared key sitting in a repo secret that outlives the person who created it.
Prefer workload identity federation where the vendor supports it: the CI job exchanges its own OIDC token for short-lived API access, so no long-lived secret sits in the repo at all. Anthropic's federation went generally available on 06-17-2026 - a federation issuer for token.actions.githubusercontent.com, a service account, and a federation rule that constrains subject, audience, and claims, returning a short-lived, single-use token. Constrain the rule to specific repos and refs: a bare repo:org/* subject also matches fork pull requests, which is not the access you meant to grant.
Subscription OAuth tokens - the kind a developer generates from their own seat - are permitted in the vendor's first-party CI action, but they bind to that person, bill against their plan, and vanish when they leave. Third-party tools have to use API keys instead; routing a third-party tool through seat credentials is against Anthropic's terms. Give each workflow class - review, repair, implement - its own identity, its own workspace, and its own spend and rate limits, so audit and budget stay attributable to a specific job, not a blended total.
A GitHub App, Not GITHUB_TOKEN
On the source-control side the agent acts as a GitHub App - the vendor's own, or one you register yourself - not the default repository token and not a person. App identity is what makes the audit log attributable and lets branch rules target the agent specifically.
Bot Commits Carry the Ticket ID
GitHub's own cloud agent is the reference pattern: commits authored by the agent, signed, the human listed as co-author, an Agent-Logs-Url trailer linking back to the session log, and pushes confined to a branch prefix it owns. See also Claude Code's own GitHub Actions integration for the first-party version of this.
Permissions
Read access to the repo. Write access only to a branch namespace the agent owns, such as agent/*. No write to protected branches, and no approving its own pull requests.
Give it a tool allow list - build, test, lint, formatter, package manager in lockfile mode - and a deny list for anything that deploys, anything that touches credentials, and anything destructive. Least privilege belongs in the harness configuration, enforced by a hook or a permission the tool call cannot bypass, not in the prompt as a request the model might honor.
No production credentials in the agent's environment, and no production data in its test fixtures. MCP servers and connectors the agent can reach get reviewed like any other dependency - who wrote it, what it can read, what it can write, and whether its responses can carry instructions back to the model. Pin them by version: several MCP servers, including reference filesystem and git servers, have shipped CVEs. Instruction files stay advisory - the model can and will deviate under pressure - so anything that must never happen is a hook or a harness permission that blocks the tool call, never a sentence in a project instruction file.
Platform Controls Worth Turning On
- Rulesets, not classic branch protection: required reviews, required status checks, requiring workflows to pass at the org level, requiring code scanning results, and the "Additional approval for unattributed Copilot pull requests" rule (preview, default on) which adds one approval whenever an agent opens a PR under its own identity.
- Actions on agent-opened PRs do not run until a user with write access clicks "Approve and run workflows." Leave that default on - the task assigner's own approval does not count toward required reviews.
- Enterprise AI Controls and the agent control plane went generally available on 02-26-2026: an AI Administrator role, per-agent enable and disable policy, an actor_is_agent field in the audit log, an agent session viewer, MCP server allow and deny lists in managed settings, and a global model policy that keeps open-weight models off by default.
- Copilot code review counting toward required approvals is a preview feature as of 09-01-2026, default off and path-scopable. Leave it off for Tier 3 paths - an AI approval satisfying a human approval requirement is the exact control failure to avoid.
- Per-user AI credit budgets on cost centers, content exclusion for paths the agent must never read, and push protection with delegated bypass so an agent cannot commit a secret.
Sandbox and Budget
Sandbox
- An ephemeral runner or container per task. Nothing persists between runs except what lands in the pull request.
- Network egress allow-listed to only the package registries and APIs the task actually needs.
- Secrets injected at runtime with the narrowest scope possible, never written to the working tree.
Budget
- A per-workflow token budget. A runaway loop should cost twelve dollars, not twelve hundred.
- Spend tagged by repo, workflow, and ticket, with a hard monthly cap on the API platform and an alert at 70 percent.
- Attempt limits: two repair attempts, then the agent pages a human and stops.
Seven Hard Rules, Enforced in the Workflow
Copy these into the workflow definition as enforced checks - a required status check, a branch rule, a harness permission - not into the prompt as requests the model is merely asked to follow.
The agent never pushes to a protected branch. It opens a pull request.
The agent never modifies an assertion, deletes a test, or adds a skip to make a build pass, unless the pull request is labeled test-weakened and routed to a human.
Two repair attempts, then it pages a human and stops.
Every repair pull request carries the original failure log.
If the agent cannot produce a failing test for a bug, it posts its analysis to the ticket and stops. No patch without a repro.
Every agent pull request states its model version, its task source, and what it verified.
The agent never approves, merges, or re-requests review on its own pull request.
Work Classes
Safe to Delegate Unattended
Open a draft pull request and let it run:
Test backfill, dependency bumps and deprecation fixes, copy and i18n, CRUD endpoints behind a flag, schema-driven forms, storybook and error-handling scaffolding, lockfile drift, formatting, type errors after dependency bumps, flaky test quarantine, snapshot updates, migration ordering, release notes, and documentation.
Never Unattended
Keep a developer in the loop for:
Money and eligibility math, auth and session handling, PII paths, rule engines, external adapters to systems of record, infrastructure and deploy config, and anything where the cheapest path to green is weakening a test.
Two Lanes
Lane A
Developer-driven, with an agent in the IDE. The developer is the author of record for every change.
Lane B
Agent-driven, on tickets labeled agent-eligible. Opens a draft pull request unattended. Starts on the safe work class only, and expands one work class at a time, gated on change failure rate holding flat.
Faros AI's 2026 report, covering 22,000 developers, is the warning for Lane B: unreviewed merges up 31% and incidents per pull request up roughly 3x on teams running agents without the guardrails above. Lane B without tiered review and a merge queue is how that happens.
Pattern: Bug to Failing Test
The agent's first deliverable is a failing test, not a patch. Without this rule you get confident patches for misdiagnosed bugs, which is worse than no patch at all.
No repro
If the agent cannot produce a failing test, it posts its analysis to the ticket and stops right there. No patch ships without a repro.
Pattern: Self-Healing Pipeline
This pattern works on a narrow class of failures and is dangerous outside it - the failure class check comes first, before the agent touches anything.
Lockfile, formatting, types, snapshots, flake, infra retry
This is the safe failure class. The agent attempts repair, continued below.
Anything else
Page a human immediately. The agent does not attempt a repair outside the safe class.
Yes
Label the pull request test-weakened and route it to a human. Same outcome as an out-of-class failure: page a human.
No
Open a repair pull request with the original failure log attached, continued below.
Yes
Merge through normal review.
No
Check the attempt counter. If this was the first attempt, the agent tries once more. If the second attempt has already been used, it pages a human and stops - the same two-attempt limit as the hard rules.
Pattern: Ticket to Production
The full pipeline, from requirements to a production change a named human signed off on.
A revision request loops back to the previous step until the plan is approved.
Default
Lane A - developer with an agent in the IDE.
Labeled agent-eligible
Lane B - agent implements, opens a draft pull request.
Tier 1
Straight to the merge queue.
Tier 2 or Tier 3
Human adjudication first, then the merge queue.
Defect
Back into the bug-to-failing-test loop, which feeds back into AI pre-review above.
Pass
On to the release branch, continued below.
The new rule feeds back into AI pre-review, so the review library keeps compounding.
Where It Fits in AIDLC
Agents in the pipeline live mostly in Develop and Launch. Develop is where the identity, permission scope, and hard rules on this page get wired into the CI configuration - the branch namespace, the tool allow list, the pinned MCP servers. Launch is where the ticket-to-production pipeline runs those agents against real risk tiers, a merge queue, and a named human approver before anything reaches production.
Frequently Asked Questions
No. Hard rule one is that the agent never pushes to a protected branch - it opens a pull request instead. Write access is scoped to a branch namespace it owns, such as agent/*, and it never approves, merges, or re-requests review on its own pull request. A human stays the approval gate all the way to production.
Test backfill, dependency bumps, copy and i18n, flagged CRUD endpoints, schema-driven forms, scaffolding, lockfile drift, formatting, flaky test quarantine, snapshot updates, and documentation. Never unattended: money and eligibility math, auth and session handling, PII paths, rule engines, external adapters to systems of record, infrastructure and deploy config, and anything where the cheapest fix is weakening a test.
As a service, not a person. Use a service account or workload identity federation - Anthropic's went generally available on 06-17-2026 - scoped to specific repos and refs, never a bare wildcard subject that could also match a fork pull request. On the source-control side the agent acts as a GitHub App, not the default repository token and not a personal seat token.
Wire the Controls, Not Just the Prompt
Grab the ticket contract and repo instruction skeleton, then see how this pipeline sits inside the Launch phase.