The Hidden Risks of AI-Generated Code
The dangerous failure mode is not a single dramatic incident. It is quiet accumulation - duplication, unreviewed diffs, and misplaced confidence stacking up commit by commit until the codebase is harder to change than anyone noticed. The controlled studies below say the same thing from different directions: watch the trend line, not the demo.
Last reviewed: September 2026
Quiet Accumulation, Not One Incident
None of the risks below look dramatic in a single pull request. A slightly duplicated block. A catch clause that swallows an error instead of handling it. A test that runs green without asserting anything of consequence. Reviewed in isolation, each one is a shrug. Reviewed in aggregate across a year of AI-assisted commits, they are a maintainability tipping point: the codebase quietly gets harder to change, harder to trust, and harder to secure, long before anyone files an incident about it.
That is what makes this category dangerous. Velocity dashboards go up. Ticket close rates go up. The signals that would normally warn you - review depth, refactoring share, duplication - move in the wrong direction underneath the good news, and they move slowly enough that no single sprint looks like the moment things turned. Treat the data on this page as an early-warning system, not an indictment of any one AI-generated diff.
The Risk Register
| Risk | Severity | What it looks like |
|---|---|---|
| Shadow AI | High | Adoption outside the official rollout - no logging, no standards, no way to assess impact, data leaving through personal accounts. |
| Inconsistent standards | High | Each team invents its own norms, so quality becomes unpredictable across the organization. |
| Security and IP exposure | High | Proprietary code, customer data, secrets, and internal architecture leaving the boundary through prompts, context windows, MCP servers, and agent tool calls. |
| Shallow code review | High | Volume rises, review depth falls, defects slip through. The most measurable risk on this page, and the one velocity dashboards hide. |
| Package hallucination (slopsquatting) | High | Models recommend packages that do not exist, and attackers register the names. About one in five package recommendations was hallucinated in one large study. |
| Prompt injection via repo content | High | Agents read issues, PR comments, READMEs, and dependency docs. Any of those can carry instructions. Tracked as OWASP LLM01 and ASI01. |
| Agent over-permission | High | An agent with write access to protected branches, production credentials, or unrestricted shell is an unmonitored engineer with no judgment. Tracked as OWASP LLM06 and ASI03. |
| Maintainability debt | Medium to High | Duplication up, refactoring down, cross-file coupling down. Individually harmless commits that together produce architectural drift. |
| Test theater | Medium | Generated tests with high coverage and no assertions of consequence. Green builds that prove nothing. |
| Prompt and context drift | Medium | Undocumented prompting habits, siloed technique, instruction files that rot, output quality that degrades unpredictably. |
| False sense of productivity | Medium | More lines and more closed tickets while correctness and maintainability decline, reinforced by the perception gap the METR study measured. |
| License contamination | Medium | Generated code reproducing licensed material without attribution. Manageable with vendor indemnity and duplicate-detection filters, but only if configured. |
| Skill atrophy | Medium | Juniors who never build the mental model, seniors who stop reading. Shows up as a review bench that cannot catch what the model got wrong. |
| Vendor and model churn | Low to Medium | Behavior changes when a model version changes. Unpinned models mean unreproducible output. |
Quality: What the Code Review Data Shows
CodeRabbit's "State of AI vs Human Code Generation" report (December 2025) compared 470 open-source pull requests - 320 AI co-authored and 150 human-only - and found AI-authored PRs carried more of nearly every defect class it measured:
- Issues per PR: 1.7x for AI co-authored PRs (10.83 vs 6.45)
- Logic and correctness errors: 1.75x
- Security vulnerabilities: 1.57x (XSS alone 2.74x)
- Maintainability issues: 1.64x; readability issues: more than 3x; error-handling gaps: about 2x
- Human-only PRs still won on two measures: more spelling errors (1.76x) and more testability issues (1.32x)
Maintainability: GitClear's Two Studies
"AI Copilot Code Quality," 2025 edition
GitClear's 2025 study analyzed 211 million changed lines from 2020 to 2024 and found refactoring giving way to duplication:
- Moved and refactored lines fell from about 25% of changes to 9.5% - roughly a 60% relative decline
- Copy-pasted lines rose from 8.3% to 12.3%; 2024 was the first year copy-paste exceeded refactoring
- Commits containing five or more duplicated lines rose about 8x during 2024 - the headline "4x growth in code clones"
- All-line churn (code rewritten or reverted within two weeks) rose from about 3.2% to 5.7%, roughly 1.8x
"The Maintainability Gap," January 2026
The follow-up study covered 623 million code changes from 2023 to 2026, indexed to 2023, and the trend deepened:
- Code-block duplication: +81%
- Error-masking constructs (empty catches, broad excepts): +47%
- Cross-file function calls: -35% - code stops reusing what already exists
- Refactoring line-moves: -70%; refactoring's share of all changes fell from 21% in 2022 to 3.8% in 2026
GitClear's own framing matters as much as the numbers: the headline is not "AI writes bad code." It is deferred structural maintenance - work that used to happen inline, now not happening at all.
Security: By Controlled Measurement
BaxBench (ETH Zurich)
BaxBench (February 2025, 392 backend tasks) found that 62% of solutions from the best model at the time were either incorrect or contained a security vulnerability; the 2026 leaderboard sits near 50%. This is the source of the widely misquoted "62% of AI code has vulnerabilities" figure - it counts incorrect-or-insecure, not insecure alone.
Stanford, Perry et al.
Perry, Srivastava, Kumar, and Boneh (CCS 2023, 47 participants) found developers with an AI assistant wrote significantly less secure code and were more likely to believe it was secure: 36% wrote SQL-injectable solutions with AI, versus 7% without.
NYU, Pearce et al.
"Asleep at the Keyboard?" (IEEE S&P 2022) found about 40% of 1,689 Copilot-generated programs across 89 CWE scenarios were vulnerable.
Veracode GenAI Code Security
Veracode's 2025 report found 45% of samples failed OWASP Top 10 checks (Java 72%; XSS defenses failed 86% of the time, log injection 88%). The 2026 edition found 44% still fail; the best model passes 68%, most models 50 to 53%, and model size has no effect. XSS pass rate: 15%. Log injection pass rate: 12%. Newer and larger models are not fixing this.
Apiiro
Apiiro's telemetry study (September 2025, Fortune 50 organizations) found AI-assisted developers committed 3 to 4x more and generated 10x more security findings; privilege-escalation paths rose 322%, architectural design flaws rose 153%. Syntax errors fell 76% and logic bugs fell 60% - the bugs moved up the stack, from typos to design.
Package Hallucination (Slopsquatting)
Spracklen et al. (USENIX Security 2025, 576,000 samples, 16 models) found 19.7% of recommended packages did not exist - 205,474 unique hallucinated names. Open-source models hallucinated 21.7% of the time versus about 5.2% for commercial models, and 43% of hallucinated names recurred on every re-query, which makes them registrable and exploitable.
Sonar Coding Personalities
"The Coding Personalities of Leading LLMs" (August 2025, 4,400+ Java tasks) found more than 90% of issues were code smells, and the model with the best pass rate also produced 93% more high-severity bugs than its predecessor. Better benchmarks did not mean safer code.
Cloud Security Alliance
The Cloud Security Alliance (April 2026) reported 74 CVEs attributed to AI coding tools by Georgia Tech's tracking, with the monthly count rising from 6 in January 2026 to 35 in March.
Trust: What Developers Say
Self-reported trust tracks the same pattern as the controlled studies: heavy use, low confidence.
- The Stack Overflow Developer Survey 2025 (49,000+ respondents): 84% use or plan to use AI tools, but only 33% trust its accuracy and 46% distrust it. The top frustration, at 66%, is that answers are "almost right, but not quite." 45% say debugging AI-generated code is more time-consuming than debugging their own.
- Sonar's State of Code Developer Survey Report (January 2026, 1,100+ developers): 42% of committed code is AI-generated or AI-assisted, yet 96% do not fully trust it. Only 48% always verify AI code before committing, and 38% say reviewing it takes more effort than reviewing human code.
- Qodo's 2025 State of AI Code Quality (609 developers): 65% say AI misses relevant context, and 76% do not fully trust generated code. Seniors with 10+ years of experience are far less willing to ship unreviewed AI code (25.8%) than juniors (60.2%).
What to Do About It
Each risk class above has a corresponding control elsewhere in the Playbook. None of them is a single tool purchase - they are practices.
- Shallow review, review fatigue, and inconsistent standards are the subject of AI Code Review at Scale.
- Test theater gets caught with mutation testing, covered in Engineering Practices.
- Security and IP exposure, prompt injection, and slopsquatting are addressed in Security and Compliance.
- Shadow AI and inconsistent standards need an organization-wide policy - see Governance Before Scale.
- Agent over-permission is a control-plane problem, covered in Agents in the Pipeline.
Where It Fits in AIDLC
These risks are not confined to one phase - they accumulate across the loop. Analyze and Ideate set the standards and scope that keep shadow AI and inconsistent norms from taking hold. Develop is where most of the quality and security data above was measured - duplication, insecure output, and shallow review all happen at the point of authoring. Launch is where agent over-permission and unreviewed merges turn into production incidents. Curate is the feedback loop that turns an escaped defect into a permanent review rule, which is the only way any of these trend lines reverse.
Frequently Asked Questions
The controlled studies say yes, with caveats. Veracode's 2026 report found 44% of samples still fail OWASP Top 10 checks regardless of model size. Stanford's Perry et al. study found developers using AI wrote more SQL-injectable code and trusted it more. BaxBench's often-cited 62% figure counts incorrect-or-insecure code, not insecure code alone.
Slopsquatting is registering a package name that AI models hallucinate, so that code generated later installs the attacker's package. A USENIX Security 2025 study of 576,000 samples found 19.7% of recommended packages did not exist, and 43% of those hallucinated names recurred on every re-query, which is what makes them practical to register and exploit.
Coverage only measures whether a line executed during a test, not whether the test would fail if the logic were wrong. AI-generated tests are good at exercising code and weak at asserting outcomes, so a suite can show high coverage while catching almost nothing. Mutation testing, which checks whether tests fail against deliberately broken code, is the check coverage cannot do.
Turn Risk Into a Control
Every risk on this page has a corresponding practice in the Playbook. Start with security, or see the full set.