Your AI agents can be right and your delivery system can still be wrong

A coding agent closes a ticket. The method compiles, the tests pass, the diff reads cleanly, and the pull request looks like something a competent engineer would write. Locally, everything is correct. Then the change sits in review for three days, comes back with questions about a service it touches two hops away, gets reworked, and ships a week later than the estimate implied. The agent was right. The system was slow.

This gap is what’s worth measuring. When AI tools produce locally plausible work faster, the question that matters isn’t whether the output is good in isolation. It’s whether complete work moves through your delivery system with fewer dependency gaps, less rework, and stable delivery. Those are different questions, and the evidence increasingly suggests they can diverge.

Local correctness has a well-known blind spot: context

The clearest mechanism evidence predates the current agent wave. Cataldo and Herbsleb (2013), in a study of two large industrial software projects, found that gaps between the coordination a change required and the coordination that actually happened were associated with more software failures. When they modeled moving structural congruence (the alignment between technical dependencies and the people coordinating across them) from its observed minimum to maximum, estimated failure-proneness dropped by roughly 18.6% in one project and 7.6% in the other under their model (Cataldo & Herbsleb 2013, IEEE TSE). The effect sizes depend on the measurement model and the context, and this is not AI-specific work. But the mechanism is durable: failures cluster where technical dependencies cross organizational boundaries that nobody is coordinating across. A systematic review by Raza et al. (2021) reinforces that coordination requirements shift dynamically, so the gaps are a moving target rather than a one-time fix.

Now overlay how code assistants behave at exactly those boundaries. Corso, Mariani, Micucci and Riganelli (2024) evaluated four assistants (Copilot, Tabnine, ChatGPT, and Bard) generating 100 Java methods drawn from real open-source projects, sorted by complexity and dependency (ICPC 2024, arXiv). Where a method had to interact with context from other classes, correct implementations fell to between 4% and 16%. Even the strongest assistant produced invalid or incorrect methods 53% of the time. This is method-level generation with 2024 tools, not agentic end-to-end work, so read it as mechanism rather than a delivery verdict. But the pattern lines up with Cataldo and Herbsleb: the assistants were weakest exactly where a change reaches across a boundary, which is where delivery systems already tend to fail.

The agent gets the local method right. The cross-boundary behavior is where correctness quietly degrades, and that is the part your reviewers, not the agent, have to catch.

More output is not the same as more throughput

Two recent studies are worth holding side by side, with their caveats intact.

Becker, Rush, Barnes and Rein (2025) ran a randomized controlled trial with 16 experienced open-source developers across 246 real tasks in mature repositories they knew well (arXiv, METR write-up). Tasks were randomized to AI-allowed or AI-disallowed. On average, the AI-allowed tasks took 19% longer, even though the developers expected a speedup beforehand and believed they had been faster afterward. This is a preprint with a small number of participants, all experienced maintainers working in codebases they know deeply. The authors explicitly warn against generalizing beyond that setting. It is not evidence that AI slows most developers, and it should not be cited that way. What it does show, credibly, is that perceived speed and measured speed can come apart, and that expertise plus a mature codebase is a setting where the tool did not pay off in wall-clock time.

Xu, Medappa, Tunc, Vroegindeweij and Fransoo (2025) looked at a different layer: open-source project panels around the introduction of GitHub Copilot (arXiv). Copilot-supported repositories showed more output and more rework. The paper reports 17.7% more added lines of code, 4.3% more pull requests, 2.4% more PR rework, and a shift in which core contributors reviewed 6.5% more code while their own commits dropped by 19%. This is also a preprint; PR rework is a proxy for technical debt, and the adoption measure and treatment design warrant caution. But the shape is telling. Volume goes up. So does the review load, and it lands on the same senior people. Generation got cheaper; integration and review did not.

That redistribution of cost has a human signal attached. Baltes, Cheong and Treude (2026) analyzed 1,154 Reddit and Hacker News discussions and found developers repeatedly describing review friction, eroding trust, quality degradation, and costs pushed onto whoever has to clean up low-quality AI-generated artifacts. This is discussion analysis, not causal evidence. But it names the same failure mode from the practitioner’s chair.

The field-level framing: amplifier, not accelerant

DORA’s 2025 report on AI-assisted software development, based on a global survey run from June to July 2025, frames AI as an amplifier of existing organizational strengths and weaknesses rather than a uniform accelerator (DORA 2025). It associates AI adoption with improved throughput and with increased delivery instability, and it points at the conditions that separate the two outcomes: platform quality, workflow clarity, value-stream management, a clear AI policy, and a healthy data ecosystem. This is correlational, industry survey data from a sponsored source, so treat it as current field framing rather than proof. But it converges with everything above. AI raises the volume flowing into your system. Whether that becomes delivered value or delivery instability depends on the system it flows into.

Put the strands together and the claim is narrow and defensible: local output speed is not the same as system throughput. None of this says AI makes teams slower. It says correctness at the task boundary does not guarantee flow across the system boundary, and that the difference shows up in review queues, rework, and change stability, not in how good the diff looks.

Make integration responsibility visible before scaling throughput

The delivery-management move is not to slow the agents down. It is to make the integration work visible and owned before you scale agent throughput, because the evidence says throughput is the easy part and integration is where the cost has moved.

Here is a diagnostic you can run on Monday. Pick one AI-touched workflow and pull the last 20 work items or pull requests it produced. Do not audit the code quality, audit the flow.

A Monday-morning flow check for one AI-touched workflow

1. What was the agent actually optimizing? Name the target for each item: a local method, a file, a PR, a feature, an incident, or a customer outcome. If most items optimized “make the PR look done,” you have a local-correctness bias.
2. Which technical dependencies crossed the local boundary? Mark every item that touched code, data, or behavior owned by another team or service. This is where Corso et al. and Cataldo & Herbsleb both predict trouble.
3. Who owned integration correctness before the agent started? If the answer is “nobody, we found out in review,” integration ownership is implicit, which means it is unassigned.
4. What downstream queue grew? For each item, note what it fed: review wait, rework, QA, a security exception, an architecture decision, or stakeholder clarification. Watch for load concentrating on a few senior reviewers (the Xu et al. pattern).
5. What was actually checked before merge? Tests and code review, yes, but also cross-service behavior, observability, rollback path, and product acceptance. Note which checks were skipped because the diff “looked fine.”
6. Did the item finish once, or come back? Count items that reappeared as follow-up work, hotfixes, or reopened tickets. Rework that returns is throughput you already paid for and did not get.
7. What moved in a system metric? Compare against your own baseline: cycle-time tail (not the median, the slow items), rework rate, change instability, review age, blocked-state age, and support load.

If items are locally correct but the cycle-time tail, review age, and rework are climbing, your agents are right and your system is not. That is a coordination and ownership problem, and no amount of better prompts fixes it. Make the integration responsibility explicit. Assign it, put it in the working agreement, and give it capacity, before you turn up the volume.

So judge AI-assisted delivery where the cost actually lands: in review queues, rework, and change stability, not in how finished each diff looks. Do that, and the agents can earn their keep.

Sources

• Cataldo & Herbsleb (2013). Coordination Breakdowns and Their Impact on Development Productivity and Software Failures. IEEE Transactions on Software Engineering. par.nsf.gov/servlets/purl/10038372
• Raza et al. (2021). Assessing the impact of socio-technical congruence in software development: a systematic literature review. consensus.app record
• Corso, Mariani, Micucci & Riganelli (2024). Generating Java Methods: An Empirical Assessment of Four AI-Based Code Assistants. ICPC 2024. doi.org/10.1145/3643916.3644402
• Becker, Rush, Barnes & Rein (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arxiv.org/abs/2507.09089
• Xu, Medappa, Tunc, Vroegindeweij & Fransoo (2025). AI-Assisted Programming Decreases the Productivity of Experienced Developers by Increasing the Technical Debt and Maintenance Burden. arxiv.org/abs/2510.10165
• Baltes, Cheong & Treude (2026). “An Endless Stream of AI Slop”: How Developers Discuss the Burden of AI-Assisted Software Development. consensus.app record
• DORA (2025). State of AI-assisted Software Development. dora.dev/dora-report-2025