Your AI-assisted team got faster. Why did the forecast get worse?

If you run a software organisation, you have probably had this conversation in the last three months. Developers ship code faster. PR throughput is up. Standups feel calmer. And the stakeholder question, "when will it be done?", has gotten harder to answer, not easier.

That gap shows up in current data. It has a structural reason that token budgets and forecasting tools cannot fix on their own.

What the latest data actually shows

Plandek's 2026 Engineering Productivity Benchmarks pulled delivery data from more than 2,000 engineering teams. AI accelerates coding across the sample. The interesting part is what happens next.

Lower-performing teams using AI cut Lead Time to Value by roughly 50%. Top-performing teams improved by 10 to 15%. About a four-fold gap in AI-driven gain. Top-quartile teams complete more than two-thirds of the work they plan in a sprint; bottom-quartile teams complete under half. Bottom-quartile cycle time runs over 62 days; top-quartile under 22.5. AI raised the floor more than it raised the ceiling, and the predictability gap between top and bottom is still wide.

Plandek sells delivery analytics and has a product framework (RACER) to push. Read the numbers directionally, not as audited benchmarks. The pattern is consistent with DORA's 2025 State of AI-Assisted Software Development, which surveyed about 5,000 developers and concluded that AI amplifies the engineering system it sits in. Mature systems convert AI into throughput and product performance; immature systems lose stability.

Predictability is a property of the distribution, not the estimate

Predictability is not the same thing as speed. A team that ships every two weeks with a standard deviation of one week is predictable. A team that averages one week but sometimes ships in two days and sometimes in five weeks is not. Reinertsen has been making this point since 2009: the dominant economics of product development sit in variability, queues, and batch size, not in mean throughput.

AI moves all three at once, and not in the same direction.

The time-to-first-draft distribution narrows. Code generation absorbs work that used to vary a lot between developers and tasks. That part is real, and it is the part that shows up on the dashboard, which is why leadership often assumes the system is more predictable than it is.

Review and verification cycle times stretch, because the queue arrives faster than it used to and the reviewer often has less context than the author who wrote the change by hand. Scope variance grows, because cheap generation invites edges to be added once the work is in flight. Delivery cycle time has a flat or better mean and a wider spread. The 85th percentile slips. The right tail thickens. Stakeholders feel the change as missed commitments even when the median is fine.

Coding faster, predictability uneven. That is the Plandek pattern.

What the evidence says about forecasting

There is also an unflattering piece of evidence about how most teams answer the "when will it be done?" question.

Miranda et al. (ACM SAC 2021) ran a controlled comparison of Monte Carlo forecasts against developer estimates on real project data. Monte Carlo forecasts based on past takt time scored an MMRE of 32% for delivery-date forecasts and 20% for effort forecasts. Where comparable developer estimates were available, their MMRE was 134%. Accuracy stabilised once the team had around 20 historical data points.

That number lands harder in the context of Fernández-Diego et al.'s 2020 systematic review of 73 papers on Agile effort estimation: expert-based methods (Planning Poker, story points) continued to dominate practice, with accuracy still a known unsolved problem.

The field's dominant estimation practice is the one with the worst track record on absolute error, and a probabilistic forecast from twenty completed items beats it by roughly a factor of four. Most teams already have those twenty items in Jira or Azure DevOps. They don't use them.

Why the standard moves are insufficient

Two reasonable instincts: install a Monte Carlo tool, or set token budgets on the AI side. Both useful. Neither moves predictability much on its own.

A Monte Carlo tool reports the variance you already have. It improves the honesty of the forecast, which is real progress when the alternative is single-point estimates that miss by 134%. The simulation sits downstream of the system. If the cycle-time distribution has a thick right tail, the forecast inherits the thick right tail. Reporting the spread between the 50th and the 85th does not move it.

Token budgets and runtime guardrails cap cost variance. They protect against the worst loops. They do not touch time variance, which is where predictability lives.

The lever that matters is the shape of the cycle-time distribution itself, and it sits in the delivery system, not in the AI layer or the reporting layer.

What a Delivery Manager can do this sprint

This is concrete enough to start in the next two weeks. None of it requires a tool purchase.

1. Replace the single-point date with a 50/85 pair. When you tell a stakeholder "we will ship in six weeks", what you mean is the 50th percentile. Add the 85th. The honest answer is usually "the 50th is week 6, the 85th is week 9, the spread is 3". That spread is now the conversation. Stakeholders accept this better than people expect.

2. Pull the cycle-time distribution for the last 50 items. Median, 85th, 95th. The gap between the median and the 85th is your real predictability metric. This number tells you what stakeholders feel.

3. Constrain the right tail. Right-tail outliers eat predictability faster than anything else. A team with median 4 days and a 95th percentile of 30 days does not need to ship faster on average. It needs the 30-day items to stop existing. Two rules: a hard WIP limit at the review gate (Sjøberg's 2018 SINTEF study on five Kanban teams over four years found lower WIP correlates with shorter lead time; quality was not measured, so do not overclaim), and a "stop and review" trigger for any item that crosses the 85th percentile in flight.

4. Tighten intake. Reinertsen's economic batch-size argument matters more under AI, not less. Cheap code generation multiplies the cost of vague acceptance criteria. A one-sentence acceptance criterion that the engineer and product owner agreed on before the agent runs prevents most scope-driven right-tail items.

5. Audit the outliers each sprint. Pull the three slowest items from the last 30. Read the PR threads. What stretched the cycle time: review wait, missing context in the prompt, missing test coverage so the model regenerated, scope leak after the PR opened, integration test flakiness? Each cause maps to an existing flow lever. The audit takes half an hour. Most teams skip it because no one's job description says they own predictability.

What this is not

Two clarifications. This is not an argument for less AI. The DORA data suggests teams with mature systems convert AI into real throughput, so a blanket pullback loses the gain. And it is not an argument for more estimation rituals; Fernández-Diego et al. is clear enough on where that road ends.

The argument is to move the predictability conversation from estimation to flow, from mean to distribution, from "when will it be done" to "what is the spread between the 50th and the 85th, and what do we do about the 85th". That conversation is a Delivery Manager's job. AI did not invent it, but it made the job harder to avoid.

The team will probably stay faster. With a few weeks of work on the right tail, it will also become more honestly predictable. Those are not the same thing, and the difference is what shows up on the stakeholder's report.

Sources

• DORA. (2025, September 23). State of AI-assisted software development 2025. Google Cloud. dora.dev/research/2025/dora-report
• Fernández-Diego, M., Méndez, E. R., González-Ladrón-de-Guevara, F., Abrahão, S., & Insfran, E. (2020). An update on effort estimation in agile software development: A systematic literature review. IEEE Access, 8, 166768–166800. doi.org/10.1109/ACCESS.2020.3021664
• Miranda, P., Faria, J. P., Correia, F. F., Fares, A., Graça, R., & Moreira, J. M. (2021). An analysis of Monte Carlo simulations for forecasting software projects. In Proceedings of the 36th Annual ACM Symposium on Applied Computing (pp. 1550–1558). ACM. doi.org/10.1145/3412841.3442030
• Plandek. (2026, March). 2026 engineering productivity benchmarks report. plandek.com/resources/2026-engineering-productivity-benchmarks-report
• Reinertsen, D. G. (2009). The principles of product development flow: Second generation lean product development. Celeritas Publishing. search.worldcat.org/title/435994279
• Sjøberg, D. I. K. (2018). An empirical study of WIP in kanban teams. In Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '18). ACM. doi.org/10.1145/3239235.3239238