Your AI pilot worked. Your delivery system did not.

Picture a familiar setup. An engineering organization runs a three-month pilot with an AI coding assistant. The numbers look encouraging: suggestion acceptance rates climb, developers say they are getting through boilerplate faster, and the demo lands well with leadership. The pilot is declared a win and rolled out to everyone.

Two quarters later someone asks the obvious question. Did anything actually ship faster? Lead time for changes is flat. Deployment frequency has not moved. The backlog is the same length it was in January. The tool works. The delivery system around it did not change.

If you have lived through this, you already know it is not a tooling failure. The assistant did exactly what it promised inside the editor. The disappointment shows up one level higher, where code becomes a working change in production. That is the level most AI rollouts never touch.

Compatibility debt, in plain terms

Compatibility debt is the distance between what an AI tool can do on its own and what your delivery system can absorb without breaking or clogging.

The phrase borrows from technical debt on purpose. Technical debt is the accumulated cost of shortcuts in the code. Compatibility debt is the accumulated cost of dropping a fast component into a system that was never set up to feed it, review its output, or act on what it produces. You do not see the debt in the pilot, because a pilot runs in a protected corner. You see it when the tool meets your real flow of work: the review queues, the approval steps, the handoffs between teams, the places where a human still has to decide something.

A concrete version: an assistant helps developers produce pull requests faster. Good. But the review capacity downstream did not grow. So the queue of open PRs gets longer, review latency rises, and the thing you were trying to speed up now waits longer in a different place. You moved the bottleneck. You did not remove it. The local win is real and the system outcome is worse, and both facts are true at the same time.

Tool readiness is not system readiness

Here the research is more useful than the marketing. The strongest, most consistent finding across the adoption literature is not that AI makes delivery faster. It is that AI value depends on readiness across several dimensions at once, and technology is only one of them.

Uren and Edwards (2023), in a qualitative study in the International Journal of Information Management, argue that operational success with AI requires people, process, and data readiness in addition to the technology itself. Put the tool in without the other three and you have bought capability you cannot yet use.

Senanayake (2026), in a systematic literature review that applies the technology-organization-environment (TOE) framework specifically to software development, catalogues 92 distinct antecedents of AI adoption. The ones that recur are telling: compatibility with existing work, security, organizational readiness, top management support, and human capital. Notice how few of those are about the model.

Two survey studies point the same way. Horani and colleagues (2025), surveying 512 senior IT and IS managers in Information Development, found that compatibility, strategic alignment, resources, top management support, and relative advantage all shaped the intention to adopt, while complexity pushed against it. A 2026 IEEE conference study by Senanayake, Wijayanayake and Jayasinghe, based on 158 software professionals, found that organizational readiness, relative advantage, and top management support predicted adoption, and that a structured strategy mattered more than ad hoc experimentation.

Read together, these give you a mechanism rather than a slogan. AI does not add value by existing in your toolchain. It adds value when it is compatible with how decisions, data, and review already flow, and when someone owns the alignment work. The 2025 DORA report on AI-assisted software development frames the same point from the field side: AI tends to act as an amplifier of an organization's existing strengths and weaknesses, which makes value a systems problem, not a procurement one.

Why this lands on delivery, not just engineering

If value is a systems problem, then the people who own the system are the ones who determine whether the pilot pays off. That is Delivery Leads, Agile Coaches, Scrum Masters, and Product and Delivery Managers, alongside the platform team that installed the assistant.

The uncomfortable implication: buying the tool is the easy part, and it is also the part most leaders over-invest in. The harder work is unglamorous. Where do we have review capacity, and where will faster upstream output overwhelm it? Which decisions still need a human, and do those humans have the context to decide well? What data and code can the tool actually see, and what is walled off by governance for good reason? Are people building the judgment to check AI output, or are they learning to wave it through?

None of that is a feature you can turn on. It is delivery work. And it is exactly the work that goes missing when AI is framed as a tool rollout with a training session attached.

A Monday-morning diagnostic

You do not need a transformation program to start. You need one workflow and four weeks. Pick a single, real path through your system, from idea to running in production, and run it through this:

1. Name the queue AI should shrink. Where do you expect the tool to remove waiting or effort? Be specific: "time spent writing test scaffolding," not "productivity."
2. Name the queue AI might grow. Every acceleration pushes load somewhere. If code arrives faster, review, integration, and QA are the usual downstream victims. Name the likely one before it surprises you.
3. Check data and context access. Can the tool actually see the code, tickets, and context it needs to be useful here? If governance blocks that access, the pilot is testing a crippled version of the tool, and you should know that going in.
4. Check review ownership. Who is accountable for what the AI produces on this path? "Everyone" means no one. If review capacity is already tight, more input will make it tighter.
5. Check decision rights. Where does a human still decide, and does that person have enough context to decide fast and well? A faster draft in front of a starved decision does not speed anything up.
6. Check learning and skill formation. Are people getting better at evaluating AI output over these four weeks, or are they outsourcing judgment to it? One builds capability. The other builds a new dependency you cannot see yet.
7. Measure one outcome and one risk signal for four weeks. Pick an outcome you care about (lead time for changes on this path, or review latency) and a risk signal that would tell you the acceleration is doing harm (change failure rate, rework, or escaped defects). Watch both. An outcome number without a risk number is how teams celebrate moving a bottleneck.

On measurement, the DX AI Measurement Framework is a reasonable practitioner reference: track utilization, impact, and cost, and look at the broader engineering picture rather than a single throughput metric. Two of its cautions are useful here. Do not use AI metrics to evaluate individual performance, which corrupts the numbers and the trust around them. And expect much of your bottleneck to sit in the outer loop, in collaboration, alignment, and focus, where a code assistant simply does not reach.

What this argument is, and is not

I want to be clear about the strength of the evidence, because overclaiming here is its own failure mode.

The literature is considerably stronger on what drives AI adoption than on whether AI causally improves delivery performance. Several of the studies above are surveys, which capture intention and perception in a specific context rather than measured throughput in yours. DORA and DX are useful and grounded in real practice, but they are industry sources with a point of view, not neutral lab results. So treat the causal claim with care.

What you can take as reasonably well-supported is the mechanism: AI pays off when it is compatible with your flow of decisions, data, review, governance, skills, and learning, and it disappoints when that compatibility is missing. The rest is a hypothesis about your own system. The four-week diagnostic is how you test it, with your workflow and your numbers, before you scale anything. If the outcome moves and the risk signal stays flat, you have earned the rollout. If not, you have found your compatibility debt cheaply, which is the point.

Sources

• DORA. (2025, September 23). State of AI-assisted software development 2025. Google Cloud. dora.dev/research/2025/dora-report
• Horani, O. M., Al-Adwan, A. S., Yaseen, H., Hmoud, H., Al-Rahmi, W. M., & Alkhalifah, A. (2025). The critical determinants impacting artificial intelligence adoption at the organizational level. Information Development, 41(3), 1055–1079. doi.org/10.1177/02666669231166889
• Noda, A., & Tacho, L. (n.d.). Measuring AI code assistants and agents. DX. getdx.com/research/measuring-ai-code-assistants-and-agents
• Senanayake (2026). Determinants of artificial intelligence adoption in software development: A systematic literature review using the TOE framework. consensus.app/papers
• Senanayake, N. S. M. I. S., Wijayanayake, J. I., & Jayasinghe, S. (2026). AI adoption in software development: TOE-based drivers of productivity enhancement in Sri Lanka. In 2026 IEEE International Research Conference on Smart Computing and Systems Engineering (SCSE) (pp. 1–6). IEEE. doi.org/10.1109/SCSE70081.2026.11499838
• Uren, V. S., & Edwards, J. S. (2023). Technology readiness and the organizational journey towards AI adoption: An empirical study. International Journal of Information Management, 68, Article 102588. doi.org/10.1016/j.ijinfomgt.2022.102588