Your Postmortem Is Not Finished When the Document Is

Three weeks after a production incident, the follow-up list has started to blur into the delivery backlog. One item has an owner but no date. Another was marked complete when a ticket moved columns. A third still sounds important, although the service has changed since the incident.

For a CTO or Head of Engineering, the awkward question is no longer whether the postmortem happened. It is whether the organisation can distinguish residual exposure from stale paperwork.

That distinction matters because incident work passes through several stages. Restoring service ends the immediate disruption. Analysing the incident develops a better explanation. Action items propose changes. Implementation alters the system. Verification checks whether the relevant exposure actually changed. A polished document can stop at stage two while everyone assumes stage five.

What the research establishes, and what it does not

Research on software incidents gives us a useful map of this handoff, but it does not justify a simple rule such as “actions older than 30 days cause repeat incidents.” The studies examined here do not estimate how the age of unfinished remediation work affects recurrence. There is no defensible universal expiry threshold in this evidence.

That absence should change the management question. The age of an item may be a warning, but age alone cannot tell you whether the action is still valuable, whether another change has removed the exposure, or whether the original proposal was weak. An old item can represent unmanaged risk. It can also represent a decision that nobody has had the courage to cancel.

The stronger evidence concerns the path from an incident to proposed change. Pope and Sillito (2025) analysed hundreds of action items from public incident reports. The proposed changes responded to triggers, failure propagation, response problems, and recovery difficulties. They targeted many parts of the system and concentrated largely on reliability and performance. Crucially, the proposals often addressed narrow scenarios connected to the incident.

That is understandable. A painful event creates urgency around the conditions people have just seen. Yet a proposal remains a hypothesis: if we change this part of the system, a similar failure path should become less likely or less damaging. The ticket does not prove the hypothesis. Closing the ticket does not prove it either.

Earlier work by Pope and Sillito (2024) places post-incident action items at the intersection of requirements engineering and software evolution. This framing is more useful than treating them as administrative leftovers. Each material action competes for capacity, carries assumptions about the future system, and needs enough definition for a team to decide what change is actually required.

Recovery, learning, and risk reduction are different work

The temptation to collapse these stages begins during the incident. The organisation needs service restored quickly. Sillito and Kutomi (2020) found that mitigation can involve rollbacks, fixes, capacity changes, resource restoration, and restarts. They also observed that resolving the original problem may not resolve the cascading conditions created during the failure. Some urgent interventions can make the situation worse.

This creates a practical boundary. The change that restores service is chosen under pressure and judged against immediate symptoms. Follow-up risk reduction asks a different question: what should change so that a trigger is contained earlier, propagation is limited, detection improves, or recovery becomes safer?

A postmortem can produce valuable learning across that boundary. Sillito and Pope (2024) found that incident learning covered triggers, contributing factors, recovery, and the role people play in reliability. That breadth matters. Recurrence may depend on architecture, monitoring, operating procedures, decision rights, or interactions among them. A narrow “fix the root cause” task can miss the conditions that shaped the outcome.

Learning still has to survive contact with the normal delivery system. A plausible operating risk is that remediation must compete in the same queues as features, maintenance, dependencies, and other risk work once the incident loses urgency. If leaders review only ticket status, they see motion without knowing whether exposure changed.

Replace the action list with a risk-reduction record

A useful governance mechanism should preserve the reasoning from the incident without pretending that every proposed action deserves implementation.

Postmortem action item governance should connect each material action to a specific failure path, one accountable owner, a deliberate review date, a verification signal, and an explicit closure decision.

Use five fields.

1. Failure path

Describe what the action should interrupt. Be specific enough to test: a configuration change bypasses progressive delivery; a scaling limit remains invisible; a dependency failure creates retries that amplify load; a recovery procedure depends on an untested backup.

This is more precise than “prevent recurrence.” Complex incidents rarely repeat in an identical form. The useful target is the mechanism that allowed impact to grow.

2. Owner

Name one person or one standing forum accountable for moving the decision. Ownership does not mean doing all the work. It means ensuring that the action is scheduled, reframed, or closed deliberately when new evidence appears.

Without that accountability, an action can remain socially important while being operationally optional.

3. Review date

Set a date to reassess the exposure and the proposed response. This is not a universal service-level agreement for all incident tasks. The date should reflect severity, expected change in the system, and the next point at which a decision can still alter risk.

Age becomes useful when it triggers judgment. It should not become a crude target that encourages teams to close tickets cosmetically.

4. Verification signal

Define what you expect to observe if exposure is lower. A failover exercise completes without manual recovery. A production limit becomes visible before saturation. A fault-injection test shows that retries remain bounded. A runbook rehearsal exposes no missing access or dependencies.

The signal need not prove that recurrence is impossible. It should provide better evidence than implementation status.

5. Closure decision

Allow three outcomes: complete with evidence, cancel with rationale, or reframe. Cancellation is legitimate when the architecture has changed, the scenario is no longer material, or the proposed action would add more complexity than protection. Reframing is necessary when verification exposes a different failure path.

This is the discipline that an ordinary backlog lacks. It makes disagreement visible and prevents “done” from becoming a synonym for “the code changed.”

A Monday experiment

Choose one old, material item from a recent incident. Do not begin by asking why it is late. Write the failure path it was meant to interrupt and the signal that would show reduced exposure. Then give one owner authority to schedule it, cancel it, or rewrite it.

Review that record within the next delivery cycle. If the team implements the change, inspect the verification signal. If the action no longer makes sense, close it with the reason. If nobody can state the failure path, the proposal is not ready to consume capacity.

Run this experiment on one item before redesigning the whole incident process. The evidence base does not support a universal aging threshold or guarantee that this record will reduce recurrence. It does support treating incident learning as socio-technical change rather than paperwork.

A postmortem is finished when the organisation can explain what it learned, what it chose to change, and what evidence supports the claim that exposure is lower. Until then, the document records intent. It does not demonstrate risk reduction.

References

Pope, M., & Sillito, J. (2024). Post-incident action items: Crossroads of requirements engineering and software evolution. In Proceedings of the 1st IEEE/ACM Workshop on Multi-disciplinary, Open, and RElevant Requirements Engineering (pp. 1–7). ACM. https://doi.org/10.1145/3643666.3648578

Pope, M., & Sillito, J. (2025). Characterizing the system evolution that is proposed after a software incident. In 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME) (pp. 321–332). IEEE. https://doi.org/10.1109/icsme64153.2025.00037

Sillito, J., & Kutomi, E. (2020). Failures and fixes: A study of software system incident response. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) (pp. 185–195). IEEE. https://doi.org/10.1109/icsme46990.2020.00027

Sillito, J., & Pope, M. (2024). Failing and learning: A study of what is learned about reliability from software incidents. In 2024 IEEE 35th International Symposium on Software Reliability Engineering Workshops (ISSREW) (pp. 295–302). IEEE. https://doi.org/10.1109/issrew63542.2024.00093

AI disclosure: AI was used to help create this article. Responsibility for the published content remains with Nico Thümler.