How to Write a Postmortem Timeline That Separates Detection Time from Diagnosis Time

Two clocks, one row

Most postmortem timelines have a single row that reads like this: 14:07 — alert fired; on-call investigated and mitigated. That row hides two different failures. Detection asks how long it took for any signal to reach a human. Diagnosis asks how long it took for a human to hold a correct model of what was breaking. A lean team that collapses both into one timestamp cannot tell whether its next investment belongs in monitoring or in runbooks, access, and observability.

This is a writing and process problem, not a tooling problem. The templates most teams already use do not have a dedicated diagnosis field. PagerDuty’s published postmortem template, for example, asks for the time the contributing factor began, the time of the page, the time the status page was updated, the time of significant actions, and the end time, each with a link to the tool or log that shows how the timestamp was arrived at. It does not ask when someone first understood the failure. The fix is to add boundary rows to the template you have, with the same evidence discipline the template already demands.

Define four boundaries before the incident

The SRE Book is direct that postmortem criteria should be defined before an incident occurs so everyone knows when a postmortem is necessary. The same logic applies inside the document. Agree on four boundary events in advance, and write them into the template as empty rows:

  1. Failure onset. When the system first departed from normal behavior, whether or not anyone could see it. This is often reconstructed from logs and is the least certain row in the table.
  2. First signal. When any signal reached a human or a system that would act on it: an alert firing, a synthetic check failing, a customer report arriving. This is detection time.
  3. First correct diagnosis stated aloud. When someone wrote or said a hypothesis that turned out to be the actual cause. This is diagnosis time. It is not the same as acknowledgment, and it is not the same as the first hypothesis.
  4. First mitigation action. When someone changed the system to reduce impact, even if the change was temporary.

Two derived gaps fall out of these rows: signal-to-human (detection) and human-to-correct-model (diagnosis). They point at different remediation work. A long detection gap is a monitoring or coverage problem. A long diagnosis gap is a context, runbook, access, or observability problem. An action item that does not name which gap it closes is hard to prioritize six weeks later.

Use the existing template as a floor, not a ceiling

PagerDuty’s template is a good floor because it insists on traceability: every timestamp should link to the tool or log that justifies it. Keep that discipline and extend the table. A workable column set for a small team is:

Time (UTC) Boundary Event Evidence link Known lag or caveat
14:02:10 Failure onset Replica lag crossed threshold pg_stat_replication sample, 10s scrape Stats flush interval; see below
14:07:00 First signal Growth-rate alert fired Alert history entry Alarm evaluation window
14:11:00 Acknowledged On-call claimed page Pager tool log —
14:26:00 First correct diagnosis Identified WAL retention exhaustion on primary Chat transcript quote Reconstructed from chat
14:31:00 First mitigation Paused batch writer Deploy log —

The caveat column matters more than it looks. Monitoring data has its own lag, and a timeline that treats a dashboard timestamp as ground truth can understate or overstate detection. PostgreSQL’s documentation notes that each server process flushes accumulated statistics to shared memory just before going idle, but not more frequently than once per PGSTAT_MIN_INTERVAL milliseconds (one second unless altered at build time), so displayed totals lag actual activity. The same documentation notes that in the default configuration, statistics accessed within a transaction are cached until that transaction ends, so a long-running query can show static numbers. On the AWS side, CloudWatch preserves alarm history for 30 days and marks each state transition with a unique timestamp, and an alarm invokes actions only when it changes state. If your first signal came from an alarm that had been in ALARM for twenty minutes before anyone looked, the alarm history will show the transition, not the human noticing. Record both.

Three rows, not one: paged, acknowledged, understood

For a team of two to fifteen engineers who are their own on-call rotation, the most common diagnosis gap is not ignorance. It is context switching. The on-call is mid-task, acknowledges the page, finishes a sentence or a deploy step, and only then begins real diagnosis. If the timeline records only “acknowledged,” that gap is invisible and the postmortem will misattribute the delay to tooling.

Split the human side into three rows:

  • Page fired. The alert or escalation event, with its tool timestamp.
  • Acknowledged. The moment a human claimed the page. This is a tool log, not a memory.
  • First correct diagnosis. The moment a correct hypothesis was stated in chat or on a call. This is usually reconstructed, and should be labeled as such.

The middle row is the one lean teams most often skip, and it is the one that makes the context-switch cost visible without blaming the person. Write it as a system state: “page acknowledged while a deploy was in progress,” not “on-call was slow to respond.”

Log the hypotheses that died

Diagnosis is iterative, and the final answer is rarely the first guess. A timeline that records only the correct diagnosis reads as if the team knew immediately, which makes the diagnosis gap look shorter than it was and hides the evidence that killed the wrong hypotheses. Add a short hypothesis log beneath the boundary table:

H1 (14:12): Primary overloaded by traffic spike. Killed at 14:18 — request rate flat in the load balancer metrics.
H2 (14:19): Network partition between primary and replica. Killed at 14:23 — replication connection alive, lag growing steadily.
H3 (14:26): WAL retention exhausted, replica cannot catch up. Confirmed by archive status and disk usage.

Each hypothesis carries the evidence that killed it. This is the decision-log part of the timeline, and it is what a reviewer needs to re-derive the diagnosis gap rather than trust the author’s summary. It also produces better action items: H1 and H2 were killed by metrics that existed, which suggests the diagnosis gap was not a missing-signal problem. H3 required a check that lived in a runbook nobody had opened, which suggests a different fix.

Keep the language factual

Blamelessness is a formatting constraint as much as a cultural one. The SRE Workbook’s critique of a poorly written postmortem calls out animated language and dramatic descriptions as distractions that erode psychological safety, and recommends providing verifiable data to justify the severity of a statement. The SRE Book defines a blameless postmortem as one that focuses on contributing causes without indicting any individual or team, and assumes everyone involved did the right thing with the information they had. Google’s CRE guidance goes further for shared documents: express the role of humans in terms of role rather than name.

In practice, that means the timeline says “the on-call” and “the alert,” not names. It says “the runbook did not cover WAL retention” rather than “the on-call did not know.” It says “the alert fired at 14:07 and was acknowledged at 14:11” rather than “response was slow.” The boundary rows make the gaps visible; the language keeps them reviewable.

A worked example, clearly hypothetical

Consider a PostgreSQL replica-lag incident on a small team running pgBackRest and streaming replication. The alert fires at T+0. The on-call acknowledges at T+4 while finishing a deploy. The first hypothesis is a traffic spike; it dies at T+11 when the load balancer metrics are flat. The second hypothesis is a network partition; it dies at T+17 when the replication connection is shown alive. The correct diagnosis — WAL retention exhausted on the primary, so the replica cannot catch up — lands at T+19. Mitigation begins at T+24 when a batch writer is paused.

The boundary table looks like this:

  • Detection gap: T+0 (signal) minus failure onset. If onset was T−6, detection took six minutes.
  • Acknowledgment gap: T+4 minus T+0. Four minutes of context switching.
  • Diagnosis gap: T+19 minus T+4. Fifteen minutes of sense-making, including two dead hypotheses.
  • Mitigation gap: T+24 minus T+19. Five minutes to act once the model was correct.

Now attach action items to rows. The detection gap suggests a review of the alert’s evaluation window and whether a leading indicator existed earlier. The acknowledgment gap suggests either an escalation policy change or an honest acknowledgment that a single on-call rotation will always have context-switch cost, which is a staffing tradeoff rather than a tooling bug. The diagnosis gap suggests a runbook entry for WAL retention and a dashboard panel that shows archive status next to replica lag. The mitigation gap suggests a pre-approved action — pausing the batch writer — that does not require a second pair of eyes at 2 a.m.

None of these action items is novel. What is novel is that each one names the row it closes, which makes the postmortem review meeting a prioritization exercise rather than a re-litigation of the incident.

What the timeline is for

A postmortem timeline with explicit boundaries is a decision log, not a narrative. The SRE Workbook notes that a good postmortem lets readers know what happened, how the issue was mitigated, and how users were impacted — and that the recovery section is where many drafts fall short. The boundary rows are how the recovery section gets specific. The SRE Book lists a monitoring failure, which usually implies manual incident discovery, among common postmortem triggers; a timeline that separates detection from diagnosis is how you tell whether a given incident was that kind of failure or a different one.

For a team without a dedicated reliability function, the timeline is also the only durable record of how the on-call rotation actually behaves under load. The recovery checklist you write before you need it — the one that covers restores, access, and escalation — is a different artifact, but it shares the same discipline: name the step, name the evidence, name the owner. If your team is still building that habit, the recovery checklist article is a useful companion to this one.

Review checklist

Before the postmortem meeting, check the timeline against these questions:

  • Are all four boundaries present — failure onset, first signal, first correct diagnosis, first mitigation?
  • Is each timestamp linked to a log, alert history entry, or chat transcript, with a note on any known lag or retention limit?
  • Are “page fired,” “acknowledged,” and “first correct diagnosis” separate rows?
  • Are the dead hypotheses recorded with the evidence that killed them?
  • Does each action item name the gap it closes?
  • Is the language factual — roles rather than names, system states rather than judgments?

If the answer to the first three is yes, the timeline will survive review. If the answer to the last three is yes, the action items will survive the next incident.

FAQ

What if we genuinely cannot reconstruct the diagnosis time?

Say so. Label the row “reconstructed from chat, approximate” and give the range. A labeled estimate is more useful than a false precision, and the SRE Workbook’s guidance to provide verifiable data applies to the caveat as much as to the timestamp.

Does this add process overhead during the incident?

No. The boundary rows are filled in after the incident, from logs and chat. The only in-incident change is that someone notes the first correct hypothesis when it is stated, which is usually already happening in the incident channel. If your team uses a scribe role, that is the scribe’s job.

Should we add these fields to our template permanently?

Yes, as empty rows. A template that asks for the diagnosis boundary will get one; a template that does not will get a narrative. The PagerDuty template’s insistence on evidence links is the model to copy.

What about incidents where diagnosis and mitigation overlap?

Record both rows anyway. Overlap is information: it usually means the mitigation was a diagnostic action, such as restarting a process to see whether the symptom clears. That pattern is worth naming in the postmortem because it often indicates a missing observability signal.