Incident Response 18 min read

Writing the Incident Ticket Someone Else Will Read

A ticket titled "Email down" closes at 13:30 UTC on 2026-11-07. Its last two entries read "Fixed. Queue normal." and "Closing. Cause: provider throttling." By the following Friday, that title is a row in the weekly operations report and a red segment on the monthly availability chart. The closing note is the summary a director forwards to the head of customer support. Nobody reading those copies was on the call. They learn that email was down, though the deferrals were to one recipient provider. They learn that the cause is known, though the ticket held a guess.

The ticket is often the first shared written account of an incident, and its fields are built to be copied. Written for the next reader, its entries say what was seen, how it is known, what is assumed, and what is still open. Its words stay true when someone lifts them out of the thread. Say Only What the Evidence Supports: Incident Language for the People Who Decide covers the language of live updates, and Say Only What the Evidence Supports in the Postmortem covers the review afterward. This piece covers the record between them and uses the same evidence states: observed, estimated, or unknown, with confirmed and correlated as qualifiers. The incident below is constructed, and so is every example.

A one-line ticket is the right record for a routine request that closes in minutes and touches nothing shared. The practice here is for tickets that coordinate work across people, change hands between shifts, or get summarized for someone who owns a decision.

Where a ticket's words travel

Four fields in particular get copied out of the ticket, by people and by reporting tools. Titles show up in queue views, handoff notes, the weekly report, and notification subject lines. Where reporting reads ticket fields, status values feed response and availability dashboards and impact figures get carried into monthly reports. A closing note becomes the summary in a problem record or the first line of a postmortem timeline.

A ticket also changes hands. An alert or a support agent opens it, an on-call engineer works it, and a second shift inherits it. A manager reviews it, and someone else may close it. Each handoff can bring a reader who lacks the context. The Google SRE book's chapter on managing incidents puts the most important information at the top of the incident document. When a ticket coordinates the incident, its top fields are the first a hurried reader sees.

Status words follow the working list from Say Only What the Evidence Supports: investigating, assessing impact, mitigating, restoring, monitoring, technical recovery complete, and resolved. Map them onto whatever states your tool offers. Each carries only what the team can show at that moment.

A title that survives being copied

A title has three parts, in this order: the service or workflow, then the observed condition, then the scope or status.

Service or workflow comes first because queue views and reports often show only a title's opening words. Next, the observed condition: what someone or something saw, in terms a reader can check. Last, the scope or status: how far it reaches or where work stands, including what is still unknown. A cause stays out of the title until the evidence confirms it. Titles are often read alone, so a hypothesis there travels without its qualifier.

Weak titleRewrittenWhat changed
Email downOutbound mail relay: delivery to one recipient provider deferred since 2026-11-07 08:58 UTC, password-reset email delayedComponent, condition, narrower scope
Website slowCustomer portal: page load 95th percentile above 4 s since 2026-11-07 14:10 UTC, cause unknownMeasured condition, start time, stated unknown
VPN broken for everyoneRemote access VPN: new connections failing for staff in one region, existing sessions passing trafficScope from what was checked
Database fixed after restartReporting database: queries timing out from 2026-11-07 02:10 UTC, service restarted 2026-11-07 02:40 UTC, monitoringAction and status replace a claimed outcome
Bad deploy took down the APIInventory API: 5xx rate above 5 percent after release 7.3, rollback under evaluationRelease as a timing fact; open decision named
Backups failed, data lost?Nightly backup job: 2 of 9 volumes incomplete, restore points for those 2 unverifiedA count; the question becomes what is unverified
Payroll export broken by vendorPayroll export: transfer rejected by receiving service since 2026-11-07 01:00 UTC, response under reviewObserved rejection; responsibility left to the review
Possible hack on build serverBuild server: unexpected outbound connections observed, security on-call engagedObservation plus routing; speculation removed
Printers not workingBadge printers at two sites: jobs held in queue, other print queues emptyTwo sites, one queue type, what was checked
Monitoring alert stormDisk-space alerts: 340 in 10 minutes from one collector, host readings unconfirmedCount and source; data doubts stated

Rewrite the title when the scope changes, and record the change in the work log: "Title changed 2026-11-07 09:33 UTC: scope narrowed from the whole relay queue to one recipient provider." The log entry is what tells the next engineer the title moved on purpose.

What each field holds

FieldHoldsKeep out
TitleService or workflow, observed condition, scope or statusA cause, a person, adjectives that severity already carries
StatusThe status word the team can support right nowResolved before the exit criteria are met; effort words like "working on it"
Severity or priorityThe level your policy assigns from impact and urgencyA level chosen for attention instead of by policy
ImpactThe affected capability; observed, estimated, or unknown; any count with its basis; the as-of timeUptime percentages with no consequence; a claim of zero customer effect before someone owns that assessment
ScopeWhat is affected, and what was checked and found normal"Everything" or "nothing else" without a check behind it
Observed conditionThe reading, its source, the date and time with its zone, and the expected valueThe interpretation of the reading
EvidenceLinks to dashboards, saved queries, log extracts, and where raw copies are heldCredentials, customer records, large pasted logs
Current assessmentEach hypothesis labeled as one, with the check that would confirm or weaken itA hypothesis written as a finding
ActionsWhat was done, by which role, on what scope, at what time, and the result observed afterAn action with no result, such as "restarted, looks good"
DecisionsWho decided, between which options, at what timeDecisions made in chat with no pointer from the ticket
Next updateA time, or a condition that triggers the update"Soon"
Exit criteriaThe measurable conditions for resolved, set while the incident is still openCriteria written after the graph turned green
Follow-ups and residual riskWork items with IDs and owners; what stays exposed"Keep an eye on it" with no owner
Related recordsIDs of any related change, alert, vendor case, problem, or security recordThe contents of a restricted record

One morning, written twice

At 08:55 UTC on 2026-11-07, a scheduled weekend-sale campaign starts sending about 60,000 messages over three hours through the outbound mail relay. It uses the same sending addresses as the customer portal's password-reset email. Times from here on are UTC, on that date unless another is given. At 09:12 a queue-depth alert fires on that relay. At 09:20 the support desk logs three customer contacts about reset email that never arrived.

The work log, as it often gets written:

09:14  Email server down. Investigating.
09:25  Customers can't reset passwords. Mail is broken.
09:45  Restarted mail service. Queue still big.
10:05  Looks like the provider is blocking us. Opened a case with them.
10:30  Paused the campaign. Queue going down.
12:30  Fixed. Queue normal.
13:30  Closing. Cause: provider throttling.

The same morning, written for the next reader:

2026-11-07 09:14 UTC  Title: Outbound mail relay: queue depth above alert
                      threshold, delivery impact unknown.
                      Queue-depth alert ALR-7731 at 09:12: 4,300 messages queued
                      on relay-b against a usual range under 200 (queue panel).
                      Status: investigating. Next update 09:45, or sooner if
                      scope changes.
2026-11-07 09:31 UTC  Observed in the relay log: 9,900 of 10,000 queued messages
                      are addressed to one external mailbox provider, deferred
                      since 08:58 with temporary (4xx) replies reading "too many
                      messages from this sender, try again later." Deliveries to
                      other domains are completing (relay log, 09:00 to 09:30).
                      Impact (observed): password-reset email to that provider's
                      addresses is delayed (relay log; 3 contacts in the support
                      queue). Number of affected customers unknown.
2026-11-07 09:33 UTC  Title changed, scope narrowed: Outbound mail relay:
                      delivery to one recipient provider deferred since
                      2026-11-07 08:58 UTC, password-reset email delayed.
                      Status: assessing impact. Next update 10:20.
2026-11-07 09:40 UTC  Assessment (hypothesis): the deferrals are correlated with
                      the campaign that started at 08:55 (campaign report).
                      They fit rate limiting by the receiving provider. Checks:
                      per-destination attempt rate against deferral onset; a
                      response from the provider.
2026-11-07 09:45 UTC  Action: mail service restarted on relay-b by the on-call
                      engineer, approved by the incident lead.
                      Result at 09:52: queue 16,300 and growing about 300 a
                      minute (queue panel). All provider attempts still deferred
                      (relay log).
2026-11-07 10:05 UTC  Action: provider case VND-5521 opened by the on-call
                      engineer with deferral samples. Result: reference
                      returned; response pending.
2026-11-07 10:20 UTC  Decision (marketing operations owner): pause the campaign,
                      over continuing unchanged.
2026-11-07 10:22 UTC  Action: campaign paused by marketing operations. About
                      29,000 of 60,000 submitted to the relay by then (campaign
                      report). Status: assessing impact, unchanged until a
                      reading shows fewer deferrals or a shrinking queue. Next
                      update 10:50.
                      Result at 10:30: queue about 25,400 and growing about 10 a
                      minute (queue panel). No campaign mail submitted since
                      10:22 (campaign report). All provider attempts still
                      deferred (relay log).
2026-11-07 10:45 UTC  Observed: retries to the provider about 840 a minute
                      (relay log). Action: retry rate to the provider capped at
                      250 a minute, effective 10:46 (mail platform team,
                      CHG-5520).
2026-11-07 10:50 UTC  Observed after the 10:45 cap: 1,040 provider attempts
                      from 10:46 to 10:50, 4 deferred, against 20,400 of
                      20,400 deferred from 10:22 to 10:46 (relay log counts).
                      Queue about 24,500 and falling (queue panel).
                      Impact (observed): delivery to the provider has resumed
                      (relay log). Reset email already queued is still delayed
                      (queue panel). Number of customers affected unknown.
                      Exit criteria: queue under 200 for 60 minutes; a test
                      reset email to an address at the provider delivered within
                      2 minutes; no new support contacts on this symptom for 60
                      minutes.
                      Status: restoring. Next update 12:30, or sooner when the
                      queue falls under 200.
2026-11-07 11:00 UTC  Per-destination check: first delivery attempts to the
                      provider rose from about 10 to about 310 a minute at
                      08:55; deferrals began at 08:58 (relay log). Provider
                      case: no response yet. Rate-limiting hypothesis still
                      correlated, unconfirmed.
2026-11-07 12:28 UTC  Queue under 200 (queue panel). No provider deferrals since
                      12:15 (relay log). Status: monitoring, exit timer running
                      to 13:28. Next update 13:30, or sooner on any new
                      deferral.
2026-11-07 13:30 UTC  Summary: deferrals to one recipient provider began at
                      08:58 (relay log) and delayed password-reset email to its
                      addresses. The queue was back under 200 at 12:28; at 12:27
                      the oldest queued message was 3 h 29 min old (queue
                      panel). Number of customers affected: unknown. Cause:
                      unconfirmed; the provider's reading and whether other
                      traffic from our sending addresses contributed are open in
                      MAIL-413.
                      Cause: hypothesis (09:40). VND-5521 checked 13:25: no
                      reply.
                      Exit criteria set at 10:50, all met. Queue under 200 from
                      12:28 to 13:30, past the 60 minutes required (queue
                      panel). Test reset email T-1315 to a provider address,
                      sent 13:15, delivered in 41 s, inside the 2-minute limit
                      (relay log). Saved support search "reset email not
                      received", 08:55 to 13:30: last match 10:58, none in the
                      last 60 minutes. No deferrals from 12:28 to 13:30 (relay
                      log). Status: resolved.
                      Residual risk: deferrals may return when bulk mail
                      resumes, since the provider's limit is unconfirmed.
                      Follow-ups: MAIL-411 per-destination send rate for bulk
                      mail, MAIL-412 detection gap, MAIL-413 causal review with
                      the provider (all mail platform team); MKT-208 campaign
                      restart plan (marketing operations); SEC-2231 release of
                      the relay log-rotation hold (security owner).

The second log is evidence-based. Each reading names where it came from, so the next engineer can find it and check the troubleshooting. Two readers show what that buys. One is a manager at 10:00 deciding whether to warn the support desk. The other is a reviewer six weeks later deciding what goes in the postmortem.

QuestionFrom the first logFrom the second log
Was customer-facing email affected, and which?"Mail is broken"Password-reset email to one provider's addresses; other domains delivering from 09:00 to 09:30
How many customers?No answerUnknown, with three contacts observed
Did the restart help?Queue still big, so probably notNo change in queue growth or deferrals: seven minutes later the queue was still growing about 300 a minute
What came before the deferrals dropped?"Queue going down" at 10:30, while it was still growingThe 10:22 pause and the 10:45 retry cap; 4 of 1,040 attempts deferred from 10:46
When did each thing happen?Entry times only, no date or zoneEvent times, dated, in UTC
What does resolved mean here?Queue normalThree criteria, set at 10:50, each with its evidence at 13:30
Is the cause established?It appears soA correlated hypothesis, still one at close
What is still open?Nothing visibleCause, send-rate control, detection gap, campaign restart plan, log-rotation hold

A provider account manager who reads "Cause: provider throttling" in a forwarded summary could read it as blame. A later reviewer could take it as a settled finding. The ticket never held more than a timing correlation and the wording of the provider's responses. Its 13:30 summary is the line written to be forwarded. It names the one provider, when deferrals began and when the queue fell back under 200, the oldest message's wait, the unknown customer count, and what is still open about the cause, each with its source.

Five situations that pressure the wording

Reopening a ticket marked resolved. On 2026-11-08, the campaign resumes at a reduced rate at 08:55, and deferrals from the same provider return at 09:02. Add an entry that says what came back and how it relates. Set the status the evidence supports. Revise the exit criteria, since the first set was met with the campaign paused:

2026-11-08 09:10 UTC  Reopened. Marketing operations resumed the campaign at
                      08:55 at the MKT-208 rate of 120 a minute (campaign
                      report). Deferrals from the same provider returned at
                      09:02, seven minutes later (relay log). Provider attempts
                      at 09:02: about 120 a minute (relay log). After the retry
                      cap, the provider accepted about 260 a minute with 4 of
                      1,040 deferred (relay log, 2026-11-07 10:46 to 10:50 UTC).
                      Impact (unknown): password-reset delivery to that
                      provider's addresses unchecked since the campaign resumed.
                      Number of customers affected unknown.
                      Assessment (hypothesis): a fixed per-minute threshold fits
                      this poorly. A limit on volume over a longer window, or on
                      sender history, is untested. Check: deferrals against
                      rolling 60-minute volume in the relay log, and the
                      provider's reading in VND-5521. Review in MAIL-413.
                      The 2026-11-07 resolution entry is unchanged: its exit
                      criteria were met with the campaign paused. Revised exit
                      criteria (incident lead): the three from 2026-11-07, to be
                      met while the campaign sends at 120 a minute. Status:
                      assessing impact. Next update 09:40.

The entry is an excerpt; the 09:40 update and those after it follow the morning's pattern. If your process opens a new ticket for a recurrence, link the two both ways so separate counts still show the relation.

A symptom the monitoring missed. The queue alert measured queue depth, which crossed its threshold only after messages piled up behind the deferrals. Record the gap in the same ticket as times and signals, and leave the fix to a follow-up item with an owner:

2026-11-07 13:27 UTC  Detection gap: deferrals began 08:58 (relay log). Queue
                      alert ALR-7731 fired 09:12. First support contact 09:20
                      (support queue). No per-destination delay alert was found
                      in the alert inventory. Follow-up MAIL-412: alert on
                      deferral rate by destination (mail platform team).

Leadership asks for the cause before the investigation is done. At 10:05 the support director asks what caused this. The ticket answers with the current assessment, its evidence, what would confirm it, and when the next answer is due:

2026-11-07 10:08 UTC  Asked at 10:05 by the support director: what caused this?
                      Current assessment (hypothesis): deferrals began about 3
                      minutes after a 60,000-message campaign started (relay
                      log; campaign report). The provider's deferral text says
                      too many messages from this sender (relay log).
                      That fits rate limiting by the provider in response to
                      our volume. Unconfirmed: the provider's own reading, and
                      whether other traffic from our sending addresses
                      contributed. Next answer by 11:00, from per-destination
                      volume and the provider case.

A broad action that feels urgent. At 10:10 someone proposes flushing the provider's deferred messages so they all retry at once. Record the proposal, its expected effect, its risk, and its disposition before anyone acts:

2026-11-07 10:14 UTC  Proposed (10:10): flush the provider's deferred messages
                      so all retry at once. Expected: about 21,600 immediate
                      retries, one for each message queued to the provider at
                      10:10 (relay log), against the suspected limit. Whether
                      retries count against that limit is unknown; if they do,
                      a flush risks a longer deferral. Declined by the incident
                      lead in favor of narrower controls on sending rate.

A security indicator inside a reliability incident. At 10:35, an engineer reading the relay log sees about 400 failed logins on the mail submission port. They come from one outside address and started at 10:15. The reliability ticket records the observation, the hand-off, and the reference, and keeps the details in the security record:

2026-11-07 10:37 UTC  Security indicator observed at 10:35 in the relay log:
                      about 400 authentication failures on the submission port
                      from 10:15. Reported to the security on-call at 10:36,
                      acknowledged 10:37; details are in SEC-2231. No connection
                      to the deferral condition is established. Relay logs from
                      08:00 are preserved as found. Log rotation is held by the
                      mail platform team with the security owner's agreement,
                      until the security owner releases it in SEC-2231.

Guesses about intent or origin stay out of the reliability ticket. Some changes alter evidence: blocking the address, clearing logs, or rotating credentials. Those go through the security incident procedure and whoever it authorizes. Preserve copies as the procedure directs.

The reviewer's questions

Before the status moves to resolved, someone else reads the ticket the way a later reader will. A peer, the incident lead, or the manager on duty can do it:

  • Does the title still describe what was observed, at the final scope?
  • Does every impact statement say observed, estimated, or unknown, with its basis and as-of time?
  • Does every action carry the result observed after it?
  • Was every promised update made on time, including early triggers?
  • Is every hypothesis labeled, and does every cause stated as established have a review behind it?
  • Were the exit criteria written before they were met, and does the closing entry point to evidence for each one?
  • Does every follow-up have an owner and a record ID?
  • Does every related change, alert, vendor case, or restricted record have a pointer, without copying restricted content?
  • Would the closing note read correctly as the only line someone sees?

Any "no" gets fixed before the ticket closes. A "no" on exit criteria, impact, or cause also blocks the move to resolved.

Before you save an entry

Run these checks on every entry:

  • A date as YYYY-MM-DD and a time in UTC or with its offset, on every entry. Times inside an entry inherit its date and zone unless they cross midnight. Every Local Timestamp in an Incident Record Needs an Offset shows what an unlabeled local time does to a timeline.
  • A source for every reading: the dashboard panel, the log and its time range, the query, the person who reported it. Save the query or a copy where the view can change.
  • Separate sentences for claims that rest on different evidence, with hypotheses labeled as hypotheses.
  • A result after every action, even when the result is "no change."
  • On a status change while the incident is open, the next update time or the condition that will trigger it.

The checks fit one order. Copy this skeleton and keep only the lines an entry needs, in this order, so a reader knows where each one sits. Leave a line out until it has something true in it: a result not yet seen is added later with its own time, as the 10:22 entry adds its 10:30 result. Times inside the entry inherit its date and zone, as the first check says. Ticket-level fields such as the title, severity, and related records sit outside the log; the closing entry repeats the follow-ups and residual risk.

YYYY-MM-DD HH:MM UTC  Observed: the reading (its source, time range, and saved
                      copy if the view can change), against the expected value
                      or state.
                      Scope: what is affected, what was checked and found
                      normal, and what is still unchecked.
                      Impact (observed, estimated, or unknown): the affected
                      capability, any count with its basis, and as of HH:MM
                      if that differs from the entry time.
                      Assessment (hypothesis until a review supports it): the
                      explanation. Check: what would confirm or weaken it.
                      Decision (who decided, and when if not at the entry time):
                      the option chosen, over the alternatives considered.
                      Action: what was done, on what scope, by which role, and
                      when if not at the entry time.
                      Result at HH:MM: the reading after, or no change, with its
                      source.
                      Exit criteria, when this entry sets or revises them: the
                      measurable conditions for resolved.
                      Status: the status word. Next update HH:MM, or sooner if a
                      named condition occurs.

Then read the entry as the person who will copy one line of it into a report without the thread around it. If that line changes meaning on the way, rewrite it before saving.

References

Sources verified 2026-11-06.

Incident Records Evidence