Microsoft 365 Global Outage Preliminary Post-Mortem: How Three Small Failures Froze the World's Email

Posted by Reda Fornera on 2026-11-17
Estimated Reading Time 18 Minutes
Words 2.9k In Total

On November 16, Microsoft published its preliminary Post-Incident Report (PIR) — the company’s first official post-mortem on the November 2026 Microsoft 365 global outage, the roughly 24-hour blackout that took down Outlook, Exchange Online, and parts of Teams, Copilot, SharePoint, and OneDrive for users in more than 70 countries. The report, dated November 15 and posted to the Microsoft 365 admin center under incident IDs EX1550543/MO1550821, was summarized on X by Charles Lamanna, EVP of Copilot, Agents, and Platform. Everything below about why the outage happened is Microsoft’s own account, from the PIR text as reproduced by Office Watch and reported by GeekWire, Ars Technica, and BleepingComputer — a first-party, self-interested postmortem, not an independent audit. I’ve kept the reported mechanism separate from my own reading, and I’ll flag where Microsoft’s explanation stops short. It is not the first time multi-service blackouts have hit this way — see also the synchronized ChatGPT, Claude, and Grok outage earlier this year — but this preliminary report is the first official post-mortem Microsoft has published on this outage, and a final PIR due November 20 will supersede it.

What happened in the Microsoft 365 global outage: a day without email

To the people living it, the outage was brutally simple. You opened Outlook — desktop, web, or phone — and your mailbox wasn’t there. Mail that trickled through was often hours late. The PIR says the “most significant impact affected Exchange Online, with most users being unable to access their mailbox,” while Copilot, Teams, SharePoint, OneDrive, Defender, and Purview hit problems in the parts that depend on Exchange. Teams chats and meetings mostly kept working — a detail that becomes important later.

The blast radius went well beyond inconvenience. The Guardian reported hospitals falling back to paper records, jails unable to release prisoners, and schools losing contact with parents.

The outage ran from 4:20 PM UTC on Thursday, November 12, to 4:10 PM UTC on Friday, November 13 — just under 24 hours of active impact. Microsoft’s own timeline tells the story in phases:

  • 1:07 AM UTC, Nov 12 — an automated failover moves Teams traffic onto the same gateway servers that handle Exchange Online. Nothing visibly breaks.
  • 1:07 AM–4:00 PM UTC — load on the gateway front-end servers rises gradually; isolated server failures begin in some regions, and automated mitigations apply without stopping the buildup.
  • 4:20 PM UTC — Microsoft declares the incident after a high-priority alert on Exchange Online. Automated systems immediately begin rerouting traffic to “healthy” endpoints.
  • ~4:30–8:30 PM UTC — the cascade spreads; the company pauses all deployments, restarts servers, and watches them fail again.
  • 1:53 AM UTC, Nov 13 — engineers disable some of the automated traffic redirection that has been feeding the cascade.
  • 7:20 AM UTC — the pivotal move: a mitigation rolls the defective code back to a known-good state on the first slice of capacity.
  • 11:35 AM UTC — Europe returns to healthy levels; Asia-Pacific recovery completes around 11:00 AM.
  • 2:17 PM UTC — global webmail availability returns to normal.
  • 4:10 PM UTC — active impact declared over; Microsoft moves into backlog-recovery mode, and delayed email continues to flush through queues into the weekend.

Roughly 38 hours after it began, Microsoft posted its public all-clear on X.

The July latent gateway bug that preloaded the outage

The seed of the Microsoft 365 global outage was a change Microsoft shipped in July. According to the PIR, a fix “prepared specifically for Microsoft 365 networking gateway servers” introduced a latent defect: under certain load conditions, lock contention inside an internal Windows process caused a machine’s CPU and memory to be exhausted until the server simply stopped responding.

Three details deserve attention. First, the change was routine in every visible way: it “went through safe deployment and saturated globally over several weeks,” then ran in production for “more than three months” with no observed impact. Second, the flaw was specific in scope — Microsoft states that “neither the underlying defect nor the fix applies to broader Windows deployments.” Third, and most curiously, GeekWire noted that Microsoft “hasn’t said what the fix was meant to address” in the first place.

The preconditions: less capacity, more traffic

Generic stock photo of Ethernet cables on a network patch panel — illustrative imagery of gateway networking hardware only, not a photo of Microsoft's data centers or the affected gateway servers

A landmine needs someone to step on it. In the PIR’s account, two preconditions loaded the trap.

The first was capacity. “Nearly 14% of server capacity for the networking gateway was out of service due to an unrelated issue from the previous day.” On its own, Microsoft says, the service had enough headroom to absorb the loss.

The second was a workload shift. Microsoft maintains two independent, heterogeneous networking gateways for resilience — a primary fronting Exchange Online and an alternate. About fifteen hours before the outage, at 1:07 AM UTC, Teams failed over from the alternate onto the primary, adding load and changing the traffic mix. Microsoft calls this “part of normal operations” — and doesn’t say what prompted it.

The PIR’s summary of how those two preconditions combined: “Taken together, the increased load and the reduced server capacity, caused the network gateway to encounter the latent bug from July. This caused requests to build up until servers ultimately stopped responding.”

How Microsoft’s own defense spread the failure

Here is the part of the report that reliability engineers will be chewing on for a long time.

When servers stop responding, Microsoft’s traffic management systems do exactly what they’re supposed to do: “automatically route users to other healthy endpoints” — behavior that happens all the time without incident. But on November 12, the “healthy” endpoints were healthy only in the sense that they hadn’t yet been pushed over the same load threshold. Every redirected user added weight to a server carrying the same July code. “The redirected traffic increased the likelihood for the same problem to occur on the remaining endpoints. This caused a cascading failure from server to server as the same amount of load was spread across fewer servers. In less than two hours, the previously unknown defect was encountered across our fleet and rendered them unresponsive. At its lowest, more than 80% of servers were unresponsive.”

The system’s automated self-healing — the mechanism designed to mask small failures — was the spread mechanism. My inference, clearly not Microsoft’s phrasing: the rerouting logic treated a load-induced failure as an isolated hardware fault, and its remedy guaranteed to replicate the fault everywhere. The defense became the attack vector. It took until 1:53 AM the next morning to switch off some of that automated traffic redirection.

Thundering herd: when recovery causes more crashes

Once you understand the spread mechanism, the botched recovery stops being surprising and starts being inevitable.

The PIR: “as servers returned to service, they were immediately overwhelmed by waiting traffic and became unresponsive again.” Every client locked out for hours was still queued up, and a rebooted server got hit with the entire backlog at once — froze again, and rejoined the pile of the dead.

Microsoft concedes its tooling wasn’t built for this: “Our normally automated recovery processes were not designed to restore service from this broad unresponsive state.” Worse, the usual escape hatches are designed to undo recent changes: “since the main code defect was a latent bug from several months previous, our normal rollback, flighting, and feature flags were unable to quickly revert the change.”

The eventual fix was almost anticlimactic: a patch that, in the PIR’s root-cause account, “returned the code path to the previously known good state in Windows,” applied by taking large groups of servers — at times more than half of a region — out of service, patching them, and restarting in controlled batches.

Containment failed twice

The most uncomfortable section of the report is also its shortest. Microsoft’s gateway is composed of “thousands of ‘capacity units’” — shards of hardware, capacity, and networking — organized into regions, with bulkheads between them designed to keep a failure from spilling from one region into another. Those blast walls did not hold.

“Due to a separate issue,” the PIR says, “the normal bulkheads between these different partitions did not stop the cascading failure. This is what allowed the impact to spread across our network gateway infrastructure globally.”

A separate issue. Which is not explained — not what it was, not whether it’s a configuration error, a second bug, or something structural. That unexplained clause is the single biggest reason the final report due November 20 matters. Everything else in the preliminary report has a root cause and a plausible fix. This one doesn’t yet.

There’s a second containment failure, and this one Microsoft does quantify, sort of. Services that could escape the broken gateway did: Teams, OneDrive, SharePoint, and Office apps “failed out of the faulty network gateway infrastructure,” which is why chat and meetings mostly survived. But in the PIR’s phrase, “not all services, like Exchange Online, can fail out of this network gateway for scale reasons.” Email — the hardest workload to shed — sat on the one gateway that couldn’t shed it. Microsoft’s flagship resiliency mechanism was available to everything except the service being destroyed.

The testing gap — and what Microsoft says it’s changing

Why didn’t anyone catch a bug that turned out to be fleet-wide and fatal? Microsoft’s own answer is startlingly frank:

“Our normal load testing did not detect the latent issue because the traffic mix and SKUs used in our labs. The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on weaker hardware with fewer cores.”

Put plainly: the labs ran older, smaller servers with a synthetic traffic mix; production runs newer machines with more cores, and the bug gets worse precisely as core counts rise. The testing fleet was a statistical sample that excluded the machines that mattered.

The scrutiny practically writes itself, and GeekWire made it explicit: Microsoft spent $45.2 billion in capital expenditures in its most recent quarter — much of it on exactly this kind of infrastructure — yet its representative lab hardware lagged far behind what it runs in production. The point isn’t underinvestment; it’s that testing fidelity, not spending, is the gap.

To its credit, the PIR lists completed mitigations: the code reversion, added capacity, extra monitoring on traffic-redistribution patterns and connection-handling health, “guardrails for traffic distribution to avoid cascading impact,” and an extended email-delivery window so nothing queued during the event bounced. Lamanna was contrite: “We are deeply sorry for the impact to all our customers and recognize the impact that this has on their normal business & operations.” Microsoft classified the incident as “a technical issue not related to cyber or AI/agents” — a claim about its own investigation, not an external finding.

Same week, different fragility: the FAA ground stop

Generic stock photo of an air traffic control tower at an airport — illustrative imagery only, not a photo of any FAA facility involved in the November 13 ground stop

While Microsoft’s gateways were still limping back, a separate and unrelated failure struck U.S. airspace — and it’s worth stressing up front: these are two different incidents, resolved independently, with no official link.

At approximately 3:30 PM ET on Friday, November 13, the FAA issued a nationwide ground stop “out of an abundance of caution” after its legacy telecom equipment “became unstable when data began flooding the FAA FTI network, at a rate of 1,000 connection attempts every 4 to 5 seconds.” Instability hit 17 of the agency’s 22 air route traffic control centers before spreading into terminal radar facilities. Air traffic resumed at 8:15 PM ET the same evening; about 3,500 airborne aircraft landed safely, and the FAA counted roughly 12,462 flight cancellations that day — with the FAA’s own caveat that not all cancellations were directly caused by the ground stop.

The FAA’s preliminary Q&A, published Sunday, November 15, points to two sites on its contractor-run communications network: “a radar site in Amarillo, Texas and a radio communications site in El Dorado, Arkansas,” where network provider L3Harris “had been doing work at the El Dorado site, including deployment of a new router.” When the anomalies began, “L3 Harris blocked those two locations and the network stabilized,” and as a precaution the broadcasting function was disabled on all new routers — and on roughly 60 similar routers systemwide. Transportation Secretary Sean Duffy told ABC’s “This Week” that the Amarillo device, installed about three weeks earlier, “started to flood our system as well.”

The ruling-out language deserves to be quoted exactly, because the timing looked suspicious: three days before the ground stop, the FAA had extended SMART — its new cloud-based AI traffic-planning platform, part of a 12-year, $875 million contract with Air Space Intelligence — to all 22 en route centers. On cyber, the FAA stated: “Based on all available evidence, we have no indication that this was a cyberattack or an outside nefarious actor.” On SMART: “No. The two systems are not related. SMART is a data aggregator, and it does not feed information into the air traffic control system.”

Nor was there any Microsoft connection, however tempting the timeline. Homeland Security Secretary Markwayne Mullin raised it himself (as GeekWire reported): “I want to make sure everybody understands that there is no connection between this and Microsoft. I know a lot of people are making that jump.” Duffy, on the AI tool: “There was no possible way SMART would have impacted what happened today.” The routers have been pulled for testing at an L3Harris facility in Florida, with the FBI assisting; the root-cause investigation continues.

What makes the FAA story a genuine co-star rather than filler is what it shares with Redmond — and it isn’t technology or geography. “Our new IP switches did not go down. It’s the old hardware that went down,” Duffy said. One incident was latent software meeting peak load; the other was aging transport meeting new equipment. In both, a small change colliding with a stressed system produced a nationwide cascade. In both, the first response was to cut the failing component out and let the survivors carry on.

The pattern: two cascades, one week

This was a week of overlapping infrastructure failure warnings — the Black October breach wave that rolled into November covered the security side of the same fragility, and now the reliability side has its own receipt.

A hyperscale cloud provider and the U.S. air traffic control system both suffered cascading failures within 24 hours of each other. In both post-mortems the proximate cause is small — a July software fix here, a router installation there. The damage came from what happened next: automatic mechanisms that rebalanced load while assuming the failure was local, plus blast-radius protections that either failed for an unexplained reason or held only partially.

The dependency question won’t wait for Microsoft’s final report. Hospitals, jails, and schools don’t buy “email” from Microsoft; they bought redundancy and got a shared front door that cannot fail out “for scale reasons.” As AI features like Copilot are woven deeper into these services, every workload that rides the gateway widens the blast radius of exactly the class of bug that froze this one — the same dynamic this site covered when Microsoft leaned into always-on autopilot agents at Build 2026. NCC Group’s Matt Hull framed it to The Guardian as the need for “not only cyber resilience, but business resilience.”

And the market beat: Anthropic’s IPO order book — reported at close to seven times oversubscribed by Bloomberg on Sunday — closed today, November 17, with pricing due Wednesday, November 18, and trading under ANT on Nasdaq expected Thursday, at up to $90.4 billion including its $26 billion private placement — a raise that, if the placement completes, would beat SpaceX’s $75 billion, the current record among private-company listings, and make it the biggest listing ever, per TNW’s report of the Bloomberg figures. The same week two nationwide infrastructures seized up, the market is pricing in a decade of building more of them.

What to watch on November 20

The final PIR is due Friday, November 20 — and three specific admissions will define how it ages:

  1. The unexplained bulkhead failure. What was the “separate issue” that let a regional failure go global? That’s the open question.
  2. The “scale reasons” admission. Does Microsoft commit to making Exchange Online actually fail out of a dead gateway — or only to more capacity, instead of decoupling?
  3. The testing-fidelity fix. Lab hardware versus newer multicore production machines is a repeatable failure mode. Testing on production-grade hardware, permanently — or reclassified as a one-off?

The preliminary report closes the easy questions. The hard one — why every containment layer failed at once — opens Friday.

References and further reading


Please let us know if you enjoyed this blog post. Share it with others to spread the knowledge! If you believe any images in this post infringe your copyright, please contact us promptly so we can remove them.



// adding consent banner