Microsoft Had Two Outages Before Dinner. The Second Was Caused by Its Own Safety Net.

6 minutes ago

4 min read

A darkened data center aisle with one unlit row of server racks, a single red status light glowing on a silent machine

Microsoft had two outages on November 12. The first one was the warmup. Microsoft Teams broke at 9:45 in the morning and was declared restored at 10:53. Then the main event started at 1:26 in the afternoon, and by 6:23 that evening the company was still telling customers to be patient while servers came back one at a time.

Nearly 130,000 people reported Outlook problems to Downdetector just after 1 p.m. Eastern. Thousands more flagged Teams, Copilot, SharePoint, OneDrive, Azure and the Microsoft Store. Microsoft's own list of impacted services included Exchange Online, SharePoint Online, Microsoft 365 Copilot and Defender XDR. Hold that last one, because it's going to matter in a minute.

Here's the part I can't get past. Microsoft says a number of servers became unavailable because they failed automated health checks. Those checks exist for a good reason. A malfunctioning machine should stop taking customer traffic before it does damage. The check is the seatbelt.

On November 12, something went wrong with the checks themselves. The system started pulling servers out of service, the remaining capacity got strained, the strain produced connection failures, and the failures looked like demand. A feedback loop with a public status page. Microsoft's own description is that the elevated load wasn't more traffic, it was servers becoming unexpectedly unavailable and straining what was left.

The fix tells you everything. Microsoft paused all deployments, moved traffic onto alternate routes, restarted machines, and then started bypassing the health-check component on a subset of servers to see if that brought them back. The repair was to stop trusting the thing that was supposed to be watching.

Two Outages Is a Pattern, Not a Coincidence

Teams fell over on resource contention at 9:45 and was called resolved at 10:53. The bigger incident landed at 1:26. Microsoft says it hasn't found a recent change that coincided with the start of the second one.

Read that sentence the way an auditor would. If you can't find what changed, you can't promise it won't change again. Every postmortem for the next year is going to have that hole in the middle of it, and every customer is going to nod politely at a root cause that stops one sentence short.

Everyone Blamed the Internet. AWS Said It Was Fine.

By early afternoon the story was already written. Macworld rounded up Slack, Reddit, Cloudflare, Gmail, gaming networks and half of western Europe as casualties. Then an Amazon spokesperson called the reporter to say AWS had been fine all day. It was.

That correction separates two very different failures. One is a backbone problem that takes everything down with it. The other is a single vendor's control plane eating its own customers while the rest of the internet hums along. Guess which one happened. The scary version of this story is infrastructure. The actual version is one company's bad afternoon.

The Security Product Was on the Casualty List

Defender XDR was in Microsoft's own impacted list. That's the stack thousands of organizations use to notice when something is wrong. During the outage, the thing that watches for trouble was itself having trouble.

I've spent most of my career on the defensive side of this, so I'll say the uncomfortable part plainly. An outage that blinds detection isn't an inconvenience. It's a coverage gap with a timestamp on it. If you run Microsoft security products, your next tabletop exercise should start with the console unreachable and the telemetry late, because on November 12 that stopped being hypothetical.

Microsoft says there's no indication a cyberattack caused any of this. I believe them. But an adversary doesn't need to cause your outage to profit from it. They only need to read your status page.

Copilot Clocked Out Too

Copilot was on the list as well. For a few hours the most expensive autocomplete in enterprise software wasn't available, and a lot of people wrote their own sentences. The work got done. Some of it was probably better.

That's not a shot at AI. It's an observation about where the value actually sits. Microsoft is selling agentic everything to the same customers whose email it couldn't route that afternoon. The pitch is that intelligence lives in the cloud and compounds every quarter. The bill of materials is that the intelligence stops when a health check misfires.

What the Incident Report Will Say

My prediction. A subset of servers. A health-check component. Traffic management. Gradual recovery. No evidence of data loss. And nothing about why the checks failed, because they still don't know.

What it won't say is the part worth knowing, which is how close the system was running to its own limits before any of this happened. Concentrated infrastructure is efficient right up until the moment it becomes a single point of failure in a compliance badge. Microsoft 365 is one control plane for a large share of the working world. When the plane wobbles, the world wobbles, and recovery takes hours because there's nowhere else to go.

Via Rail told passengers they might not get their trip emails and pointed them at a website instead. That's the whole outage in one sentence. The system that was supposed to inform you that the system failed was also part of the system that failed.

So no, I don't need another status update with the word mitigated in it. I want the incident report. Root cause, change history, and the paragraph where somebody explains why a health check had the authority to remove healthy capacity at scale.

Until then, the honest summary of November 12 is this. Microsoft had two outages before dinner, and the second one was caused by the system that was supposed to prevent the first. The internet was fine. Microsoft wasn't.

Sources

Rich Washburn is a technologist, strategist, and Founder & Chief AI Architect of ARIA AI Labs, working at the intersection of AI, infrastructure, communications, and capital. He also serves as Managing Partner and Chief AI Officer at Eliakim Capital.