H/T to reader Skip..
Microsoft 365 took a broad hit today (Nov. 12). Outlook was the epicenter, but Teams, Copilot, OneDrive, SharePoint, Office on the web, and even Defender XDR were caught in it.
Peak: roughly 130,000 Outlook reports on Downdetector around 1 p.m. ET, with tens of thousands more on 365 and Teams.
Scope: started in the U.S./Americas, then Microsoft acknowledged all regions were affected.
Not resolved cleanly: by late afternoon they were still at it — "We're opening all available pathways to restore service, as our actions taken thus far have not provided the anticipated relief." That's not a confident postmortem voice.
What Microsoft actually said the cause was: a large number of servers became unexpectedly unavailable, failed automated health checks, and the remaining infrastructure got strained by the load. They explicitly said the elevated load was not a traffic spike. They also said they found no recent changes that correlate with the start of impact — which is corporate for "we don't know why our own servers fell over."
Worth noticing: AWS and Google users were reporting problems at the same time. Amazon insisted its services were "operating normally" and blamed "an unrelated disruption elsewhere on the internet" for inaccurate reports. When three major providers wobble in the same window, the interesting question is what they share — DNS, CDNs, transit providers, upstream dependencies — and that's precisely the layer nobody gives straight answers about. but "coincidence" isn't a technical explanation either.
Wierd my Microsoft Outlook work email disabled but I can get it on the cellphone.
ReplyDeleteNothing like putting so much of your world on a internet system, eh?
Haven’t been getting emails on my Live/Outlook account since mid afternoon. Still not working at 2048 CT
ReplyDeleteIf the average person knew the condition of the fiber-optic cables these networks run on, they would be amazed anything works at all. I run a small fiber contracting company that does a find-and-fix for one of the network providers for these big companies. Based on what I've seen, there should be no fear of AI taking over. One good thump on any number of splices and the whole thing goes down with loss of signal.
ReplyDeleteSame here in the UK
ReplyDelete"Not a Hack. Not a Mystery. Just One Company Too Big to Fail, Grading Its Own Outage." Microsoft already told you it wasn't a cyberattack. So drop the dramatic frame — this wasn't an adversary, it was a config change, a health-check system that ejected servers into a stampede, and a "fix" that didn't hold. Fourteen hours, 70+ countries, Congress and two spy agencies locked out of their own email.
ReplyDeleteHere's the eye-opener: the actual postmortem — timeline, root cause, remediation — goes to enterprise admins through a portal Microsoft owns, on a timeline Microsoft's own service description sets, in a format Microsoft picks. No public version promised. Personal Outlook users aren't even in the room.
So the company that caused the outage is the company that writes the report, decides who reads it, and decides when the 48-hour clock starts. That's not accountability. That's a closed loop with a status page bolted on.
One config change, four continents, zero public sentences owed. That's what "too big to fail" actually looks like.
Microsft has always sucked, this should come as no surprise.
ReplyDeleteBACK TO THE TIDDIES!!!
Switched to pure Linux last year; no problem
ReplyDelete