Yesterday was another reminder that moving everything to the cloud doesn’t mean outages become somebody else’s problem. Microsoft may own the servers, but those of us responsible for IT still get to explain why email isn’t working, what employees can do, and when things will be back. That last question gets a little difficult when the answer is essentially, “We’re waiting on Microsoft, too.”
Naturally, the internet immediately filled up with explanations. Bad updates, AI, layoffs, leadership decisions. Apparently, an Outlook outage turns everyone into a Microsoft infrastructure engineer. I have my own concerns, but I want to separate what we know from what we suspect.
Microsoft’s updates described servers failing health checks, a process responsible for secure connections becoming overwhelmed, and recovery attempts that weren’t producing the expected results. They were investigating configuration changes and testing recovery approaches that involved isolating infrastructure and gradually reintroducing traffic. That tells us something about the symptoms and the difficulty of recovery. It doesn’t give us a complete root cause.
My theory is that the original problem may have created a second problem that kept the outage going. A server comes back online, gets hammered by clients trying to reconnect, becomes overwhelmed, and drops out again. The remaining servers inherit more work, and the cycle continues. Engineers call one version of this a metastable failure. I call it a very expensive game of whack-a-mole. To be clear, that’s a possible explanation, not a confirmed account of what happened inside Microsoft.
It also raises a question about how you recover these environments. If your tools for pushing the fix depend on the same infrastructure that’s struggling, you have another problem. “Just roll it back” sounds simple until the system needed to deliver the rollback is having a bad day, too. Finding the original mistake and getting the service healthy again can become two very different jobs.
Which brings me to AI and the human brains we seem increasingly eager to replace.
I use AI. I like what it can do, and I’m putting it to work myself. But I have a real problem with treating it as a substitute for experienced people who understand the systems keeping a business running. AI can generate a configuration, write a script, and explain—with impressive confidence—why its answer should work. Experienced people bring judgment, context, and common sense. They know to ask whether making that change right now is a good idea in the first place.
Common sense is knowing that a technically valid change can still be an operationally terrible decision. It’s remembering the undocumented dependency, questioning an unusually clean test result, or saying, “Let’s slow down before we turn one problem into six.” AI can produce language that sounds like that judgment. I’m still putting a person with experience in charge of the decision.
I have no evidence that AI caused this outage, or that staffing decisions caused the recovery problems. Those conclusions need evidence. But my concern with Microsoft, and any company racing to put AI into everything, is whether the investment in experienced engineers, reliability, and recovery is keeping pace with the push to ship faster. If AI becomes the justification for having fewer people who understand the machinery, that’s a business decision customers will eventually feel.
There is tremendous value in the person who knows why a system was built a certain way, which shortcuts were taken, and what happened the last time someone tried the obvious fix. That knowledge rarely makes a product demo. At 3 a.m., with services down and customers waiting, it can be the most valuable thing in the company. Before calling those people an expense AI can eliminate, I’d suggest spending an outage on the bridge call with them.
From the CIO chair, there’s also a lesson we can act on ourselves. We need to check how many of our backup plans depend on the same services as our primary plan. If email is unavailable, how do we reach employees? If the collaboration platform is affected, too, where do we go? Can we get to our contacts and recovery instructions without signing into the environment having the incident? “We’ll email everyone instructions” probably needs another pass through the planning process.
I want to see Microsoft’s post-incident review, especially what made recovery difficult and what they’ll change. In the meantime, I’ll keep using AI, keep testing our recovery plans, and keep valuing the people who understand what happens when the technology stops cooperating. Faster tools are useful. So is a human being with enough experience and common sense to say, “Hang on a minute.”
