Users of Microsoft Teams went without the program for almost three hours before Microsoft eventually deployed fixes which solved the outage issue.
A message came up on screen for users stating that HTTPS connections had failed.
Teams initially went down just after 2pm yesterday, with Microsoft 365’s Twitter account for incident updates sending out the following message:
We’ve determined that an authentication certificate has expired causing, users to have issues using the service. We’re developing a fix to apply a new certificate to the service which will remediate impact. Further updates can be found under TM202916 in the admin center.
— Microsoft 365 Status (@MSFT365Status) February 3, 2020
Fixes began to be rolled out at around 4:20pm, with issues being completely fixed for most Teams users at 5pm.
At around 9:30pm, Microsoft notified users that the problem was completely solved.
We successfully deployed the fix to the affected infrastructure and conducted additional remediation actions to resolve the issue. More information can be found under TM202916.
— Microsoft 365 Status (@MSFT365Status) February 3, 2020
“Yet again we’ve seen another hyperscale provider fall victim to an expired security certificate,” said Andy Simpson-Pirie, CTO at Cyberfort Group. “This was an easily avoidable lapse in security management, and again it is businesses paying the price, suffering disruption when there didn’t need to be any.
“However, we know this kind of mistake is more common than not. The fact this happened to such a large and established technology business like Microsoft highlights the need for security teams to embrace the use of an automated system which can carry out certification checks and regular updates, error-free.
“By doing so, organisations can remove the burden of this mundane but important task from security teams so they can focus on defending against the evolving array of threat actors within the cyber world.”
Kevin Bocek, vice president security strategy and threat intelligence at Venafi, said that this incident “happens every day to Global 5000 businesses”.
“These mistakes can cause a service or application to go down for hours, days, and, in some cases, even longer,” he said. “This is not a unique occurrence, and unfortunately Microsoft Azure and LinkedIn have experienced outages due to expired certificates in the past.
How to manage digital certificates (ahead of AWS refresh)
“The main issue is that certificates act as authenticators for machines, and they authorise machine-to-machine connections and communications. Keys and certificates serve as machine identities, and they are critical to making today’s global economy work. When they expire, business stops.
“The problem is that most businesses and government agencies are using thousands of certificates, but they don’t have the insight or automation needed to replace certificates before they expire, and an outage based on a failed certificate is really painful, not just for consumers but also for the IT and security teams trying to fix them.
“Finding an expired certificate manually is like looking for a very specific needle in a stack of needles.”
Teams, which is used by around 20 million company employees and executives worldwide, is planning to have Skype integrated into its platform some time in Q1 2020.
