Santosh Janardhan, VP, Infrastructure at Facebook, in a blog post, has revealed the main reasons for the outage for nearly 6-hours on October 4, 2021.
Nearly 3.5 billion customers of Facebook, Instagram, Messenger and WhatsApp could not access the social media and messaging services during the break down.
Facebook, the world’s largest seller of online ads after Google, was losing about $545,000 in U.S. ad revenue per hour during the outage, according to estimates from ad measurement firm Standard Media Index.
Facebook, the # social media network, said sorry for the inconvenience caused by today’s outage across our platforms.
“We’ve been working as hard as we can to restore access, and our systems are now back up and running. The underlying cause of this outage also impacted many of the internal tools and systems we use in our day-to-day operations, complicating our attempts to quickly diagnose and resolve the problem,” Facebook said in a statement.
“Our engineering teams learned that configuration changes on the backbone routers that coordinate network traffic between our data centers caused issues that interrupted this communication. This disruption to network traffic had a cascading effect on the way our data centers communicate, bringing our services to a halt,” Facebook said.
Facebook did not reveal the name of the IT service provider or the supplier of backbone routers that coordinate network traffic between its data centers. Facebook does not reveal its IT budget to keep the service running.
Leading router suppliers include Cisco, Huawei, HPE, among others.
“Our services are now back online and we’re actively working to fully return them to regular operations. We want to make clear at this time we believe the root cause of this outage was a faulty configuration change. We also have no evidence that user data was compromised as a result of this downtime,” said Facebook.
