By DailyMail
Facebook, Instagram, WhatsApp all remain down in a four-hour-long global outage that experts say was caused by an internal error on Monday morning.
Between 11.50am EST and 11.52am EST, Facebook made a series of updates to its border gateway protocol (BGP) which caused it to ‘disappear’ from the internet.
The BGP allows for the exchange of routing information on the internet and takes people to the websites they want to access.
Facebook’s changes included withdrawals which removed its properties from the domain naming system it operates and essentially made it impossible for anyone to connect to the sites because they could no longer be found online.
Facebook is yet to explain why the changes were made or why they caused such outages.
A New York Times reporter tweeted that staff at an unspecified Facebook office were unable to use their keycards to gain access to the premises on Monday.
It’s unclear if that is related to the outage or not.
NetBlocks, which tracks internet outages and their impact, estimate the outage has already cost the global economy $160m (£117 million).
Experts say that even once the problem is fixed, it will be a nightmare task rebooting the system because it is so large.
The scandal-battered company’s shares had dipped by 5 percent on Monday amid the outage and after a whistleblower went public on Sunday night with claims of how it regularly places profit above morals.
CloudFare’s Chief Technology Office John Graham-Cunningham tweeted on Monday that Facebook accidentally ‘disappeared’ from the internet after making a ‘flurry’ of updates to its BGP – Border Gateway Protocol.
‘Between 15:50 UTC and 15:52 UTC Facebook and related properties disappeared from the Internet in a flurry of BGP updates,’ he said.
Facebook does not use CloudFare but it runs one of the world’s largest DNS resolvers. When sites go down because of failures in DNS systems, CloudFare tries to repair them.
Usman Muzaffar, SVP of engineering at CloudFare, explained to DailyMail.com: ‘Humans access information online through domain names, like facebook.com and DNS converts it into numbers, called an IP address, computers use.
‘From what we understand of the actual issue —it is a globalized BGP configuration issue. In our experience, these usually are mistakes, not attacks.
‘Border Gateway Protocol (BGP) is the routing protocol for the Internet. Much like the post office processing mail, BGP picks the most efficient routes for delivering Internet traffic.
‘Today, the directions for how to get to Facebook’s DNS server’s addresses weren’t available (and seem to still be unavailable).
‘Without being able to contact the DNS servers, visitors trying to reach a Facebook property, like facebook.com, will not get an answer and so the page won’t load.’
Facebook has not commented on the cause of the outage.
‘We’re aware that some people are having trouble accessing our apps and products.
‘We’re working to get things back to normal as quickly as possible, and we apologize for any inconvenience,’ Andy Stone, the company’s director of communications, said.
Initially, there were reports that AT&T, Verizon and T-Mobile were all down too, however those reports stemmed from people being unable to access Facebook-run apps on their mobile devices.
Kevin King, Director of Communications at Verizon, said there was no outage with the network but that it appeared as though some users were having difficulty accessing certain apps or sites on their Verizon devices.
Almost all of those who reported problems had issues with mobile internet, and not with making calls or texts.
Cyber security specialist Jake Moore said there is a ‘chance’ the issue could be related to a cyber attack.
He PA: ‘There have been many reports and I’m struggling to find out exactly what has happened- I’m reading it could be DNS related, which means there is an issue with the connection not knowing where to go to your device.
‘It could well be a human error or a software bug lurking in the shadows but whatever it is Facebook needs to do its best to mitigate the problem of causing more panic about this.
‘The biggest problem is fears over a cyber attack but as we saw from Fastly in the summer I would hedge my bets on that not being the case as we’re talking about one of the biggest companies in the world, but there’s always a chance.’
Adam Leon Smith, of BCS, The Chartered Institute for IT and a software testing expert, said: ‘The outage is caused by changes made to the Facebook network infrastructure. Many of the recent high-profile outages have been caused by similar network level events.
‘It is reported by unidentified Facebook sources on Reddit that the network changes have also prevented engineers from remotely connecting to resolve the issues, delaying resolution.
‘Notably, many organizations now define their physical infrastructure as code, but most do not apply the same level of testing rigor when they change that code, as they would when changing their core business logic.’
