We use the internet every day and almost all modern applications have something to do with the internet. So what happens if a country loses internet availability in half the cities? When an ISP makes even the tiniest mistakes the outcome can be close to catastrophic which can result in delayed flights, people being unable to work, online transactions not being made and all kinds of chaos ensues. This is exactly what happened to Japan this week when Google made a big blunder and botched a Border Gateway Protocol (BGP) causing half of the country to lose its internet connection!
— Channel 1 World (@Channel1World) August 29, 2017
The wide Internet outage caused by Google messing up occurred on Friday that resulted in half of the country being unable to use the web. The few moments it lasted and the aftermath was a big reminder of how important internet related services are and how any disruption in the chain could affect day to day activities and services that we believe to be unrelated to the internet.
— Justin (@ThePickMan236) December 16, 2017
On Friday, several people reported that they had lost connection to the internet. What’s surprising is that the outage lasted only 40 minutes but what’s even more surprising is that the damage and inconvenience people had to face in that short time frame. The outage was reportedly enough that a fair majority of people were unable to access their train reservations and bank assets, according to local media reports. The mess up by Google mostly affected the services of NTT Communications, which runs the largest service provider in the country OCN, and KDDI.
— Arnaud ASCENSI (@AAsc49) August 29, 2017
According to reports, by botching up a border gateway protocol (BGP), the number of peer prefixes being sent by Google to Verizon increased dramatically. The downtime occurred because Google supposedly “leaked” a route table to Verizon, which is also known as a BGP route hijack. If you aren’t aware of what BGP is it’s a protocol for distributing routing information between networks which is as important as it sounds. This caused the traffic that was meant to be directed towards NTT Communications and KDDI to be sent to Google instead. Since Google didn’t know what to do with all the extra traffic, things ended up being the way they did.
Quite a few Twitter users complained about their internet not working with the phrase “connection disruption” in Japanese trending on the social media service.
“I was using the Suica app and all of a sudden I got a message saying I was disconnected, although my connection was supposed to be working. I couldn’t restart it and can’t charge it now,” an artist who uses the Twitter handle @kanzakihiro reported to more than 53,000 followers.
BGPmon, an iindustry-recognized expert in network monitoring and routing security, said over the weekend that more than 135,000 prefixes on the Google-Verizon route were the cause of the internet disruption. “Google accidentally became a transit provider for Jastel by announcing peer prefixes to Verizon. Since Verizon would select this path to Jastel it would have sent traffic for this network towards Google. Not only did this happen for Jastel, but thousands of other networks as well,” the company wrote.
The report added, “It’s easy to make configuration mistakes that can lead to incidents like this… In this case , t appears a configuration error or software problem in Google’s network led to inadvertently announcing thousands of prefixes to Verizon, in turn, propagated the leak to many of its peers.”
Google later admitted its mistake in a statement to The Asahi Shimbun, with a spokesperson saying, “We set wrong information for the network and, as a result, problems occurred. We modified the information to the correct one within eight minutes. We apologize for causing inconvenience and anxieties.”
Even after the internet was restored for the day, users still experienced relatively slow internet speeds. This just goes to show, just how important internet is and even how the biggest companies can mess up every now and then,