A Typo That Shut Down Amazon

On February 28th a typo took out the internet. Many may have felt or heard of the 4-hour outage Amazon AWS had in one of its main datacenters. They were initially troubleshooting some minor issues they were having but due to a mistyped command, one of their engineers inadvertently wiped out several of their primary operating servers which took down the entire datacenter!

“At 9:37AM PST, an authorized S3 team member using an established playbook executed a command which was intended to remove a small number of servers for one of the S3 subsystems that is used by the S3 billing process. Unfortunately, one of the inputs to the command was entered incorrectly and a larger set of servers was removed than intended.”

The Amazon S3 system is used by 148,213 sites according to market research firm SimilarTech. Sites such as Salesforce, Netflix, Spotify, Pinterest and Buzzfeed.

It then took them Amazon 4 hours to get everything running again, unfortunately that was in the heart of the business day and many businesses were severely impacted.

“During AWS’ four-hour disruption, S&P 500 companies lost $150 million, according to analysis by Cyence, a startup that models the economic impact of cyberrisk. US financial services companies lost an estimate $160 million, the company estimates.” From <https://www.businessinsider.com/aws-outage-hurt-internet-retailers-except-amazon-2017-3>

It’s estimated that over the next year, Amazons AWS services alone will be a 14-Billion-dollar business and as it stands it already represents 56% of all their operating income. Last month Amazon said that their AWS arm brought in $926 million in operating income from $3.53 billion in revenue in fourth quarter 2016 which is up 47% from this time last year.

So, does the recent outage mean that we should be concerned about the cloud? No way. The fact was under reported but AWS has several other datacenters that continued to operate fine. Those that were affected by the outage have options to utilize georedundant datacenters to avoid interruptions in just this type of scenario. Many AWS customers may not have subscribed to those redundancies due to additional costs in services and since AWS has had such a great track record. I’m sure now many are considering those available services.

As you can see from the quick numbers shared, Amazon AWS is experiencing massive growth in the cloud services arena. Others such as Microsoft Azure and Google Cloud Platform are also experiencing massive growth. In the tech world, there are so many more benefits to moving to the cloud and we’ll continue to see products and services move that way.

There are simply way too many benefits to moving to the cloud, as technology continues to advance.

For now I thought it would be fun to share a few details about what happened with the recent Amazon AWS outage especially for those of you where were affected by it. In the near future I hope to write about why so many things are moving to the cloud and what advantages can be of benefit.