How a single typo brought the internet to its knees

jeff bezosMireya Acierto / Getty

This week, an Amazon Web Service (AWS) failure caused a massive outage all over the internet. Now we know why: a typo.

The company released a detailed report explaining what happened. An employee entered what they thought was a routine command to remove servers from an S3 subsystem. By mistake, they entered a larger number than intended. These servers supported two other S3 subsystems, both of which manage the storage and metadata for the entire region. Down went the dominoes.

AWS assures everyone that it’s prepared for the occasional failure. Fixing the employee’s error should have been as simple as rebooting the subsystems. However, AWS admitted that it hasn’t actually restarted those subsystems in years, and S3 has grown considerably in the meantime.

If you’ve ever rebooted an older computer and notice it chugging on start-up, you’ll understand the feeling AWS must have had while waiting for the system to come back.

In response, AWS says it’s putting in safeguards to prevent this kind of error from happening again. Apparently, creating these safeguards had been in the works for a while, and AWS is making it a priority after this outage. Considering it even brought down AWS’s own conference mid-speech, this is probably for the best.

NOW WATCH: 7 little-known benefits of Amazon Prime

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s