No More Hustleporn: AWS Is Too Big

By Yiren Lu — Dec 18, 2021

Tweet by Corey Quinn

Chief Cloud Economist at @DuckbillGroup. Father to @QuinnyPiglet & @theMunchQuinn. he/him Get my snarky take on AWS news: http://lastweekinaws.com/t/

So a question I posed in slack.lastweekinaws.com led to an unfortunate realization on my part:

@awscloud is too big, and has too many customers for the overall good of society.

"Well were things more reliable before @awscloud?" No! Good lord no! The difference is that I could have a bad day and take down a hospital. AWS has a bad day and takes down all the hospitals.

It's the simultaneous outage of everything that's the problem.

The worst part is that I don't even have the slightest clue how to fix it. You can plan and plan and plan around this. You can build out multi-region or multi-cloud until the cows come home.

And then one of your third parties did none of this and you're just as down.

A multi-day full outage of us-east-1 will have an observable effect on the world economy.

That is not an exaggeration. I don't know how to fix any of this. I just know that we should be talking about it.

And also in slack.lastweekinaws.com we confirmed the root cause of today's outage: the Managed NAT Gateways in us-east-1 overflowed and jammed ujp with money.

Yeah, this doesn't work. I assure you, no federal regulation or proposed penalty is going to make @awscloud say "oh, outages are bad, we should be more careful." They already say that! Constantly!

Today's event wasn't from a lack of care or diligence.

To be explicit, I don't think AWS has done anything wrong here. This is the natural end result of their success at massive scale.