No More Hustleporn: On Fiber Links that Connect Google Datacenters

Tweet by Urs Hölzle

SVP Engineering at Google, working on the infrastructure powering Google Cloud and Google. Opinions are my own and are worth exactly what you pay for them.

Here's another story from the trenches of the internet. Last month, a fire in a rural area of South Carolina damaged one of the fiber links that connect Google datacenters (picture below). 100Tbps of capacity lost and restored in 12 hours!🧵

What you see is a "splice box" where the fiber briefly come to the surface, for example, to connect to a different fiber route. Often these are underground, too, but not here.

In this case, there also was a "slack loop", extra fiber that's coiled underneath to make future repairs easier. Because of the intensity of the brush fire, both the splice box and the fiber were melted.

Before it was destroyed, the link carried 100 Tbps (100 million megabits/sec) of capacity for Google. So we did notice :-) But because we have redundant links there was no customer visible impact.

Our excellent network deployment team, together with a local provider, fully rebuilt everything in less than 12 hours.

So while we're all looking forward to a quiet holiday period, please thank these unsung heroes of internet infrastructure. They never get a quiet time because there are disruptions every day along the millions of miles of fiber they are responsible for. Hats off to them! 👏👏👏