Wired: Where in the World Is Google Building Servers?

July 6th, 2012

By Cade Metz

Though few realize it, Google is now one of the world’s largest hardware makers.

Since 2000, the company has designed the computer servers that underpin its empire of online services, including Google Search, Gmail, and Google Docs, and since 2005, according to one former employee, it has also designed the networking equipment that connects these servers.

For years, the company kept all this very quiet, seeing its hardware design efforts as perhaps its most significant advantage over competitors. But now that the rest of the world has caught on — with many companies trying to duplicate its methods — Google has opened up a bit. It even acknowledges that it’s designing at least some of its own networking gear. But much of its operation remains a mystery, and even the stuff we know can be misleading.

A decade ago, Google turned the hardware world on its head when it started building its own servers in tandem with various manufacturers in Taiwan and China. Rather than buy gear from big-name sellers such as Dell and HP, it streamlined the process, going straight to Asia — where the Dell and HP gear was actually built. Google’s move was so successful, others followed, including Amazon and Facebook. In a way, Google shifted an entire market to Asia.

But it seems the company’s manufacturing operation now extends elsewhere.

Urs Hölzle — the man who oversees Google’s worldwide network of data centers — indicates that at least some of the company’s data center hardware is manufactured outside of Asia. Last week, at Google’s annual developer conference in San Francisco, when we asked about the company “designing its own servers and going to Asia to manufacture them,” Hölzle demurred. “That’s not strictly true. But I can’t tell you why that’s not true,” he said. “Certainly some of them are manufactured in Asia.”

He did confirm that the company is designing its own servers — and that it has done so for the past 12 years — but he clearly indicated that its machines are not manufactured exclusively in Asia. It’s a perplexing admission, but it may provide another window into how the giants of the web operate. Typically, as they revamp the infrastructure that drives their online services, the web’s biggest names will follow Google’s lead. This is true of software, but also of hardware.

Hölzle declined to provide additional information about Google’s manufacturing habits. He says that although the company is willing to share some information about its internal infrastructure — as a way pushing certain technologies into other companies so that it can force prices down or encourage the adoption of industry standards — it still sees its core operation as a competitive advantage. Just as we don’t know exactly where its latest machines are manufactured, we don’t know what these machines look like.

The Asian Advantage

In today’s world, when you’re building enough hardware to feed Google’s worldwide network of roughly 40 data centers, you typically team with manufacturers in Taiwan or China. This is not just how Dell, HP, and Cisco manufacture the beefy data center gear they sell to the world’s businesses. It’s how a company like Apple builds phones and tablets. The cost of labor is cheaper in Asia, and Asian manufacturers have the infrastructure required to produce this hardware in vast quantities.

Yes, some companies have their gear manufactured in the United States and other parts of North America. SeaMicro, a server maker recently purchased by AMD, for instance, works with a manufacturer right down the road from its Northern California offices. But typically, such companies are smaller operations. When you get big, the economics change.

Larger operations may handle part of the process in the United States. Facebook, for instance, has its servers built in Asia, but then it uses a company in Silicon Valley to “integrate” the machines, loading them into server racks and plugging them into network equipment. But Hölzle’s comments seemed to refer to more than just integration and assembly.

Hölzle may be saying that Google now operates its own manufacturing plants. If you get large enough, the economics change again. It may make more sense to do it all yourself. Google has gotten to the point where it’s designing gear entirely with in-house engineers, according to one former employee, and you wouldn’t be surprised if it starting building the hardware on its own as well.

“Google does seem to have the chutzpah to think it could pull that off,” says Jennifer Reed, an analyst with Charlie Barnhart & Associates, an outfit that closely tracks manufacturing trends across the globe. “Whether they could actually pull that off, I don’t know.”

At its latest shareholders meeting, Google chief financial officer Patrick actually called the company “probably … one of the largest hardware manufacturers in the world,” and he spoke as if Google does its own manufacturing, rather than farming things out to someone else.

“Google actually builds servers in a factory,” he said. “So we know hardware. We know about flash. We know about equipment. We know about supply chain. So we were very well-equipped from the hardware side, to be very competitive in that space.” At the time, most assumed that he was referring to a setup that involved third-party manufacturers, but perhaps his words should be taken at face value.

That said, Hölzle did add that Google uses multiple “contract manufacturers” outside the company. So if it does run its own manufacturing operation, that’s only part of the pie.

In calling them “contract manufacturers,” Hölzle seems to confirm that Google is designing the machines entirely on its own. In this respect, it seems to operate a little differently from Facebook, which works hand in hand with “original design manufacturers,” or ODMs, on the design of its machines. Typically, an ODM helps with the design while a contract manufacturer merely builds — though the lines between the two aren’t always so clear.

‘Silly Rabbit, No One Builds that Kind of Stuff in the U.S.’

Google already manufactures some hardware here in the United States. Last week, when the company unveiled its new Nexus Q music player, it also revealed that the device is built at a plant that’s no more than a short car ride or plane flight from Google’s headquarters. Much like SeaMicro, the company says it likes having the plant close enough that it can easily monitor the manufacturing process. But again, like SeaMicro, this is a relatively small operation. Building the first run of a music player few are likely buy is very different from building thousands of servers for one of the world’s largest data centers networks.

Jason Hoffman — the chief technology officer of Joyent, a cloud computing company that has explored the possibility of going straight to Asia for servers and other gear — believes it would be impossible for Google to build its servers in the United States. “Silly rabbit,” he says, “no one builds that kind stuff in the U.S.”

But Google could be using manufacturers in Mexico — or perhaps even Eastern Europe, according to Eric Miscoll, another analyst with Charlie Barnhart & Associates. He points out that some Asian ODMs operate plants in Mexico — including Wistron, one of the ODMs that works with Facebook — and he says that most all of the big Asian contract manufacturers are there too, including Foxconn and Flextronics.

Some operations will avoid manufacturing plants outside of Asia because so many component parts are already built in Taiwan and China, but according to Miscoll, as of a year ago, contract manufacturers in Guadalajara were flying electronics parts from China at least once a week. If Google does manufacture some of its servers in Mexico, Miscoll says, this could make it easier to ship machines to data centers in North America.

Urs Hölzle may simply be saying that Google builds a small number of prototypes in the States, where it can keep that close eye on production. Companies will often build prototypes at nearby plants before shifting the larger operation to Asia or some other place where the costs are significantly lower. But Hölzle was apparently discussing machines that are actually put to use in Google’s data centers.

He said that all the servers that underpin Google public services are designed by the company and manufactured in Asia — or somewhere else.

Wherever Google is manufacturing its gear, you can bet the web’s other big players will keep a close eye on its progress. The rumor is that Amazon runs an internal software application that seeks to determine when it will make economic sense to move server manufacturing inside the company. And with its Open Compute Project, Facebook is working to move so many other outfits to the streamlined servers designed for its data centers. It doesn’t make sense for everyone to build their own servers, but it certainly makes sense for the web’s biggest players — and perhaps others.