N.Y. Times Data Center Indictment Misses Big Picture

A New York Times examination of increasing data center use and its environmental impact focuses on aging enterprise data centers. A more important issue: How much environmental benefit can we reap from today's modern cloud data centers?

A New York Times Sept. 22 article now causing heated conversation in the cloud computing community paints a gloomy picture of the average data center as a wastrel and energy glutton. In the eyes of the writer of the initial piece "The Cloud Factories: Power, Pollution and the Internet," data centers are prime suspects in the ongoing process of environmental degradation.

I am in total accord with the intent of the Times piece, seeking to make data centers more efficient. But I am dismayed at the casualties that the writer, James Glanz, inflicts as he goes about making his point. Surely, he realizes that the first part of the story describes an aging, mixed-system data center straight out of the 1980s or 1990s. Then he goes on to say that both the enterprise IT manager and cloud data center creator are culprits (two very different things lumped together). And, he concludes that the Internet user is an eager, witless accomplice to them both.

Either the Times underestimates the intelligence of its readers or it has lost touch with them. After all, it delights in publishing stories about the iPhone, and there are few drivers of data center construction that are equal to the use of smartphone apps. I don't remember reading about the dark side of iPhone 5 in all those iPhone 5 advance articles.

It's difficult, but what we should be trying to calculate is how to keep the economy growing with digital services, while protecting the environment.

Everyone is doing a lot more computing, as the story notes. But as we do so, the amount of electricity consumed per unit of computing is going down, which the story somehow misses. Nowhere does the Times address this salient point. Instead, it concludes we are doing a lot more computing and, therefore, we are all guilty of driving environmental degradation. If you're going to reform the world, you need to build a better soapbox than this.

There's a lively discussion going on regarding these points at the New York Times' collection of related opinion columns, Big Data's Big Costs. "Is the convenience of cloud computing worth the energy and pollution it requires? Or should we wean off of 24/7 access?" are some of the questions asked.

Let me start with some contrary evidence. InformationWeekreported a year ago that the Environmental Protection Agency had predicted that between 2005 and 2010, the electricity consumed by data centers around the world would double, based on the rate of new construction. In 2011, the EPA had to back off that prediction because during that period, data center electricity consumption had increased by 36%, not 100%. The author of the study for the EPA speculated that the pace of data center construction had slowed, due to economic recession. I for one have driven around Silicon Valley looking for staked-out, but not-yet-built data centers; they're more scarce than Samsung handsets on Apple's Infinite Loop.

While some data centers were mothballed during the recession, most were built, and were much more efficient in using electricity than their predecessors. For example, PUE is the ratio of energy imported by a facility to the amount of power consumed by IT computing devices; it stands for Power Usage Effectiveness. The 1990s data center, with air conditioning pouring through a raised floor, and featuring a water-cooled mainframe at the center, had a PUE of 1.92 to 2. That means it consumed about twice as much power as the amount used in actual computing; cooling was the largest power consumer after computing itself.

Today, modern data centers built by Microsoft, Google, Facebook, or Yahoo have reduced that ratio to somewhere between 1.22 and 1.07--the latter mark set by Facebook's new Prineville, Ore., facility. This type of modern data center does so by delaying the stepping down of power line voltage until it's close to where servers and switches are running; that reduces the loss to resistance in the line. Instead of air conditioning, the modern data center uses ambient, outside air cooled by evaporation; the air is piped to the cool side of servers, blown into channels between baffles that steer it to the hottest components. Then it's collected in a hot aisle and flushed from the building. The process uses a fraction of the electricity of air conditioning.

The Times article doesn't mention PUE. It does cite a McKinsey study, commissioned by the newspaper, that found across several industries, the average server used only 6% to 12% of its available CPU cycles. To me, this describes the data center running one application per server, lest one application's activity overwrite the data of another (which causes a crash). In the past, InformationWeek has cited such a practice as utilizing 7% to 15% of the server, with the rest of the CPU cycles idled away. The practice reflects an IT manager's fear of downtime. At one time it was a realistic calculation that the business could afford the extra electricity, but it couldn't afford to go off the air for a few minutes or few hours.

But to some extent, the Times is busy pillorying a straw man. I don't know how to explain the McKinsey figures at this late date, but it's now 2012 and the times are a changin'. Maybe with his focus on the big picture, the writer missed the trend toward virtualization within the enterprise data center, where each server is loaded with 15 to 20 applications, or more. They run under lean hypervisor software, which manages CPU, memory, and I/O limits to avoid application conflicts. At sites where virtualization is used effectively, half or two-thirds of the number of physical servers formerly in use are now gone.

Well, I have seen multiple instances of data centers that are really inefficient with huge white space across 1000's of servers. I mean on average as a group of servers doing 10% utilization. Really bad I must say. The good news is these Data "Pig" Centers, really energy pigs, have their respective companies investigating this, doing trade off analysis to determine cost benefits of optimizing their IT landscape. While it will happen it will be slow extensively because of cultural and operational inertia that roadblocks progress in those companies. The top companies (speculation based on what I read) such as Google and Facebook are much better citizens and have shown social responsibility in releasing their designs and strategies that help laggards improve. So while some of this post resonants with me I felt when I read the original NY Times post, I am a subscriber, it was reasonably accurate in depicting the state of the Data Center.

It is obvious that the New York TImes is once again out to create an incendiary controversy about its competition. Just as the economics cretin Paul Krugman keeps getting his soapbox, this writer is out to pillory the competition that is eating The Times' breakfast, lunch and dinner.

What the NY Times article accomplished was to get views, reads, hits, follow-up posts, stories referencing it and so forth all of which are moving bits and bytes of data through various data centers/cloud facilities and actually using those servers, storage, networking and other technologies that as they claim, may have been underutilized.

What the NY Times article also accomplished was to bring back to life some of the themes, claims, challenges and discussions of the 2008 era that while not all have been addressed, many are being implemented. These range from data footprint reduction (DFR) such as archiving, backup modernization, compression, dedupe, storage tiering, data management along with intelligent or smart cooling and power management, more effective power supplies and distribution, faster servers that can do more work within the same physical and energy density supporting more memory and IO. There is also virtualization being used for consolidation as well as agility, along with clouds, IO performance optimization with SSD and many many more.

One of the challenges with the NY Times article, however IMHO it might also be a good example of the data center facilities focused centric view around the Greengrid PUE metric as a means of gauging efficiency. The challenge around PUE as by itself, it only measures the efficiency of the facility as opposed to the effectiveness of the work being done by the servers, storage, networking, hardware, software and other occupants of those habitats of technology. Thus you can have a data center that looks to be efficient based on PUE, however if the hardware and software that support business functions is not effective, than you actually have an inefficient information factory.

On the other hand, you could use the NY Times point of view and cite a data center or cloud site as being inefficient, yet, if that site is productive in terms of the level of services being delivered (performance, availability/reliability, cost, work done per watt of energy, etc), you in fact would have an effective and productive information factory.

If it took as some have pointed out a year of research for the NY Times piece, IG«÷m not sure what was done during that year because in 2008 while doing my normal day job as an independent IT advisory analyst and consultant, I also managed to research and write the book "The Green and Virtual Data Center" (CRC/Taylor and Francis) which is on the Intel Recommended Reading List for developers (shameless plug ;) )...

If you are interested in learning more about the above and related themes, here are some links that Im assuming the NY Times wont be interested in...

The article misses that the cloud allows upgrades to equipment without taking customersG«÷ applications down, allowing further improvements. Virtualization leader VMWare (of which AiNET is a partner) shows significant savings and uptake from progressive well run businesses looking to their data center operations as critical to their operation.

With AiNET technology, we can do what is call High Availability and Fault Tolerance. But even without super-advanced technology, using the basic technology allows you to stay up longer by not having to go down for routine thingsG«™ like the hard drive failed in your machine or the machine just got old. You can move your applications around without worrying about the actual hardware (the server) anymore. ThatG«÷s a game changerG«™.

The new york times is a newspaper that is living off it's reputation, and puts pushing it's political view before reporting accurate news. Just because it's published in New York doesn't mean it is accurate, or worth reading, despite it's arrogant view of itselft.

The problem with any data center is that it concentrates the energy usage into one spot and with that the environmental impact. For centuries we found that distributed organizations and systems are more sustainable and in the end work better than putting everything into one spot. Data centers are like the big screw that holds everything together. One big flood, earthquake, labor strike, hurricane, etc and the center is off the grid. And with that the hundreds of organizations that blindly trust that the cloud will always be there.

I read with interest the first Cloud Factory article, "Power, Pollution and the Internet," in the New York Times and hope that future entries in their series will delve deeper into the "many ... solutions [that] are readily available" to decrease wasted power, including virtualization, which the author briefly referenced at the end of the story. This disruptive technology is being provided today by several companies to provide "virtual machines" that can do the exact same jobs as their physical counterparts while minimizing the need for additional hardware to run business-critical applications. Implementation of data storage optimization technologies, like deduplication, which removes the need to physically store redundant (or duplicate) data, helps reduce the need for additional storage hardware (i.e. disk drives) and thereby the floor space, power and cooling required for it. The adoption rate of these technologies is soaring because of their positive impact on TCO.Tom Cook, CEO Permabit Technology Corporation

The larger question here is why would anyone take the NYT seriously? This is an organization as biased and untrustworthy as there is in this country. The writer - as the majority of their journalists - already had a preconceived idea about computing that he wanted to disseminate. He certainly wasn't going to let a few facts and some basic research get in his way. There are a couple of million ninnies in Manhattan that eat that type of garbage up. Your readers are a lot smarter than that.

Enterprise cloud adoption has evolved to the point where hybrid public/private cloud designs and use of multiple providers is common. Who among us has mastered provisioning resources in different clouds; allocating the right resources to each application; assigning applications to the "best" cloud provider based on performance or reliability requirements.