Posted
by
Zonk
on Saturday November 17, 2007 @06:39PM
from the only-problem-is-the-view dept.

Kurtz'sKompund writes with word of a Sun project in Japan, one that's taking a somewhat non-standard approach to data center construction. To save on power, heating, and water costs, the consortium is going to be building their center in an abandoned coal mine. The outpost will be created by lowering Blackbox systems into the ground; estimates on savings run to $9 million annually in electricity alone.

SUN is one of the only IT companies that not only "cares" about Eco Responsibility, but actually talks the talk AND walks the walk. This is not just me saying this, this is from the likes of Forbes http://www.forbes.com/home/2007/09/25/sun-green-software-tech-sciences-cx_ag_0925techsun.html [forbes.com] and so on.
This is not ironic that SUN is putting a datacetner underground. This is perfect. SUN *IS* the SUN with their new energy saving servers. And as oil races to and past 100USD$/barrel - this kind of thinking onl

The temperature in a cave means nothing, unless you take into account the cave's ability to dissipate heat somewhere (water or air moving through the cave). If you go inside a cave that's been at constant 55F for a thousand years and you suddenly heat it with 50 kilowatts of power from your data center the temperature will settle at 255F in a hurry. About the only advantage you get from a cave is a constant supply of really cold water (if sufficient rain that year). Ambient air temperature is irrelevant since usually you don't have a strong draft in a deep cave and static air will heat up pretty quickly.

The next big monster from Japan will be the Sunjira: The Sun datacenter goes 'live' after a big earthquake. Its brain consists of 30000 CPUs directly connected to the internet, filling it with hentai tentacle porn. Its target is, strangely enough, not Tokyo, but Seattle (Redmond, to be more specific)...

On the other side of the Pacific ocean, chair production reaches an all-time high.

Its not a cave, though. Its an abandoned coal mine. That means that there's ventilation infrastructure of some sort. And Sun's datacenter will hardly fill up the entire mine. That means that they can use the unused portions of the mine as a heat exchanger: bring in air from the empty portion to cool the datacenter, and dump the hot air back out to that same area to allow for cooling.The issue I see is humidity. Mines, caves and other underground passages are usually more humid than open areas, simply b

Does not the air conditioning cycle in these black boxes also remove humidity ? I worked at Sun and got to play with these containers. They remove the humidity from in coming air and are cooled with water.

It seems like the idea is to use the mines water to cool the containers and dump it back into the mine to be cooled and reused. They also have dehumidifiers built into the Black box to prevent condensating moisture inside.

I worked on wiring one with a couple cohorts and even sweating in these things is a joke , it's pretty much sucked up in about 5 minutes of being sediment in the box.

I'm pretty sure there are some flaws in your thinking. I was in a coal mine recently (Iron Mt.), and the tour center there used air from the cave to keep the place cool in the summer. They said that the temperature is very very stable.

But I guess it does have air moving at a respectable clip from the air hole at one end of the mine.

(It was amazing that men with hand tools dug a hole big enough to put the entire Empire State building in with only the antenna sticking out! And in the dark too.)

You raise an interesting point about heat dissipation in an underground datacenter. I remember seeing something on NORAD years ago about the construction of the command center inside Cheyenne Mountain. One of the things that stuck with me was the fact that there was no dedicated heating system: they merely ducted the waste heat from their 150+ mainframes throughout the entire installation. Kept 'em all nice and toasty warm, even in a Colorado winter.

People have been using geothermal heat pumps [wikipedia.org] for maintaining house temperature as a sustainable alternative to the traditional gas/electric air conditioner/furnace combination for years now (1 million installed base currently). A liquid is moved through pipes placed in the ground - in direct contact with earth or water - to remove the waste heat for cooling, and to absorb heat for warming.

While this approach may save money for the company, and is certainly a 'greener' approach than traditional data centers

2. EMP security. In the event of a nuclear war or similar event that could produce significant EMP, an underground site is your best bet. If wired properly, I am sure Sun's Black Boxes also serve as nice Faraday cages as an added bonus. Aside from a direct hit on the mineshaft, your data will be secure.

Except for the fact that while my data would be safe, I would be toasted by said nuclear explosion, your argument looks like an excellent marketing point, Mr McNeally.

According to TFA, $9M could be saved on electricity when using 30'000 server cores. Also according to TFA, 10'000 cores are planned with a $405M budget. If power demand scales directly with the number of cores, this would equate savings of $3M annually. Based only on these savings (which of course won't be the only factor, but since TFS and TFA single them out so clearly), this project breaks even after a measly 135 years or about five and a half times Sun's current age.

According to TFA, $9M could be saved on electricity when using 30'000 server cores. Also according to TFA, 10'000 cores are planned with a $405M budget. If power demand scales directly with the number of cores, this would equate savings of $3M annually. Based only on these savings (which of course won't be the only factor, but since TFS and TFA single them out so clearly), this project breaks even after a measly 135 years or about five and a half times Sun's current age.

Your savings estimate is only correct if it costs nothing to create a datacenter. Google recently spent $600 mil on their Lenoir datacenter. To calculate value, you'd have to compare the installation and upkeep costs of other facilities.

Yes, that would be true if ground level facilities and the hardware itself was all freely available. Oh, and construction crews, technicians, engineers, supporting staff, *list goes on*, are also entirely willing to volunteer their time and equipment.

What you should be doing is comparing the cost of this project to a comparably equiped ground level datacenter.

The Blackbox containers are robust enough to withstand earthquakes, being capable of withstanding a quake of magnitude 6.7 on the Richter scale.

I don't know, but placing servers 100m underground in a place that routinely is hit by large earthquakes seems a dubious idea. The containers themselves may survive a quake, but what happens when the disused coal mine collapses onto and around them? Even if the containers and servers survive, will the power and data cables? If the tunnels collapse how will you get to and from the servers for maintenance?

And you know what happens when Godzilla comes around - The military fires rockets and microwaves into him, just making him mad, and he wipes out Tokyo - AGAIN. That Godzilla, he's a fickle one - he's only your friend until you make him bleed profusely...

If it happens and you end up with damaged hardware you consider it a loss.However Sun and most heavy iron server companies now Have ILOM enabled systems. Quite easily managed from remote locations. I have loaded firmware onto e10k's in New Jersey from a project in Colombia via remote connections.

*/me shudders with the thought of the new DST and patching all the Sun boxes earlier this year*

Really are great tools Integrated Lights Out Management is , I have to say one of the best inventions that they have mad

So...essentially, they're using the same process as (what Wikipedia refers to as) Geo-exchange, [wikipedia.org] only instead of bringing the constant-temperature (hot or cold, depending on surface temperature) to a building on the surface with heat exchangers, they are bringing the 'building' to be cooled underground.

what about setting up water cooled servers, with water running on a closed circuit, through a stainless steel sink at sea? no corrosion there. The coasts of my country (Argentina) face the antarctic currents of the Atlantic ocean, and keeps it COLD all year long.

Damn flu medicine makes my head feel like its full of glue. I could have sworn the title was "Sun To Create Underground Japanese Detector". I had to go read the article to try and figure out what underground Japanese are, and why you would want to detect them.

I remember the last San Fran Earthquake and we had to get a warm site up and running using all the backup tapes from our offsite storage company. The storage vault was 100% ok, the warm site was 100% ok and I couldn't get anyone to drive the truck through a post apocalyptic thunder dome. I suspect that getting a bunch of nerds to work in an abandoned coal mine will be greeted by dumbstruck looks when you see a giant fire breathing dinosaur.

I agree that it's always cool underground (in the 50s F range) however a problem they might run into is the humidity. Depending on the cave/mine the air can be quite humid and could pose a problem for the machinery. Seems like a tough thing space to "air condition" the water content.

Since the computers produce a lot of heat the humidity wouldn't be much of a problem, try putting a computer in a humid garage, the computer will be just about the only thing dry in there.I do wonder how much this thing will really save, I wouldn't be so surprised if the costs are comparable to the normal installation (remember, the normal installation costs for these things is near 0, they just need a power, network and water plug). If they'd just put the server somewhere with some other cooling source ava

A solar neutrino observatory is installed in an old mine in Sudbury, Ontario, Canads. It has the advantage of being impervious to almost all kinds of radiation, except of course for neutrinos. http://www.sno.phy.queensu.ca/ [queensu.ca]

As I look at the other posts, I see lots of naysayers. Well there are at least a couple of cases where old mines have been used successfully for other things.

Remember seeing a documentary on an ultra-deep salt mine in Utah, now largely played out. Big empty space, no light except for what you bring with you. Lots of worn out machinery that's just abandoned because its value as scrap is less than the cost of bringing it back up. Creepy.

Sounds like the perfect geek plot for a Half-Life style game, only with servers and geeks.

Your name is Theo de Raadt. The year is 2048 and you've been set on a mission to check out what's wrong with that server in Remote Data Mining Facility #2345. Once you're lowered into the bowels of the datacenter, all you have on you is a hazmat suit, a 1U rack server and a crowbar. You notice that the container you're supposed to service is open, and a feint glow is emitting from it...

duh how the hell does this reduce heating problems and shit? it seems to me if you build the darn thing where the sun don't shine as an earlier poster said, then it would get hotter not colder cuz the cottonpickin earth all around would serve as an insulator and cause the datacenter to fry itself and shit.

Putting it in a cave is not a win. The ground water cooling is great, but by putting it in a cave they are severely limiting their access to air-side economization, which is bringing in filtered outside air directly when it is cooler than the rack's exhaust temperature. And if you're doing it right, the rack's exhaust temperature is 95F+ (when we do a hot aisle, we make it a hot aisle dangit). As a professional in the efficient datacenter arena, the mention that they are still using chillers at all when the

As another datacenter efficiency advocate, I've been looking for references of using natural (low temperature) water for free-cooling. Is there a place/consultancy/resource to look up figures like this? -- e.g. 30MW of cooling with 60F water ~flow rate of 13,650gpm -- without chillers?
Thanks in advance!
FF

The coolant will be ground water and the site's temperature is a constant 15 degrees Celsius (59 degrees Fahrenheit) all year, meaning no air-conditioning will be needed outside the containers. This reduces the energy required for the water chillers, used with surface-level Blackbox containers.

The containers will be lowered 100m into the mine and linked to power, water cooling and network lines via external connectors.

They are going to use the mine as a heat-sink, computers are going only 100m under ground.

I'm sure the box will survive an earthquakes, but what about the contents? Most servers don't like to be shaken very hard. You also need to worry about the roof caving in.

When they talk about the blackbox they aren't talking about the container but rather the entire system. The whole floor where the equipment sits is shock isolated and the racks are further cushioned from that. They have to design it that way because these things are expected to be shipped from place to place, there is no packing material