When less is more: the basics of physicalization

Momentum behind the ten-month-old trend called physicalization is accelerating …

The word "physicalization" is ten months old. This January, Rackable Systems launched a strange line of servers which defied all the conventional wisdom of server design by disaggregating larger servers into many smaller ones based on consumer parts, and in the process lowering power and performance density. Ars, among others, expressed skepticism over the strange design decisions, but launches from other major vendors suggest that, for some market segments, the server space is taking a turn in a novel new direction. Ars has covered this trend before, but we now take a more detailed look at it, with a survey of available physicalization offerings, analysis of the reasons for their adoption, and some predictions about the future.

The standard server: a bit unbalanced

Let's start by detailing the status quo a little. Conventional server products for datacenter use on the current market from major vendors are the same: a pair of x86 server-class processors from either Intel or AMD, and an appropriate complement of memory, with hard disks. This same product can be bought from almost every vendor, typically in 1U but also in 2U and various blade configurations. Double-density products fitting two such servers per rack unit are available from HP and SuperMicro, among others, pushing the density threshold to 16 cores per rack unit and power density above 25KW per cabinet under full load. We'll call this product, and its forthcoming descendants, the standard server.

The standard server is the product of a number of longstanding trends: increasing density, increasing core count, increasing power density, and increasing performance per watt. Some of these trends may have been taken too far; the power density of current Nehalem solutions is high enough to consume the power budgets of many datacenters in a fraction of their space, unnecessarily concentrating their cooling needs in a small area and leaving the rest of the center empty. These days, silicon and power are more expensive than concrete and steel, and some offerings don't adequately reflect this.

Meanwhile, the standard server is a comparatively unbalanced machine; lots of computing power, memory, and network bandwidth concentrated into a few nodes with relatively little storage bandwidth or IOPS capacity. Web serving and other database applications need little processor power to provide these functions, which can leave expensive Nehalem Xeon processors idling en masse. Even for compute-intensive applications, a single OS instance is no longer enough to effectively use all the resources of a modern Nehalem server.

For these reasons among others, virtualization has rocked the server space over the last few years, with end-users adopting virtualization techniques en masse to meet their various needs. In the Ars Server Room forum, virtualization skeptics are treated like creationist lunatics, so thoroughly has the relatively new technology proven adept, when properly used, of helping to resolve the resource problem above.

Our hypothetical database company, stricken with storage servers that have underutilized processing power, might virtualize instances of compute-intensive tasks onto these servers, allowing them to simultaneously fill a number of different roles, fluidly. In this way, virtualization has made itself essential to the efficient use of the standard server, and this trend will only continue over time.

At the same time as it introduces a cost in the form of relatively unbalanced performance for many workloads, and the additional abstraction and load-balancing work needed to rectify this, the standard server also traps the buyer in a sandboxed marketplace of very high margins. Consider the fact that the Xeon X5570 currently sells for almost three times the cost of a practically identical desktop processor, the Bloomfield Core i7 940. This difference can contribute upwards of one third the total cost of blade servers from large vendors, so anyone who can find a good way of using desktop and laptop components in the datacenter can make a great deal of money mining the margins.

Beyond processors, consumer hardware can offer other cost advantages, alleviating the need for server-class networking like Fibre, 10GigE, and Infiniband by harnessing the "free first gigabit" offered by GigE. SATA instead of SAS disks, and other such changes, can further lower costs.

At times in the past, it would have been lunacy to try to use consumer hardware for heavy datacenter workloads, because every available FLOP of per-node performance was dearly valuable. The market has changed, though. More and more applications are adopting a clustering, "scale-out" model, as software and network improvements better allow smaller nodes to cooperate at larger tasks. The same forces that drove high-end x86 processors like Xeon to compete with other microarchitectures are now allowing other x86 processors to compete with Xeon.

It's a good example of what has been called "IT consumerization." The server market is now wrapping up a transition to x86 from other architectures begun in the 1990s, primarily driven by the economies offered by x86's prevalence on the wider market. In an extension of this process, the market may soon make another transition, this time to a closer-to-consumer segment in the x86 space, or at the least using the threat of this transition to force Intel and AMD to take lower margins in the server space.

Physicalization: the theory

The transition to smaller, closer-to-consumer-components servers with lower-power processors and single-socket designs, then, has several advantages, in theory. It can:

Lower costs by using consumer components.

Rebalance the available performance in an I/O direction to be more suitable for loads like hosting and database apps

Reduce (but not eliminate) the need for virtualization, especially in the near term

Reverse, or at least halt, the trend of unnecessarily high-density servers exhausting power and cooling budgets in a small fraction of available rack space

Allow the datacenter to pack more independent servers into every rack unit, for easier marketing of "dedicated hardware" to Web hosting and other customers

The disadvantage is this: disaggregation is, if not a violation, an unorthodox use of Moore's law. As we've covered before, there are good, grounded-in-physics-and-economics reasons why the die-level integration of a standard server, and its multicore design, will be cheaper to produce, and more power-efficient, than a physicalized alternative. The standard server is standard for a reason: it's got roughly the highest available performance per watt and manufacturing economy metrics.

Adventures in physicalization, then, will rest on attempts to efficiently serve those workloads which, like Web hosting, are not suited for the standard server, while emphasizing the cost advantages and finding ways to reduce the hit in performance per watt imposed by the lower level of integration. Let's take a look at the entrants so far, and at one novel entrant which is, thus far, hypothetical.

It still sounds like virtualization is a better option for most circumstances. If you have 10 virualized servers on very powerful hardware, if one of those servers suddenly needs to do some number crunching, there is going to be the available capacity. With physicalization on an Atom processor, your maximum processing speed is going to be extremely limited.

I can only see it being useful in webhosting for providing fully dedicated servers for cheap(tm). Other than that I can only see it in a fringe niche. though I know I wouldn't turn now a nice 32core Cortex-A9 box if someone gave me one

An interesting concept for sure, but it seems to me like it will be limited to some very specific types of workloads; though the people with those kinds of workloads have a lot of market pressure due to their very substantial budgets.

It still sounds like virtualization is a better option for most circumstances.

Depends on the software. In the end virtualization is a kludge as well trying to solve a software problem. If you run software that already takes care of nodes that might break and scales out well it will always be better than virtualization.

Of course 99.9% of all software does not work in that way so virtualization is still a great thing.

Physicalization is a cool idea, if your web app can even do sharding/LB. Seeing Rackspace doing this I understand. They have a lot of really small websites and people want their own "dedicated" server.

I would love to see some real world stats. 2U Nehalem(virtualization) vs 2U "Microserver".

And we all know these new Microservers cost a lot. Not even sure there is really cost savings at all. Think about the whole rack setup and admin side.

My work site can and will peg a 3ghz Nehalem CPU for 10-30sec because of the amount of data it has to deal with, and its not bad code, its just a lot of data. Thank you PHP for not being multithreaded.

I'm amused to see vmware supported on the CloudRack platform. Based on the cost per node of vmware licensing you'll pay 2-8 times the cost of the server blade for your vmware license. Free Hypervisors may make a touch more sense, but these physicalization platforms seem quite unsuited to virtualisation.

I also hope the come with internal switches like bladecenters, as a cabinet with 264 network ports (and that's without redundancy, some people would want to multihome them) will require a _lot_ of network cables. Not to mention consuming the better part of a 6509 (5.5 48 port blades) or 6Us of "high density" 48 port switches.

An interesting product, but it seems intended for very specific niche markets.

if they added PCI slots, I would be all over this. I frequently deploy far more CPU power then required to essentially get video capture cards connected to the SAN. While some customers prefer cpu intensive codecs (XDCAM35 being the most popular of those now) most do not (DVCProHD) and we end up racking a lot of dual xeon systems just for their dual power supplies and ability to host a PCI card.

For sure I'm in a pretty niche market (Broadcast) but across the board I am far more bandwidth constrained than CPU constrained - most of my servers never see 50% CPU utilization.

If they could give me enough CPU to encode raw RGB into DVCProHD, a 10GigE port, and a PCI slot or even a firewire port I could roll these out pretty massively. An Express Slot would do nicely as well.

While the web/DB server node is an interesting case, I think these guys would do well to offer the flexibility of some kind of external port, as it would allow the products to replace many 'standard servers' that are deployed because of their combination of both reliability and flexibility...

For an organization as large and well funded as Condté Nast, I find it very disturbing that your editors do not seem to know the difference between "then" and "than"!

And lately, your writers have even been mixing up the proper use of the words "amount" and "number of". Never, should you refer to an "amount of people"! Look it up in the dictionary! Hire a few (a number of) educated proof readers!

Maybe it's just my experience, but in 12 years of database administration I've never had a box with "wasted" resources. It's far more typical to get pages alerting me that server X has been at 90% CPU for X hours.

Of course, I don't admin Windows boxes or all that many web servers, so my experience is skewed towards big DB installations.

Disk bandwidth is definitely huge, but when I'm looking at specs right now to budget adding an additional set of processor cards, "I have too much processor going to waste" isn't really my problem.

I was pretty interested in this as a cheaper-than-VMware way to split up a lot of small workloads until I took a harder look at the hardware. The Atom solution is the only viable one available in the "six per slot" Atom configuration of the SGI (Rackable) product; the AMD one is based around the X2. Unlike what this article claims, their web site shows support only for single-core Atoms, not the 330. Which is a shame.

That wasn't what killed me on the idea though. I get the idea of using lower-end parts, but they can't seriously expect me to deploy a single-LAN solution and then run that single LAN on a Realtek chip? If the stated goal of these things is to maximize I/O per processor, how exactly is this fabled I/O supposed to get in and out of the machine?

Oh yeah, and managing a 14U chassis with 54 nodes without IPMI? (shudder) No thanks.

I can only assume that saying that these are optimized for the dedicated server market means "cutting the corners you can only cut when the workload you're running isn't your own."

the power density of current Nehalem solutions is high enough to consume the power budgets of many datacenters in a fraction of their space, unnecessarily concentrating their cooling needs in a small area and leaving the rest of the center empty

This still seems odd to me. So...you've got a lot of empty space, and your solution is to use less space-efficient computers? Why not use that space for something else? For example, you could partition off the servers, and lease out the rest of the room as office space. As office space, it'd be slightly awkward (not much room to grow), but it'd be great for a small startup that also needs to rent rack space.

Originally posted by metageek:lease out the rest of the room as office space. As office space, it'd be slightly awkward (not much room to grow), but it'd be great for a small startup that also needs to rent rack space.

Keep in mind that "the rest of the room" has no power and no cooling. Assuming you partition it off, you can pretty much only use it yourself, not rent it out, because nobody else is going to put up with all your datacenter security protocols every time anyone goes in or out. And it sounds like they do 24x7 jet engine testing in the next room.

You can use the leftover space for storage, and that's about it.

The merit to using less space-efficient gear is that it simplifies the task of distributing power and cooling. Think of how much the fab cost increases when Intel does a process shrink. In a very inaccurate and poorly-considered way, power and cooling have the same issues.

It may only be a market niche, but the fact that people are buying these things and the sellers are noticing implies that there may be plenty of centers with too much CPU power as compared to I/O bandwidth.

If the general pattern holds, I'd expect to see high end "server" processors get cheaper, and that means a whole new set of markets for them. Maybe there's a market for more processing power in the home, at least for media center and hobbyist types?

Originally posted by Article:In the Ars Server Room forum, virtualization skeptics are treated like creationist lunatics, so thoroughly has the relatively new technology proven adept, when properly used, of helping to resolve the resource problem above.

Thanks, Ars, for that snide, unwarranted jab at other people's beliefs. Nothing like religious tolerance, huh? Can't have opinions other than the almighty Ars or you're a lunatic.Again, thanks for that.

Originally posted by Article:In the Ars Server Room forum, virtualization skeptics are treated like creationist lunatics, so thoroughly has the relatively new technology proven adept, when properly used, of helping to resolve the resource problem above.

Thanks, Ars, for that snide, unwarranted jab at other people's beliefs. Nothing like religious tolerance, huh? Can't have opinions other than the almighty Ars or you're a lunatic.Again, thanks for that.

You may want to consider re-reading the text you quoted. The article specified "creationist LUNATICS," not creationists in general. Lay off the victim act, it's counter-productive to learning and comprehension. Evolution is the widely accepted theory. Lunatics of any belief tend to have such tunnel vision that their ability to comprehend anything outside of their narrow scope is severely impaired. I am sure evolutionist lunatics exist as well, they are probably just quieter and are content with things.

Now, if you are a lunatic and you are taking issue with that portion of the quote, I can't help you.

Originally posted by Article:In the Ars Server Room forum, virtualization skeptics are treated like creationist lunatics, so thoroughly has the relatively new technology proven adept, when properly used, of helping to resolve the resource problem above.

Thanks, Ars, for that snide, unwarranted jab at other people's beliefs. Nothing like religious tolerance, huh? Can't have opinions other than the almighty Ars or you're a lunatic.Again, thanks for that.

You may want to consider re-reading the text you quoted. The article specified "creationist LUNATICS," not creationists in general. Lay off the victim act, it's counter-productive to learning and comprehension. Evolution is the widely accepted theory. Lunatics of any belief tend to have such tunnel vision that their ability to comprehend anything outside of their narrow scope is severely impaired. I am sure evolutionist lunatics exist as well, they are probably just quieter and are content with things.

Now, if you are a lunatic and you are taking issue with that portion of the quote, I can't help you.

the way i continue to read it is that the author meant all creationists == lunatics. We aren't(at least not all). I see evolution has it's place in the world, that doesn't mean I believe everything. But because I am technically a "creationist," I am labeled a lunatic.It may not be how they meant it, but it is how it comes across. I am not playing victim, I am trying to understand why a flippant remark about creationists is put into an article on server "physicalization".

I am surprised no one mentioned the history behind it. The Pentium Pro could support SMP with 4 processors. The Pentium II crippled it to only two processors. For 4 processor SMP systems, Intel later introduced the Xeon line with the Pentium II Xeon. Later the Pentium 4 was crippled to only one CPU, and later a separate P4-based Xeon DP line was introduced for dual processors. Later the DP Xeon line got renamed to Xeon 5000 series, and the MP Xeon line got renamed to Xeon 7000 series, with a new single processor only Xeon 3000 series being introduced later that is mostly versions of desktop CPUs with enhanced features like ECC in the most recent Xeon 3400/3500 series (they even use the same socket).

the way i continue to read it is that the author meant all creationists == lunatics.

That would be the case if he said, "In the Ars Server Room forum, virtualization skeptics are treated like creationists..." Which he did not. There are lunatics everywhere. For your personal health, never come into GESC and say that Planescape: Torment was a horrible game. Lepers are treated better

Your concern about why it is used as an analogy at all? Well placed, IMO.

The way this article characterizes database applications is rather off. Some database workloads are sensitive to random I/O and true database commit rate; moving toward consumer targeted disks and their controllers is a useless step for them. Other workloads have occasional large queries that can't be spread across more than one CPU; while CPUs may sit idle most of the time, the slowdown when one of these shows up would be intolerable.

The only types of database apps this "physicalization" trend targets well are databases for web applications, where commit guarantees tend to be loose and the data easier to shard across multiple nodes. For all other types of databases, this really isn't an improvement over the "standard server" that has a powerful CPU available and a battery-backed RAID controller as an option.

Virtualization is great. Where I think Physicalization shines the most is in small server setups. In large clusters with virtualization it's easier to tolerate hardware failures. When you get down to smaller setups though, it's not nearly as cut and dry. Yes, with virtualization I can setup a few really high power processors in a much smaller area. I can then utilize the power for various tasks. What happens when a server goes down though? In a smaller setup there just isn't a lot of tolerance when a virtualized setup goes down, even if the other servers aren't operating at peak.

Enter physicalization. If one small section goes down, it's cheap to replace for the client, and they might only have one or two services down, instead of the whole schpiel going to hell quickly, and oh by the way, please lay out $xxk to replace this thing. Add virtualization on top of that, and you have yourself a real winner. Suddenly, the client can more easily afford to keep one or two spares around.

The real sweet spot for this technology seems to be in its compliment to specifically "cloud-based" technologies such as the Azure platform. If each server node is one full machine (or a collection of VM's), it might not make a lot of sense to use cheaper, less capable servers.

I think the story changes, however, with something like the Azure platform. In the Azure case, each server core is one of perhaps thousands of generic "compute nodes". In the generic "compute node" paradigm, scalability and lower per-unit cost and is the ultimate goal, and the lower cost of commodity hardware starts to make a lot more sense.

Of course, many server applications are fundamentally dependant upon a "scale up" approach; these applications will not be good applications for the "scale out" that commoditization/physicalization can provide. I would be very surprised, however, if commoditization was not the long-term winner any efficiency contest, server physicalization notwithstanding.