AMD plunks custom Opteron into SM15000 microservers

Looked at Centerton Atom, wasn't impressed

AMD's SeaMicro microserver and fabric unit has started shipping the "Piledriver" Opteron server nodes for its new SM15000 machines, announced in September sporting quad-core "Ivy Bridge" Xeon E3 processors as well as promising the then-impending Piledrivers.

At the time, AMD did not divulge what Piledriver part it would use, and considering that this is a single-socket microserver, you might be thinking that AMD would have gone with a quad-core Opteron 3350 HE part at 45 watts or an Opteron 3320 EE at 25 watts, both of which the company announced in early December aimed specifically at microservers.

There is also an eight-core Opteron 3380 with a normal 65 watt thermal envelope, which could be used to provide brawnier cores and up to four memory slots. (In the Opteron world, HE is Highly Efficient and designates a low voltage part, and EE means Extremely Efficient and designates a low-volt part that also has its clock speed scaled back to lower power consumption even further.)

As it turns out, AMD is not using a stock "Delhi" Opteron 3300 part in the SM15000 systems, and AMD is not even using one of the "Seoul" eight-core Opteron 4300 chips, which are also available at 35 watts for an EE part with four cores and at 65 watts for an HE part with six or eight cores. Rather, for the SM15000 system AMD is doing a deep bin sort and using a custom Opteron 4300 chip, which it has not named and which it is not selling to other customers.

This custom processor has eight cores and a base frequency of 2GHz, AMD tells El Reg, and the cores can turbo boost to 2.3GHz. With a single core activated, that chip can run at a top speed of 2.8GHz. The Opteron server card has one socket and two SODIMM memory slots that can support from 8GB to 64GB of main memory for that single processor.

The Opteron 4300 server node for the SM15000

AMD did not divulge the wattage of this part – it's a secret – but given its description, we would venture a guess that it is a clocked-down version of the Opteron 4376 HE part, which is rated at 65 watts and which runs at 2.6GHz with a turbo core speed of 3.6GHz.

It is hard to say how much lower the wattage can go. A reduction of 500MHz and a drop in voltage from the standard Opteron 4386 to the 4376 HE part cuts out 30 watts. If you drop the voltage a bit more to make a quasi-EE part and then drop the clock speed another 600MHz, you might buy another 20 watts or so, which would put the custom part at around 45 watts. That's the same as the four-core Opteron 3300 HE part, which explains why AMD didn't use that part in the SM15000 system. If we're right, it can have the same thermals and have twice as many cores as the Intel alternative.

"We are knocking it out of the park with that part – it's a first-rate part," says Andrew Feldman, cofounder of SeaMicro and now general manager of AMD's Data Center Solutions Group. More than that, he wouldn't say.

Feldman was happy to chat about the Atom alternatives and the impending ARM-based alternatives that AMD is cooking up that will make their way into SeaMicro boxes. He is not at all impressed by Intel's dual-core "Centerton" Atom Series processors announced last week, after being initially very excited about Atoms when SeaMicro debuted.

But don't get the wrong idea. Feldman is keeping an open mind about Intel Xeon and Atom processors, including next year's "Avoton" kicker to Centerton. And Feldman needs to do that because that is good business.

"We looked at Centerton and we found that it did not have the right mix for us," Feldman explained. "We will look at Avoton, and if it does have the right mix for us, that would be great. Centerton is weak in memory, with only 8GB of DRAM. That's not quite enough memory, and with our experience customers are looking for a minimum of 16GB. With think Avoton and the direction Intel is going is exciting and interesting, and we think we will have very compelling parts at AMD in the Avoton timeframe with ARM technology. And I think the world is unfolding in exactly the same way SeaMicro said it would unfold three years ago."

While there are some applications where 4GB (the upper limit on the Atom N570 chips that were in the second-generation SM10000 machines from July 2011) or 8GB is sufficient, says Feldman, there are others where 16GB is the baseline. "The issue is when you want to load up memory with a database or a cache and have all of the cores in the system work on it. At that point, 4GB or 8GB is not enough."

The original single-core Atom with a 2GB of memory limit was "not quite the sweet spot that we had hoped," said Feldman, and it was followed up by that dual-core Atom with 4GB of memory, which "was real progress and we did really well."

But when SeaMicro added the Xeon to its system in January of this year, "the business really took off." And that is no accident, says Feldman. It wasn't about it being a Xeon, but being a slightly beefier core and having more main memory per core.

"If you look at where Avoton is aimed, it is roughly where Sandy Bridge was in terms of core count and performance," he says, referring to the Xeon E3 versions of Intel's "Sandy Bridge" family of chips. Intel has not confirmed that Avoton is a four-core part, but everyone expects it to be with the process shrink from 32 nanometers with the two-core Centerton to the 22 nanometers to be used with Avoton.

"There is a sweet spot in microservers, and it is somewhere between 2GHz and 2.4GHz with four cores and 32GB of main memory ideally with maybe 16GB at a minimum," Feldman said. "And that is exactly where – lo and behold – where everybody's ARM part will be in 2014."

Still, on jobs where the compute is modest and the memory requirements are also modest – something like Hadoop, where you basically tie one core to one disk drive, or at least try to in a balanced configuration – then a more modest memory footprint can work. At least with the way Hadoop is architected today. And that may change as microservers and flash and other technologies are brought to bear on big-data munching jobs.

"What the guys who write software and don't pay for disk would like you to do is this: buy a ton of disks to overcome some challenges in their software," says Feldman. "That is an answer, and that is one of the reasons why we now support over 1,400 disks with 64 sockets in the SM15000s.

"But I think what will happen as you see the maturation of Hadoop and the rise of Cassandra and other software is that the ideal will no longer be one disk per core. Right now, it doesn't matter what you pay for Hadoop clusters because they are cheaper than buying a traditional data warehouse or data analytics appliances. The guys who buy Hadoop machines are so innovative, and the Apache community doesn't like waste, and they will continue to improve the software. You can't just throw hardware at the problem."

That's an odd thing for the guy in charge of data-center hardware to say, but there you have it. What has Feldman fired up is that Intel is on board, validating the microserver space that SeaMicro was the first to commercialize in a big way and that AMD hopes to capitalize.

"The cool thing is that Intel came out and said how important this market is going to be and that it is not a segment only for Xeon. There will be many different chip flavors and many different workloads, and Intel is going to have to compete against ARM and compete with parts that two years ago they said they were not interested in putting into servers."

As AMD knows it must compete with ARM chip makers like Calxeda and Applied Micro, who have embedded fabrics on their processors, and so they're working on expanding the "Freedom" fabric that lashes all of the server nodes in a SeaMicro box to each other and to the virtual I/O for storage and external networking. That Freedom ASIC was limited to a single chassis, but with the SM15000 machines it was stretched to include the external storage arrays that are necessary for running big-data munchers such as Hadoop.

"We got it part way there this year by bringing the Freedom fabric out of the system to get to the storage, and I think you will see us bring the fabric out again to get to additional compute, and there you have a switchless pod," says Feldman.

Right now, the important thing is that the Opteron variant of the x86 server card for the SM15000 has twice as many cores and twice as much main memory as the Intel-based server card using the "Ivy Bridge" Xeon E3-1200 v2 processor.

The pricing for the Opteron and Xeon machines is similar, at around $139,000, for either processor with base memory and disk. That's the kind of competitive edge AMD needs to deliver if it wants to be a server vendor. ®