Oracle Exalogic – the emperor has no clothes!

Just a little over a year ago (in December 2010 to be precise), Oracle made the first official Oracle Exalogic Elastic Cloud X2-2 box. Oracle claimed that it was “The World’s Best Foundation for Cloud Computing”. There is still a lot of hype around Exalogic and in this post I will try to separate that hype from reality and answer the questions about Oracle Exalogic, such as:

What exactly is Oracle Exalogic?

Is Oracle Exalogic a true appliance?

How is Exalogic different from any other x86 server hardware sold by HP, Dell, IBM and others?

Exalogic is a lot of hardware – can it be virtualized?

Exalogic is called “Elastic Cloud” by Oracle. What makes it cloud enabled?

Is Oracle really first to market with the hardware and software engineered system?

How many customers are using Exalogic today in production?

How can Oracle claim 2-3X improvement in performance with Exalogic over traditional hardware?

Is it true that Oracle Exalogic is “Open and Standards Based”, while IBM’s 795 server is proprietary?

What kind of a solution would you recommend to customers who want to adopt cloud technology today?

Summary – comparison of IBM System z, Power7 System and Oracle Exalogic

Q: What exactly is Oracle Exalogic?
In one sentence Oracle Exalogic is a rack of up to 30 compute nodes mounted in a rack with InfiniBand backplane and ZFS storage, where each compute node has 2 Intel Xeon x86 CPU’s with 6 cores each. This means that a full rack has a total of 360 cores. All of these individual servers are interconnected with each other via InfiniBand networking with the ability to connect together up to 8 racks of Exalogic or Exadata on one InfiniBand network. Each of these 30 servers has 96 GB of RAM and 64 GB of SSD. Additionally the rack has 60 TB of SAS disk storage to be shared between those 30 servers (see full specs).

All this sounds fairly impressive on paper, however unlike traditional hardware, customers don’t have the flexibility to select the components they need in this hardware configuration, except to buy a (1) quarter of the rack, (2) half rack, or (3) full rack. Recently Oracle added a forth option – you can now buy one eights of a rack (I guess not a lot of customers went for the full package). However customers cannot select the right amount of memory, or SSD, or shared storage capacity for any of the above configuration options. What if the customer already has storage and does not need all that 60 TB from Oracle or 96 GB of RAM per machine? This is like buying a house with 10 bathrooms and 15 garages when you only need two bathrooms and one two car garage for your family.

Q: Is Oracle Exalogic a true appliance?
Nope. One other marketing aspect of this “Cloud machine” is the software stack that can run on it. In his Oracle World announcement, Larry Ellison said that patching the machine requires a single file to download, which updates the entire stack on the machine– from BIOS, to OS, JDK, Application Server and “all” other software on the appliance. None of it is true. All nodes in the Exalogic must be manually imaged with the OS using Oracle provided tool and the rest of the software must be (1) purchased separately, (2) downloaded individually, (3) installed manually, and (4) configured using an Oracle provided installation manual, which is several hundred pages long! This is not what is normally called “an appliance” nor is it a “single click” install process. Typically with a true appliance you expect it to be powered on and somewhat useful out of the box with very little configuration (IBM DataPower appliances are a great example). The Exalogic machine requires several days (if not weeks) of planning, installation, configuration and setup to be operational – just like any other server. Not to mention the time it takes to procure all the licenses and get the “right” software shipped or downloaded.

Bottom line is that out of the box, this Exalogic machine is nothing else, but a 42U rack of x86 servers mounted on a single rack and weighting 966 kg (2131 lb for those of us metrically challenged). You, the Oracle customer must apply your skills and expertise to turn this rack into something useful.

Q: How is Exalogic different from any other x86 server hardware sold by HP, Dell, IBM and others?
The first difference that I can see is that with those other hardware vendors, you have a choice of components and specs to fit your exact requirements, while Oracle sells a “one size fits all” Exalogic configuration (e.g. you are paying and getting that 96 GB of RAM and 60TB of storage whether you need it or not). And, there is really not much of a difference from a software perspective – all software must be installed and configured by the user anyway. Exalogic comes from a factory exactly the same way as other hardware systems did for the past few decades – you get bare bones hardware and you must initialize and install and configure it with your choice of OS (Oracle flavor of Linux or Solaris 11), then add JDK, application server and any other software you need it to run – all manually. How is it different from Dell or HP servers? It is not. Except for the fact that you can not install other flavor of Linux or Windows and there is specific Exalogic patch that you need to install into WebLogic domain – and this too is a manual install and you must not forget to do it (after you are done with installation, you then need to configure WebLogic – take a look at this huge configuration manual for WebLogic– not even including Exalogic hardware configuration and setup steps). In other words, Oracle provides you with the software and hardware, and you need to apply your expertise to configure it properly to do what you need it to do.

Q: Exalogic is a lot of hardware – can it be virtualized?
Here we get to the main point – the emperor has no clothes and no virtualization. I was trying hard to find any significant differences between Exalogic and other “normal” x86 servers sold by HP, Dell and IBM, and I found a second significant difference: unlike other x86 servers, Exalogic does not support virtualization.
Perhaps I need to repeat – as of today, there is no virtualization support on Oracle Exalogic. You cannot run VMware or even OracleVM on Exalogic today! What? Cloud with no virtualization? Are you serious? Perhaps Oracle has built some cool cloud software to run on Exalogic? Nope. There is no special cloud software. Even Oracle Virtual Assembly Builder does not support Exalogic.

What it really means is that the Exalogic machine is likely to get 20% utilization at best, just like any other non-virtualized x86 server environment. Just to give you a more specific example – customers will pay about $1.2M for hardware and another $4M for the software (assuming 50% discount), but with 20% utilization of the server they will actually use only $1M of that investment and never take advantage of another $4M that they have spent on this “engineered system” because 80% of their capacity will sit idle doing nothing. All Exalogic provides is a set of up to 30 independent servers in one rack with ability for those servers to be combined in WebLogic and Coherence clusters–all business as usual, no new capability, except for fast networking, lots of SSDs and RAM. When I first saw the Exalogic announcement, I presumed that all those servers could be virtualized; however I was amazed to learn that all 96 GB of RAM and 12 cores on each of those 30 servers in the rack can only run a single instance of the Operating System per server. In other words, the most you can do today is to run 30 OS instances on that rack. No more, no less – one OS per physical server with a choice between Solaris or Oracle Linux. You can’t even run multiple virtual machines on those 12 core servers!

In contrast, large multi-core machines with lots of memory can be very efficient server consolidation platforms where instead of “server sprawl” of hundreds of little machines with 5-10% utilization, workloads are moved into a fewer number of virtualized machines with much higher density of applications per compute resource and utilization rates often well above 80%. Two great examples of this are the IBM System z and IBM Power Systems. Oracle compares its Exalogic to the IBM Power 795 server, however there are huge differences between the two systems:

IBM’s Power 795 server is an SMP computer, which means a user could run all 256 cores as a single AIX image. This is great for database or other workloads where extreme vertical scalability is required. The greatest number of cores that can be used for a single OS instance on Exalogic is 12 cores which is not really a scalable environment.

IBM’s PowerVM is a highly scalable virtualization platform that can partition those 256 cores on the 795 server to run several hundreds of virtual machines with AIX or Linux workloads. Partitioning can be done in groups of several cores or multiple VMs that can share the same cores depending on user requirements. Intel x86 based version of the Oracle Exalogic today has no virtualization capability. The best you can do is to run a single OS instance on 12 cores–no more, no less. And you must license all 12 cores for your WebLogic. See my comparison of IBM and Oracle licensing in virtualized environments.

Oracle claims that InfiniBand provides for a fast network fabric and low latency. They definitely need that as 30 OS images need to communicate to each other via network. What is the comparable capability on the IBM Power 795? Well, because IBM PowerVM does the partitioning, we don’t even need to worry about Ethernet because all of the hundreds of virtual machines communicate to each other directly via memory. This is faster than using InfiniBand. Exalogic architecture is inherently limited in this respect and can never achieve what an SMP architecture can.

Another significant benefit of using PowerVM is the ability to dynamically move VMs between cores or even physical machine boundaries. For example, the administrator can move a VM instance without stopping it to a completely different server (perhaps located in a different building), or add/remove capacity to the existing VM. Exalogic does not provide any of this and is fixed and frozen on that single “compute unit”– one of 30 that sits in the box.

I would also like to mention that IBM currently ships a number of its software products packaged as a hypervisor edition, meaning that those products come as virtual machines pre-installed with the OS. The ever growing list of these products includes WebSphere Application Server, WebSphere Process Server, WebSphere Portal Server, DB2, WebSphere Message Broker, WebSphere Business Monitor, WebSphere Message Queue. The time to install and configure hypervisor versions of these products is significantly smaller than doing classic product installation into the existing OS. Plus IBM Workload Deployer provides pre-built templates for ultra-fast one click deployment as I have discussed in this article.

Q: Exalogic is called “Elastic Cloud” by Oracle. What makes it cloud enabled, or rather how does it enable cloud?
The word “cloud” seems to be the latest fashion at Oracle and even their latest version of WebLogic Server is called 12c (“c” for cloud – see my analysis of WLS 12c). Exalogic does not provide any cloud management capabilities in the current release. Without a virtualization or cloud management layer, it’s hard to accept Exalogic as the building block for the private cloud.

Q: Is Oracle really first to market with the hardware and software engineered system?
Oracle claims to be the “first to market” with everything, including the software and hardware “engineered to work together”. I have news for them – IBM has been in this market for over four decades with mainframe, CICS, AS/400 and other software and hardware integrated systems. IBM has done extensive research over years to optimize WebSphere and DB2 to run well on Power5, Power6 and Power7 architecture, and conversely some design decisions that went into Power7 were made to make Java and WebSphere and DB2 run faster on Power7.

Q: How many customers are using Exalogic today in production?
As of today, Oracle has provided only a handful of customer examples using Exalogic in real production. Contrast that with IBM’s thousands of customers using IBM System z and IBM Power Systems. IBM has enhanced the System z and Power platforms over decades of innovation. The first Exalogic platform shipped little over a year ago. The level of maturity is incomparable between these IBM and Oracle products. It takes years to iron out all the bugs from a system. At the moment, the IBM System z and Power Systems represents the first and second most advanced and mature server platform in the world.

Have you seen the server market share report from Gartner that was published this week? IBM is #1 with the server market share while Oracle is struggling to stay in top 5.

Q: How can Oracle claim 2-3X improvement in performance with Exalogic over traditional hardware?
Oracle compared Exalogic to “unspecified other servers”. The real question is not even an absolute performance, but rather price performance. What is the cost per transaction? Oracle claims they have the fastest machine, but mysteriously enough, have never published a credible SPECjEnterprise2010 benchmark result. If these machines are so fast, why not use them for benchmarks? The only two explanations I can think of are (1) the machines are not nearly as fast as Oracle says they are, or (2) the Exalogic machines are fast, but their cost makes price performance uncompetitive.

Exalogic has been on the market for one year and Oracle had plenty of time to submit benchmarks, yet they have chosen to use HP, Dell and CICSO hardware, not their own Exalogic for the SPECjAppServer2010 results. Isn’t it strange?

Q: Is it true that Oracle Exalogic is “Open and Standards Based”, while IBM’s 795 server is proprietary?
By now you probably can figure out the answer is “No”. In fact, I think Exalogic is more proprietary than IBM’s 795 server or for that matter any other IBM hardware:

With IBM hardware, including the 795 server, you can pick a wide variety of desired configurations and still have a supported platform. With Exalogic you don’t have that flexibility. If you change the Exalogic hardware setup, you terminate your support automatically.

With IBM 795 server you can pick and choose between a number of Operating Systems, including AIX, i/OS, SuSe Linux, RedHat Linux. There are many thousands of 3rd party applications, systems management tools, development tools and frameworks certified to run on these Operating Systems (including, but not limited to WebLogic, JBoss, WebSphere, JDE, Siebel, Oracle eBS, PeopleSoft, SAP, and endless list of others). What can you run on Exalogic? Well, not much. Your choice is limited to the new Solaris 11 or the brand new Oracle Linux. How many 3rd party or Oracle applications can you run on these OSs? Last time I counted, you could only run WebLogic, Coherence and a couple of new Oracle eBS v12 modules – that was it. Will ISVs start enthusiastically supporting the new unproven Oracle Linux Kernel or Solaris 11? That remains to be seen.

In summary with Exalogic, customers get locked-in to Oracle software and Oracle hardware and hand the key to that lock over to Oracle.

Q: What kind of a solution would you recommend to customers who want to adopt cloud technology today?
I t all depends on requirements and it is hard to prescribe one solution without knowing your situation. For some companies, a private cloud is the right solution and IBM can help build that. For others, public clouds, such as the IBM SmartCloud, Amazon EC2, Rackspace, to name a few, might be the best option. And, for some, LotusLive, SalesForce.com or similar SaaS solutions might be the best fit. It all depends. To help our customers save money and get started with clouds IBM offers Strategy and Change Services for Cloud for cloud providers and cloud adopters.