UPDATE #1: I received several comments about the testing methodology used in these benchmarks so I wanted to explain further. Each server instance at the Rackspace Cloud, Linode and Amazon are running the exact same Linux distribution (Debian 5.0 Lenny), kernel, architecture (x86_64) and version of Apache (2.2.9 w/ worker MPM).

UPDATE #2: After much concern was raised I went ahead and reran the benchmarks on the small instances after tuning a few settings on the Linode server.

echo 99999 > /proc/sys/net/ipv4/netfilter/ip_conntrack_max

Also, instead of running the tests from a consistent remote location I ran them on the local machines this way I could guarantee that network latency wouldn’t impact the test results in any way.

—–

In the past few weeks there has been a lot of talk about the differences between various Cloud and VPS providers (see my post), mainly revolving around performance. I decided to take some time last week to run some very straight forward and real world tests to gauge performance among the most popular providers: The Rackspace Cloud, Amazon EC2 and Linode.

For the benchmark I decided to use ApacheBench and Siege as *most* users on these platforms are serving some type of web content via Apache. Even if different web server software is used, it is unlikely that the results and trends would differ. As part of a series of posts, I will be running the same benchmarks against static content and dynamic content. This way we can be sure to look fairly at all aspects of each particular platform.

As mentioned, I decided to use ApacheBench and Siege as the testing platforms so I performed a default install of Apache 2.2 with the Worker MPM on each server instance. The server OS platform is Debian 5.0 Lenny on all servers.

These benchmarks are very easy to duplicate, so if you would like to test the results simply replicate the total connections or time and concurrency settings mentioned below.

I did not tweak any of the Apache settings on any of the server instances. I used the default settings of:

I ran the same tests against each platform and since I was only hitting the default HTML page generated by the Apache install I decided to raise the concurrency limit up to something significant to generate some real load. Each benchmark was run three times over three different days and I took the best result from each set of tests.

The Rackspace Cloud Server instance, with only 256 MB of RAM, performed well under the stress as did the Linode 360 server, but there is still a significant difference in the overall performance. This could be due to host server utilization, but according to the Linode control panel the host machine my instance was running on is “low”.

In the Siege results we get some similar insights. The 256 MB Rackspace Cloud Server processed significantly more transactions than the Linode server. Importantly, the transaction rate was also much higher.

There is always a lot of discussion around the Linode 360 server instance being a better value than the 256 MB Rackspace Cloud Server, but the tests show otherwise. In my opinion this is most likely due to Rackspace Cloud’s lower utilization of host hardware and a more robust network.

Large Size Instances

Now lets take a look at the larger instances. I wanted to work Amazon’s EC2 platform into these performance tests and their large instance is most comparable to the Cloud Server and Linode instances of the 8GB variety.

ApacheBench Results

Rackspace 8192MB Cloud Server

Linode 8640MB

Amazon EC2 Large

Concurrency Level

100

100

100

Requests [#/sec] (average)

10,729.61

9,109.91

6,782.93

Time Per Request [ms] (average)

9.320

10.977

14.743

Transfer Rate [Kbytes/sec] Received

3,363.62

2,829.17

2,113.04

Siege Results

Rackspace 8192MB Cloud Server

Linode 8640MB

Amazon EC2 Large

Transactions

1,564,152

1,636,652

891,956

Availability

100.00%

100.00%

100.00%

Response Time

0.04 secs

0.04 secs

0.07 secs

Transaction Rate

2,606.79 trans/sec

2,727.34 trans/sec

1,485.80 trans/sec

Successful Transactions

1,564,152

1,636,652

891,956

Failed Transactions

0

0

0

Longest Transaction

0.49

0.22

0.65

Shortest Transaction

0.00

0.00

0.00

Review

As we saw with the smaller instances in the ApacheBench test, the Cloud Server with fewer resources is outpacing both the Linode server and the Large EC2 instance. But in the Siege benchmark we see the Linode instance take the crown for total transaction rate. I would note the larger memory allocation on the Linode server instance might have something to do with these results so be sure to compare price for your particular application.

Amazon’s Large EC2 instance was the slowest of the bunch during the ApacheBench testing but did complete all of the transactions thrown at it during the Siege benchmark.

Final Thoughts

The tests show some significant performance benefits when running on the Rackspace Cloud Servers platform on the low end and similar performance to larger Linode and Amazon instances on the high end even though the comparable Cloud Server had much less memory and therefore lower cost. In my opinion, I believe lower hardware utilization and better host server hardware contribute to these results.

Apache settings can definitely be tweaked to increase performance on any of these platforms; this was just a baseline test using the defaults provided. The next round of tests will focus on serving a dynamic web applications.

If you have any thoughts, questions or comments about the benchmarks performed here please leave a comment below. I’d love to hear from you!

If you have been in the hosting business or simply hosting your own web sites for any length of time, you are probably familiar with the most popular commercial control panels on the market: Plesk, cPanel & Ensim. These control panels, while powerful in many respects often introduce more trouble than they are worth in the form of increased resource overhead, buggy implementations, cost, usability, etc.

The Rackspace Cloud has taken a different approach and decided to design and develop their own control panel in house from the beginning. Today, the Cloud control panel is an extremely powerful tool that is built specifically for the three product offerings from the Rackspace Cloud (Cloud Sites, Cloud Servers & Cloud Files). This control panel has many benefits over using a generic commercial CP, for example:

I recorded a video tutorial showing the process of installing JungleDisk Server Edition on a Rackspace Cloud Server. Many customers ask about the best way to do file-level backups on Cloud Servers and now the answer is simple: JungleDisk Server Edition. This is a great supplement to the snapshot based image backups that are already available with the Cloud Servers platform.

Lately I’ve found myself in the middle of several discussions about how Rackspace Cloud Servers and other platforms like it (Amazon EC2, Linode, GoGrid, etc) differ from traditional VPS environments. While there are similarities, they are by no means the same thing. Let’s take a quick look at some of the key differentiators and advantages.

A Cloud Server is a virtual dedicated server instance based on our custom implementation the Xen Hypervisor that run on very robust hardware. These server instances offer truly dedicated and protected resources which isn’t the case with a VPS (virtual private server). While you may get a small portion of dedicated memory, there is generally no dedicated CPU allocation, disk i/o, network i/o, etc. In addition, most VPS providers only offer smaller amounts of resources, 256MB-1GB of memory and no options above this. And the largest problem by far is VPS platforms are typically oversold, so one physical host machine is running far too many customers and this turns into a resource allocation nightmare causing server crashes or poor performance (or both). There is no isolation from other customers.

Cloud Servers does not yet have built-in reseller functionality in our control panel and it may be some time before we see something similar to Cloud Sites implemented. This leaves you with a few different options: