All Cloud Performance Testing Benchmarks Are Not the Same

Cloud performance testing is important. Whether deploying applications or moving workloads to the cloud, your users will judge your team's ability on the actual performance. If it's too slow, you risk low productivity or even a lack of use. And, let's face it - there are plenty of marketing claims that say one cloud is faster than the other cloud.

You've also read about or experienced inconsistent cloud performance. We have the results and share the methods here you need to make an assessment.

Synthetic Cloud Computing Performance Benchmarks

UnixBench

DBENCH

Iperf

See the Results

Workload Based Cloud Computing Performance Benchmarks

Apache Benchmark

Nginx Benchmark

SysBench (MySQL)

Pgbench (PostgreSQL)

PostMark

See the Results

As Seen In

Synthetic Benchmarks: Measuring Cloud Computing's Raw Performance.

Synthetic benchmarks are designed to identify broad performance measurements, such as the UnixBench benchmark, or to test individual components of a system, such as the disk performance or networking performance. Cloud Computing's diverse architectures and foundational infrastructure combine to make it difficult to compare cloud providers' performance.

Performance engineers at ProfitBricks are constantly commissioning or running performance tests against all IaaS platform components of Amazon AWS EC2, as well as our own. They strive to identify bottlenecks, throttling, inconsistencies and opportunities for improvement in the underlying hardware, configurations, virtualization layers, platform software and networking that make up an IaaS platform. Cloud Computing providers have long used Synthetic benchmarks to compare Cloud providers. In this report you'll learn how to interpret these results and how to run your own.

Workload Specific Cloud Computing Performance Benchmark Testing

Why finding the right cloud requires benchmarking that is closer to your workload's requirements.

Benchmarks that are designed to mimic a particular type of workload on a system are a critical factor you should consider in your short list of vendors. Whether you intend to test your own applications and databases on a few cloud providers or not, workload specific benchmarks provide results that go beyond the popular synthetic benchmarks.

We've taken the time to set up cloud environments and all of their components: applications, databases, and network infrastructures, and run popular workload level performance benchmark tests to see how cloud computing providers compare with individual workloads. ProfitBricks has always encouraged cloud buyers to test their applications, and with these workload performance benchmarks we come closer to your actual workloads. In this latest series of tests we compare MySQL, PostgrSQL, Apache, nginix, and file transactions, and share the results with ProfitBricks engineers and our customers. Our goal? To ensure that we maintain or improve performance on a consistent basis, and provide guidance to customers on how to optimize their deployments for better performance. Our methods and data are transparent. Download the report and learn how to run your own reports or assess how your applications might perform in the cloud.

Learn why each workload has its own needs and performance results that vary.

As you have seen, Cloud Computing pricing varies widely; from how some vendors that only offer pre-packaged instances to those who have simple or extremely complex pricing schemes, it's more important than ever before to learn about cloud pricing.

Cloud performance also varies among cloud vendors. In some cases the variation is very high, up to 17x. Using the synthetic and workload performance tests found on this page, you'll be able to better understand how your performance requirements might drive your cost projects.

We encourage cloud buyers to give our cloud consultants a call - we've become experts at assessing workloads and their performance requirements, as well as the resulting costs, across providers. Our sales engineers can help you design the best cloud deployment architecture plans and help your team assess the costs or compare the costs among providers.