First Look at Google Compute Engine for Video Transcoding

For those of us in the cloud computing world, the most exciting thing that came out of Google I/O this year wasn’t skydivers wearing Glass, and it wasn’t a new tablet. The big news was that Google is getting into the cloud infrastructure-as-a-service space, currently dominated by Amazon Web Services (AWS). Specifically, Google has launched a new service called Google Compute Engine to compete with Amazon EC2.

This is exciting. The world needs another robust, performant, well-designed, cloud virtual machine service. With apologies to Rackspace and others, this has been a single-player space for a long time – EC2 is far and away the leader. Google obviously has the expertise and scale to be a serious competitor, if they stick with it. How does it look?
Early reports are positive. Google Compute Engine (GCE) is well designed, well executed, and based on infrastructure Google has been using for years. Performance is good – especially disk I/O, boot times, and consistency, which historically haven’t been EC2′s strong suit.

But how well suited is GCE for cloud video transcoding? We have some preliminary results, acknowledging that more testing needs to be done. Here are some basic tests of video transcoding and file transfer using Zencoder software on both GCE and EC2.

Raw Transcoding Speed

Performance is our top priority, so Zencoder uses the fastest servers we can find. On EC2, we use Cluster Compute instances, which are fast dual-CPU machines in two sizes: 4XL and 8XL. We compared these with the fastest GCE instance type, which is currently a single-CPU 8-core server.

Server

CPU

GCE 8-core

Intel Xeon (Sandy Bridge – probably E5-2670) – 8 cores @ 2.60GHz

EC2 cc1.4xlarge

Dual Intel Xeon X5570 – 8 cores @ 2.93GHz/core

EC2 cc2.8xlarge

Dual Intel Xeon E5-2670 – 16 cores @ 2.60GHz/core

These tests were done using an H.264 source video at 640×360 and 1280×720 resolutions, and were encoded by Zencoder using the same single-pass output transcoding settings (H.264 Baseline profile, AAC, one-pass Constant Quality transcoding, etc.).

Server

Resolution

Simultaneous Encodes

Time (seconds)

Cost per thousand

EC2 cc1.4xlarge

640×360

6

15.87

$0.96

EC2 cc2.8xlarge

640×360

6

9.93

$1.10

GCE 8-core

640×360

6

21.05

$1.13

GCE 8-core

640×360

1

6.01

$1.94

EC2 cc1.4xlarge

640×360

1

5.96

$2.15

EC2 cc1.4xlarge

1280×720

6

48.58

$2.92

EC2 cc2.8xlarge

640×360

1

4.99

$3.33

EC2 cc2.8xlarge

1280×720

6

30.74

$3.42

GCE 8-core

1280×720

6

68.15

$3.66

EC2 cc1.4xlarge

1280×720

1

12.89

$4.65

GCE 8-core

1280×720

1

16.01

$5.16

EC2 cc2.8xlarge

1280×720

1

10.92

$7.28

Using default Zencoder settings, both types of EC2 instance are faster than GCE. The economics are a bit closer, and there isn’t a clear winner between 4XL EC2 instances and GCE. So GCE is a viable option for transcoding where cost is a higher priority than raw speed, though AWS customers can make use of Reserved Instances and Spot Instances for further cost reductions.

We noticed that the 16-core EC2 instances were roughly twice as fast as GCE 8-core instances when under load with 6 simultaneous transcodes. Given the similar clock speeds, but half the number of cores, this is what you would expect. However, if Google adds similar 16 core machines, they could have comparable transcoding speeds.

Transfer Speeds

When transcoding video in the cloud, network I/O is almost as important as CPU. This is especially true for customers working with high-bitrate content (broadcasters, studios, and creatives). So how do GCE transfer speeds compare to EC2?

To test this, we ran four sets of benchmarks:

Amazon S3 to Amazon EC2

Amazon S3 to Google Compute Engine

Google Cloud Storage to Amazon EC2

Google Cloud Storage to Google Compute Engine

We did this by testing the same 1GB video file stored on Google Cloud Storage (GCS) and on Amazon S3. Transfer was performed using 10 HTTP connections. (Zencoder does this by default to optimize transfer speeds, and it can dramatically speed up large file transfers over HTTP.)

Transfer speed (Mbps)

Server Bandwidth

S3 to GCE

470.96

1 Gbps

S3 to EC2 c1.xlarge

644.29

1 Gbps

S3 to EC2 cc2.8xlarge

1458.32

10 Gbps

GCS to GCE

202.60

1 Gbps

GCS to EC2 c1.xlarge

378.28

1 Gbps

GCS to EC2 cc2.8xlarge

641.34

10 Gbps

This is interesting. We expected Amazon-to-Amazon transfer to be fast, which it was. But we also expected Google-to-Google transfer to be fast, which it wasn’t. In fact, it appears that GCS is slower than S3, and GCE transfer is slower than EC2, such that even if you’re using Google for compute, you may be better off using S3 for storage. Transfer was 2.3x faster from S3 to GCE than from GCS to GCE.

More Tests Needed

Consider these results preliminary. Further testing needs to be done to take into account more variables, such as:

a) Instance-to-instance differences. This is especially true for file transfer, which can vary widely based on network conditions and instance variability.

b) Additional applications. These benchmarks only cover transcoding, which is a CPU-bound benchmark. Other applications are limited by disk, memory, etc., and these tests don’t speak to anything other than transcoding.

c) Scalability. Scalability is extremely important for anyone using the cloud for video transcoding. More tests are needed to see how GCE compares with EC2 when it comes to enormous scale – tens of thousands of servers (or more). At what point do users run into capacity issues? Performance problems? Design limitations? Instability?

Exciting Future for Cloud Infrastructure

Even though EC2 wins in these early tests, we’re excited about Google Compute Engine. To be a serious competitor for high-performance transcoding, Google needs to add larger instances with faster CPUs. But adding new instance types is easy. Nothing prevents Google from doing this. What is hard is building a robust, performant, feature-complete, scalable cloud platform, and Google seems to have succeeded. If Google is committed to this product and developers for the long run, the cloud virtualization world may have just gotten a second legitimate player.

Anonymous

thanks for the analysis, i’ve been interested to see how Google’s cloud computing compares with EC2, I especially appreciate this analysis because I’m a developer using Zencoder so it is great to see Zencoder testing the bleeding edge!

Anonymous

thanks for the analysis, i’ve been interested to see how Google’s cloud computing compares with EC2, I especially appreciate this analysis because I’m a developer using Zencoder so it is great to see Zencoder testing the bleeding edge!

David Pokorny

Without more information, the network performance stats are essentially worthless. Transfer performance depends on the distance between points A and B. Are A and B in the same {region, datacenter, rack}? This is going to make the difference, not whether the components transferring the bits are owned by Google or Amazon. So give us the details, don’t omit important information.

http://brandon.arbini.com Brandon

Since we’re talking about Amazon and Google you can eliminate datacenter and rack from the list. However, we should have mentioned region. Our tests were run in Amazon’s US East region and Google’s us-east1-a region.

Grateful

Thanks for posting. Your data is certainly not “worthless”, although more tests would be helpful. Haters gonna hate – keep it up.

http://www.facebook.com/osewa Oluwaseun Osewa

You should have compared the performance per dollar to level the playing field meaningfully.

Interesting article having read the Amazon updates I will imagine it is no where near as advanced as Amazon now is.

http://twitter.com/williamlbell William Bell

I think that you would find some of the smaller players can still run circles around both GCE and EC2 our Phoenix NAP Cloud servers are designed from the ground up for mission critical apps…mix of e7 and e5 procs, entirely 10gbit inf. We have a few transcoding and big data customers that found our pricing models (allowing you to shutdown instances) super beneficial as well. The big guys do it OK…but the companies that specialize in multi-tenant infrastructure do it better.

Dan

Great analysis, thanks for sharing!!! Looks like Google is way behind. I would have expected much more from Google if they are serious about this space. The pricing they offer is not competitive and these numbers make things even worse. How is it possible that transfer speeds between GCS and GCE is slower than when using S3 with GCE?
BOTTOM LINE: It will take a while for Google to be competitive in IaaS

Toedad

Very interesting data. Thinking of drilling down to understand why such differences, other than dual socket (AMZN) vs single socket (Google). Things like raw CPU benchmark, Xen vs KVM would shed more insights.
GCS to GCE thruput is lower than 1Gbps. Was the storage benchmark test done in the way of writing data to the local harddisk?

http://brandon.arbini.com Brandon

I believe the table that shows the cost per thousand does essentially that. We took the time to encode the files, multiplied by 1000, and calculated the cost for each instance type on each platform. What else are you looking for there?

http://twitter.com/KaurKuut Kaur Kuut

I doubt it’s the
E5-2670, because that’s a 8-core CPU. The GCE cores represent 1 hyper-thread, so the 8 virtual core instance gets 4 physical cores, with each core providing two hyper threads. This is documented in their I/O videos, and possibly on the GCE website.

http://www.cloudcentral.com.au Kristoffer Sheather

I’d love for you to run your test suite against our new EnterpriseCompute cloud. Ping me if you’re interested.

Chris

Would love to see Rackspace Cloud Services and HP Cloud thrown into this testing.

http://www.victusspiritus.com/ Mark Essel

Really appreciate this post, even more than the comparison of Google to AWS it was insightful to see how the different ec2 instance types yielded different costs per video. Do you have similar upload numbers from ec2 instances to s3 @brandonarbini:disqus ?

i work with an awesome startup in NY creating video (FastSociety), and we’re always looking for ways to speed up our performance, as well as shrink our costs.

http://brandon.arbini.com Brandon

Unfortunately, we don’t have a good set of numbers for ec2 to s3 transfer. We should run some tests there though.

Chris

Compute Engine and Cloud Platform have evolved a lot since July 2013, for instance they now have High CPU and High Mem instances, and Compute Engine is now available. Have you done other testing since then?

Sunny

Hi, What were the lengths of the videos you used in your test?

Anonymous

was that a cost per thousand videos?

http://brandon.arbini.com Brandon

Yes, that’s what we are showing in the “cost per thousand” column.

interestedinclouds

who funded this study? who verified the methodology? who verified the results?

Mohan Vamsi

I couldnot completely understand your starts around time.For ex. Do the numbers against 640*360-1 mean show the times it took to encode just one file on each of these platforms? If yes, what is the file size?