joys and sorrows of AWS

Menu

A wish list for the Amazon Elastic Transcoder

How long does it take to transcode a job? It depends, usually not too long but if you need real-time or almost real-time encoding you need to look somewhere else or wait and hope they will add the feature in the future.

Over a year ago Amazon paid apparently $296 million to acquire Elemental Technologies, a company that provides real-time video and audio encoding and still operates as a standalone company. But who knows, maybe there are some hopes for real-time video encoding in the AWS offer in the future.

I pay one minute of encoding, I have a nice output and I am a happy customer. I now take the output and reiterate the process. Same pipeline, same preset, just a new job. I might hope to have the output identical to the input – the Amazon Elastic Transcoder has transcoded it, from the metadata of the output it’s obvious. Instead a new output is generated, I pay one more minute and I can keep iterating. The output every time is similar but the quality decreases.

OK, I submitted the job but I am not sure it makes sense to generate an output that is not as good as the input and that already matches the preset as best as your FFmpeg transcoder could. Would not be better to avoid any transcoding operation when the input is already in a “margin of error” of the specs of the preset? Output files cannot (obviously) be better than input ones. But AWS will try and charge for it anyway.

All this might sound like a corner case, but let’s think of a scenario where your end users upload the videos from their iPhone or tablets or laptop. And you are looking for surprises if you do not have good filtering in front of the pipeline.

3 – Processing a few seconds of video

Actually that is not true, the Amazon Elastic Transcoder can process video of any length, even just a few seconds. Whatever they are 6.5 second looping videos (Vines) or short video from Snapchat or whatever other source at a higher resolution. But it might not be the most effective way and it can be quite costly.

“Fractional minutes are rounded up. For example, if your output duration is less than a minute, you are charged for one minute.”

So for your 3 seconds you pay 60 seconds for every output. Any way around it? There are two more efficients way to process short videos. You can use the new Clip Stitching feature that allows you to stitch together parts, or clips, from multiple input files to create a single output and fractional minutes are then better rounded up. But you then need to handle the splitting of the output yourself. Or you can replace the Amazon Elastic Transcoder with aws-lambda-ffmpeg or similar open source projects based on AWS Lambda + FFmpeg and that are more cost effective for very short videos. You can then determine the minimum length that triggers the job on the Elastic Transcoder.

4 – Run a spot job & bidding price

I can bid on the price of a EC2 instance. I can run Elasticsearch on Spot Instances. I can rely on cheaper spot options on a Amazon EMR cluster. But I have no real chance to lower the costs of running Amazon Elastic Transcoder. I cannot (at least automatically from the console) commit to a certain amount of encoded hours to have a reserved capacity at a lower price. Many times, when I do not need real-time video encoding, I do not have a strong constraint of having the items in just a few minutes. Amazon Elastic Transcoder does a great job to process them (usually) in a few minutes but I would mind the option to bid a price or have a significantly longer queue time in exchange of a lower price when the service is less used. To achieve that, you are back to a cluster of FFmpeg running on EC2 instances.

5 – Encoding available in every regions

The Amazon Elastic Transcoder is currently available only in a subset of regions (8) and in Europe only in Ireland. From a technical point of view, you can always configure the Amazon Elastic Transcoder to handle the S3 bucket in one region and run the encoding in a different one. Latency is not that big of an issue, given that it’s anyway an asynchronous operation. And the cost of data transfer is as well negligible compared to the cost of the service itself. But the blocker is often a legal requirement and you cannot process a user video outside a specific region where the data is stored.