[Flávio Ribeiro] Snickers: Open Source HTTP API for Media Encoding

At the beginning of this year we created a group on our video engineering team to deal with the ingesting, encoding, publishing and syndication of The New York Times videos. The main goal of the team was to build a pipeline that is vendor agnostic, cloud-based, efficient and elastic. We also wanted to build an easier workflow for video producers in the newsroom and third-party partners who distribute our videos. We named this team Media Factory.

After a few months of development, the pipeline is almost ready to be deployed to production. It consists of three different steps: Content Acquisition, Transcoding and Distribution. The content acquisition part is responsible for receiving ProRes/Raw videos, the transcoding API transforms the source files in playable videos by our supported browsers and devices and the distribution step sends the final videos to our CDN.

Probably the most important and complex part of the pipeline is the transcoding process. Instead of using heavy and expensive hardware for that we decided to create clients for different encoding services like Encoding.com, Elastic Transcoder and Elemental. This enabled us to decide which one to use based on a set of variables such as availability, speed, formats supported (HLS, webm, mp4, DASH, etc) and even cost.

While discussing the design of the pipeline and the integration with cloud services, we thought it would be useful (and FUN!) to implement another provider that would be just a tiny wrapper for FFmpeg or GStreamer. By using this tiny wrapper, we would have the freedom to deploy anywhere, implement new features (such as the generation of thumbnails) and perhaps, one day, be used to encode our lower priority jobs.

We have plans to write a blog post about the details of each step and how they work together on the Open Blog. We should work on that as soon as we have people at the company using the pipeline.

Time To Learn Some Go

After having a lot of fun on the journey to remove Flash from our player I got attracted by the chance to be part of this team. I was already willing to work with back-end services again and knowing that the company is adopting Go for API’s turned this opportunity into something even more attractive. Although I was participating on meetings and design decisions, I actually started working on the team in April.

I had never worked with Go before and after a few years plunged on dynamically typed languages I felt a bit of pain during the first steps working with it. The rest of the team was already fluent and I though I’d need something to exercise myself and keep up with the team’s pace.

Creating this dummy provider that is a wrapper for FFmpeg seemed like a good chance for me to evolve my Go skills. A month ago I started writting it, I called the project Snickers. I put this name because it’s my favorite chocolate and I was holding one when I created the GitHub repository.

To be honest, the title of this post doesn’t actually represent the current state of the project. Right now, it looks much more like a proof-of-concept/embryo project than anything else.

It supports the download of sources from HTTP and Amazon S3, encode them in H264/MP4 or VP8/WebM using a FFmpeg binding and uploads the results to Amazon S3. We still need a plethora of features to be able to actually call it a Media Encoding API, but one of the purposes of this post is to broadcast the idea and try to engage people to collaborate. Snickers already has contributors!

Next Steps

I have a bunch of ideas for this project. The next steps include building clients in JavaScript so people can use Snickers directly on the browser (with nice progress bars and stuff) as well as a client in Go, probably as part of the encoding-wrapper project. We also need to write some glue code for the Transcoding API so Snickers can be considered in the decision process of Media Factory pipeline. We use a lot of HLS at the company so HLS support is mandatory and probably the highest priority feature.

I’m also interested in working on the creation of a GStreamer driver for the encoding part so we can have benchmarks using different drivers. I saw that there’s already a Gstreamer binding for Go and it seems that the code is simpler than handling libav calls.