I Make Fun Stuff

WebRAW: Using Emscripten and asm.js to Process RAW Images

Jun 4th, 2013

Asm.js is getting lots of visibility because it has successfully brought near-native code performance to the web. This is a huge accomplishment, but I think it will take some time until developers figure out how to realize its potential. Many of the initial high profile examples have been games, such as BananaBread or engines, such as Epic’s Unreal Engine 3, powering the Epic Citadel demo. This is with good reason – games demand high performance, low latency, and consistent execution.

But there are other interesting use cases for asm.js and the Emscripten compiler. I do photography as a hobby, and I use Adobe’s Lightroom to manage my photos and to perform color correction and full image tasks. A few years ago, with the introduction of Tracemonkey, the original JavaScript JIT for Mozilla’s Spidermonkey JS engine, I created a simple “Darkroom” demo. The app allowed loading a JPEG image, and performing some image adjustments.

But now, I wanted to see if it was possible to go further – I wanted to work with the original raw files from my camera in the browser. To see what the base performance would be like, I compiled LibRAW using Emscripten. It required few changes (I mainly added some access methods that an approach like embind would do automatically). The initial results were quite good, and directly in line with what we have consistently seen with Emscripten and asm.js performance. These are times required to unpack and process a 36MP losslessly-compressed RAW file from my D800. The input NEF file is about 40MB in size.

Native: 7,812ms average
asm.js: 12,868ms average (1.64x)

You can play with a demo I put together. For best results, use a recent Firefox Nightly build. I believe the Aurora alpha builds should also work well. Drag and drop one of your own images onto the page. The embedded thumbnail will be extracted and put into the filmstrip at the top, along with some basic information about the image. Clicking on the thumbnail will load and decode the full raw image. The full image display will let you click to zoom and drag to pan around.

The demo’s UI is entirely done in regular HTML. The RAW processing happens on a web worker, where the asm.js-based LibRAW is loaded and the processing executed. The results are then sent via postMessage for display to the user.

It’s obviously not functional for real usage, and is purely an interesting tech demo. In particular, it uses some hardcoded defaults instead of selecting a proper white balance or choosing other appropriate processing parameters. It is also rather careless with memory; it could be much improved, but memory usage was not my focus. The performance is promising, and shows that Emscripten and asm.js are powerful tools that are usable in conjunction with standard web technologies instead of as just a porting method for native applications.

After I created this demo a week ago, I read an article about pics.io who is working to bring the full photo processing task to the browser. I hope they succeed! They’re not using asm.js, and instead have custom decoders written to take advantage of WebGL, thus offloading much of the processing to the GPU. This is a great approach for performance, but it does mean that they are limited in the formats that they support – currently, only support for Adobe’s DNG is present. A combination of an asm.js-compiled existing decoding library along with their WebGL-based manipulation tools could be a winner.