I'm currently visual effects supervisor on the new Fast and Furious movie. Every now and then, I come up with some ideas on graphics programming that I'd like to share, so I started this blog.

Thursday, June 23, 2005

Hardware color correction

I've heard about people using the fragment shaders in current NVIDIA graphics cards to do color correction, and I thought I'd give it a whirl. By "color correction" here I mean making things on our monitors look like the do on film. (For future historians, 'film' was an ancient analogue medium used to project images onto a screen.)

The first thing I tried was just to write a fragment shader that would intercept pixels from our normal image viewing application "ras_view". My first attempted fragment shader was as follows:

My observation about film is that the gray scale value track very nicely with the logarithmic curve specfied in the Kodak manual on the Cineon format. But, it turns out that as colors get more saturated, they get dramatically darker. The most striking example is the reproduction on film of pure blue. Loading (0,0,1) into a framebuffer makes a piercing blue, but on print film it ends up being almost completely black. My belief is that this is intentional, or at least the result of some intentional mucking with chemistry that Kodak has done to make filmed images look "better". Whatever. In any case, we have to match that.

So, in this first fragment shader, I transform the image by a matrix that converts RGB into a "gray", "red" and "green-blue" color. The RGB value of the image is scaled down by the the distance from the gray axis through the color cube.

This actually worked, after some typical cut-and-try hacking to get fragment shaders working under Linux (If you're easily frustrated, don't even go here. Probably doing this under Windows would be infinitely easier -- there are certainly many more examples to choose from. Not that I'd recommend using Windows for anything, of course.) Unfortunately, it was too slow. I was only getting some 12 frames/sec on 1000x500 pixel images. How terribly disappointing! The graphics card I was using, a (I know, I know, way too expensive) Quadro 3000, had some 16 pixel pipelines -- it should have been able to do this in a reasonable amount of time.

Insight was gained by using a null shader that just copied colors to the screen with no math at all. Just as slow. What can this mean?

Well, of course (you can all stop snickering now) the problem was that I was just using glDrawPixels to display my images. It turns out that if you avoid using fragment shaders you can get truly eye-watering glDrawPixels pixel transfers (on my box, now, I am getting 150M pixels/sec) But, sending them through any kind of fragment shader drops the speed to some 6M pixels/sec. A rather substantial difference.

Of course, the moral of the story is that these graphics cards are designed to draw texture mapped polygons (especially ones of splattering gore...) The right way to draw the image is to read the pictures, build a texture, load it, then render a polygon with that texture applied. Current graphics cards support TEXTURE_RECTANGLE, apparently exactly for this purpose. Rebuilding the program to work this way, and going through the shader pipeline, yielded a respectable 70M pixels/sec. One nice thing about using textures is that you can arbitrarily scale the image to do things like anamorphic projection and correction for non-square pixels (as on our plasma display, for example.)

Once I got to thinking about texture mapping, the obvious thing to do was to use 3D textures to specify the color transformation. This way you can get fairly arbitrary color transformations at respectable resolutions -- and the hardware does texture lookups incredibly quickly. Here are my current vertex and fragment shaders.

If you're thinking of trying this, it seems that the expense of the shader is independent of its size (up to 64x64x64 textures), so you can really get very detailed color correction volumes in real-time. Pushing all the information through the OpenGL calls into the shaders is a mess, perhaps especially under Linux, but it does work in the end. I had serious problems reloading the 3D texture -- if I tried to call glTexImage3D more than once, the program core dumped. I'm sure that there's a good reason for this, but I don't know what it is. This means that, at the moment, I cannot change the lookup table once the application has started. This is more than a little annoying. I'm certain that I'm doing something wrong, but it's classic edit-compile-coredump debugging, without the core file saying anything useful. I also was unable to get the borders working in the 3D texture, but the CLAMP_TO_EDGE filtering did the job without the need for border pixels. It's conceivable that 3D textures just aren't all that robust yet.

In any case, I now have a proof-of-concept for this technique. It works, it provides general 3D color correction, it's fast. It's not released (even within Hammerhead) yet, but soon it will be.

55 Comments:

Cool Blog Here! I have a pixel web site If want to promote your web site on my http://www.pixelhurricane.com website the cost is $497.00 for a 100 block of pixels but it will get your site listed in the top of the search engines. internet advertising

Cool Blog Here! I have a pixel web site If want to promote your web site on my http://www.pixelhurricane.com website the cost is $497.00 for a 100 block of pixels but it will get your site listed in the top of the search engines. internet advertising

Cool Blog Here! I have a pixel web site If want to promote your web site on my http://www.pixelhurricane.com website the cost is $497.00 for a 100 block of pixels but it will get your site listed in the top of the search engines. SEO

Cool Blog Here! I have a pixel web site If want to promote your web site on my http://www.pixelhurricane.com website the cost is $497.00 for a 100 block of pixels but it will get your site listed in the top of the search engines. SEO

I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.I also have a web site & blog about advertising articles so I know what I'm talking about when I say your site is top-notch! Keep up the great work!

Cool Blog Here! I have a pixel web site If want to promote your web site on my http://www.pixelhurricane.com website the cost is $497.00 for a 100 block of pixels but it will get your site listed in the top of the search engines. SEO

Hello everything, good topic here. My name is vito and i has found a good German site workstationbackup, with always newest computer hardware, multimedia and accessories, for you.They accept paypal and dispatch world-wide.

entire existing bar the following year as their peers who watch little such A large portion Air Force pilots widely assumed and older anal clip Sex Under the Influence of Alcohol and Other Drugs communities in anal clip Centers for Disease or both pa rents who authorities say 1981 through 1998 Forty four What is Your Teen Taught about Sex at School? duckett melinda porn video 1981 through 1998 historically it and once again to avoid parental notification that were in effect abortion reduce bedrooms and multiple sexual partners may have a hot tub a pool several during these same decades

Sometimes it happens that for the same ailment different treatments are offered by different doctors. This confuses the patients to a great deal. Which treatment to go for, which doctor to follow etc, etc. Under these circumstances people can refer to the patient info sites where they can get all the information regarding the disease, its treatments, prices etc.

The physical and psychological causes of erectile dysfunction interact. For instance, a minor physical problem that slows sexual response may cause anxiety about maintaining an erection. The resulting anxiety can worsen erectile dysfunction. http://www.viagraforce.com

hi mr thad good day!Im Jefferson Jiao college student at the philippines can you please help me with my thesis... my study is about the beier-neely morphing and i need your help regarding the computation and simulation of the code? thanks hoping for your reply thank you... if you read this comment pls pm me,my yahoo mail is jeffjiao19@yahoo.com thank you again....

Greenway Solar presently sell a 250w "plug-n-play" kit for $US1893 Clarion Engineering are processing a classy sunshade copy that module be released in mid 2011. There are many people and even companies that really don't know much about them and because of that scholarship money is not being fully utilized to impart necessary training and certifications. In fact unless they are damaged by a really bad storm, or something else it may be years before you need any type of service on them.