Pixel Perfect Chroma Key Blend Using GPUImage

Dec 19, 2014

If you ‘re doing chroma key blend with GPUImageChromaKeyBlendFilter from GPUImage library like what I did in this post GPUImage Movie Writer Exposure Problem, then you’re very likely to find that it is very tricky to get pixel-perfect blend result:(e.g. annoying green lines around edges no matter how hard you tweak the threshold and smoothing parameter, also you need ask animation creator to be careful to not picking any close colors against the background green color which is super frustrating to them).

The reason is that if you look into the OpenGL ES GLSL code in GPUImageChromaKeyBlendFilter.m, you could find that actually it masks out the color which is close to background color(like green color by default). And this masking part is not very precise when it comes to some colors between background and character.

Then you probably wonder how Action Movie FX got this pixel-perfect blending video. After some hack, I got a clear idea about it. Following is the diagram:

+

:

+

:

=

As you see, it is not using green color as reference for cutting color, instead it got from a alpha-channel video (black-and-white).

Already someone posts an question for it on GPUImage issues list :Chroma or alpha on videos? #255, Brad has given some suggestions on it but there wasn’t any code about it. However, very obviously that instead of use two input filters like GPUImageChromaKeyBlendFilter, this has to use three input filters like GPUImageThreeInputFilter.m. If you looks at the code inside GPUImageChromaKeyBlendFilter, it notice that it doesnt’ have custom fragment shader to it, yes, it is designed to be subclassed to implement custom rendering.

After spent some time on tweaking it, I finally got it work. If you use GPUImage as pod dependency, I suggest to git clone a copy to your local computer, then change Podfile in XCode spike project to refer to local copy, so that you could easily debug it.

Extreme care should be taken for the order of addTarget:self.filter part, as this order maps strictly to the order in GLSL fragment shader.

There you go, you should be able to get pixel-perfect blended videos, no pulling hairs from animation creators.

All videos(including alpha-channel, fx, source) mentioned in above code could be found here : sample videos. Just quick reminder, the sample vidoeos are just for studying code purpose, do not distribute or use for commercial.