Tuesday, December 28, 2010

CRT Pixel Shader Filter for SNES Emulation

Update 05/18/2011: More screenshots for more new filters in my new post. :-)

Update 05/02/11: After many changes, it looks like the CRT shader development has settled down, so there's less need for me to maintain the older versions of the shaders. From here on out, I recommend visiting Screwtape's git repo for all of your XML shader needs (my mediafire account will still be there, but Screwtape has all of mine and more).

This post covers the use of filters to upscale pixel art--specifically as it applies to SNES emulation--with special attention to CRT reproduction. If you just want the pictures and to download the filters, skip to the bottom of the post.

Background:

As everyone who dabbles in old-school emulation knows, artwork that was intended for a 480i CRT television that has been upscaled to an HD resolution looks like absolute garbage on an LCD monitor. The chunky sprites with their often thick, cartoony outlines just weren't designed to be reproduced with sharp edges resulting from nearest-neighbor upscaling.

To get around this ugly upscaling effect, many emulators now include upscaling interpolation filters, which apply complex mathematical algorithms to the original picture to fill in the gaps between things that are impossible to represent in chunky low-res, such as curves and smooth diagonal lines. You're probably familiar with some of the more common and popular interpolating filters, such as SuperEagle, SuperSaI and HQ2x. Unfortunately, none of these filters gets everything quite right, especially numbers and letters, which can look bubbly or overly smoothed (you can learn more about pixel art scaling algorithms here).

Purists have long been turned off by the inaccuracies of interpolating filters and have instead used scanline masks to try and capture the effect of an interlaced display, relying on the human brain's natural ability to recognize patterns and fill in the gaps between lines (you can learn everything you ever wanted to know about scanlines here). However, this too falls short from a true representation of a CRT display, as it ignores the existence of phosphors--the tiny red-, green- and blue-colored lenses that the electron gun in the back of a CRT tube shoots with a beam of electrons to recreate a colored pixel--and the color bleed that naturally occurs in these displays.

Recently, a number of determined individuals have set out to try and capture all of the different effects of a CRT display, warts and all, to truly reproduce classic pixel art the way it was meant to be viewed.

The Comparisons:

(Each of these images is presented as it would be displayed onscreen, at a resolution of approximately 800x600, then again at 400% scale without any interpolation used when scaling; as always, click the thumbnail to embiggen)

First, we should look at the baseline. This was scaled up to size using nearest-neighbor and is otherwise untouched:

Next, we'll add blargg's NTSC filter, which emulates the noise and color bleed of an NTSC video signal (this filter has several presets; I will only be showing the RGB preset, which reproduces the look of an SNES hooked up via RGB connection [not available in the U.S.], and the RF preset, which reproduces the look of the SNES RF modulator attachment, respectively):

(<- Look at that noisy RF signal!)

As a note, blargg's NTSC filter is so accurate that byuu, the author of bsnes, recommends its use along with bsnes' accuracy profile to achieve proper blending on games that use halftones to simulate transparency (Jurassic Park and Kirby's Dreamland, for example).

Next up, we'll look at cgwg's CRT shader, which includes a phosphor mask and barrel distortion to simulate the screen curvature of a CRT television (just look at those RGB phophors!):

Similarly, there is a version of cgwg's CRT shader, which doesn't include the barrel distortion and represents an idealized flat CRT (actual flat CRTs tended to have slight blurring at the edges where the tube curvature would normally be). Incidentally, this version also has no visible garbage pixels (the occasional black specs that are visible in the curved version):

Pixel Shaders vs. Software Filters

cgwg's CRT shader is a special kind of filter known as a pixel shader. Unlike regular filters, which rely on the CPU to do all of the complex upscaling calculations, pixel shaders draw on the awesome computing power of the video card to do the calculations, thereby leaving the CPU to focus on emulating the SNES. Additionally, since the pixel shader is calculated separately from the filter in bsnes, you can stack blargg's NTSC filter with cgwg's CRT shader:

Finally, for non-purists, we'll look at the combination of cgwg's CRT shader with the popular SuperSaI filter, which creates a pleasing--though not quite as accurate--output:

As amazing as cgwg's CRT shader is already, there is still some room for improvement. For example, the current implementation misses the intensity-based bloom effect on individual phosphors that can be seen in a true CRT. DOLLS (J) [!], one of the contributors to the CRT reproduction effort, intends to write a more complete CRT emulation shader in the future that will incorporate these and other idiosyncracies.

UPDATE (3/4/2011): Themaister did a rewrite of the flat version of cgwg's CRT shader, moving many of the calculations from fragment to vertex, which provides a substantial ~20% increase in speed (making it usable on many older and less powerful video cards). This rewrite also appears to conform more rigidly to the GLSL shader spec, making it compatible with more cards from different vendors. I have labeled it v4 of cgwg's CRT Flat, and it is available in the aforemented mediafire account.

15 comments:

Anonymous
said...

As CRTs continue dying out and becoming increasingly harder to find, this kind of 'emulation' will become increasingly important. Twenty years from now, no one under the age of 30 will probably have even seen a CRT operating.

That's a great point. What seems like pointless nostalgia right now will someday mean the difference between proper and improper reproduction for future generations.

I'm reminded of classic Greek and Roman sculptures, which recent discoveries reveal were originally painted in bright, lifelike pigments. These colors were lost over time to the point that generations of scholars have thought the sculptures were *supposed* to be viewed in haunting, pale, bare marble.

To be honest, any of the filters you present fail horribly at emulating a crt tv. I think that the problem is that they all still think in pixels. A real CRT deals with lines on phosphor. What you see is a line that gets thinner or broader depending on how hard the electron beam hits the phosphor. This line is continuous. Consecutive pixels are blended blobs that should be calculated and placed at a somewhat higher frequency than the horizontal pixel frequency.I for one never ever have seen squarish pixels on a TV set. Some PC monitors came closetho. These seem similar to the cwgw's CRT shaders. But still, they emulate a high precision monitor and not a tv.

If you take a look at cgwg and DOLLS' commented code and related calculations, you'll see that they have taken a number of CRT characteristics, such as beam decay and phosphor blending, into account and reproduce them accurately. Many of these calculations are based on published whitepapers from display technology journals.

However, one area that is essentially unapproachable with current technology is the shadow mask, and I think that's mostly what you're referring to.

Once screen resolutions get to something like 2500p, we'll be able to just draw a shadow mask on the screen and divide each CRT pixel into a grid of many LCD pixels, but until then adding a shadow mask overlay looks like garbage and creates issues with moire as it interacts with the LCD subpixels.

cgwg did use LCD subpixel characteristics to try and recreate the interplay of the individual phosphors, but the result is still less-than-optimal and doesn't render properly on all displays.

If you want to see another approach that focuses more on the phosphors themselves, check out 'caligari's scanlines' shader on my other shader post. It still has square pixels in a grid, but again, that's more a limitation of existing display resolution than a failing of the implementation.

Kind of sucks that there's no way for me to download any of the CRT Shaders (and even if I could theres like 4 folders filled with so many shader files each one I have no idea what to do with to get working with either bsnes of snes9x), some instructions on how to install them would be handy but also when I go to mediafire and click on a folder or file to download, all I get is a darkened screen overlap with a help button with no download starting, even after signing up for an account :/

@Chris AndersonI'm not sure why Mediafire isn't working for you but you can download many of the shaders I've covered from here, alternatively:http://gitorious.org/bsnes/xml-shaders/trees/master/shaders/OpenGL/v1.0

For bsnes, the *.OpenGL.shader files (you might have to rename them to get the .OpenGL part in the name; bsnes only recognizes shaders named in that fashion) go into a folder named either 'shaders' or 'video shaders' depending on the bsnes version you have. For snes9x, I believe you can apply them from the video section of the preferences menu. For both bsnes and snes9x, the GLSL shaders only work if you're using the emulator's OpenGL display driver.

Similar trick that's used to emulate old analog gear could presumably be applied in this problem, however in practise I suspect it might be too slow.

I saw a video on tube of Commodores Amiga CRT with some guy showing it side by side with plasma. It was very obvious how much better the CRT looked even when just viewing it through a video.

So if a CRT picture of low res game can look much better through video, obviously that can be algorithmically reproduce. Volterra kernels are used in acustica's audio product called nebula to emulate the dynamic response (capacitors etc) of analog gear to analog input.

I believe you could shoot a video of a CRT and align the video with the digital pixels - then use the volterra kernels to generate the response data of the CRT to do similar trick as the audio convolution but to the visuals.