AuthorTopic: non-interpolating raw converter? (Read 19484 times)

Is there a raw converter that doesn't interpolate when it de-mosaics? This will of course mean that the output is a lower resolution, but on the other hand it should be a heck of a lot sharper and have less artifacting. I honestly think you'd end up with an overall higher quality image by working this way, with the only downside being a slightly smaller maximum print size. So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

P.S. If it doesn't already exist, I'm going to start writing one asap.

So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

Yes, and no. Google search "Canon sRAW", and you'll find some newer Canon cameras already do this. The problem is that to eliminate any need for interpolating, you need to downsize your pixel dimensions by 50%, turning 20MP into 5MP (albeit 5 really good quality megapixels). The problem is that existing RAW converters do a good enough job interpolating that you get most of the benefit of your 20MP even though they are interpolated. Downsizing to 5MP throws away a LOT of detail.

Before you start wasting time coding this idea, try this experiment:

Pick an uncropped image, process it to the best of your ability, and then print it as large as you can.

Make a copy of the image file you just printed, and downsize it 50% (2000 pixels > 1000 pixels, etc) using Bicubic Sharper interpolation.

Print the downsized image the same size as the original image.

Compare the prints side by side, realize the pointlessness of the exercise, and then go find something else useful to occupy the rest of your life.

Is there a raw converter that doesn't interpolate when it de-mosaics? This will of course mean that the output is a lower resolution, but on the other hand it should be a heck of a lot sharper and have less artifacting. I honestly think you'd end up with an overall higher quality image by working this way, with the only downside being a slightly smaller maximum print size. So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

P.S. If it doesn't already exist, I'm going to start writing one asap.

And exactly how are you going to do the demosaicing without interpolation?

Is there a raw converter that doesn't interpolate when it de-mosaics? This will of course mean that the output is a lower resolution, but on the other hand it should be a heck of a lot sharper and have less artifacting. I honestly think you'd end up with an overall higher quality image by working this way, with the only downside being a slightly smaller maximum print size. So does this already exist, or have I stumbled upon the greatest breakthrough in digital photography ever?

P.S. If it doesn't already exist, I'm going to start writing one asap.

As mentioned, dcraw (command line interface) will take an RGGB Bayer quartet of raw data, average the two green values, and output the resulting triplet as RGB pixel data. RPP (Mac only, with a functional but rudimentary GUI) is a raw converter based on dcraw that carries over this feature of allowing a "half-size" interpolation. Because the three colors were not located at the same point on the sensor, such an approach necessarily generates aliasing of high spatial frequency image content. As Graeme says, it's better to do a full demosaic of the raw data, and then downsample.

Canon mRAW and sRAW show little in the way of aliasing artifacts, so it is very likely that they follow this second route (especially mRAW, which downsamples by a linear factor 11/16; hard to imagine how you're going to do that without interpolation!). It would be silly not to, since the camera is already doing the full interpolation to generate the embedded jpeg.

As mentioned, dcraw (command line interface) will take an RGGB Bayer quartet of raw data, average the two green values, and output the resulting triplet as RGB pixel data... Because the three colors were not located at the same point on the sensor, such an approach necessarily generates aliasing of high spatial frequency image content

If dcraw really works this way, then the aliasing is a negligable issue, for the output is not truely RGB. Putting the "red", "green" and "blue" raw channel values together as R, G and B components is not demosaicing.

You can do that just by extracting one of the RGGB RAW channels and ignore the other 3. The image will be half the size in both axis, sharp and should have less aliasing artifacts than when demosaicing. The problem is it will be a monochrome image

You can do that by extracting the RAW channels in DCRAW:

dcraw -v -d -r 1 1 1 1 -4 -T file.cr2

Rescale the resulting TIFF to 50% using nearest neighbour (no idea the name nearest neighbour is given in the English version of PS). Add two extra rows/columns of pixels before rescaling to pick up the preferred RAW channel.

One sample (no sharpening at all applied, 100% crop):

Aliasing still happens (see nearly horizontal hairs on top of the girl's head). I guess the gaps between the photocells in the sub-grid used to form the image make them behave closer to delta samplers than to the more desired light averaging samplers, therefore easily producing aliasing. Accutance is higher this way though.

Ok, so when I first thought of this idea the simplest and easiest way to do it was the way that Canon seems to be doing it. You cut the height and width each in half, leaving only 25% of the original pixel count but with the advantage of giving you full color pixels with 0 interpolation. So to use Panopeeper's example numbers, you'd get something like:

Alright, now that's a pretty quick and dirty way of doing it, but is there a better way? The method above samples four different pixels in order to get all the color information needed to generate one pixel of output. So I thought, how else could I sample four pixels?

The grid shows the outline of the post-conversion pixels in spatial relation to pre-conversion. The dots with letters represent samples taken per color channel. On the left is supposed to be a normal interpolating algorithm. The marked pixel is supposed to be showing that for that pixel it can sample the green directly, but has to create sample values for red and blue by looking at neighboring pixels and interpolating. To the right is my second idea. You only lose half of the pixels along the outer edge, so on large images you retain nearly all the original pixel count. By shifting all the pixels, each one can now sample all the color information it needs. I think this will reduce aliasing, but at the cost of introducing a different type of artifacting. I suspect that this method will not increase IQ above standard methods, but maybe I'll write it out in Erlang just for fun.

I'm starting to see why no companies really take this approach. Except for Fuji, which I've heard has a camera that doesn't interpolate. But they get around that by laying out their color filter in a different arrangement than the standard Bayer. I think that's a pretty cool solution. Anyway, thanks everyone for your input.

If dcraw really works this way, then the aliasing is a negligable issue, for the output is not truely RGB. Putting the "red", "green" and "blue" raw channel values together as R, G and B components is not demosaicing.

Try it with a contrasty, near-Nyquist texture and see what you get. The aliasing is not negligible in situations where it tends to arise (eg as moire). While not demosaicing, it is interpolation, albeit a crude form -- the RGB information in the output data is coming from locations that are slightly shifted from the input values, and then the data is subsampled.

Alright, now that's a pretty quick and dirty way of doing it, but is there a better way? The method above samples four different pixels in order to get all the color information needed to generate one pixel of output. So I thought, how else could I sample four pixels?

The grid shows the outline of the post-conversion pixels in spatial relation to pre-conversion. The dots with letters represent samples taken per color channel. On the left is supposed to be a normal interpolating algorithm. The marked pixel is supposed to be showing that for that pixel it can sample the green directly, but has to create sample values for red and blue by looking at neighboring pixels and interpolating. To the right is my second idea. You only lose half of the pixels along the outer edge, so on large images you retain nearly all the original pixel count. By shifting all the pixels, each one can now sample all the color information it needs. I think this will reduce aliasing, but at the cost of introducing a different type of artifacting.

Suppose there is a sharp diagonal edge passing from lower left to upper right, with the green pixels along the diagonal having midtone values, the pixels to the upper left being low values, and to the lower right having high values. Your diagonal edge will be alternating RBRBRBR averaged in with the midtone green.

There is lots of image information that this sort of simple approach throws away, and that a more sophisticated, nonlinear demosaic algorithm takes account of.

To the right is my second idea. You only lose half of the pixels along the outer edge, so on large images you retain nearly all the original pixel count. By shifting all the pixels, each one can now sample all the color information it needs. I think this will reduce aliasing, but at the cost of introducing a different type of artifacting. I suspect that this method will not increase IQ above standard methods, but maybe I'll write it out in Erlang just for fun.

I guarantee you it will suck. You have no provision for taking into account the variations in the location of the source pixels, so you're going to have weird color alternations along all hard edges, whether horizontal, diagonal, or vertical. You have to do averaging of all the pixels that fall within the area covered by an output pixel, and you need to fill in the gaps in the Bayer matrix with interpolated values or you'll get weird color artifacting and aliasing. I've been playing around with interpolation stuff a LOT the last few weeks (see my thread in Digital Image Processing for more details), and very little of it is anywhere near as simple as it appears at first glance, at least if you want to maximize output sharpness without aliasing. Achieving both of those goals simultaneously is a cast-iron-b**ch.

I meant that the aliasing is negligable, because the result is pretty worthless anyway, for the raw channel pixel values are far from the R, G and B components of RGB.

Example for the color checker rendered by using the raw pixel values as RGB components and by demosaicing:

In your example, it isn't the demosaicing that causes the difference, it's the color space transform. In uniform flat areas there will be little difference between the sampling method suggested by the OP, and a full demosaic.