In any event, the histogram shows the values in the rendered file, but does not indicate clipping in the raw file.

The histogram in ACR has always been the output refereed data, not the raw data. In Lightroom the histogram is Melissa RGB (ProPhoto RGB, sRGB tone curve–so pretty much output referred) or whatever RGB profile is set for soft proofing which is obviously output referred.

So, the question is, when RawDigger is showing that the raw data is clipped, what is PV 2012 doing with it to eliminate the clipping in the output referred data? Is it "recovery" as stated by Guy? Well, PV 2012 is NOT doing "recovery" the way recovery was done in PV 2010. I can guarantee that! So, exactly what is PV 2012 doing? I have a sneaky suspicion but I would defer to Eric Chan if he feels he can describe what's actually happening. When I describe PV 2012 as image adaptive and auto-ranging, that characterization was vetted by Eric. But understanding EXACTLY what is happening and what algorithms are being employed would require more imaging science than what I have. If you can read the Magic or Local Laplacian Filters? and understand it, then I think you'll be closer to understanding what PV 2012 is actually doing. But PV2012 handling of tone mapping is NOT the same as PV2010's recovery...which is a very good thing in my book.

The histogram in ACR has always been the output refereed data, not the raw data. In Lightroom the histogram is Melissa RGB (ProPhoto RGB, sRGB tone curve–so pretty much output referred) or whatever RGB profile is set for soft proofing which is obviously output referred.

So, the question is, when RawDigger is showing that the raw data is clipped, what is PV 2012 doing with it to eliminate the clipping in the output referred data? Is it "recovery" as stated by Guy? Well, PV 2012 is NOT doing "recovery" the way recovery was done in PV 2010. I can guarantee that! So, exactly what is PV 2012 doing? I have a sneaky suspicion but I would defer to Eric Chan if he feels he can describe what's actually happening. When I describe PV 2012 as image adaptive and auto-ranging, that characterization was vetted by Eric. But understanding EXACTLY what is happening and what algorithms are being employed would require more imaging science than what I have. If you can read the Magic or Local Laplacian Filters? and understand it, then I think you'll be closer to understanding what PV 2012 is actually doing. But PV2012 handling of tone mapping is NOT the same as PV2010's recovery...which is a very good thing in my book.

Jeff....who cares if it is the exact same recovery as PV2010. I think everyone knows that Eric changed the algorithms (for the better).

Consequently who care's what you call it? The clipping isn't there. The software doesn't show it's clipped and you don't get clipping. Seems accurate (accurate enough) to me.

Hi, Andrew...

The point I was making is that auto-recovery is being done, which we were told was not (even though anyone who tested would know it).

I agree that this recovery is great for most people....and in most cases....and protects them from "oh shucks" over exposure. However, it will result in probable changes to color and texture and the ability to do further exposure reduction. It doesn't take much testing to show how poorer the recovered areas are vs the same area in a properly exposed (non-recovered) image.

The point I was making is that auto-recovery is being done, which we were told was not (even though anyone who tested would know it).

I agree that this recovery is great for most people....and in most cases....and protects them from "oh shucks" over exposure. However, it will result in probable changes to color and texture and the ability to do further exposure reduction. It doesn't take much testing to show how poorer the recovered areas are vs the same area in a properly exposed (non-recovered) image.

The remapping of highlights that are blown in the raw file is definitely improved with PV2012. The highlights can be placed below 255 and color shifts are less likely with PV2012. However, this improved rendering can mask overexposure resulting from excessive ETTR and it would be helpful to have a raw histogram such as is available in Rawtherapee.

The remapping of highlights that are blown in the raw file is definitely improved with PV2012.

Thanks for using a better term–remapping–because that's what I think is happening. It's obviously not "recovery" related to PV 2010. That's clear...and Guy, when he did the demo with the girl and violin chose to prove his point using PV 2010, not PV 2012. The result would have been considerably different if he had compared PV 2012 to Aperture.

With regards to ACR/LR's histogram, I think the argument for providing a scene referred raw histogram is only useful for image analysis purposes, not actual raw image processing. The output referred histogram is what's important for raw image optimized processing. For analysis purposes, RawDigger provides a useful set of tools which I applaud. But I wonder how useful a redesigned ACR/LR histogram would actually help image processing. I rarely pay much attention to the histogram when adjusting raw images–there is no such thing as a perfect histogram. It's simply a tool to evaluate your image. What's really the most important aspect is what does the image look like and what do you need to do to adjust it?

As to what EXACTLY happens with PV 2012 and clipped highlights, I've pinged Eric to see if he has an interest in explaining...but I got an auto-reply that he's on the road for a while. So, if/when he answers, it'll be down the road. As I said, I really don't understand the math behind PV 2012 and will defer to Eric...

Thanks for using a better term–remapping–because that's what I think is happening. It's obviously not "recovery" related to PV 2010. That's clear...and Guy, when he did the demo with the girl and violin chose to prove his point using PV 2010, not PV 2012. The result would have been considerably different if he had compared PV 2012 to Aperture.

With regards to ACR/LR's histogram, I think the argument for providing a scene referred raw histogram is only useful for image analysis purposes, not actual raw image processing. The output referred histogram is what's important for raw image optimized processing. For analysis purposes, RawDigger provides a useful set of tools which I applaud. But I wonder how useful a redesigned ACR/LR histogram would actually help image processing. I rarely pay much attention to the histogram when adjusting raw images–there is no such thing as a perfect histogram. It's simply a tool to evaluate your image. What's really the most important aspect is what does the image look like and what do you need to do to adjust it?

As to what EXACTLY happens with PV 2012 and clipped highlights, I've pinged Eric to see if he has an interest in explaining...but I got an auto-reply that he's on the road for a while. So, if/when he answers, it'll be down the road. As I said, I really don't understand the math behind PV 2012 and will defer to Eric...

The raw histogram is important if you want to know where your camera clips, which is important if you want to optimize your exposures.

FYI...from EC on 1/30/2012:

Highlight recovery is always enabled in PV 2012. Combined with the highlight shoulder in PV 2012, this means that areas that would've shown as blown in 2010 may actually show as not clipped in 2012. (Also, 2012's HL recovery is improved over 2010, for better detail extraction.) This is the reason for the HL clip warning differences between 2010 and 2012.

In any event, the histogram shows the values in the rendered file, but does not indicate clipping in the raw file. This could cause problems when one is judging ETTR exposures.

Hi Bill,

Although the thread's topic originally was more about sharpening before it turned into Guy bashing, you step-wedge images show the issues with tonecurve rendering in PV2012 nicely. The fact that highlights that are clipped in Raw do not show that in PV2012. In fact, it does an automatic highlight recovery and, IMHO more importantly, a default highlight compression. That latter fact is exactly what Guy Gowan was harping on about.

As we can see in the attached chart I made from your Stouffer stepwedge conversions, and compare the PV2010 and PV2012 with the original step-wedge data, it is clear that the default PV2012 conversion suffers from significant upper midtone and highlight compression. That would indeed be detrimental for bride's dress and white cloud image content. The PV2010 conversion is much closer to how the original data (the blue line in my chart) would look in a straight conversion.

Of course Guy Gowan doesn't mention that the Highlights and Whites controls in PV2012 can help to restore highlight tonality, because that doesn't suit his agenda, but he does have a point that one would need to work the highlights in PV2012 conversion much more than usual, and similarly the shadows in a PV2010 conversion (which is what he advocates).

All this demonstrates that there is a bit of truth in all positions that are defended and that PV2012 can create a good image, with recovered highlights, but one really needs to work the PV2012 files very differently. One should e.g. not apply an simple S-curve or Clarity to a regular PV2012 conversion to boost overall contrast, if you want to keep some life in the highlight rendering. Work the highlights if you want the images to sparkle, and apply a curve adjustment if necessary.

Thanks for using a better term–remapping–because that's what I think is happening. It's obviously not "recovery" related to PV 2010. That's clear...and Guy, when he did the demo with the girl and violin chose to prove his point using PV 2010, not PV 2012. The result would have been considerably different if he had compared PV 2012 to Aperture.

One can use the term recovery in a specific or generic sense. If one uses the term recovery to indicate the algorithms used in PV2010, then the remapping that PV2012 does is different. George Jardine has some excellent examples with back lit scenes in his LR4 develop tutorial where he was able to get considerably better results with PV2012. Perhaps Guy could learn something from George's tutorial.

With regards to ACR/LR's histogram, I think the argument for providing a scene referred raw histogram is only useful for image analysis purposes, not actual raw image processing. The output referred histogram is what's important for raw image optimized processing. For analysis purposes, RawDigger provides a useful set of tools which I applaud. But I wonder how useful a redesigned ACR/LR histogram would actually help image processing.

I agree that the histogram should show the distribution of levels in the rendered image when one is actually editing an image and am not suggesting that the behavior of the histogram should be changed, but it would be nice to have a switch in LR/ACR for use in image analysis such as is available in Rawdigger. This would avoid unnecessary trips to Rawdigger when one needs to have an indication of clipping in the raw file.

With regards to ACR/LR's histogram, I think the argument for providing a scene referred raw histogram is only useful for image analysis purposes, not actual raw image processing.

Hi Jeff,

I do not agree, but that may also have to do with how one processes one's files. It is because LR/ACR in PV2012 doesn't allow control over the Raw Whitepoint selection with the Exposure control any longer, but the Whites control might have some. It depends on how it is implemented under the hood. A Raw converter like RawTherapee does give the user direct control over the Raw Whitepoint, and it offers a Raw histogram, both of which together with clipping indicators to allow to make an optimal conversion.

Attached are 2 crops, one with Lightroom PV2012 (with Exposure -0.50 to get closer to the default RawTherapee average brightness),and one with RawTherapee (with Raw Whitepoint to 0.80 to pull slightly more of the specular highlights into showing detail).

Besides the significantly more overall contrasty rendering by Lightroom, the huge difference in highlight detail is obvious. Also obvious is that both renderings are only basic conversions with lots of potential for tweaking, but this is the virtually straight out of the box rendering difference.

Cheers,Bart

P.S. I've added a Capture One Pro conversion, coming in with also more detailed highlights which, a bit like RawTherapee, requires little work to get where they should be.

The raw histogram is important if you want to know where your camera clips, which is important if you want to optimize your exposures.

Which is why such a raw histogram needs to be on the cameras. Then one must account for exposure + development just like the old analog days, and different people will use different raw processors (the developing part). In that case, one would examine PV2012 and have a specific idea of how their development and their exposure work together.

I'm not sure that a RAW data histogram is specifically useful for the purposes of highlight clipping. The blinking highlight overlay we have discussed in the past is much more useful in that respect as it also shows how much of the data is affected perceptually. But yes, that overlay should be based on the rawdata.

However, I would like to make a case for being able to adjust the scale of the x-axis in the converter, because that I do find a tremendous help when making specific adjustments to dynamic range.

The attached images show an example. - The first image shows the histogram in F-stops (incidentally in this case also the true RAW data). Because F-stops spread the dark tones and compress the highlights, I can easily judge the integrity of the darktones and at the same time adjust the level of absolute black with more ease and precision. (Note that the RAW data without multipliers is not particularly comprehensive, even if it would only show the brightnesslevels).

- The second image shows the histogram in linear space,This obviously does the exact opposite of the F-stops scale, and therefore allows me to check the integrity of the highlights and adjust the clipping level accordingly. It would also immediately show the parts that would be affected by highlight recovery algorithms.

- The third image is the final histogram in perceptual space,Whatever I want to do next, I am certain that the dynamic range is fully utilized and the data in perceptual space is fully optimized because it spreads across the entire width of the histogram.If I now want to apply local contrast enhancement, at least I can immediately judge with all certainty that I have exhausted the other possibilities of contrast optimization.

I think we all agree about that. But since we don't have that, we need to bracket and it would be useful if the Rawconverter would allow us to (without prior inspection by RawDigger) directly pick the best exposure based on actual Raw exposure, and tweak its rendering.

Besides the significantly more overall contrasty rendering by Lightroom, the huge difference in highlight detail is obvious. Also obvious is that both renderings are only basic conversions with lots of potential for tweaking, but this is the virtually straight out of the box rendering difference.

Ironically, I prefer the Lightroom rendering (having recently shot a lot of waves and water on a vacation to Hawaii). Also, pulling the Whites slider down would allow you to tune the brightness of the whites considerably...

BTW, I really don't care about the "virtually straight out of the box rendering difference" comparisons of any raw converters...what I care about is the ability of a raw converter to optimize the image. When I use LR or C1, I tend to touch all the controls that are needed to get the image the way I want it so "out of the box" is pretty meaningless for me.

Although the thread's topic originally was more about sharpening before it turned into Guy bashing, you step-wedge images show the issues with tonecurve rendering in PV2012 nicely. The fact that highlights that are clipped in Raw do not show that in PV2012. In fact, it does an automatic highlight recovery and, IMHO more importantly, a default highlight compression. That latter fact is exactly what Guy Gowan was harping on about.

As we can see in the attached chart I made from your Stouffer stepwedge conversions, and compare the PV2010 and PV2012 with the original step-wedge data, it is clear that the default PV2012 conversion suffers from significant upper midtone and highlight compression. That would indeed be detrimental for bride's dress and white cloud image content. The PV2010 conversion is much closer to how the original data (the blue line in my chart) would look in a straight conversion.

Of course Guy Gowan doesn't mention that the Highlights and Whites controls in PV2012 can help to restore highlight tonality, because that doesn't suit his agenda, but he does have a point that one would need to work the highlights in PV2012 conversion much more than usual, and similarly the shadows in a PV2010 conversion (which is what he advocates).

All this demonstrates that there is a bit of truth in all positions that are defended and that PV2012 can create a good image, with recovered highlights, but one really needs to work the PV2012 files very differently. One should e.g. not apply an simple S-curve or Clarity to a regular PV2012 conversion to boost overall contrast, if you want to keep some life in the highlight rendering. Work the highlights if you want the images to sparkle, and apply a curve adjustment if necessary.

Bart,

Thanks for taking the trouble to plot out the values for the raw file and various renderings. To investigate these differences in another manner, I selected a wedge exposed just short of clipping as shown by Rawdigger.

I then rendered it in PV2012 in sRGB with no exposure adjustment and the resulting plot was fairly linear except to the kink in the low pixel values where sRGB assumes a linear ramp. I then increased exposure by 1 and 2 EV using the exposure control of the raw converter with the results shown. Rather than clipping, the highlights are rolled off close to 255 but not clipped. I then selected another wedge exposed by the camera with +1 EV, and the results are very similar to those obtained with a +1 EV exposure increase made with the raw converter.

Rendering the overexposed file with PV2010 and a linear tone curve and the highlights in the 4 brightest steps were clipped to 255. The highlight differences in the brightest steps are hard to see in the graph, so I included them in tabular form below the legend. These results illustrate the image adaptive processing algorithms used in PV2012. With the overexposure, the brighter steps may appear blown on the screen with the PV2012 but are different and the normal appearance of the wedge can be restored with the tone mapping tools.

George Jardine also has a new tutorial "Image Correction Master Class".

Does a great job showing how to manage tone and contast corrections....highlights being one of the key areas.

Hi John,

Yes, (almost) everybody comes to the same conclusion, i.e. that managing the compressed highlights is the main issue to address in PV2012 (just as is managing the shadows in PV2010). The complicating factor is that ETTR exposed images require much more correction than e.g. a 1 stop under-exposed shot that will have more default highlight contrast in a PC2012 conversion. Changing the overall exposure will require to adjust the Highlights and Whites as well.

Instructors like George Jardine and Martin Evening are generally considered to be reliable sources of information.

Ironically, I prefer the Lightroom rendering (having recently shot a lot of waves and water on a vacation to Hawaii). Also, pulling the Whites slider down would allow you to tune the brightness of the whites considerably...

Hi Jeff,

It's not so much the brightness of the whites that becomes a challenge in PV2012, it's the highlight contrast in those tone-compressed whites. That doesn't have to be a problem as long as we can adjust the Raw conversion before it is demosaiced. Otherwise, after demosaicing, we will have lost some of the definition in the brighter tones already.

When you re-read the great article by Charles Cramer you'll see that he needs to use a maximum setting of -100 Highlights Slider control to tame the highlight tonality. Also, the Clarity control compresses the highlights even further because it expands mostly the mid-tone contrast. Brightening the image with the Exposure slider also compresses the highlights. Working the highlights is a task of major importance in PV2012. Charles' summary is spot on.

When an image has significant highlight content, one may alteratively even prefer to process those in PV2010 (assuming the exposure was correct and not clipped), or blend 2 renderings in Photoshop from PV2010 (for the highlights) and PV2012 (for the mid-tones and highlights).

Quote

BTW, I really don't care about the "virtually straight out of the box rendering difference" comparisons of any raw converters...what I care about is the ability of a raw converter to optimize the image. When I use LR or C1, I tend to touch all the controls that are needed to get the image the way I want it so "out of the box" is pretty meaningless for me.

I agree that the straight out of the box rendering is only a starting point, however I'd rather start with a decent starting point. It's a bit like having to climb a steep hill, I'd rather start half way than at the bottom. Besides, some of the tonality may already be lost when the Raw conversion/demosaicing itself doesn't incorporate the taming of the highlights, parametric processing doesn't help with that. Doing it as post-Raw conversion is like climbing that hill with one leg.

Imho the need for a raw histogram in raw converters comes from the realization by IQ conscious photographers that as long as all desirable Raw image information/data is contained within 14-16 integer bits - making use of most of them - a perceptively 'good quality' image can be generated with full control of typical Raw Converter/Editor variables, by making a number of informed choices and compromises. These choices and compromises can be quite different whether the desirable highlight in question is clipped or not, although both final images may at first appear similar to a naive observer.

So given that the objective of the IQ conscious photographer in the field is to ETTRv2 subject to his artistic constraints in order to ensure that s/he starts with all desirable information captured and at its best, the objective of the Raw Histogram/blinkies in the Raw converter is to confirm that that is indeed the case - and if it's not to give him/her the information s/he needs in order to make his/her informed choices and compromises.

For instance if one knows that desirable highlights are irretrievably clipped in the raw data, one may not even try to reduce image values linearly (-EC) in order to attempt to give them more detail in a subsequent PP step. Conversely, if they are not clipped in the Raw data but are apparently so in the finally rendered image, one may decide to apply -EC in the Raw Converter and compensate for it by adding brightness/contrast (local and otherwise) in other ways.

There are equivalent examples in the shadows, and I personally find myself tripping back and forth to RawDigger at the beginning of a conversion to check just this sort of thing out. I would find it very useful to have some of its functionality built into current Raw conversion software.