24Peter wrote:
I'm no expert but shouldn't the image resolutions be "normalized" to make a valid comparison? Won't the noise in the 5DIII image look "better" at 18MP, whereas the noise in the 1DX images look "worse" at 22MP?

Yes, but postprocessing the images comes at a cost, maybe introducing an additional variable in the already difficult comparison. I think that the other approach (to measure the noise difference and compare it with the correspondence in resolution terms) is also valid: that is, if a given camera has one full stop advantage in noise over other camera but has half the pixels, both have the same ISO performance upon physics laws (well, we depend on noise reduction algorithms performance when trading image resolution for noise). The 1DX vs 5D3 resolution difference should end in about 1/3 f-stop (log2(22/18)) noise advantage for 1DX at pixel level (eg. 1DX at ISO 1000 should match 5D3 at ISO 800) ending in the same noise at "image level" after applying the proper cleaning+downsizing+sharpening software to 5D3.

mmurph wrote:
That looks like it will be a very nice lens, assuming it performs to expectations. Canon has a very good track record with their recent lens updatyes. In additiona to the higher optical and build quality than previous models, the focusing is also much more precise on the newer camera bodies.

If you are able to get it in a kit at a good price it would definitely be a good purcahse. I think Lensrentals.com will likely have the first "authoritive": say whenm they get their copies.

My version I 24-70 is my favorite lens ever. I am still hoping to get a version II at a good price when I cvanb afford it.

Otherwise you might look to buy a copy of the 24-70 4.0 IS that is coming from a kit that gets split (by enbd user or eBay seller), or you might have to wait 6-9 months to find it at a reasonable price.

There is a cost penalty for being an early adopter, though it is much less on a lens than on a camera.

Thanks for sharing your comparison on the 6D! Whaterver the feedback you get here, it is something alot of people having been anxious to see.

You did a very, very good job in format and presntation. I know that was a lot of work on your part! Thanks again!

Using imagemagick is not such a hard job... the most difficult part was to download the files and to select the Width x Height + X + Y area to crop (6D required two zones, since the photographer moved the camera starting from ISO 25600). These are the commands to generate the first comparison crops:

(The none.png is a 1x1 black pixel, but it should have been properly sized, and properly placed, but the final picture looked ok despite the error). I think that in Windows it could be a little more tedious, but Mac OS X users also have command line shells with the for command (bash) for loops already available. They only need to install dcraw and imagemagick, if not yet installed. I did it on Linux.

I said that because a FF sensor has 2.6 times the area of a canon 1.6 crop. Since 2^1.38 = 2.6, that means that a FF sensor made with the very same technology should gather 1.38 f-stops more of light. In fact, one can crank up the ISO by 1.38 f-stops, getting the same noise at the image level (full picture) and turning the F4 lens "as bright" as a F2.5 lens is in a crop sensor, with the same noise. At the same time, that F4 lens is equivalent to 4/1.6 = F2.5 crop lens in terms of depth of field. The same reasoning may be applied to diffraction for a given resolution, concluding that both systems (Xmm F4 in FF compared to 1.6*Xmm F2.5 in crop) are totally equivalent....Show more →

Well, no. Depth of field, yes, exposure, no. The FF sensor doesn't gather "1.38 f-stops" more light, it gathers more light in total, but spreads it over a larger area, and in the end is no further ahead. If a shot is properly exposed at f/4 on crop, then it is properly exposed at f/4 on full-frame, with the same shutter speed and (real) ISO setting. There is no "equivalence" measure here.

This is one of the advantages of the smaller formats. Smaller lenses, same light gathering ability, for the correct exposure. For example, the Olympus 150/2.0 is really one stop faster (i.e. light gathering) than the "equivalent" (angle of view) 300/2.8 full-frame lens, but the smaller lens will have more depth of field, in spite of its smaller max aperture.

cgarcia wrote:
I personally prefer FF systems because, despite the cameras being a bit bulkier (the 6D is nearly a crop-size camera, though); same-quality, truly-optically-equivalent lenses seem not to be compact at all when paired with smaller sensors.

There is no such concept as "truly optically equivalent" small sensor and large sensor lenses. You can have the same angle of view, but as for aperture, you can either make the max aperture (light gathering ability) OR the depth of field (2 stops faster for MFT) equal, but not both. It depends what you are interested in. And of course, the camera is not passive in this equation, but adds more factors to "equalise".

Ultimately, it doesn't really make a whole lot of sense to try to keep comparisons equal over all these factors, and it is not possible anyway. For example, although the sensor in the E-M5 is 4 times less area (roughly) than a D800 sensor, the noise difference is less, due to different technological compromises having been made by the respective manufacturers, and this is ignoring the IBIS of the E-M5, which in many situations can more than equalise image quality.

What makes most sense is that the prospective buyer decides which factors are most important, in what order, and compares based on that. There is no single sensible "general" way to compare.

carstenw wrote:
The FF sensor doesn't gather "1.38 f-stops" more light, it gathers more light in total, but spreads it over a larger area, and in the end is no further ahead. .

Sorry, that is wrong. It is the total light that matters for image quality. This is because we compare images at the same final output size, so the image from the larger sensor need less enlargement. At the same exposure, the larger sensor is further ahead in image quality.

When we have the same total light (and different exposure): If we want to spread the light over a small sensor and then enlarge the image more, or spread it over a large sensor and enlarge the image less, we end up with the same result. We can simplify it and say we have the same total light, and spread it over the same total image size.

carstenw wrote:
For example, although the sensor in the E-M5 is 4 times less area (roughly) than a D800 sensor, the noise difference is less, due to different technological compromises having been made by the respective manufacturers, and this is ignoring the IBIS of the E-M5, which in many situations can more than equalise image quality.

Whatever difference there is in the technology is negligible compared to the sheer size effect. The sensor efficiency per area for those sensors is very close (56% vs 53% per sensorgen.info), so you cannot say that the E-M5 sensor tecnology invalidates equivalence.

Equivalence exists, and you cannot deny it. Of course you have practical considerations and other factors like lens performance that make direct comparisons difficult.

carstenw wrote:
Well, no. Depth of field, yes, exposure, no. The FF sensor doesn't gather "1.38 f-stops" more light, it gathers more light in total, but spreads it over a larger area, and in the end is no further ahead. If a shot is properly exposed at f/4 on crop, then it is properly exposed at f/4 on full-frame, with the same shutter speed and (real) ISO setting. There is no "equivalence" measure here.

This is one of the advantages of the smaller formats. Smaller lenses, same light gathering ability, for the correct exposure. For example, the Olympus 150/2.0 is really one stop faster (i.e. light gathering) than the "equivalent" (angle of view) 300/2.8 full-frame lens, but the smaller lens will have more depth of field, in spite of its smaller max aperture....Show more →

We are exchanging "aperture" and "f-stop" terms too liberally (well, this is normal at any talk!). And we must not forget the ISO advantage of bigger sensors in the equivalence equation. BTW the truly equivalent lens in FF to the four thirds 150mm @ F2 would be the 300mm @ F4.

A bigger sensor always captures more light (all other exposure parameters equal) at the same f-stop (thanks to using a bigger effective lens aperture diameter). What I tried to mean in my post is that when using a bigger sensor, one can always use a smaller f-stop ("downgrading" to the same aperture) to exactly mach the smaller sensor, of course using an equivalent (not the same) focal length. Better with a numerical example, remembering the old definition of f-stop (f-stop = focal length / aperture):

A 35 mm lens in FF using f-stop 4 has an aperture diameter of 35/4 = 8.75mm. To get the same shot in 1.6x Canon crop (same depth of field, same angle of view, same diffraction, etc) we need a 22mm lens at f-stop 2.5 (because 4/1.6=2.5) and fortunately this also yields an aperture of 22 / 2.5 = 8.75mm. Yes, both systems (FF at F4, crop at F2.5) gather exactly the same light as long as they use a "light cone" of the same 8.75mm width, and the proper lens to spread that light over their respective sensor surfaces. Of course, the light is spreaded in FF in a sensor surface 2.6 times greater. Thus, for the same shot (same shutter speed) in FF it would appear underexposed, unless the FF shooter increases the ISO 1.33 fstops to push the exposure 1.33^2=2.6 times to compensate (for example, by using ISO 250 instead of ISO 100). Now the two shots are identical, and because the full frame sensor has also a 1.33 f-stop advantage in ISO (its pixels are 2.6 times bigger and count 2.6 times the photons, increasing the signal to noise ratio) both match even in the noise matter. As it should be, because both sensors have the same performance in our assumption and the same amount of light was used taking the picture.

In crop cameras, in addition to multiply the focal lengths by the crop ratio to get the FF-equivalent focal lenght, we need to divide the f-stop by the crop ratio to achieve the same depth of field and diffraction effect. But we can also multiply the f-stop by the crop ratio to determine the FF f-stop with a equivalent light gathering ability (of course, this requires entering the ISO into the equation, which is fair if we also want equivalent noise levels).

The equivalence of systems is true, despite usually it is not available (smaller sensors usually have no comercially available fast enough lenses to be able to make certain shots that bigger sensors can -ok, trading depth of field-). Or the equivalence is made available at the expense of no advantage of the smaller format in terms of equipment size, like the example of the Zuiko 14-35 F2 vs Canon 24-70 F2.8 weight/size (or the bulky 35-100 F2 lens of the same manufacturer vs Canon 70-200 F4). Or the otherwise bulky designs are partially mitigated by creating lenses which desperately need heavy correction by in-camera software (many mirrorless lenses).

cgarcia wrote:
BTW the truly equivalent lens in FF to the four thirds 150mm @ F2 would be the 300mm @ F4.

Please re-read my post completely on this point. You seem not to have seen anything I wrote. Also read the part you cut. There is no such thing as true equivalence in the sense you use it or any other sense. There is light-gathering-equivalence (equal aperture) or depth-of-field-equivalence, and the two are mutually exclusive.

D. Diggler wrote:
I like to see both but non-normalized is more important to me.

Why? If you are making equal sized prints the normalisation will already be done for you anyway. Let me give you an example. The Nikon D3X and D700. At high ISO the D700 is cleaner at the pixel level. But if you resize the D3x 24MP to 12MP, you end up with essentially the same noise, yet noticeably more detail. The prints from the D3x will look better. You are never worse off and at low ISO you are way ahead and can print larger.

Pixel Perfect wrote:
Why? If you are making equal sized prints the normalisation will already be done for you anyway. Let me give you an example. The Nikon D3X and D700. At high ISO the D700 is cleaner at the pixel level. But if you resize the D3x 24MP to 12MP, you end up with essentially the same noise, yet noticeably more detail. The prints from the D3x will look better. You are never worse off and at low ISO you are way ahead and can print larger.

+ 100
That is true. But my own tests showed that the difference at 18/20/21/22/24 MP does not make a real big and visible difference (to me). At 21/36 it does indeed.

Just a quick comment, probably to multiple posters, so don't take it personally.

The notion that comparisons must somehow neutralize the cameras to some default state makes some logical sense but little photographic sense. No one simply leaves images as they come from the camera - we tweak, adjust, compensate, and so forth to pull our ideal best image out of what the camera captures.

I've always felt that the best real world test would be one in which a photographer takes images from all of the test cameras and then uses a workflow optimized for each camera to produce the best possible prints for comparison. This reflects that way that photographers actually use cameras to make real photographs. To make it more interesting, you might want to have, say, ten skillful photographers try to produce the very best images that they know how to produce using the same approach and then compare.

To shoot all cameras in exactly the same way and the leave all files in the their default state tells us as much about the actual potential performance of the cameras as putting drivers in three cars and making sure that they always push down the accelerator peddle to exactly 50% of its range of travel tells us about automobile performance. ;-)

For an expressive form that has beautiful and compelling photographs as its ultimate goal and which measure success by the effectiveness of those photographs and the degree to which they move us and communicate a personal vision of the photographer, we sometimes seem to focus excessively and almost obsessively on tiny, insignificant technical differences in hardware that matter very, very little. They are not completely insignificant, just way less significant that our attention to them seems to suggest.

Dan

cgarcia wrote:
Yes, but postprocessing the images comes at a cost, maybe introducing an additional variable in the already difficult comparison. I think that the other approach (to measure the noise difference and compare it with the correspondence in resolution terms) is also valid: that is, if a given camera has one full stop advantage in noise over other camera but has half the pixels, both have the same ISO performance upon physics laws (well, we depend on noise reduction algorithms performance when trading image resolution for noise). The 1DX vs 5D3 resolution difference should end in about 1/3 f-stop (log2(22/18)) noise advantage for 1DX at pixel level (eg. 1DX at ISO 1000 should match 5D3 at ISO 800) ending in the same noise at "image level" after applying the proper cleaning+downsizing+sharpening software to 5D3....Show more →

I agree with gdanmitchell on this occasion, but contrariwise, I’d like to point out that cgarcia’s original comparison image is flawed in a subtle but significant way. That is this: the photos have greatly differing saturation, particularly of reds. (Look at the burgundy material at the right of the test target. The 5D Mark III has the highest saturation, followed by the 6D, while the D600 has much less.)

Chroma noise is greatly affected by saturation, especially of reds. (Taken to an extreme, a fully desaturated greyscale photo would have no chroma noise at all.) And since no noise reduction has been applied, we’re mostly comparing chroma noise in this test.

That said, cgarcia didn’t tag his compilation JPEG with a colour profile, so all bets are off as to how it should really look.

S Dilworth wrote:
I agree with gdanmitchell on this occasion, but contrariwise, I’d like to point out that cgarcia’s original comparison image is flawed in a subtle but significant way. That is this: the photos have greatly differing saturation, particularly of reds. (Look at the burgundy material at the right of the test target. The 5D Mark III has the highest saturation, followed by the 6D, while the D600 has much less.)

Chroma noise is greatly affected by saturation, especially of reds. (Taken to an extreme, a fully desaturated greyscale photo would have no chroma noise at all.) And since no noise reduction has been applied, we’re mostly comparing chroma noise in this test.

That said, cgarcia didn’t tag his compilation JPEG with a colour profile, so all bets are off as to how it should really look....Show more →

You have reason, and I warned about the reliability. The 6D is using the 5D3 matrix for color correction in the second image I posted (since the first is uncorrected). But D600 results are not reliable at all (perhaps I could have downloaded the D800 raws, if available, since that camera is properly supported by dcraw). The 6D colors visually seem far better after the color correction fix, and I hope it won't be a lot of difference in noise levels once the converter is properly updated for this camera (in fact, noise didn't made a big difference compared with the first image posted).

I personally prefer to compare chroma noise. In the end, in the digital sensor noise is not grain but "wrong pixels" each of a single primary color. And chroma noise smeared to the neighbour pixels while demosaicing the bayer pattern further "helps" to see it better... we are looking for noise, not cleanness!. I suspect that a sensor with poor "raw chroma noise" compared to other better in this test, will not be able to achieve a cleaner luminance noise once postprocessed with the same algorithms (ok, it could appear relatively not so bad then).

The jpeg images were saved in sRGB (dcraw's default) in ppm files, then were cropped into png and composed into jpeg. Likely any image browser will interpret the missing profile as sRGB.

I must say. I find it a little bit strange that the much cheaper 6D has a better sensor than the 5D3. - Yes, even if it is only marginally better.. All tests I've seen shows that the 6D is improved over the 5D3 when it comes to RAW noise, on both low and high ISO settings.

I.m.o, the 6D sensor should be in the 5D3, and the 5D3 sensor should be in the 6D.

But oh well. I don't own any of these two cameras, so I personally don't care. And SoNikon's sensor blow both of them out of the water.

Rickuz wrote:
I must say. I find it a little bit strange that the much cheaper 6D has a better sensor than the 5D3. - Yes, even if it is only marginally better.. All tests I've seen shows that the 6D is notably better than the 5D3 when it comes to RAW noise. On both low and high ISO settings.

I.m.o, the 6D sensor should be in the 5D3, and the 5D3 sensor should be in the 6D.

I don't own any of these two cameras, so I personally don't care. And SoNikon blow both of them out of the water.

I'd hazard a guess that most of the IQ increase of the 6D is a firmware/software issue and not just a newer sensor design.

cgarcia wrote:
I have done a proper true RAW comparison (using dcraw only for demosaicing) between 5D3, 6D and D600 using the RAW files from www.focus-numerique.com. These are 100% crops from each camera, comparing aproximately the same subject (thus, the D600 image has the greater area and the smaller comes from 6D). Note that dcraw still doesn't natively supports the 6D, thus these results must be taken with caution (anyway, at least they are useful compared with any jpeg from camera or most jpeg conversions, usually garbage in order to make any conclusion).

Each picture has a label bellow indicating the camera name and ISO level (100-102400, except for the D600 which maxes out at 25600). Left is 5D3, center 6D and right D600. Warning, huge file (32 Mbytes) at 98% jpeg quality:

It seems that 6D has appreciable less noise than 5D3, more importantly at low ISO settings, far beyond the improvement expected for the slightly higher pixel size. It closes some of the gap in the shadows with the Sony sensor (D600 has still a bit less noise, though). At high ISO settings the 6D has a bit less noise than D600 (but note that D600 has nearly 20% more pixels, thus at image level they both could be very similar). I suspect that 6D will improve the dynamic range at low ISO settings. Canon is hearing, but needs more time... :-)

I have used the dcraw tool, "upgraded" for supporting the 6D. Using default values, of course no noise reduction, and the white balance calculated by averaging the full image (instead of by the camera). Dcraw conversion (at least without tweakings) maybe is not the best, and artifacts are visible, but indeed has no noise reduction at all.

I'm not sure if the colors are accurate for the 6D, since I manually upgraded dcraw for the 6D, only to take into account the masked pixels at the image border. This could also impact in the noise (I'm not sure) thus these results should still be taken with caution, until dcraw is properly upgraded by its author. Meanwhile the current version can be easily patched by searching the line containing "5920" (the 5D3 raw image width) and adding a new block (e.g. just bellow it) like this:

You might want to repeat the same test in wider DR, you will be supprise the noise differences that collects between canon and nikon sensors. Wider DR test result is really not close to what we have here: 6D vs 5D3 vs D600. Your data is not dynamic enough to say how those cam really work in real world. I suggest to have the candle light test if you want to do it in house. To test a heart we need to check its lung as well, we all know the sensor does not work alone.