4 Answers
4

The reviewer is measuring scientifically in the lab, pixel peeping using test charts and compiling MTF curves. Owners of the lens are taking vacations shots and pictures of the family dog.

the reveiwer has experience with a number of other lenses, including pro lenses. Owners of the 18-200mm? It may be the only lens they own.

the reviewer is measuring 1% distortion that most users will not see in real life images. Most wouldn't know what pincushion distortion is, or notice it unless you pointed it out. I have the lens and distortion is only noticeable to me in shots of brick walls or skyscrapers, and photoshop corrects it anyway!

the reviewer is using test charts meant to expose any weaknesses in the lens. An owner of the lens is just taking pictures in real life situations and probably can't tell which images were taken with the 18-200mm and which were with the 50mm prime. I can't, not in terms of sharpness or distortion.

the reviewer is judging the quality of the lens vs cost to arrive at a overall value relative to other lenses. He will no doubt think it's pricey and may judge that you could obtain a better value (either sharper or less expensive). But an owner of the lens has already paid (or overpaid) and paid the credit card bill and they're not concerned about how it compares on a test chart against another lens. They're taking pictures, and able to zoom to 200mm or out to 18mm and catch shots they wouldn't get if they had to switch lenses, or left the other lenses at home.

I would also wager than 90% of amateur photographers don't know about or care about vignetting or chromatic aberration either. Even the trendy bokeh is probably not in most people's vocabulary :)

I bought the lens expecting it to be reasonably sharp, but mainly versatile and convenient, for a walk about lens. It's amazing that a super zooms exist IMO, much less that they are reasonably sharp. If I had experience using professional lenses, I might feel this lens was a bit soft or slow. But hey, it's basically a kit lens. Most of the people buying these are not pros and not interested in test charts.

What he says about distortion is probably true of all samples. Not sure about the softness at 135mm, I've not noticed it, and other reviewers like Thom Hogan didn't mention it. To be honest, I use the lens mainly between 18-50mm, and occasionally zoom out to 200mm to get some detail. I would rarely use 135mm.

I have done basic testing of my 18-200 at 50mm f/8 and compared to my prime 50mm at f/8. I didn't use a proper test pattern, but some newspaper. To my eye they were almost the same. The prime had a bit more contrast, and slightly sharper. If it had been a normal picture of a landscape I don't know if I could tell them apart to be honest.

As much as I love the my 85mm prime and a few others, if I could only own one lens I guess I'd stick with the 18-200mm for versatility. For that versatility it's worth the price IMO. So I'd give it a good review, but if I worked for dpreview and had all the gear to measure it against a database of other lenses, I might be more lukewarm in my assessment.

A great deal here comes down to the simple fact that most of what's measured in a typical lens test has almost nothing to do with how that lens will perform in real life.

First of all, most lens tests emphasize resolution. This gives some idea of the largest print you could produce from a picture and still have it look sharp -- but doesn't tell you much (if anything) about how it will look when sized down to fit on the screen of a computer or tablet -- and the last I heard, that's the primary way of viewing a large (and growing) majority of pictures.

Second, for most people almost none of that matters anyway. Lens testing is normally done at the lowest ISO the camera supports. Many (most?) people routinely use an ISO considerably higher than that, immediately reducing their best resolution to quite a bit worse than what the testers would rate as really poor performance.

Third, even when/if the do shoot at minimum ISO, most people can't plan on getting even close to the resolution of test anyway. Neither autofocus systems or hand focusing will get you even close to the resolution shown in a lens test. Since the testers can't focus accurately enough, they don't even try -- instead they just bracket the focusing, taking multiple pictures (moving the camera minutely from one to the next) and choosing the sharpest one.

Along with that, they are (of course) doing quite a bit to ensure the sharpest possible picture -- mounting the camera extremely solidly, pre-firing (or locking up) the mirror, using a cable release, etc. Many people will buy a camera, use it for years, and discard it without ever, even once, taking a single picture with nearly the care that's considered the bare minimum for a lens test.

Fourth, there are rather specific accepted rules about how lens tests are done, some of which keep the tests from having much to do with how the lens will work in real life. Just for example, it's generally accepted that you focus for maximum sharpness at the center of the frame, and then measure the results for the rest of the frame with that same focus. This will show a lens that has field curvature as having extremely "soft" edges/corners. When you take real pictures of three dimensional subject matter, you may easily see results that are exactly the opposite of what the test seems to indicate (i.e., a lens that looks bad in the test looks great in real pictures, while one that looks pretty good in the test doesn't look nearly as good in real pictures).

Bottom line: it's not just a matter of subjective opinion versus objective measurements, differences in standards, or anything of that sort at all. In reality, most of what you see in a typical lens test tells you next to nothing about how good of pictures that lens will produce in real use.

Most of the time when people get pictures that aren't sharp, it's caused by missed focus or camera movement (or both). Blurry pictures from an 18-200 lens will most likely stem from the fact that these lenses are fairly slow (small maximum aperture). Unless you have quite bright light, this will lead to either boosting the ISO or hand-holding at too slow of a shutter speed. Either will typically lose far more sharpness than lack of lens resolution.

+50. Reviews are taken more seriously by gadget lovers when they include scientific measurements done scientifically. The same applies to camera body reviews. This naturally leads to the respected review sites focusing on that which can be easily quantified, and on emphasizing quantifiable differences as significant. Really, this is just one piece of a balanced review. It's interesting technical information, and can be of use to a photographer, but shouldn't be the centerpiece.
–
mattdmJan 18 '12 at 14:33

I think what you're seeing here is the difference between an objective review and subjective reviews. The DPreview review includes a highly technical interactive diagram showing exactly how the image quality varies with focal length and aperture, with actual photos taken with the lens to demonstrate this. The statements you quote are statements of measured fact, not opinion.

The reviews on B&H are all very subjective, using little or no factual basis to justify their comments. I couldn't speculate as to the reasons behind individual positive reviews, but they could vary from a need to justify the expenditure, to low expectations which are met, or exceeded, by the lens.

A reviewer will specifically point out the problems of a product and notice the most minute details.

The average user, especially those who buy such (ridiculous) 18-200mm (and similar) lenses don't have very high expectations in the first place. Any design that covers such a focal length will be extremely compromising over using 2 or 3 lenses.

That is also why professional lenses often do not cover a huge zoom range. About "4x" tends to be the biggest range found in professional lenses, examples being the 70-300mm L from Canon and the 100-400mm lens.

Considering that you would expect the reviewer to have a serious interest in photography, he or she would most likely own one of the better lenses and use it as a comparison too, again negatively influencing the opinion of the 18-200mm lens.
And lastly, nearly any lens on an SLR gives better results than a compact camera. Many of the people who buy such 18-200mm lenses possibly won't use their SLR as anything other than a fancy automatic camera on "auto" mode anyway.