Sony Xperia Z sinks in DxOMark Mobile Report

DxOMark Mobile Report: Sony Xperia Z

Touted as Sony's new "superphone" and leading the charge as one of the first to foster a growing trend toward 13MP camera phones, the Xperia Z certainly piqued the interest of mobile photography enthusiasts when it was launched at the start of this year. We know we were eager to see how the phone fared under DxOMark's rigorous scientific testing.

The DxOMark Mobile Report includes DxO Lab's usual industrial-quality scientific measurements. Its imaging experts have analyzed 14 aspects of mobile imaging including detailed image quality assessment, flash performance, autofocus reliability and more to calculate a final score. This report will be integrated into our full review once it is finished. We do not yet have a review unit at the DPR Connect offices but will start working on our own evaluation as soon as we get one. For now scroll down to find out how the Sony Xperia Z's camera performed in the DxO lab tests.

Sony Xperia Z headline features:

13-megapixel camera with Exmor RS sensor

2MP front-facing camera

5-inch touchscreen (1920 x 1080 pixels, 441ppi)

1.5GHz quad-core CPU

2GB RAM

16GB of internal storage

MicroSD card slot, up to 64 GB

Summary

With a DxOMark Mobile score of 61 the Sony Xperia Z is ranked low among DxO smartphone rankings. It came in only two points above Apple's New iPad and the only other devices it left behind are older phones like the Samsung Galaxy SII or the Apple iPhone 4. However, it's far below the best in class such as the Nokia 808, Apple iPhone 5 and Samsung Galaxy S3. In particular, it exhibited a poor performance in low light conditions.

The DxOMark team report that the Sony Xperia Z captures images with "good overall exposure," "nice colors in every lighting conditions" and "low noise level in low light without chromatic component."

On the downside: DxOMark reports that many pictures were out of focus. Images show "very strong low frequency chroma noise in the shade in bright light conditions" and "some white balance errors outdoors." In low light, the camera demonstrated poor texture preservation, a strong loss of texture, poor autofocus repeatability and excessive exposure time, even when the image stabilizer is off. Color shading occurs under a tungsten light source.

In video mode, the Xperia Z redeemed itself a bit. DxOMark's engineers reported that the Sony produced video with "excellent contrast and dynamic: best device tested to date" and "good noise reduction."

However, the report also found the Xperia Z video testing showed "loss of texture, even in bright light," "artifacts such as aliasing (staircase effect on straight lines) and flickering" and "inefficient video stabilization." Autofocus was not very reactive to scene change without moving the device.

Color, Exposure and Contrast

Still Photography

The DxOMark team found that the Sony Xperia Z images show "good overall exposure" and "nice colors in every lighting conditions."

However, the Xperia Z also displayed "excessive exposure time in low light, even when the image stabilizer is off." In difficult lighting conditions, contrast can be lost in the shadows. The Xperia Z shows color shading under tungsten light and some white balance errors outdoors. Because of this, DxOMark scored the Sony Xperia Z at 78/100 for color in bright light and 67/100 in low light.

Overall DxOMark awarded the Sony Xperia Z scores of:

4.2 out of 5 for Exposure

3.5 out of 5 for White Balance accuracy

4.0 out of 5 for Color shadingin low light*

3.8 out of 5 for Color shadingin bright light*

3.0 out of 5 for Color Rendering in low light

4.0 out of 5 for Color Rendering in bright light

*Color Shading is the nasty habit cellphone cameras have of rendering different areas of the frame with different color shifts, resulting in pictures with, for example, pinkish centers and greenish corners.

Texture Acutance is a way of measuring the ability of a camera to capture images that preserve fine details, particularly the kind of low contrast detail (textures such as fine foliage, hair, fur) that can be blurred away by noise reduction or obliterated by excessive sharpening.

Sharpness is an important part of the quality of an image, but while it is easy to look at an image and decide visually whether it's sharp or not, the objective measurement of sharpness is less straightforward.

An image can be defined as "sharp" if its edges are sharp and if fine details are visible, but in-camera processing means it's possible to have one of these (sharp edges) but not the other (fine details). Conventional MTF measurements tell us how sharp an edge is, but have drawbacks when it comes to measuring fine detail preservation. Image processing algorithms can detect edges and enhance their sharpness, but they can also find homogeneous areas and smooth them out to reduce noise.

Texture Acutance, on the other hand, can qualify sharpness in terms of preservation of fine details, without being fooled by edge enhancement algorithms.

A detail of target made of a dead leaves pattern, designed to measure Texture Acutance. It is obtained by drawing random shapes that occlude each other in the plane, like dead leaves falling from a tree. The statistics of this model follow the distribution of the same statistics in natural images.

In this example from a DSLR without edge enhancement, sharpness seems equal on edge and on texture. Many details are visible in the texture.

In this second example edges have been digitally enhanced, and the edge looks over-sharp, with visible processing halos ("ringing"). On the texture part, many details have disappeared.

At first sight, the images from these two cameras may appear equally sharp. A sharpness measurement on edges will indeed confirm this impression, and will even show that the second camera is sharper. But a closer examination of low contrasted textures shows that the first camera has a better preservation of fine details than the second. The purpose of the Texture Acutance measurement is to qualify this difference.

Note:Acutance is a single value metric calculated from a MTF result. Acutance is used toassess the sharpness of an image as viewed by the human visual system, and is dependent on the viewing conditions (size of image, size of screen or print, viewing distance). Only the values of texture acutance are given here. The measurements are expressed as a percentage of the theoretical maximum for the chosen viewing condition. The higher the score, the more details can be seen in an image.

For all DxOMark Mobiledata presented on connect.dpreview.com we're showing only the 8MP equivalent values, which gives us a level playing field for comparison between phone cameras with different megapixel values by normalizing all to 8MP, suitable for fairly large prints. DxOMark also offer this data for lower resolution use-cases (web and on-screen). For more information on DxOMark's testing methodology and Acutance measurements please visit the website at www.dxomark.com.

In low light texture acutance is slightly better under daylight than tungsten illuminant but things even out at brighter light levels.

In terms of texture acutance the Xperia Z scores well below other devices in its class, though it improves in brighter light conditions.

Edge Acutance

Edge acutance is a measure of the sharpness of the edges in images captured by the phone's camera, and again we're only looking at the most demanding of the three viewing conditions that DxOMark reports on, "8MP equivalent."

The Sony Xperia Z's ability to retain fine texture in images is best at 100 Lux (very dark overcast day), results decline in darker conditions and less so in brighter light.

In terms of edge acutance the Sony Xperia Z performs better than even the best in class at 100 Lux.

Visual Noise

Visual Noise is a value designed to assess the noise in an image as perceived by the human visual system, depending on the viewing condition (size of image, size of screen or print, viewing distance). The measurements have no units and can be simply viewed as a weighted average of noise standard deviation for each channel in the CIE L*a*b* color space. The lower the measurement, the less noise in the image.

Noise levels in daylight conditions are low across all light levels.

Noise levels are slightly higher under tungsten lighting conditions.

The Sony Xperia Z's noise levels are lower than its competitors' at low light levels, but images exhibit slightly higher levels of noise in brighter conditions.

Noise and Detail Perceptual scoring

DxOMark engineers don't just point camera phones at charts, they also take and analyse scores of real-world shots and score them accordingly. Their findings for the Sony Xperia Z were:

Artifacts

Phone cameras, like entry-level compact cameras, tend to suffer from artifacts such as sharpening halos, color fringing, vignetting (shading) and distortion, which can impact on the visual appeal of the end result. DxOMark engineers measure and analyze a range of artifacts. Their findings after testing the Sony Xperia Z are shown below:

Pros:

n/a

Cons

Slight color fringing but at an acceptable level

Perceptual scores

Sharpness 4.0 out of 5

Color fringing 4.0 out of 5

Measured findings

Ringing center 7.2%

Ringing corner 5.4%

Max geometric distortion -0.31%

Luminance shading 14%

Distortion and Chromatic Aberrations

The graph shows the magnification from center to edge (with the center normalized to 1). The Sony Xperia Z shows a slight pincushion distortion, which you are not going to notice in normal photography.

The Xperia Z shows some lateral chromatic abberations but still withing acceptable limits.

Autofocus

DxOMark also tests autofocus accuracy and reliability by measuring how much the accutance -- sharpness -- varies with each shot over a series of 30 exposures (defocusing then using the autofocus for each one). As with other tests these results are dependant on the viewing conditions (a little bit out of focus matters a lot less with a small web image than a full 8MP shot viewed at 100%). Using the 8MP equivalent condition the Sony Xperia Z results are especially poor in low light. The overall score is 67/100 in bright light and 26/100 in low light.

Pros:

Few oscillations

Cons:

Poor repeatability in low light conditions. Many pictures are out of focus.

Flash

Pros:

Cons:

Poor texture preservation.

Autofocus sometimes lacks repeatability.

Red color cast with tungsten light source.

Overall DxOMark Mobile score for Photo: 62 / 100

Video Capture

DxOMark engineers put phone cameras through a similarly grueling set of video tests, and you can read their full findings on the DxOMark website here. We'll simply summarize for you. DxOMark found the Sony Xperia Z's exposure to be the best in any device it's tested to date. However, results exhibit loss of texture, artifacts and flickering, and image stabilization performance is inefficient.

Pros:

Good exposure: best device tested to date.

Good noise reduction.

Cons:

Loss of texture, even in bright light.

Artifacts such as aliasing (starcaise effect on straight lines) and flickering.

Inefficient video stabilization.

Autofocus not very reactive to scene change without phone motion.

Overall DxOMark Mobile score for Video: 61 / 100

DXOMark Mobile Score

61

DXOMark Image Quality Assessment

The Sony Xperia Z achieves a DxO Mark score of 61 which means it can only place itself in front of older phones such as the Samsung Galaxy SII or Apple iPhone 4. However, it is far below the best in class such as the Nokia 808, Apple iPhone 5 and Samsung Galaxy S3. In particular, it exhibited a poor performance in low light conditions.

The Sony Xperia Z offers "good overall exposure," "nice colors in every lighting conditions" and "low noise level in low light." On the downside: DxOMark reports that many pictures were out of focus. Images show chroma noise in shadow areas and in low light the camera preserved textures poorly, with poor autofocus repeatability and excessive exposure time, even with the image stabilizer switched off. Color shading occurs under tungsten light sources.

In video mode, the Xperia Z captured footage with "excellent contrast and dynamic range" and "good noise reduction" but also showed a "loss of texture, even in bright light" and "artifacts such as aliasing (staircase effect on straight lines) and flickering." Image stabilization in video mode was found to be inefficient and the AF was not very reactive to scene changes without moving the device. For a more detailed analysis, visit www.dxomark.com.

Strange being Sony's compacts and NEX take superb pictures. (I have an NEX 5R 16-50mm PZ and got some outstanding shots in the lovely British sunshine today, so why didn't Sony continue in that tradition with the Xperia Z when phones like the Galaxy Note (1) that I once owned take superb photos that are hard to tell from even a mirrorless or NEX in good light?

stop acting like a fanboy and education yourself, Nikon scores are like pentax and sony scores, because they use the same sensors in most camera. Sony iso is lower because it use slt and that has some light lost. Canon used older tech so it has less DR, the iso is lower, but canon cook the raw to get improve real world effects. Some Olympus and hasselbald are using sony sensors now, not canon.

This comment needs more recognition. On the basis of that thread, it's incredible to believe Sony messed up the device with such OTT noise reduction. I imagine future software updates will improve image quality.

Interesting test, though pointless. Phone cameras are mainly used by persons not in the least interested in any kind of serious "photography". Any snap will be good enough.I personally know a few guys who even don't bother downloading their images from phone to PC. All they do is watching their photos on the phone screen. They would never ever care about 100% accurate sharpness, grain or whatever.Although I have to admit that it's quite a shame for a company like Sony to incorporate such a mediocre camera. If even Apple can do better, Sony should seriously consider some improvements.

@wetsleet The point is, Sony is a renowned camera maker with a good reputation and no matter whether users really "need" an above average camera or not, Sony should provide its phones with cameras that are able to vie with the elite of competitors. Even more since Sony charges quite a bit for its top notch phones. It's all about reputation.

A few years ago, many people would say 'Digital cameras are mainly used by persons not in the least interested in any kind of serious "photography". Any snap will be good enough.' and yet somehow dpr survived...

This review meets my findings and it's a shame because what this phone's camera really needs is less pixels. Sony maybe follows the trend for this type of sensor in mobile phones and maybe wants to show the way to anyone serious about photo quality in their serious digital cameras, but what they only manage (thanks to dpreview as well) is to fail in their most important task at the moment which is, of course, to make their name synonymous (in people's mind) with great photographic tools.(PS. sorry about the long sentence. This is a very Greek way of writing)

I may choose the S4, too, as far as I've seen preview, the camera quality seems to be very decent.The comparison tool is very helpful.

@Cartagena Photo:Even more amazing is the fact that the Samsung Galaxy S4 features "only" the Exmor R (not RS) sensor!Either sony built a "new" sensor that is way worse than the old one (with same MP), or they failed at some software "optimazations"(post-processing) at the camera of the Xperia Z.

A smaller screen works but a larger one with more pixels it's just nicer...works the same way with megapixels. When I had my D70 I thought 6MP is more than I'd ever need. When I shoot a D800 nowadays I would not want to go back though :-)

In USA most people drive. In other countries a lot of people use their phones on buses and subways and during work breaks and it's very much an internet device and gaming and video device so a larger screen is preferable.

An easy to understand analogy from Lars, yet so many seem to miss the point. The point being in general, that what we found perfect in the past, had a context. The context being the norm and stage of technology as a reference. Both are ever changing, as does what we feel fits our needs best. Experiencing change is the only way to know what you could be missing.

A shame, knowing that their own sensor can do so much better in terms of detail and even in low light. See the S4 (compare the Z and S4 at GSMArena). The S4 shows that with better processing, the 13 MP sensor can take much cleaner and still much more detailed pictures than the 8 MP sensor found in the S3. Which means that more MP isn't the real problem. Processing is.

Well it is just a camera phone, what do you expect? Downsized to your typical 800px web resolution or a 4x6" print, the images look pretty good for a phone. Many of the issues such as sharpening etc.. that it is knocked for at a pixel level work in it's favor for small prints, the web, and a phone's intended use. Acutance is more important than texture at these small sizes. Considering what most phone cameras are used for, it's more than adequate. I had an Xperia X10 and it's 8mp camera wasn't very good, but worked ok for the above said uses and this one is better.

When phones catch up to the basic IQ level of things like a Fuji x10 or Sony RX100 (about the lowest quality for anything serious), then these tests are splitting hairs. We have at least 5 years until a phone will make a quality small enlargement (8x10). The 5mp camera on my Motorola is unusable for even the web, so I'd say the Sony did quite well for it's intended use. Others may be better, but not good enough to matter.

SHood, in many ways they are. The camera is just one aspect of a phone, and one of the smaller aspects. It's primary use is as a phone, and it does good at that. The basic waterproofing is nice since it can take a light drop in the water. I know many people who have killed their iPhone due to an accidental dunk in the river or lake. As an over all package, it's a great phone and the camera works well for what it is intended to be used for. If you need anything more IQ wise, there is no substitute for a real camera. I think some people are expecting too much from a phone when it comes to the performance of the camera. That's why cameras like RX100, x10, Coolpix A, Sigma DP's, smaller m4/3's, Nikon 1 etc.. Are taking off because of the ultra portability (compared to a DSLR) along with the performance compared to a phone. The phone has replaced cheap P&S cams for those who don't require great IQ. But most enthusiasts still use a real stand alone camera and will until the phone catches up.

It´s not just the camera, the screen is also bad.Compared to e.g. LG 4xHD it really shows it´s shortcomings despite higher res. (I have both so I know). It is very uncomfortable to hold and you can´t change battery. Only good thing about it is native USB host.... hyped POS.

Starting October 1st, Getty Images will no longer accept images in which the models have been Photoshopped to "look thinner or larger." The change was made due to a French law that requires disclosure of such images.

A court ruling our of Newton, Massachusetts has set an important legal precedent for drone pilots: federal drone laws will now trump local drone regulations in situations where the two are in conflict.

macOS High Sierra came out today, but if you use a Wacom tablet you need to wait a few weeks before you upgrade. According to Wacom, they won't have a compatible driver ready for you until "late October."

Vitec, the company that owns popular accessory maker Manfrotto, has just acquired JOBY and Lowepro for a cool $10.3 million in cash. The acquisition adds JOBY and Lowepro to Vitec's already sizable collection of camera gear brands.

A veteran photojournalist, Rick Wilking secured a spot in the path of totality for the August solar eclipse. While things didn't quite pan out as predicted, an unexpected subject in the sky and a quick reaction made for a once-in-a-lifetime shot.