DxOMark Adds Selfie Benchmark to Its Phone Scores

This site may earn affiliate commissions from the links on this page. Terms of use.

Love them or hate them, selfies are here to stay. Phones are by far the most popular cameras today, and selfies are a rapidly growing use of them. So it makes sense that consumers are increasingly choosing their next phone based partially on its ability to capture selfies.

Phone makers have been paying attention. They’ve introduced dual-camera front-facing camera setups for better portrait effects and wide-angle selfie cameras for group selfies. They’ve also bumped up front-facing camera quality and resolution. But until now, there hasn’t been an industry-wide published benchmark for selfie quality. DxOMark Image Labs (most famous for dxomark.com) is aiming to change that with the release of its DxOMark Selfie test suite, along with initial benchmark results for several representative phones.

Rise Of the Selfie

Selfies got their start in 1839, when chemist Robert Cornelius took a photo of himself, at some considerable investment of time and expense. Andy Warhol later took what might be some of the most famous selfies, starting in the 1950s. But it wasn’t until Apple added a front-facing camera to the iPhone in 2010 that the selfie began to take off.

The ability to quickly post to social media added fuel to the fire. The current number of selfies captured is in the billions. By 2014, Google estimated that Android phones alone were used to capture 93 million selfies each day. By 2015, over 25 billion had been uploaded to Google Photos. Samsung estimates the average phone user will take 25,000 selfies over the course of their lifetime.

Selfies Are Harder Than They Look

A panel of models are part of the testing so that skin tones from various races are included

Unfortunately, simply pointing a camera at your face from a foot or two away doesn’t automatically yield a great picture. The cheap front-facing cameras on early smartphones didn’t produce quality images in the first place, and the combination of short distance to the subject and a relatively wide-angle lens distorts faces and bodies in the foreground. Lighting on the subject of the selfie can also be dramatically different from that on the background, making it hard for the camera to pick an exposure that will make the subject look natural and still work for the background.

Ideally, the camera would also know whether to try to blur the background — as most rear cameras now do in portrait mode — or make it as sharp as possible, in the case where the photographer wants to show the subject or group of subjects in a scene.

To address these issues, phone makers have upped their game in both hardware and software. Front-facing cameras are now based on better sensors, often helped out with additional methods of estimating distance. Some companies like Google have opted for wider-angle front cameras to make better group selfies possible, while others have doubled up on their front cameras to provide better Portrait images. HDR mode and multi-frame processing have also migrated from the main to the front camera on many newer phones. Going beyond features typically found in the main camera, most phone makers have also started to add automatic “beautification” capability to their selfie cameras.

With the increasing importance of selfies to both phone owners and phone makers, it makes sense for DxOMark to extend its phone camera test suite to include testing and scoring of a phone’s front-facing camera. Hence, DxOMark Selfie.

This simple comparison shows how differently selfie cameras can capture the same scene. There are variations in color rendering, skin tone, and skin detail between these three phones.

Challenges in Testing Selfie Quality

At its core, measuring image quality for a front-facing camera includes the same basic aspects as for the main, rear-facing, camera. So DxOMark Selfie includes almost the same scoring categories as DxOMark Mobile. For Photo, that’s Exposure, Color, Focus, Texture, Noise, Artifacts, Flash, and Bokeh. Only Zoom is left out. For Video, Exposure, Color, Focus, Texture, Noise, Artifacts, and Stabilization are measured — identical to the categories for main cameras. The testing also follows the same overall structure, including over 1,500 images captured in the lab and outdoors, both tripod-mounted and handheld, along with dozens of sample video clips. But the specifics of testing selfies posed some unique challenges, including short working distances and skin tone variety, that required novel solutions.

DxOMark Selfie includes images in lighting ranging from 1 lux to many thousands of lux.

By definition, selfies include at least one person close to the camera. So for starters, the selfie test protocols focus on the close-range portrait performance of the front-facing camera. DxOMark Selfie includes tests at three distances: 30 cm, 55 cm, and 120 cm, to allow for various ways of holding the phone and the possible use of a selfie stick. You can see the use of this range of distances in these examples:

Selfies captured with subject distances ranging from about a foot out to several feet.

Skin tone rendering is also crucial for selfies, particularly since a light or dark skin tone might require different exposure settings. Group selfies may include a range of skin tones, making the camera’s choice of exposure even more complex. DxOMark has chosen to test phones with the international version of their firmware and default settings. But the company also had to come up with some unique test equipment to make its tests both accurate and repeatable.

First are a couple of purpose-built test charts for measuring bokeh (background blur) as well as one for measuring noise and detail. Slightly more exciting is the HDR portrait setup DxOMark created to allow it to evaluate image capture in backlit scenes — typical in casual selfies.

The most challenging innovation was the creation of a pair of mannequins, in conjunction with a Hollywood special effects firm, that accurately reproduce the spectral response of various human skin tones, while allowing for repeating test results without the issues associated with trying to have human subjects look, act, and pose identically for multiple day tests.

Benchmarking Beauty

An even harder challenge is evaluating images captured by phones that use automatic beautification techniques to smooth skin, alter skin tones, or even change the shape of facial features. DxOMark Selfie takes a conservative approach to this evaluation by only penalizing a phone’s score for causing unintended artifacts when applying beautification. There is no penalty for a phone not attempting beautification. Given the differing notions of beauty found in various cultures and social groups around the world, this method of scoring avoids having to make subjective value judgments about which standards to apply.

DxOMark Selfie Complements DxOMark Mobile

DxOMark Selfie scores are completely independent of, and can’t directly be compared with, DxOMark Mobile scores for the main camera. Starting now, DxOMark is releasing initial Selfie scores for some phones and will continue to score new models on DxOMark.com, the same way it does front-facing cameras.

UPDATE: DxOMark Image Labs has now released its initial Selfie scores:

Disclosure: The author wrote some of the material DxOMark Image Labs is using to explain and promote DxOMark Selfie.

Tagged In

This site may earn affiliate commissions from the links on this page. Terms of use.

ExtremeTech Newsletter

Subscribe Today to get the latest ExtremeTech news delivered right to your inbox.

Email

This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our
Terms of Use and
Privacy Policy. You may unsubscribe from the newsletter at any time.