Advertisement

Advertisement

New Scientist Live

Skin care gets smart with AI

Choosing the cream that best matches your skin is a tricky business. Now Olay has trained a deep learning algorithm to study your face and help you make the best decision

Get smart with your skin care

When it comes to choice, more is not always better. The all too familiar conundrum of choosing between dozens of chocolate biscuits or hundreds of toothpastes can sometimes induce a kind of decision paralysis, a phenomenon that psychologists call the paradox of choice.

Skin care conundrum

It’s a problem often seen in the confused faces of customers surrounded by hundreds of products in the skin care section of department stores.

A 2013 survey by consumer product giant Procter & Gamble found one third of women were unable to find what they were looking for in facial skin care aisles. Almost two-thirds say they have unused facial products at home.

“There’s been an explosion of skin care brands and products in the past 10 years or so,” says Dr Frauke Neuser, principal scientist for P&G brand Olay. “One result is that women are shopping in places where they can get advice. However, that can be intimidating for those who might not want to buy a £150 skin cream recommended for them.”

Advertisement

P&G says it has the solution. Its web-based Olay Skin Advisor analyses make-up-free selfies uploaded by users to estimate skin age and make personalised product suggestions. It is believed to be the first artificial intelligence-based skin care advice tool.

It’s the result of decades of endeavour. P&G scientists have been studying skin for over 60 years and developing improved image capture and analysis technology for almost three decades (see “A history in imaging“). That includes a 2015 clinical study comparing the facial skin of 330 women of different ethnicities. These included a subset judged to look at least 10 years younger than they actually are and who proved to have a common pattern of gene expression.

Inspired by this study, P&G scientists wondered whether artificial intelligence could replace the subjective human judgements of age with something more objective.

The idea of machines that can perceive the world, learn from examples and understand humans has been around since the 1950s. It is however only in recent years that major strides in building such machines have been made thanks to key advances.

These include the rapid growth in large datasets that include images and the ability to store such vast amounts of data. Also important has been a shift to using computer chips known as graphics processing units (GPUs) originally developed for video gaming. These offer enhanced power over normal chips for certain tasks. “There’s a natural fit between the workloads of deep learning models and gaming. Both involve doing very large batches of computations, and doing so in parallel,” says Thomas Bradley, Director of Developer Technology at NVIDIA, which created the GPUs used to train and run the Olay Skin Advisor deep learning algorithm.

That algorithm, the result of experimenting with image recognition software, is the outcome. It was developed by the P&G bioinformatics group with artificial neural networks, computer simulations inspired by the way human brains process information. They allow so-called deep learning by a machine.

Deep learning models

“Human brains learn to estimate people’s ages by building a model that associates features we see in faces with the ages we know those people are,” says Dr Jun Xu, a P&G computational biologist and lead developer of the Olay Skin Advisor’s VizID™ algorithm. “Deep learning mimics this process.”

Deep learning models are made up of hierarchies of filters called “layers” designed to detect the presence or absence of features.

Each pixel in an image has red, green and blue colour values. By analysing these, a layer can, for example, look for the edge of a feature or the edge of a colour patch. It then passes its finding on to the next layer, which might look at what shapes are formed by those edges. Early layers look at smaller areas and later ones assess larger ones.

By going through 20 such layers, VizID™ builds skin age predictions. By comparing these repeatedly with known real ages, it can tweak the weightings it gives to different elements to improve its accuracy. Olay researchers used 50,000 selfies of women of different ages and ethnicities to “train” their model. This included feeding these images back through the algorithm 250 more times, each time with a slight change to lighting, resolution or positioning, for example. This ensures the model learns to predict the right age despite these image variations (a technique called data augmentation).

“Deep learning mimics the way the human brain learns to estimate people’s ages”

Once trained, the model was tested with a new set of 630 selfies of women aged 18-65. The Olay Skin Advisor’s average age difference was 3.8 years. This compared to an average of 7.3 years for a group of dermatologists.

By retracing the model’s calculations, the Olay team found it assigned most weight to eye wrinkles, pigmented age spots and “laughter lines” running between the corners of the mouth and the edge of the nose. These are among key features that humans also use to estimate age.

Launched last year, Olay Skin Advisor gives users an estimate of their skin age and tells them which of their forehead, cheeks, mouth, and eye areas are in best shape, as well as which could benefit most from improvement. Based on this and questionnaire answers, it gives personalised product recommendations.

More than 1 million women have already used the web-based tool. The images they uploaded are helping further improve its accuracy. A survey of users carried out by P&G found 94 per cent agreed that the products recommended by the algorithm were right for them and 88 per cent were still using the same products four weeks later.

“We’re not doing this because deep learning is cool,” says Neuser. “Ultimately the proof is in the pudding. If women buy the recommended products, see results, and keep using those products, then we can say that Olay Skin Advisor is a success.”

A history in imaging

THE use of deep learning in the Olay Skin Advisor may seem bold, but it is no leap in the dark. In fact P&G has been breaking new ground in skin imaging and analysis for 30 years.

These advances were initially designed to assist product development, before later being used in devices that assess skin age by analysing features such as texture, tone and radiance in images.

During the 1990s, for example, it developed software to correct colour variations caused by changes in lighting and camera response by comparing them with reference colours incorporated into images. It also standardised facial positioning using “ghost images” from earlier capture sessions to precisely line subjects up in later ones.

In 2001, P&G licensed its imaging know how to Canfield Scientific as the basis for its in-store, feature-based skin analysis devices, and later to create VISIA, a similar system for medical professionals.

In 2006, P&G automated the assessment of the molecular basis of skin colour using digital cameras and custom software to create maps of melanin and haemoglobin.

“Adopting deep learning algorithms hasn’t been as difficult as it might have been, thanks to Olay’s rich heritage in image acquisition and analysis,” says Dr Paul Matts, Olay Research Fellow.