Automatic age progression: Ars staffers stare down the Grim Reaper

App takes baby photos and projects how they'll age. In our case, not so well.

Our once-cute author, seen here at the ages of 3 and "roughly 60," according to a University of Washington age-progression application. Still holding on to those baby teeth, apparently!

For years, the “have you seen this child?” part of missing persons’ reports has been the most difficult one. Not just emotionally but literally. How can someone be recognized based on a childhood photo—often a low-detail one, at that—along with an awkward-looking artist's estimate of their current appearance?

Ira Kemelmacher-Shlizerman, an assistant computer science professor at the University of Washington, stumbled upon this challenge after roughly a decade of research in similar fields. Kemelmacher-Shlizerman and her colleagues have worked for years to accurately re-render photos, mostly in converting 2D faces to 3D models.

The results of her teams’ studies thus far have been intriguing—and even hilarious. In 2010, for example, her team made it possible to be John Malkovich, translating live facial movement onto another face on a screen. Her most recent project was a little different, as it began with a nudge from the National Center for Missing and Exploited Children to find out what more could be done with so much facial data.

“They were saying how challenging it is to create [age-progressed] images, particularly when they are limited to stock images given by families,” Kemelmacher-Shlizerman said. “The lighting could be different between photos, as could the facial pose, and those issues are particularly challenging for photos of kids.”

Enlarge/ When Kemelmacher-Shlizerman's application gets things right, it can transform a baby photo into an older one with nothing more than smart analysis of thousands of photos.

Her effort to improve matters, detailed in a paper that will be presented at June’s Computer Vision and Pattern Recognition conference, has been dubbed Illumination-Aware Age Progression. Its aim is to create a realistic aging estimate with as little source material as a single, low-res photo of a child.

The application her team developed will soon be released for public use. Anybody can choose a photo, preferably a young, front-facing one, and the app will automatically find the eyes, nose, and other facial points—no user input necessary—and create a 3D model that is auto-aged to estimate anything from years to decades.

Rather than wait for the app’s launch, we reached out to its creators and asked if we could cut in line to stare the Grim Reaper dead in the eyes. Give us your baby photos, the team responded, and they’d hit the fast-forward button to the age of “roughly 60.”

Nate Anderson, from 4 to "roughly 60."

Nate Anderson, from 4 to "roughly 60."

Nate Anderson, overalls intact.

Modern Nate.

Jonathan Gitlin, from 3 to "roughly 60."

Jonathan Gitlin, overalls intact as well!

Modern Jonathan. Not nearly as squinty!

Casey Johnston, from 2 to "roughly 60."

Casey Johnston, noticeably missing overalls.

Modern Casey.

The only input Kemelmacher’s team received was a set of childhood photos, one per Ars staffer, and the results in our case were… a little weird. The main issue is that our age-estimate shots sit side-by-side with our childhood photos, meaning we can see the system’s current failings, particularly in handling teeth, wrinkles, and hair color.

Kemelmacher-Shlizerman admitted that her application relies on a purely mathematical model as opposed to drawing biological data or ethnic differences, because her first goal was to create a fully automated system with higher accuracy than any other similar, untouched methods. At the very least, that goal has been achieved.

Her team gathered more than 40,000 photos from Google image searches, broken down into about 1,500 per age range, in order to create average face templates at any stage in life, with only gender as a differentiator. The team also used its prior work on 3D transformations of 2D images to ensure that all averaged faces were perfectly aligned.

Merely creating an average wasn’t enough for realistic age-progression generation, however, so the team created relightable average images, which were crucial to emphasize shape changes like eye narrowing, forehead sloping, and nose-size increase. With that data in hand, the application can automatically apply those changes along with textural changes (wrinkles and lines, for example) to a photo of a child. Impressively, it does this without even converting the original child’s photo to 3D.

“It’s not perfect,” Kemelmacher-Shlizerman said. Without elements like cranial-facial research and hair-color modeling, results will suffer compared to any real sets of photos that catalog a single person from birth to age 80. In particular, wrinkle modeling has proven most difficult because “wrinkles are in different places,” which means they can get lost in a spread of thousands of photos.

Once I got the results back, I figured I’d turn to the person who has watched me age the most over the years: my mother. “They took that pixie nose of yours and kept too much of it,” she complained about the results, before proceeding to remind me how “handsome” my nose and chin turned out (aw, geez). That’s probably reason enough to toss her opinion on account of bias.

What about some unbiased opinions? The study put people to the test by handing them real baby photos, then a mix of age-progression estimates and actual, aged photos, and asked them which shots were the real ones. The results were split with 44 percent favoring the real ones and 37 percent estimated, prompting the paper to note that there are “limits of human evaluation for assessing age progression results.”

Sam Machkovech's 30-year-old photo put next to his age-progressed estimate of 30.

My photo also came with a “30-year-old,” mid-level estimate, which I thankfully had a shaven-beard shot to compare to (in a tux, no less!). Pardon my attempt at “human evaluation,” but I believe it’s a decent estimate for how I turned out. The nose and eyebrows are just about spot-on, but the cheeks are still a little baby-ish, and the chin appears to have been left untouched.

For the sake of our morbid curiosity, this app doesn’t quite cut it. Its ability to automatically render an estimate that doesn’t look hideous is remarkable, but without more biological data modeling, issues with late-life estimates, like teeth and wrinkling, will continue to raise eyebrows as big as the uncanny valley. (Kemelmacher-Shlizerman admitted with a laugh that the public and press have mostly requested that she render Prince William and Kate Middleton’s child. She has, thus far, declined to do so. Sorry, The Sun!)

Still, widespread deployment will help more families use this app for its intended purpose: shorter-term age-progression estimates that can be used to find missing persons. That’s more than successful enough for this UW team.

“You can't just Photoshop a child’s photo to create an aged version,” Kemelmacher-Shlizerman said. “And now we can analyze photos taken in uncalibrated conditions, even with shadows and glasses. It’s all automatic, and for the people who need this, that’s the important thing right now.”

Very clever, but the software doesn't do well with teeth - permanent dentition looks very different to childhood dentition, the teeth are larger, usually more closely spaced and significantly change the facial appearance.. There should be a dramatic change in the appearance of the teeth between ages 6 and 13.

I wonder why they didn't do a more rigorous test: 1) Age up some old baby/kid photos of subjects for whom we have a current picture2) Throw in the real current pictures into a huge database of pictures3) Use facial recognition software to try to match the aged-up photos to the "real" photo4) Report error rates

What they actually tested, asking people to pick between the real and aged-up photo, seems beside the point.

I agree that it is not a great match, but the main issue is whether it is better than what they are currently doing. If it is quicker, better or more reliable than what they have been doing, then this is a win. In later versions they can add some of the features that were discussed in the article. We shouldn't be comparing this against processes driven by experts - this is an automated process acting on a small set of input information.

I don't get it, did that moderated post try and claim Casey is or isn't a fox?

Scientifically speaking, her appearance seems to conform to the norms that most heterosexual males of Homo sapiens sapiens find preferable based on sexual selection. The post was probably moderated because comments on staff appearance are somewhat tactless even if they are true.

And, on a psychological level, she probably knows that and doesn't need random forum comments to remind her. Such comments might even qualify as "creepy".

I don't get it, did that moderated post try and claim Casey is or isn't a fox?

Scientifically speaking, her appearance seems to conform to the norms that most heterosexual males of Homo sapiens sapiens find preferable based on sexual selection. The post was probably moderated because comments on staff appearance are somewhat tactless even if they are true.

And, on a psychological level, she probably knows that and doesn't need random forum comments to remind her. Such comments might even qualify as "creepy".

I don't think there's much chance you could identify Casey from her age-progressed photo.

Sounds like the most accurate picture they could make is an age progression of your wallet with money sticking out and the "roughly 10 minutes later" picture showing none.

Pretty funny results.

Nate had the unfortunate accident that must have stretched out his whole face, Sam's eyes have migrated closer somehow, Jonathan's eyes are apparently much more sensitive to the sun, and Casey seems more cynical with a grumpier brow (maybe that's accurate).

Well you guys certainly aged well, haha. Those computer-aged people are horrendously ugly. It's probably because we are remarkably good at detecting inconsistencies, and an algorithm isn't too good at avoiding them...