Moving Portraits

Ira Kemelmacher-Shlizerman describes creating "Face Movies" from numerous still photos of a subject, the topic of "Moving Portraits" from the September 2014 Communications of the ACM (cacm.acm.org/magazines/2014/9/177944).

---BEGIN TRANSCRIPT---

00:00-00:20 VO: A portrait may be worth a thousand words, but what story does a thousand portraits tell? Often, not much of a story at all. For differences among them can make for some jarring transitions. And the sheer volume is overwhelming.

00:20-00:35 VO: Now, researchers have developed a system that automatically selects and sorts portraits based on such factors as head orientation and facial features. Then it creates "Photobios" -- a hypnotic, animated display.

00:35-00:45 TOM: Join guest journalist Teresa Meek as she talks with lead researcher Ira Kemelmacher-Shlizerman about this new system of Moving Portraits.

00:55-01:10 VO: Photo albums have come a long way since the days of paper and glue. One thing that *hasn't* changed is our love of the human face. When put in order, we see a life story.

01:10-01:15 VO: With enough pictures, that story becomes something like a movie — although a rather jerky one.

01:15-01:25 VO: You can smooth it out with morphing, a labor-intensive technique that combines a blur effect with point-to-point warping.

01:25-01:30 VO: But new research shows that you can get impressive results without all that work.

01:30-01:55 IRA K-S: So what we do in our method is that we take lots of photos of the same person, we assume it's the same person... We find similarities in facial expressions all these automatically using our algorithms. And Then we build this huge graph where each node in this graph is a single photo... And the connections between nodes -- edges -- represent the similarities between photos.

01:55-02:05 VO: With enough portraits in the collection, the system can move smoothly between *any* two. It's a lot like morphing, but with one important difference.

02:05-02:35 IRA K-S: Ah, classical morphing is... you're given as input two photos... And then you would try to synthetically create in-between frames to kind of go from one face to another... What we are doing is really different in the way that these in-between frames actually come from the photo collection.... So all the photos in between are real photos. That's the cool part.

02:35-02:45 VO: Dr. Kemelmacher-Shlizerman's team previously used a similar technique to match video frames of two people, allowing one to "puppeteer" the other.

02:45-02:50 VO: Now, her work has gone into Google's photo program Picasa, where it appears as the "Face Movies" feature.

02:50-03:10 IRA K-S: At some point we found out that these movies, face movies, looked really, really really interesting... So Steve Seitz, who was, he's a professor here... spent some time in Google... he showed it to some people, and then they said, Oh! That's really exciting! Let's, um, we must have it in, um, in Picasa!

03:10-03:25 VO: There's more to these face movies than just finding and arranging similar portraits, though. The team's research also documented how the much-used cross-dissolve transition can create a realistic effect of motion.

03:25-03:40 IRA K-S: We've estimated... how close the edges of the face should be in the two photos. And if they're close enough... cross-dissolve will create an effect of motion... We don't physically move one face toward the other... but you would create this effect of motion.

03:40-03:55 VO: Life is a continuous story; we capture only glimpses of it. This work aims to help us relive that story -- a story of a thousand portraits.

04:00-04:05 VO: Find out more in this month's Communications of the ACM, in the Research Highlights article, "Moving Portraits".