In our last blog post, we showed that you could read President Obama’s and Mr. Romney’s heart rate right off of their faces during their first presidential debate. We got a lot of questions asking us to explain the technology behind it a little more, and we’re eager to show other ways it can be used. It turns out that this works with all sorts of video, including our favorite Hollywood movies. To demonstrate this technology to you a little further, today we’ll be looking at Robert De Niro during his seminal “You talkin’ to me?” scene in Taxi Driver (1976).

First, how does this work? The basic idea is that the color of your face fluctuates slightly as blood perfusion under the skin changes from your heart’s pumping.If you measure the color change, you can measure heart rate.It sounds simple, but it’s not.One thing that makes this difficult is user motion: if the person being measured moves, it becomes difficult to disentangle cardiac-induced color changes from changes in lighting conditions on the face, rotations of the face, etc. But in some cases, we’re actually able to filter out some of this motion. Pretty cool, right?

Now let’s walk through the scene as a way to show how this technique works. Fig. 1 (below) depicts the motion of Mr. De Niro during the performance.Big spikes indicate lots of motion and flat, low regions indicate periods of relative stillness.We see that the sequence where Mr. De Niro says,“You make the move,”is the most still period.It’s about 15 seconds prior to “You talkin’ to me?”, but it’s a good starting point to try this.

Figure 1. Mr. De Niro undergoes significant motion in the “You talkin’ to me?” scene. The most still period is when he says “You make the move”, which precedes “You talkin’ to me?” by about 15 seconds

There’s still motion during this sequence, so to get rid of it, we’ll do a simple feature tracking on Mr. De Niro’s face, as shown in the movie below.

Now taking that patch of skin and analyzing its color changes, we can derive the cardiac signal for Mr. De Niro and get his heart rate. Fig. 2 shows a nice sequence (below). In this chart, you can see the color intensity swing up and down in a visible pattern that corresponds to specific heart beats. The rest is just arithmetic to figure out the rate at which the heart is beating. Mr. De Niro’s heart rate for this scene:95 bpm! The average heart rate for adults falls more around 72 bpm. Since Mr. De Niro was in his early 30’s at the time of the movie and seems to be in good shape, his heart rate was probably slightly elevated, which isn’t suprising since it was a pretty exciting moment in the movie where he pulls out a gun with a crazy mechanical contraption. Or maybe Mr. De Niro did jumping jacks right before the scene?

Figure 2. Mr. De Niro's cardiac signal during the "You make the move" sequence. The overlaid hearts denote heart beats.

Now that we’ve cut our teeth on the easy part, let’s see if we can find Mr. De Niro’s heart rate during the epic, zeitgeist-defining moment of truth. This movie shows the “You talkin’ to me?” sequence with the feature tracker going to filter out motion.

Notice that there’s significant motion throughout, but there is one sweet spot when things are still. Fig. 3 below shows the corresponding cardiac signal during that interval with beats overlaid. The answer: 79 bpm!

Did Robert De Niro know that he would change popular culture permanently with a single line? …Ultimately, we don’t know. What we do know is that his heart rate was a little elevated, so he was at least probably excited about his work. Check out the scene in its entirety below. Warning: there is some profanity in this clip!

There’s a lot that could be done with this technology in the long run, from tracking the heart rates of our nation’s leaders to tracking your friends’ or your own heart rate during a video chat. It could be really interesting to see how heart rate changes when you’re interrupted by your boss or coworkers at the office. Of course, some care still needs to be taken with video compression, optical and digital filtering, motion, and the like – all of which interferes with the cardiac signal. Nevertheless, it’s still a lot of fun to think about.

Finally, some caveats: this should not be taken as a rigorous scientific work, but more as a fun show-and-tell prototyping project. There may be confounding factors in this analysis that have not been investigated in detail. If you can think of them or have any additional thoughts or comments, we’d enjoy hearing them all.

After last night’s presidential debate, the nagging question seems to be “who won: President Obama or Governor Romney?” It’s a subjective, squishy question that can only be made somewhat quantitative by polling a lot of people on how they felt about the debate. But what if, instead, you could get some objective measurements on the candidates themselves? What if you could get insight into their mental state while they debated? That would be cool… and it’s the object of today’s Fitbit blog post.

You can tell a lot from a person’s face. A person smiles, they’re happy (usually); a person frowns, they’re not. That’s the easy stuff. Believe it or not, you can go much deeper. Did you know that you can actually measure a person’s heart rate from his or her face? Really. And from someone’s heart rate, there are various inferences that we can make: such as if they’re stressed, or even lying.

Once we figured out that this was possible, our first thought was, “Wouldn’t it be cool if we could use this to measure the stress levels of our nation’s leaders? Say, for example, during the next presidential debate?” Last night, under the bright lights of the presidential debate stage, that’s exactly what Fitbit Labs did. Just by looking at their faces.

In the above images, a faint blue box shows the area we’re focusing on for our results. It’s pretty cool: using a regular video camera and some advanced computer algorithms, it’s possible to observe small color changes on someone’s skin as his or her heart pumps. It’s very much like checking for someone’s pulse by touching his wrist, except in this case the “pulses” are color fluctuations in the face coming from blood volume changes under the skin. In order for this technique to work, the person shouldn’t move too much or the motion ends up interfering with the very subtle colors that you’re trying to measure. And forget about fancy camera panning like you see used in movies for dramatic effect.

Of course, “normal” heart rates vary from person to person a lot as well. Luckily, in the US we actually publish our leaders’ physicals for the public to see, so we know what each candidate’s heart rate was starting off.

Table 1: Resting heart rate for Pres. Obama and former Gov. Romney from their most recent medical reports

During Gov. Romney’s opening remarks, he stayed still for a few seconds and we saw that his heart rate was 73 bpm, or a 83% elevation above his resting heart rate of 40 bpm. Not bad. President Obama’s rebuttal a few minutes later came up at 96 bpm, or a 43% elevation above his resting heart rate of 67 bpm. Even though Romney’s heart rate was lower in the beginning, compared to his resting heart rate he was more worked up right from the start.

Now here’s a breakdown of the results across the debate:

Romney’s opening argument: 73 bpm (+83%)

Obama’s rebuttal: 96 bpm (+43%)

Obama’s closing speech: 59 bpm (-12%)

Romney’s closing speech: 75 bpm (+88%)

Or, maybe it’s easier to see as a timeline across the debate from opening to close:

Figure 2: Heart rate elevation for Pres. Obama and former Gov. Romney through the debate

What does this mean? Let’s walk through the debate.

There’s actually a number of different inferences you can make from the data we found. A person’s heart rate naturally accelerates during times of stress — such as a crucial debate being televised to millions of people that decide who will run the entire country. We could look at an elevated heart rate as a measure of nervousness. In that case, we would infer that President Obama was more nervous towards the beginning of the debate, but proved to be one cool cat as the pressure mounted towards the end. In this line of reasoning, Romney, was steadily nervous throughout – and to a higher degree than Obama was. But in this case, we’re making the assumption that the elevated heart rate is related to stress, when that’s not necessarily the case. What if, instead, Romney’s heart rate is a reflection of how amped up he is about his ideas? This could just mean that Romney was really “in the zone” for the debate, while Obama failed to bring a certain excitement for the task at hand. This was, in fact, a popular criticism of Obama’s performance.

So who DID win the debate? Well, if you want the candidate who kept his cool, it was Obama. If you want the one who was most excited, it was Romney. Maybe before the next debate Obama should drink some caffeine to get that energy up. It will be interesting to see how the next debate compares, now that the pressure is really on to be more energetic.

This also makes us here at Fitbit wonder if this sort of technology might have a place in debates in the future. Would a candidate ever agree to be hooked up to an EKG during the debate? Or at the very least, could such data be used to train for debates? We can’t wait to find out.

Disclaimers:

This is just our best guess at the candidates’ heart rates during the debate. There are a lot of reasons to take these with a grain of salt, but we’d be thrilled to see if others could reproduce or refute our analysis. Of the things to consider:
1. The heart rate measurements were taken during small clips when there’s no movement. No movement is necessary for accuracy, but ideally we’d have used longer clips (~15 seconds, at least).
2. There’s some natural variation in heart rates as well, especially when a person is calm, and “heart rate” is usually defined as the average heart rate over some time. Since we’re using short clips, we might not have seen enough heart beats to make an accurate measurement.
3. Lighting can change on stage and that would affect the results.
4. In all likelihood, both candidates are wearing makeup for the debate. Since we’re measuring changes in skin coloring, makeup can mask some of this.
5. The videos we used were compressed, and of lower quality. A higher definition video would likely be more accurate.
6. Heart rate alone is not the best measure of stress. Measures of heart rate variability, skin conductance, etc. used together are much more reliable.

Meet Zip: the latest addition to the Fitbit Tracker line! In developing the Zip, we reexamined every aspect of what makes our trackers great – from how the clip would work, to the shape of the device — but in the whole design process, the one feature that we kept coming back to was: how do we make a tracker that doesn’t have a button so that it’s more water resistant, but still has a display that you can interact with and is fun to use?

This inspired several lines of investigation on motion-based device interaction. We tried different gestures, from flicks of the wrist to turning the device upside down. “It was interesting to see that men and women naturally prefer different motion interactions,” observed Dr. Shelten Yuen, our Chief Scientist. “Guys flick one way – towards their bodies – but women flick in the opposite way. And most of the gestures were either too hard to learn or not very fun to do. The one universal, easy, and enjoyable motion was the tap,” a simple motion that most of us are already used to from using smartphones or tablets.

Making it even trickier, though, we wanted to leverage the motion sensors inside the tracker to detect taps but NOT add any false steps or false calories at the same time (because they use the same motion sensors!). We did a lot of research on how people tap the Zip: how hard, how soft, where, tapping while walking, tapping while running, tapping standing still… everyone at Fitbit could be found testing out the tap feature at one time or another. We even went to the more extreme motions of dropping the Zip, or throwing it against a table. This allowed us to build some very sophisticated pattern recognition algorithms to accurately detect taps and steps without fear of them being mistaken one for the other.

It took some work, but it was worth it so that you can enjoy your Zip, and have fun with the new tap interface while also resting well in the knowledge that your Zip is sweat, rain, and water-resistant. You can learn more, or buy a Zip online now. Let us know what you think by heading to our Facebook page!

I had never consistently used a scale before getting my Fitbit Aria Wi-Fi Smart Scale. I’ve always been one who measures my weight based on how well my pants fit. Although, I have occasionally told myself that my pants just shrunk. So I was surprised, after using my Aria Scale for a few weeks, how much I love this scale — and I think I like it so much especially because I’m not a weight watcher.

With other scales, I never really knew what my weight number meant. I could never remember what my weight was last week, yet alone the month before. And more importantly, I never knew what was driving my weight. And if I saw any weight decrease (or increase), was it from fat? With Aria, I can see everything. It’s easy, and dare I say it… it’s even fun.

Maybe calling it fun is going a little too far, but I can’t deny that I get a little excited about checking out my weight and % body fat charts online, or on the Fitbit mobile apps. And since I log food and use a Fitbit Tracker, I can see how my active (or inactive) day and my food consumption affects my weight. We all understand the basics of how food consumption and lack of activity really packs on the weight. With Aria, you can actually see the effect of food and activity on your weight, through the data it tracks.

So I wasn’t much of a scale person, but being able to view my health as simple, measurable data is priceless. Let us know what you think about the Aria Scale by going on Facebook’s Fitbit Fan page or by emailing us at welovefitbit@fitbit.com. Or, you can learn more about the Aria Scale by going to http://www.fitbit.com/product/aria.

We at Fitbit love when we find unique, thoughtful blog posts about our products. The below is one such blog about the power of Fitbit badges, originally posted on Louis Gray’s personal blog. Louis Gray is a Google employee, technology blogger, and dad. We came across his blog by accident, but loved it so much that we wanted to share it with everyone. You can read more of Louis Gray’s blog at louisgray.com.

I am not going on a diet – and I have no interest in going to the gym, even if Google makes it incredibly easy to eat healthy on campus, and gym membership is free with equipment abundant. It’s just not me. But despite this clear disinterest in my making any kind of physical life change, I have been wearing a Fitbit the last week, obsessively counting my steps, climbing the stairs and tracking how many miles I make on foot. I’ve even been wearing the lightweight tracker at night to see how long and how well I sleep – working to optimize that as well.

So why would I resort to such silliness? It’s the stinkin’ badges – helped along by casual competition with friends, and now, despite my best attempts to not make any actual alterations to how I behave, I am sure I am doing things that are actually better for me, in the same way that Foursquare recommendations have pushed me to new venues and trying new things, based on badges and recommendations from friends.

Yesterday's Fitbit activity shows average walking, and lots of climbing.

The Fitbit itself is not entirely new – having debuted in late 2008, and so far, I’ve been uninterested. I recognize that my mostly sedentary activity of holding down a desk, and chasing after my kids being my main form of exercise would not be particularly interesting. Even now, while I managed 10,000+ steps and 50 flights of stairs yesterday, I still managed to scarf down a great bacon and cheddar sandwich for lunch, so weight loss is not the target.

After scads of occasional tweets and other status updates from acquaintances updating me on the minutiae of their daily fitness activity, it took a simple email of a friend’s weekly dashboard last week to recognize this was a device I needed. In minutes, I’d not only purchased the $99 Fitbit Ultra tracker, but also pre-ordered the Wifi-capable Aria scale for another $129. It was the stats, and the idea of competition, that made me knew I had to get it.

A day's activity, showing spikes of walking across campus and at home.

Like a true geek, I’m understandably curious about the Fitbit’s accuracy. Does it count 5 steps as a flight of stairs? What about 10? What about small steps, big steps? Do I get credit for manually shaking the tracker or running in place? But despite my moments of tinkering, I’ve found the tracker’s daily reports to be especially accurate. I can spot when I walked to and from my car, to and from lunch, and even when I went from building to building for meetings. I can see when I chased my kids around the backyard, and by looking at the sleep tracker, get a good idea for when they started yowling in the morning, begging to get up.

Gaining one’s first badges, such as 5,000 or 10,000 steps, or 10 flights of stairs traversed, is pretty straight forward. But I wanted more. When I got home and put the kids to bed, I was at a mere 14 flights of stairs, so I literally, alone in a quiet house, went up and down my 15 stairs at home 11 times, to get to 25 flights. It must have been quite the sight. That got me a 25 flights badge, and later, when I interrupted each chore with 5 more flights, I finally made it to 50 flights of stairs, which earned me a new badge, not to mention a little bit of sweat and some tightness in my calves, which said the exercise might actually have been working. Tricked again!

A badge for 50 flights is one thing. What about 100 flights?

Had it not been for the allure of the 50 floors badge, there’s no question I wouldn’t have been hiking up and down in my house in some solitary unfulfilling challenge. Had it not been for the intrigue of comparing my daily steps accumulated against my friends, and seeing if I could walk more steps than the previous day, or sleep more efficiently one night versus the previous night, I wouldn’t be thinking about it at all. Once the scale arrives and threatens to send my weight to my own internal profile, I wonder if it too is going to impact how I eat, measure and commit to something that resembles good behavior.

As for the Fitbit itself, I can’t complain at all. It’s very light, inconspicuous, and the software is practically invisible. Just connect to the computer, hit sync, and it’s good to go. I’m now addicted to these stats, like any blogger chasing page views, or your favorite fantasy football fan whose future hangs on every rushing yard. The badges are driving the behavior. So if you have a Fitbit and want to challenge my stats, invite me by email. Let’s do this.

This blog was reposted with permission and all content remains the property of louisgray.com.

Our free Fitbit app for Android has arrived! Of all the digital fitness device companies out there, we’re happy to announce that we’re among the first to offer both an iPhone and Android app. You can download the Fitbit app for Android for free from Google Play.

When we released our iPhone app last October and saw the warm reception it received, we started working on our Android app. We made a point to design our app specifically for Android phones, rather than just try to make the existing app work on Android. We hope you’ll be happy with the results.

With this app, you can log your food, activities, weight and water while on the go, as long as you have a wireless connection. Everything syncs to (and from) your Fitbit.com account, and you can view up to two weeks of data for easy access when you’re away from your computer.

The updated Food Plan is also on the Android app, so that you have all the information you may need to make smart decisions about your calorie intake throughout the day; to learn more, check our earlier blog entry. For our iPhone users, the updated Food Plan will be available with our next iPhone app update at the end of the month. We’ll be adding new features and updates to both apps, so keep your feedback coming and make sure you keep your Fitbit app up to date.

And a special thank you to all our Android users for your patience! The app is only available in the US, Canada, and the UK right now, but will be expanded to further countries as we expand our direct international sales. Happy tracking and logging — wherever you go!

After our release of Food Goal, we got a lot of great feedback and suggestions from our Fitbit fans about it. We’re excited to announce our answer to those suggestions with a large update to Food Goal. In fact, we’ve changed so much that we completely renamed it to Food Plan.

What’s changed? Quite a lot, actually, though the Food Plan has kept the same general look and feel as it had before. Everything that was in Food Goal is still available, like the gauge to show whether or not you’re in the “calorie zone”. But now, we’re providing a lot more information right at your fingertips, and the Food Plan has become even better at helping you manage your calorie intake each day.

Smarter calorie estimates: Your Food Plan sets each day based on your historic activity levels. It will calculate how many calories you burn each day on average, and base your suggested calorie intake on this. As you wear and sync your Tracker throughout the day, your Food Plan adjusts dynamically based on whether you’re exercising more or less than that average, so that your goal remains as accurate as possible. And as your activity trends change over time, your Food Plan’s starting estimate will adjust to your new activity levels as well.

Calories in vs calories out: Additionally, many of you wanted a way to judge how well you were doing at any one point of the day. You can now view your total calories burned compared to your total ca lories eaten, and see whether or not you are currently within your plan’s deficit goal zone. So if you’ve just logged your lunch and wonder how you’re doing for the day so far, you can see how your total calories burned so far for the day measure up against the total calories you’ve eaten. There’s no more guessing if you’re behind or on track so far for the day. With this meter, it’s always clear where you stand.

Tips and Tricks: To get the most out of your Food Plan, make sure to log your food throughout the day. You can do this on the go through your smartphone on our apps, or by visiting our mobile website.

Overall, we’ve tried to make changes that provide you with the information you need to make smart, healthy decisions and be better prepared to achieve your goals. We hope you enjoy the Food Plan, and welcome any feedback you might have. “Like” us on Facebook to join the conversation there.

The last holiday cookie has been munched. The feasting is almost done. Now that the holiday season is coming to an end (and 2012 is around the corner), everyone seems to be making fitness or getting into shape a New Year’s Resolution… mostly because we‘ve been binging over the holidays and need to get rid of those extra pounds.

Here at Fitbit we want you to put fitness on your Resolution list, but for a very different reason: because it leads to amazing health benefits — like mitigating diabetes and Alzheimers… and even living longer!! Just dedicating a small amount of time each day to light exercise can garner all these amazing health benefits. And it’s fun too.

In a very entertaining and motivating video, Dr. Mike Evans walks through the scientific research that shows how simple exercise — like walking just 30 minutes a day (every day) — can be the single most important thing you can do for your health. Check it out. It will get you to put moving on your resolution list, and keep it too!

It seems simple — 30 minutes a day — or for Fitbitters, we talk about hitting 10,000 steps a day. If fitness is on your 2012 New Year’s Resolution, let us know what you are aiming to do each and every day. We are aiming to do 10,000 steps 100% of the days in January. And we’ll report back our status at the end of January. Let us know at welovefitbit@fitbit.com if you want to join our resolution or “Like” us on Facebook to tell us there.

Studies have shown that working out with a friend can help you meet your fitness goals. Adding a friend to your work out, even remotely, can provide you with motivation and incentive to keep your fitness plans on track.

At Fitbit.com, you can share your stats across your social network. Integrations with popular apps and websites, including Facebook, allow you to share your achievements with your friends, and you can share as much (or as little) as you want with your Fitbit friends. If you’re Fitbit friends with someone, you also have access to leaderboards that show everyone’s averages by week and can compare how you’re doing in relation to your friends.

Now that we’re in the middle of the holiday season, having that extra support to get you through all of the obstacles is even more important. During the holidays we are so much more at risk to fall off the fitness wagon. With meals that can add up to over 4,000 calories in one sitting, the need for fitness friends during this time is so much higher.

Fitbit Trackers also make great holiday gifts for anyone on your list looking to live healthy for the new year, but you don’t need to wait to get your fitness friends together. Find your motivation over the holidays with friends, and stay healthy all year long!

Order now to make sure your package arrives in time. “Like” us on Facebook or email us at welovefitbit@fitbit.com to let us know how you stick to your fitness goals over the holidays.

This past Saturday morning, I found myself on the road going for a run. Actually it was more of a walk/run. Or maybe a walk/run/walk. I’ve always been big proponent of fitness, but before working at Fitbit I had found myself in a little downward spiral of inactivity. I would read articles about getting fit and talk about fitness trends and studies for hours with friends, but at the same time I would tell myself to postpone working out for another day, or that I needed to rest and relax from a tough work week — and I slowly became less fit (and chubbier).

Here at the Fitbit Offices, it’s hard to avoid being motivated to be active every day. In fact, this past weekend I got a big dose of motivation in the shape of a contest our CEO posed to a bunch of us late Friday night: walk 20,000 steps in one day that weekend. The consequences of the bet were quickly flushed out. There weren’t rewards for the highest steps; it was simply whoever didn’t hit 20,000 steps on Saturday or Sunday had to write an email to the rest of the company, declaring their shame in not living up to Fitbit love. We discussed quickly the exceptions to the bet (what if we broke our legs?); in the end, we all agreed.

So there I was on Saturday morning, walking/running/walking. And there I was, on Saturday afternoon, walking to the grocery store. And there I was Saturday evening, hitting all the errands I was postponing to get those steps. And at my friend’s dinner, I was very eager to jump up and help out just to get a few more steps. We were all texting each other our progress and it came right down to the wire. But I made it! Whew.

Apparently the CEO was walking around late Saturday night to get the last 2,000 steps before midnight, much to the amusement of his wife. A bet is a bet.

Have you guys ever participated in a stepping contest? Let us know all about it by emailing us at welovefitbit.com.