Microsoft's New 'Xbox One' Can Measure Heart Rate

To be clear,
Microsoft made no official healthcare announcement today. At least none that I'm aware of. They did, however, announce their latest home entertainment console - the new Xbox One (pictured above) - and that does make for some really interesting speculation on the intersection of home entertainment and home health. That intersection hasn't arrived - but it's definitely coming and it's likely to get here a lot sooner than we realize.

The press event was carefully orchestrated on Microsoft's Redmond campus inside a specially built tent. From that vantage point - I was able to follow the live events onstage as recorded by fellow Forbes Contributor Matt Hickey (h/t and thanks to Matt!). Using the coveritlive.com service (recorded here) Matt provided live text updates throughout the main event.

By all accounts, it was fairly standard stuff - although the anticipation has been building for months. The long awaited announcement. The official unveiling of the new console. Lot's of demos and descriptions around its sleek new capabilities wrapped inside a whole new hardware box. I'm not really a gamer, so there was only a few hints of excitement for our home setup:

* A more seamless switch between games and live TV (ah - much needed)

* Kinect included with every console (ok - nice touch)

* Built-in Blue Ray Disc player (yup - finally)

* 500GB Hard Drive (ok)

* 8-core, X86 Processor (ok - but no specs around GPU?)

* HDMI - In/Out (hmm - 1080P in and out? ok - this is interesting)

* USB 3.0 ports

In a nutshell, all very interesting - but all fairly evolutionary - not revolutionary. That is until about the 10:23am mark. Here's how Matt described it from inside the tent:

10:23: It's more sensitive, can read more data points of articulation [referring to the new Kinect camera]

10:23: The sensor can read your heartbeat

10:23: IT CAN READ YOUR HEARTBEAT

My reaction was the same as Matt's last one - where he used all caps. Say what? I dug into the data sheet (here):

Redesigned Kinect: It includes a 1080p, HD camera that captures video at 30 frames per second. All-new, active-infrared capabilities increase precision, allowing it to work in nearly any lighting condition and expanding field of view to accommodate a greater variety of room sizes. Microsoft proprietary Time-of-Flight technology measures the time it takes individual photons to rebound off you to create unprecedented accuracy and precision. The new noise-isolating multimicrophone array filters ambient sound to recognize natural speaking voices even in crowded rooms.

It even looks a little different than the first Kinect:

I quickly connected with Matt to find out more. He tracked to sources at Microsoft and replied with this update:

He also referenced a link as a possible source of the technology at work. It's called Eulerian Video Modification and from the MIT link (here) comes this abstract:

Our goal is to reveal temporal variations in videos that are difficult or impossible to see with the naked eye and display them in an indicative manner. Our method, which we call Eulerian Video Magnification, takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signal is then amplified to reveal hidden information. Using our method, we are able to visualize the flow of blood as it fills the face and also to amplify and reveal small motions. Our technique can run in real time to show phenomena occurring at temporal frequencies selected by the user.

The four reference frames used to highlight this technique look like this:

In other words, the technology to measure heart rate is entirely optical and not based on physical contact with the controller. Later in the day - Matt confirmed that the technology was developed entirely in-house by Microsoft.

"The Xbox One's technology is all developed in-house." Yusuf Mehdi, Corporate Vice President of Marketing and Strategy, Interactive Entertainment Business (Matt's interview with Yusuf Mehdi here)

Matt also captured this image - which he shared for use here:

I've always been intrigued by the direct connection between home entertainment and home health - because it's such a logical intersection for broader elements of patient-centered healthcare. It's also a much more seamless experience.

Clinical devices (and apps) that are exclusive to major healthcare events are often required for serious medical conditions - but as we move into the low acuity spectrum, we quickly arrive at categories of health like remote diagnostics (for kids and aging at home), chronic disease management and physical therapy. Those are big, exciting opportunities. In that spectrum of healthcare - standalone/dedicated devices aren't likely to be as effective - and the capabilities of an in-home console like the Xbox are pitch-perfect as a natural extension to other aspects of our home life. Including health data capture and remote sharing.

More importantly, the future for consoles as dedicated gaming devices is limited. That was also part of the landscape of quotes that appeared throughout the day from various sources. This is a great example of one filed by The Telegraph (here) :

"Gaming is on the rise: More people will play more games for more minutes on more devices than ever before. But ironically even as gaming rises, the Xbox will only continue its growth by moving beyond gaming. Xbox Live Gold members already spend a majority of console time not with gaming but with video. The opportunity for Microsoft to deepen its customer relationship with this next Xbox is there, as long as it confidently reaches beyond the hardcore gamers Sony is so focused on, making its console more important for everything from gaming to video to shopping to home management." James McQuivey - VP, Principal Analyst at Forrester Research