What’s the best way to measure attention online? Upworthy thinks it has the answer

For a medium in which every aspect of a user’s behavior can be tracked down to the pixel, from their mouse activity to what browser they are using, it’s still incredibly hard to arrive at a measure of reader attention that works. Analytics companies from Google to comScore each have their own metrics, but they are all flawed. Now Upworthy, the site that specializes in viral content, says it has come up with a comprehensive way to measure attention, and it wants to share it.

As the company points out in a blog post announcing its new metric, measurements of reader engagement or attention are all over the map: some sites continue to use raw pageviews, although that is almost entirely discredited as a way of measuring value, while others focus on unique visitors. Then there’s the amount of time spent on a page, and other custom measurements.

Advertisement

Chartbeat, for example, shows how far down a page a reader got before they decided to click away to somewhere else (a statistic that is usually quite depressing for the author). Other services show what they call “heat maps,” that track eyeballs across the page and show where they spent the most time, something advertisers often want to see. But everyone has a different toolbox and in many cases the numbers don’t add up.

“Shares per piece of content are quite a valuable signal, but they don’t get you all the way there. And time on site, as Google measures it, works great for e-commerce but is often confusingly broken for media companies. Google Analytics at one point had us at 21 minutes on site per visit on average; we’re good, but we know we’re not that good.”

A fine-grained and unforgiving metric

What Upworthy has come up with instead is its own measurement called “Attention Minutes,” which it tracks on both a site-wide basis and on a per-article basis. In order to arrive at this metric, the site says it looks at a wide range of signals about user behavior — including mouse movement, open browser tabs, whether a video player is activated, etc. — to see whether someone is still paying attention. Upworthy says the result is a “fine-grained and unforgiving metric.”

While many services such as Google Analytics show the amount of time spent on a page, those numbers are often flawed. In some cases they simply look at whether a webpage is still open in a browser tab, which can be distorted by the fact that many users leave tabs open but never return. As Upworthy notes, Chartbeat and other services have their own ways of measuring what they call “engaged time,” which adds as many signals as possible.

Chartbeat, for example, has something it calls “Average Engaged Time” — which is based on data that comes from the site being measured pinging Chartbeat’s servers every few seconds — while YouTube looks at “Time Watched” and Medium has its own measurement called “Total Time Reading,” which like Chartbeat’s system measures things like how far the reader has scrolled through a page. As Medium describes it in a blog post:

“Away from the publicity glare of the Valley tech blogs, every web company should have some not-so-bullshit metrics that guide the business and provide an indication of its health. Ideally, there is one number to rule them all. Josh Elman calls this The Only Metric That Matters. At Medium, our number is Total Time Reading.”

Upworthy says its new attention metric allows the site to determine how well it is actually performing when it comes to creating content that captures its readers’ attention. And the company says that it plans to stick with the new measurement even if it means that “we don’t do quite as well on uniques or pageviews… that’s a trade-off we’re happy to make.” And Upworthy plans to make the code for its new metric open-source so that other publishers can also use it.

I’m actually really, really excited about this. I think advertisers are definitely going to be interested in the attention-minute data once it’s made public. Once THAT happens, the entire content landscape is going to look pretty different.