What to Measure? The Question Journalism Has to Answer

How should we measure articles' success?

A few months ago I wrote an article for a website where the standard agreement for writers is a bonus incentive on social shares for the article. This was both usual, and unusual for a lot of reasons.

First, most websites don’t pay writers anything. A good portion of writing online is done for exposure (which a lot of people laugh at but content marketing can be hugely lucrative and I encourage my clients to do it). So that was slightly unusual. What was more standard was the fact that for a site that did pay, the payment was partially contingent on page views (there was a bonus for how many social shares the article got).

Slightly more unique, I am in a financial position where because of my books and other income, I did not need to bother with such a bonus and could refuse—the standard, token payment for the article itself was plenty fine and plenty motivating. Why would I write something if I didn’t want lots of people to see it and like it?

That article did something like 5,000 Facebook shares—a good portion of them because I worked hard to promote it. Now that I am working with Betabeat, I’ve been thinking about all this quite a bit. Because it’s a dilemma every writer is thinking about—as well as every editor tasked with growing an online property.

Though I could paint it otherwise, the site was not trying to corrupt me with some compromising offer. Nor was I some moral paragon for refusing it. After all, my posts at Forbes resulted in some nice little checks due to the per-pageview rate they charged. My contract with Betabeat also includes traffic goals and rewards tied to them.

When I see Marc Andreeseen’s intelligent critiques of journalism, particularly the bad metrics of current journalism, l agree. But the other side of me thinks: well, what the hell else are people supposed to do?

What am I—or any publisher—supposed to measure then? How should media outlets in 2014 determine what “success” looks like and create employee compensation parameters around it?

If pageviews or uniques aren’t it, what should it be? Gut instincts? Length? Comments? What?

It’s the lack of a clear, easy answer to these questions that has us in the state we are currently in. As much as I’ve criticized it, I empathize with the quandary. Page views are easy to track. Quality is harder. It’s not only relative and subjective, it takes time and resources that many people don’t have. It’s easier to fall back on “whatever everyone else is doing.”

I’ve used this analogy before but here it goes again: We can all admit that “bodycount” is a pretty terrible metric to use in a war–one responsible for all sorts of calamities–but what’s a better one? What’s one that’s not only better but more practical than this obvious and straightforward heuristic.

Because it’s real easy to criticize, but hard to come up with a genuine alternative

Maybe step one is to start having this discussion publicly–to start figuring out and hearing from readers about how they want it to go. I’m not talking about what they’re clicking, that is notoriously inaccurate (as Henry Ford said, if he’d listened to customers he’d have invented a faster horse). I’m talking about eliminating the opaque veil that hides how the business works.

When the public understands the economics of the information, they can begin to understand why the information is how it is. They can also make decisions about who they choose to read and why, just as they might pick the ad-free Consumer Reports over the potentially biased Car and Driver. Right now it’s the decisions being made at BuzzFeed and Upworthy and Gawker that are driving how our information is produced, instead of the values of readers and the public.

Andreessen posited that quality came to the news industry as it was taken over by “oligarchies.” Rich families care about money, but they also care about how they look to people in their social circles. They have connections and relationships that they value in addition to the raw bottom line. I passed on that bonus for that exact reason. That came from a privileged position–which not everyone is able to do.

Maybe we need more of that. In a hyper-connected world where relationships and networks matter more than ever, we should consider incentives—and a culture—that makes people think twice before making decisions that externalize bad outcomes on the consumer.

Holy crap, this is insightful. I wasn't expecting to find such clear insight here.

It's worthwhile (to me) to see that others are considering the same issues -- not just that "the media is broken," but WHY it's broken, and HOW things got that way -- when pondering "how do we fix this mess?"

Despite my disdain for the profit motive being the genesis of everything, there's a lot of bitter truth to the concept that "nothing is free" (or, alternatively, "free ain't"). As soon as we move towards "freemium," ad-driven content, we suddenly make the "value" of a piece of creative work (and news blogging and, say, computer programming -- my profession, and my bias -- are both just as creative as music composition or painting, IMO) *how many ads it can sell*. It loses any concept on intrinsic value, and simply becomes an "ad delivery mechanism," like network TV currently is. This is why page views are so problematic; they're not really interested in measuring "how many people are looking at my story" but "how many eyeballs can we attract to *the ads we deliver alongside* my story?" This perverse incentive produces ever more inflammatory headlines and ledes.

I really don't know the solution. Shaming helps, but the problem is that proper shaming relies upon a fair, intact system of reporting. We have to be able to accurately report a shameful story, to use the power of shame to suppress "broken" journalism; but when the infrastructure itself is suspect, you run into a "bootstrapping" problem. Who do you trust to tell you who to be ashamed of? How do you know the oligarchs are lying when they say "oh, the Daily Muckraker reported a shameful story about me? Well, they're just a leftist/rightist rag."

Betabeat is now the newly launched Innovation section of the Observer. All your favorite features and columns—as well as exciting new areas of tech coverage—can now be found at Observer.com/Innovation.

Don't miss the latest and best writing on technology and the future of business innovation. Add the Innovation section to your RSS feed and follow the Observer on Twitter and Facebook.