Sep Kamvar: 'We need more nourishing metrics than downloads'

This article was taken from the November 2012
issue of Wired magazine. Be the first to read Wired's articles in
print before they're posted online, and get your hands on loads of
additional content by subscribing online.

There is a Zen story about a man riding a horse galloping
frantically down a path. His friend, who's sitting by the side of
the road, calls out, "Where are you going?" The man replies, "I
don't know. Ask the horse!"

When we build our tools, we often depend on metrics to guide
development. We keep graphs of unique visitors and page views and
watch them closely. This keeps us honest. It's hard to convince
anybody that we're building a useful tool if our metrics show
that nobody is using it.

But we must take care when we use metrics. Metrics can be like
the horse in the Zen story. Once we decide on them, they have a
habit of setting the agenda. As the adage goes, what gets measured
gets managed.

The standard metric for a country's economic welfare is GDP. I
find this strange. If the government decided to give millions
of pounds to the country's richest people so that they can buy
yachts from one another, that would increase GDP. So would felling
national forests to build shopping malls, outsourcing the raising
of children, and incarcerating large swaths of the poor. If we
temper the language a bit, we might find this description is not so
far from reality.

My point is: metrics shape behaviour. Joseph Stiglitz,
economist and professor at Columbia Business School, describes this
mechanism nicely: "What we gather our information about, and how we
describe success, affects what we strive for." Political leaders
who want to grow the economy, he says, will focus policies on
things that grow GDP, even when GDP does not correlate with
societal well-being.

Which brings me to my second point: all metrics leave something
out. Often, they leave out the most important things. In 2007,
Stanford offered a course called CS377W:
Creating Engaging Facebook Apps. The course assignment was to build
a Facebook application that, according to the course website, would
"focus on solving a problem for a broad audience". It was an
intensively metrics-driven class, and the key metric was user
numbers. By the metrics, the results were astonishing: during the
ten-week term, the apps collectively reached 16 million users.

The flipside was that the applications themselves were
underwhelming. Most of them allowed users to do things like rank
the attractiveness of their friends. The substance of the
applications reflected what the metric left out. If it were
possible to measure the value of a user's attention, or how
enriching an application is to his or her life, the course projects
would likely have been quite different. But, sometimes, the
important things can't be measured.

It is useful, therefore, to have missions to balance our
metrics. If I were to suggest one mission, it might be: every tool
should nourish the things upon which it depends.

We see this principle at varying levels in some of our tools
today. I call them cyclical tools. The iPhone empowers the
developer ecosystem that helps to drive its adoption. A bike
strengthens the person who pedals it. Open-source software educates
its potential contributors. A hallmark of cyclical tools is that
they create open loops: the bike strengthens its rider to do things
other than just pedal the bike.

Cyclical tools are like trees, whose leaves fertilise the soil
in which they grow. At the top of the stack, all tools depend on
nature and human nature. And they depend on users' attention. A
fully cyclical app might use peer-to-peer data centres powered by
its users, consisting of biodegradable, fertilising
microprocessors. It would be open source and provide APIs to
empower the creativity of builders, and a clean design and useful
purpose that cultivates users' concentration. If this sounds like
sci-fi, so did manned lunar vehicles in 1950, or self-driving cars
in 2000. We have a tendency to achieve what we focus on.

It's difficult to build cyclical tools because the alternative
is so tempting. Cyclical tools appear to be lower-power,
slower-growing, and more expensive than extractive tools. But you
can't measure the impact of tools on their own. You must measure
them by the ecosystems that they co-create.

Sep Kamvar is associate
professor of media arts and sciences at the Media Lab

Edited by David Cornish

Comments

Wrong on incarcerating poor people. Nearly everyone, even the poor, is a net producer of goods. And incarceration is expensive.

But the article is ignorant about economics and accounting in so many way...

Alistair Morley

Nov 13th 2012

"The standard metric for a country's economic welfare is GDP. I find this strange." >> Still, high GDP is usually correlated with higher standards of living. In today's world, it is a necessary but insufficient development factor. But yeah, economics ought to continue rethinking and developing better metrics--as it has done before (Gini coefficient, the Human Development Index, various health economics indicators, and so on).