I run an executive think-and-do tank called REX, where we harness the forces that are ripping business, government and society inside-out. Before this, I worked for a dozen years as a tech industry analyst (half of them writing Release 1.0, a tech newsletter that put me at the 50-yard line of the tech revolution), and then headed off on my own to advise startups and consult to BigCos about the future they were heading into.
Raised in South America and curious about everything, I'm a pattern finder who connects things in ways a lot of other people miss.
More on my online profile and blog. I'm @jerrymichalski on Twitter and jerry@sociate.com for traditionalists.

Big Data and the Stalker Economy

Crack is being served in Silicon Valley. An enthusiastic crowd of geeks and suits — all of them “data scientists” — just spent three days at the O’Reilly Strata conference (#strataconf) in Santa Clara. All over the event’s menu is the crack cocaine of our day: big data.

A couple decades ago, when “big data” was mostly the dominion of a few credit-rating and mailing-list management companies such as Acxiom, Experian, InfoUSA (now Infogroup) plus some disparate, disconnected monster databases built by telcos, physicists and national security agencies, I used the metaphor of catnip instead of crack. I’d describe how access to rich data about customers was like catnip to executives, managers, statisticians and others in the consumer-mass-marketing economy. It was irresistible.

But that was long before most of us had poured our hearts and lives into Blogger, Facebook, Twitter and dozens of other online sites, for all to see — and capture. Catnip now evokes images of LOLcats and frenzied cuteness, which is too lighthearted for the topic at hand. The bits of data that a few players had about us have turned into a torrent of data that’s out and about. And less expensive than before. It’s addictive like crack.

Big data is strategic now. Facebook is valued at around $100 billion because it has collected a treasure trove of data that may unlock the secrets of selling more things to more people. Most other companies would like to have whatever they’re having. Google offers free email, word processing, mapping, analytics, video, videoconferencing and much more because they’re selling us to advertisers. The byword these days is, “if you’re not paying for the service, you’re the product.” Before turning to the ethical choices at hand, let me offer a mini paean to big data. I love big data, and not just because of my background in econometrics. A few things to love about it include:

It’s helping solve big problems. Early detection of epidemics. Automated spell-checking. Crowdsourced astronomy. We seem to have entered the Age of Big Data.

It’s creating useful feedback loops. In participatory medicine, people opt in to share data so they can analyze it and continually improve their health outcomes. This kind of feedback is spreading from field to field.

It’s eroding the culture of expertise. Read Daniel Kahneman’s new book and you’ll stick to statistics. Add a pinch of Taleb and you’ll never speak to tie-wearing experts again.

It’s nurturing a culture of collaboration. From participatory medicine to open science and open government, scientists and citizens alike are resetting the terms of innovation.

It’s a major new source of employment. Ok, maybe not major, but one of the few bright spots on the job horizon is the desperate need for those “data scientists.”

The dark side: Now we can talk about big data’s dark side, one aspect of which was spotlighted recently in a piece by Charles Duhigg that must have caused Target‘s senior management to break out in a cold sweat. I’ll focus mostly on personal data here, even though there are dark sides to other collections, as well.

We are in the cross-hairs: When companies say they are “customer-centric,” we might be tempted to think that it means that they honor customers above everything. That the customer’s wish is their command.

Post Your Comment

Post Your Reply

Forbes writers have the ability to call out member comments they find particularly interesting. Called-out comments are highlighted across the Forbes network. You'll be notified if your comment is called out.

Comments

CJ, there’s a strong argument to be made that the set point for privacy is shifting, that the new generation simply feels different about it. I’m happy to debate that, but my current PoV is that it isn’t moving that much, and that human behavior (in this area) will remain pretty stable in the long run.

Great piece, Jerry. I’m just finishing Kahneman’s book and it is enlightening and timely. We’re just discovering how little we know about ourselves at a time when big data knows us better than ever (how appropriate that Forbes asked for my twitter and email account to post this comment).

Working in health care, we hear a lot about “patient-centricity” nowadays, and for the most part, that’s a good thing. We haven’t known a lot about what actually works in the contexts of delivering care or keeping people on a healthy course. The data simply hasn’t been available, and in a fee-for-service model, not particularly valuable, but times are changing.

We’re just beginning to learn that to enable better health, we’ll need more data about how healthy behavior spreads and what works for which people. As was elegantly written in Fowler’s “Connected”, health, both mental and physical, spreads quickly through social networks. How far can we go in our patient-centricity to understand healthy behavior and it’s social contexts? Social networs are private, yet deeply valuable sources of health information. PatientsLikeMe is the obvious example where people are willing to share personal information if it will help others. What about before you get sick?

As Kahneneman points out, physicians are notoriously bad (like the rest of us) in interpreting statistical information, so many decision in this realm should likely be decision-supported or automated.

It’s no accident that health insurers are buying up health information networks and app makers. Lots to work out in defining the line between privacy and public good and figuring out who’s in charge.

Health care is one of those tricky spaces. HIPAA is pretty specific about what providers inside the system can do, and actually cuts down on potentially useful avenues of social medicine.

Then the “outsider” sites like CureTogether and PatientsLikeMe get participants to share astonishing amounts of deeply personal data, all because their conditions are severe and chronic, so they need the mutual problem-solving very much.

I wish there were more middle ground, and I hope that we find ways to nurture that middle ground.

Separately, I get a kick from reading Atul Gawande, who talks about the huge effect paying attention to statistics and checklists has on health outcomes.