A forum organized by the Massachusetts Technology Leadership Council addresses one of the hottest questions in social media: how do you measure the return on investment (ROI) for these platform? The panel, part of a “Social Media Summit” hosted in Microsoft’s Cambridge offices, was moderated by Dave Vellante, co-founder of the Wikibon Project and featured Fred Cremo of Humana, Leslie Forde of Communispace, Chuck Hollis of EMC and Katrina Lowes of Market Bridge. The panel followed danah boyd’s keynote on “social media evolution and digital ethnography.”

Chuck Hollis kicked off the panel by defining the challenge of measuring this kind of interaction and usage. “How do you measure a good conversation? A good idea? You guys are measuring the wrong thing.”

Lowes, whose focus on results and specific case studies throughout, put ROI in the context of creating relationships with Medicare recipients. The campaigns she has been involved with have been razor-focused on measuring all of the interactions, including what people are interested in. She described a partnership with Eons to host and provide discussion groups. Using them, they watch what people are talking about. As people move towards trigger point for Medicare, they watch more closely. As Lowes noted, “you get one chance to get a 65 year old into Medicare. If you can get people interacting with you three times before 64, you become relevant. That will have an impact on conversion rates.” At present, they’re taking a research-based approach to measuring impact utilizing a control group for direct mail and comparing it to the conversion rates of different groups based on a mix of social media presentations.

After a while, the audience grew restive, looking for a measure of hard ROI that could be used to justify social media use. The panelists understand the issues, especially at a large enterprise:

“When executives ask about social media ROI, they’re asking about risk. Why should I change decades of experience?”-@ChuckHollis

Hollis noted, in following, that managing risk in social media is challenging but possible: “negativity is passion that needs to be channeled to constructive conversations,”

Forde also sees the challenges for engagement marketing. With consumers (and users in general on the public Internet, you simply don’t know what you’re going to hear. (Note the Skittles experience). As she noted “in opening dialogue, you get serendipity & surprises.” For instance, Forde cited a case study provided by Kraft. People on their discussion boards were talking about weight loss through portion control. “Why can’t you make a tiny bag?” Kraft listened — and in the first six month, Kraft’s “Calorie Pack” earned more than $100 million dollars of revenue. Forde noted that the marketing campaign and manufacturing cycle in a one third of the time.

Forde noted as well that “It’s amazing how self-policing communities can be.” In her experience, community managers rarely have to step in and intervene. It is necessary, on occasion, to send private emails or direct messages and pull aside members to assert norms. How do you manage risk? Hollis noted that “EMC had a governance board for each project. They met once — and never met again. We never had a problem – but the structure was there to address it if necessary.”

When queried about adoption of social media by enterprises, Lowes voiced a key concern: “Everyone is in love with the technology. They haven’t thought about maintaining the conversations.” In her view, a company needs to have someone passionate to engage people and answer questions. The issue that many organizations are having with community management and conversation curation lies in a widespread tendency to put lower-paid people customer service reps. It’s not about technology or governance. It’s about skills, behaviors and attitudes. In Forde’s view, it’s about “trust, transparency and demonstration of listening.” That means that organization need to allow customers to be heard, with the understanding that it’s crucial to nurturing a long term relationship. That means “building websites around their interests and preferences, raising awareness of a company as a trusted partner,” according to Cremo — not through pushing sales directly.

When I stepped out, however, I returned to a groundswell of pushback for the panel. Where are quantitative social media metrics? Hard ROI? “The problem with social media is that we’re all talking to each other,” as one audience member put it. He stated that the total social media spend is “0.4% of the total annnual advertising budget in Fortune 500.” (That number was cited as $250B). Where’s the real return?

In response, Katrina Lowes offered the most substantive response of the day. “Consider: I’ve got a video to put online or on broadcast. You need to calculate the advertising comparison impact between the two mediums… How much would I have had to pay to get this exposure in traditional media?” She suggest looking at click through rates (CTR) of a cluster demographic from a social media platform or campaign back to the launch page of your website. Measure “Media equivalent purchase value” and conversion traffic, in other words, when it comes to ROI.

Forde noted that it’s also key to consider cultural differences, especially overseas, particularly with respect to hierarchical processes. If decisions are made once a month by a small group, observe how that can be improved. For instance, asynchronous tools can help – a lot – with time to market for products or campaigns. She cited one client where a 52-week time to market was cut to 14 weeks.

Considerable concern still remained in the audience with regard to unleashing social media internally. “What about the sexting that’s going to happen in my company.” Executives are worried about risk.

They should be, as Lowes noted. By tracking & gathering people’s personally identifiable information (PII) at Humana, they’re liable under HIPAA. That’s a major responsibility. Given the longevity & permanency of data on these platforms, organizations must be mindful of measuring ROI in more than conversion; they need to consider the risks of the overall project.

The audience reaction is similar to the hesitation and lack of understanding one feels upon signing up for Twitter. There is no evidence to provide encouragement. However, if you dive in and participate the value becomes apparent.

I wasn’t surprised at all by that resistance, given that I also showed up with the expectation of hearing more hard numbers and case studies. The panel offered some of the latter; I blogged above about the relevant comparison of paid media buys to what a broadcast placement might cost.

The metrics that I expected to hear — return on marketing objective, for instance, as measured by mentions or links from influencers — weren’t introduced. Twitter entered the conversation (and was an active back channel, at #MassTLC) but didn’t receive much notice with respect to measurement. The somewhat shallow post I wrote last month, “How do you measure social media ROI on Twitter? A ReTweetability Index?,” was more granular than what was offered.

I think the audience needed hard numbers to take back to execs to justify why this stuff is worth getting into. Doing social media “right” takes an investment of time, both for listening, learning the social mores and identifying people within the organization and outside of it who can communicate effectively about a brand, product, service or even personality.

I was surprised that we didn’t hear at all about inbound links, given the relationship between social media marketing and SEO. There’s a direct business case there to be made.

That said, there were other case studies that noted improved time to market; I’d imagine any manufacturer that spends money on focus groups would take notice of that.

With respect to your second comment, Jim, it occurs to me that we now have two years worth of examples of how, where, when and why Twitter can work well, if people choose to read them before joining. You’re right; it’s necessary to listen and follow people for a while before that value becomes apparent. Perhaps it’s not clear to enough folks because the listening skill isn’t one that we value enough as a culture.

I attended the conference as well. The reason to attend is that I wanted to better understand the current market place with respect to social media ROI. I was very surprised to find that there is basically none.

Last week, I pulled together a Social Media ROI spread sheet. It is based on work by Charlene Li and Bill Johnston. I put in on my blog and had so far close to 300 downloads. To me that shows there is a need. (http://dagholmboe.wordpress.com).

A solid ROI calculation deals with everything from traffic, conversions, demographics of users, decreased sales cycle (saving cost of sales), decreasing time to manage negative UGC and so on.

This is the week for seeking feedback on open government in the United States. 4 days ago, the White House published a collaborative online document that digitized the notes from an open government workshop held during Sunshine Week in March. … Continue reading →

In a followup post, the White House shared a link to a collaborative online document where the notes from an open government workshop held during Sunlight Week were posted online for comment. In doing so, they moves from sticky notes to … Continue reading →

Yesterday, I wrote up 15 key insights from the Pew Internet and Life Project’s new research on the American public’s attitude towards open data and open government. If you missed it, what people think about government data and the potential … Continue reading →

Today, a new survey released by the Pew Research Internet and Life Project provided one of the most comprehensive snapshots into the attitudes of the American public towards open data and open government to date. In general, more people surveyed … Continue reading →

On April 1st, some reporters, open government advocates and people in industry may have hoped that a new redesign of USASpending.gov, the flagship financial transparency website of the United States government, was just a poorly conceived April Fool’s joke. Unfortunately, … Continue reading →

In hiring Stephanie Hannon to be her campaign’s chief technology officer, former Secretary of State Hillary Clinton is not just bringing on a former Cisco software engineer with product management experience at Google and Facebook and time in the trenches … Continue reading →

Federal financial regulators and the industry that they regulate are fretting over Freedom of Informatiom Act reform in Congress, per The Hill. At least the concerns about sensitive info are being aired in public this time, albeit not on the record: … Continue reading →

In its search for technology talent, the White House has been recruiting heavily from Google of late, including U.S. chief technology officer Megan Smith. Today, President Barack Obama showed that his administration also likes Facebook, announcing that engineer David Recordon would upgrade the White House’s technology … Continue reading →

There are roughly 1,361 .gov domains* operated by the executive branch of the United States federal government, 700-800 of which are live and in active use. Today, for the first time, the public can see how many people are visiting 300 … Continue reading →

This is how @opengovpart National Action Plans get made pic.twitter.com/3d9gvmFUOs — Nathaniel Heller (@Integrilicious) March 17, 2015 Yesterday, the White House hosted an “Open Government Workshop” in Washington, DC, a portion of which was livestreamed at though whitehouse.gov. The workshop … Continue reading →

For many people in developing countries, the cost of data incurred by using social networks, sharing files, watching videos and playing games makes such activities prohibitively expensive. But if the idea behind a new startup catches on, "data back" programs might become the new cash back reward programs for mobile users around the world. When peop […]

Over the centuries, many projects with stratospheric ambitions, from pneumatic trains to space ships -- have come to naught. Facebook's project to provide airborne Internet to remote regions through an autonomous aircraft, however, is ready to launch. In a video posted to his Facebook account on Thursday, Mark Zuckerberg announced th […]

No matter how many different ways we ask the question, the answer from child development experts remains the same: humans under age 2 learn best from people, not screens. "The real emphasis shouldn't be on the taking away of screens," Dr. Claire McCarthy, a pediatrician at Boston Children's Hospital, recently said on HuffPost L […]

Imagine asking your phone to calculate the optimal route for cycling to work while enjoying the best air quality, or calculating whether it's better to just take the train or bus to reduce pollution. Similar requests might soon be possible. On Tuesday, Google and Aclima, a San Francisco-based startup that makes environmental sensor networks, announced […]

The White House responded on Tuesday to all of the unanswered online petitions that have met the minimum requirement of 100,000 signatures on its "We The People" platform. Twenty petitions received responses, including one requesting a pardon for Edward Snowden, one to require labels for genetically modified foods and another to ref […]

In a story for Vox Monday, Matt Yglesias argues that we shouldn't be worried about losing our jobs to robots -- or to any other kind of sophisticated tech that can do the work of a human. What should worry us, Yglesias suggests, is the possibility that this doesn't happen. To Ygelsias, the sluggish productivity gains in the American […]

Now that the Uberization of everything is breaking into mainstream politics, it's time to talk about what on-demand apps such as Uber or Pager will mean for transport, health care or other aspects of modern life and work. MIT research professor Andrew McAfee is the co-author of The Second Machine Age, an acclaimed book about how […]

As the 25th anniversary of the Americans with Disability Act draws near, how the landmark accessibility law will apply to websites is still in limbo. In a tweet Friday, Apple CEO Tim Cook made one of the clearest statements by any business leader to date about the importance of making technology accessible to everyone. Accessibility rights ar […]

Former Secretary of State Hillary Clinton hasn't given many national media interviews on the campaign trail, so when she took to Facebook on Monday for a question-and-answer session, journalists eagerly peppered her with queries. (Clinton answered a question I asked her there, signaling her support for more flexible benefits for work […]

The federal government has been taking down websites that host illegal activity for years, but in taking over a website -- specifically, FAFSA.com, which until recently had been operated by the company Student Financial Aid Services, Inc. -- the Department of Education is doing something unusual. FAFSA is the Free Application for Federal […]

Putting a dollar value on clean water, stable markets, the quality of schooling or access to the judiciary is no easy task. Each of these elements of society, however, are to some extent related to and enabled by open government. If we think about how the fundamental democratic principles established centuries ago extend today purely […]

In an age where setting up a livestream to the Web and the rest of the networked world is as easy as holding up a smartphone and making a few taps, the United States Supreme Court appears more uniformly opposed to adding cameras in the courtroom than ever.

On January 10th, 2013, the OpenGov Hub officially launched in Washington, DC. The OpenGov Hub has similarities to incubators and accelerators, in terms of physically housing different organizations in one location, but focuses on scaling open government and building community, as opposed to scaling a startup and building a business. Samantha Power, special a […]

Last September, I gave a 5 minute Ignite talk at the tenth Ignite DC. The video just became available. My talk, embedded below, focused on what I’ve been writing about here at Radar for the past three years: open government, journalism, media, mobile technology and more. The 20 slides that I used for the Ignite were a condensed version of a much longer prese […]

After years of steady growth, open data is now entering into public discourse, particularly in the public sector. If President Barack Obama decides to put the White House’s long-awaited new open data mandate before the nation this spring, it will finally enter the mainstream. As more governments, businesses, media organizations and institutions adopt open da […]

There are few ways to better judge a nation’s character than to look at how its children are educated. What values do their parents, teachers and mentors demonstrate? What accomplishments are celebrated? In a world where championship sports teams are idolized and superstar athletes are feted by the media, it was gratifying to see science, students and teache […]

Simon Rogers Twitter has hired its first data editor. Simon Rogers, one of the leading practitioners of data journalism in the world, will join Twitter in May. He will be moving his family from London to San Francisco and applying his skills to telling data-driven stories using tweets. James Ball will replace him as the Guardian’s new data editor. As a data […]

Creating the conditions for startups to form is now a policy imperative for governments around the world, as Julian Jay Robinson, minister of state in Jamaica’s Ministry of Science, Technology, Energy and Mining, reminded the attendees at the “Developing the Caribbean” conference last week in Kingston, Jamaica. Robinson said Jamaica is working on deploying w […]

Last winter, around the same time there was a huge row in Congress over the Cyber Intelligence Sharing and Protection Act (CISPA), U.S. Attorney General Holder quietly signed off on expanded rules on government data sharing. The rules allowed the National Counterterrorism Center (NCTC), housed within the Department of Homeland Security, to analyze the regula […]

When I went to the 2013 SXSW Interactive Festival to host a conversation with NPR’s Javaun Moradi about sensors, society and the media, I thought we would be talking about the future of data journalism. By the time I left the event, I’d learned that sensor journalism had long since arrived and been applied. Today, inexpensive, easy-to-use open source hardwar […]

GitHub has been gaining new prominence as the use of open source software in government grows. Earlier this month, I included a few thoughts from Chicago’s chief information officer, Brett Goldstein, about the city’s use of GitHub, in a piece exploring GitHub’s role in government. While Goldstein says that Chicago’s open data portal will remain the primary m […]

When it comes to government IT in 2013, GitHub may have surpassed Twitter and Facebook as the most interesting social network. GitHub’s profile has been rising recently, from a Wired article about open source in government, to its high profile use by the White House and within the Consumer Financial Protection Bureau. This March, after the first White House […]

This week, I found that one of my Facebook updates received significantly more attention that others I’ve posted. On the one hand, it was a share of an important New York Times story focusing on the first time a baby was cured of HIV. But I discovered something that went beyond the story itself: someone who was not my friend had paid to sponsor one of my pos […]