Measuring the value of communities of practice

Ed Mitchell over at Platform Neutral picks up on the issue of Return on Investment (ROI) for on-line communities, quite rightly identifying this as growing topic for debate and argument over the coming months. In Ed’s words:

"ROI
for â€˜communitiesâ€™ is going to appear on our horizons for proper this
year. We will have the debate from the sponsors asking if itâ€™s worth
it, and another one about which way around it should be – ie: ROI for
whom? The sponsors, or the participants. Likewise there is much talk
about the ROI of â€™social mediaâ€™, and all of the projects I am working
on have measurement built into them at the strategy level"

Personally, I haven’t quite cracked this nut for the virtual communities using the IDeA platform (240 of them at the last count). Since many of the communities are closed, self-organising networks, I only really see the platform-wide metrics. So, for example, I can see:

Total number of communities

Total registered users

Total contributors

Members per community

Total topics

Threads with responses

Responses per thread

Participating users

Total number of blogs

Total posts

Total comments

Total number of Wikis

Total edits

Articles per community

Total number of documents

Total number of document comments

Number of Documents per community

Total number of messages

Number of users sending messages

Number of users receiving messages

Total events posted

Events per community

I can get the same raw data from each
individual community (but it’s a tedious process doing this for 240 communities!)
– though this does at least give a more accurate indication of the health of
the individual community (where my definition of ‘health’ means
‘activity’), but none of this information can provide me with
a ‘value’ or ROI – no matter how I slice and dice the data.

Hence the need for a more qualitative approach, in the form of membership and user surveys etc. Picking up on Ed’s comment that all of his community projects have measurement built into them at a strategy level – which sounds like the right approach – it should also be noted that even this can be difficult to quantify in terms of an ROI. The IDeA community strategy was developed to "improve local government services". I can infer from the platform statistics that the communities appear to be active, but I cannot yet connect a specific output (service improvement) to the work of any individual community.

So, allowing for the subjective nature of user surveys, I still think they are more likely to give an indication of ROI if not an absolute measure. If I’m right, the next step is in asking the
right questions!

Stephen is Director and founder of Collabor8now Ltd, an organization focussed on developing collaborative environments (e.g. Communities of Practice) and the integration of knowledge management tools and processes to support business improvement. He is a certified knowledge manager with the Knowledge Management Institute (KMI) and the author of several published research papers on collaborative behaviours and information technology.

2 Comments

ROI for social networks isn’t nearly that complicated. When Ragan started “MyRagan” and Melcrum started their network, I asked both of them how they were going to measure their success and they both said “membership.” I wondered how “membership” translates into revenues, but then both organizations do conferences and the rule for conferences is, the bigger the mailing list the more successful the conference. So the true ROI for those networks, is ultimately tied to the attendance at their conferences. It’s as simple as that. What’s the desired outcome to the organization? and then you take the cost of the network and subtract it from the revenue benefits.

Thanks for the marvelous posting! I seriously enjoyed reading it, you happen to be a great author.I will ensure that I bookmark
your blog and may come back very soon. I want to encourage you to ultimately continue your great posts,
have a nice weekend!