Assessing stories for the Knowledge Management Impact Challenge

We were contacted late last year by colleagues from the Impact Alliance to participate in a small piece of research and knowledge sharing about the impact of knowledge management interventions in our organizations.

About the Challenge
“Are you challenged to…

Assess whether your KM activities actually improved performance?

Link KM measures to other organizational measurement frameworks?

Provide evidence of successful KM processes to support institutional scale-out?

Demonstrate the value of KM to senior leaders?

The Knowledge Management Impact Challenge is bringing people together to discover better methods to meet these challenges. Figuring out how to effectively measure the results and impact of investing in KM and learning can help us focus our efforts and resources, and make the case for knowledge management to senior managers.”

The KM Impact Challenge is sponsored by USAID’s Knowledge-Driven Microenterprise Development project as a key part of the project’s Assessing & Learning component which seeks to improve our understanding of how investing in learning can increase and extend the overall impact of our development efforts. The KM Impact Challenge is being carried out in collaboration with KM4Dev and the Impact Alliance.

Louise Clarke, a consultant working on the project, and Jeffrey Kwaterski from the Impact Alliance contacted us to contribute with initial feedback on the project concept, and thus we joined the technical advisory group (TAG). This TAG is composed of many colleagues and friends who provided feedback, helped translate project documents from English into other languages and who reviewed the 40 or so proposals that were received through the call for short (about two page) descriptions of an “experience assessing knowledge management activities”.

About the process

The process consisted in a review of the case stories submitted, to be followed by a one day face to face event to discuss the best case studies and next steps. While this is a rather classical approach, there are many things we liked about it:

Partnerships: The project coordinators tapped into their existing networks (KM4dev, CGIAR centers, and friends) to share ideas, adjust the concept, distribute the work load, and assure a good outreach of the call for case stories.

On-line interaction: A whole range of social media is used to promote the initiative; a blog serves as a sharing mechanism of the progress and insights gained during the project. The stories are accessible to all public and can be commented on, before and through the review process. We really appreciated this part of the approach: it adds a lot of value to the proposals, which end up becoming an information resource for all public instead of being only an application for a jury. This was something we did differently for the Knowledge Share Fair stories, but we plan on adopting the more ‘widely sharing’ approach used here.

The review process: The same proposals were evaluated by several reviewers. All proposals are available through an online excel sheet, where we were able to see each other’s evaluations. The criteria and comment fields were clear and facilitated the process.

.
About the case studies

Among the 40 or so stories, quite a few didn’t focus the analysis on the KM evaluation approaches per se but on the results of the project interventions. Thus we learn little about what worked and what didn’t related to KM impact assessment, although some of these divergent stories did present interesting KM initiatives!

Among the 10 proposals I had to review my preferred story was “Preventing HIV/AIDS among Rickshaw Pullers in India through Knowledge Management”. I was struck by the high degree of vulnerability of an illiterate and highly socially excluded group: “The Rickshaw Pullers possess some common characteristics like unsafe sexual behavior, multiple sex partners, homo sex, substance abuse, lack of family planning etc. which makes them highly vulnerable to HIV/AIDS”. The prevention approach seems very interesting as the project formed 31 Community Peer Educators and 310 Peer Educators who serve as connections in the network of Rickshaw pullers. I also liked the idea of having two types of M&E: one with the peer educators and one with the rickshaw pullers using improvisational theatre: “Through use of magnet theaters we are building and checking the knowledge of community simultaneously which is also increasing community participation.” Finally, and in this context perhaps not surprising, the game and theater methods seemed to work better than the written evaluations (peer educators diaries).

We are looking forward to the results of the face-to-face debates of this Challenge, which will take place in Washington D.C. in May 2011.