Tuesday, 4 October 2016

It has been two years since the World Bank published a report that stated that over 30 percent of its policy reports have never been downloaded even once and only 13 percent of policy reports were downloaded at least 250 times. The debate among development practitioners that followed made it clear that the World Bank is by far not alone with this phenomenon and that most international organizations, including UNDP, face the exact same challenge.

As UNDP provides support services for implementation of the Sustainable Development Goals (SDGs), we in UNDP’s Knowledge Management Team see the importance of getting insights into the perceived value of our knowledge products and therefore UNDP’s thought leadership in various SDG topics.

In fact, UNDP’s Knowledge Management Strategy 2014-2017 pointed out that UNDP needs to invest in its process of planning, developing and disseminating knowledge products in ways that make them “more relevant to clients’ needs, more flexible and timely in their development and format, and more measurable in their quality and impact.”

During the debate that followed the World Bank’s report, we the Knowledge Management Team at UNDP thought long and hard how to get meaningful data about who is actually reading its publications, to what extent those readers find individual publications useful, and most importantly, for what end those products are actually being used. In order to do this right one would almost need to talk to each individual reader and ask them one by one, which is kind of impossible on an ongoing basis. Or is it?

Well, after several prototypes and tests during the last year, we’ve finally come up with a model to do just that. In March 2016, we tweaked UNDP’s Public Library of Publications so it would present users with a post-download pop-up asking them whether they would be willing to leave their email address so we could contact them later.

In the six month since we introduced this question, over 42,000 users left us their email addresses, and we since followed up with 27,000 of them (through a weekly survey issued a few weeks after a download of a specific publication), asking them how useful they found the specific document they downloaded, what organization they are with, and whether/how the publication made a difference in their work. As of September 2016 we received 1186 survey responses, and the insights we get from our audience goes far and beyond any of the intel we had in the past.

We can now see how useful our publications are to our users, and to what extent specific publications reflect on UNDP’s thought leadership in that topic:

Even with the possibility of a voluntary response bias, the numbers serve as a valuable baseline to track changes in perceived usefulness over time. In addition, we now for the first time get a clear picture who is getting value out of our publications:

And most importantly, we learn from our audience how and for what purpose they use the downloaded publications in their work:

We are also getting great qualitative feedback on how we can improve specific publications in the future, and the individual comments provide great anecdotal evidence at project or community level that demonstrate the impact of UNDP’s work on the ground. Here are some of the impact stories we’ve received:

“The publication was used in the development of our food security and livelihood strategy for the Uganda refugee operation.”

“The publication has been useful as a starting point to persuade managers of Nature Reserves and Forest Reserve to consider ecotourism planning besides conventional forest management planning.”

“Some of the inputs were used in our legislative agenda setting, especially those that are applicable to the Philippines situation.”

“I am working in Rwanda’s Environment Management Authority and the publication is useful for public sensitization.”

“I introduce the paper to PhD students in my development administration class and asked them to prepare a paper on SDG targets.”

“The publication was of fundamental importance for the Pedagogical Political Plan formulation for professional training courses developed within my organization, the Military Police of Mato Grosso, Brazil.”

Going forward, we are making this qualitative feedback available to all our staff, so they will be able to look up their publication and go through all the individual comments the publication received. It is this kind of evidence that shows us where investment in the quality of our publications pays out, and where we need to switch gear, improve our efforts, or shift our focus entirely with regards to specific thematic areas. Most of all, it is these stories that inspire us as staff on a daily basis as they remind us why we are doing what we are doing in our pursuit of sustainable human development.

Of course, this measurement approach is only reaching those who download publications online, and will miss out on all those who receive them through hard copies or through presentations at workshops and conferences.

What did your organization do to get feedback from your offline audience, and do you have any suggestions for how UNDP could fine-tune the above measurement approach? Leave comments below, I’d be glad to hear your suggestions!

Thursday, 18 June 2015

I was recently reviewing a number of texts which my
organization collected from past projects and initiatives (some through an
internal mandatory monitoring tool, others gathered as part of After Action
Reviews or Lessons Learned Papers), which all meant to capture ‘lessons learned’
from specific experiences.

And while these texts were not wrong per se, I realized that
there seems to be a fundamental misconception what constitutes a good lesson,
and what doesn’t. Here are a few typical examples of what we often collect as
part of such lessons learned exercises:

“Ensure that
the [Team] Manager has excellent leadership, project and team management
skills, understanding of programming and experience working in [the subject
matter].”

“Project
outputs must be compatible towards project goals. Throughout the project there
is a need for careful identification of project goals and outputs to ensure
that they are compatible with each other. This can be only ensured through a
consultative and participatory approach in project design with target
institutions, implementing partner and experts.”

“Managing
relationships between key national and international players during [the project
activity] is very important. Recognizing and respecting national ownership and
leadership of the process is vital and key to winning the trust of the national
authorities.”

“The
better local authorities are involved in the process, the better the expected
results are easily achieved and durable.”

The above examples are representative for a common type of
lessons learned write up, which fails to pass what I would call the “Duh-test”:

If a ‘lesson learned’
statement is so obvious that it is self-evident to every reader, and at the
same time so generally applicable to almost any type of project or initiative,
it basically becomes meaningless.

It is good when a team realizes that it failed to put in
place a team leader who has leadership and team management skills (and yes, it
should remind itself do better next time), but there is literally no value in
sharing that learning point with others outside the team, simply because everyone
already knows that this should always be a criteria for selecting team leaders.
There is nothing new to learn here that
would change anyone’s views or actions.

Also, if a lesson is so generic that it could apply to any
scenario, we deprive ourselves of the learning effect that comes from understanding the particular conditions responsible
for making your project work or not work, so others can go and try to
replicate or avoid those conditions.

Such lessons that are either too obvious or too generally
applicable produce ‘lessons learned noise’ because these same lessons are
reported from countless projects over and over again, without anyone actually
learning from them. At the same time, this noise detracts everyone’s attention from
the meaty lessons learned pieces that really provide value to a wider audience.

So what is it that makes lessons learned write-ups actually
add value? Maybe asking ourselves the following three questions could help make
lessons learned statements worth capturing and sharing:

Will
anyone else actually learn something new from this lesson, as opposed to
self-evident truths that everyone usually already knows?This is the “Duh-test” and should always be the first criteria.

Is this
lesson particularly relevant to your specific situation, as opposed to a lesson
that it so general that it would apply to any scenario?The more general a lesson is, the less useful it is.

Does
the lesson include or lend itself to a concrete action that you or someone else
can take in order to effect a change in future practice? Capturing a lesson is only meaningful if there is an actual change triggered by
it

But aren’t the ‘bad’ examples mentioned earlier still true and important to
highlight, even if they are not particularly new or context specific? Doesn’t
the fact that everyone agrees to them intuitively and that they apply to all
our projects and initiatives all the more valuable?

Absolutely! But I would never call them ‘lesson learned’. Rather, these are important principles that
anyone should abide by, no matter what subject matter expertise or functional roles
someone has. We should treat them as guiding lights for our work, teach
them in our training curriculae, communicate them our onboarding and induction sessions
and embed them our policy guidance. Some
lessons from projects, if they are collected often enough, might eventually be
added over time to such a common canon of principles. But we should stop
collecting what is already part of that canon over and over again from
individual projects, which is no good use of anyone’s time.

Monday, 13 October 2014

During the last few weeks I was heavily involved with the SHIFT Week of Innovation Action,
a series of parallel events taking place in 21 different country offices. Over
50 practitioners were invited to ‘shift’ from one country office to another to
share their experience on innovation methodologies and what they learned from
their ongoing innovation projects (many of them funded by UNDP’s
Innovation Facility), learn from others, and ‘shift mindsets’ in the
process.

As part of the team that coordinated the event week I was in
awe of the incredible energy coming from country office colleagues and the
enthusiasm, creativity and time commitment on the side of organizers,
participants, and the coordination team here in New York. And from the feedback
that has been rolling in so far (the evaluation survey shows about 95% of
participants were satisfied or very satisfied with the event) it seems the
SHIFT initiative was a success all around.

Yet, we all remember other instances of well-organized
events which achieved great visibility, but when people were asked there months
later what the impact of the event has been, we didn’t have much to show for
it.

So you had a nice
event that brought people together and left everyone happy and excited, but so
what? What came out of it?

I believe we have to be very honest about how we define
success of events. Yes, it is good when participants convey in a survey how
much they enjoyed the gathering. And it is also great when the event achieves visibility
and external recognition with good communication during and immediately after
the event, such as national media coverage of the SHIFT hackathon in Belarus,
great videos produced about SHIFT events in Haiti, Montenegro or Georgia, or outreach
products such as the SHIFT
Exposure compilation, that give audiences a glance of what happened.

But it is not enough. Because if 12 months from now, none of
the new ideas generated will have inspired actual initiatives, projects or
products, if none of the innovative prototypes developed will have been applied
in real life, none of the solutions shared will have been successfully replicated
or brought to scale, and no one who couldn’t participate in person has a chance
to learn from what was discussed the event – then I don’t think we can call the
exercise a success.

Then it will just
have been a bright bonfire that burned for a single night. We have a nice
picture of it, but it will not warm anyone going forward.

So here is what I think is needed to make events worth the
investment we put into them in terms of time and money. And please feel free to
add your own bullet points to this list:

1. Set up
an after-event communication plan, and follow up diligently

Rather than letting organizers and
participants disperse after a good event, let’s use the current momentum and
excitement when people return to their offices. Make a plan on how we want to
communicate the results, increase visibility and leverage the event’s
discussions and activities to initiate new collaborations, products and
projects. Maybe this is the opportunity to promote an existing Community of Practice
(COP), or establish a network of mentors around your topic! Make sure to use
all available channels, from internal COPs, to external online networks
(LinkedIn, Devex, DGroups, World Bank networks, etc.) to public social media
channels (Twitter, Facebook, Slideshare) and try to engage new audiences.

2, Relentlessly
focus on knowledge and learning products

Communication products and activities are
crucial for getting recognition and visibility, and for reporting back to
donors. But the important substance, the ‘meat’ of knowledge and learning
points is what others really need in order to apply the results of the event to
their work. Where can new colleagues who join the organization six months from
now access the video recordings and slides of the presentations given so they
can follow the event’s learning points? Where can they find blog posts and
short interviews with personal insights and reflections of participants on what
they learned at the event and how they intend to apply that to their own work?
And where are the hands-on knowledge products that help them review the
examples shared and apply the solutions that were discussed? If there are only
glossy brochures and good-looking PR videos, but no substantive project examples,
how-to articles, lessons learned summaries, guidance notes or toolkits coming
out of the event, then we might look good externally, but the event was still a
failure for the organization as nobody other the handful of on-site
participants will learn anything from it.

3. Track
status initiatives and projects coming out of the event

One of the reason we as organizations facilitate
working-level events is to fulfil our role as a broker of exchanges to inspire
and improve our projects and programming. We
must come to an understanding that we cannot afford to organize events that
look great from the outside but that do not result in concrete, improved approaches,
projects and initiatives that are being replicated and scaled up in other
countries and regions. We need to wrap up events with concrete commitments
on what will happen next, and be diligent in checking-in with organizers and participants
at different intervals after the event on how their commitments, prototypes and
follow-up activities are evolving (and no, just planning for the next event to
discuss the issue further doesn’t count! ;). That means that as an organization
we have to expect more from participants than showing up and consuming
presentations, but rather for all to become part of an active knowledge
production and application process that extends far beyond the event’s closing
session.

Thursday, 29 May 2014

Since the World
Bank published its report early this month which states that over 30% of
its policy reports have never been downloaded even once (!) and only 13 percent
of policy reports were downloaded at least 250 times, a fascinating debate on
the purpose and value of knowledge products is flourishing the web, and the
posts from KM practitioners all over keep pouring in.

It’s not just the
World Bank, but most international organizations

Interestingly, I have been thinking about exactly the same questions
for the last 9 months now as I was drafting UNDP’s new Knowledge Management Strategy for the upcoming years. Here’s a passage which
captures UNDPs own dilemma regarding knowledge products:

“The current process of knowledge product
definition, development, dissemination and measurement does not yield the
quality, reach and impact that is needed for UNDP to be a thought leader in
development.” The Strategy goes on to stress that UNDP intends to revise its
process of planning, developing, and disseminating knowledge products in a way
that makes them “more easily accessible,
more relevant to clients’ needs, more accountable towards the community they
seek to engage, more flexible and timely in their development and format, and
more measurable in their quality and impact.”

Format matters

A lot of contributors to the debate, such as the commenters of
the respective
Washington Post article, the DevPolicy
Blog, Crisscrossed or my KM colleagues
from the KM4dev
network highlight how we have to get much smarter in developing formats
that actually appeal to an audience that is increasingly passing on lengthy unappealing
reports and paper. And there is a lot of truth to this. Colleagues at UNDP are
increasingly learning that short and snappy products, such as blog posts, 2-pagers
or infographics will allow communicating important key points from their work to
a larger audience and also more just-in-time. Compared with heavy research
reports which take months and years to finalize, the advantage of light-weight
formats is that they allow for adjusting content quickly as new data and
evidence emerges, which makes the product more relevant and timely the moment
it is distributed.

The launch of a paper
cannot be the end of the project

Ian
Thorpe (who arguably came up with the most crisp blog title in the debate
so far ;) also makes an excellent point in clarifying that we have to invest
much more in dissemination and outreach. All too often the launch of a product
is declared the successful end of a research project, when in fact, this should
be just the starting point of a whole new phase where we reach out to potential
audiences through all possible traditional and social media channels, organize
webinars and on-site events to raise awareness of the knowledge product and its
key points, and inject ourselves into ongoing debates where our product can add
real value. Budgets for development of knowledge products leave this part of
the process chronically underfunded, and we as KM practitioners need to make a
point that a dissemination and public engagement strategy has to be an integral
part of any knowledge production process.

The real issue is the
lack of community feedback loops

But while clear abstracts, interesting illustrations, good
formatting and focused outreach will go a long way in mitigating the “too long;
didn’t read” (TL;DR) problem, my personal belief is that we must pay much more
attention to where the problem of unread knowledge products starts: at inception.
The Complexia
blog nails it when it points out that there is a “lack of demand-driven research” in which “research projects tend to be more driven by the interest of individual
researchers”.

How can it be that organizations give authors green light
for the development of papers and reports for which they haven’t done any preliminary
analysis of what the targeted community needs and whether the product to be
developed is likely to find an audience? How
is it possible that we can go through an entire cycle of a product production
process without probing with the relevant communities of practitioners outside our organizations whether the
questions we ask and the conclusions we draw resonate with the audience that is
supposed to benefit from them? And not just once in a peer review when the product
is almost finished, but at every step, from inception to formulation of
research questions, outline and early drafts?

It is clear to me that we need to get rid of our internal
navel-gazing posture and get much better at involving the relevant communities much
earlier in the process, and at much more frequent intervals than we do today.
This is not rocket science, as such ongoing feedback loops can be achieved
through regular blog posts about work in progress, a targeted e-discussion at
an early stage, and frequent participation in external online fora to vet ideas.
But it requires that authors start seeing themselves not as isolated writers,
but as facilitators of a larger debate who are tasked to feed the essence of
that debate into their product. Authors who make a living of the actual impact of
their publications understand this, as you can see from countless books of business
advisors and speakers. Authors who are just hired to deliver a product for an
organization by a certain deadline (often without even being credited for it)
don’t have that incentive.

Are we at international organizations ready to change this? What
can we do to turn this pattern around and start thinking about the relevance of
knowledge product from the users’ perspective?

Tuesday, 22 April 2014

In November 2013 I got deployed for 3 months to Amman on a consulting assignment to
support the setup of UNDP’s Sub-Regional Response Facility for the crisis in
Syria. A key role of the Facility is to operationalize the Strategic Plan’s key
area of ‘Resilience’ in an environment of crisis by marrying the humanitarian
response for Syria with a development response. So far there has been a
primarily humanitarian angle to the Syria crisis, with OCHA, UNHCR, WFP and FAO
leading the response efforts in the region. UNDP’s interest in this situation
is to widen the perspective and highlight that there is a dramatic development
cost for Syria’s neighbor countries Lebanon, Jordan, Turkey and Iraq, which
deal with the largest refugee movement since the WWII. Given that most refugees
are not staying in camps are embedded in host communities with families and
friends, the host communities face a heavy strain on local services such as
access to housing, water, sanitation, health care, education and the labor
market, as well as on social cohesion more generally. UNDP finds itself in a
situation where it needs to explore new solutions to something that in this
scope and in this way hasn’t been done before. This calls for innovation, and
my job at the Facility was to help the facility to establish a KM and
Innovation framework and action plan to define what the Facility can do from a
knowledge and innovation perspective in the next two years, to help UNDP
implement a resilience-based response to the Syria crisis.

I have been following UNDP’s work on innovation, and I closely followed
the Global Innovation Meeting that took place in November 2013 in Montenegro,
including its outputs such the excellent Budva Declaration. Still, much of this to
me was rather theoretical and I didn’t have a lot of practical experience on
how to approach and manage an innovation initiative. So I contacted a number
UNDP colleagues who work on innovation and asked them to participate in Peer
Assists – a knowledge management methodology that brings together a group of
peers to elicit feedback on a problem, project, or activity, and draw insights
from the participants' knowledge and experience (to learn more about Peer
Assists, watch this excellent
6 min video here).

I was lucky to win Arndt Husar from the UNDP Global Centre for Public
Service Excellence, and George Hodge from the UNDP Country Office in Armenia
for a Peer Assist session I conducted in Dec 2013 to tap their brains on
implementing innovation initiatives, and I hope the following shortened
transcript of the conversation can be of as much help to others as it was to
me!

Johannes: I invited you to this conversation, because
you both have been involved in practical innovation events and initiatives in
UNDP in the past. My hope is to get to a better understanding of the conditions
under which certain innovation initiatives make sense and how we would plan for
something like this. Where would a team like ours here in Amman start? And what
would be the conditions under which it would make sense to have e.g. an
innovation camp in one of Syria’s neighbor countries to identify new solutions
together with municipalities and local actors?

George: I like that you are trying to incorporate different
approaches in addressing these challenges. The first thing I usually do is to
get out of the office and talk to municipal officials and the other
stakeholders in order to get a better sense of the problems. When running
social innovation camps and innovation challenges, the better you can define
the problem at the very beginning, the happier you will be about what happens
later in the cycle.

If you run an open innovation challenge where you ask “tell us about
your problems, and suggest solutions”, you get a better sense of where the
problems lie. Whereas if you want to address more concrete problems of your
stakeholders, then you should run an innovation camp or challenge around a
specific question.

I recommend visiting your stakeholders to get a sense of which public
services are under strain. Then you can run a series of concurrent challenges
asking “Can you come up with ways in which we can overcome this particular
issue”. But if you are at an early stage and you are trying to make sense
of the environment, an open challenge may be best. You should expect over 100
responses, and from that you will get a sense of where people see the most
pressing problems.

Arndt: That was great feedback from George. I think I
need more clarity on the scale: Is this something you want to do at an
inter-governmental level, or do you want to look for solutions in each specific
country? This will define your immediate counterparts and your outreach.
Municipalities are good, because they are the ones delivering your cutting-edge
services. You could break your issues in the sub-region down into national
challenges, and then go out and do the sensing with local partners. For me the
big question is the connection between that massive scale of four countries,
and the local services that you are looking at.

Johannes: To give you feedback on the scale, we are talking
more or less about three countries: Jordan, Lebanon and Turkey, because those
are the three countries that host the biggest population of refugees. They are
also at a more advanced development stage with larger middle income population,
so we may have a higher chance to get some of those technology solutions with
private sector off the ground.

Arndt: Ok, I think that’s very good. This sounds similar to
what we had in Singapore with our social innovation camps. We had activities in
multiple countries and at the end convened a regional summit where we brought
the various countries together to combine different country perspectives, which
worked quite well.

Regarding the conditions that have to be in place to organize something like
this, Mm experience in our regional initiative in Asia-Pacific was that the
outputs depended hugely on the local organizing partner and also on the network
that the local partner brought in. If you partner with some agency that has a
very specific urban network, then you get that kind of result. If you partner with
an agency that has a lot of techies in it, then you are going to get a lot of
IT solutions. So it’s really important to pick the right partners (ideally a
consortium of partners) so you can have a wider range of solutions.

Another lesson that we learned is that during the short duration of a three
day camp people really just scratch the surface. I think if you want to get
real solutions from municipalities to be innovative, you need to along the
lines George suggested: First the sensing, then the definition of the problem,
then the call, then probably some research (almost like the production of a
case study), and then you go into prototyping and so on. That’s a really
thorough preparation process, which we ourselves were not able to do through
just a social innovation camp. The above requires a bit more work.

George: What Arndt has just identified is why we – after
running a few Social Innovation Camps – decided to set up an Innovation Lab,
because we realized we needed a bigger support structure around our events. You
need to do the sense making first and then come up with a series of really
specific challenges - this will give you results that are much better aligned
with your mission.

After the first social innovation camp, we gave the teams grants and just
said “good luck, come to us of you have any problems”, but this
approach is too passive. By now we have a much bigger support structure where
we get our hands dirty with the teams and actively invest in the initiatives
that pass through the lab – much more like a incubator.

Johannes: So how many of these three-day social innovation
camps in Armenia did you run? And have they all been successful?

George: We have run four of them, and they were definitely
worth it. I would say if you run a three-day event, it is useful to look at
what is already out there of which UNDP is not aware, and identify people who
have a deep understanding of the social problems in question. Maybe there are
teams that are already established, but don’t know how to scale their activities,
in which case you could throw UNDP’s institutional weight behind an initiative
and scale it quickly.

Johannes: And then you can go a step further and turn that
to a ‘lab’ structure. What is a lab exactly?

George: It’s an incubator for social projects where we
mimic business incubators: Identify ideas, conduct rapid prototyping, get a
very basic product into the world, test it with users, and then look at the
feedback. Is this working? Is this gaining traction? If yes, let’s invest more!
And all the while you are giving the team access to mentors and design
workshops, and develop their capacity. With a lab you can test multiple ideas
and hypotheses at the same time, look for results, and then scale. This is very
different to traditional programming approaches. The lab in Armenia
attempts to solve big social challenges by harnessing citizens’ experiences,
insights and ideas alongside public services. It applies approaches like
horizon scanning, design challenges, user-innovation and service design.Johannes: And what does it take? How much does it cost and
how many people are involved?

George: Social Innovation Camps all-in-all cost about
$25,000 here in Armenia. The event team should include an Event Coordinator
working part-time, plus a full-time assistant, plus a full-time intern over the
course of four months. And all that a Lab does is to extend this team people on
an ongoing basis.

In terms of time you are looking at four months from the launch of the call
for ideas until the event itself. During the preparation phase the team goes
out running workshops, talking to people and developing ideas about the
problems or challenges. Once the applications come in, they are also looking to
find the experts that will complement the idea owners. For example, we had a
doctor approach us who said she wanted to digitize Armenia’s blood registry. Of
course she had no idea of how to build a database and how to build a web
interface. She was just a doctor who understood the problem and had an idea how
it could be fixed. So the organizing team then goes out and finds the extra
skill sets and people needed to build a crude prototype of the initiative (in
this case an online blood registry database) at the event.

Johannes: But even that is a lot of things to do in just
four months.

Arndt: Well, in your case we may need to rethink the whole
approach. For emergency response situations you need to get to solutions much
more rapidly. It is almost like ignoring what Design Thinking says: You have to
give it time through various iterations. But you are in a situation where you
need quick solution which you can rapidly scale up. Maybe we need to design a
process that can be more integrated into government and existing institutions,
so we skip the part of building a team. Instead we do stakeholder engagement,
and bring in NGOs, relief organizations and governments, host community and
refugee representatives and come up with something that can be immediately
picked up by the municipalities. Thereby you could crunch the time that you
would usually need for the incubation.

George: But then, it is still an untested idea, and the
whole point is to establish whether an idea works.

Arndt: Yes, you would still have to test that idea and
develop it further.

George: Another idea that builds on Arndt’s could be to do
an open call for existing initiatives that are already working on a pre-defined
problem. If they have developed a prototype which has generated results we can
help them scale up. You are basically looking to adopt alpha or beta products
in the late stages of development. You can apply the same skill sets that you
would need for a Social Innovation Camp, but you get to scale faster. The
challenge is finding these existing initiatives.

Johannes: How do you identify who you can actually talk to?
It’s not like you have in a Country Office a list ready with all the actors
that you could potentially approach on specific issues.

George: Well, I would start with whatever list you have,
and talk to them. First you talk about the problems they face and then
you ask them who else they know in their network who is good at addressing this
kind of issue, and contact them as well. Repeat this process until you find
existing initiatives or citizens who are addressing the problems in creative ways.

Arndt: In your case in Amman, the approach is really a lot
about user experiences, either from refugees that are coming from other
countries, or the host communities. Think of who has the best information and
can identify the crunch points for which you want to develop solutions. Then
interview people and collect stories, so you do the sensing exercise in that
particular environment that surrounds the issue you are looking at.

Johannes: In both the Armenia Social Innovation Camps and
in the regional camps in Singapore, who did that sensing. Was it both of you,
or did you hire consultants to do that?

Arndt: In our case it only happened in one camp, and I must
say, that we didn’t go with the standard Social Innovation Camp approach
because it didn’t do as much of sensing. It basically relied on the
participants, the idea givers, to do that. Some of them had done a decent job,
but a lot of them had not. The one camp that did this really quite well was the
one in Singapore where we had a local counterpart institution that had the
mandate to engage on this in the long term, and they did a fantastic job on
this.

George: In the case of Armenia, we did it in a team of
three: a consultant, an intern and me. The three of us travelled across the
country delivering the sensing workshops. Because as Arndt rightly highlighted,
you need to get a lot of different perspectives on the problem. I recommend
keeping the team in-house instead of giving it to contractors, so you can
develop and keep the knowledge, relationships and networks. Also, if you
outsource there can be issues of quality assurance. It takes a little bit more
work, but at the same time, it really supercharges your learning. We now have
an entire list of organizations and people with whom we would never have
developed relationships had we not run the event and the lab in-house.

Arndt: It sounds to me that if you want to do a three
country project, it will be a quite elaborate process. If you want to create
real value it make sense to invest some funds into it and have people do it
full time.

Johannes: Do you think it is better to start off with only
one country, rather than doing it in three countries at once?

George: It depends on how you are going to do it. If you do
it in-house start with one country and make sure you build the skills within
the team. But if you are going to outsource it, you might as well go big from
the start.

Arndt: I would tend to say go big, because for the
Sub-Regional Facility, that will prove your value.Johannes: Well, it’s not like I am going to do this
outreach myself, I am here on a detail assignment for another few weeks. But
what I need to do is to put the train on the rails and a plan in place so the
Facility can hire someone full time to take that plan and run with it.

Arndt: The other thing is: So far we only looked at the
option of Social Innovation Camps at the grassroots level. The other
possibility is to think of this rather as a public service initiative in
support of an innovation-type exercise. You can look at examples of Policy Labs
that were presented in at the Global
Innovation Meeting in Montenegro, and now again at our event
in Singapore, such as like the Mind Lab (Denmark) or designGov (Australia).
The latter is more a governmental initiative which also applies a design
thinking approach where it is government officials who do all the sensing. They
have almost a purely government-driven lab, but they hire service providers
which will accompany this process. I wouldn’t be surprised if some of Syria’s
neighboring countries had a few companies providing design thinking consultancy
services that you can hire to facilitate governments finding solutions. This is
different than just coming up with a solution from a UNDP office and then developing
a pilot, which is the traditional way UNDP does business. Instead, it means
that people really go down to the ground level, get a sense of problems, go
back and develop prototypes, test them out, and then scale up what works.Johannes: In the case of Singapore, did the government
already have something like that in place, or did they ask UNDP to help them
establish the lab?

Arndt: Here in Singapore the Human Experience Lab (THE) Lab is
a relatively recent initiative, while Denmark, Australia and UK have policy
labs since while. The Singapore lab is only half a year old, but they have been
practicing design thinking for a bit longer. Supporting such government labs is
possibly an alternative option for UNDP, although these modalities don’t necessarily
exclude each other. You could have a government-driven process going on, with
social innovation camps as satellite events. That would be really nice because
the innovation camps could feed into an official government-driven process and
you could have a dialogue between the processes. That might actually be even
better than each modality on its own. But of course you need more money for
that.

Johannes: From your experience, how many of these solutions
that come out of such modalities are IT-driven, like Web 2.0 or e-government
solutions, and how many of them are non-tech, like processes or policies?

Arndt: The social innovation camps I’ve seen were almost
all tech, but that is because we had specifically asked for tech. The output
from the government lab was mostly non-tech, such as project re-engineering,
cutting silos, cutting bureaucracy, finding new ways of communication with
citizens, etc.

George: From its origin, the specific objective of social
innovation camps is to apply Web 2.0 to social problems. So if you are looking
beyond tech then you are looking at innovation challenges or design teams.
Service design approaches do not involve crowd sourcing but can be a good way
of generating lots of ideas for alternative approaches to service delivery. You
certainly get non-tech solutions emerging from this. However, if you embrace an
open process and just collect ideas, it actually doesn’t matter whether the
solutions are IT-based or not. The point is: if you open up to many different
inputs, you get good ideas.

Arndt: I second that. It is also easier to scale if the
solution is non-tech.

Johannes: From all that you said, I understand that at one
point we will have to make that national , regional or global call when you put
the question out there who has a solution to a particular problem. How do you
actually plan for that?

George: From our experience here in Armenia, once you have
a sense of a problem, you can either do an open challenge where you say “tell
us about problems and suggest solutions” or a slightly more specific one “tell
us about problems in this sector and suggest solutions”, or you ask a
really focused question like “tell us about solutions for this specific
problem”. Once you have the question you build a small website and
integrate social media into it. Whatever the social media platform of choice is
in the country you are working in, focus on the top two. Then you have to go
into different communities, both online and offline, and starting talking to
people. You can’t just build a beautiful garden and expect people to come, you
have to go out and engage with people where they are. Our intern worked very
hard going into every single web forum in Armenia she could find, talking to
people and looking for activists with whom we could work. Just talking to them
at a personal level about the issues, and through that engagement process, a
lot of hits were generated on our website, and then turned into applications
for the event itself. You then complement that with a series of sensing
workshops, where you are targeting specific groups of stakeholders. In your
example you would talk to host communities, refugees, to state officials,
municipal officials – especially in front line service delivery, and
journalists. You invite them to a workshop where you ask them to define the
problem more clearly. Or if you have already identified the question or
challenge, you let them brainstorm on solutions. You also ask them “who do
you know who is already working in this”, so you can meet them and see if
you can support their work. From that engagement process you will get a lot of
connections and ideas. Then you take the best 5 or 6 ideas into a prototyping
event, much like a social innovation camp.

Johannes: Well, thank you so much guys, this all really
helps our thinking here a lot. We will have to develop an action plan on this
going forward. I will write this up, both in a blog but also in the form of
some concept note for the Facility. And I’ll keep you updated on how I can
inject those thoughts into what we are doing at the Facility, and we’ll be
happy to share how far we will have come with it in a few months’ time!

Monday, 31 March 2014

Following my earlier posts on my
assignment with the Sub-Regional Response Facility for Syria in Amman, Jordan, where
we identified the general directions for KM for this business unit, here are now
the details of the KM plan that I introduced, based on our earlier needs
assessment.

1.An Online Collaboration Space for the Facility, targeted
at the Regional Working Group and invited guests

To support of the Facility’s role as a broker, the creation of an
online collaboration space hosted by the Sub-Regional Facility will allow the
team to provide an online home for the Regional UNDG Working Group for the Arab
States/MENA to share draft papers and relevant resources on an ongoing basis. Even
more it creates a space to discuss questions and collect comments from colleagues
on the Facility’s ongoing work. In the spirit of ‘working out loud’ we will
also invite a number of selected colleagues from all UNDG agencies into this
space.

2.Establishment of a UNDG-wide Community of Practice on
Resilience-Based Development, including selected guests from academia

A key element of the Facility’s broker function
is fostering a Community of Practice of colleagues working
on resilience in context of humanitarian and development work. The Facility will use
the above space as a launch pad for e-discussions and ad-hoc queries and benefit
from the input of UN colleagues. A first formal e-discussion on
vulnerability criteria has already taken place in December 2013, and the Facility
will reach out to selected experts in the field of resilience, humanitarian
work, local governance, etc., followed by additional discussions and surveys
among the new community members in 2014.

3.Mapping of stakeholders for research on resilience and partners
for engagement on Resilience-Based Development

In order to identify relevant stakeholders that the Facility can
engage and work with in its role as a knowledge broker, a knowledge mapping
exercise is recommended. This will be targeted at two levels:

Mapping of potential members for the above Community of Practice on Resilience-Based Development for Syria.

Mapping of stakeholders for collaboration on research, innovation and substantive projects outside the UN system.

4.Exploration of
organizing social innovation camps in Jordan, Lebanon and/or Turkey to identify
and prototype e-governance solutions for a priority issue (e.g. local services)

UNDP’ experience in the Europe & CIS region suggests that there
is great potential in bringing together citizens, local actors and innovative
NGOs and companies to identify innovative solutions to local issues that would
benefit from the UN’s support in prototyping, testing and scaling up. This can
include the Social
Innovation Camp methodology and could also be precluded by a public
innovation competition to crowd-source practical solutions to local challenges
around resilience-based development. Depending on the evaluation of the
experience, this could be a precursor to widening the scope of audience and establishing
at a later stage an ongoing innovation lab facility in cooperation with one or
several of Syria’s neighbour governments, similar to Kolba
Labs in Armenia.

5.Series of targeted consultations on questions related to resilience
in context of a sub-regional development forum for Syria

In the process of identifying innovative solutions for operationalizing
a resilience-based development approach, the Facility should draw on input from
a larger audience. UNDP/KICG’s experience with large scale consultations
targeted at a large pool of external experts and interest communities, such as
the Rio+20 Dialogues or the Post-2016 Consultations have the
potential to add significant value to the above process. The objective would be
to inform the operationalization process of the resilience-based approach with
substantive experiences, suggestions and prioritizations from selected actors
in the sub-region, including government planners, public service workers, host
and refugee communities, as well as selected civil society organizations and
private sector companies.

6.Creation of data visualizations and infographics, to use
as communication, advocacy and capacity building tool

Once the resilience-based approach has been operationalized in more
detail, the Facility’s dissemination and capacity building efforts will benefit
the use of simplified infographics and data visualizations for communication
purposes. This is particularly suitable for data-heavy research findings that
allow for clear conclusions, narratives and calls to action. This
visualizations can then be used in print, video or online knowledge and
communication products.

While the focus of the Facility is on brokering and innovating,
there is still a natural need to disseminate results to at least the immediate
audiences the Facility is working with, such as UNDP and UNDG agencies. In order
to create visibility for the work of the Facility, to communicate results of
research, projects, events and initiatives, as well as to foster learning within
the Community of Practice, the Facility could host regular online webinars that
would serve as both learning and advocacy instrument.

8.Maintaining a regular blog about the Facility’s ongoing
work and results, to increase visibility and influence the general debate on
resilience

In order to influence the debate on Syria among stakeholders, and
ultimately influence decision making of development actors, the value of the
tool of blogging cannot be underestimated. Maintaining a regularly updated
public blog where different authors provide personal views and reflections
based on UNDG’s work can have a significant impact on framing the conversation
along the outputs, activities and objectives of the Facility. I order to do
this right, the Facility must be committed to publish at least two blog posts
per month. This can be reasonably achieved by rotating authors among the
facility’s substantive development experts.

9. Peer Assists, applied as needed to get input from peers
on internal tasks and challenges of the Facility

A Peer Assist is a KM methodology (see instructional video) that brings
together a group of peers (on site or online) to elicit feedback on a problem,
project, or activity, and draw lessons from the participants' knowledge and
experience. Peer Assists are useful when starting a new activity or project and
a team wants to benefit from the advice of more experienced colleagues, or
another group that has faced a similar situation in the past.

This methodology is targeted at the Facility internally to improve
its own work as a team. The Facility should map the planned projects for the
upcoming, determine which of the new projects and initiatives would benefit
from a formal Peer Assist, and then identify potential peer experts who could
be invited to participate. However, they can also be organized ad-hoc when
problems emerge that the team is not quite sure how to address.

10.After-Action Reviews, after key events or activities to
reflect on the ongoing work of the Facility and capture learning points

An After Action Review (AAR) is an internal
process used by a team to reflect on a recent activity or event to capture learning
points with the goal of improving future performance. The facilitator of an AAR
will ask the team three questions: “What was supposed to happen?”, “What
actually did happen?” and “What can we learn from it?” AARs can also be
employed in the course of a project to ‘learn while doing’. They should be
carried out with an open spirit and no intent to blame. The American Army, which
invented the methodology, used the phrase "leave your rank at the
door" to optimize learning in this process. The KM Specialist of the
Facility should constantly look out for opportunities to conduct AARs with the
team, which can be as brief as 15-30 min, even though AARs of up to 2 hours
might be suitable for reflecting on larger projects or events.

Below you can see an overview on how the different suggested KM action items
will be rolled out in 2014.

Tuesday, 21 January 2014

Following my earlier post
on my assignment with the Sub-Regional Response Facility for
Syria in Amman, Jordan, I'd like to follow up with the results of the
needs assessment that we conducted, and the resulting directions for
knowledge management that the Facility identified for itself.

Role and audience as prioritized by the Facility
After exploring the potential roles, audiences and challenges of the
Facility, its mandate, the management of the Facility defined the
Facility’s role and audience for 2014-15 as follows:

The Facility sees its knowledge management role in
brokering partnerships and exchanges, and facilitating innovation on the
issue of a resilience-based development in context of the Syria crisis.
It will do so by also facilitating, investing in and drawing on
data-driven catalytic research and development, however, it does not see
itself as a research institution.

In terms of audience, it will take an incremental
approach over time in which it will – through the work of the Regional
UN Working Group on Resilience – focus in the short-term on serving UNDP
Country Offices, then expand this work to UNCTs and UNDG agencies, and
later target the larger development community of governments, NGOs and
other partners.
The incremental approach to widen the audience of the Facility’s work over time is illustrated in the following figure:

Recommended Knowledge Management Activities for the Sub-Regional Facility
One concern raised by the management team of the Facility was that –
given the Facility’s limited team size –its KM approach should not be
too complex in order to avoid any capacity issues (recruitment of a
Knowledge Management Specialist position is planned for Q1 2014). With
the direction for knowledge management of the Facility derived from the
knowledge needs assessment, and keeping in mind the need for
prioritization due to capacity constraints, UNDP was then able to
identify potential KM initiatives that can support a knowledge agenda
focusing on brokering and innovating – as well as to some extent
research – and that targets first UNDP Country Offices and UNDG
audiences, as shown in this figure:

The following are the recommendations for knowledge management activities for the Sub-Regional Facility:

An online collaboration space for the Facility, targeted at the Regional Working Group and invited guests;

Establishment of a UNDG-wide Community of Practice on resilience-based development, including selected guests from academia;

Mapping of stakeholders for research on resilience and partners for engagement on Resilience-Based Development;

Exploration of organizing social innovation camps
in Jordan, Lebanon and/or Turkey to identify and prototype e-governance
solutions for a priority issue (e.g. local services), potentially in
combination with a public innovation competition to crowd-source practical solutions to local challenges around resilience-based development.

Series of targeted consultations
on questions related to resilience with staff across UNDG agencies, as
well as invited external guests from academia, international
organizations, NGOs and private sector partners.

While not immediately being in the focus of the Facility in terms of
role and audience, the following activities can also add value to its
knowledge agenda:

Conducting monthly webinars, to periodically
inform UNDP and UNDG stakeholders about the Facility’s ongoing work,
outputs and results, and foster learning among stakeholders;

Creation of visualizations and infographics,
to use as communication, capacity building and advocacy tool, packaging
evidence from research and results of the Facility’s work;

Maintaining a regular blog about the Facility’s ongoing work and results, to increase visibility and influence the general debate on resilience

Finally, to support the Facility’s internal work, the application of the following is also recommended:

Peer Assists, applied as needed to get input from peers on internal tasks and challenges of the Facility;

After-Action Reviews,
applied consistently after key events or activities of the Facility to
reflect on its ongoing work and capture learning points.

What could those priorities entail in detail? Expect a last blog post
elaborating on each of the initiatives, as well as some first results in
implementing this work plan!

Blogs I read

About this blog

The way we work and communicate is constantly evolving. With the evolution of knowledge management and social business, I find myself learning about new tools, approaches and strategies to KM and ICT for development in an incredible pace. I enjoy incorporating these learnings this into my work on KM, organizational effectiveness and change, but it also I also find myself spinning thoughts on the social and philosophical implications for society and human interaction in general. I realize that I am part of this tremendous change that is happening and I sense that I will not be the same person than I used to be. This blog is to reflect on this personal change, looking at myself diving into the flow of social and organizational change, curious where the waters will take me.