Crisismapping has never been just about Twitter feeds; it’s always been about data. But what data, and how do we know what’s useful? I’ve been looking back over 4 years of archived data to start answering that one.

In truth, I’ve been having a bit of an identity crisis. I see all the “big data” work on social media feeds, and although I can swing an AWS instance and the NLTK toolkit like a data nerd, for me personally, that’s not where the value of crisismapping has been.

It’s been about the useful, actionable data, and about connecting the people who have it with the people who need it. And whilst some of that data lies hidden in Twitter streams and Facebook requests, most of it is already on people’s servers and hard drives, often in formats that can’t be combined or understood easily.

So, some first things that make a difference every time:

Rolodexes: knowing which response groups to follow, and who’s likely to bring what helps. 3Ws are part of this – but before the 3W (who’s doing what where) is the “who’s”.

GIS data: knowing where medical facilities, schools, roads, bridges are makes a difference. Knowing what communications is available is important, so also knowing where cell towers are helps, but might be too coarse-grained: using signal maps to know which areas have cell coverage is often more useful. For me, mapping cell towers is problematic for the same reason that mapping military bases is problematic: they’re both potential sources of help in a crisis, but they’re both critical infrastructure whose locations are potentially sensitive information. But many maps include them (e.g. open signal map).

Demographics. Very useful data, but finding even population counts at sub-country levels can be difficult. They’re usually there (except perhaps in countries like DR Congo where surveying is difficult) but finding the “there” can be hard. I’d add technology and social media use to demographics, because there’s no point sniffing Twitter if only 0.5% of the country (and mainly expats) use it – there used to be sites available that listed, e.g. Facebook, Twitter etc percentages in each country, but they all seem to be behind paywalls now.

After that, it’s the emerging data: the 3Ws, the situation reports (both official, via news sources and on social media), the field notes about what’s happening.

We also now have 4 years of historical crisis data collected and collated by volunteers, often in areas prone to repeated crises, on top of the data already available through organisations and groups that existed before crisismapping was a “thing”. I’m not entirely sure what the value of that data is to the next crisis (like wars, every crisis is subtlely different), but it’s certainly worth working that out.

I’m sitting in the iHub Nairobi today, waiting for Space Apps Challenge to start. I’ve been in Kenya for a few days now, spending time with the amazing techs here, and getting more and more impatient for the first Brcks to go on sale (and not just because I’m about to be working in rural Tanzania). Because the one thing that holds people back here, and in many other “developing” countries, is connectivity. The ability to connect to the global community of information that the internet and all its pages, apps, and datastores offers without having those “I would get you that information but the power just went off / the internet just went down” moments. Brck is designed to help solve that type of connectivity problem. But there are other forms of connection happening across the internet: the human ones.

Back in 2010, Spike Foote and I spent 48 very long hours connecting people across the world working on similar problems (or the same problems) in different Random Hacks of Kindness cities. I still remember the shock from the iHub, and especially the AkiraChix, that people outside Nairobi were watching what they did, and wanted to reach out and connect to them. Yes, there are many comments about how effective the projects from global hackathons are and aren’t, and there are things that could be done better, like organizing global lists of for-good hackathons and projects done and to do (which is one reason that OpenGoodHacks was created), but the one thing that I always point to, that I always come back to, is that global hackathons create global hacker communities. These are the people who really solve the problems in a country – the people from those countries, working together with people around the world, sharing ideas. In 2010, I watched the Nairobi hackers teach US hackers about mobile phone programming. These are the kinds of connections that make us strong together.

And now, as I sit here waiting for Nairobi to start, whilst checking comments from SpaceApps sites around the world, in a space that has fostered many local tech companies and many great local hackers, I think about the challenges ahead for this globalized world. I am more aware than ever that we have to connect across borders and continents, and that the best way in for many many people is through global hackathons like this. That, and remembering to create low-bandwidth versions of the tech pages that people here need access to!

Here’s a copy of the draft results (i you want edit access to the real thing, just ask… and blame the spammers for this – they’re even targetting maps now) – my other unanswered questions include whether there’s been a drop-off, rise or steady number of new maps, and how the categories lists have changed over the past few years (I’ll put the scraper for that into github).

What’s Happened?

Past month: Deployments

What: South Sudan crisis

who: HOT OSM, GISCorps, Ushahidi, OpenCrisis, MapAction

where: South Sudan

what: OpenCrisis set up coordination skypechat and googledoc (http://bit.ly/1f9C457) for groups mapping the South Sudan crisis. HOT OSM has mapped several affected areas, in collaboration with UNITAR/UNOSAT, HIU (State Dept) with assistance from GISCorps and, for area identification, OpenCrisis (BOH). MapAction has also been working with OCHA. sending a two-person team to Juba for a period.

OpenCrisis platforms: Migration to WordPress almost complete (just link integrity survey and fettling to be finished off) – site now fully functioning.

Membership: 27 people on WP site (few more to be asked if want to be added), including 1st applications direct from new site; 24 on OpenCrisis Googlegroup; (37 on total list…but some of these have just cruised by once…and therefore probably shouldn’t be counted)

Twitter a/c: much more active over last 3 weeks – but could still be moved up notch

Personal Democracy Forum suggested that we submit a talk to them on our South Sudan dataset experience and learnings.

OpenCrisis involved in ongoing discussions (as observer, minor contributor) with HOT, State Dept’s HIU about mechanics of provision of sat imagery to community, workflows prior to HIU’s rollout of MapGive program

Other community activities

Humanitarian Data Project: getting going again – have started an online list of community-created datasets as part of a discussion on how to license and open these. Have also connected HDP to the Ushahidi CrisisNet project (which isn’t aimed at crisismapping, but does have some overlaps with it).

Accademics / SBTF Academics: Joseph launching room and collaboration space this weekend: “I’m in the process of setting a large group of academics for the purpose of creating joint university-level courses in Crisis Mapping, Crisis Management, Geo-Informatics. In addition I think this group is a great forum to discuss inter-institutional consortiums for the design of new approaches and technologies for humanitarians and civil protection. It’s another great way to meet the top academics committed to social good through technology. Join us by adding me on skype @ josephrichardpollack or sending me an email. Looking forward to your participation!” – Joseph.

Data handling guidelines – we need these,and SBTF have asked for some too. Who’s up for research and writing some? BOH is…

ODbL license: may be more restrictive than we (and HOT OSM) think. We need to investigate this asap (especially since all recent OC datasets have ODbL licenses on them), and see if we can get some legal help with it.

info4disasters website: Info4 is regrouping regarding “Country Pages” and EOCs in light of the “found” materials Sara pointed us to on the Wayback Machine (a great archive site for looking for “lost” internet pages)

WHS: info4disasters & SBTF working on the World Humanitarian Summit (deployment of DHN). Proposed that OpenCrisis assist Info4Disasters with part of their remit for this.

Cool tool: MapWarper.net, for aligning old maps with modern location-corrected maps

Has been ‘strongly suggested’ that OpenCrisis contribute to Personal Democracy Forum, June 2014, New York

OpenCrisis is now registered at ALNAP – meaning we get updates on events, jobs, opportunities in humanitarian learning field. “The Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP) is a system-wide organisation dedicated to improving the accountability and performance of humanitarian action by strengthening the humanitarian evidence base through sharing lessons, researching key issues and, where appropriate, providing leadership to find collective approaches and solutions.”

GDELT, the conflict data warehouse, is back! see here. And…it has a new home – here.

Sara – post about open data cambodia (but only with permission from team)

Sara – run training session on open data.

Needs

Blogposts for the new website – please write them!

Next month’s plan

By-laws discussion and drafting…and consultation…and adoption

ODbL licensing issues: We need to investigate this asap (especially since all recent OC datasets have ODbL licenses on them), and see if we can get some legal help with it. BOH to talk to Lillian; BOH to draft workflows that community can use with e.g., placename databases to ensure that is clear which data are OSM-compatible, and which not

I’ve recently been tasked with building and managing a Monitoring and Evaluation Team for an organization within the digital humanitarian response community. Although after-action reports have been created by many groups since at least 2010, Monitoring and Evaluation is a relatively novel and absolutely vital function in the digital humanitarian response space. We need to be monitoring and evaluating how our work-products are performing on the ground, so that we can optimize our workflows and increase the effectiveness of our products. In digital humanitarian response, the main goal is for us to provide timely and accurate information to field humanitarians (or affected populations) who are responding to save lives or aid crisis-affected communities in an emergency situation on-the-ground. Within this core function, we must also work to ensure that any of the information we release adheres to the “do no harm principle”.

To monitor and evaluate the effectiveness of our workflows and product utilization, we must start by taking a broad view of the humanitarian response ecosystem and probe to discover our relative position in that ecosystem. To get a visual idea, please see the infographic posted below. We need to ask questions about on-the-ground information product usage (by field response organizations), information product utilization and tracking (by other digital humanitarian response organizations), information product monitoring (i.e.; quality assurance, quality control), and M&E team function and efficiency (a project management and implementation task).

Beyond asking the right questions and evaluating responses, as part of building an M&E team, I have also researched categorical functions of sub-teams within a typical monitoring and evaluation implementation. As stated above, M&E is a novel function in the digital humanitarian space, and the technical nature of digital humanitarian response cannot be omitted from consideration when building a team. Borrowing from and adapting UNDP’s Handbook on Monitoring and Evaluation for Results, my initial recommendation is that an M&E Team in digital humanitarian response should have the following 4 functional groups:

Outcome Monitoring Team (An Implementation Role)

Outcome Evaluation Team (An Implementation Role)

Partner Engagement Team (An Implementation Role)

Monitoring Tools and Mechanisms (A Technical Team – Research and Development Role)

And from there moving forward, I have formulated a basic set of questions, the answers to which could be assessed to form some basic plans about how an M&E Team might best operate within the disaster response framework.

Last year, I published an article discussing the challenges to Documenting Rape in Africa which continue to hamper effective prevention efforts across the continent and internationally. In this general review, informed heavily by Caroline Muthoni Muriithi the Sexual Violence and Trafficking program officer at Equality Now in Nairobi, constraints on effective programming such as lack of reliable data on actual occurrences due to a complexity of social factors that inhibit reporting were identified. This discussion is not new, what is however are the many innovative and dynamic applications to track sexual violence and hate-motivated crimes through social media and collaborative mapping such as the Women Under Siege project, HarassMap, and Bachao. These and other efforts have increasingly been utilized by advocates seeking to strengthen the base of evidence available to practitioners. Which can be particularly useful in countries or regions where weak or ineffective public institutions lack quality assurance in data informatics. However, recent discussion on this site and iRevolution on the public availability and ownership of such data particularly in situations of limited statehood raise important questions for ongoing efforts to document these abuses.

As an example, the Being LGBT in Asia project is a crowdsourced instance that seeks to map the ‘Successes and Barriers to LGBT Rights in Asia’. This Ushahidi-based map tracks a wide diversity of social trends and population indicators across regional outlets, intended to provide an understanding of the challenges faced by Lesbian, Gay, Bisexual and Transgender people in Asia. This joint analysis is being undertaken by grassroots LGBT organizations and community leaders in the region, together with the United Nations Development Program (UNDP) and the United States Agency for International Development (USAID), utilizing a participatory approach to the documentation of legal, social and environmental challenges faced by LGBT persons across civil society institutions. In the context of such a project, who can be said to “own” this data and what potential conflicts of interest could arise between the international organizations providing technical assistance for the project and agents on the ground mired in the daily battles of this work?

Another map called the Geography of Hate, developed by Dr. Monica Stephens at Humboldt State University, drew data on 150,000 tweets sourced by the DOLLY project out of the University of Kentucky. Heat maps of hate speech were generated after researchers hand-filtered tweets categorizing them as positive, neutral or negative based on a pre-defined rubric. Negative messages were mapped to the county level to protect privacy and actual tweet locations, which highlights an interesting gray area of security concern. By blinding this data, one has effectively “protected” perpetrators of these crimes, an issue also identified in the proposal found here on Praxis’s Mapping Sexual Assault proposal site. This is an area of legal and policy research that is currently being prioritized across gender, public health, judicial and development fields.

The International Lesbian, Gay, Bisexual, Trans and Intersex Association (ILGA) has produced a comparative mapping on the status of sexual and gender minority rights globally. The map covers a range of social, legal and institutional indicators and its dataset provides researchers and advocates with a critical resource while also linking out to timely analysis of current trends across sectors. Such information can be one tool for lobbyist and development workers advocating for change, but organizations and digital activists also need to be careful when publicizing socially mined data that has the potential to be utilized by repressive governments to further human rights violations.

More notable examples of this happening can be found from lessons learned in the Arab Spring also touched upon by Brendan in his post on issues around the use and publication of Crisis Data in the context of South Sudan . Privacy concerns, which is an echo of security risk concerns faced by Volunteer Technical Communities (VTCs) operating in disaster settings more generally. A post last November on Anahi Ayala Iaccuci’s Diary of a Crisis Mapper blog covers this issue drawing on lessons learned in the context of Typhoon Yolanda (Haiyan) in the Philippines.

These experiences raise important issues on the implications on activists working to document sensitive human rights abuses that are sexual in nature or motivated by hate. Who defines a particular form of speech as hateful and what standards are used to support this valuation? Further, what can be done to ensure that data is not captured and then withheld from agents operating on behalf of survivors? As covered by Lillian Pierson in her upcoming post on Monitoring & Evaluation for Digital Humanitarian Response – a Novel Approach, there must be explicit efforts to address the unintended consequences in humanitarian crisis mapping and response to complex human rights and security issues.

In the interest of furthering informed critical dialogue and open debate on mapping sexual violence and hate crimes: interested readers and practitioners are encouraged to submit for the recent LGBTI in Africa: Call for Articles issued by the Pan-African social justice press, Pambazuka News, due February, 21st 2014.

This blogpost seeks to answer a number of honestly expressed concerns – which have been recently conveyed to us – about possible risks attached to the publication of different kinds of information from the public domain (principally Twitter names, radio stations and event locations) in a data resource doc for South Sudan that Sara-Jayne Terp of OpenCrisis set up before Christmas 2013 and which has since been curated by myself, Brendan O’Hanrahan.

We have examined these concerns in great detail and have sought and received the views of several humanitarian information and media professionals who are either based in South Sudan or were based there until the outbreak of this crisis,

Twitter Lists

We believe that there is a fundamental misapprehension at the heart of the concerns that have been raised about our work, particularly as it relates to Twitter: contrary to those assumptions, South Sudan is not experiencing a (so-called) “Arab Spring” or Iranian “Green Revolution” situation at the minute – rather, it is a minor civil war with ethnic undertones, which is essentially based on two factions jockeying for control of one party at the heart of what is, effectively, a one-party state. I wouldn’t classify anybody on the Twitter list contained within our resource doc as a proper social media activist in the sense one would use this term for other recent political crises: there’s one (very high-profile) rap artist in the diaspora who has strongly identified with one side, but somebody who appears on the BBC, Al Jazeera, etc doesn’t really need his identity protected. Then there are 3-4 people who might be categorised as peace activists – but while in some countries this might leave them vulnerable to persecution of some kind, there has been absolutely zero evidence of this for South Sudan. Indeed, if anything, my analysis would be that the peace activists have been more aligned with the status quo than against it – with more of their efforts focused on getting the rebels to abandon their rebellion…and being critical of the UN in a way that sometimes echoes regime criticism.

Twitter does not exist in some grey samizdat-style space; thanks to Twitter lists (not to mention blogposts like Lesley Anne Warner’s), anybody who wants to get hold of prominent Twitter accounts relating to South Sudan can do so in seconds. To illustrate, here’s a table of the most well-known Twitter lists for (or including) South Sudan.

List Name

No. of a/cs

Owner

Sudan/South Sudan

69

Gintare Eidimtaite

South Sudan

12

Quinn MacDonald

South Sudan

19

Rachael Akidi

South Sudan

39

Alun McDonald

South Sudan

26

Veronica Ledesma

South Sudan

72

Gabriela Jacomella

South Sudan

87

Geoffrey York

South Sudan

273

Brendan O’Hanrahan

South Sudan

24

Daniel Stuckey

southsudan

81

Clemens Lerche

SS

11

AbdulRahman

Sudans

90

Robyn Dixon

South Sudan

27

Paul Brown

Sudans

125

Lauren P Blanchard

SouthSudanCrisis

100

Carlo Pisani

Sudanese

169

John Fenning

South Sudan

47

Ben Byerly

R.O.S.S. Related

100

Christine Elizabeth

south sudan

55

Hosanna Fox

South Sudan

75

U.S. Mission in Juba

South Sudan

30

Akshaya Kumar

Sudan/South Sudan

164

Teresa Reiter

SouthSudan

41

Anna Bagadi

both sudans

142

sonjasugira

And, in the interests of shining more light about what sort of people actually populated the resource doc’s Twitter list, I extracted the following summary breakdown

South Sudanese citizens

No. in (or mainly in) South Sudan

Diaspora

Journalists/news outlets

Analysts

Humanitarian or UN

21

12

11

23

11

4

One minor point to highlight here is that the resource doc’s Twitter page has very few full names (this is simply because it was abandoned as a continuing project – because it was felt that public Twitter lists did the same job pretty much as adequately) – and those few nearly all relate to journalists and analysts. Though considering how conspicuous anybody on that list is, I really don’t see a problem with putting their full names there. It would save some evil mukhabarat-style agent 15 minutes of due diligence at most. A media and social media specialist normally based in South Sudan and with extensive experience of social media dynamics in East Africa concurred with our view that there were no vulnerable activists on this list.

Radio Station Lists

I am aware that radio stations (as well as several other kinds of media outlets) have been targeted in South Sudan – there is absolutely no question that South Sudan has become an increasingly hostile environment for journalists to work in over the last 18 months – see the many examples detailed on CPJ’s South Sudan page. However, again, I really, really doubt whether any of the information provided in our document would be necessary for any media suppression goons to do their job. From the government side all radio stations are licensed – so they have that information already…and from the rebel side, it does not make a plausible scenario that they would use our information to potentially target any radio operations when they have local knowledge already at hand. It should also be mentioned that there is already a detailed map of radio stations for South Sudan.

It’s important to stress that we deliberately did not keep any notes as to stations’ political alignment nor are any details about names of those working for them or their actual addresses given.

We have spoken to professionals working in the media development and humanitarian information spheres for South Sudan (two present in-country) – showing them the resource doc, and they have said that as long as no names or detailed addresses were published they saw no problems, since such information is widely available, or the radio stations are well known in their local areas.

Placenames and associated information

The source of over 50% of the named locations (and all of the IDP locations) in this doc is the body of humanitarian updates issued, principally, by OCHA, as well as IOM and USAID. See e.g., this and this, so any condemnation of this ‘loose’ information policy is effectively extended to them. 95% of the remainder of the locations in the placenames and coordinates sheet originate in news reports, or from Twitter updates from trusted journalists – and the vast majority appeared in several different news sources. Many of the placenames first mentioned in news reports subsequently appeared in the maps of violent incidents produced by USAID, e.g. this one for 27 January. The remainder are either from otherwise researched and credible sources or are included for assistance in geographical orientation.

While there is no direct verification protocol, only reports from trusted sources have been included (and since I have been following this crisis from close to the beginning in daily detail and been in dialogue with journalists and analysts who know the situation intimately, I think I am relatively qualified to make judgements as to who is trusted and who isn’t) – and if there was still some doubt attached to them, this is mentioned in the notes column. There were no attempts to categorise or otherwise analyse the events mentioned. This is because while it was thought that there would be organised attempts to map the violence and the humanitarian crisis, in practice the only non-OCHA, IOM, Enough Project or USAID mapping initiative was an abortive one, which was suspended after 2-3 days close to Christmas. The purpose of the placename map therefore evolved to be a resource for either any media people who might seek to know where particular places were, or as the basis for assisting humanitarian mapping projects. One example of this were the maps I produced to help HOT in their identification of some of the priority areas for their South Sudan volunteer mapping response (in support of UNITAR/UNOSAT). It was never, and this is important to stress, intended as the basis for an event map or a violence map – the notes on what put that place “in the news” were just intended for context.

Which brings us to the specific claims that the mentioning of these places brought inherent danger with it to any people staying at those locations. The claim about people being put in danger because the location of a refugee camp is given doesn’t really withstand proper scrutiny – these refugee camps are BIG. All of them with the exception of those at Minkamman and the UNMISS compounds are existing entities which are extensively documented (and mapped – see especially UNITAR’s collection of maps for South Sudan here) all over the Internet, and it simply wouldn’t be possible to come into an area and talk to local people without becoming aware of them. The new ones have also been extensively publicised and mapped elsewhere (again, especially via UNITAR). Neither does it make any sense that mentioning e.g., a village which has been raided would make that village vulnerable to another attack in some way.

We spoke to a prominent humanitarian information officer in South Sudan and they said that they had seen no problem in mapping IDP locations and expressed no concern about mapping other locations included in the database.

We have honestly tried to conceive of (realistic!) scenarios whereby some bad actors would use this document to plan or inform their malfeasance, but we honestly couldn’t find any that belonged in the realm of commonsense.

Circulation

We have only recently put this up on the OpenCrisis site (although frankly you wouldn’t find it unless you knew to look for it), and the version there is published without any notes on the events associated with the placenames (though this is mainly because the metadocumentation of my agonising about where difficult-to-find places might be was considered to be probably rather tedious to casual visitors) and apart from the mention on Twitter of this version of the doc, it has otherwise had no circulation except to those involved directly in crisis mapping or media coverage of the crisis (e.g., OCHA in South Sudan, BBC, a couple of freelance journalists, MapAction, HOT). But, again we in OpenCrisis would like to stress that we see no a priori reason why individuals with legitimate interest in events in South Sudan should have less right to this information than those with organisational credentials. We believe in open data – while absolutely taking all due care that no harm is done – and by making these data available we hope to make a contribution, however small, to the deeper understanding and analysis of the complex recent events in South Sudan.

Which brings us on to the final and most important point – whether we have been sloppy in observing humanitarian criteria and being mindful of the safety of those who may be affected by the information in our document.

As the guidelines on the first page of the resource doc state – the overriding principle is to “do no harm”; and although it originally said “don’t identify people in-country,” it was felt that this was better and more practically phrased as “don’t give any details about people which may result in they (or their relations) being put at risk,” while maintaining the crucial first principle. Thus, we are extremely confident that nobody in any danger has been identified, i.e., all those named here are prominent on Twitter already (two possible exceptions are completely apolitical) and no personal information has been attached to either mappable data or radio station details. And, while verification of events associated with placenames has been limited to ensuring that only reports from trusted sources (e.g., OCHA, USAID, Radio Tamazuj, Gurtong Trust etc.) are noted, we feel that this has been to an appropriate level in view of the degree of risk attached to the publication of these names. We should have had a note explaining this for the document before, but we have now rectified this by adding such a clarification to the first page.

So, while acknowledging that it will not always be easy to reconcile the twin aims of making useful crisis data widely available and ensuring that no risk is thereby entailed, we honestly think that, at least for this project, we have managed to achieve a sensible balance.