Month: February 2013

i admit that i am working from an n of 2, purposively selected from Ghana and Nigeria. nevertheless, it seems worth asking, how is national history *not* part of the standard middle or high school curriculum, insofar as knowing history is an important part of educating citizens?

Like this:

@edwardcarr, i also have a confession, which is that i have a small crush on you right now for the post in which you make a confession about causality and try to disentangle causes, mechanisms, and information that can be used to understand, revamp, and scale programs. i‘d like to try to tweak the argument, especially in light of the recent excitement about the publication of null results from a bombay-based cluster-randomized trial on health and pregnancy, described here.

the tweak is to separate out two categories of information that are important and may be better gleaned from qualitative inquiry and analysis, though more adaptive/iterative and process-focused quantitative data can also play a role. i think the general mindset on what constitutes (and who has) useful and useable information is as important as the difference between quant or qual analysis.

1. how the program actually brought about an effect. this point is the focus of ed’s post, as well as levy paluck’s nice paper on qualitative methods and field experiments, which distinguishes between causal effects and causal mechanisms. mechanisms can, to some degree, be pursued and examined through ever-proliferating treatment arms in RCTs… but observation and purposive, systematic conversation are very helpful. ifi read ed carr’s piece correctly, he wants more (data collected with an eye towards) explanation in order to pursue (a) a deeper understanding – beyond ‘story time’ – of what moderates and mediates the (potentially causal) relationship between X and Y within the study context, to also help us gain (b) a deeper understanding of the external validity of the findings, which could inform adaptation and replication. both are important goals for studies with any intention of scaling.

2. how the program was experienced by a range of stakeholders and how it could have been done better. this part doesn’t feature in ed’s post or levy paluck’s piece but is important. sometimesi feel like when i talk about process, everyone breaks out in log-frame hives. take a deep breath. i don’t just mean process checklists and indicators. i mean recording how things went, deviations from the study design, and seeking feedback from study participants, study facilitators, study staff, and other study stakeholders. in the bombay experiment referenced above, the team had a process evaluation officer, who consistently surveyed staff and documented meetings with the participants. these data allowed the researchers to know, among other things, that the participating urban women “balked” at collective action but were happy to share information one-on-one — a fairly useful finding for anyone else designing a program with similar goals or in a similar population. i think the researchers could have gone slightly further in asking participants about possible explanations for the similarity of outcomes in the treatment and comparison group — but the centrality of process evaluation is clear nevertheless. in a similar vein, camposet al draw lessons from experiments that didn’t happen and propose that researchers need to “work more on delivery and better incentivize project staff.” this, too, suggests a need to better collect (and use) information on delivery process and staff perceptions of projects, which means making time (and setting aside money) to solicit this information and finding ways to incorporate it into study findings. it also means taking program design as seriously as experimental design.

in sum, explanation matters. collect data that allows for better explanation of the mechanisms underlying causal effects as well as the process by which those mechanisms were put in place. in the meantime, everyone needs to do more work on figuring out how to present these types of data in a way that is easily accessible to and valued by a variety of researchers and practitioners.

Like this:

that’s the title of a short article in the jan/feb 2013 atlantic— and i couldn’t think of a better one.

i have written previously about the joys of getting and giving directions in lower-income countries – specifically for research and household follow-up, although the general taxi/auto/tuk-tuk stories of trying to reach any specific location purposefully are equally fun (in hindsight).

after reading my initial post, at least one friend reminded me that people at home (in the US) aren’t always so good at directions either, too familiar with a route to think about landmarks or to remember street names, and already too accustomed to google maps & similar being able to get the job done. theatlantic article from the title, about westvirginia, re-emphasizes, for one, that a lack of street names, the use of landmarks, etc, is hardly only a poor-country phenomenon – rather, that “addresses have historically been an urban commodity” and one that probably belonged to highly literate urban areas with people who moved around the city a good deal.

formalizing addresses is more important than the inconvenience of trying to find a location or getting mail delivered. it is also essential for emergency services to find you and is presumably useful for tax collection and other basic services of the state.

which brings us to the second important part of the article: west virgina relied both on 911-services and a deal with verizon to get the mapping and road-naming word underway. knowing the power and visibility of mobile companies in many low- and middle-income countries, would this not be a reasonable way to move the task forward? of all the potential projects for m-dev (e.g. and here, h/t tompaulson), it seems to me that mapping, paying taxes, and vital registration are some of the most promising and fundamental – as well as good public-private ventures. these would be fairly top-down and possibly foucauldian projects, and may be faulted for that, but i think we need more thinking about how the state can connect with its citizens.

finally, the atlanticarticle also points out the fun/difficulty of coming up with that many new street names. on absurd street names presumably combined by some random generator (although the linked article points to a single woman), i think my parent’s town has to take the cake.

drive-by truckers (highway 72):

“Don’t know why they even bother putting this highway on the mapEverybody that’s ever been on it knows exactly where they’re at.”