Sunday, December 16, 2012

They say that three times is a trend, so I will have to make do with two thirds of a trend regarding Nicholas Kristof's use of social science research. Or more precisely, regarding Kristof's use of research to back what he presents as inconvenient truths regarding the behavior of poor people or the efficacy of attempts to help them.

Instance 1: Kristof is currently taking a lot of heat from liberal social policy experts for using anecdotal evidence to suggest that significant numbers of parents of children receiving Supplemental Security Income (SSI) deliberately keep their children illiterate to avoid losing the child's disability check. More precisely, Kristof uses anecdotal evidence supported by statistical sleight-of-hand.

Harold Polllack rather gently points to the questionable deployment of data. I'd like to elaborate a bit. Pollack's case first:

Kristof’s essay left the impression that welfare dependence is a
growing, chronic problem fed by ballooning SSI rolls. As he put things,

More than 1.2 million children across America — a full 8
percent* of all low-income children — are now enrolled in S.S.I. as
disabled, at an annual cost of more than $9 billion.

That is a burden on taxpayers, of course, but it can be even worse
for children whose families have a huge stake in their failing in
school. Those kids may never recover: a 2009 study found that nearly
two-thirds of these children make the transition at age 18 into S.S.I.
for the adult disabled.

I tracked down the 2009 study that
Kristof had read. It’s by Jeffrey Hemmeter, Jacqueline Kauff, and David
Wittenberg....As Kristof reports, about two-thirds of youth on at age 17 are
recertified at age 19 for the adult SSI program. Yet I read these
numbers differently. Indeed the study’s other main finding is that
“youth with behavioral disorders and mental disorders other than mental
retardation are much less likely to receive SSI at age 19.” Moreover,
the authors express concerns that these youth who do not transition into adult SSI do poorly.

It seems to me that the childhood SSI program runs a reasonably tight
ship. Most children deemed eligible for childhood benefits have
identified, serious, and chronic impairments that make them SSI-eligible
when they are older. Youth with the fuzzier conditions that Kristof
questions are precisely those most likely to leave the program at
adulthood.

Pollack is a little too polite regarding the mis-deployment of this research (though I would echo his caveat: Kristof "has risked his life to report on many profound global health problems
and violations of human rights. I admire him greatly for that)." Regarding those "nearly two thirds" who transition into adult SSI: in the study in question, 49.6% of the children were mentally retarded. Another 14.8% suffered from "systems and sensory diseases," which include "diseases of the nervous system and sensory organs, and systemic disorders (i.e., diseases of the circulatory, digestive, genitourinary, musculoskeletal, and respiratory systems)." Another 9.8% suffer from "other disorders," e.g., "infectious and parasitic diseases...diseases of the blood and blood-forming organs" and others. Just 25.8% are diagnosed with "mental and behavioral disorders" -- Kristof's "fuzzier conditions" (which Pollack suggests are not so fuzzy'- qualification on this basis is difficult).

Kristof complains that at present, "55 percent of the disabilities [child SSI] covers are fuzzier intellectual
disabilities short of mental retardation, where the diagnosis is less
clear-cut." If so, the study he cites to demonstrate that too many of the covered children end up on adult disability is out of date: notwithstanding his use of its 2009 date to suggest that it captures near-current unfortunate changes, it analyzes data from 2001-2002. Insofar as it is not out of date, however, to elaborate Pollack's point, those in Kristof's "fuzzy" category -- are the least likely beneficiaries to transition to adult SSI. 46.8% do so, compared to 82.8% of those with "systems disorders" (physical diseases), 67.5% of those with mental retardation, and 64.9% with other disabilities.

As Pollack points out, the study authors focus most of their concern on the 53% of SSI children with mental and behavior disorders who do not transition to adult SSI. High percentages of this group had been in trouble with the law or suspended from school, and while higher percentages than in other groups had earned $2,000 or more as teens, only 25% earned as much in the year after they turned 18 than they would have received on SSI. Those with mental and behavioral disorders, in sum, were likeliest to have been in trouble with the law, to have made relatively substantial amounts of money, and to have gone off SSI when they reached adulthood. The authors conclude:

Of particular interest are the relatively poor outcomes of child SSI recipients with other mental and behavioral disorders who, relative to others, have both higher rates of problem behaviors prior to age 18 and lower rates of adult SSI participation after age 18. While we cannot assess the efficacy of the age-18 redetermination in influencing outcomes, our findings do indicate that many child SSI recipients do not develop their human capital sufficiently to make a successful transition to adulthood. Few of those no longer receiving SSI at age 19 had enough earnings to offset their pre-redetermination SSI benefit.

The findings suggest that additional research is needed to determine whether effective intervention options can be developed to better prepare and support those who do not continue receiving benefits after the age-18 redetermination. A substantial share of child beneficiaries will not qualify for adult benefits. Many of these youth have limited employment prospects and high rates of reported social problems.

Those conclusions do not contradict Kristof's claims. And it may be true that the incentives built into the children's SSI program work perversely for parents in some circumstances. But the study does not suggest that a disproportionate amount of recipients with mental and behavioral disorders move on to adult SSI. And if 55% of current recipients are in the mental and behavioral disorder category, I'd like to know where Kristof got that number. The composition in the study he cites, based on 2001-02 data, is far different.

* While Kristof claims that the 1.2 million children on SSI represents 8% of low income children, this 11/28/12 letter from 85 aid and advocacy groups to Obama, thanking him for preserving funding for children's SSI, claims that the approximately 1.3 million beneficiaries represent 4% of low income children.

Instance 2 of a dicey data deployment occurred in a May 2010 column in which Kristof suggested that the poor in many countries need to spend less money on alcohol and tobacco and more on education.. Then, too, the study providing principle support did not say quite what Kristof implied it did, as I noted at the time:

Kristof's basis is largely anecdotal -- families he interviewed in Congo
Republic. But he calls for backup from a study, to which he helpfully
provides a link:

Two M.I.T. economists, Abhijit Banerjee and Esther Duflo, found that the world’s poor
typically spend about 2 percent of their income educating their
children, and often larger percentages on alcohol and tobacco: 4 percent
in rural Papua New Guinea, 6 percent in Indonesia, 8 percent in Mexico.
The indigent also spend significant sums on soft drinks, prostitution
and extravagant festivals.

The results Kristof pulls from
this study need some qualifying. First, Banerjee and Duflo reported
findings from thirteen countries, with large variations in spending
habits among them. It's true that Banerjee and Duflo find that the
extremely poor in a range of countries spend between 4.1% and 8.1% of
their income on alcohol and tobacco, and just about 2% on education
averaged across the countries studied. But: "The reason spending is low
is that children in poor households typically attend public schools or
other schools that do not charge a fee" (9). Where fees are charged, the
percentage of household income spent on education rises: it is 6% in
Cote D'Ivoire. The study did not include Congo Republic, but one might
imagine that since fees are ubiquitous there, school spending would also
be relatively high.

In the 13 countries studied by Banerjee and Duflo, moreover, "food
typically represents from 56 to 78 percent among rural households, and
56 to 74 percent in urban areas" (5). Perhaps one tenth of the average
poor family's consumption budget goes to alcohol and tobacco, then --
and these averages cover large differences between countries and
communities. Families in the study also spent an average of 7% of their
food budget on sugar, and 10% on sugar, salt and processed foods.
Kristof might as well have lamented that fact -- though who wants to
live without sugar and salt?

A 750-word column is a tight space. The research cited in each of these cases is complex and not easily boiled down to policy prescriptions. In each case, the data might be read to supply an equivocal support to Kristof's claims (in the SSI study, at least, that the current constellation of services available to children with mental and behavioral disorders could be more effective). But neither study lends the particular support he claims for it.

About Me

I'm a freelance writer and media consultant with a lasting interest in how democracy works, how it malfunctions and self-corrects. Since fall 2013 I've focused increasingly on the unfolding drama of Affordable Care Act implementation and health reform more generally.
I have a Ph.D. in medieval English literature and a propensity to parse the rhetoric and logic of our political leaders as well as that of media pundits and scholars who jump into the national debate. I wrote a dissertation on the remarkably humane and subtle medieval English anchorite Julian of Norwich, a mystic nun whose knack of squaring circles and framing paradoxes reminds me a little of our current president. A sampling of that work (mind the google gaps) is here: http://bit.ly/OzwsrR