Study: Nope, government pre-K programs still not actually helping kids

posted at 10:01 pm on November 20, 2013 by Mary Katharine Ham

In January of this year, the federal government finally released a long-awaited study of Head Start’s impact on the low-income children it’s meant to help. The impact was pretty much none. As I wrote in January:

For the second time in two years, a Congressionally mandated study of the Health and Human Services program has shown it does not work. Its published date is October 2012, but it wasn’t released until after Obama’s election, presumably so Democratic campaigns could safely accuse Republicans of not caring about children and ignore the actual results of the programs they favor…

The theme of this evaluation is “no statistically measurable effect,” and what tiny positive effects there are among subgroups in behavioral and parental improvements are outweighed by statistically measurable harmful impacts in others. This is not a wise way to spend billions of dollars.

Because this program is for low-income children and families, it’s easy to demagogue. All of its critics just hate low-income children, the speeches go, making it rather hard to criticize the program. But the reality is, when you’re taking billions out of the federal budget and out of American citizens’ budgets to prop up a program that has been shown repeatedly not to work for the very people who need it to work, you are doing a disservice to those children. It is not enough to pass legislation that funds a giant federal program if that program does nothing to help the people you claim to be helping. It is not compassionate to perpetuate that program when scientifically rigorous studies done during a Democratic administration have shown it does not help, and sometimes hurts. There are a thousand private entities doing work on education, social-emotional development, and parenting skills in low-income communities that would make better use of a 1/100th of Head Start’s budget than Head Start is. Head Start’s allotment in Obama’s 2013 budget is $8 billion— more than Twitter’s entire net worth and four times the worth of Pinterest, for some budgetary perspective. Head Start was upset there wasn’t more “investment.”

You know, study after study shows that the sooner a child begins learning, the better he or she does down the road. But today, fewer than three in ten 4-year-olds are enrolled in a high-quality preschool program. Most middle-class parents can’t afford a few hundred bucks a week for private preschool. And for poor kids who need help the most, this lack of access to preschool education can shadow them for the rest of their lives.

So, tonight, I propose working with states to make high-quality preschool available to every single child in America.

Making high-quality preschool available to every single child in America. So, what would we get for that money? And, more importantly, what do students get? A new study of Tennessee’s Voluntary Pre-K program suggests, once again, the answer is not much.

Like HHS’s Head Start Impact study, the study of the TN-VPK program is a randomized trial, which meets the highest standard for scientifically evaluating programs. The study followed 3,000 4-year-olds starting in 2009. A state lottery system determined which children of the 3,000 who applied for pre-K were admitted to the program. Those who won the lottery became the pre-K group and those who lost, the control group to which they were compared. Three quarters of those in the control group did not end up in a pre-K program, so you’re looking at a pretty close approximation of state pre-K vs. nothing. A subset of 1,100 students were tested as they progressed from pre-K to first grade to determine what gains, if any, stuck around, as pre-K proponents claim they do.

An intensively studied subset of about 1,100 children drawn from both groups was directly tested on cognitive skills, such as knowledge of vocabulary, at the beginning and end of the pre-k year and at the end of kindergarten and first grade. These same children were rated by teachers at the end of first grade on a number of non-cognitive characteristics, such as the ability to play well with other children and work independently…

The research team previously reported positive impacts on cognitive measures favoring the TN-VPK participants at the end of the pre-k year. The recently released findings are with respect to how TN-VPK participation affects children’s later performance in kindergarten and first grade. The whole justification for investing in pre-k is that it provides long-term benefits, so these follow-up data are critically important.

The relatively large effects of TN‐VPK on the Woodcock Johnson achievement measures found at the end of the pre‐k year were greatly diminished and no longer statistically significant at the end of the kindergarten year. The only exception was a marginally significant negative effect on Passage Comprehension such that nonparticipants had higher scores at the end of the kindergarten year than TN‐VPK participants.

Similarly, at the end of first grade, there were no statistically significant differences between TN‐VPK participants and nonparticipants on the Woodcock Johnson achievement measures with one exception. There was a significant difference that favored the nonparticipant group on the Quantitative Concept subscale.

These diminished effects were not entirely unexpected in light of the findings in other longitudinal studies of the effects of early childhood programs on economically disadvantaged children. For preschool programs, a typical finding is that the cognitive effects are not sustained for very long after that initial year.

Aside from the fact that it sounds like a 13-year-old boy made up the name of the achievement test, this is a sobering assessment of a program very much like the kind the president would like to extend to all American children at great expense.

Whitehurst puts it in graph form. A shorthand explanation: If the bar is below the zero line, the control group of non-pre-K kids beat out the pre-K kids. If it’s above the zero line, the pre-K kids outperformed. Ouch:

In only one area of testing did pre-K kids outperform the control group, and the only statistically significant result was a negative affect on Passage Comprehension (Correction) Quantitative Concepts. Heckuva job, pre-K.

As with the Head Start Impact Study, one can find tiny, tiny bright spots in this research, but one would have to be a pretty determined pre-K proponent with a bad case of confirmation bias. (I’m acknowledging and attempting to inoculate myself from the same, by the way, hence the Brookings scholar.) I’ll leave you with Whitehurst’s words, which echo mine from January.

I see these findings as devastating for advocates of the expansion of state pre-k programs. This is the first large scale randomized trial of a present-day state pre-k program. Its methodology soundly trumps the quasi-experimental approaches that have heretofore been the only source of data on which to infer the impact of these programs. And its results align almost perfectly with those of the Head Start Impact Study, the only other large randomized trial that examines the longitudinal effects of having attended a public pre-k program. Based on what we have learned from these studies, the most defensible conclusion is that these statewide programs are not working to meaningfully increase the academic achievement or social/emotional skills and dispositions of children from low-income families. I wish this weren’t so, but facts are stubborn things. Maybe we should figure out how to deliver effective programs before the federal government funds preschool for all.

It is not compassionate to use billions in taxpayer money that could be better used by those who might actually get results. How many private tutor sessions could more than a hundred billion buy? You are not serving those you wish to serve by simply saying so.

Exit quotation: “Poor children deserve effective programs, not just programs that are well-intentioned.”

Breaking on Hot Air

Blowback

Note from Hot Air management: This section is for comments from Hot Air's community of registered readers. Please don't assume that Hot Air management agrees with or otherwise endorses any particular comment just because we let it stand. A reminder: Anyone who fails to comply with our terms of use may lose their posting privilege.

Based on what we have learned from these studies, the most defensible conclusion is that these statewide programs are not working to meaningfully increase the academic achievement or social/emotional skills and dispositions of children from low-income families. I wish this weren’t so, but facts are stubborn things.

There is a referendum on the Memphis, TN ballot tomorrow to raise the sales tax by .05% “for the children” to 10%. Pre-K is nothing more than a dumping ground by people who don’t want to take care of their young kids, teach them at home, and can’t afford daycare.

There is a referendum on the Memphis, TN ballot tomorrow to raise the sales tax by .05% “for the children” to 10%. Pre-K is nothing more than a dumping ground by people who don’t want to take care of their young kids, teach them at home, and can’t afford daycare.

simkeith on November 20, 2013 at 10:32 PM

the same thing happened in San Antonio. It was sold as “aw its just peanuts, c’mon you wingnuts, it’s just a fraction of a percent….” but you’re exactly right in your assessment–it’s the government acting as baby daddy.

Kids are not going to learn unless there are readers and books in their home. We did not own many books but my mother would take us to the Bookmobile. a library in a bus, that stopped by our neighborhood once or twice a week. Reading builds up the “muscles” in the brain that can then be used to accomplish almost anything.

Ben Carson said that his illiterate mother would assign him book reports and pretend to grade them to make him into a reader. After awhile, she learned also.

My family’s “high quality pre-school program” is called parenting. Showing them how to read starting at age two, describing numbers and their behaviors as soon as they can grasp. When my daughter was somewhere between 2 and 3, my only child at the time, she was given a plate of hamburger patties to bring to me outside at the grill. She saw the 6 formed hunks of meat, looked up at me, and said, “That’s two for each of us!”. I can take a hint, so the next day, I began asking her to figure out the “missing number” in equations like “5 whats make 10?”, and when she started getting bored with that, explained variables to her and really started. She didn’t need to realize that she was learning at age 3 and 4 what most kids don’t get until 7th grade (or later).

The same was true for reading and “citizenship”. Now she’s in her third year teaching kindergarten and coaching high school cheerleading, and the first grade teachers at the school complain that they gets kids who already understand their entire curriculum.

and the only statistically significant result was a negative affect on Passage Comprehension.

The largest bar on the chart is Quant. Concepts, which I’m guessing means Quantitative, not Quantum, at a delta of around -0.21%. Passage Comprehension is actually the righthand bar of the Language group, somewhere between -0.05% and -0.1%

The largest bar on the chart is Quant. Concepts, which I’m guessing means Quantitative, not Quantum, at a delta of around -0.21%. Passage Comprehension is actually the righthand bar of the Language group, somewhere between -0.05% and -0.1%
Freelancer on November 20, 2013 at 11:15 PM

Effect size and statistic significance, though related, are not the same. It is possible to have statistical significance for an effect size that is smaller than one that is not statistically significant. Depends on sample size (which is probably roughly the same here) and within group variability. If MKH made a mistake, so did the person she quoted.

I knew one of the original designers of Head Start, James Payne, when he was on faculty at UVA. He admitted the program had never delivered the results it promised, but he had a list of reasons (excuses) for it.

Four decades later, they’ve tried everything and spent like drunken sailors trying to make it work, with no visible results at all. It’s at best federally subsidized day care. And some studies show the children in Head Start actually are worse off by 1st grade than those who had no pre-K program at all.

Mary Katharine, it doesn’t MATTER what the data shows. This administration is intent on tracking students from cradle to grave. You need to start them in school asap so the government can start the tracking of human capital.

I haven’t read all the studies, but I have been a children’s librarian who had a weekly assignment to read to the local Head Start facility.

The Head Start instructors were pleasant, obviously caring, and just about competent enough to deliver instruction on the level of most high-school- graduate lower-middle-class parents, while decidedly inferior to most of the teachers in my (and my kids’) elementary schools, as well as most of the serious home-schooling parents of my acquaintance (the ones who actually did the work, not the ones just too lazy to get the kids up in the morning).

It seemed to me that their skill sets were, in fact, more on par with regular non-academic child-care workers than with educational faculty.

The theme of this evaluation is “no statistically measurable effect,” and what tiny positive effects there are among subgroups in behavioral and parental improvements are outweighed by statistically measurable harmful impacts in others. This is not a wise way to spend billions of dollars.

It’s hard to see how the children were harmed by getting decent meals, some social interaction, and basic educational instruction, but if they were being brought up to the same academic level as non-disadvantaged children at the start of KG, I would be surprised.

“Poor children deserve effective programs, not just programs that are well-intentioned.”

“Poor children deserve effective programs, not just programs that are well-intentioned.”

The problem for Dim politicians is that “effective” programs for poor kids would be programs designed to do things like encourage “baby daddies” to marry the mother of their children and stay in the kids’ lives.

First, we should ask what group is intended to benefit from this federal program? It is not good enough to say “kids from poor families”. Let’s be honest and say that the target group is really kids from inner-city, one parent homes in high crime areas. I realize there are plenty of kids outside this demographic who qualify for assistance and who may even benefit from it to a degree. But the truth is that those kids and their families will do fine on their own. Not great. But fine.

It is the inner-city, one parent family kids that are the issue. These are the kids who year after year fall further and further behind the results of kids their age who come from intact families and/or from families not in the inner cities.

So we are spending money to help the inner city kids without addressing the many issues that contribute to the problem.

IMO you cannot expect kids to do well in school simply by giving them a few hours a week in an “education” program. These kids go home to mothers who simply are not equipped to build on the foundations laid by these programs (assuming these foundations are well built). These kids live in communities where achievement is suspect, where violence is the norm, where parental ignorance regarding childcare is almost non-existent.

Once the Titanic struck the iceberg, you could throw money at it all night long and it would not alter the course of events.

THE ISSUE IS NOT MONEY.

We need to find a way to alter the culture that these kids are born into. I’m afraid that the only way to do so is to admit that a generation or two of inner-city kids must pay the price so to speak… that the best way to help future generations is to admit that the existing generation is beyond help and that the next generation or two probably is as well. And that if we follow the same course of action (welfare, entitlements, etc), all subsequent generations will experience the same low acheivement.

We need to implement something that will ultimately change results by ignoring the pain that this will cause to current families, as hard as that will be.

IMO, this means doing away with the entitlement programs as they are. We need to find a way to help adults who are trying without subsidizing them forever. We need to expect that all adults participate in the reality that is life, meaning that we cannot forever reduce the cost of living by assisting with food, housing, etc. We need to convince this culture that accepting subsidies long term harms instead of helps.

Because the sad irony is that social programs that were ostensibly implemented to raise conditions have contributed to creating a greater downturn of the same.

Until we change the culture that kids are born into, we cannot reasonably expect that Head Start programs will have any lasting impact.

The progs know that the problem is the dysfunctional home life of too many of the low income children but PC prevents them from actually addressing it.

Rather than just keep proposing to throw more money and programs at schools that only have the kids for 8 hours 5 days a week they should be looking at how to move the children permanently into group homes where they can live in a stable environment with set rules and expectations of behavior. Such a solution could even indirectly benefit the parents, or in most cases the single parent, from whom the children are taken.

Since most government benefits are based on aiding the children they, assuming they were mentally and physically able, would be exposed to the motivation of “let him who will not work not eat” to pursue enrolling in one of the many job skill training or other educational programs that are available.

First, we should ask what group is intended to benefit from this federal program?
Chitownmom on November 21, 2013 at 6:56 AM

Follow the money….Pre-K is nothing more then a federally funded expansion of the teachers unions. More teachers, more union dues, more democrat skimming. Easy to figure out if you have never attended Pre-K.

The progs know that the problem is the dysfunctional home life of too many of the low income children but PC prevents them from actually addressing it.

Nomas on November 21, 2013 at 7:48 AM

Exactly. We conservatives always look at the effectiveness and logic of legislation and laws. These programs have nothing to do with that and those studies will be completely ignored. Head Start is a jobs program, period. It is a place to employ low-skilled, minimally educated people just like the people described in a previous post. Keeping people on the government dole is a perfect way to lock in votes. Head Start is just not a direct transfer like food stamps.

late to this story but I’ll chime in. My wife works in Head Start. And my two kids go to her school, but not in her classroom.

We are a rural head start, so no inner city, but we do have a lot of hispanics there (it’s mainly meant for them, I think).

I have mixed feelings about it, honestly. The pay is atrocious, so anyone good leaves soon (as well as the beaucracy–tis horrible). My wife is good, but she admits she’s burned out. The children come from broken homes, many of them. WE have some dedicated parents, like my wife and I (In all actuality, the training my wife has received is incredible for toddler/Pre-K age kids). The materials are not bad. The curriculum seems pretty good.

In other words, a good teacher in a head start program is likely able to do good things. But my children, in other classrooms in the same head start as my wife, aren’t progressing like I’d like them too. They learn far more from me than they do at school. And I watch their teachers and it’s no mystery why: they are burned out or not interested.