“In a recent study, two economists at the University of Chicago, Matthew Gentzkow and Jesse Shapiro, came up with a different way to test the long-run impact of television on kids—by reaching back to the distant past of the information age. When Americans first started getting television in the 1940s, the availability of the medium spread across the country unevenly. Some cities, like New York, had television by 1940. Others, like Denver and Honolulu, didn’t get their first broadcasts until the early 1950s. Whenever television appeared, kids became immediate junkies: Children in households with televisions watched their boob tubes for close to four hours a day by 1950.”

…

“The key point for Gentzkow and Shapiro’s study is that depending on where you lived and when you were born, the total amount of TV you watched in your childhood could differ vastly. A kid born in 1947 who grew up in Denver, where the first TV station didn’t get under way until 1952, would probably not have watched much TV at all until the age of 5. But a kid born the same year in Seattle, where TV began broadcasting in 1948, could watch from the age of 1. If TV-watching during the early years damages kids’ brains, then the test scores of Denver high-school seniors in 1965 (the kids born in 1947) should be better than those of 1965 high-school seniors in Seattle.”

…

“From the 1966 Coleman Report, the landmark study of educational opportunity commissioned by the Civil Rights Act of 1964, Gentzkow and Shapiro got 1965 test-score data for almost 300,000 kids. They looked for evidence that greater exposure to television lowered test scores. They found none. After controlling for socioeconomic status, there were no significant test-score differences between kids who lived in cities that got TV earlier as opposed to later, or between kids of pre- and post-TV-age cohorts. Nor did the kids differ significantly in the amount of homework they did, dropout rates, or the wages they eventually made. If anything, the data revealed a small positive uptick in test scores for kids who got to watch more television when they were young. For kids living in households in which English was a second language, or with a mother who had less than a high-school education, the study found that TV had a more sizable positive impact on test scores in reading and general knowledge. Evidently, Bozo the Clown was better than we remember.”

The conclusion above gives the impression that this study is actually looking at the test results of kids who grew up watching a lot of TV compared to kids who grew up watching substantially less.

That is actually not the case, instead this study is comparing the test results of whole cities. That is, the authors assume that in a city where a TV station started broadcasting a few year earlier, that children living there watched substantially more TV growing up than children who grew up in a city where a TV station came later.

This is a huge assumption to make. For example, the study is comparing cohorts where a TV station arrived in their city in 1948 with cohorts where a TV station arrived in 1954. As their own graph shows in 1948 only about 20% of households even had TVs, and by 1954 that percentage had only increased to about 30%. (Remember, back in the 1940s and 1950s, a TV was an expensive luxury item.) So basically the test results are diluted (unevenly) by a large percentage of kids who lived in homes without a TV (despite living in a city with a TV station).

Also, there is no real evidence that children, in the 1950’s, living in households with a TV instantly started watching over 4 hours of TV per day (as the Slate article argues). There is some evidence that in homes with a TV, that by 1950, the TV was on for 4:35 per day. Just as by 2008, the average American home has their TV on for 8:18 hours per day. I don’t think that anyone is arguing that today, kids watch over 8 hours of TV per day.

Never mind the economic, cultural, and demographic differences among the compared cities. Never mind the fact that it took years from the time a TV station first arrived for most households to even get a TV. And never mind, that most new TV stations had few channels making it hard for even the most dedicated TV fan to actually watch large amounts of TV right away. These are huge confounding factors.

The authors acknowledge some of these factors, and do attempt to take them into consideration when doing their statistical calculations. Nevertheless, there are a large number of known and unknown confounding historical factors making it impossible to come to any kind of accurate conclusion. This explains why this “working paper” was never published in a peer-reviewed journal (note it was released in January 2006, so there has been plenty of time for peer-review). This lack of peer-view did not stop Slate from publishing an enthusiastic and uncritical article describing this non-peer-reviewed study, as “Proof that TV doesn’t harm kids”.

Instead of using a large number of assumptions and estimates to try to compare a large group of kids (some of who grew up watching various amount of TV) with another group of kids (some of who watched slightly less), why not just do a longitudinal study?

“These findings indicate that excessive television viewing is likely to have a negative impact on educational achievement. This is likely to have far-reaching consequences for an individual’s socioeconomic status and well-being in adult life.23 Although we cannot prove that watching television is causally related to poor educational achievement, the associations between viewing time and educational outcomes were strong and independent of the known confounding influences of intelligence, socioeconomic status, and childhood behavioral problems. Furthermore, this study fulfills many of the other criteria often used to infer causality in an observational study, including temporal sequence, dose-response relationship, and biological plausibility. However, we cannot rule out the possibility of reverse causation. This is likely to be at least part of the explanation for the strong association between television viewing during adolescence and leaving school without any qualifications. By adolescence, some individuals will be poorly motivated toward schoolwork and may, for example, fill their time by watching television instead of doing homework. This is less likely to be the explanation for the strong inverse association between television viewing in childhood and attainment of a university degree. The finding that childhood viewing was a better predictor than adolescent viewing of not obtaining a university degree makes reverse causation unlikely and indicates that excessive childhood television viewing has a long-lasting association with poor educational outcomes.” Archives of Pediatrics & Adolescent Medicine (July 2005)