This is an anti-quackery and pro-neurodiversity blog. Most posts here have to do with the author's view that autism is not a new man-made phenomenon, but instead a natural part of our species that has always been with us, and should continue to be. Reasonable corrections and rebuttals are welcome and encouraged.

Tuesday, November 24, 2009

A Word About Hertz-Picciotto & Delwiche (2009)

The recent Chicago Tribune Articles have resulted in a flurry of reactions, particularly from AoA. I'd like to discuss just one of them in this post. Kent Heckenlively was complaining that the Chicago Tribune reporters did not give weight to any of the documentation he had provided them. Specifically, to support the assertion that we are in the midst of an autism epidemic, Heckenlively cited a press release by the MIND Institute.

Why cite the press release and not the paper? Well, that's because the conclusions of the paper said the following.

Other artifacts have yet to be quantified, and as a result, the extent to which the continued rise represents a true increase in the occurrence of autism remains unclear.

A true increase... remains unclear. Is that clear enough?

The paper is titled "The rise in autism and the role of age at diagnosis." I'll refer to it as Hertz-Picciotto & Delwiche (2009). I've already criticized multiple aspects of the paper (last time here) but I'd like to say a few more things about it.

The conclusions of the paper seem refreshingly honest, but I'm guessing they are that way simply to get through peer-review. The missing artifacts are not just any artifacts, either. One of the artifacts that did not enter the calculations is key: awareness. It would be nonsensical to assume that awareness of autism has not changed since the early 1990s.

But what did the press release say?

“It’s time to start looking for the environmental culprits responsible for the remarkable increase in the rate of autism in California,” said UC Davis M.I.N.D. Institute researcher Irva Hertz-Picciotto, a professor of environmental and occupational health and epidemiology, and an internationally respected autism researcher.

Hertz-Picciotto said that many researchers, state officials and advocacy organizations have viewed the rise in autism's incidence in California with skepticism.

This is completely at odds with the conclusions of the paper, and I find it quite dishonest. To make it perfectly clear, yes, I'm accusing Dr. Hertz-Picciotto of intellectual dishonesty.

Dr. H-P further states that:

“These are fairly small percentages compared to the size of the increase that we’ve seen in the state,” Hertz-Picciotto said.

Is that even true? The paper finds that a 2.2-fold increase may be explained by changes in diagnostic criteria, 1.56-fold due to inclusion of "milder" cases, and 1.24-fold due to changes in age at diagnosis. If you multiply these factors, you come up with a 4.26-fold increase that may be explained by just these 3 artifacts.

That's 62% of the entire rise the authors were attempting to explain. Maybe 62% is a "fairly small percentage." In this case I'm going to give Dr. H-P the benefit of the doubt and say that she likely didn't know the factors needed to be multiplied. There are no indications in the paper that a calculation of the overall contribution of the 3 artifacts combined was even attempted. I won't even get into statistical uncertainty.

The flaws of the analysis are not what bother me the most, however. Consider this. What is the value of this study? Why was it carried out? If you're trying to determine whether artifacts can explain the rise in autism service classifications in California, and you cannot estimate the contribution of all relevant artifacts, what is the point of the analysis?

Here's an analogy. Suppose you wanted to determine if global warming can be explained by the greenhouse effect. In order to do this, you estimate the contribution of methane, water vapor and nitrous oxide to recent temperature increases, but you leave out CO2. You conclude that those 3 gases alone cannot explain the entire rise in temperatures, but perhaps that's because you did not consider the biggest contributor to the greenhouse effect: CO2. Then you tell the media that greenhouse gases cannot fully explain the rise in temperatures.

Wouldn't such a study be better understood as a propaganda effort, rather than a contribution to scientific knowledge?

Someone might complain that Hertz-Picciotto & Delwiche (2009) does contribute to scientific knowledge, because it tells us about the impact of certain artifacts. But does it actually do that?

Take what is perhaps the most important artifact the paper does take into account: changes in diagnostic criteria. What would you do if you wanted to determine the impact of changes in criteria? You might carry out a prevalence study with good case-finding that uses two different criteria on the same population: DSM-IV and DSM-III (or perhaps Kanner criteria.) Hertz-Picciotto & Delwiche (2009) does not do anything that even resembles this. They use data from a separate study, so they didn't even contribute new data. We can't even be sure how well the data from Finland might apply to California. The case-finding of the Finnish study is not necessarily very good either. Plus it's just one data point, with all the uncertainties implied by that.

To determine the impact of inclusion of "milder" cases (i.e. anything that is not "autistic disoder"), what would you do? I think you could evaluate a random sample of CalDDS autistic children and diagnose them with either autistic disorder, PDD-NOS or Asperger's. What the researchers did instead was use data from a separate MIND Institute study where CalDDS children had been evaluated with the Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Inventory (ADI.) Are these diagnostic tools even able to accurately distinguish between autism spectrum diagnoses? I'm not aware of any evidence that they do.

In other words, the MIND Institute study is not even informative about the impact of the artifacts it did take into account. I frankly can't see this study as a contribution to scientific knowledge at all. It gives the appearance of being part of a propaganda effort.

I've found that the two "camps" among autism is really not "Pro-Neurodiversity" and "Anti-Neurodiversity" but "Pro-Vaccine" and "Anti-Vaccine."

I do wish "Neurodiversity" would spend more time on critical issues that relate to autistic people and the diversity they supposedly advocate for.

I get tired of the same articles and studies being written about over and over again and the same arguments being made by the same people. I've been in the online autism world for over a year now and it is still the same.

You left me this response on LB/RB:

"Institutionalization is just a terrible idea. It’s interesting to read Kanner’s findings on this. No other researcher has ever looked into the institutionalization of autistics, as far as I’m aware.

Read more: http://leftbrainrightbrain.co.uk/?p=3642#ixzz0XsnpsVn1"

Can you direct me to Kanner's findings? As a person with autism who was institutionalized I'd like to read them.

Subscribe To

About Me

I am the parent of a classically autistic boy, and I'm on the spectrum myself. The science of autism is my main area of interest, but I have a general interest in science and data analysis. My formal training is in Computer Science.

Followers

Comment Policy

Natural Variation has an intentionally lenient comment policy. I'd hate to have to change it, so please do not abuse it.

Comment moderation is disabled at Natural Variation in posts that are less than one year old. The only comments that end up getting deleted are those that are in violation of Blogger's Content Policy (typically spam), or are authored by John Best Jr.