One of my goals in writing for this blog is to educate the general public about how to evaluate a scientific study, specifically medical studies. New studies are being reported in the press all the time, and the analysis provided by your average journalist leaves much to be desired. Generally, they fail to put the study into context, often get the bottom line incorrect, and then some headline writer puts a sensationalistic bow on top.

In addition to mediocre science journalism we also face dedicated ideological groups who go out of their way to spin, distort, and mutilate the scientific literature all in one direction. The anti-vaccine community is a shining example of this – they can dismiss any study whose conclusions they do not like, while promoting any horrible worthless study as long as it casts suspicion on vaccines.

Yesterday on Age of Autism (the propaganda blog for Generation Rescue) Mark Blaxill gave us another example of this, presenting a terrible pilot study as if we could draw any conclusions from it. The study is yet another publication apparently squeezed out of the same data set that Laura Hewitson has been milking for several years now - a study involving macaque infants and vaccinations. In this study Hewitson claims a significant difference in brain maturation between vaccinated and unvaccinated macaque infants, by MRI and PET analysis. Blaxill presents the study without noting any of its crippling limitations, and the commenters predictably gush.

The first (and really only) thing you need to know about this study is that it involves 9 vaccinated monkeys and 2 controls. That’s right – just 2 controls. The fact that Hewitson bothers to do statistical analysis on such a small set of subjects is laughable. Let’s keep in mind that most pilot studies turn out to be wrong – they are called pilot studies because they are intended to point the way to further research, not as a basis for any conclusions. Serious researches recognize that pilot studies are shots in the dark – and that counts even for good pilot studies, which this is not.

If the outcome were something hard and dramatic – like survival vs death, then 2 subjects would be a reasonable pilot study. But in this case Hewitson is doing a somewhat tricky measurement of brain volume changes over time and binding of opioid ligands in the amygdala. It is also worth noting that there were originally 4 controls, but one was eliminated due to improper protocol. We never learn what happened to the third monkey, we are just told there is data on two controls. This kind of missing data, especially when the overall numbers are so pathetically low, is very concerning.

She is also making multiple analyses (another red flag by itself), which means she can compare multiple variables looking for any difference. Then she invokes the sharpshooter fallacy and declares any change she does find to be clinically meaningful. So while there is no difference in brain volume or amygdala volume between exposed and unexposed monkeys, she finds differences in the change over time. We don’t know if still other variables were looked at and not reported – this is another weakness of pilot studies and why follow up studies replicating the specific effects reported are necessary before any conclusions can be drawn.

As further evidence of looking for any difference then declaring that the outcome of interest, we can look back to Hewitson’s 2008 reporting of her monkey data, in which she wrote:

“Compared with unexposed animals, exposed animals showed attenuation of amygdala growth and differences in the amygdala binding of [11C]diprenorphine.”

But in the current study she finds increased amygdala growth in exposed monkeys:

Not surprisingly, given the different maturational trajectories in exposed vs. unexposed animals, (unexposed decreasing and exposed increasing) there was a statistically significant interaction between exposure and time on total amygdala volume (Wald χ2=10.93; P=0.001). However, there were no significant main effects on total amygdala volume of either exposure (Wald χ2=0.75; P=0.39) or time (Wald χ2=1.14; P=0.29).

So which is it? Reading the results of the current study, especially in light of previous publications, gives the overall impression of a random scatter of data with incredible cherry picking in order to make the argument that there are any meaningful results at all.

Taken by itself, this is a worthless study. The numbers of subjects is too small to do any meaningful analysis. The results are all over the place, and not even consistent with prior publications by the same authors. The analysis is also far-fetched. Hewitson argues that both thimerosal-containing vaccine and MMR (which does not contain thimerosal) contribute to the alleged brain changes she is reporting. While the word “autism” does not appear in her report, Blaxill is concluding in his reporting that these brain changes are the same as those found in autism (an absurd conclusion given how non-specific these changes are, even if real, which cannot be concluded from this study). The anti-vaccine agenda is now clear – they get to have their cake and eat it too. They can now argue that an interaction between thimerosal and MMR cause or contribute to autism, through completely independent mechanisms, apparently.

To put this study further into context, this research is being conducted by the Thoughtful House Center for Children – Andrew Wakefield’s home after he was essentially kicked out of the UK and subsequently struck off. Wakefield’s name, however, does not appear anywhere on the current study, although he was listed as final author on previous publications from the same research. Apparently his name has become too toxic for the Thoughtful House.

The current study also appears in a obscure journal, Acta Neurobiologiae Experimentalis – which dedicated an entire issue to publishing dubious research on autism. The same issue includes two articles by the father and son Geier team – other vaccine and autism researchers who are off in their own world and whose research cannot be replicated.

Conclusion

This current study, as well as the entire macaque research program by Hewitson, is a good example of terrible research. The subject numbers are far too small for any meaningful statistics, and the outcomes being followed are numerous and tricky with a random scatter of results not even consistent between different publications of the same research.

What we have is far worse than ideological reporting and spinning of the scientific research – apparently we have the ideological conduction of research in the first place. This is similar to the research program of Benveniste on homeopathy.

In general it is a good rule to be suspicious of research that seems to be unique to one researcher or research team and is out of step with the broader research community. Unfortunately, such research contaminates the literature and is easily exploited to confuse the media and the public who often do not distinguish crank research from legitimate science.

___________________

Others reporting on this study:

Respectful Insolence – Orac also points out that Hewitson failed to disclose her COI – that she has a child with autism who is part of the Autism Omnibus suit.

Ibrb – Author, Sullivan, also points out that amygdala size should increase in macaques, so it is especially odd that the non-exposed monkeys’ amygdalas shrank. That makes no sense, and is likely due to the quirkiness of having only two controls. So the authors conclusions are entirely based upon a weird result in their tiny control group – i.e. this is completely bunk science.

21 Responses to “Terrible Anti-Vaccine Study, Terrible Reporting”

I think thats the prime example of bad research. How did it even make it past peer review? I wouldn’t be surprised if we have another Wakefield scenario where the researcher has arrived at a conclusion before the research has started.

It’s pretty easy to show mathematically that if you have 9 experimental monkeys and 2 controls, the odds of anything measurably different in any of the experimental monkeys compared to the controls is pretty high.
Imagine randomly picking 11 people from a bus and singling out two at random. Even with a typical measurement, like height or weight, there is a pretty good chance the measured average will be different between both groups by chance alone, simply because the 9 person group has a greater chance of having an outlier to throw off the average.

We can go back even further: how did it make it past IACUC review? If the protocol stated that there would only be 3 (or 4 or 2…they can’t seem to settle on a number) controls, how can the use of the monkeys have been justified?

Also, about the AoA. To them anyone even attempting this kind of research is deified. But it is still amazing to see how easily they are ready to make the leap between this paper and autism (a paper that doesn’t directly study humans or autism) when they refuse to accept the many negative studies which involved much larger sample sizes and studied actual humans with autism.

I remember one anti-vax commenter on Orac’s blog admitting that MMR had nothing to do with autism, yet she still greatly admired Wakefield “for having the guts to stand up to the pro-vax groupthink” (paraphrased from memory).

I wouldn’t be surprised if we have another Wakefield scenario where the researcher has arrived at a conclusion before the research has started.

Nige, are you aware that Laura Hewitson has a claim in the vaccine court for vaccine-injury-auitsm for her child?

She neglected to note her COI when she presented this research at IMFAR in 2008, leading to a stricter reporting of COI’s for that conference.

She works for Thoughtful House. Her husband works for Thoughtful House.

In her abstract for IMFAR she claimed that the amygdala’s of the vaccinated monkeys didn’t grow as fast as controls, indicating that vaccines cause injury. Now she claims the amygdalas grow faster, indicating that vaccines cause injury.

I don’t understand the rationale for the pilot study. After reading the introduction, it is still unclear. It is written in a way that indicates to me that they want the reader to connect dots that aren’t there (because they don’t actually say what they are implying).

That’s an excellent question, and one for which no one here seems to have an answer.

In my field, there are certainly vanity journals and they will publish just about anything. Can anyone comment knowledgeably about the prestige of this journal? Is this a low-credibility/low prestige journal?

I’m not fond of case studies, but they have a place — but this seems to go far beyond that. How on earth do you milk a study for multiple publications with fewer than 15 participants?

The comments here dismiss vaccine harm of any kind (both by the vaccine or the toxic content) and therefore subject infants to vaccines without consideration for harm.

More than a million infants have died otherwise healthy after following the vaccine regimes.

We have no accepted reason for this short of harm from repeat vaccines and anaaphylaxis now research more than a hundred years ago.

This 2010 research shows even in very small animal groups that with blinded procedures we get an increase of head size from 84800 to 87400 in 8 weeks while those not vaccinated remain essentially the same size.

My own research into falsely imprisoned parents always shows this head size increase over and above the normal. Figures of 50 to 90 per cent head increase can only mean colossal damage to the brain.

Here the increase for just the few animals is disturbing even using Steven’s analogy of results in animal experiments will be all over the place.

While animals may totally recover from a 5 per cent head size increase those that increase by 50 per cent as happens will not recover so easily.

I have to say that this research is going over what has been known for a very long time, that vaccines with organomercury are toxic and one way this shows is by an insult to the brain causing inflammation in EVERY animal receiving the thimerosal.

More than a million infants have died otherwise healthy after following the vaccine regimes.

@johnfryer, could you please be more explicit with this statement, such as would be in the form of literature citations?

This 2010 research shows even in very small animal groups that with blinded procedures we get an increase of head size from 84800 to 87400 in 8 weeks while those not vaccinated remain essentially the same size.

No john, the unexposed group (n=2, by the way) resulted in a significant decrease in amygdala size; that is not normal. The exposed group resulted in, what certainly appears to be, normal brain growth. Could you do yourself a service and actually read the study before commenting? I already know the torturous and outright false interpretations by those with a vested interest in ‘vaccine damage’.

My own research into falsely imprisoned parents always shows this head size increase over and above the normal. Figures of 50 to 90 per cent head increase can only mean colossal damage to the brain.

Would you be so kind as to share this research with the rest of us and how you reached this conclusion?

I have to say that this research is going over what has been known for a very long time, that vaccines with organomercury are toxic and one way this shows is by an insult to the brain causing inflammation in EVERY animal receiving the thimerosal.

Again, that is an erroneous conclusion of the study and I would appreciate it if you could show me the part that supports your statement? Ethylmercury requires orders of magnitude higher exposure than what was found in paediatric vaccines in order for toxicity to occur. But please feel free to demonstrate how the former vaccine schedule in the U.S. had toxic amounts of mercury.

Since the recognition of SIDS the death rate has been around 1 sudden death per thousand.

This means 2 000 deaths per year in the UK.

Probably 10 000 deaths per year in USA.

This has been going on with different numbers since 1969 and today sadly we still get reduced but significant sudden deaths.

As the babies are healthy before death we have to assume some sort of toxic death in a common sense world.

There are many such toxins and some are new and increasing and others are being reduced or eliminated.

I think a reasonable figure for summing up these deaths for every country around the world would exceed 1 million.

In addition there is strong pressure today to find causes of deaths other than SIDS.

On one blog my comment that Harry Clark died within 6 hours of an organomercury vaccine elicited the response it was SIDS. None of the Clark family that died after vaccines was or ever will be diagnosed as SIDS. This gives for me missed numbers of cases.

Is an organomercury vaccine toxic? Well in 1970 they were shown by simple tests to be toxic even when diluted 100 fold.

What tests today would you do to check for toxicity? What tests are done for example on the millions of organomercury H1N1 vaccines not used in France and resting in a SECRET repository where few people are aware of.

Why can’t we see toxicity studies on these vaccines today? Why aren’t we informed where this repository is? Is secrecy and throwing out messengers of bad news the trademarks of tyrants?

And of course vaccines are NOT the only cause of SIDS.

We are all painfully aware that NO ONE officially knows why a SIDS baby dies. But they most certainly die when othwise in PERFECT HEALTH of the diagnosis is INVALID.

I find it apalling that no one is shouting like me of the deaths to more than a million healthy infants many immediately after toxic vaccines when non toxic vaccines exist.

This study 2010 with monkeys is not perfect and I am not here to tear down the study which others do so well.

However the increase in HEAD size (nothing to do with any other things) is an increase of head size from 84800 to 87400 in 8 weeks compared with a zero increase for the controls.

Even assuming some of this is normal growth it also shows some abnormal swelling as well and this is disturbing.

If the toxic vaccines are directly causing this then, yes, a check of the results is needed and possibly withing 4 days of the toxic insult and not 8 weeks would add clarity to this investigation and its findings against mercury vaccines.

While the addition of toxic vaccines was historically happening the number of convictions for child abuse curiously went up 20 fold. This is either a measure of the increasing violence or perspicacity of the police or a sign of something very rotten in the state of denmark and every other western country.