Monday, December 01, 2008

In this study, 106 autistic children (mean age 8.5yrs, range 3-14.2yrs) were receiving intensive (27-35hrs/wk) one-on-one ABA-based interventions (the ABA school group), while 51 children received "eclectic" non-ABA autism-specialized services (27-30hrs/wk; the ASD unit children).

Some of these children (109) were judged to be in various ranges of intellectual ability, but this information was gathered by questioning individuals who worked with the children and was not verified in any way. Similarly, there is no information about these children's diagnoses, beyond that they were "children with autism spectrum disorder," and for seven children, even their age was unknown.

All 157 autistic children's challenging behaviours were assessed through a version of the Behavior Problems Inventory (Rojahn et al., 2001; Murphy et al., in press, take their description of this instrument nearly verbatim from this earlier paper). This instrument involves interviewing informants (rather than direct observation) and divides challenging behaviours into three categories: stereotypy, self-injury, and aggression.

Regardless of the reasonable sample size, this study's authors chose to organize their data in ways that made many statistical comparisons useless. There is a clear trend, though, if you slog through what information they do provide. This is summarized in the discussion, about comparisons between children in the ABA school group and children in the non-ABA ASD unit group:

With regard to the type of intervention received being a risk factor, findings show that the type of intervention is not related to the prevalence of challenging behavior. However, in relation to participants receiving an ABA intervention, they displayed a higher frequency of stereotypy and also showed higher levels of aggression.

If I adhered to autism advocacy standards, I could simply repeat some or all of this excerpt endlessly as some kind of "proof" of what intensive ABA programs accomplish.

But the first sentence in the quote above is not supported by the authors' analyses. The other sentence is accurate in the sense that what is claimed can be traced to statistical comparisons that were actually carried out.

However, these statistical comparisons are within the context of a very poor quality study where the findings, even within the given highly questionable parameters (e.g., definitions of challenging behaviours that, among many other limitations, encompass presumed mental states) are virtually impossible to interpret.

Available data that might assist in interpreting the findings are simply not reported (e.g., demographics of the two different intervention groups). But even were the data better reported, this study is based on an very poorly characterized sample (poor characterization being the standard in the vast ABA literature), and one that according to what scant information is provided, is not representative of autistic children in general.

Indeed, an inventory of the errors (some of them instantly obvious) and major weaknesses in this paper would be lengthy and would include, somewhere down the list, the authors' contention that ABA must have been effective in their study in reducing challenging behaviours in autism because, well, ABA is effective.

This paper was accepted (within two days of being submitted) by an autism journal edited by a famous behaviour analyst. In my view, in its epublished form, this paper contributes nothing reliable or worthwhile to autism research.

What is worthwhile, though, is the reminder that quality and standards matter. The selective reading and promotion of research of overwhelmingly poor quality is central to autism advocacy--is one of its signatures. Autism advocates have successfully insisted that poor quality research is what autistics deserve.

But poor quality research can be selectively invoked to support any position, as I've pointed out here. For example, the low/no standards of science and ethics that autism advocates have imposed on autistics have led to public funding for RDI. But pushing poor standards and poor quality research is not going to lead to better outcomes for autistics. Claiming otherwise, as autism advocates loudly do, harms autistics.

12 comments:

Do you feel that all studies which refer to the subjects simply as "autistic" or as having an "autism spectrum disorder" without identifying the specific ASD of the children involved reflect poor science?

Should the severity of the subjects disorders also be identified in the interests of better science? For example should a distinction be made between autistic subjects who are very intelligent and enjoy excellent communication skills and those who are non-verbal and cognitively impaired?

If I might ask one more question. You have spent much of your post-Canada Post career attacking ABA. Can you recommend a more scientifically solid, or evidence based, health OR education intervention for assisting persons with Autistic Disorders and profound developmental delays? And can you provide some sources and authorities for this alternative, in addition to your own opinion?

In response to Mr Doherty, my concerns re diagnosis are not those Mr Doherty mentions. Also, it is unclear what Mr Doherty means by "severity" of autism, as Mr Doherty uses level of measured intelligence as an example of "severity."

Murphy et al. (in press) try to use ranges of intellectual ability as a variable, but do not report how, when or whether this was measured.

These studies have also shown, in different ways, how important it is that these autistics' competent communication be responded to (see also the successful pilot RCT of an early intervention, Aldred et al., 2004). Unfortunately, autism advocates have widely spread the false information that these autistic children can't communicate at all.

According to Mr Doherty, any scrutiny or criticism of the science and ethics of ABA-based autism interventions is an "attack" on ABA. Also, any attempt to achieve human rights for autistics--even the most basic of human rights--is considered by Mr Doherty to be an "attack" on ABA. And according to Mr Doherty, any attempt to promote recognized standards of science and ethics for autistics is an "attack" on ABA. In my view, this all speaks volumes about autism advocacy.

Because of the success of autism advocacy, going back decades, the quality of autism intervention research is generally poor. FEAT in Canada, which is fully supported by Mr Doherty, has opposed any money being spent on more autism intervention research.

However, individuals in the profound range not excluded in the multi-site RCT reported by Tryer et al. (2008), who state that:

"either the placebo effect, the psychological effect of a formal external intervention, or spontaneous resolution, or all three, are substantial and would be difficult to surpass by even the most effective of drugs."

Yes Harold makes a valid point about the severity of disorders in the children studied,but until I see other wise,I will say the fault lies with the journals that accept these articles for publication.If you read a lot of these articles,they will simply say that X number of children had an autism spectrum disorder,and do not break down the results to say the children with Aspergers had this result,or the children with PDD-NOS had this result.

Some will just say the child had an ASD,with no specifics.Is it up to the journals to demand more of the studies they publish,or should the researchers do better work on their own?

Research in Autism Spectrum Disorders seems especially bad in this.I have read a lot of their other articles.There are a lot that you can read without payment or registration

You can start here: http://www.sciencedirect.com/science/journal/17509467

I agree this is shoddy science,the editors and publishers ought to expect better than accepting some of the largely worthless stuff here like "Age, social engagement, and physical activity in children with autism spectrum disorders"

Are there people at these journals ,who share the view that all autism is created equal,that some who promote neurodiversity espouse?If this is their agenda,they should come out and say it.I wouldn't use some of this stuff to prove a point.

I was diagnosed as an adult,so I am still learning about ABA.This article says ABA makes sterotypy and aggression worse.From what I have read,it seems to vary from child to child.Some kids improve, in these areas,some get worse.

I know if it made the behaviour worse,in a child of mine,I would stop it.

You mention Relationship Development Intervention ™.Do you believe this is a valid alternative to ABA ?Dr.Gutstein seems to allow very little third party information to get out.He seems more interested in protecting his financial interests,than in autistic children.Is this the case?What success has Relationship Development Intervention ™ had in lower functioning autistics?

I don't know what Mr Doherty means by "severity" of autism, so I am not sure what point he is making.

Re RDI, there is virtually no published research to support this highly-marketed autism intervention. There is one paper (Gutstein et al., 2007), but it is an uncontrolled follow-up to an earlier study that has not yet been published in a peer-reviewed journal.

Children in Gutstein et al. (2007) were required to have pre-treatment IQs of 70 or more (no instruments given), but IQ was not an outcome measure.

"According to Mr Doherty, any scrutiny or criticism of the science and ethics of ABA-based autism interventions is an "attack" on ABA. Also, any attempt to achieve human rights for autistics--even the most basic of human rights--is considered by Mr Doherty to be an "attack" on ABA. And according to Mr Doherty, any attempt to promote recognized standards of science and ethics for autistics is an "attack" on ABA. In my view, this all speaks volumes about autism advocacy."

Ms Dawson you make statements about what I claim. I have never taken the positions that you stated. Please stick to facts not fiction.

I gave you example when I asked my question about severity. You chose to ignore the example in your reply.

You avoided altogether answering my question concerning whether you could specify any intervention which has more evidence in support of its effectiveness than ABA. If you can I am sure the American Academy of Pediatrics, the NY DOH, the MADSEC Autism Task Force, the US Surgeon General, the Association for Science in autism treatment and hundreds of thousands of parents of autistic children would be interested in learning of any intervention that surpasses ABA as an evidence based intervention in helping autistic children realize gains in several domains, intellectual, communication, social and behavioral.

In response to Mr Doherty, his position, as I describe it, can be verified through his public writing, including his comments on this blog.

Re "severity" of autism, this term has a specific meaning in autism research. But as is common practice with autism advocates, who also promote novel diagnostic categories and criteria, and as I noted above, Mr Doherty has apparently invented a novel definition of autism "severity."

As anyone can see by reading what Mr Doherty wrote, Mr Doherty's original question was:

"Can you recommend a more scientifically solid, or evidence based, health OR education intervention for assisting persons with Autistic Disorders and profound developmental delays?"

I did answer that question, and my answer can be verified by reading primary sources.

I can add that if you put together all the ABA group designs, you will find very few of the individuals who Mr Doherty asked about (autistics scoring in the "profound mental retardation" range, as defined in the DSM or ICD), certainly too few to draw any conclusions about the effectiveness of ABA-based interventions for this population.

I've responded to Mr Doherty's boilerplate here. And since that time, another systematic review has been published (Ospina et al., 2008), which describes the evidence in support of ABA-based autism interventions this way:

"However, in relation to participants receiving an ABA intervention, they displayed a higher frequency of stereotypy and also showed higher levels of aggression."

All I can say is "NO KIDDING!!!" My happy, go-lucky, friendly little boy has been loved and integrated by all EXCEPT the ABA Therapists. Those he hated. He finally clawed (5 actions against others) the Male T who was kicked out permanently and then we went to work getting rid of the rest of them... they didn't leave willingly. And to this day I feel guilty about believing them at first when they blamed HIM for his being upset. And yes, you should be able to read your NON VERBAL children's body language and changes in behaviour and believe them not the professionals when something is wrong.

Harold's afraid to discover his "severe" son... may actually not be as MR as he hopes. Since many of us who have worked with, played with, enjoyed, integrated our severe children have had AMAZING growth, speech, independance, and learning (the 3R's) with them.

Yet those stuck in those self-contained classrooms... have I high rate of assault. I know one local class that's had 2 such assaults on the teachers already this year.

......interesting. My 15 yr. old son and I are both Aspies. I've witnessed that the functioning level of individuals with ASD is not indicative of intelligence. We all communicate differently as do neuro-typicals.

I've also seen how misnomers regarding communication levels have set up a society which often does not understand that individuals with ASD can and do communicate at all levels. There is much work to be done with regards to helping others "get it" and truly understand Neurodiversity.

I really have yet to see you engage in wider aspects of human rights for autistic people beyond your own personal conceptions and objectifications.

I know it is not really PC in this blog world to have a go at the "blessed" Michelle, which in a sense determines why I do it. No one is immune from criticism (yeah cos I know the usual suspects will be bunging the brickbats myside) and as I have said before, your politics suck because you fail to engage in wider debates.

There is more to life than the spectre of Lovaas, and I do think you overestimate the influence here, I don't think you give a true account of the situation in Canada at all because you are so biased, determinate and "short sighted"

In response to Mr Rex, I go around asking for criticism--genuine criticism (meaning, related to things I've actually said or done). Unfortunately, Mr Rex still hasn't offered any. If ever he does, I will be sure to thank him.