Biostatistical Recommendations

H.C. Kraemer

Little more can be learned either from cross-sectional or from retrospective research on many key issues in autism. Autism is a developmental disorder, with very early onset, and is chronic over the lifetime of the patient. There is a serious need to understand what are the stable traits of patients and to distinguish these from what are the stages of the disorder, and to distinguish both traits and stages from states and random variation. To do this requires prospective, longitudinal studies. The problem is that such studies are costly in terms both of research time and research cost.

New statistical methodologies are currently emerging to make such studies more informative as well as most cost-effective. Individual growth models have been mentioned frequently in this report, for example. Such models acknowledge both the consistent individual differences (traits) within groups of persons with autism and those differences expressed in different trajectories (stages). Moreover, such approaches are much more tolerant of unequal follow-up times, or irregular scheduling of follow-up times, and are much more robust to the less than perfect reliability of many available and pertinent outcome measures.

To reduce both time and cost of such studies as well, accelerated lifetime sampling methods are available, where subjects are entered into study at different ages and followed for some period of time (say 5 years), in such a way that age span over which different subjects are followed overlap each other. One can, by such methods, accumulate a depiction of the general growth patterns over the first 20 years of life, for example, using only 5 years of follow-up per patient.

There are many other such strategies either currently known but seldom used, or under current development, or that could be developed that are particularly appropriate to the study of this disorder. Development and dissemination of such methodological strategies might be supported for researchers in the field.

It is known that some persons with autism are high- and some low-functioning; that some are mute and some vocal; that some respond to a certain treatment and some not. Dr. Grandin made the point most strongly that there is a great degree of heterogeneity among persons with autism that is not well understood, and sometimes not even acknowledged.

Identification of subtypes is important, that is, subgroups of those appropriately diagnosed with autism who may have different etiologies, different course, and/or different response to treatment. If such subtypes exist, they are currently being lumped into one group. The heterogeneity so introduced by "lumping" diminishes the power to detect any signals, whether they be the genetic basis of the disorder, risk factors for the disorder, discrimination between persons with autism and those with normally developed brain structure and function, or treatment efficacy/effectiveness. It is crucial to future research and development of knowledge that, if such subtypes really exist, they be identified. On the other hand, we do not know the boundaries of autism or any subtype of autism. For example, we can reliably distinguish autism, Rett syndrome, and childhood disintegrative disorders (CDD), but are these simply different expressions of the same disorder or of different disorders (again, different etiologies, course, or treatment response)? It should be remembered that before the organism for syphilis was identified, it was thought that there were multiple different diseases depending on which organ system was primarily affected. To "split" when there is no valid reason to do so may also undermine a research study’s results. The search for a biological marker(s) is critical here.

Comorbidities are yet another problematic source of heterogeneity among persons with autism. Some comorbidities are random—one might have a cold and corns at the same time, and they have nothing to do with each other. Some comorbidities may be different expressions of the same disorder, or one disorder might lead to another. In such cases, these are not necessarily separate disorders, but perhaps different manifestations of the same disorder, or different stages of a single disorder. Some comorbidities are indeed separate but related disorders, due to linked genes or related environmental effects, or with common risk factors, some causal, some not. When comorbidity exists, each disorder may or may not affect the success of treatment of the other.

Should we "lump" or should we "split"? Each is appropriate in different situations, and whichever is inappropriate will compromise research success in understanding autism. It is essential to gain a greater understanding of the heterogeneities among persons with autism, and a recognition of which sources of heterogeneity are clinically important and which are not, for these issues have major repercussions in terms of research design and research success.

Another recurrent theme has been that of fostering closer connections between research efforts and real life. Patients, parents of patients, as well as interest and support groups should be involved in clinical research studies, both for the traditional purposes of fund raising and help with patient recruitment, but also to help researchers formulate the questions most important to patients. Along this line: (a) We should increase emphasis on long-term effectiveness rather than short-term efficacy studies. (b) We should reconsider the appropriate choice of control group (When is a placebo group the appropriate choice?), which may not be the same in all studies. (c) We should include consideration of both financial and emotional cost to families as outcomes in clinical trials as well as quality of life measures.

With the difficulty of defining samples that control for relevant variables and the severity and impact of autism, there should be special emphasis on high quality research, for example, diagnosis, sampling, measurement, design, and power. Frequently, the argument is made in the opposite direction: Since the issues are so important, we should allow researchers more latitude in designing and executing their studies. It is important to realize that funding poorly designed research is not only a waste of time and money that might better have been invested elsewhere but the results may actually mislead the research field and misinform the clinicians working with patients with autism.

Moreover, it may well be that new modes of research collaboration need to be forged. Multisite trials are certainly one such example. Collegial agreements between independent research centers studying autism that a finding at one site should immediately be followed by an attempt to replicate and confirm that finding at another site, is another example. Autism registries, brain banks, and gene banks have also been mentioned as possible resources to foster excellent and cost-effective research efforts.