Essay # 3:

How to set up comparisons

In this essay, we will examine a number of ways in which epidemiologists design their studies to compare disease risks across groups of people. The designs vary in how their data are collected, how the results are expressed, and the kinds of inference that can be drawn from them.

Introduction

Epidemiologists are usually interested to compare the risks of disease between various groups of people, sectors of society, over time, or between different places. In order to make these comparisons, a variety of methods is used to set up the appropriate comparison groups and to estimate corresponding disease risks for each of them. We will now review several of the most important study designs used by epidemiologists for this purpose.

Case-series design

The first approach is known as the case-series method. In this approach, information is gathered on all the cases of a disease or other health event of interest over some period of time and within a certain geographic area.[Vandenbroucke, 2001] For example, doctors in a city hospital might record the number of patients admitted to their emergency department for injuries related to traffic accidents for a twelve month period. They might gather details about the age and sex of the patients, medical history, and clinical details of the injuries the patients have sustained.[Freeman and Hutchison, 1980] The resulting analysis of this data will probably be largely descriptive, for instance summarising the age and sex distribution of all of the patients seen. It may also describe the number of patients seen each month during the data collection period.

Case series play a very important part in medicine. First, they are often easy to collect. Second, they are a fruitful source of new insights and possible alarms about the emergence of a new disease, or the development of an adverse effects of drugs. For example, when a several cases of a new strange disease were seen in the US, wherein previously perfectly healthy young men developed infections that are usually only seen in cancer patients whose immune defence system is compromised by the cancer and the anti-cancer drugs. This was called an “Acquired form of Immunodeficiency Syndrome.” “AIDS” as we know now, was a purely descriptive term indicating that these people hade become immunodeficient ‘out of the blue’. [Gottlieb et al., 1981] Likewise, when a new drug for ‘athlete’s foot was given to people, and several of them told their doctors that they had suddenly lost their appetite (which is a very odd phenomenon), this heralded the discovery of a - fortunately reversible - adverse effect of this drug.[Ottervanger & Stricker, 1992]

Despite the important uses of case series in discovery,, the inferences that we can draw from such case series are often quite limited, because the cases constitute the numerator counts (as described in Essay 2), but we have no comparable information on the corresponding denominators. So, for instance, it is difficult to know what a larger number of injuries in one age group vs. another might mean; we certainly cannot conclude that members of the age group with the most injuries are the people at highest risk of an accident, because we do not know how many people in the general population are in each age group. More specifically, we cannot tell (from this data) how many people in each age group are driving or travelling by road and therefore at risk of a traffic accident. Similarly, we could not conclude that simply because more patients were admitted during the winter months compared to the summer that winter driving is necessarily more hazardous. In order to make a valid comparison of the risks of driving at different times of year, we would need to know about the amount of traffic on the road, which would then tell us about the denominator of people at risk of having an accident.[Frost, 1941]

Sometimes a failure to recognise that denominators are lacking, and basing the analysis on numerators only, leads to incorrect conclusions. For example, some years ago a study was published on the average age of death of doctors in various specialties.[Warren, 1956] The results showed that radiologists died younger than doctors in other medical specialties such as dermatology or pathology, and that physicians who had no exposure to radiation had the highest average age at death of all groups. It is incorrect to conclude from these observations that being a radiologist is necessarily any more hazardous to one’s health than being in some other medical specialty. Naive comparisons of the average ages of death in these groups fail to recognise the age distribution of the doctors in the various specialties. When this study was done, radiology was a relatively new specialty, and accordingly had relatively young doctors practicing it. So even if the risk of death was identical at any given age for the doctors in all specialties, one would expect to see an average younger age of death among the group (i.e. the radiologists) who had a younger age distribution.

Similar thinking can debunk other fallacious arguments based on numerators only. For instance, one may note that kings and queens tend to die at older ages than princes and princesses. It would be incorrect to conclude that the protected lifestyle of a reigning monarch provides some reduction in their death rate. The real answer is that kings and queens have to survive long enough in order to ascend to the throne, and are therefore relatively old even when they become the monarch, compared to the age distribution of other princes and princesses who do not succeed to the throne.

Cohort study design

A second approach, and one which is important to epidemiologists and which does take denominators into account, is known as the cohort study design.[Weinberg, 1913] In this method, groups of people are enrolled into a study at some point in time and then followed and monitored for the occurrence of health events of interest.[Doll and Hill, 1954] In some cohort studies, samples of the population or indeed an entire community can be followed for many years. This was done, for instance, in a follow-up study of residents in the town in Framingham, Massachusetts, to examine their risk of health-related events, including diseases and deaths from various causes.[Doll, 2004] This study was instrumental in elucidating that people with a higher cholesterol level in their blood had a higher risk of heart disease, such as myocardial infarction. Another example is the studies of survivors of the atomic bomb explosions in Japan during the Second World War; these survivors were followed with a particular focus on the occurrence of radiation-related disease including a variety of cancers.[Beebe et al., 1971]

The cohort method is also used in occupational medicine. Workers in a particular industry may be enrolled into the cohort if they are present in the workforce on a particular date, or when they begin their individual employment in the industry.[[Doll, 2004] and others] Another well-known example of the cohort design is the study of doctors in the United Kingdom that was one of the earliest investigations that identified significant health effects of smoking: In 1951, the doctors completed a very simple half-page questionnaire, about whether they smoked, what they smoked and what amount they smoked daily; that information was then linked to their death certificates collected over the following 50 years. [Doll and Hill, 1954;Doll and Hill, 1964(a) & 1964(b);Doll, 2004b]

In all these scenarios involving cohorts, base-line information is collected on the cohort members. This information can be gathered by questionnaire, through existing documents such as occupational and medical records, by interview, or by physical examination. This data is then used to establish baseline counts or denominators of the number of people at risk in various levels of exposure to possible risk factors. For example, we may wish to compare the risk of death among people who are defined as obese or non-obese, the risk of cancer according to radiation level received in the Japanese cohorts,[Beebe et al., 1971] the risks of various disease outcomes according to the specific job activities within an occupational group,[Stellman, 2004] or in the United Kingdom doctors study the risks of lung and other cancers among heavy smokers, light smokers, or non-smokers.[Doll and Hill, 1954]

Some cohort studies also involve repeated contacts or data gathering from the cohort members, so that information on risk factors can be updated.[Nurses' Health Study] This is because for some investigations, the risks of disease may depend on the duration or intensity of exposure to a risk factor. For example, the risks of cancers and heart disease may depend on the length of time that a woman has been using hormone replacement therapy. Similarly, the number of years that a person has smoked and how much was smoked each year may be the relevant factors that determine the risk of cancer.[U.S.Dept.of Health, 1964] Finally, the risk of side effects from taking medication may depend on how long it has been used and the daily dosage that was prescribed.

Unfortunately, cohort studies are often difficult or expensive to carry out.[Hammond and Horn, 1958] This is partly because one frequently needs to follow the cohort for many years in order for a sufficient number of disease cases to be observed. Even for the commonest diseases, the probability that any particular individual will develop as a case of disease will be quite small. If only a limited number of cases of disease are identified in the cohort, any association with risk factors will not be clearly identified. There will be considerable chance variation in the calculated risk, and so the evidence will likely be insufficient to clearly establish whether a risk factor does or does not affect the chance of disease.

Cohort studies are also used frequently in clinical medicine, to study the outcome of patients, for example to see which patients benefit more from therapy, or which patients are more likely to have poorer disease outcomes.

Quite apart from the fact that only a limited number of disease cases may occur in a cohort, there is the additional problem that considerable resources may be required to administer the study while investigators are waiting for those cases to occur. The heath status of the cohort members must be monitored during the follow up, and the expense of doing so over several years can be considerable.

Case-control study design

Recognizing the practical difficulties of carrying out cohort studies, an alternative design which may be more efficient is often used. This alternative is known as the case-control method.[Lane-Claypon, 1926],[Stocks and Karn, 1933],[Lombard and Doering, 1928] In this approach, epidemiologists compare people who have already developed disease (the cases) with other people - known as controls - who do not have the disease. So, for example, if we were interested to identify risk factors for breast cancer, [Lane-Claypon, 1926] the case-control method would identify a series of cases, such as all newly diagnosed breast cancer who are admitted to a hospital within a certain time period. Additionally, the controls would be selected in some way as a comparison for the breast cancer cases. These controls might be drawn from people who were admitted to the same hospital for some other reason, or sampled from the general community. The idea is that these control persons should reflect the ‘exposure that the cases would have had, if they had not become diseased’ - i.e., the expected distribution of exposure in the general population.

For both the cases and the controls, epidemiologists then measure the exposure to risk factors at various points in the past. For example, if the investigators think that breast cancer might be related to diet, level of exercise, and smoking, they would try to find out what the distributions of those variables had been for the cases as compared to the controls.

Sometimes the information on risk factor status can be established through historical documents; for instance, the use of cholesterol lowering medications (a possibly protective risk factor) could be identified by consulting physician records for previous years. However, for other factors such as level of exercise and smoking, one will probably have to rely on reports by the cases and controls themselves, as given through interview or questionnaires.

The appropriate comparison that is made in a case-control study is that of the distribution of exposures to risk factors between the two groups. If relatively more exposure to a risk factor is observed in the cases than in the controls, then one might conclude that the risk factor was associated with an increased risk of disease, and vice versa if less exposure is seen in the cases compared to the controls. Studies on smoking and lung cancer, before the British cohort study, were case-control studies, in which smoking habits of patients admitted to hospital for lung cancer were compared to smoking habits of patients with various diseases that were thought to be unrelated to smoking (and thus would represent the usual or background smoking frequency in the population).[Doll and Hill, 1950],[Levin et al 1950],[Wynder&Graham 1950]

The case-control design has a major advantage, because we are not required to wait for a long period of time for cases to develop, as often occurs with the cohort design. Case-control comparisons are focused more directly on a larger number of cases, with typically about the same number of controls. If necessary, the new cases of disease can be identified by a network of hospitals or a system such as a population-wide cancer registry, so that the required number of cases can be enrolled relatively quickly. By gathering the cases and controls over a short period of time, the case-control study can be carried out rapidly and typically at lower cost. There is a fundamental conceptual unity between the case-control and the cohort approach: In a sense, a case-control study samples the same group of people as a cohort study would do, but does this by enrolling only those who become diseased and those who are not diseased in the population.

These advantages have to be traded off against the difficulty of actually obtaining good quality data on risk exposures for previous years. It is often difficult for individuals to accurately recall their prior exposures, and indeed their ability to do so might differ between the case and control groups. Accordingly, in a case-control study there may be concerns about the fundamental accuracy of the data. Imperfectly recalled exposure, or even bias in recalling the exposures may lead to uncertainty about how valid the claim of an association of the risk factor with disease might be.

Randomised trial design

In some cases, epidemiologists are able to use the randomised trial method of setting up their comparisons.[The James Lind Library] In this approach, participants are randomly assigned to one of two or more interventions and then followed up to observe outcomes of interest. For example, a randomised trial might be used to compare two alternative strategies for smoking cessation among smokers; study participants would be randomly assigned to have an individual counsellor, to take part in group sessions, or to receive educational literature. The outcome for such a trial would be the proportion of smokers who successfully quit after some given period of time. This would be an example of secondary prevention of disease, which is attempting to reduce exposure to a health hazard among people who have already become exposed.

The randomised trial method could also be used to evaluate public health interventions such as screening for cancer.[Shapiro et al., 1988] Individuals, or sometimes communities of individuals, would be randomly selected to be offered screening (on one or more occasions), while comparable controls in the same population would not receive this offer. The outcome for such a trial would be death from the cancer in question. The objective would be that the screening examination would detect some cancers at an early stage, which might then translate into a more effective treatment and improved prognosis.

The randomised trial method is frequently used by clinical epidemiologists to evaluate the relative benefits of two or more alternative treatments for a given disease. Randomised assignment of interventions has the advantage that on average one expects that all other risk factors for the outcome will be equally distributed between the two treatment groups. This would include not only risk factors that have been identified and possibly measured in the participants, but also be true for other risk factors which cannot be measured or which have not even been identified. The randomised trial design focuses specifically on the effect of the experimental intervention, and the analysis is relatively uncomplicated by the existence of other risk factors which might otherwise affect the outcome.

Randomised trials usually involve follow-up of their participants over time, and so therefore they may suffer from the same practical difficulties as were mentioned earlier for cohort studies. Furthermore, randomisation is only possible for risk factors or interventions that are actually modifiable. So, while we may potentially be able to affect a risk factor such as smoking in a preventive trial aimed at smoking cessation, and while we can also choose to offer a screening program or not, it will not be possible to use a randomised approach to non-modifiable exposures such as genetic risk factors or other personal characteristics of the participants such as age, sex, or ethnicity, even though those factors may indeed be predictive of disease risk.

Other designs

There are a many other designs available to the epidemiologists, but most of them are variants of the major designs we have discussed above, while others are more specialised. For instance, there is a range of specialised design and analysis methods that attempt to identify clusters of disease occurring near some presumed hazard (for example, a land-fill site or a potentially hazardous factory). Similarly, specialized techniques may be required to investigate the causes of an apparent outbreak of disease.[Morabia and Hardy, 2004] Although the technical details of how these specialised studies are carried out may differ, they also fundamentally attempt to find out if the risk of disease in one part of the population is higher than elsewhere, so here once again, making a suitable comparison is an integral part of the epidemiologist’s approach.

Methods to summarise comparisons

The various designs described above may lead to the results being expressed in somewhat different ways, depending on the circumstances. For example, follow-up investigations such as cohort or randomised trial studies may express their results in terms of relative risks or absolute differences in risk; in contrast, results based on data such as is collected in a case-control design cannot be expressed in terms of absolute measures of disease risk, but they can be expressed in terms of the relative exposure levels of cases vs. controls that correspond to relative risks.[Cornfield, 1951],[Miettinen 1976] In the clinical arena, results are sometimes expressed in terms of the NNT, or the Number Needed to Treat, a value that indicates the number of patients that one would need to treat with a new intervention in order to prevent one adverse outcome (such as death), compared to the situation if the patients were treated with the existing standard therapy.

While these details of the way results are reported differ according to their associated study designs, all of the study designs have in common that they are fundamentally making comparisons between various population sub-groups, over time, or between different geographic areas. And they all have a similar purpose: to demonstrate if the risk of disease is higher or lower according to the exposure to certain risk factors, or if the risk of good or bad outcomes depends on which of two or more alternative treatments are used.