Grant Support: Dr. Redelmeier was supported by a career scientist award from the Ontario Ministry of Health, the de Souza Chair in Clinical Trauma Research of the University of Toronto, and the Canadian Institute for Health Research. Mr. Singh was supported by the Jane and Howard Jones Bursary at the University of Toronto.

Critical revision of the article for important intellectual content: D.A. Redelmeier, S.M. Singh.

Final approval of the article: D.A. Redelmeier, S.M. Singh.

Provision of study materials or patients: S.M. Singh.

Statistical expertise: D.A. Redelmeier.

Obtaining of funding: D.A. Redelmeier, S.M. Singh.

Administrative, technical, or logistic support: D.A. Redelmeier.

Collection and assembly of data: D.A. Redelmeier, S.M. Singh.

From University of Toronto, the Clinical Epidemiology and Health Care Research Program, Sunnybrook and Women's College Health Sciences Centre, and the Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada.

Clinical Slide Sets

Terms of Use

The In the Clinic® slide sets are owned and copyrighted by the American College of Physicians (ACP). All text, graphics, trademarks, and other intellectual property incorporated into the slide sets remain the sole and exclusive property of the ACP. The slide sets may be used only by the person who downloads or purchases them and only for the purpose of presenting them during not-for-profit educational activities. Users may incorporate the entire slide set or selected individual slides into their own teaching presentations but may not alter the content of the slides in any way or remove the ACP copyright notice. Users may make print copies for use as hand-outs for the audience the user is personally addressing but may not otherwise reproduce or distribute the slides by any means or media, including but not limited to sending them as e-mail attachments, posting them on Internet or Intranet sites, publishing them in meeting proceedings, or making them available for sale or distribution in any unauthorized form, without the express written permission of the ACP. Unauthorized use of the In the Clinic slide sets will constitute copyright infringement.

This feature is available only to Registered Users

2 Comments

The large survival advantage "“ almost 4 years -- for Academy Award"“winning actors and actresses over their less successful peers [1] continues to receive attention. We write to point out that the statistical method used to derive the statistically significant survival difference gave the Oscar winners an unfair advantage, and to help readers recognize or avoid similar artifacts in other research reports.

The report was based on 235 Oscar winners; 527 other nominees who never won; and 887 "controls" who were never nominated, selected from performers of the same sex and approximately same age who played in the movies for which the nominees were nominated. In the primary analysis, survival was measured from birth, but other definitions of "time-zero" were also used. In all but one of the Kaplan-Meier, Log-Rank and Cox proportional hazards analyses reported, each performer was classified as a "winner" or "not a winner" One reported analysis employed "winner" as a time-dependent covariate, to reflect the fact that all started out as "non -winners", but that some changed status over time.

In the (more emphasized) authors' principal comparison, the Kaplan- Meier curves showed a 3.9 year advantage for the winners. The Cox model, with "winner" as a fixed-in-time covariate , yielded mortality rate reductions ranging from 28% (no adjustment) to 23% (adjustment for seven other covariates), all with 95% confidence limits above 0%. The one reported set of analyses that treated each performer's status (as winner or loser) as dynamic (time-dependent) yielded a reduction of 20%; the lower limit of the CI was 0%, i.e., the reduction was "just significant" at the conventional P=0.05 level. The abstract and the Figure focused on the 3.9 years and the 28% reduction in death rates, which were obtained without adjustment, and without taking into account that a performer's status changed with time.

The analyses that classified those who ultimately won as "winners from the outset" gave them an inbuilt survival advantage, by crediting the years prior to winning towards survival subsequent to winning an Oscar. These "˜immortal' years [2,3] were in fact a requirement for membership in the winners' group: winners had to survive long enough to win (more than 79 years in the two most extreme cases: see Figure). Conversely, actors who did not win had no minimum survival requirement, and some died even before some winners had won, i.e., before some "longevity contests" could begin. For example, 145 "non-winners" had already died by age 65, before 15 of the winners had even won; these unfair pairings (e.g., Burton vs. Burns) were implicitly included in the overall longevity contest between the two groups, and contributed to the apparent survival advantage of the winners, even if winning brought no survival benefit.

To estimate the longevity benefits of winning an Oscar, the comparison should begin at the time that each actor first won, and the "˜remaining-longevity' contest should only include those alive at the same age as the winner was when the winner won. A winner may legitimately be included in comparisons (risksets) prior to winning, but as a "non- winner".

An analysis where the status of a performer who ultimately won is treated as a winner throughout, even in the risksets prior to winning, produces "˜immortal time' bias. As the figure illustrates, a longevity that is measured from a time-zero that precedes the performer's Oscar win (e.g., an individualized one such as the day each performer's first film was released, or a common one such as each performer's 0th or 50th birthday, as in [1]) will necessarily contain some immortal time; there is no such immortality guarantee for those who do not win. Similarly, the matching process, involving a performer "who played opposite" a nominee, ensured that a "control" was alive when a person who ultimately won was nominated, but not necessarily at the time that winner won (the comparison of the 235 nominees who won vs. the 527 other nominees did not involve a matching process).

The authors did report one analysis where each performer's status was updated in each risk set. Annals plans to publish Our methods are described more fully in a technical appendix and our results are presented in a table that will be published by Annals. All of our analyses treat each performer's status as dynamic.

In our re-analyses, which take the immortal time, as well as the covariates sex and year of birth, into account, the point estimate of the actuarial advantage is approximately 1 year, and not statistically significantly different from 0 (the 95% CI is compatible with 0). The estimated percentage mortality reduction is also correspondingly smaller.

We also directly estimated the magnitude of the "immortal time" bias. In the winners vs. nominated comparison, we estimated that not accounting for "˜immortal time' produced an artifactual longevity advantage of 0.8 years (mortality rate ratio 0.94). In the winners vs. controls, not accounting for the immortal time "“ now more substantial -- between the year of a winning actor's first film and the year (s)he first won produced an artificial longevity advantage of 1.7 years and a mortality rate ratio of 0.87. In 1843, William Farr (4) described the statistical artifact created by classifying persons by their status at the end of follow-up, and analyzing them as if they had been in these categories from the outset. He used as examples the greater longevity of persons who reached higher ranks within their professions (bishops vs. curates; judges vs. barristers; generals vs. lieutenants). Despite textbook warnings (2,5,6), analyses that overlook this subtle bias are still common today.

In some longevity comparisons, e.g., (1,7,8), the consequences of an incorrect conclusion are minor. In the evaluation of the time-extension benefits of therapy (3,9,10), the consequences are more serious. How therefore to detect potential immortal time bias? We suggest that when reports compare two "groups", such as winners vs. nominees, one should carefully examine when and how persons entered a "group". Did being in (or moving to) a group have a time-related requirement? Was the classification based on the status at time-zero, or later? If later, was this accounted for? Is the term "status", which implies it could change, more appropriate than the term "group", where, as in a clinical trial, membership is fixed from the outset? Is it sufficient to classify the person just once, or do we need to re-classify the person-moments (the person at different times)? Showing the timelines may help. And, of course, readers and commentators should be doubly cautious whenever they encounter statistics that seem too extreme to be true.

Conflict of Interest:

None declared

Donald Redelmeier

Toronto

June 6, 2006

Lingering Concerns about Immortality Bias

To the editor,

Sylvestre et al. correctly comment that survival statistics are fallible. Our primary analysis was based on the Kaplan-Meier method because life expectancy is the preferred metric in medical decision analysis [1]. Our article also provided 40 other secondary analyses to explore different models since no one statistic is ideal. Sylvestre et al. argue that the multivariate-adjusted Cox proportional hazards model with a time-varying step function is preferred over our primary analysis approach, do not discuss the limitations of such models, and intimate that other models give an unfair advantage. This position disagrees with us and with other reviews involving our work [2, 3].

We agree that time-varying functions are valuable for addressing a change in status from winning. One drawback with such models can be in assuming the same hazard for all winners following first win; for example, Jodi Foster (first win age 25) and Judi Dench (first win age 62) are assigned identical hazards from age 63 until death. However, we found earlier wins were associated with greater advantages, contrary to this assumption. Adding fixed covariates that additionally model age (linear or quadratic) is no simple solution because the likelihood of winning is no simple function of age. The models also have limited power on small datasets, assume no unmeasured heterogeneity, and rarely capture complex trajectories (eg, multiple films, nominations, and wins) [4, 5, 6, 7].

We thank many scientists for analyses of our database. We have also done an update to March 29, 2006 and observed 122 more individuals and 144 more deaths since first publication. Our primary unadjusted analysis shows a smaller survival advantage of 3.6 years (79.7 vs 76.1, p = 0.005). Applying Model 1 of the Sylvestre technical appendix so that winners are treated in a time-varying manner yielded a 8% mortality reduction (95% confidence interval: -14 to +26, p = 0.455). Modifying Model 1 so that both winners and non-winners are treated in a time-varying manner yielded a 15% mortality reduction (95% confidence interval: -6 to +31, p = 0.140). These estimates overlap earlier results. Apparently, the survival advantage depends on the analytic method chosen.

The statistical debate concerns inbuilt survival advantages that yield an immortality bias. We provided methods for addressing this bias, observed multiple findings suggesting this bias was not large in our cohort, and estimated the hidden confounding that would need to be postulated. We found no survival advantage comparing individuals with many nominations to individuals with no nominations, for example, contrary to estimates of a large immortality bias. Moreover, we presumed individuals not reported dead were alive, which is a different type of immortality bias that causes almost all of our analyses and Sylvestre et al.'s analyses to underestimate survival differences.