Nuanced Statistical Methods Advocated For Clinical Trials

Action Points

Be aware that the overall results and subgroup analyses presented in many clinical trials may miss subgroups of high-risk and low-risk patients who are more likely to be helped or harmed by the treatment in question.

ANN ARBOR, Mich., Nov. 11 - By relying on "average" statistical results reported by most clinical trials, physicians may make treatment decisions that do more harm than good for significant numbers of patients, researchers here said.

Because of the statistical methods they employ, the vast majority of clinical trials overlook identifiable subgroups of patients likely to be harmed by a new treatment while greatly overestimating or underestimating benefits to others, said Rodney A. Hayward, M.D., of the Veterans Affairs Ann Arbor Health Care System.

But the problem can be fixed, Dr. Hayward and colleagues said in a paper in the November-December issue of Health Affairs.

Rather than presenting results as the average benefit across all people in the trial, with subgroup analyses based on a single variable such as age or sex, investigators should conduct risk-stratified analyses that include several variables at once. Risk-stratified analyses identify high-risk patients likely to be helped and low-risk patients who may be harmed,

The researchers reviewed 108 clinical trials reporting results on major patient outcomes that were published during 2001 in the Journal of the American Medical Association, The Lancet, and the New England Journal of Medicine. Of the 108 studies, four employed risk-stratified analyses.

In addition, the Michigan team identified ad-hoc, risk-stratified analyses conducted by other researchers that revealed surprising gaps and weaknesses in the original results of important clinical trials on which treatment recommendations were based.

For example, the European Carotid Surgery Trial, published in TheLancet in 1998, reported that carotid endarterectomy reduced the absolute risk of major stroke or death by almost 12%. A conventional "one variable at a time" subgroup analysis failed to identify any patient subgroup that would not benefit from the surgical procedure.

Subsequent re-analysis with a risk-stratified model, however, found that patients with the highest risk of stroke benefited dramatically, but the typical patient in the study received no benefit. Furthermore, the surgery resulted in net harm for many patients with the lowest risk of stroke.

"Overall, they found that 16% of the patients, whose risk was more than three times greater than that of the remaining 84% of study subjects, accounted for almost all of the benefit seen in the 'average' result," Dr. Hayward and colleagues wrote.

When analyzing data from clinical trials, "it can be far easier to deal with simple averages and artificial dichotomies," Dr. Hayward and colleagues wrote. "Thus, we predict that the most difficult challenge for risk-stratified analysis will come from the ways in which this approach will inevitably make decision-making more challenging and nuanced."

"Given the obvious economic incentives for industry (and researchers with strong financial connections to industry) to get treatments approved for as broad a population as possible, regulatory agencies such as the U.S. Food and Drug Administration (FDA) and the U.K. National Institute for Clinical Excellence (NICE) should consider requiring risk-stratified analysis," the Michigan team concluded.

Reviewed by Zalman S. Agus, MD Emeritus Professor at the University of Pennsylvania School of Medicine

Accessibility Statement

At MedPage Today, we are committed to ensuring that individuals with disabilities can access all of the content offered by MedPage Today through our website and other properties. If you are having trouble accessing www.medpagetoday.com, MedPageToday's mobile apps, please email legal@ziffdavis.com for assistance. Please put "ADA Inquiry" in the subject line of your email.