Comparative effectiveness research gaining traction, finding best practices in the real world

By Jan Bowers, contributing writer, January 02, 2012

Comparing two or more treatments in a real-world setting is not a new concept, but comparative effectiveness research (CER) has recently become a hot topic in medicine, thanks to an infusion of funding from the American Recovery and Reinvestment Act (ARRA) of 2009. At the national level, CER is a key component of emerging health policy, with ramifications for clinical practice and reimbursement. Its influence could be especially significant for dermatology, where experts say solid evidence for many treatments is lacking. “I think it’s very important in dermatology, because there just hasn’t been that much research in our field that has compared treatments so that we know what works best,” said David J. Margolis, MD, PhD, professor of dermatology and epidemiology at the University of Pennsylvania’s Perelman School of Medicine. “Many of our therapies don’t even have good efficacy studies to support their use. Like other branches of medicine, dermatology is full of treatments that are based on what I was told to do’ or what I think works best,’ and not necessarily more compelling evidence than that.”

Effectiveness, not efficacy

The Institute of Medicine (IOM) committee charged with setting priorities for federally funded CER defined it as “the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.” A dermatologist currently conducting CER for wound care put it more succinctly: “Essentially, the goal of CER is to figure out what works, and for which patients, and when. And it looks at real-life outcomes,” said Robert S. Kirsner, MD, PhD, vice chairman, professor, and Stiefel Laboratories chair of the department of dermatology and cutaneous surgery and chief of dermatology at the University of Miami Miller School of Medicine. “CER often provides data that traditional randomized controlled studies don’t provide, and the methodologies are different.”

The efficacy studies conducted by drug manufacturers seeking U.S. Food and Drug Administration approval for their products typically are designed with restrictions that limit their usefulness in clinical practice, Dr. Kirsner said. “In psoriasis studies, for example, people with erythrodermic or pustular psoriasis typically are not enrolled,” he noted. “We’re often not able to look at patients with liver or kidney disease because they’re excluded from the studies, along with certain medications. Certainly you can do follow-up studies, but each one is expensive and they may continue to exclude different populations or co-morbid patients.” In contrast, he said, a CER study will include “a data set of all comers” that would allow researchers to compare, for example, how patients with and without kidney disease respond to a particular treatment.

Other dermatologists emphasize that just because a drug has been proven to work in the controlled practice environment of an efficacy study doesn’t necessarily mean it will be effective in day-to-day practice. “In an efficacy situation, in a clinical trial, you’re very well-educated in how you use a drug or device; you have documentation in front of you every time you see a patient to make sure you’re doing it properly,” Dr. Margolis said. “So I have a drug that in the ideal situation works really well, but then we release it into the general community and no one can use it properly, and it doesn’t seem to work. You have to ask yourself, is that a good drug? Maybe it’s not. There are lots of therapies that we think we’re using properly, and we’re probably not.” The level of evidence from CER should be regarded at least as highly as that from efficacy studies, Dr. Margolis maintained. “The efficacy study shows that something does work; the effectiveness study shows whether it works in the population you’re trying to treat. They go hand in hand.” [pagebreak]

Federal funding fuels new CER efforts

The 2009 federal stimulus legislation authorized $1.1 billion for CER and directed the funds to the National Institutes of Health, the Agency for Healthcare Research and Quality (AHRQ), and the Department of Veterans Affairs. The ARRA also directed the IOM to recommend national priorities for research questions to be addressed by CER. In doing so, the IOM solicited and received input from a broad range of stakeholders, including consumers, provider groups, patient advocacy groups, insurers, manufacturers, and academic institutions. A Web-based questionnaire requesting nominations received 1,758 submissions naming more than 2,600 topics. Among the 100 research topics selected as high priority, four involved dermatology: psoriatic arthritis, psoriasis, wound care, and acne. (See "Dermatology in the IOM Top 100" sidebar)

The Patient Protection and Affordable Care Act of 2010 established a public-private partnership known as the Patient-Centered Outcomes Research Institute (PCORI), charged with establishing its own research priorities and conducting research to help patients and providers make more informed decisions. Beginning in 2013, Medicare and private health insurance companies will pay a tax into a trust fund to support PCORI research. PCORI announced its first major funding opportunity, the $26 million Pilot Projects Grant Program, in the fall of 2011.

“Although CER has existed for 20 or 30 years, this new federal funding will create a lot of research opportunities unless the health reform legislation gets overturned,” Dr. Margolis said. “Governments that provide health care to their population, like the European Union, have been much more aggressive about determining effectiveness, and how well a treatment will work in the population at large. But now the U.S. is very interested as well, and I believe that CER will play a more important role in health care as we go forward, especially if there’s government health insurance.” In the debate preceding passage of the Affordable Care Act, some critics voiced concern that government-funded CER might lead to “rationing” of care or denial of costly treatments. In fact, as noted in a Health Affairs policy brief on CER (“Health Policy Brief: Comparative Effectiveness Research,” Health Affairs, Oct. 5, 2010), Congress rejected using cost-effectiveness analyses to aid Medicare coverage and reimbursement decisions. Dr. Margolis emphasized that while CER might eventually be used to help determine which treatments are reimbursed, it is not cost-effectiveness research. “That’s not to say cost-effectiveness studies might not be worthwhile to do, but that’s not what CER is about,” he said. “A study might conclude that a cheap therapy works better than a very expensive one, but the opposite could happen as well.” [pagebreak]

CER in dermatology

Dermatologists across the United States are engaged in CER projects with and without government funding. The Dermatology Clinical Effectiveness Research Network (DCERN), established in 2009 through a grant from the National Institute for Arthritis, Musculoskeletal and Skin Diseases, conducts CER to improve the quality of care for patients with moderate to severe psoriasis. “Our first study was an attempt to understand dermatologists’ preferences for first-line therapy in moderate to severe psoriasis,” said DCERN founder Joel M. Gelfand, MD, medical director of the Clinical Studies Unit and assistant professor of dermatology and epidemiology at the University of Pennsylvania Health System. “We also wanted to figure out how much uncertainty and difference of opinion is out there regarding first-line treatment how much variability exists in the community, and where there’s a lot of variability, why that is and whether it’s appropriate.” The group’s findings, published online in the Journal of the American Academy of Dermatology in August 2011, indicated that “while most dermatologists prefer phototherapy, there’s variability that has nothing to do with the patient’s situation,” Dr. Gelfand noted. “For example, male dermatologists show a stronger preference for phototherapy than female dermatologists, and derms in the Midwest prefer adalimumab more than those in other parts of the country.” DCERN also examined data on 1,800 psoriasis patients from 10 dermatology practices across the U.S. to rank the effectiveness of different therapies in terms of treatment satisfaction and objective response rate, Dr. Gelfand said. Those results are currently under review. (“Psoriasis Therapy and Management,” a session highlighting the Academy’s evidence-based guidelines for psoriasis and psoriatic arthritis, which cover first-line and advanced therapies, took place at the Academy’s 70th Annual Meeting.)

CER is “incredibly necessary” in advancing the treatment of psoriasis, Dr. Gelfand said. “We really don’t have a good sense, for patients, about which therapy is going to be the best treatment for them over the long term. We have a little bit of head-to-head data from short-term clinical trials that don’t reflect what happens in the real world. We have big knowledge gaps about how best to treat the disease and how to deliver these therapies.”

Dr. Kirsner analyzed the effectiveness of advanced biological therapies for diabetic foot ulcers using data from a company that administers wound centers throughout the U.S. His findings, published in Archives of Dermatology (2010;146(8):857-62), revealed that wounds treated with engineered skin as the first advanced biological therapy were 31.2 percent more likely to heal than wounds first treated with topical recombinant growth factor, and 40 percent more likely to heal than those first treated with platelet releasate. He said he is also working with Dr. Margolis, using a data set from a different wound healing corporation, to determine when hyperbaric oxygen is most beneficial. “We’re studying factors such as wound size, stage, depth, and whether or not there’s a concomitant bone infection or arterial disease,” he noted. “Part of the project is funded by the wound healing company, but it’s also supported by the AHRQ. So it serves as an example of a partnership between industry and government to try to find what works best for what patients.”

Dr. Kirsner remarked that the increasing use of electronic health records (EHR) will help speed the progress of CER. “Often as we transition to EHR, we resent it because it’s new, it’s change, it’s more work,” he said. “But once you begin to collect well-done, accurate data about dermatologic disease and treatment, people can go back and analyze that data and compile a robust data set. So I think all of us, in some way, shape or form, will be contributing through the use of EHRs, and that information will be valuable in many ways.”

Presentations at the 2010 annual meeting of the Cochrane Skin Group at the University of Colorado Denver provided a snapshot of additional CER efforts in dermatology. The Cochrane Collaboration, registered as a charity in the United Kingdom, is an international network of more than 28,000 researchers dedicated to conducting systematic reviews of primary research in health care and health policy. The Cochrane Skin Group produces and updates reviews and meta-analyses of published and unpublished trials relating to skin conditions.

The two-day meeting, dedicated to discussions of CER in dermatology, included presentations of several different types of research, not just the systematic reviews conducted by the Cochrane Skin Group. Dr. Margolis discussed the limitations of randomized clinical trials and described alternative study designs for CER such as large pragmatic studies, traditional cohort studies, and cohort studies that use propensity scores or instrumental variables. Such alternatives are useful, he maintained, because “as soon as you start interfering, having a study protocol, you change the nature of treatment. You start telling people exactly what they’re supposed to do, and they treat their patients better. So the most rigorous CER studies have very, very simple protocols.”

Other presentation topics included:

A review of all studies that assess the therapeutic value of topical devices indicated for atopic dermatitis and an assessment of the comparative efficacy;

The role of long-term safety in determining the comparative effectiveness of psoriasis treatments;

CER in keratinocyte carcinoma; and

The comparative efficacy of the multiple treatments containing benzoyl peroxide and clindamycin for acne. [pagebreak]

Long-term impact

Robert Dellavalle, MD, associate professor of dermatology and public health at the University of Colorado and Cochrane Skin Group editor, said he believes that CER is gaining traction in the U.S. and that “eventually, the government will not approve new drugs just because they work better than placebo alone. I think they’re going to have to be compared to other drugs. The FDA doesn’t currently require CER in its approval process, but I hear that it’s being increasingly scrutinized and encouraged.”

Although the conditions selected as top priority by the IOM, PCORI, and other federal health agencies are likely to be the first to benefit from CER, Dr. Margolis insisted that “everything will be impacted, not just those conditions on the list. All of medical practice, you could argue, should at some point be studied in a more careful way. Dermatologists, just like everybody in medicine, need to embrace the importance of these kinds of evaluations and not hide from them but learn from them.”

Dr. Gelfand maintained that CER is “crucial for our specialty. If we don’t do this research, we’re not going to understand how to best treat our patients. Increasingly, payers are linking payment to performance, and CER is a key opportunity for us to demonstrate the value of our care in improving the lives of patients with skin disease.”

Dermatology in the IOM top 100

In formulating its priority list of 100 research questions, the Institute of Medicine took into account factors relating to specific conditions (e.g., burden of disease, cost and variability) as well as appropriateness for comparative effectiveness research, gaps in existing knowledge, and the likelihood that the research results would improve health.

Four dermatology-related research topics made the IOM’s top 100 priorities:

Psoriatic arthritis: Compare the effectiveness of different strategies of introducing biologics into the treatment algorithm for inflammatory diseases, including Crohn’s disease, ulcerative colitis, rheumatoid arthritis, and psoriatic arthritis.

In formulating its priority list of 100 research questions, the Institute of Medicine took into account factors relating to specific conditions (e.g., burden of disease, cost and variability) as well as appropriateness for comparative effectiveness research, gaps in existing knowledge, and the likelihood that the research results would improve health.

Four dermatology-related research topics made the IOM’s top 100 priorities:

Psoriatic arthritis: Compare the effectiveness of different strategies of introducing biologics into the treatment algorithm for inflammatory diseases, including Crohn’s disease, ulcerative colitis, rheumatoid arthritis, and psoriatic arthritis.