ABSTRACT

BackgroundThe Food and Drug Administration (FDA) and pharmaceutical manufacturers conduct most postmarketing pharmaceutical safety investigations. These efforts are frequently based on data mining of databases. In 1998, investigators initiated the Research on Adverse Drug events And Reports (RADAR) project to investigate reports of serious adverse drug reactions (ADRs) and prospectively obtain information on these cases. We compare safety efforts for evaluating serious ADRs conducted by the FDA and pharmaceutical manufacturers vs the RADAR project.

MethodsWe evaluated the completeness of serious ADR descriptions in the FDA and RADAR databases and the comprehensiveness of notifications disseminated by pharmaceutical manufacturers and the RADAR investigators. A serious ADR was defined as an event that led to death or required intensive therapies to reverse.

ConclusionProactive safety efforts conducted by the RADAR investigators are more comprehensive than those conducted by the FDA and pharmaceutical manufacturers, but dissemination of related safety notifications is less timely.

Figures in this Article

Adverse drug reactions (ADRs) result in health care costs of $3.6 billion annually and are among the top 10 leading causes of death.1- 3 After product recalls for rofecoxib and cerivastatin, concern exists over the ability of the Food and Drug Administration (FDA) and pharmaceutical manufacturers to identify serious adverse events and to notify physicians about these findings.4- 8 Serious ADRs can go unrecognized if initial safety signals are unclear.2 New approaches to identifying unexpected pharmaceutical toxic effects are needed.9,10

One concern is that current pharmacovigilance efforts are hindered by low reporting rates, and when ADRs are reported the information is of variable quality.11 When pharmaceutical manufacturers and the FDA become convinced that an event represents a serious ADR, safety notifications are disseminated.2,12 Physicians may not be aware of serious ADRs if dissemination of these notifications is delayed, the safety notification is not comprehensive, or they do not read the notifications.

Academic researchers and nonprofit organizations have developed systems to identify and evaluate serious ADRs.13- 16 As noted previously, surveillance efforts conducted by these groups differ from those of the FDA: the organizations focus on serious ADRs, and additional reports are obtained after queries to health care professionals who provide care for patients with the relevant toxic effects.17 The deputy director of the FDA proposed that adverse event reports obtained by proactive pharmacovigilance should be transmitted to the FDA and would augment existing pharmacovigilance systems.18 However, before augmentation can occur, comparisons of these pharmacovigilance efforts are needed. Herein, we evaluate 2 questions that can inform this comparison: how complete are adverse event reports included in databases obtained by these systems, and how timely and complete are safety notifications disseminated by these programs? We limit these comparisons to recent safety evaluations conducted by one of the largest nonprofit pharmacovigilance programs, the Research on Adverse Drug events And Reports (RADAR) project.14

METHODS

The RADAR project is a pharmacovigilance program that focuses on identification, evaluation, and dissemination of information describing serious ADRs, such as those resulting in death or severe organ failure or precipitating major therapeutic interventions. The RADAR project is funded by grants from the National Heart, Lung, and Blood Institute, the National Cancer Institute, the American Cancer Society, and the Department of Veterans Affairs. Pharmaceutical manufacturers do not provide support, although they are asked to provide relevant clinical information on each serious ADR.

The RADAR project is led by a hematologist-oncologist and health services researcher (C.L.B.) and consists of 25 investigators with training in internal medicine and the medical subspecialties of clinical pharmacology, epidemiology, statistics, and pharmacy. An investigation is initiated when a clinical event that represents a possible occurrence of a serious ADR is seen by a RADAR coinvestigator or reported by unaffiliated physicians. Senior members of the RADAR team review the indicator event, as well as associated literature and package inserts, to determine if the event represents an ADR. If further investigation of the ADR is meritorious, queries for additional case reports are submitted to the FDA and institutional review board approval is requested at collaborating institutions. The FDA reports are subject to a preliminary review to inform hypothesis generation and refine data collection tools. The case forms include demographics, source of information (eg, clinical trial or physician queries), medication history, event description (eg, date of the event, time elapsed between exposure and ADR event, event duration, other relevant history, physical findings, and study results), organ-specific history, and treatments. World Health Organization criteria are used to score causal associations between the suspect drug and the event. The RADAR investigators refine hypotheses about pathophysiologic features. Data elements and coding accommodate the range of data available in the FDA databases and address underlying causality hypotheses. Coding of the case classification form is designed to facilitate algorithmic analysis of case findings (Figure).14

DATA SOURCES

Initial sources for RADAR investigations are descriptions of single events of serious ADRs identified by a RADAR investigator, often originating with queries from health care workers, attorneys, or patients. The RADAR data sources for obtaining additional information include communications with clinical trialists or referral centers that treat large numbers of individuals with the given toxic effect. The FDA data sources for obtaining follow-up reports include spontaneous adverse event reports contained in MedWatch19 (for drugs) and the Manufacturer and User Facility Device Experience Database (MAUDE)20 (for drug-coated devices). Dissemination of RADAR findings occurs via published abstracts or articles. Dissemination of FDA and pharmaceutical supplier findings is made via “Dear Doctor” letters and package insert revisions.

OUTCOME MEASURES

Safety elements were abstracted onto syndrome-specific forms.21- 32 These forms are based on the ADR systems of the World Health Organization33 and Naranjo et al.34,35 Data elements included history and physical examination, laboratory,radiologic, and basic science findings; age and sex; clinical diagnosis for which the drug was being prescribed; duration and dose of the drug administered; use of concomitant drugs; relevant clinical and laboratory findings that supported the diagnosis of the adverse event; duration of clinical abnormalities; exposure-adjusted incidence rates; treatments and prophylaxis; diagnosis date; date of reporting to the FDA or RADAR investigators; rechallenge and dechallenge; and outcome.

Data elements in the summary safety notification analysis included date of publication (for manuscripts and abstracts) or distribution (for package insert revisions or Dear Doctor letters); descriptions of clinical findings, exposure-adjusted incidence rates, and treatment and prophylaxis regimens; and references to relevant peer-reviewed articles. For each of the safety evaluations, completeness percentages were derived for individual case descriptions contained in the respective safety databases.21- 32 Specific categories included history and physical examination findings, laboratory and radiologic test results, and basic science correlative studies. For each summary safety notification published as a peer-reviewed publication, Dear Doctor letter, or package insert revision, mean percentages were derived for frequency of inclusion in these reports of relevant clinical findings, estimated reporting and exposure-adjusted incidence rates, treatment and prophylaxis recommendations, and references to relevant peer-reviewed articles. A time to initial publication was derived for each summary notification, operationally defined as the time from FDA approval to publication (for RADAR notifications) or ADR notification via a Dear Doctors letter (for FDA or pharmaceutical supplier notification) or revised package inserts (for pharmaceutical supplier notifications).

STATISTICAL ANALYSIS

Overall unweighted mean percentages of completeness for inclusion of information from history and physical examinations, laboratory and radiologic tests, and basic science correlative studies were derived for clinical adverse event reports included in the FDA and RADAR safety databases. Statistical comparisons were made of completeness percentages for these individual data elements in FDA vs RADAR databases. With respect to dissemination, we reported mean completeness percentages for inclusion of relevant safety information in notifications from RADAR and the pharmaceutical manufacturers, as well as median time to publication of RADAR notification vs safety notifications from pharmaceutical manufacturers and/or the FDA.36- 47 Statistical analyses were conducted using optimal discriminant analysis, a nonparametric exact statistic.48 The project received institutional review board approval from Northwestern University.

These clinical events required primarily intensive interventions, and occasionally patients died of the toxic effect. For each ADR, descriptions of additional detailed case information were obtained from health care professionals who conducted early-phase clinical trials, had referral practices that evaluated or treated patients with the relevant toxic effect, or directed specialty centers such as blood banks or leukemia referral centers. The number of individual event reports associated with each of the ADRs ranged from 3 to 1348. Overall, RADAR databases included 472 individual reports and FDA databases included 2296 reports.

Timing and completeness of safety notifications disseminated by pharmaceutical manufacturers and RADAR investigators differed (Table 2). Safety notifications were disseminated earlier by pharmaceutical manufacturers, usually as package inserts or Dear Doctor letters (median of 2 vs 4 years after FDA approval). For 4 ADRs (thrombotic thrombocytopenic purpura associated with clopidogrel, venous thromboembolism associated with thalidomide and lenalidomide, nevirapine-associated hepatotoxicity, and bisphosphonate-associated osteonecrosis of the jaw), RADAR safety notifications were the primary sources of summary safety information for the RADAR project, as well as for the related Dear Doctor letters and package insert notifications disseminated by pharmaceutical manufacturers.22,31 For 2 ADRs (fulminant hepatic failure with nevirapine that contained post–HIV-exposure prophylaxis regimens and enoxaparin-associated severe hemorrhages after invasive cardiologic procedures), safety notifications were made via RADAR publications but are not described in materials distributed by the pharmaceutical manufacturer because these toxic effects occurred in off-label settings.

Compared with notifications disseminated as pharmaceutical package insert revisions or Dear Doctor letters, RADAR notifications were more likely to include reporting rates or exposure-adjusted incidence rates (93% vs 46%; P = .02), outcomes (100% vs 8%; P<.001), treatment and prophylaxis recommendations (93% vs 25%; P<.001), and peer-reviewed references (80% vs 8%; P<.001). Of note, for drug-eluting cardiac stents, a Dear Doctor letter issued on October 29, 2003, described rare instances of hypersensitivity when drug-eluting stents were placed in coronary arteries.52 One month later, a follow-up notification from the FDA indicated that these cases were most likely related to hypersensitivity to concomitantly administered drugs, primarily clopidogrel.53 In 2004, a RADAR notification indicated that some instances of hypersensitivity appeared to be caused by the stents, since autopsies revealed eosinophilic infiltrates at the stent site for 4 patients and information in FDA notifications rarely supported clopidogrel as the cause.27,54

COMMENT

In this review of pharmacovigilance efforts associated with 16 serious ADRs, we found that clinical information obtained by the proactive RADAR program was of higher quality, but lower quantity, than that contained in the combination of spontaneous adverse event reports submitted to the FDA and adverse event reports submitted to the FDA by pharmaceutical manufacturers. Also, safety notifications from the RADAR investigators were more complete than those from pharmaceutical manufacturers or the FDA, although they were less timely. Our findings highlight differences between proactive pharmacovigilance efforts conducted by the RADAR group vs pharmacovigilance efforts conducted by the FDA and pharmaceutical manufacturers.55 In interpreting our findings, several factors should be considered.

Pharmacovigilance efforts of the FDA or the pharmaceutical supplier did not result in the initiation of investigations of any of the serious ADRs described herein. Indeed, none of the 5 ADRs for which more than 200 individual case descriptions resided in the FDA safety databases were investigated before the initial reports of serious clinical events associated with these ADRs were published (Table 1). In contrast, each of the RADAR investigations was initiated when a serious clinical event that represented a potential serious ADR was identified by a RADAR investigator. As noted by others, despite having a large number of ADR reports in FDA databases, it is a difficult task to clearly identify a “signal” that suggests a new ADR has occurred vs “noise” associated with complex clinical settings.10

With respect to follow-up evaluations, the FDA receives more than 400 000 reports of serious and nonserious ADRs annually.56 It is impractical for the FDA to request follow-up information on most of these cases. However, in rare instances in which follow-up clinical information for an individual case report has been requested by pharmaceutical manufacturers or the FDA, health care professionals rarely comply with these requests.10 In contrast, health care professionals almost universally responded to follow-up queries from RADAR investigators. Although not paid for these efforts, participating physicians have subsequently contributed as coauthors to the summary RADAR publications that are disseminated in peer-reviewed medical journals.11 The end result is that large numbers of incomplete ADR reports are submitted by many health care professionals to the FDA and pharmaceutical manufacturers, whereas a small number of health care professionals submit fewer but more completely described safety reports to the RADAR program. It is the quality, not the quantity, of the case report information that primarily supports successful pharmacovigilance efforts. For 8 serious ADRs, RADAR summary reports were based on comprehensive review of a median of only 20 case reports for the individual toxic effect. Causality assessments, the most difficult and one of the most important aspects of pharmacovigilance studies, were included in several RADAR investigations, such as biopsy or autopsy findings or antibody test results for a small number of patients, but not in the pharmaceutical or FDA summary safety reports.

The RADAR notifications were disseminated a median of 2 vs 4 years after FDA approval. These notifications were disseminated primarily as peer-reviewed manuscripts, although some of the more urgent notifications were rapidly distributed as electronic preprints (clopidogrel-associated thrombotic thrombocytopenic purpura) or research letters (lenalidomide-associated thromboembolism) within 1 year of FDA approval of the relevant drug. The RADAR notifications, but not the FDA or pharmaceutical supplier safety notifications, included information on expected clinical outcomes, recommendations for treatment and prevention, and common strategies for diagnosis of the ADR. The recently released report from the Institute of Medicine's Committee on Assessment of the US Drug Safety System proposed changes that would improve the timing and comprehensiveness of safety notifications disseminated by the FDA.57

For several RADAR investigations, resultant RADAR safety notifications were followed by actions that had a direct impact on the safety of pharmaceuticals in the postmarketing setting (Table 3). These actions were facilitated by market-related factors (clopidogrel rapidly replaced ticlopidine, in part for safety reasons related to lower rates of thrombotic thrombocytopenic purpura as outlined by the RADAR project), support from the Connecticut attorney general (for thalidomide-associated venous thromboembolism), and support from national health authorities (mandating intravenous epoetin alfa administration to patients with chronic kidney disease or contraindicating nevirapine-containing post–HIV-exposure prophylaxis regimens). The Institute of Medicine's Committee on Assessment of the US Drug Safety System recommended that the FDA be granted regulatory authority to affect the manner in which drugs are used or marketed when safety concerns are identified.57

After publication of the recent Institute of Medicine report on drug safety, experts proposed funding a consortium of nonprofit pharmacovigilance centers, such as the Centers for Education and Research on Therapeutics.10,56,58 Collaborations could build on the unique strengths of these centers. For example, the FDA may initially identify a weak safety signal by data-mining efforts and then issue a request for proposal to FDA-approved independent pharmacovigilance centers that have cooperative agreements. These centers might competitively bid on the proposed project, and the selected centers would then conduct proactive pharmacovigilance, identify additional reports of the serious adverse event, and obtain detailed case information on these individuals. The FDA and the selected pharmacovigilance centers would ultimately share safety information. Alternatively, the pharmacovigilance centers might identify signals of serious ADRs and present these findings to FDA officials at regularly scheduled meetings, similar to those that occur between the FDA and the Centers for Education and Research on Therapeutics. In addition, FDA data-mining efforts (application of statistical techniques to spontaneous ADR reports to identify potential associations between drug exposures and outcomes) would look for additional cases. Concise safety summaries may be prepared by pharmaceutical manufacturers with input from the FDA and the proactive safety centers, and dissemination may occur as revised package inserts and/or Dear Doctor letters. Academic investigators could prepare articles for submission to peer-reviewed journals that summarize reporting rates, outcomes, treatment, prophylaxis, and clinical and basic science findings. By tapping into the interests and expertise of safety centers while still maintaining control and focus, the FDA could highlight its strengths (sending rapid, nationwide feedback about new ADRs) while reducing bureaucratic burden.

Our study has limitations. First, additional sources of safety reports exist in databases maintained by institutional review boards, cooperative clinical trial groups, and insurance companies.59 Second, physicians frequently do not respond to Dear Doctor letters or black-box advisories.12,60- 63 Third, the RADAR project focused primarily on ADRs related to drugs or drug-related devices (drug-eluting stents). In 2002, the Office of Devices and Radiologic Health of the FDA initiated the Medical Product Surveillance Network, a collaboration that resembles the proposed FDA-RADAR collaboration. However, the RADAR project, unlike the Medical Product Surveillance Network, benefits from institutional support of multidisciplinary researchers at a National Cancer Institute–designated comprehensive cancer center. As such, the RADAR effort would be difficult to replicate within the FDA. The RADAR project focuses on obtaining a small number of comprehensive safety reports from select centers and provides extensive review and analysis of the individual ADR reports to distinguish signal from noise. Finally, similar assessments of other nonprofit pharmacovigilance efforts, such as the Drug-Induced Liver Injury Network64 and the National Registry of Drug-Induced Ocular Side Effects, should be reported.65

In conclusion, our study identified differences in data quality for safety reports contained in databases obtained by proactive pharmacovigilance efforts conducted by the RADAR project vs the FDA, as well as in the completeness and timing of dissemination of safety notifications. Moving forward, collaborations could be developed that build on the strengths of independent nonprofit organizations that proactively conduct pharmacovigilance efforts and of safety efforts of the FDA and pharmaceutical manufacturers.

Funding/Support: Funding for this project was provided by grants 1 R01 CA102713-01 from the National Cancer Institute (principal investigator: Dr Bennett), 3 R01 CA102713-02S from the National Cancer Institute (principal investigator: Dr Bennett), HL69717-01A1 from the National Institutes of Health (principal investigator: Dr Bennett), 05-215-01-CPPB from the American Cancer Society (principal investigator: Dr Belknap), P30 CA60553 from the National Cancer Institute (Dr McKoy), RCD-02-176, TRP-02-147 from the Veterans Administration Health Services Research and Development (Dr Nebeker), K22 LM 8427-01 from the National Library of Medicine (Dr Dorr), and 5K23HL077404-02 from the National Heart, Lung, and Blood Institute (Dr Courtney).

Acknowledgment: We thank Matthew Fisher and Mark Mai for helpful comments and contributions to early drafts of the manuscript.

Baciu
AStratton
KBurke
SCommittee on the Assessment of the US Drug Safety System, The Future of Drug Safety: Promoting and Protecting the Health of the Public. Washington, DC National Academies Press2006;http://www.iom.edu/CMS/3793/26341/37329.aspx. Accessed October 5, 2006

Johnson
LNPassman
RSBennett
CL Amiodarone-Associated Optic Neuropathy: A Review of 294 Cases From the Research on Adverse Drug Events and Reports (RADAR) Project. Fort Lauderdale, Fla Association for Research in Vision and Ophthalmology2006;

Pomeranz
HAngelotta
CRaisch
DW Sildenafil- and Tadalafil-Associated Optic Neuropathy: An Overview From the Research on Adverse Events and Reports (RADAR) Project. Fort Lauderdale, Fla Association for Research in Vision and Ophthalmology2006;

US Food and Drug Administration, Updated information for physicians on sub-acute thromboses (SAT) and hypersensitivity reactions with use of the Cordis CYPHER sirolimus-eluting coronary stent. http://www.fda.gov/cdrh/safety/cypher2.html. Accessed March 10, 2006

Institute of Medicine, The Future of Drug Safety: Promoting and Protecting the Health of the Public: Report of the Institute of Medicine (IOM) of the National Academies. Washington, DC Institute of Medicine September25 2006;

Baciu
AStratton
KBurke
SCommittee on the Assessment of the US Drug Safety System, The Future of Drug Safety: Promoting and Protecting the Health of the Public. Washington, DC National Academies Press2006;http://www.iom.edu/CMS/3793/26341/37329.aspx. Accessed October 5, 2006

Johnson
LNPassman
RSBennett
CL Amiodarone-Associated Optic Neuropathy: A Review of 294 Cases From the Research on Adverse Drug Events and Reports (RADAR) Project. Fort Lauderdale, Fla Association for Research in Vision and Ophthalmology2006;

Pomeranz
HAngelotta
CRaisch
DW Sildenafil- and Tadalafil-Associated Optic Neuropathy: An Overview From the Research on Adverse Events and Reports (RADAR) Project. Fort Lauderdale, Fla Association for Research in Vision and Ophthalmology2006;

US Food and Drug Administration, Updated information for physicians on sub-acute thromboses (SAT) and hypersensitivity reactions with use of the Cordis CYPHER sirolimus-eluting coronary stent. http://www.fda.gov/cdrh/safety/cypher2.html. Accessed March 10, 2006

Institute of Medicine, The Future of Drug Safety: Promoting and Protecting the Health of the Public: Report of the Institute of Medicine (IOM) of the National Academies. Washington, DC Institute of Medicine September25 2006;

Correspondence

The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.
The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with
the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.

Indicate what change(s) you will implement in your practice, if any, based on this CME course.

Your quiz results:

The filled radio buttons indicate your responses. The preferred responses are highlighted

For CME Course:
A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes

Indicate what changes(s) you will implement in your practice, if any, based on this
CME course.

Instructions

Thank you for submitting a comment on this article. It will be reviewed by JAMA Internal Medicine editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.

Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.

* = Required Field

Comment Author(s)* (if multiple authors, separate
names by comma)

Example: John Doe

Affiliation & Institution*

Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.

This feature is provided as a courtesy. By using it you agree that that you are requesting the material solely for personal, non-commercial use, and that it is subject to the AMA's Terms of Use. The information provided in order to email this article will not be shared, sold, traded, exchanged, or rented. Please refer to The JAMA Network's Privacy Policy for additional information.

Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.