Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

How Is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature

EXECUTIVE SUMMARY

Measuring and understanding the relationship between volume and outcome in the delivery of health services has engaged the attention of an increasing number of researchers since the 1980s. More recently, health care purchasers have begun to debate whether and how they should make use of these research findings in their relationships with health plans and individuals.

We conducted a systematic review of the published research literature to assess the methodological quality of the research and to understand what is known about how volume and outcome are associated, by what causal pathways they might be linked, and how generalizable the research findings are. We developed an 18-point scale to rate the methodological rigor of each study. The scale was based on a conceptual model that identified a series of critical factors that need to be addressed to fully delineate the relationship between volume and quality of care. These factors include: adequacy of risk-adjustment for differences in severity and comorbidity, sample size, unit of analysis, assessment of specific clinical processes of care, organizational processes, skills and experience of the various relevant clinicians, appropriateness of patient selection, and the spectrum of important clinical outcomes.

We identified published, English-language studies by electronic searches of MEDLINE, by consulting with experts, and by hand searching the reference lists of identified studies. We excluded studies with patient cohorts treated prior to 1980, samples that were not community or population-based, those in which a health outcome was not the dependent variable, and those in which volume was not an independent variable. This monograph reports the results of our review of 8 conditions and procedures: coronary artery bypass graft (CABG) surgery, pediatric cardiac surgery, carotid endarterectomy, abdominal aortic aneurysm repair, cancer surgery, percutaneous transluminal coronary angioplasty (PTCA), acute myocardial infarction, and acquired immuodeficiency syndrome (AIDS).

Overall, 88 studies were included in this review. The median quality score was 8, less than 50% of the maximum possible score of 18. The highest score assigned (to 1 study) was 13, 72% of the maximum possible score. All of the remaining studies scored 11 or less. The principal omissions in the published research, which led to lower quality scores, included:

Only 16 considered the independent effects of both physician and hospital volume;

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Some (n = 24) used clinical data to adjust for differences among patients in severity and comorbidity, but only 4 reported statistically robust models;

Few (n = 4) adjusted for differences in specific processes of care; and

Few studies (n = 2) addressed appropriateness of patient selection.

Several conclusions emerged from this review. There can be little doubt that for a wide variety of surgical procedures and medical conditions higher volume (whether assessed by hospital or by physician) is associated with better health outcomes. Although a publication bias against negative findings appearing in print cannot be excluded, the uniformity with which the published research documents or confirms the existence of the association is compelling. Fully 77% of the studies we reviewed found statistically significant associations between higher volume and better outcomes. The remaining 23% did not find statistically significant relationships. No study demonstrated a statistically significant association in the opposite direction. Finally, all 16 of the studies with the highest quality scores found statistically significant associations.

Overall, however, the methodological rigor of the studies was modest, limiting generalizability and leaving many questions about the nature and causes of the association unanswered. Studies of the same procedure or condition typically employed widely varying definitions of high and low volume, precluding definitive conclusions about the nature of its relationship to outcome. We do not know, for example, whether a volume threshold exists—or more than one—above which outcomes are better but do not continue to improve with further volume increases. Is there a threshold effect for physician or hospital volume? Both? Neither? Very few studies addressed the question of whether the only relevant volume measure is that of the specific procedure or condition under study. Thus, we do not know the extent to which experience with procedures (or conditions) that are closely related affects outcome.

Available research does not shed much light on what specific factors explain outcome differences between high and low volume providers. No longitudinal studies (and very few of any design) address the important question of how much of the variability, especially among low-volume providers, is due to chance. Few investigations have assessed differences in specific clinical processes of care, especially those known to affect outcomes. One intriguing exception showed that about one third of the mortality difference between high- and low-volume hospitals for acute myocardial infarction could be attributed to more frequent use of proven-effective medications at high-volume hospitals. We believe that this direction is the most promising avenue for future research: a detailed understanding of those critical clinical and organizational processes that affect outcomes and that differentiate high- and low-volume hospitals and physicians. Such knowledge could provide a blueprint for improvement.

These research findings have several potential implications for public policy in health care. Public policy might be directed at informing consumers and referring physicians about the nature of the relationship between volume and outcome or at improving outcomes, or both. Data on volume of procedures by hospital are publicly accessible in many states through hospital discharge abstract databases. Many of these databases also permit identification of treating physicians. Making these data widely available is one potential way of informing consumers and clinicians. Some of the challenges such efforts would face are the difficulties of explaining what the data mean (and what they do not), the technical challenges of carefully identifying conditions and procedures for which research is conclusive that higher volume is associated with better outcomes (and at what threshold), and the need to update the data regularly.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Regulation in some states has limited the number of low-volume hospitals for some specific services, but such programs have become increasingly politically unpopular and require significant infrastructure to support. Recent actions by private employers suggest that some purchasers may be prepared to take an active role in steering patients toward high-volume hospitals, using incentives directed either at managed care plans or at employees themselves. No data exist on which to base predictions of the impact of such efforts. Available data suggest that significant numbers of patients do not use data on outcomes or volume where they exist to choose where to receive care.

We believe that the most promising use of the research findings linking volume and outcome is to fuel quality improvement. To the extent ongoing research is successful in demonstrating what specific clinical and organizational processes of care are associated with improved outcomes, this knowledge can be put to direct and immediate use to improve hospital and physician care. Public policy can accelerate this improvement by supporting the necessary research, by fostering and evaluating demonstration programs to implement it, and by disseminating successful models.

HOW IS VOLUME RELATED TO QUALITY IN HEALTH CARE? A SYSTEMATIC REVIEW OF THE RESEARCH LITERATURE

“We must formulate some method of hospital report showing as nearly as possible what are the results of the treatment obtained at different institutions. This report must be made out and published by each hospital in a uniform manner, so that comparison will be possible. . . . A set of statistics had been prepared comparing the mortality at a certain semi-private hospital of 200 beds with that of four of the best general hospitals in America, having a total of 1,200 beds. These statistics were obtained from published reports. They clearly showed that the semi-private hospital not only did more operations, but that the mortality was much lower, especially in some of the more difficult branches of surgery.”

Introduction and Historical Context

This author goes on to speculate about what factors might have produced such an apparently anomalous result—that a small hospital might have better outcomes than larger, more prestigious institutions. Among the factors he cited in addition to the volume of operations were the skill of the surgeons, the comorbid conditions of the patients, the financial condition of the hospitals, and the way in which surgical departments were organized to triage particular patients to particular surgeons. Data to answer these questions continue to prove elusive, some 86 years after a Boston surgeon, Ernest Avery Codman, first posed them. (Codman 1914)

Codman's questions lay dormant for more than 50 years, until the National Halothane Study, undertaken in the 1960s to assess possible hepatotoxic effects of halothane, unexpectedly reported a 24-fold variation in operative mortality among participating U.S. hospitals. (Moses 1968) This finding led to a series of further investigations. (Stanford 1976, Flood 1979, Flood 1984a, Flood 1984b) More recently, beginning with the work of Luft and colleagues (Luft 1979, Bunker 1982, Luft 1987), a large body of research has focused on measuring and explaining the relationship between patient outcomes and the volume of specific health services provided by hospitals and physicians.

Many studies have documented that higher volume is associated with better outcomes for a variety of different surgical procedures and medical conditions. Two principal hypotheses have

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Although a few studies have found some indirect evidence that is consistent with either or both of these hypotheses, we believe a reconceptualization is warranted of the model that describes the potential mechanisms by which volume and outcome might be related in health care. This reconceptualization is prompted both by several inconsistencies in the older hypotheses and by evidence from recent research. Although the “practice makes perfect” hypothesis has some intuitive appeal, extant research has been unable to demonstrate what skills or practices improve as hospitals or physicians gain experience with particular kinds of patients and why they are uniquely related to volume. For the “selective referral hypothesis” to be an important explanatory factor, the outcomes of different physicians, surgeons, and hospitals would have to be widely known for referring physicians (if not patients) to use in making treatment recommendations. But we know that such data are not readily available and cannot be the basis for very many patient referrals. Even where they do exist, physicians report that they do not use them to make referrals. (Schneider 1996, Hannan 1997) Nor is there evidence that patients either shun hospitals with well-publicized poor outcomes or flock to those with good ones. (Chassin 1996, Schneider 1998) Further, both of these hypotheses focus on explaining how particular hospitals or physicians might have achieved high volume and good outcomes over time. Neither offers much help in explaining the causes of differences in outcomes among hospitals or physicians with different volumes or in suggesting how the poorer performers might improve. Some recent evidence documenting systematic differences among high-and low-volume providers in the use of specific interventions points to a different set of explanatory factors. (Thiemann 1999)

We, therefore, begin this systematic review of the research evidence linking volume and outcome in health care by describing a new conceptual model. In subsequent sections of this monograph, we describe how we used this model to develop criteria to judge the quality of the published research we reviewed. We then define our literature search strategies, inclusion and exclusion criteria, and methods for reviewing and rating included studies. We discuss the results of the review separately for each included topic area and then discuss generalizable findings and issues. Finally, we discuss the potential policy implications of the findings.

Conceptual Framework

Figure 1 displays the factors we believe are likely to be the most potent explanatory variables in understanding how volume of services is related to health outcomes. Patients are selected to receive surgical or medical services in a variety of ways. Some have little time or choice (e.g., those suffering acute myocardial infarction or ruptured aortic aneurysms). The opposite is true for many surgical procedures, especially for the vast majority of elective operations. We know little about the relationship between volume and patient selection. Are patients of high-volume hospitals or physicians likely to be more or less appropriate candidates for surgery than patients of low-volume providers? A single study documented that high-volume surgeons were more likely to perform inappropriate carotid endarterectomies than low-volume surgeons were. (Brook 1990b) Few investigations have addressed this important dimension of quality.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

More attention has been focused on identifying the key variables that define the severity of patients' presenting illness and the comorbid conditions that also affect the outcomes of treatment. How well these factors are measured and taken into account (i.e., how well the data on outcomes are “risk-adjusted”) are crucial components of the validity of outcome comparisons between high- and low-volume providers. If high-volume surgeons or hospitals treat patients who are systematically sicker (or healthier) than their low-volume counterparts, comparisons of their outcomes are not valid without scientifically rigorous risk adjustment.

Once the patient population undergoing treatment is adequately characterized, it is then necessary to consider exactly what treatments the patients receive. Volume of services cannot directly produce better outcomes. If volume is related to outcome, that association must be expressed by differences in the components of care or in the skill with which treatments are provided. For some conditions or procedures, specific component treatments (processes of care) have been demonstrated to improve outcomes. For example, it is essential to know whether patients undergoing treatment for acute myocardial infarction received therapies designed to achieve reperfusion (acute angioplasty or thrombolytic drugs in appropriate settings). Other effective processes for this condition include aspirin, beta blockers, and angiotensin converting enzyme (ACE) inhibitors, each with its own specific indications. Although data on effectiveness and efficacy for most surgical processes of care are lacking, it is equally essential that researchers investigate the extent to which carotid endarterectomies or coronary bypass operations are performed the same way at high- and low-volume centers and by different surgeons. Do high-volume surgeons employ different surgical techniques compared with low-volume surgeons? If they do different operations, we can characterize the differences and assess the extent to which these measurably different processes of care are associated with different outcomes.

If they do exactly the same operations but obtain different results, we may attribute such outcome differences to differences in skill (or “unmeasured process” differences). Several aspects of this issue are worth noting. First, if physician experience (volume) is a proxy measure for the attainment of a certain skill level that permits superior performance, it is important to consider how that experience is best measured. Does superior performance in carotid endarterectomy or aortic aneurysm surgery derive from experience with just those procedures, or is the volume of other major vascular surgical procedures important also? This issue may be termed the “volume of what” question.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Second, it is important also to identify exactly what skills are related to outcomes. In surgical procedures, specific physical skills may be vital—for example, the speed and dexterity with which a carotid artery is dissected, minimizing trauma to adjacent tissues. Performance of tasks like these may improve as volume increases and may require a certain level of volume to achieve or maintain proficiency. The acquisition and maintenance of certain cognitive skills may also be important to producing good outcomes and related to volume. Recognizing uncommon presentations of medical conditions or anatomic variations in surgery and taking appropriate and timely action are examples. To the extent the physical and cognitive skills most crucial to outcomes can be characterized and specified, they may move from the realm of “unmeasured processes” to assume their proper role in the set of crucial and measurable processes that are integral to producing the best outcomes.

Third, is the experience of other key participants important? Is the volume of experience in a hospital's emergency department or coronary care unit important to its outcomes for patients with acute myocardial infarction? What about the experience of key consultants? Is the amount of experience of the team managing the cardiopulmonary bypass pump important in contributing to the outcomes of coronary artery bypass surgery? Fourth, what about “organizational skill?” Do high-volume hospitals adopt specific organizational strategies that are particularly effective in enhancing outcomes? Do they employ sophisticated computerized reminder systems or standardized protocols that reduce errors of omission or commission? Do teaching hospitals, which are often high in volume, have an advantage because house staff and fellows provide 24-hour physician coverage? The more complex the treatment process, the more likely it is that physician or surgeon skill will be only one of many important components of the full complement of effective care. Defining the relative contributions of all important contributors is an essential research challenge.

Criteria for Rating the Quality of Published Studies

Using this conceptual model as a guide, we developed a scoring system to assess the quality of the research studies included in our systematic review. The full list of criteria is attached as Appendix A. Our aim was to create a quantitative method of assessing the research design of the studies we reviewed such that higher scores would reflect increasing likelihood of the study's ability to discern generalizable conclusions about the nature and magnitude of the relationship between volume and outcome. The first four criteria assess various aspects of the patient sample used in the research. We assigned one point if the sample was representative of the general population of all patients who might receive the treatments under study. Thus, studies of managed care plan enrollees or Medicare beneficiaries were not considered representative. We assigned two points if the study included patients of 50 or more physicians and 20 or more hospitals. If only one of these criteria was met, we assigned one point. No points were assigned if neither criterion was met. In some studies authors reported the number of hospitals in their sample but not the number of treating physicians. In these cases we estimated the number of physicians by assuming it would be at least equal to the number of hospitals. The vast majority of these studies included hundreds of hospitals from administrative databases, so we estimated the number of physicians as 50 for scoring this criterion. If the total sample size was 1000 patients or more, we assigned one point. Because statistical power to detect significant relationships in logistic regression models depends more on the total number of adverse events represented in the sample than on total sample size (and because the various conditions and procedures in this lit

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

erature have widely varying adverse event rates), we assigned 2 points if the total number of adverse events was greater than 100, one point if it was 21–100, and no points if it was 20 or less.

We assigned no points if the study assessed the relationship between outcome and either hospital or physician volume. If both were assessed separately, we assigned one point. If the joint relationships of hospital and physician volume were assessed independently in a multivariate analysis, we assigned 2 points. And if a study examined both of these and the volume of another important component of the care process, we assigned 3 points. If the appropriateness of patient selection was not addressed, we assigned no points. If appropriateness was measured, we assigned one point. If it was measured and taken into account in the analysis of the volume/outcome relationship, we assigned 2 points.

If volume was analyzed in only 2 categories, we assigned no points. If more than 2 categories were assessed or if volume was treated as a continuous variable, we assigned one point to credit a more sophisticated assessment of a possible dose-response relationship. In considering the various ways in which outcomes might be risk-adjusted, we assigned no points if no risk-adjustment at all was done. If data from insurance claims, hospital discharge abstract databases, or other sources of administrative data were used, we assigned one point. If data from clinical sources (e.g., medical records or prospectively designed clinical registries) were used for risk-adjustment, we assigned 2 points. If clinical data were used in a logistic regression model that demonstrated good calibration by the Hosmer-Lemeshow test and good discrimination (by a C-statistic of 0.75 or greater), we assigned 3 points.

If specific clinical processes of care were not measured, we assigned no points. If a single process was measured and its impact on risk-adjusted outcomes assessed, we assigned one point. If two or more such processes were measured and evaluated, we assigned 2 points. Finally, if death was the only outcome evaluated, we assigned no points. If other adverse outcomes in addition to mortality were assessed, we assigned 2 points.

Quality scores were summed across all 10 criteria for each study. The maximum possible total score was 18.

Literature Review Methods

We performed two electronic subject-based searches of the literature on MEDLINE (1966– 1999). A professional reference librarian assisted us in the development of our search strategy. First, we selected twelve conditions and procedures for which volume and outcomes had been studied (coronary artery bypass, carotid endarterectomy, peripheral vascular surgery, trauma, transplant, hip surgery, knee surgery, liver resection, lung resection, cancer surgery, angioplasty, and myocardial infarction).

We then developed a list of search terms based on subject headings from articles known to be highly relevant to our topic and from the official indexing terms of the MEDLINE database. We performed multiple searches with combinations of these terms and evaluated the results of those searches for sensitivity and specificity, with respect to our topic of volume and outcomes. The search algorithm that yielded the greatest number of highly relevant articles combined the 12 conditions with the terms volume, utilization, frequency, statistics, and outcomes. In order to broaden our search to include articles on regionalization of care, we added another search that combined the 12 conditions with the term regionalization. (See Appendix B.)

We also performed MEDLINE searches on authors known to have published widely on the study topic, and we searched the Cochrane Collaboration Database for systematic reviews. In addition to performing electronic database searches, we consulted experts in the field for further

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

references. Finally, we reviewed the references cited by each article that was ultimately included. We did not hand-search any journals. This review was limited to the English-language research literature.

This monograph includes the findings of our review of eight procedures and conditions: coronary artery bypass graft (CABG) surgery, pediatric cardiac surgery, carotid endarterectomy, abdominal aortic aneurysm repair, cancer surgery, coronary angioplasty, acute myocardial infarction, and AIDS. Our electronic search also identified articles about volume and outcome in trauma, newborn intensive care, organ transplantation, hip surgery, knee replacement, neurosurgery, prostatectomy, cataract surgery, endocrine surgery, and general surgical procedures for benign disease, but we did not include them in this review. In the areas of trauma, newborn intensive care, and transplantation, the provision of services has already been regionalized or is otherwise highly regulated by government or other administrative bodies. Other areas were generally limited to very few articles per condition.

We developed an initial set of inclusion criteria and tested it on studies of three conditions: cardiac surgery, carotid endarterectomy, and AIDS. After revision, the final inclusion criteria were:

Time: patient cohorts treated from 1980 forward.

Sample: community or population-based sample—case series or convenience samples were excluded.

Multiple publications from the same database excluded; only the most recent or most complete publication was included

Health outcome(s) must be assessed as the dependent variable(s).

Volume must be an independent variable.

We limited the review to studies of patients treated from 1980 to the present, because of the rapidity of changes in hospital care, available treatments, and surgical techniques. In our view, data from patient cohorts prior to 1980 would have questionable relevance to today's policy issues. In a few instances, we included studies if part of their patient sample included patients treated in 1978 or 1979, but most of the sample comprised patients from the 1980s. We excluded studies from single institutions, from voluntary registries, or other convenience samples because of the weak generalizability of such studies. We excluded a few studies in which the only dependent variable was a composite of deaths or long lengths of stay, because, formulated in this way, the dependent variable was not purely a health outcome. We also excluded a few studies in which the only dependent variable was a composite of death or complications, with the latter determined solely by secondary diagnosis codes in administrative databases. These studies were excluded because of the notorious unreliability of using such data to identify complications. In general, we excluded multiple publications from the same set of data, selecting only the most recent or complete, unless different publications reported substantially different analyses (e.g., one reported the relationship of hospital volume to outcome and another analyzed physician volume and outcome).

Three reviewers assessed the articles for inclusion or exclusion, with at least two reviewers independently examining each article and applying the criteria. Discrepancies in the application of the criteria were resolved by discussion between the reviewers. We developed an initial set of assessment criteria and tested it on the studies of CABG surgery, carotid endarterectomy, and AIDS which had passed the inclusion criteria. Our final criteria for quality assessment and the

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

scoring system were described earlier and are listed in Appendix A. The same pair of reviewers who assessed each article for inclusion or exclusion then independently evaluated each article and assigned quality scores. Discrepancies were resolved by discussion between the 2 reviewers.

Overall Results of Literature Review

Identification of Studies

The MEDLINE searches of subject headings resulted in 740 citations of all types of articles, including reviews, editorials, and letters. The MEDLINE searches of authors known to have published on the topic yielded 160 citations (including overlap with previous searches). Consultation with experts in the field yielded 136 citations (including overlap). The Cochrane Collaboration Database contained no relevant current or pending reviews.

We reviewed the titles and abstracts of the 850 citations. We were able to exclude 600 based on the title or the abstract. Four of these were because of a language other than English. 250 citations remained for possible inclusion into the study. Of these, 128 were about one of the eight chosen conditions. We found ten additional citations by examining the references of all included articles. We were unable to obtain the articles for one citation (Choti 1998). Thus, we reviewed a total of 137 articles for possible inclusion into our systematic review. We counted an article that studied more than one condition as more than one study. For example, we counted the article by Hannan published in 1999 as two studies because it examined CABG and AAA. The total number of studies was 162.

Exclusion of Studies

Of the 162 studies, 74 were excluded, and 88 were included. The most common reason for exclusion was because volume was not an independent variable (31 studies, 19%). The second most common reason for exclusion was a study sample that was not community or populationbased (22 studies, 14%). This was particularly common among the studies of coronary angioplasty and cancer. These were primarily early studies of multiple surgical procedures that analyzed data from the 1970s. A few studies were excluded because the study outcomes were not health outcomes.

Overall Results of Systematic Review

Overall, 88 studies were included in our systematic review. Two authors scored each study on the 10 item methodological quality scale described previously. Possible total quality scores range from 0 to 18. The actual distribution of quality scores ranged from 2 to 13 (See Figure 2). The mean total quality score was 7.8 ± 1.9. The median score was 8 with an interquartile range of 6 to 9.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

* We counted an article that studied more than one procedure as separate studies for each.

** The articles that studied more than one procedure that were included are reflected in the numbers for each procedure.

Since we excluded articles that analyzed data collected prior to 1980, the publication dates of those included in our review ranged from 1984 to 2000 (Our last MEDLINE search was March 2000). We found evidence of a dramatic increase in interest in this area as nearly half of all the studies we identified were published since 1998.

The methodological characteristics of the 88 articles we reviewed are as follows. Approximately, two-thirds (68%) of the studies were judged to be truly representative of the underlying population of interest (i.e, all patients undergoing the specified procedure). The remaining one-third of authors focused exclusively on the care of a certain category of patients by insurance status such as Medicare or Veterans Affairs patients. Nearly all of the studies were based on administrative data sources, so it was no surprise that 78% of them had sample sizes of 1000 cases or more—many had tens of thousands of cases. Approximately 80% of studies were based on samples that included 20 or more hospitals and 50 or more treating physicians. We used this measure as a rough indication that there was the potential for substantial variation at the two provider levels of interest. However, 9% of authors reported on samples with fewer than 20 hospitals and 50 physicians. Twelve percent of studies had an intermediate number of providers.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

While we had no easy way of measuring the statistical power of each study, we did abstract the absolute number of primary outcomes (e.g., inpatient deaths) as a proxy indicator of the potential power of the study to detect modest associations or perform any significant multivariate modeling. It is worth noting that event rates are a function of both the overall sample size and the absolute rate of the adverse outcome of interest. Three-fourths of studies reported more than 100 adverse events upon which the volume-outcomes analyses were based. The rest of the studies had 20 to 100 events, except for one which had fewer than 20 (Wade 1996).

The primary outcome was mortality in 78% of the studies. This was most commonly inpatient death, though many reported 30 day death rates, and a few reported on 1, 3 or 5 year survival. The nineteen studies (22%) that examined more than just mortality were primarily studies of carotid endarterectomy that considered death or stroke as primary outcomes and investigations of coronary angioplasty that focused on death or emergency coronary artery bypass grafting together as major adverse events.

The vast majority of investigators examined provider volume as a categorical variable. Twelve percent dichotomized volume into low and high without any more detailed look for doseresponse relationships. The remaining 88% reported at least three categories of volume, usually textiles or quartiles, or numerous categories broken into even-numbered ranges. A few authors analyzed volume as a continuous variable.

Among the 88 studies we reviewed, there were 78 attempts to detect hospital level volume-outcome effects and 35 comparisons of physician level influences. The predominance of analyses of hospital volume effects is largely due to the fact that most of the studies were based on state or national hospital discharge databases that often lacked unique physician identifiers.

Most studies (72%) were primarily focused on examining either the effects of hospital or physician volume on outcomes. Another 10% of studies reported both hospital and physician levels effects, but separately. Sixteen studies (18%) performed some type of multivariate or stratified analysis designed to try to tease apart the independent or synergistic contributions of physician and hospital volume to outcomes.

We paid particularly close attention to the level of sophistication of the risk adjustment techniques used by investigators (if any), because failure to account for imbalances in patient case mix among providers can seriously confound the relationship between volume and outcome. Fifteen percent of studies performed no risk adjustment at all. The majority of studies (58%) relied exclusively on risk adjustment models based on administrative data, most commonly sociodemographics and ICD-9 discharge diagnoses. The limitations of these risk adjustment techniques are beyond the scope of our discussion but have been reviewed elsewhere (Iezzoni 1997). Only 24 studies (27%) used any clinical data in their risk adjustment models. Among this group, only six studies reported risk adjustment models that were robustly discriminating (model C statistic ≥ 0.75) and calibrated (by Hosmer-Lemeshow or other testing).

There were two other higher level methodological issues that few studies seemed to address. Only 2 of 88 studies (2%) measured the appropriateness of patient selection—both were studies of carotid endarterectomy where this important issue has been recognized for a decade (Brook 1990a, Karp 1998). Similarly, only 4 of 88 authors (5%) made any attempt to adjust for the use of any key processes of care. Two studies of breast cancer adjusted for use of adjuvant therapy or

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

type of surgery (Sainsbury 1995, Roohan 1998). Thiemann and colleagues ' (1999) study of myocardial infarction adjusted for the use of key medications (e.g., aspirin and beta-blockers) and revascularization. Malenka et al (1999) adjusted for use of heparin or intra-aortic balloon pump in coronary angioplasty cases.

Finally, while we found marked heterogeneity in the methodologic quality of studies for each publication year we reviewed, we were interested in more formally assessing whether the overall quality of published research was improving over time. The results of this preliminary analysis are displayed in Figure 3. We did find a significant, though modest, increase in quality scores over time (R = 0.26, p = .014). Studies with more recent publication dates received higher scores than earlier ones, suggesting that the level of analytic sophistication in this field is improving.

Volume and Outcome in Coronary Artery Bypass Graft (CABG) Surgery:

Nineteen articles treating various aspects of the subject of volume and outcome in CABG surgery were identified by our various search strategies. Ten failed to meet our inclusion criteria. Two reported data from hospitals that were not community or population based samples. (Zelen 1991, Clark 1996) One was a review article and presented no primary data. (Snowden 1995) Three dealt with various issues concerning regionalization of CABG surgical services but 2 presented no data on outcomes (Anderson 1989, Rohrer 1997), and one had no data on the relationship between volume and outcome. (Roos 1989) Two were excluded because either mortality by hospital was not evaluated (Banta 1990), or because the relationship of neither hospital nor physician-specific volume was examined with respect to outcomes. (Tu 1996). One was excluded because health outcome was not the dependent variable; that study used a composite variable of inpatient death plus long length of stay. (Hughes 1987). And one study was excluded because, although risk-adjusted mortality data and hospital volume data were presented together, their association was not explored. (Hannan 1994)

Figure 3: Quality of the Volume/Outcome Literature over time (larger circles represent larger numbers of studies)

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Of the 9 articles meeting our inclusion criteria, overall quality scores ranged from 7 to 11, with a median of 8. These studies and their main characteristics are listed in Appendix C. Two studies received a score of 7, four obtained scores of 8, and one each received scores of 9, 10, and 11. The three highest scoring studies achieved these ratings either by examining the joint effects of hospital and surgeon volume (n = 2) or by using clinical data to risk-adjust (n = 2), or both (n = 1). No studies examined any outcomes other than death, and none assessed appropriateness of patient

Figure 3: Quality of the Volume/Outcome Literature over time (larger circles represent larger numbers of studies) selection or clinical processes of care. All 9 studies employed risk-adjustment methods in analyzing outcomes; 7 used administrative data and 2 used clinical data. All 9 studies had large sample sizes (n > 5,000); 7 studies examined data from more than 10,000 cases, and 4 had sample sizes greater than 50,000.

Seven of the 9 studies demonstrated a statistically significant association between mortality and surgeon or hospital volume, or both. The earliest study was limited by its use of Medicare data to assess the relationship of death (within 60 days of surgery) to hospital volume of CABG operations on Medicare patients in 1979–1980. (Riley 1985) Because older people received this operation much less often 20 years ago, Medicare hospital volumes were very low by today's standards; 76% of cases were performed in hospitals with volumes less than 100. Showstack and colleagues used administrative data from California hospitals in 1983 to demonstrate that risk-adjusted inpatient death rates were higher at hospitals performing 100 CABG operations or fewer (5.2%) compared with those performing at least 350 such surgeries (3.1%). Grumbach and colleagues demonstrated similar findings in their study of CABG surgery in California, New York, and Canada (Ontario, Manitoba, and British Columbia). (Grumbach 1995)

Hannan and colleagues have published the only investigations to examine the impact of both surgeon and hospital volume on mortality, finding significant inverse relationships for both. They first examined administrative data from New York State in 1986 and found that high-volume surgeons (>116 cases) had lower risk-adjusted inpatient mortality across all strata of hospital volume than lower-volume surgeons. High-volume surgeons at high-volume hospitals (>650 cases) experienced a risk-adjusted mortality of 3.2% compared with low-volume surgeons at low-volume hospitals (<224 cases), who demonstrated a risk-adjusted mortality of 5.7%. (Hannan 1989) In the most detailed and rigorous assessment of these relationships, these authors used prospectively collected clinical data to risk-adjust 1989 inpatient mortality in New York hospitals. (Hannan 1991) They again found statistically and clinically significant associations between mortality and both surgeon and hospital volume. Table 2 summarizes these findings.

Two studies employed several years' of data to examine changes in the volume outcome relationship over time in CABG surgery. Farley used administrative data from 62 hospitals participating in the Hospital Cost and Utilization Project from 1980 to 1987. They examined the relationship between inpatient mortality and hospital volume both cross-sectionally and longitudinally. (Farley 1992) They found that volume at low-mortality hospitals increased over time and interpreted the findings as evidence for the “selective-referral hypothesis.”

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

TABLE 2 Risk-Adjusted Inpatient Mortality (%) Following CABG Surgery: New York State in 1989

Surgeon Volume

Hospital Volume

<55

55–89

90–259

260

Total

<200

14.07

6.67

6.42

3.33

7.25

200–889

9.01

5.72

3.97

2.88

4.32

890

5.33

3.39

3.11

2.18

2.85

Total

8.14

5.56

3.61

2.43

3.68

Hannan and colleagues used clinical data to examine changes in surgeon volume and their relationships to risk-adjusted inpatient mortality in New York from 1989 to 1992. (Hannan 1995) They demonstrated that risk-adjusted mortality for low-volume surgeons (50 cases per year) fell 60%—from 7.94% to 3.20%—during the study period. This decline was greater than that observed for high-volume surgeons (34%) and for all cases (43%). In a closer analysis of their data, however, they depicted an important dimension of complexity not previously described. In each of the 4 years studied, the composition of the low-volume surgeon group changed markedly. Although the number of low-volume surgeons was similar in each of the 4 years (between 33 and 39), only 10 surgeons remained in the low-volume group in all 4 years. In each year, some new surgeons entered practice, some became low-volume surgeons transiently, and some stopped doing CABG surgery altogether. The 10 persistently low-volume surgeons had the highest risk-adjusted mortality rate in the final study year of any identified subgroup. Their mortality rate in 1992 was 8.96%, compared to all low-volume surgeons in that year (3.2%), low-volume surgeons who were new to New York that year (2.07%), low-volume surgeons who had previously had higher volumes (1.82%), or the statewide average (2.45%).

Two studies found no association between volume and outcome. Shroyer and colleagues examined data from 44 Veterans Affairs Hospitals performing CABG surgery between 1987 and 1992 and found no relationship between hospital volume and death within 30 days of surgery. (Shroyer 1996) The range of volume among the hospitals studied was limited. No hospital performed more than 400 CABG operations, and only 3 performed between 300 and 400. The lack of hospitals in high or very high volume strata (as characterized in other research) precluded the ability of this study to detect an association between outcome and a more complete range of volume. In contrast, Sollano et al examined the experience in New York State between 1990 and 1995 and found no relationship between inpatient death and hospital volume. (Sollano 1999) In this study, too, the range of volume was limited, but in an opposite direction compared with the Shroyer study. Only 1 of 31 hospitals averaged fewer than 100 CABG operations during the study period, 1 averaged between 100 and 199 procedures, and 4 averaged between 200 and 299 operations. In this sample, 18 hospitals (58%) averaged more than 400 cases per year, a volume not reached by any of the hospitals in the Shroyer study. It seems likely that the Sollano investigation lacked sufficient hospital representation in the low-volume range to adequately assess the association between volume and outcome in CABG surgery.

Volume and Outcome in Pediatric Cardiac Surgery

Three studies met our inclusion criteria for this review. No studies were excluded. Appendix D summarizes the characteristics, findings, and quality scores of the 3 studies. Quality scores

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

were modest (6, 8, and 9). None of the studies examined appropriateness or processes of care or studied an outcome other than inpatient mortality. None of the studies included large numbers of hospitals, because pediatric cardiac surgery is concentrated in a very small subset of hospitals, much smaller than the number performing adult cardiac surgery. All 3 studies took a similar analytic approach to examining how mortality was related to volume of surgery for congenital heart disease in children. In contrast to many adult surgical procedures, the frequency of occurrence of specific pediatric cardiac surgical procedures (e.g., repair of a ventricular septal defect or correction of tetralogy of Fallot) is too low to support comparisons of volume across different hospitals. All 3 of the research teams assessing the relationship between volume and outcome in this area divided into 4 complexity categories all of the surgical procedures performed to correct congenital heart disease in children, aiming to construct categories with relatively homogeneous mortality risk. The volume variable in these analyses is then the total number of all such procedures, with the complexity category entering a multivariate model as an independent risk-adjustment variable.

Jenkins and colleagues used administrative data from California (for 1988) and Massachusetts (for 1989) and were the first to create complexity categories. (Jenkins 1995) They constructed their categories using clinical judgment to sort ICD9 procedure codes into 4 groups. When they applied the resulting classification scheme to their data set, they did demonstrate increasing average mortality rates across the 4 categories: 4.8%, 6.4%, 10.7%, and 15.9%. In their multivariate model, adjusting for demographic characteristics and complexity, volume was strongly related to inpatient mortality. Risk-adjusted inpatient mortality was 18.5% for hospitals at which fewer than 10 cases were performed, 7.9% for those in the range of 10–100, 8.2% for those performing between 101 and 300 cases, and 3.0% for hospitals at which more than 300 of these operations were done. Although the 95% confidence intervals for these estimates over-lapped considerably for the lower 3 volume categories, the confidence interval for the estimate of the highest volume category did not overlap with any of the others. The authors did not present data on model calibration or discrimination, but used generalized estimating equation procedures to control for intra-hospital correlations.

Sollano and colleagues used the complexity categories created by Jenkins et al. in their analysis of administrative data from New York State (1990–1995). (Sollano 1999) They used standard logistic regression to assess the relationship between hospital volume and inpatient mortality, adjusting for procedure complexity, and found a significant odds ratio of 0.944 for hospital volume (in hundreds of cases), suggesting a decrease of 5.6% in the odds of dying for every additional 100 cases in volume. The average overall mortality was 7.3%. The model's C statistic was 0.617; calibration was good. Further analysis by age group showed that the relationship of volume to outcome was especially strong for neonates, age 30 days, (OR = 0.64) and for infants, age 31 days–1 year, (OR = 0.72) but not statistically significant for children over 1 year of age.

Hannan et al. took a different analytic approach to constructing complexity categories and had access to prospectively collected clinical data to evaluate volume and inpatient mortality in New York State from 1992 to 1995. (Hannan 1998a) These researchers ordered the cardiac surgical procedures by their actual mortality rates in their data set. Expert clinicians then grouped the procedures into clinically sensible subgroups and repeated the process until 4 categories were identified. This procedure resulted in a classification system with more widely separated average mortality rates than that created by Jenkins et al. Death rates by category were 1.4%, 4.5%, 11.0%, and 20.1%. These categories were employed as independent variables in a multivariate logistic regression, which also controlled for age and comorbidities (as identified in the clinical database). The resulting model demonstrated excellent calibration and discrimination (C = 0.818). The researchers used the model to explore the relationship between inhospital death and volume for both hospitals and surgeons. Table 3 shows the findings. Both the hospital and surgeon effects were statistically significant. The authors chose the volume categories that demonstrated the greatest differentiation between low and high volume providers. In general, they noted that higher volumes were associated with lower risk-adjusted mortality rates across all categories of procedure volumes.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

In each of the 4 complexity categories, both high-volume hospitals and high-volume surgeons demonstrated lower risk-adjusted mortality than their lower-volume counterparts.

Volume and Outcome in Carotid Endarterectomy

We were able to identify 19 studies of carotid endarterectomy that met our inclusion criteria (See Appendix E). Six additional studies initially identified by our search algorithm were excluded (2 because there was no explicit volume/outcome analyses (Tu 1998, Hsia 1992) and 4 because they were not population-based (Hertzer 1984, Rubin 1998, Friedman 1988, Slavish 1989).

In general, there are two types of studies in the literature. The most common type is based on a state or national hospital discharge database and reports on ten of thousands of carotid endarterectomies, but has limited clinical information. These studies have a large number of cases, but tend to have very limited if any risk-adjustment, and infrequently have both surgeon and hospital information. They are also limited to mortality as the only believable outcome, since post-operative stroke can not be reliably ascertained by administrative data. This is because stroke can be a pre-operative diagnosis as well as a post-operative complication, and ICD-9 discharge codes do not distinguish between the two. The other type of study is much smaller (usually 500 to 1500 patients), but has more detailed clinical information from medical chart review that facilitates measurement of stroke and death rates (usually over 30 days) and allows more clinically grounded risk adjustment models.

There was considerable heterogeneity in the overall quality scores ranging from 3 to 10 (out of a possible 18 point scale). The median score was 6 (inter-quartertile range 5–8). Only 9 of the 19 studies performed any type of risk adjustment. Among studies that did attempt to adjust for patient risk factors, five relied on administrative data, and four had clinical data. All four that collected clinical data received a quality score of 8 or greater. Only two studies (Hannan 1998b,

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Khuri 1999) presented moderately robust risk-adjustment models with C statistics of 0.715 and 0.72, respectively. Only two studies attempted to adjust the analyses for appropriateness of the procedure, and none adjusted for processes of care.

The unit of analysis was also quite variable. Four studies looked solely at surgeon volume, six examined only hospital volume, and nine examined both (3 of which examined them together). Nearly all studies reflect carotid endarterectomy practice from the 1980s and mid-1990s. Half (10) of the studies considered inpatient mortality to be the primary outcome. The other half (9) focused on 30 day death rates, some with combined death, stroke, and myocardial infarction rates. Because of the well-documented difficulties of using administrative data to differentiate pre-operative versus post-operative strokes, we only considered the 4 studies that had medical chart review data to have valid measures of perioperative stroke or myocardial infarction.

While a few studies considered procedure volume as a continuous variable, most investigators divided annualized caseload rates into two to five categories and compared the outcomes of the lowest versus the highest-volume providers. There was ten-fold variation in the definition of low and high-volume providers among the studies. The definition of low volume surgeon ranged from ≤1 per year to <30 per year; and the definition of low-volume hospital ranged from ≤6 per year to ≤100 per year. This heterogeneity makes formal meta-analysis virtually impossible.

Of the 13 studies that analyzed surgeon volume, nine found some positive relationship between surgeon volume and outcomes. Of the four highest rated studies (scores of ≥9), three found that the low volume surgeons (<5 per year to <10 per year had significantly worse risk-adjusted outcomes compared with highest-volume surgeons. The one higher-quality study by Cebul and colleagues (1998) that found no effect of physician caseload had a relatively small sample size (n = 678) and absolute number of adverse events (n = 32). The authors themselves calculated that they had less than 10% statistical power to find a significant relationship between observed differences in high compared with low-volume physicians. However, their study did have the statistical power to find a significant relationship between hospital volume and outcome.

Of the 15 studies that analyzed hospital level effects, seven found a significant relationship (and one a borderline trend p = .08) with clinical outcomes and institutional volume. Of the three higher rated studies (scores of ≥ 9) that measured volume effects, one (Katonen 1998) conducted in Finland found no hospital effect, and the two in the U.S. (Cebul 1998 and Hannan 1998b) did find significantly worse outcomes for hospitals with ≤ 62 and ≤ 100 carotid endareterectomies per year.

Among the eight studies that examined both surgeon and hospital volume, only two performed any risk adjustment. Cebul and colleagues (1998) found that only hospital volume was significantly associated with risk-adjusted 30 day stroke and death rates, not surgeon volume. Hannan et al. (1998b) which included data on 28,207 operations over a six year period in New York found both hospital and surgeon volume to be independently associated with risk-adjusted inpatient mortality rates, with the surgeon effect being larger than the hospital effect. Hannan et al. also found a significant interaction between hospital and surgeon volume as shown in Table 4. Low volume surgeons operating at low volume hospitals had the lowest risk-adjusted mortality rates (1.96%), low volume surgeons at high volume hospitals and high volume surgeons at low volume hospitals had identical, intermediate mortality rates (1.18%), and high volume surgeons operating at high volume hospitals had the lowest death rates (0.94%). It is also worth noting that in New York State, there were very few operations performed by low volume surgeons in high volume hospitals, so estimates of mortality rates in this subgroup had very wide 95% confidence intervals.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Volume and Outcome in Abdominal Aortic Aneurysm Surgery

Sixteen studies on abdominal aortic aneurysm (AAA) repair were reviewed. Four were excluded. Two were excluded because volume was not an independent variable (Evans 1999, Adam 1999). One study (Veith 1991) was excluded because it was based on the same data as another study (Hannan 1992). Another study was excluded because its data was from the 1970s (Pilcher 1980). A summary of the articles on AAA is provided in Appendix F.

Twelve studies met the inclusion criteria. Overall quality scores ranged from 5 to 10, with a median of 8. These moderate scores reflect the frequent use of administrative databases for risk adjustment and the exclusive use of mortality as the outcome. The unit of analysis was the hospital in seven studies, the hospital and the physician (analyzed separately) in three studies (Pronovost 1999, Dardik 1998, Rutledge 1996), and the hospital and the physician analyzed together in two studies (Hannan 1989, Hannan 1992). No study measured the appropriateness of patient selection.

The definition of a low-volume hospital was heterogeneous, making comparisons among studies difficult. It ranged from less than four to less than 50 unruptured AAAs per year. The definition of a low-volume surgeon was more homogeneous, ranging from less than three to less than five unruptured AAAs per year.

A somewhat unique characteristic of many of the AAA studies was an attempt to measure the volume of something other than the index condition, and to determine the association of this volume with the outcomes of that condition (i.e., the issue of “volume of what”). These studies asked whether experience with vascular surgical procedures other than the one under study might be associated with the outcomes of the studied procedure. Three studies examined the relationship between unruptured AAA volume and ruptured AAA outcomes (Dardik 1998, Rutledge 1996, Amundsen 1990). One study looked at total AAA volume and outcomes for ruptured AAA and unruptured AAA (Manheim 1998). Three studies looked at the relationship between the volume of operations other than AAAs and AAA outcomes (Khuri 1999, Amundsen 1990, Hannan

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

1992). Khuri and Amundsen measured the volume of all peripheral vascular operations, and Hannan looked at the volume of any aorta operation —i.e. not just aortic aneurysm repairs.

All but the Amundsen study performed some type of risk adjustment, and the Khuri study of Veterans Affairs hospitals was the only one to use clinical data for risk adjustment. The Khuri study was the only study in this entire review to perform hierarchical linear modeling. Its C statistic for AAA was 0.75.

No study measured clinical processes of care. Pronovost et al (1999) came the closest, measuring the effect of intensive care unit characteristics on outcomes. This included physician staffing, (e.g. daily rounds by an ICU physician), nurse staffing (e.g. nurse-patient ratios), and some in practice policies (e.g. use of a critical pathway, morbidity and mortality reviews, routine extubation in the operating room). All of these variables, except for routine extubation in the operating room, are structural characteristics or general policies and not actual clinical processes of care measured at the patient level. Routine extubation in the operating room was analyzed only in terms of its effect on length of stay, and not on any clinical outcome.

Every study looked at only one outcome – mortality. Khuri measured 30-day mortality, and all other studies measured inpatient death. Complications of AAA repair such as bleeding, renal failure, lower extremity ischemia, ischemic colitis, myocardial infarction, and respiratory failure were not measured.

All but one study (Khuri 1999) found a significant relationship between volume and outcomes. Six of the seven studies that looked exclusively at hospital volume found higher mortality among low-volume hospitals. In a study of unruptured AAAs in New York State, Sollano et al found that an increment of 100 cases per year was associated with a 22% decrease in the odds of death (Sollano 1999). Similarly, in a study of Ontario hospitals, Wen et al found that an increase in hospital volume of 10 unruptured cases per year was associated with a 6% decrease in the odds of death from unruptured AAA (Wen 1996). They found no such relationship for ruptured AAAs.

Both Dardik and Rutledge looked at outcomes for ruptured AAAs. Dardik et al found that hospital volume (unruptured or ruptured) was not related to outcome of ruptured AAAs, but that surgeon volume of ruptured cases was (Dardik 1998). Similarly, Rutledge et al found only a non-significant trend between hospital volume (unruptured or ruptured) and outcome, and a significant relationship for surgeon volume (Rutledge 1996).

Two studies received a quality score of 10 (Hannan 1989, Hannan 1992a). These studies had large numbers of patients, physicians, and hospitals, and they examined surgeon volume, hospital volume, as well as the surgeon and hospital volume together. Hannan et al evaluated all AAAs performed in New York State in 1986 (Hannan 1989). Hospital volume was related to outcome only when the surgeon was a high-volume surgeon (≥4 cases per year). When the surgeon was high-volume, high-volume hospitals (>36 cases per year) had a risk-adjusted mortality of 11%, and low-volume hospitals had a risk-adjusted mortality of 19%. Surgeon volume was related to outcome, regardless of the hospital volume. Overall, high-volume surgeons had a risk-adjusted mortality of 15%, and low-volume surgeons (<4 cases per year) had a risk-adjusted mortality of 20%. The effect of surgeon volume was particularly strong within high-volume hospitals. High-volume surgeons operating in high-volume hospitals had a risk-adjusted mortality rate of 11%, whereas low-volume surgeons in high-volume hospitals had a risk-adjusted mortality of 20%. The study also found that for high-volume surgeons, mortality continues to decrease as hospital volume increases over the measured range of hospital volumes.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

In another study of New York State, Hannan et al (1992a) examined the relationship of hospital and surgeon volume to outcomes of unruptured and ruptured AAA over a three year period (1985 to 1987). They found that for unruptured AAAs, hospital volume was related to outcome, but that surgeon volume was not. By contrast, for ruptured AAAs, surgeon volume, but not hospital volume, was related to outcome. They repeated the analysis using the volume of all aorta operations and found the same results.

In order to try to evaluate the “selective referral” hypothesis, they identified groups of hospitals and groups of physicians with the lowest and highest mortality rates in 1982–1984, and then followed their changes in volume during the period 1985–1987. Although all groups of surgeons experienced some increase in volume, low-mortality surgeons had the greatest increase in volume (523 operations to 751 operations, or 44% increase). To evaluate the “practice makes perfect ” hypothesis, they looked at the outcomes for those low-volume surgeons in 1982–1984 who became high-volume surgeons in 1985–1987. There were 22 low-volume surgeons who increased their volumes to 14 or more cases per year. The decrease in their risk-adjusted mortality rate was negligible (6.8% to 6.2%). Six of those 22 surgeons, however experienced an increase in volume to 22 or more cases per year, and the drop in their mortality rate was larger (5.8% to 2.5%), although still not statistically significant because of the small number of surgeons. The authors concluded that some evidence for selective referral as well as the practice makes perfect hypotheses exists.

The only negative study was Khuri and colleagues' (1999) evaluation of all AAAs performed in Veterans Affairs (VA) hospitals from 1991 to 1997. In this study, 107 hospitals performed 3,767 procedures. Thirty-day mortality was measured, and clinical data was used to build a fairly robust risk-adjustment model. Annual hospital volume ranged from 0 to 32, with a mean of 6.9. The study found no relationship between hospital volume and outcome for AAAs or for any of the seven other procedures it examined. The findings of this study may not be generalizable, because it included only VA hospitals and patients. The authors point out that many VA surgeons operate at other institutions as well, so the volume of the VA institutions may not in fact reflect the experience of their staffs.

Volume and Outcome in Cancer Surgery

We examined a total of 38 studies on cancer. All of the eight studies of medical treatment of cancer were excluded because none of them looked at volume as an independent variable. Of the 30 studies of surgical treatment, 10 were excluded. The most common reason for exclusion was a sample that was not community or population-based (7 studies). Two studies did not evaluate volume as an independent variable (Gordon 1998, Whittle 1998). One paper was a review article, not primary research (Steele 1996).

Thus, 20 papers, all about cancer surgery, were included in the systematic review. Three of these studies looked at more than one procedure (Hannan 2000, Gordon 1999, Begg 1998). To analyze these articles, we examined the data for each procedure separately. In total, 11 papers studied pancreatic resection, five studied colorectal resection, three studied esophagectomy, three studied lung resection, and two studied breast surgery (summarized in Appendix G). The three articles that looked at other cancer procedures are summarized in a table in Appendix G called“Cancer Miscellaneous.”

We did not include other papers that studied these operations for benign as well as malignant disease, with the exception of Gordon (1999). We included Gordon (1999) because it studied

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

pancreaticoduodenectomy and esophagectomy, both of which are rarely performed for benign disease.

Pancreatic Resection

Eleven studies evaluated pancreatic resection. The quality scores varied greatly, ranging from 3 to 10, with a median of 7. The study with the lowest quality score had a small sample that was not representative of the entire population and did not perform any risk adjustment (Wade 1996). The study with the highest quality score had a large, representative sample, and it examined physician volume, hospital volume, and the interaction between the two (Lieberman 1995).

The unit of analysis was the hospital for all studies, except for two that looked at both surgeon and hospital volume (Lieberman 1995, Sosa 1998a). No study evaluated appropriateness of patient selection. The definition of low hospital volume ranged from less than 1 to less than 9 procedures per year. Begg et al defined volume as the annual volume of procedures done on Medicare patients. Two studies of Maryland had only one high-volume hospital (Gordon 1995, Gordon 1999). In Lieberman and colleagues' study of New York State, two hospitals were high-volume, and four surgeons were high-volume. The two analyses of surgeon and hospital volume interaction were limited by the fact that most of the high-volume surgeons practiced only in high-volume hospitals.

No study effectively addressed the question of “volume of what.” Gordon et al studied the association between the total volume of 6 “complex gastrointestinal” procedures (total colectomy, esophagectomy, total gastrectomy, hepatic lobectomy, biliary tract anastomosis, and pancreaticoduodenectomy) and individual procedure mortality. They did not also study, however, the association between individual procedure volumes and mortality (Godron 1999). No study evaluated the appropriateness of patient selection.

Risk adjustment was based almost exclusively on administrative data. Only Begg et al used some clinical data (cancer staging from the Survival, Epidemiology, and End Results database). None of the studies examined clinical processes. Inpatient death was the primary outcome of interest. Three studies looked at death beyond the inpatient stay (Simunovic 1999, Birkmeyer 1999a, Birkmeyer 1999b), and one measured rates of complications, specifically infection and hemorrhage (Glasgow 1996). Other complications such as pancreatic or biliary leak, gastric dysmotility, pneumonia, and other outcomes such as recurrence and quality of life were not examined.

Of the nine studies that looked at hospital volume only, all but one (Wade 1996) found a significant relationship between volume and outcomes. The highest quality score of 8 was achieved by a study of 1705 pancreatectomies at 298 hospitals in California from 1990 to 1994 (Glasgow 1996). In this study, the risk-adjusted mortality at high-volume hospitals (>50 cases per year) was 3.5%, compared to 14% at low-volume hospitals (≤5 cases per year).

Lieberman et al (1995) analyzed both physician and hospital volumes; 1,972 procedures were performed by 748 surgeons in 184 hospitals in New York State from 1984 to 1991. In separate analyses of surgeon volume and hospital volume, high-volume surgeons (≥41 cases per year) had lower risk-adjusted mortality rates than low-volume surgeons (<9 cases per year)—6% versus 13%, and high-volume hospitals (>8 cases per year) had lower risk-adjusted mortality rates than low-volume hospitals (<10 cases per year)—5% vs. 19%. When surgeon volume and hospital volume were analyzed together, however, only hospital volume was significant.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Sosa et al (1998a) analyzed both physician and hospital volumes for 1,236 procedures by 373 surgeons at 48 hospitals in Maryland. They found that the relative risk of death at low-volume hospitals (< 5 cases per year) was 19 times that at high-volume hospitals (>20 cases per year). Analyzing physician and hospital volume together, they found hospital volume to be significant regardless of physician volume.

Although the studies on pancreatic resection had a great deal of methodological heterogeneity, they suggested that outcomes were related to provider volume and to hospital volume in particular. The magnitude of this volume effect was relatively large compared to most of the other procedures we studied. This is a function of both the high absolute mortality rate for pancreatic cancer as well as a very strong volume and outcome relationship. The number needed to be treated by a high volume provider to prevent one inpatient death attributable to low volume was only 10 to 15 for most higher-quality studies.

Esophagectomy

The three studies of esophagectomy had low quality scores (6, 6, and 8). The two lower-scoring studies had relatively small sample sizes – 518 patients in one (Gordon 1999) and 503 patients in another (Begg 1998). The unit of analysis was the hospital in all three studies. The definition of low volume was relatively similar across studies, ranging from less than 6 to less than 10 procedures per year. Begg et al measured volume of Medicare cases only. All studies performed some risk adjustment, and only one utilized clinical data (Begg 1998). No study evaluated clinical processes such as operative approach (abdominal versus thoracoabdominal) and method of reconstruction.

The only outcome evaluated was inpatient mortality. No study examined long-term survival, recurrence, or quality of life. Complications such as anastomotic leak, respiratory failure, pneumonia, and digestive dysfunction were not measured.

All three studies found large differences in mortality between low-volume and high-volume hospitals. Gordon and colleagues found that the relative risk of death at a low-volume hospital was 3.8 times that at a very-high-volume hospital, although there was only one institution in this latter category (Gordon 1999). Begg et al found that the risk-adjusted mortality at high-volume hospitals was 3.4%, compared to 17.3% at low-volume hospitals. Patti et al. (1998) found similar mortality rates—6% at high-volume hospitals and 17% at low-volume hospitals. This study had the highest quality score of 8, in part because of its large size. Overall, the magnitude of the volume and outcome relationship for esphagectomy was striking. The number needed to treat by a high volume provider to prevent one inpatient death attributable to low volume was seven to nine patients.

Breast Cancer Surgery

The two studies of breast cancer surgery had relatively high quality scores (10 and 11) because they had large numbers of patients, surgeons, hospitals, and adverse events, and because they utilized clinical data from cancer registries in their risk adjustment models. The unit of analysis was the hospital in one study (Roohan 1998) and the surgeon in the other (Sainsbury 1995). Neither study looked at the appropriateness of patient selection. Roohan et al defined “ very low” hospital volume as fewer than 10 cases per year. Sainsbury et al defined low surgeon volume as fewer than 30 cases per year. Sainsbury et al attempted to include extent of disease

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

and tumor grade in their risk-adjustment model, though this information was missing for 50% of patients.

The two studies were noteworthy for their measurement of clinical processes. Roohan et al included the type of operation (mastectomy or breast-conserving surgery) as an independent variable in the multivariate analysis. Sainsbury et al included the percentage of patients treated by mastectomy (versus local excision), chemotherapy, hormone therapy, radiation therapy, or surgery alone for each surgeon. These two studies were unique in that they both selected a long-term outcome (5-year survival) as their dependent variable. Neither study measured other outcomes such as recurrence, complications of surgery, or complications of adjuvant therapy.

Roohan et al looked at 47,890 cases of breast cancer surgery performed in 266 hospitals in New York State from 1984 to 1989. In a multivariate regression model, they found volume to be related to 5-year mortality, with a clear “dose-response” relationship. The increased risk of death was 19% in moderate-volume versus high-volume hospitals, 30% in low-volume versus high-volume hospitals, and 60% in very-low-volume versus high-volume hospitals. The authors conjectured that since breast surgery has negligible operative and inpatient mortality, the volume-outcome relationship might be caused by higher-volume hospitals providing more effective adjuvant treatment.

Sainsbury et al studied 12,861 cases of breast cancer surgery performed by 180 surgeons in the Yorkshire Regional Health Authority area from 1979 to 1988. Risk adjustment included age, extent of disease, tumor grade, socioeconomic status, date of treatment, and type of therapy (surgery, radiation, chemotherapy, hormone therapy, surgery alone). They found that the risk of death was significantly lower for patients of high-volume surgeons (greater than 29 cases per year) compared to low-volume surgeons (fewer than 10 cases per year). There was no difference in survival between moderate-volume (10 to 29 cases per year) and low-volume surgeons. The volume effect was slightly smaller after risk adjustment (risk ratio of 0.86 versus 0.82 before adjustment). Variation among surgeons in use of mastectomy, radiation, chemotherapy, hormone therapy, and surgery alone accounted for 8% of the variation in survival. Surgeon volume and use of chemotherapy accounted for 20 to 25% of the variation in survival.

Lung Resection

The quality scores of the three studies of lung resection were relatively high (8, 8, and 10). The numbers of patients, physicians, hospitals, and adverse events were all high. The unit of analysis was the hospital in two studies (Begg 1998, Romano 1992) and both hospital and physician in one study (Hannan 2000). No study evaluated the appropriateness of patient selection. The three studies looked at different types of lung resection—lobectomies (Hannan 2000), pneumonectomies (Begg 1998), and all resections (Romano 1992).

The definitions of low hospital volume were heterogeneous, ranging from less than 6 to less than 38 procedures per year. Risk adjustment was based on administrative data in two of the studies (Hannan 2000, Romano 1992) and clinical data in one (Begg 1998). No study looked at clinical processes of care. The outcome of interest was inpatient death in all three studies. Complications such as bronchopleural fistula, respiratory failure, and pneumonia were not measured. In addition, no study evaluated other outcomes such as long-term survival, recurrence, or quality of life.

In the study with the highest quality score of 10, Hannan et al (2000) looked at 6,954 lobectomies by 373 surgeons at 178 hospitals. The risk-adjusted mortality rate at low-volume hospitals

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

(≥37 cases per year) was 1.65% higher than at high-volume hospitals (≥169 cases per year). There was no difference between medium-volume and high-volume hospitals. The vast majority of hospitals were low-volume (133 hospitals). Only 4 hospitals were high-volume. No significant relationship between surgeon volume and outcome was found.

Begg and colleagues examined 1,375 pneumonectomies performed on Medicare patients at 313 hospitals in the U.S. They utilized clinical data for risk adjustment. No difference in outcomes existed between high-volume and low-volume hospitals. Romano and colleagues found 40% lower risk of death after pneumonectomy at high-volume hospitals compared to low-volume hospitals. They also found a similar volume-outcome relationship for lesser resections.

Colorectal Resection

The five studies of colorectal cancer resection had quality scores ranging from 7 to 10, with a median of 9. The studies were very heterogeneous. Three studies evaluated resections of all types of colorectal cancer (Hannan 2000, Harmon 1999, Parry 1999), one looked at total colectomy for benign and malignant disease (Gordon 1999), and one looked at resections for rectal cancer (Porter 1998). The unit of analysis was the hospital in 1 study (Gordon 1999), the physician in one study (Porter 1998), and both hospitals and physicians in 3 (Hannan 2000, Harmon 1999, Parry 1999).

The definition of low volume was variable, even among the three studies that looked at volume of all colorectal resections. Among these three studies, the definition of low surgeon volume ranged from less than 6 to less than 12 procedures per year. The definition of low hospital volume ranged from less than 40 to less than 84 per year. Gordon et al looked at the relationship between 6 complex gastrointestinal procedures including total colectomy and the outcomes of total colectomy.

All studies performed risk adjustment, and two studies (Porter 1998, Parry 1999) used clinical data. Two studies examined clinical processes, but neither incorporated the processes into their risk adjustment model. Parry et al measured whether or not an abdominoperineal resection was performed, use of ultrasound or CT scan, and operating “after hours.” Porter et al looked at the type of operation (low anterior resection versus abdominoperineal resection) and the use of adjuvant therapy. The outcome studied was primarily inpatient mortality. One study (Parry 1999) measured local recurrence rates as well as disease-specific survival. No study measured complications such as anastomotic leak, intra-abdominal abscess, wound infection, or genitourinary dysfunction.

Three of the four studies that assessed hospital volume did not find a significant relationship to outcomes. Harmon et al studied all resections in Maryland and found a trend toward lower mortality at high-volume hospitals, but this was not significant (odds ratio 0.78, p < 0.10). Parry et al studied all resections in the Northwestern United Kingdom and found no relationship between volume and outcomes. Gordon et al found no relationship between volume of complex gastrointestinal surgery and outcome of total colectomy. The only study to find a significant relationship for hospital volume found that the risk-adjusted mortality rate at low-volume hospitals was 1.9% higher than at high-volume hospitals (Hannan 2000).

Of the four studies that measured physician volume, three found a significant volume-outcome relationship. Only Parry et al found no relationship between physician volume and outcomes. Porter et al found that patients of low-volume surgeons had worse disease-specific survival than patients of high-volume surgeons (hazard ratio = 1.40) and a higher risk of local recur-

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

rence (hazard ratio = 1.80). High-volume surgeons were more likely to perform a low anterior resection as might be expected. They were no more likely, however, to use adjuvant therapy than low-volume surgeons were.

Three studies analyzed physician volume and hospital volume together. The physician effect found by Hannan et al disappeared when hospital volume was controlled for in the analysis. Harmon et al found that surgeon volume was related to volume regardless of hospital volume.

The studies of volume and outcome in colorectal surgery do not uniformly find a significant relationship. The magnitude of the volume effect on mortality is relatively modest—an absolute difference in inpatient mortality of 1% to 2% corresponding to a number needed to treat of 50– 100.

Summary

The twenty studies of cancer surgery suggest that a significant relationship between volume and outcomes does exist. The largest differences between low and high-volume providers were found for the most complicated operations in rare cancers—pancreatectomy and esophagectomy. For colorectal resection and lung resection, two operations for more common cancers, the relationship between volume and outcome is not as clear.

The common methodologic issues for these studies point to a need for more clinical data. Information about the type of tumor and cancer stage would be highly desirable, particularly in studies that look at long-term survival. An examination of the different clinical processes being employed and how they vary with provider volume, might elucidate the differences in outcomes. For example, the use of adjuvant therapies is particularly important but has not been well-studied with respect to volume. The roles of other providers besides the surgeon have also not been examined. Particularly when long-term survival is being evaluated, characteristics of other providers who care for the patient years after surgery, such as the medical oncologist and radiation oncologist would be relevant. More appropriate referral to these providers or better coordination of the many elements of cancer care, such as diagnostic testing, adjuvant therapy, and follow-up surveillance, may underly the hospital volume effects that have been found.

It is worth noting that the literature on volume and outcomes in cancer has disproportionately focused on rare operations for rare cancers. For the most common cancer operations – breast cancer surgery, colon resection, and lung resection, we found 10 studies that met our inclusion criteria. By contrast, the most rare operations – esophagectomy and pancreatectomy had 13 publications. In addition, we found no studies of medical treatment of cancers.

Volume and Outcome in Coronary Angioplasty

Nine studies on the relationship between provider angioplasty volume and outcomes met inclusion for our systematic review (See Appendix H.) Eleven other studies were identified by our search algorithm but excluded for the following reasons: two did not have volume and outcome data (Gray 1999, de Belder 1999), two were based on data from the early 1970s, and seven were not population-based samples —either small convience samples (Ellis 1997, Kastrati 1998, Klein 1997, Shook 1996) or nested within a randomized controlled trial (Gilchrist 1999) or trial registry (Grassman 1997, Kimmel 1995).

All of the studies were based on tens of thousands of patients. Seven of the nine were derived from large statewide or national hospital discharge databases. Overall quality scores ranged from 7 to 14, with a median score of 8 (IQR, 8–9). All of these articles included some attempt at

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

risk adjustment, though most of them (6) did this using administrative data from hospital discharge abstracts. The exceptions were Hannan (1997a), Malenka (1999) and McGrath (1998) which used databases that had more detailed clinical data that could be used for more rigorous risk adjustment. The study by Hannan and colleagues (1997a) using the New York State Coronary Angioplasty Reporting System reported the most robust risk adjustment model with a C statistic of 0.89. No studies adjusted for appropriateness, though Malenka (1999) adjusted for clinical indication. Also of note, only one study, Malenka et al. (1999) adjusted for important processes of care.

The unit of analysis among the the studies also varied. Five studies examined hospital volume alone, two looked at physician volume alone, and two analyzed both. All studies reflect practice from the late 1980s through the mid-1990s. None of the articles included data on angioplasty performed after 1996, so these data do not reflect the use of newer antiplatelet therapies as well as other more recent innovations in technique.

Compared with several of the other conditions which we reviewed, the literature on angioplasty was somewhat more homogenous. All studies were very large databases of tens of thousands of patients, included attempts at risk adjustment, and examined the outcomes of inpatient death and emergency (or same stay) coronary artery bypass grafting (CABG). Emergency CABG is an important outcome because it usually is performed when angioplasty is unable to open the target clogged coronary artery or when the procedure results in a dangerous complication such as coronary artery dissection. A few studies included other additional outcomes such as 30 day survival (Jollis 1994 and Jollis 1997) and radiographically defined successful angioplasty (patency; McGrath 1998, Malenka 1999).

Nearly all investigators compared outcomes of providers across a broad spectrum of volume categories, most commonly tertiles or deciles of caseload. The definition of low volume physician ranged from <25 per year to ≤85 per year. It is worth noting that the American College of Cardiology/American Heart Association guidelines recommend a minimum physician volume of 75 procedures per year. This is particularly significant in interpreting the finding of the two articles from the Northern New England Cardiovascular Disease Study Group (NNEDSG; Malenka 1999, McGrath 1998), which used as their lowest-volume physician volume thresholds < 84/yr or ≤ 85/yr which are often used to define “high-volume” providers in policy statements and other analyses. Relative to the rest of the country, the NNEDSG was really a study of high-volume, very high-volume and extremely-high-volume providers. This may explain why both investigators were unable to find any relationship between procedure volume and mortality. The definition of low-volume hospital ranged from <50 per year to <400 per year. The ACC/AHA guidelines recommend a minimum hospital caseload of 200 angioplasties per year.

Of the four studies that analyzed the effects of physician volume, only one found a significant association between procedure volume and mortality alone (Hannan 1997a), and one found a trend towards such a relationship (Malenka 1999, p = .086). Three studies found that lower angioplasty volume correlated with higher rates of emergency CABG. Emergency CABG rates average about 3% to 5% compared to mean inpatient mortality rates of about 1.0 to 1.5%, so studies that measured rates of both types of major angioplasty complications likely had greater statistical power to detect modest influence of volume on outcomes.

Of the seven articles that examined hospital volume, three found an inverse relationship between volume and mortality alone, though four did not. However, all seven found that lower volume was associated with higher rates of emergency CABG alone as well as the combined endpoint of inpatient death or emergency CABG. The two studies that examined both physician

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

and hospital volume influences (Hannan 1997a, Jollis 1997), both reported that physician and hospital procedure volume were independent predictors of outcomes in multivariate analyses. That is, adjusting for physician volume, hospital volume was still important, and vice versa. While neither investigator found any significant formal interactions between physician and hospital volume, both observed the lowest risk-adjusted event rates among the highest volume cardiologists performing angioplasty in the highest volume hospitals.

Three studies stand-out for their methodological sophistication. The highest rated study by Hannan and colleagues (1997a) examined all 62,670 procedures performed in New York in 1991 to 1994 using the New York State Department of Health Coronary Angioplasty Reporting System database. This dataset contains details on several important pre-procedural clinical variables identified by the New York State Cardiac Advisory Committee as necessary for rigorous risk adjustment including location, percent stenosis, and morphology of the diseased coronary vessels. This level of granularity of information may help explain how Hannan and colleagues were able to develop a robust risk adjustment model with excellent model fit and discrimination (C statistic = 0.89 and Hosmer-Lemeshow test not significant). They found that patients undergoing angioplasty by cardiologists with annual volumes < 75 per year had significantly higher risked adjusted rates of inpatient mortality and emergency CABG than the state as a whole (death: 1.03% vs. 0.90%; CABG: 3.93% vs. 3.43%). Hospital volume was also important. Hospitals that performed <400 angioplasties per year had higher risk adjusted rates of inpatient death and same stay CABG compared to the statewide average (death: 1.12% vs. 0.90%; CABG: 4.19% vs. 3.43%). They also found that both cardiologist and institutional caseload made independent contributions to explaining the variance in clinical outcomes. They found no formal interactions between physician and hospital volume.

Two additional points are worth comment. Hannan and colleagues (1997a) compared the outcomes of the lowest volume categories to the statewide average, not to the highest volume categories. For all of the comparisons, there did not appear to be a smooth linear relationship of decreasing event rates for increasing volume, in part, because the highest-volume categories tended to have higher event rates than the intermediate volume categories. In addition, the relative difference in outcomes was fairly modest resulting in an approximate 13% difference in results by cardiologist volume and 19% difference by hospital volume. The absolute difference was even smaller, yielding a number needed to treat to prevent one death attributable to low volume of 454 for cardiologist caseload and 769 for hospital caseload.

Jollis and colleagues (1997) used the national Health Care Financing Administration hospital discharge abstracts to compare the outcomes of 119,886 Medicare beneficiaries undergoing angioplasty in 1992. Their volume categories are based on numbers of Medicare patients undergoing angioplasty, who are thought to account for only one-third to one-half of angioplasty patients, depending on the physician or institution. They relied on administrative data for risk adjustment. The authors reported that higher physician volume was associated with lower risk adjusted rates of the combined endpoint of in-hospital death or CABG, even after controlling for hospital volume. Similarly, they found that higher hospital volume was associated with lower risk adjusted rates of both in-hospital death and the combined outcome of death or CABG, even after controlling for cardiologist volume. However, the actual numerical estimates of the risk-adjusted analyses were not reported in the paper.

The unadjusted numbers were reported and are as follows. Low-volume cardiologists (<25 per year) had higher unadjusted rates of the combined endpoint of inpatient death or CABG compared to those with high-volume (>50 per year)—6.0% versus 4.7%. Low-volume hospitals

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

(<100 per year) had higher unadjusted death or CABG rates than those with high-volume (>200 per year)—6.5% versus 4.9%. The magnitude of these relative differences range from 22% to 25% and correspond to numbers needed to treat to prevent one death or CABG attributable to low provider volume of 77 and 62, respectively. However, they found no significant differences in unadjusted or adjusted inpatient or 30 day mortality rates by volume categories.

Finally, the study by Malenka et al. (1999) is noteworthy because it was one of the few studies in our entire systematic review that collected information on key processes of care and also adjusted for these variables in their multivariate modeling, in this case, the use of heparin, coronary stenting, or intra-aortic balloon pump.

Volume and Outcome in Acute Myocardial Infarction

While there is an enormous health services research literature on acute myocardial infarction, we were only able to identify three studies that met our inclusion (See Appendix I.) Eight other studies were identified by our search algorithm but excluded because they did not explicitly evaluate volume and outcome relationships. Most of these articles focused on the effects of physician specialty or hospital type (teaching vs. non-teaching) on clinical outcomes. One study (Barbash 1994), did explicitly analyze the effects of hospital volume, but was excluded because it was based on a thrombolytic trial database, which was considered a non-population based sample.

All three studies that did meet our inclusion criteria utilized very large hospital discharge databases that contained data on ten of thousands of patients. Thiemann and colleagues (1999) received the highest quality rating of the group because they utilized a robust, clinically-based risk adjustment model and also adjusted for important processes of care known to influence outcomes following myocardial infarction, such as certain medications (aspirin and beta blockers) and revascularization procedures.

Thiemann (1999) used the Health Care Financing Administration Cooperative Cardiovascular Project database that contains clinical and administrative information on a nationwide sample of Medicare patients with acute myocardial infarction during an eight month period during 1994–1995. In addition, to the standard data, the CCP contains information on important processes of care known to influence outcomes, such as the use of aspirin, beta-blockers, ACE inhibitors, thrombolytic therapy, and revascularization procedures. They compared 30 day and one year survival rates according to quartiles of hospital volume defined as < 1.4 MI patients/wk, 1.4– 2.5 per week, 2.6–4.4 per week and >4.4 per week. Unadjusted 30 day mortality rates in the four groups ranging from lowest to highest volume were: 16.7%, 15.3%, 14.7%, and 14.4%, respectively.

Using a detailed clinical risk adjustment model (C = 0.79), they found that compared to the lowest hospital volume category, the hazard ratio for 30 day survival was 1.17 for the highest volume quartile, 1.07 for the second highest quartile and 1.05 for the third quartile. Overall, a decrease of 5.5 Medicare patients per week was associated with a hazard ratio for 30 day death of 1.10 and a long term hazard for 1 year mortality of 1.05. While the risk of death was highest among the lowest-volume hospitals (<1.4 per week), hospital volume remained a continuous predictor even among the two highest volume quartiles.

The authors also stratified by the specialty of the admitting physician, and found that the association between low hospital volume and higher mortality rates was consistently found among patients treated by cardiology, internal medicine, or family practice physicians. Interestingly,

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

they found no relationship between individual physician volume and survival in models with and without physician specialty.

The Theimann (1999) article is most noteworthy for being one of the few studies in our systematic review that examined the independent influence of key processes of care on the volume-outcomes relationship. For example, they reported that adjusting for differences in processes of care such as use of aspirin, thrombolytic agents, beta-blockers, ACE inhibitors, and revascularization procedures accounted for about one third of the survival benefit attributed to high volume hospitals. This indicates that different patterns of use of specific treatments or procedures can explain part, though not all, of the variations seen in outcomes of patients treated in high versus low-volume hospitals.

Casale and colleagues (1998) used administrative database from Pennsylvania to analyze 30,715 admissions for acute myocardial infarction in 1993. The Pennsylvania database also contained clinical data on patient severity using the MedisGroup classification (now called the Atlas System). This clinical data enabled the researchers to produce one of the more robust risk adjustment models identified by our review (C statistic = 0.881). They divided hospitals into low volume providers, those below the median number of myocardial infarction patients ( ≤ 12 per year) and high volume, those above the median (>12 per year). Patients treated by physicians a high volume of cases had 11% lower odds of risk-adjusted inpatient death (OR = 0.89).

Of note, the primary purpose of the study by Casale and colleagues was to examine the effects of physician specialty on outcomes in myocardial infarction. They reported that patients treated by cardiologists had lower in-hospital mortality rates than those treated by primary care physicians (OR = 0.87). However, much of this effect was probably mediated through physician volume effects as 85% of cardiologists were high volume providers compared to 31% of primary care physicians. In addition, high volume remained an independent predictor even after controlling for physician specialty. This study serves as a harbinger that many of the specialist versus generalist debates in the outcomes literature may be seriously confounded by volume effects.

The study by Farley et al. (1992) took a somewhat different approach by focusing on the potential effect of changes in provider volume over time. This is in comparison to the vast majority of articles we reviewed that evaluate volume and outcome associations in a cross-sectional manner. Farley and colleagues used longitudinal data from the Hospital Cost and Utilization Project to analyze patterns of myocardial infarction care in a national sample of 600 non-Federal hospitals over the period of 1980 to 1987. One of the things these authors did differently from most others was to logarithmically transform volume to adjust for skewness (the asymmetric distribution of many low volume hospitals and only a few very-high-volume ones with extremely large caseloads). They then considered the natural log of volume [In(volume)] as a continuous variable. In analyses that used administrative data (including the Disease Staging severity system using ICD-9 discharge diagnosis) for risk adjustment, they reported that a 10% increase in caseload corresponded to a 2.2% absolute decrease in inpatient mortality rates.

Using a set of sophisticated statistical models, they concluded that the strength of the relationship between hospital volume and inpatient mortality increases when one moves from analyzing cross sectional data (as most investigators have done) to longitudinal or “panel” data. Examining data within a given hospital over time increases the likelihood of adequately controlling for hospital-specific factors that could influence both volume and outcomes and thus bias estimates of the direct relationship between volume and outcomes. They also conducted a set of regressions using instrumental variables that they assert shows no selective referral effect for the conditions of myocardial infarction, inguinal hernia repair and neonatal respiratory distress syn-

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

drome. Their study also suggests that other factors such as “nursing intensity,” which are often unmeasured, may be associated with inpatient mortality rates.

Volume and Outcome in AIDS Treatment

Nine research articles on AIDS were reviewed. Three failed to meet inclusion criteria for the systematic review. One was excluded because the same data on outcomes and volume were reported in an earlier article by the same researchers. (Bennett 1990). One was excluded because the patient sample was not community or population based; it reported data from a single institution. (Wachter 1986) The third study was excluded because volume was not an independent variable in its analyses; it investigated the experiences of generalists and specialists as the “predominant providers” for patients with AIDS. (Turner 1994)

Six studies met the inclusion criteria and were analyzed in the review. (See Appendix J for summary overview.) Overall quality scores were low: 3 studies received scores of 7, 2 received scores of 6, and 1 received a 2. Principal common factors underlying these low scores is that all of the investigations assessed death as the only outcome and none evaluated the effect of appropriateness or processes of care. Five studies examined the relationship between outcome and hospital volume alone and one examined the association between survival and physician volume. None examined both hospital and physician experience together. All used at least administrative data to risk-adjust rates of death, and 2 used clinical data but none presented data documenting good discrimination or calibration. All six demonstrated statistically significant inverse associations between hospital or physician volume and death rates. Patient sampling, the formulation of the volume variable, and analytic presentations differed widely among the 6 studies, precluding meta-analysis and rendering generalizations hazardous.

Bennett and colleagues first reported this relationship in a study of administrative data from 15 California metropolitan area hospitals. (Bennett 1989) They studied inpatient mortality for patients admitted with AIDS and their first episode of Pneumocystis carinii pneumonia (PCP) in 1986–1987. They compared outcomes at hospitals with high “familiarity” with AIDS (▯30 admissions of patients with HIV infection per 10,000 total admissions) to those with less familiarity (<30 AIDS-related admissions per 10,000). Unadjusted death rates were 33% at low AIDS-experience hospitals and 12% at high experience hospitals. In their multivariate model (using administrative data to risk-adjust), the odds of a patient dying in a hospital with low-AIDS familiarity were about 3.6 greater than in a hospital with high AIDS familiarity.

Using a similar approach with administrative data from 73 New York City hospitals, Bennett and colleagues also studied inpatient deaths for patients with AIDS and first episodes of PCP in 1987. The volume variable in this study was the number of cases of AIDS-related PCP treated in the study year. Hospitals were classified as low-volume (<160 cases) or high volume (▯160 cases). High-volume hospitals exhibited an unadjusted death rate of 21% compared with 30% at low-volume hospitals. Logistic regression using the administrative data for risk-adjustment demonstrated statistically significant odds ratios of 0.64 (for hospital volume of 160– 239 cases) and 0.67 (for ▯240 cases), both compared with hospitals with fewer than 80 cases.

Stone et al. used clinical data from a state AIDS registry to risk-adjust both inpatient and 30-day death rates for patients with AIDS admitted to Massachusetts hospitals in 1987–1988. (Stone 1992) These investigators chose to define volume as the number of discharges of patients with AIDS per 10,000 discharges at each hospital. The 20% of hospitals with the greatest volume were designated “high-experience” (▯43 AIDS discharges per 10,000), the remainder as “low-experience”(<43 AIDS discharges per 10,000). The unadjusted relative risk of dying in a low-

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

experience hospital was 2.16 times that in a high-experience institution (19.0% vs. 9.8%). The unadjusted relative risk of dying within 30 days of admission was similar (1.93). When adjusted by logistic regression for a variety of demographic and severity of illness variables derived from the AIDS registry, the relative risk of inpatient death remained significant (RR = 2.92 at low versus high-experience hospitals) as did the relative risk of death within 30 days of admission (RR = 2.51).

Turner and Ball used administrative data from a nationally-representative sample of 258 hospitals (from the Hospital Cost and Utilization Project) in 1986–1987 to examine the relationship between death during hospitalization for patients with any manifestation of AIDS and the total number of AIDS discharges in each hospital in each study year. (Turner 1992). In a logistic regression model controlling for demographics and comorbid conditions, an increase in a hospital's volume of AIDS cases of 100 was associated with a decrease of 4.3% in the odds of an inpatient death.

In the only study of hospital care to examine more than 2 years of data, Hogg et al used administrative data to assess inpatient mortality for all patients with AIDS admitted to all Canadian hospitals (n = 513) from 1987 through 1994. In a logistic regression model controlling for age, year of admission, and clinical stage, the average number of AIDS admissions per year was significantly associated with inpatient mortality. Compared with hospitals experiencing fewer than 1 AIDS admission per year, hospitals with 1–9 demonstrated an odds ratio of 0.79; 10–99, OR = 0.77; and those with 100, OR = 0.64.

In the sole study in which the dependent volume variable was physician experience, Kitahata and colleagues found a significant relationship between survival from the time of diagnosis and physician experience with AIDS. (Kitahata 1996) In their sample of 403 patients with AIDS treated by physicians at Group Health Cooperative of Puget Sound, they found a decreased relative risk of death for patients treated by physicians whose experience consisted of more than 5 AIDS patients, compared with those with low experience (1 patient). Using clinical data to adjust for severity of illness and CD4 counts, the relative risk of death was 0.57 for patients of high-experience physicians. The median survival from diagnosis was 14 months for low-experience physicians compared with 26 months for their more experienced colleagues.

Despite their methodologic weaknesses, all six included studies showed statistically significant and, in some cases, clinically impressive associations between hospital volume and mortality (usually inpatient mortality). Whether a publication bias exists that acts against the submission or publication of studies demonstrating no association between volume and outcome in this disease is unknown. None of the 5 studies examining hospital experience assessed physician experience or measured specific processes of care. These studies did little to illuminate the underlying reasons for the associations they documented. No study reported data from the recent era of AIDS treatment, characterized by the emergence of protease inhibitors and an increasing array of complex multidrug regimens. The role of the hospital has diminished greatly in the care of patients with AIDS. How generalizable the findings of these studies are to present-day AIDS care is unclear.

Conclusions

Several conclusions emerge from this systematic review of the research since 1980 on the association between volume and outcome in health care. There can be little doubt that for a wide variety of medical conditions and surgical procedures, patients treated at higher volume hospitals or by higher volume physicians experience on average lower mortality rates than those treated by

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

low-volume hospitals and physicians. The 88 studies we reviewed contained 113 different assessments of the relationship between hospital or physician volume and outcomes; 53 examined hospital volume only, 10 focused on physician volume only, and 25 assessed both. Counting the 25 studies that evaluated both as separate assessments, 77% of all the assessments in the studies in our review found statistically significant associations between either hospital or physician volume and health outcomes. In all cases the association was in the direction of higher volume being associated with better outcomes. No study documented a statistically significant association between higher volume and poorer outcomes. The remaining 23% of evaluations found results that were not statistically significant. This conclusion is strengthened by the further observation that all 16 studies with the highest quality scores (10 or greater) demonstrated statistically significant associations.

The majority (60%) of studies investigated hospital volume effects only, a smaller proportion examined only physician volume (11%), 10% assessed hospital and physician volume but as separate independent variables, and 18% analyzed their relative contributions taken together. Across all conditions and procedures, an almost identical proportion of studies reported statistically significant, positive associations for better outcomes and hospital volume (78%) as for physician volume (74%). Of the 16 studies reporting the independent effects of both physician and hospital volume, 12 (75%) found statistically significant independent effects for both, 3 (19%) found a significant effect for hospital but not physician volume, and 1 (6%) found a significant physician volume association but none for hospitals. A majority (63%) of these 16 studies examined data from one state (New York).

One important cautionary note must be appended to our degree of confidence in the conclusion that the association between volume and outcome is firmly established. We cannot exclude the existence of a significant publication bias that would diminish the likelihood of studies failing to document such associations appearing in the research literature. Although it is possible that such a bias might operate at the level of review and editorial decisions at individual journals, it is also possible, and perhaps more likely, that such a bias could operate at the level of individual investigators, who choose not to pursue or report analyses that fail to demonstrate significant associations. Such decisions could occur because of loss of interest in the research topic or fear a bias on the part of journal editors against studies reporting negative findings. In addition, when faced with data about a number of procedures or conditions, researchers may choose to report only the positive findings.

Risk-adjustment methods varied in their sophistication, with the most rigorous using prospectively collected clinical data to adjust outcomes for severity of illness and comorbidities. Many of these risk-adjustment models achieved high levels of statistical performance in discrimination and calibration. In the vast majority of these studies, statistically significant associations between outcomes and hospital or physician volume were documented. It is thus most unlikely that differences in severity of illness or distribution of comorbid illnesses explain the association between volume and outcome.

The strength and magnitude of the statistically significant relationships described in these studies varied greatly. Many of the included studies analyzed data from tens of thousands of cases, having sufficient sample sizes to detect statistical significance even in small differences in outcomes. In absolute terms, significant differences between high and low volume hospitals ranged from about 1 death (or fewer) per 100 patients treated in PTCA and carotid endarterectomy to 5–10 per 100 in some studies of CABG, pediatric cardiac surgery, and AAA surgery.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

The greatest differences were seen in some studies of complex cancer surgery (e.g., operations on the pancreas or esophagus), where differences as high as 10–15 deaths per 100 were reported.

Overall, the methodological rigor of the published research was modest, with only 18% receiving quality scores of 10 or higher; 75% of these received a score of 10, which is 56% of the maximum score of 18, and only 1 study received the highest score (13), which is 72% of the maximum. Thus, while the research documents the fact that high volume is often associated with better health outcomes, many questions about the nature of the relationship, its causes, and significance remain unanswered. For example, the studies in our review examined a wide variety of volume categories for the same procedures, typically comparing 2 or 3 volume strata or assessing the linear association between volume and outcome. Thus, we know little about whether a volume threshold exists (or more than one) above which outcomes are more favorable but do not continue to fall with further increases in volume. We do not know how these relationships vary among procedures or conditions or how different they are when considering physician volume.

Further, the correlation between volume and outcome, even if statistically significant, is true only on average (Hannan 1999a). Some low-volume hospitals will have outcomes comparable to the best of the high-volume hospitals and some high-volume hospitals will have poor outcomes. Moreover, in any given time period, some portion of the variability among hospitals (or physicians) will be due to chance and not to systematic differences in quality. This variability is not independent of volume. Lower volume hospitals or physicians will demonstrate greater variability, because proportions based on smaller sample sizes will generate greater chance variability compared with larger ones.

This effect is even more pronounced when the event representing the numerator of the proportion is uncommon. Figure 4 illustrates this phenomenon by plotting the hospital mortality rates for all 5705 hospitals treating one or more Medicare beneficiaries with an acute myocardial infarction in 1984 (Chassin 1989). The plotted mortality rates were adjusted for differences in age, sex, and race. As volume increases, the variation among hospitals in mortality rates diminishes dramatically. Further study of these data revealed that 62% of the variability among all hospitals in death rates was due to chance, even after using clinical data to adjust for differences among hospitals in severity, comorbidity, and processes of care (Park 1990). Extant research does not address how much of the observed outcome differences between high and low volume hospitals might be due to chance. This deficit in our knowledge is due in part to the lack of studies using longitudinal data to assess hospital or physician performance.

Another consequence of the paucity of longitudinal studies is the lack of insight provided into the question of the shape and duration of the learning curve, either for new physicians learning standard procedures or treatments or for experienced physicians learning new procedures. A related, rarely-asked question is what level of volume is required to maintain proficiency once acquired. The questions about the learning curve and maintenance of proficiency are equally relevant (and equally unanswered) for hospitals and other important members of the clinical team in addition to the treating physician.

In only 4 procedures were outcomes other than death evaluated—stroke in carotid endarterectomy, emergency CABG surgery in PTCA, local recurrence in colon resection, and bleeding and infection in pancreas surgery. Most of these studies showed the same association with these non-fatal outcomes as with mortality—that they occurred less frequently when hospitals or physicians had more experience. Many other vital health outcomes have not been investigated (e.g., physical or psychological functioning and other local surgical complications).

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Current research did not address the extent to which (if any) the differences between high and low volume providers may be associated with differences in appropriateness of patient selection. Nor does current knowledge permit conclusions to be drawn about what unit of volume is most highly correlated with outcome. For example, a surgeon's proficiency with colon resection may be affected not only by his or her experience with that procedure, but also by experience with small bowel resection, because some very similar surgical tasks are involved in both kinds of procedures. The few studies that have addressed this question have reached variable conclusions and suffered from a lack of methodological rigor. Thus, the “volume of what” question remains unanswered.

This research has also failed to illuminate what underlying causes produce the association between volume and outcome. It is obvious that volume itself cannot directly produce good (or bad) outcomes. Other features of patient care, correlated with volume, are the most likely explanatory factors. Our conceptual model (Figure 1) suggests that specific clinical processes of care that bear a direct relationship to outcome may be performed more often in high-volume hospitals (or by high-volume physicians). The converse may also play a role (that harmful processes are performed less often by high-volume providers). The research literature we reviewed provided scant opportunities to test this hypothesis, as specific clinical processes were rarely studied. In one important study on elderly acute MI patients, the differential use of proveneffective treatments (thrombolytics, aspirin, beta blockers, and ACE inhibitors) explained about one-third of the survival benefit attributable to high-volume hospitals (Thiemann 1999). This

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

study documented that patients at high-volume hospitals with lower mortality rates were more likely to receive effective pharmaceutical treatments than those treated at low-volume hospitals.

We believe that future research directed at uncovering differences in processes of care between high- and low-volume providers carries the best chance of explaining why outcome differences exist between these groups. This research should include investigations of the relationship of the experience of other important members of teams of caregivers (e.g., consultants, intensivists, operating room teams). Organizational practices should also be studied, following on work done in intensive care (Shortell 1994, Pronovost 1999) and surgical services. (Young 1998) In addition, research of this type could help establish the effectiveness of specific processes, particularly in surgery, where such evidence is lacking today. Advancing knowledge in this direction could also provide a blueprint for improvement by identifying precisely what poorperforming providers (whether low-volume or not) need to do to improve.

Policy Implications

Is there a role for public policy to play in the application of this body of knowledge? Our discussion of policy implications is predicated on the assumption that health care policy in this area may have one or both of the following objectives: to inform consumers and clinicians about important information on outcomes and hospital and physician characteristics that may be important in decision making, or to improve patient outcomes.

We see four alternative policy directions that might address these objectives:

Data on volume might be made publicly available.

State governments might take action to limit the number of hospitals permitted to perform certain procedures.

Private purchasers might require or provide incentives for managed care plans or employees to steer patients to high volume providers.

Government (or other state or regional entities) might use data on volume and outcomes to improve performance.

A program of public availability of data on hospital or physician volume of services would have the advantage of access to reliable and reasonably timely data (at least on hospital volume) from many, if not most state hospital discharge abstract data bases. In the absence of reliable and valid data on outcomes, data on volume may be of interest to consumers and useful to referring physicians. Such a program would face the difficulty of explaining in understandable terms the complex relationships between volume and outcome that research has delineated, being careful to clarify what is known and what is not. For example, it will be important to explain the limitations in our knowledge about whether volume thresholds exist for different procedures or conditions, for hospitals, or for physicians. Updating data on a regular basis to reflect not only more recent information about volumes, but also new research findings will present further challenges. The best of our knowledge today suggests that in some instances volume may be a proxy for outcomes that patients experience. But it is only a proxy, and sometimes a poor one.

Few state governments have continued the hospital certificate of need (CON) programs popular in the 1970s. Even when such regulatory programs were popular, only a small proportion focused on the development of requirements for specific services or procedures. New York State is one of the very few remaining states that has an active CON program that does include requirements for specific services (e.g., CABG surgery, PTCA, organ transplantation). New

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

York's successful regionalization of CABG surgery services has been cited as responsible for severely limiting the number of low-volume hospitals performing this operation in the state and, together with its quality measurement and improvement program, for reductions in statewide mortality following the procedure (Grumbach 1995, Chassin 1996, Chassin 1997, Peterson 1998, Sollano 1999). There may be a role for such programs in other states and for other procedures. Political opposition to such regulation is widespread, however, and its implementation and operation require substantial organizational expertise and infrastructure.

Several recent initiatives have aimed to use private sector purchasing power to encourage managed care organizations to contract only with high volume-hospitals or to provide incentives to patients to seek care at such institutions. No published studies have evaluated the impact of such attempts to move substantial numbers of patients from low to high-volume hospitals. How strong such incentives would have to be for significant changes to occur is unknown. Previous research on the impact of publicly available data on mortality following CABG surgery documents the absence of significant movement of patients toward low-mortality hospitals (Chassin 1996), suggesting that information on outcome (or volume) alone may be insufficient to induce substantial changes in where patients receive care. A recent review documents the lack of use of data on performance of various parts of the health care system by consumers in their decision making (Marshall 2000).

Perhaps the most promising approach to the use of these data for improving outcomes is to employ them in quality improvement programs. To the extent that low volume (and poor performance) can be associated with underuse of effective processes of care, demonstrated to improve outcomes (or with an increased use of harmful practices), such knowledge can be put to direct use by these institutions to improve their performance. As a new generation of research produces more such data, the prospect of substantial improvement will increase. Among the most important research efforts to encourage are those which build prospectively-compiled, clinical databases that include all patients and all providers. Such databases can produce powerful risk-adjustment models that permit the production of valid outcome comparisons among providers and provide unparalleled opportunities to examine the effects of specific processes of care on carefully risk-adjusted outcomes. Government and private purchasers can accelerate this improvement by investing in the necessary research, by establishing and evaluating collaborative demonstrations, and by disseminating successful models.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM A: RATING THE QUALITY OF RESEARCH ON VOLUME AND OUTCOME

Objective of Scoring System: designed to measure the degree to which the study design is likely to reveal generalizable conclusions about the magnitude and nature of the relationship between volume and outcome.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM C: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN CABG SURGERY

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM D: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN PEDIATRIC CARDIAC SURGERY

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM E: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN CAROTID ENDARTERECTOMY

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

MD: Ruptured AAA mortality related to surgeon vol of ruptured cases but not to surgeon vol of unruptured cases; Hosp: Trend, but not significant relationship to hosp vol of ruptured or unruptured cases

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM G: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN CANCER SURGERY

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM H: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN CORONARY ANGIOPLASTY

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM I: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN ACUTE MYOCARDIAL INFARCTION

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

ADDENDUM J: SUMMARY OF STUDIES OF VOLUME AND OUTCOME IN AIDS

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

38. Collette L,Sylvester RJ,Stenning SP,et al. Impact of the treating institution on survival of patients with “poor-prognosis” metastatic nonseminoma. European Organization for Research and Treatment of Cancer Genito-Urinary Tract Cancer Collaborative Group and the Medical Research Council Testicular. J Natl Cancer Inst. 1999; 91(10):839-46.

45. de Belder MA. Cardiac intervention procedures in the United Kingdom 1997: developments in data collection. Council of the British Cardiovascular Intervention Society. Heart. 1999; 82 Suppl 2:II2-9.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

76. Grassman ED,Johnson SA,Krone RJ. Predictors of success and major complications for primary percutaneous transluminal coronary angioplasty in acute myocardial infarction . An analysis of the 1990 to 1994 Society for Cardiac Angiography and Interventions registries. Journal of the American College of Cardiology. 1997; 30(1):201-8.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

83. Hannan EL,O'Donnell JF,Kilburn H, Jr.,Bernard HR,Yazici A. Investigation of the relationship between volume and mortality for surgical procedures performed in New York State hospitals. JAMA. 1989; 262(4):503-10.

85. Hannan EL,Kilburn H, Jr.,O'Donnell JF,et al. A longitudinal analysis of the relationship between in-hospital mortality in New York State and the volume of abdominal aortic aneurysm surgeries performed. Health Serv Res. 1992a; 27(4):517-42.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

118. Imperato PJ,Nenner RP,Starr HA,Will TO,Rosenberg CR,Dearie MB. The effects of regionalization on clinical outcomes for a high risk surgical procedure: a study of the Whipple procedure in New York State. American Journal of Medical Quality. 1996; 11(4):193-7.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

124. Jones J,Rowan K. Is there a relationship between the volume of work carried out in intensive care and its outcome? International Journal of Technology Assessment in Health Care. 1995; 11(4):762-9.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

146. Kreder HJ,Deyo RA,Koepsell T,Swiontkowski MF,Kreuter W. Relationship between the volume of total hip replacements performed by providers and the rates of postoperative complications in the state of Washington. Journal of Bone and Joint Surgery. American Volume. 1997; 79(4):485-94.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

264. Verloove-Vanhorick SP,Verwey RA,Ebeling MC,Brand R,Ruys JH. Mortality in very preterm and very low birth weight infants according to place of birth and level of care: results of a national collaborative survey of preterm and very low birth weight infants in The Netherlands . Pediatrics. 1988; 81(3):404-11.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Suggested Citation:"Appendix C: How is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature?." Institute of Medicine. 2000. Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/10005.

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.