Huber PJ. The Behavior of Maximum Likelihood Estimates Under
Non-standard Conditions: Proceedings of the Fifth Berkeley Symposium on Mathematical
Statistics and Probability.Berkeley: University of California Press; 1967:221-233.

Objective To determine the association between the patient-to-nurse ratio and
patient mortality, failure-to-rescue (deaths following complications) among
surgical patients, and factors related to nurse retention.

Design, Setting, and Participants Cross-sectional analyses of linked data from 10 184 staff nurses
surveyed, 232 342 general, orthopedic, and vascular surgery patients
discharged from the hospital between April 1, 1998, and November 30, 1999,
and administrative data from 168 nonfederal adult general hospitals in Pennsylvania.

Main Outcome Measures Risk-adjusted patient mortality and failure-to-rescue within 30 days
of admission, and nurse-reported job dissatisfaction and job-related burnout.

Results After adjusting for patient and hospital characteristics (size, teaching
status, and technology), each additional patient per nurse was associated
with a 7% (odds ratio [OR], 1.07; 95% confidence interval [CI], 1.03-1.12)
increase in the likelihood of dying within 30 days of admission and a 7% (OR,
1.07; 95% CI, 1.02-1.11) increase in the odds of failure-to-rescue. After
adjusting for nurse and hospital characteristics, each additional patient
per nurse was associated with a 23% (OR, 1.23; 95% CI, 1.13-1.34) increase
in the odds of burnout and a 15% (OR, 1.15; 95% CI, 1.07-1.25) increase in
the odds of job dissatisfaction.

Conclusions In hospitals with high patient-to-nurse ratios, surgical patients experience
higher risk-adjusted 30-day mortality and failure-to-rescue rates, and nurses
are more likely to experience burnout and job dissatisfaction.

The past decade has been a turbulent time for US hospitals and practicing
nurses. News media have trumpeted urgent concerns about hospital understaffing
and a growing hospital nurse shortage.1- 3 Nurses
nationwide consistently report that hospital nurse staffing levels are inadequate
to provide safe and effective care.4- 6 Physicians
agree, citing inadequate nurse staffing as a major impediment to the provision
of high-quality hospital care.7 The shortage
of hospital nurses may be linked to unrealistic nurse workloads.8 Forty
percent of hospital nurses have burnout levels that exceed the norms for health
care workers.4 Job dissatisfaction among hospital
nurses is 4 times greater than the average for all US workers, and 1 in 5
hospital nurses report that they intend to leave their current jobs within
a year.4

In 1999, California passed legislation mandating patient-to-nurse ratios
for its hospitals, which goes into effect in July 2003. The California legislation
was motivated by an increasing hospital nursing shortage and the perception
that lower nurse retention in hospital practice was related to burdensome
workloads and high levels of job-related burnout and job dissatisfaction.
Stakeholder groups advocated widely divergent minimum ratios. On medical and
surgical units, recommended ratios ranged from 3 to 10 patients for each nurse.9- 11 In early 2002, California's
governor announced that hospitals must have at least 1 licensed nurse for
every 6 medical and surgical patients by July 2003, a ratio that will move
to 1 to 5 when the mandates are fully implemented.12

This study reports on findings from a comprehensive study of 168 hospitals
and clarifies the impact of nurse staffing levels on patient outcomes and
factors that influence nurse retention.13 Specifically,
we examined whether risk-adjusted surgical mortality and rates of failure-to-rescue
(deaths in surgical patients who develop serious complications) are lower
in hospitals where nurses carry smaller patient loads. In addition, we ascertained
the extent to which more favorable patient-to-nurse ratios are associated
with lower burnout and higher job satisfaction among registered nurses. We
also estimated excess surgical deaths associated with the different nurse
staffing ratios vigorously debated in California. Finally, we estimated the
impact of nurse staffing levels proposed in California on nurse burnout and
dissatisfaction, 2 precursors of turnover.13 Our
findings offer insights into how more generous registered nurse staffing might
affect patient outcomes and inform current debates in many states regarding
the merits of legislative actions to influence staffing levels.

METHODS

Patients, Data Sources, and Variables

Our study combines information about hospital staffing and organization
obtained from nurse surveys with patient outcomes derived from hospital discharge
abstracts and hospital characteristics drawn from administrative databases.14 The study protocol for linking anonymized nurse data
and handling denominalized patient data was approved by the institutional
review board of the University of Pennsylvania.

Hospitals

Data were collected on all 210 adult general hospitals in Pennsylvania.
Information about hospital characteristics was derived from the 1999 American
Hospital Association (AHA) Annual Survey and the 1999 Pennsylvania Department
of Health Hospital Survey.15,16 Ultimately,
168 of the 210 acute care hospitals had discharge data for surgical patients
in the targeted Diagnosis Related Groups (DRGs) during the study period, as
well AHA data, and survey data from 10 or more staff nurses. Six of the excluded
hospitals were Veterans Affairs hospitals, which do not report discharge data
to the state. Twenty-six hospitals were excluded because their administrative
or patient outcomes data could not be matched to our surveys because of missing
variables, primarily because they reported their characteristics or patient
data as aggregate multihospital entities. In 10 additional small hospitals,
the majority of which had fewer than 50 beds, fewer than 10 nurses responded
to the survey.

A nurse staffing measure was calculated as the mean patient load across
all staff registered nurses who reported having responsibility for at least
1 but fewer than 20 patients on the last shift they worked, regardless of
the specialty or shift (day, evening, night) worked. This measure of staffing
is superior to those derived from administrative databases, which generally
include registered nurse positions that do not involve inpatient acute care
at the bedside. Staffing was measured across entire hospitals because there
is no evidence that specialty-specific staffing offers advantages in the study
of patient outcome17 and to reflect the fact
that patients often receive nursing care in multiple specialty areas of a
hospital. Direct measurement also avoided problems with missing data common
to the AHA's Annual Survey of hospitals, which imputed staffing data in 1999
for 20% of Pennsylvania hospitals.

Three hospital characteristics were used as control variables: size,
teaching status, and technology. Hospitals were grouped into 3 size categories:
small (≤100 hospital beds), medium (101-250 hospital beds), and large (≥251
hospital beds). Teaching status was measured by the ratio of resident physicians
and fellows to hospital beds, which has been suggested as superior to university
affiliations and association memberships as an indicator of the intensity
of teaching activity.18 Hospitals with no postgraduate
trainees (nonteaching) were contrasted with those that had 1:4 or smaller
trainee:bed ratios (minor teaching hospitals) and those with ratios that were
higher than 1:4 (major teaching hospitals). Finally, hospitals with facilities
for open heart surgery and/or major transplants were classified as high-technology
hospitals and contrasted with other hospitals.19

Nurses and Nurse Outcomes

Surveys were mailed in the spring of 1999 to a 50% random sample of
registered nurses who were on the Pennsylvania Board of Nursing rolls and
resided in the state. The response rate was 52%, which compares favorably
with rates seen in other voluntary surveys of health professionals.20 Roughly one third of the nurses who responded worked
in hospitals and included the sample of 10 184 nurses described here.
No special recruiting methods or inducements were used. Demographic characteristics
of the respondents matched the profile for Pennsylvania nurses in the National
Sample Survey of Registered Nurses.21 Nurses
employed in hospitals were asked to use a list to identify the hospital in
which they worked, and then were queried about their demographic characteristics,
work history, workload, job satisfaction, and feelings of job-related burnout.
Questionnaires were returned by nurses employed at each of the 210 Pennsylvania
hospitals providing adult acute care. To obtain reliable hospital-level estimates
of nurse staffing (the ratio of patients to nurses in each hospital), attention
was restricted to registered nurses holding staff nurse positions involving
direct patient care and to hospitals from which at least 10 such nurses returned
questionnaires. In 80% of the 168 hospitals in the final sample, 20 or more
nurses provided responses to our questionnaire. There were more than 50 nurse
respondents from half of the hospitals. We examined 2 nurse job outcomes in
relation to staffing: job satisfaction (rated on a 4-point scale from very
dissatisfied to very satisfied) and burnout (measured with the Emotional Exhaustion
scale of the Maslach Burnout Inventory, a standardized tool).22,23

Patients and Patient Outcomes

Discharge abstracts representing all admissions to nonfederal hospitals
in Pennsylvania from 1998 to 1999 were obtained from the Pennsylvania Health
Care Cost Containment Council. These discharge abstracts were merged with
Pennsylvania vital statistics records to identify patients who died within
30 days of hospital admission to control for timing of discharge as a possible
source of variation in hospital outcomes. We examined outcomes for 232 342
patients between the ages of 20 and 85 years who underwent general surgical,
orthopedic, or vascular procedures in the 168 hospitals from April 1, 1998,
to November 30, 1999. Surgical discharges were selected for study because
of the availability of well-validated risk adjustment models.24- 29 The
number of patients discharged from the study hospitals ranged from 75 to 7746.
Only the first hospital admission for any of the DRGs listed in the BOX for
any patient during the study period was included in the analyses.

In addition to 30-day mortality, we examined failure-to-rescue (deaths
within 30 days of admission among patients who experienced complications).24- 29 Complications
were identified by scanning discharge abstracts for International
Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes in the secondary diagnosis and procedure
fields that were suggestive of 39 different clinical events. Distinguishing
complications from previously existing comorbidities involved the use of rules
developed by expert consensus and previous empirical work, as well as examination
of discharge records for each patient's hospitalizations 90 days before the
surgery of interest for overlap in secondary diagnosis codes.27- 29 Examples
of complications included aspiration pneumonia and hypotension/shock. Patients
who died postoperatively were assumed to have developed a complication even
if no complication codes were identified in their discharge abstracts.

Box. Surgical Patient Diagnosis Related Groups Included in the Analyses of Mortality and Failure-to-Rescue

Risk adjustment of mortality and failure-to-rescue for patient characteristics
and comorbidities was accomplished by using 133 variables, including age,
sex, surgery types, and dummy variables indicating the presence of chronic
preexisting health conditions reflected in the ICD-9-CM codes in the discharge abstracts (eg, diabetes mellitus), as well
as a series of interaction terms. The final set of control variables was determined
by a selection process that paralleled an approach used and reported previously.27- 29 The C statistic (area under the receiver operating characteristic curve)
for the mortality risk adjustment model was 0.89.30

Data Analysis

Descriptive data show how patients and nurses in our sample were distributed
across the various categories of hospitals defined by staffing levels and
other characteristics. Logistic regression models were used to estimate the
effects of staffing on the nurse outcomes (job dissatisfaction and burnout)
and 2 patient outcomes (mortality and failure-to-rescue). We computed the
odds of nurses being moderately or very dissatisfied with their current positions
and reporting a level of emotional exhaustion (burnout) above published norms
for medical workers and of patients experiencing mortality and failure-to-rescue
under different levels of registered nurse staffing, before and after control
for individual characteristics and hospital variables. For nurse outcomes,
we adjusted for sex, years of experience in nursing, education (baccalaureate
degree or above vs diploma or associate degree as highest credential in nursing),
and nursing specialty. For analyses of patient outcomes, we controlled for
the variables in our risk adjustment model, specifically, demographic characteristics
of patients, nature of the hospital admission, comorbidities, and relevant
interaction terms. For analyses of both patient and nurse outcomes, we adjusted
for hospital size, teaching status, and technology.

All logistic regression models were estimated by using Huber-White (robust)
procedures to account for the clustering of patients within hospitals and
adjust the SEs of the parameter estimates appropriately.31,32 Model
calibration was assessed with the Hosmer-Lemeshow statistic.33 We
used direct standardization to illustrate the magnitude of the effect of staffing
by estimating the difference in the numbers of deaths and episodes of failure-to-rescue
under different staffing scenarios. Using all patients in the study and using
the final fully-adjusted model, we estimated the probability of death and
failure-to-rescue for each patient under various patient-to-nurse ratios (ie,
4, 6, and 8 patients per nurse) with all other patient characteristics unchanged.
We then calculated the differences in total deaths under the different scenarios.34 Confidence intervals (CIs) for these direct standardization
estimates were derived with the Δ method described by Agresti.35 All analyses were performed using STATA version 7.0
(STATA Corp, College Station, Tex), and P<.05
was considered statistically significant in all analyses.

RESULTS

Characteristics of Hospitals, Nurses, and Patients

Distributions of hospitals with various characteristics, distributions
of nurses surveyed, and patients whose outcomes were studied are shown in Table 1. Fifty percent of the hospitals
had patient-to-nurse ratios that were 5:1 or lower, and those hospitals discharged
65.9% of the patients in the study and employed 64.4% of the nurses we surveyed.
Hospitals with more than 250 beds accounted for a disproportionate share of
both patients and nurses (45.5% and 43.4%, respectively). Although high-technology
hospitals accounted for only 28.0% of the institutions studied, more than
half (55.3%) of the patients discharged and 53.8% of nurses surveyed were
from high-technology hospitals. A majority of the patients studied and nurses
surveyed were drawn from the 61 hospitals (36.3%) that reported postgraduate
medical trainees in 1999.

As shown in Table 2, 94.1%
of the nurses were women and 39.6% held a baccalaureate degree or higher.
The mean (SD) work experience in nursing was 13.8 years (9.8). Thirty-one
percent of the nurses in the sample worked on medical and surgical general
units, while 19.6% and 9.8% worked in intensive care and perioperative settings,
respectively. Forty-three percent of the nurses had high burnout scores and
a similar proportion were dissatisfied with their current jobs.

Of the 232 342 patients studied, 53 813 (23.2%) experienced
a major complication not present on admission and 4535 (2.0%) died within
30 days of admission. The death rate among patients with complications was
8.4%. The surgical case types and clinical characteristics of the patient
cohort are shown in Table 3. Slightly
more than half of patients (51.2%) were classified in an orthopedic surgery
DRG, with the next largest group of patients (36.4%) undergoing digestive
tract and hepatobiliary surgeries. Chronic medical conditions, with the exception
of hypertension, were relatively uncommon among these patients. Patients who
experienced complications and were included in our analyses of failure-to-rescue
were similar to the broader group of patients in our mortality analyses with
respect to their comorbidities, but orthopedic surgery patients were less
prominently represented among patients with complications than in the overall
sample.

Staffing and Job Satisfaction and Burnout

Higher emotional exhaustion and greater job dissatisfaction in nurses
were strongly and significantly associated with patient-to-nurse ratios. Table 4 shows odds ratios (ORs) indicating
how much more likely nurses in hospitals with higher patient-to-nurse ratios
were to exhibit burnout scores above published norms and to be dissatisfied
with their jobs. Controlling for nurse and hospital characteristics resulted
in a slight increase in these ratios, which in both cases indicated a pronounced
effect of staffing. The final adjusted ORs indicated that an increase of 1
patient per nurse to a hospital's staffing level increased burnout and job
dissatisfaction by factors of 1.23 (95% CI, 1.13-1.34) and 1.15 (95% CI, 1.07-1.25),
respectively, or by 23% and 15%. This implies that nurses in hospitals with
8:1 patient-to-nurse ratios would be 2.29 times as likely as nurses with 4:1
patient-to-nurse ratios to show high emotional exhaustion (ie, 1.23 to the
4th power for 4 additional patients per nurse = 2.29) and 1.75 times as likely
to be dissatisfied with their jobs (ie, 1.15 to the 4th power for 4 additional
patients per nurse = 1.75). Our data further indicate that, although 43% of
nurses who report high burnout and are dissatisfied with their jobs intend
to leave their current job within the next 12 months, only 11% of the nurses
who are not burned out and who remain satisfied with their jobs intend to
leave.

Staffing and Patient Mortality and Failure-to-Rescue

Among the surgical patients studied, there was a pronounced effect of
nurse staffing on both mortality and mortality following complications. Table 4 also shows the relationship between
nurse staffing and patient mortality and failure-to-rescue (mortality following
complications) when other factors were ignored, after patient characteristics
were controlled, and after patient characteristics and other hospital characteristics
(size, teaching status, and technology) were controlled. Although the ORs
reflecting the nurse staffing effect were somewhat diminished by controlling
for patient and hospital characteristics, they remained sizable and significant
for both mortality and failure-to-rescue (1.07; 95% CI, 1.03-1.12 and 1.07;
95% CI, 1.02-1.11, respectively). An OR of 1.07 implies that the odds of patient
mortality increased by 7% for every additional patient in the average nurse's
workload in the hospital and that the difference from 4 to 6 and from 4 to
8 patients per nurse would be accompanied by 14% and 31% increases in mortality,
respectively (ie, 1.07 to the 2nd power = 1.14 and 1.07 to the 4th power =
1.31).

These effects imply that, all else being equal, substantial decreases
in mortality rates could result from increasing registered nurse staffing,
especially for patients who develop complications. Direct standardization
techniques were used to predict excess deaths in all patients and in patients
with complications that would be expected if the patient-to-nurse ratio for
all patients in the study were at various levels that figure prominently in
the California staffing mandate debates. If the staffing ratio in all hospitals
was 6 patients per nurse rather than 4 patients per nurse, we would expect
2.3 (95% CI, 1.1-3.5) additional deaths per 1000 patients and 8.7 (95% CI,
3.9-13.5) additional deaths per 1000 patients with complications. If the staffing
ratio in all hospitals was 8 patients per nurse rather than 6 patients per
nurse, we would expect 2.6 (95% CI, 1.2-4.0) additional deaths per 1000 patients
and 9.5 (95% CI, 3.8-15.2) additional deaths per 1000 patients with complications.
Staffing hospitals uniformly at 8 vs 4 patients per nurse would be expected
to entail 5.0 (95% CI, 2.4-7.6) excess deaths per 1000 patients and 18.2 (95%
CI, 7.7-28.7) excess deaths per 1000 complicated patients. We were unable
to estimate excess deaths or failures associated with a ratio of 10 patients
per nurse (one of the levels proposed in California) because there were so
few hospitals in our sample staffed at that level.

COMMENT

Registered nurses constitute an around-the-clock surveillance system
in hospitals for early detection and prompt intervention when patients' conditions
deteriorate. The effectiveness of nurse surveillance is influenced by the
number of registered nurses available to assess patients on an ongoing basis.
Thus, it is not surprising that we found nurse staffing ratios to be important
in explaining variation in hospital mortality. Numerous studies have reported
an association between more registered nurses and lower hospital mortality,
but often as a by-product of analyses focusing directly on some other aspect
of hospital resources such as ownership, teaching status, or anesthesiologist
direction.19,27,36- 42 Therefore,
a simple search for literature dealing with the relationship between nurse
staffing and patient outcomes yields only a fraction of the studies that have
relevant findings. The relative inaccessibility of this evidence base might
account for the influential Audit Commission in England concluding recently
that there is no evidence that more favorable patient-to-nurse ratios result
in better patient outcomes.43

Our results suggest that the California hospital nurse staffing legislation
represents a credible approach to reducing mortality and increasing nurse
retention in hospital practice, if it can be successfully implemented. Moreover,
our findings suggest that California officials were wise to reject ratios
favored by hospital stakeholder groups of 10 patients to each nurse on medical
and surgical general units in favor of more generous staffing requirements
of 5 to 6 patients per nurse. Our results do not directly indicate how many
nurses are needed to care for patients or whether there is some maximum ratio
of patients per nurse above which hospitals should not venture. Our major
point is that there are detectable differences in risk-adjusted mortality
and failure-to-rescue rates across hospitals with different registered nurse
staffing ratios.

In our sample of 168 Pennsylvania hospitals in which the mean patient-to-nurse
ratio ranged from 4:1 to 8:1, 4535 of the 232 342 surgical patients with
the clinical characteristics we selected died within 30 days of being admitted.
Our results imply that had the patient-to-nurse ratio across all Pennsylvania
hospitals been 4:1, possibly 4000 of these patients may have died, and had
it been 8:1, more than 5000 of them may have died. While this difference of
1000 deaths in Pennsylvania hospitals across the 2 staffing scenarios is approximate,
it represents a conservative estimate of preventable deaths attributable to
nurse staffing in the state. Our sample of patients represents only about
half of all surgical cases in these hospitals, and other patients admitted
to these hospitals are at risk of dying and similarly subject to the effects
of staffing. Moreover, in California, which has nearly twice as many acute
care hospitals and discharges and an overall inpatient mortality rate higher
than in our sample in Pennsylvania (2.3% vs 2.0%), it would be reasonable
to expect that the difference of 4 fewer patients per nurse might result in
2000 or more preventable deaths throughout a similar period.

Our results further indicate that nurses in hospitals with the highest
patient-to-nurse ratios are more than twice as likely to experience job-related
burnout and almost twice as likely to be dissatisfied with their jobs compared
with nurses in the hospitals with the lowest ratios. This effect of staffing
on job satisfaction and burnout suggests that improvements in nurse staffing
in California hospitals resulting from the new legislation could be accompanied
by declines in nurse turnover. We found that burnout and dissatisfaction predict
nurses' intentions to leave their current jobs within a year. Although we
do not know how many of the nurses who indicated intentions to leave their
jobs actually did so, it seems reasonable to assume that the 4-fold difference
in intentions across these 2 groups translated to at least a similar difference
in nurse resignations. If recently published estimates of the costs of replacing
a hospital medical and surgical general unit and a specialty nurse of $42 000
and $64 000, respectively, are correct, improving staffing may not only
save patient lives and decrease nurse turnover but also reduce hospital costs.44

Additional analyses indicate that our conclusions about the effects
of staffing and the size of these effects are similar under a variety of specifications.
We allowed the effect of nurse staffing to be nonlinear (using a quadratic
term) and vary in size across staffing levels (using dummy variables and interaction
terms) and found no evidence in this sample of hospitals that additional registered
nurse staffing has different effects at differing staffing levels. Limiting
our analyses to general and orthopedic surgery patients and eliminating vascular
surgery patients (who have higher mortality and complication rates) did not
affect our conclusions and effect-size estimates. Also, our findings were
not changed by restricting attention to inpatient deaths vs deaths within
30 days of admission. Results were unaffected by restricting analyses to patients
who were discharged after our staffing measures were obtained, rather than
to the patients who were discharged from 9 months before to 9 months following
the nurse surveys that produced our staffing measures. They were also unchanged
by restricting the sample of nurses from which we derived our staffing measures
to medical and surgical nurses, as opposed to all staff nurses. Finally, they
were neither altered by adjusting for patient-to-licensed practical nurse
ratios and patient-to-unlicensed assistive personnel ratios (neither of which
were related to patient outcomes) nor affected by excluding the hospitals
in our sample with smaller numbers of patients or nurses.

One limitation of this study is the potential for response bias, given
a 52% response rate. We find no evidence that the nurses in our sample were
disproportionately dissatisfied with their work relative to Pennsylvania staff
nurses from the National Sample Survey of Registered Nurses (a national probability-based
sample survey performed in 2000).21 Furthermore,
with respect to demographic characteristics (sex, age, and education) included
in both surveys, our sample of nurses also closely resembles those participating
in the National Sample Survey of Registered Nurses. We are confident that
these results are not specific to this particular sample of nurses. Ultimately,
longitudinal data sets will be needed to exclude the possibility that low
hospital nurse staffing is the consequence, rather than the cause, of poor
patient and nurse outcomes.

Our findings have important implications for 2 pressing issues: patient
safety and the hospital nurse shortage. Our results document sizable and significant
effects of registered nurse staffing on preventable deaths. The association
of nurse staffing levels with the rescue of patients with life-threatening
conditions suggests that nurses contribute importantly to surveillance, early
detection, and timely interventions that save lives. The benefits of improved
registered nurse staffing also extend to the larger numbers of hospitalized
patients who are not at high risk for mortality but nevertheless are vulnerable
to a wide range of unfavorable outcomes. Improving nurse staffing levels may
reduce alarming turnover rates in hospitals by reducing burnout and job dissatisfaction,
major precursors of job resignation. When taken together, the impacts of staffing
on patient and nurse outcomes suggest that by investing in registered nurse
staffing, hospitals may avert both preventable mortality and low nurse retention
in hospital practice.

Huber PJ. The Behavior of Maximum Likelihood Estimates Under
Non-standard Conditions: Proceedings of the Fifth Berkeley Symposium on Mathematical
Statistics and Probability.Berkeley: University of California Press; 1967:221-233.