Patients: 152 outpatients with heart failure or patients admitted to hospital because of heart failure, who were included in a randomised
trial to assess the impact of a pharmacist led intervention to improve drug compliance. Duration of follow up was at least
18 months.

Main outcome measures: Multivariable logistic regression modelling was used to evaluate information from history, physical examination (for example,
blood pressure), drug use, and quality of life questionnaires that independently contributed to the prediction of death. The
area under receiver operating characteristic curves (AUC) was used to estimate the predictive ability of the prognostic models.

Results: During the 18 months of follow up, 51 patients (34%) died. Independent predictors of mortality were diabetes mellitus, a
history of renal dysfunction (or higher creatinine), New York Heart Association (NYHA) functional class III or IV, lower weight
or body mass index, lower blood pressure, ankle oedema, and higher scores on a disease specific quality of life questionnaire.
The use of β blockers was predictive of a better prognosis. These factors were used to derive various prediction formulas.
A model based on medical history, weight, presence of oedema, and lower blood pressure had an AUC of 0.77. Addition of use
of β blockers to this model improved the AUC to 0.80. Addition of NYHA class increased the AUC to 0.84. Data on quality of
life did not improve the AUC further (AUC 0.85).

Conclusions: A prognostic model produced on the basis of easily obtainable information from medical history and physical examination can
adequately stratify heart failure patients according to their short term risk of death.

Mortality among patients with heart failure discharged from hospital has repeatedly been reported to be high.1–4 Determinants of early death or readmission to hospital in these patients have been investigated in several studies. A wide
variety of factors is reported to be associated with an increased risk of hospital admission or death, including demographic
factors (for example, male sex and single marital status), clinical characteristics (lower systolic blood pressure, renal
dysfunction), history of heart failure (previous hospital admissions), and comorbidity (diabetes and depression).2,5–8 Some of these studies focused on invasive and non-invasive test results (such as echocardiographically determined ejection
fraction) which are not widely available for all patients, particularly those managed mainly in primary care.5–7

Our aim in this study was to develop a comprehensive and easily applicable prognostic model predicting the risk of death in
patients with heart failure, based on information that is readily available in medical practice.

METHODS

Patients and prognostic determinants

The study group consisted of 152 patients (average age 69.7 years, range 37–91 years) enrolled in a randomised controlled
trial evaluating the effect of a pharmacist led intervention on drug compliance in patients with heart failure. All patients
in the study were treated with loop diuretics and were either admitted to one of the participating hospitals because of heart
failure (ICD-9, 428) or were treated in a specialised outpatient heart failure clinic. Eligible patients with heart failure
were included by their cardiologist. The diagnosis of heart failure was validated using the patient’s hospital records, including
cardiac imaging findings. Patients with severe psychiatric problems or dementia, those with a planned admission to a nursing
home, those who did not manage their own drug treatment (for example, where the treatment was given by relatives or district
nurses), and those with a life expectancy of less than three months were excluded from the study.

Patients were enrolled in seven hospitals in the province of Utrecht, Netherlands (one university hospital and six regional
hospitals). Most of the patients (70%) were enrolled in two large regional hospitals (containing more than 500 beds). The
prognostic model was developed in all 152 patients included in the study.

As potential prognostic determinants, information from the patients’ medical history, physical examination, and laboratory
tests was obtained from hospital records, while quality of life was assessed using a generic questionnaire (COOP/WONCA charts)
and a disease specific questionnaire (Minnesota “living with heart failure” questionnaire). Information on survival during
the 18 months follow up period was retrieved from the hospital records, from the patients’ general practitioners, and from
the community pharmacy.

Data analysis

Crude risk ratios for 18 months mortality were calculated for all potential prognostic determinants. Continuous variables
were initially analysed without categorisation, but various cut off values were evaluated as well. All factors with a probability
value of p < 0.10, together with age and sex, were included in multivariable logistic regression analyses.

Models were constructed in accordance with the chronology in which predictors are available in clinical practice. Hence, we
first included all variables from the patient’s history into an overall “history model.” Model reduction was done by excluding
variables with p values > 0.10. The reduced history model was consecutively extended with data from physical examination—for
example, body mass index, blood pressure variables—and laboratory tests to determine their added value in predicting death.
For each model, the reliability (goodness of fit) was quantified using the Hosmer and Lemeshow test. Goodness of fit statistics
examine the difference between the observed frequency and the expected frequency for groups of patients. The statistic can
be used to determine if the model provides a good fit for the data. If the p value is large, the model is well calibrated
and fits the data well.9 The predicted values from the logistic regression model were used to construct receiver operating characteristic (ROC) curves
and to calculate the area under the ROC curves (AUC).10 The ROC area is a suitable parameter to summarise the discriminative or predictive value and can range from 0.5 (no discrimination,
like a coin flip) to 1.0 (perfect discrimination). Subsequently, to obtain an easily applicable prediction rule, the adjusted
regression coefficients of the model were multiplied by a factor 10 and rounded to the nearest integer.

Missing values

Deleting subjects with a missing value on one of the predictors included in the multivariable model (so called complete case
analyses) commonly leads to biased results and definitely to a loss of power.11,12 To decrease bias and increase statistical efficiency, it is better to impute these missing values rather than doing a complete
case analyses.11,12 Accordingly, we imputed our missing data using the expectation maximisation method available in SPSS (version 10.0) software.
Such imputations are based on the correlation between each variable with missing values and all other variables, as estimated
from the set of complete subjects.

RESULTS

Within the 18 months follow up period, 51 (34%) of the 152 heart failure patients died (mortality at six and 12 months, 26
(17%) and 43 (28%), respectively). The cause of death was known in 35 of these patients: they all died from a cardiovascular
cause (for example, terminal heart failure, sudden cardiac death). The remaining deaths occurred at home and no documentation
of the cause was available. Table 1 shows the results of the crude association of potential prognostic determinants with the
18 months mortality. The strongest predictors were New York Heart Association (NYHA) classification, Minnesota heart failure
score, renal dysfunction, and the use or non-use of β blocking agents.

The overall clinical model (that is, the first column of table 2) had an AUC of 0.77. The inclusion of the use of β blockers
in this model improved the AUC to 0.80, while the introduction of NYHA class III or IV further improved the AUC to 0.84. The
introduction of the Minnesota heart failure score yielded a predictive accuracy with an AUC of 0.85. The fit of all models
was good: p values of the Hosmer and Lemeshow statistic ranged from 0.2–0.9.

As the clinical model with information on drug treatment combines data readily available for practising clinicians we transformed
this model to a scoring rule: age/17 + 4 for male + 9 for presence of diabetes + 17 for history of renal dysfunction + 10
for presence of ankle oedema + 7 for systolic blood pressure < 110 or diastolic blood pressure < 70 − weight/3 + 13 for absence
of use of β blockers (table 3). Such a scoring rule can be considered as one overall measure for predicting mortality in heart
failure patients. The score was calculated for each subject by assigning points for each predictor present and adding these
points. For instance, a woman of 60 years and weighing 70 kg, with a history of renal insufficiency, diabetes, ankle oedema,
and a blood pressure of 130/80 and who does not use a β blocker receives a score of (60/17 + 9 + 17 + 10 − 70/3 +13) = 29.2.
In our data, the score ranged from −32.2 to +31.7 (mean −0.3, median 0.5) and the AUC of the rule was 0.80 (95% confidence
interval 0.72 to 0.88).

Regression coefficient and score of each predictor included in clinical model + drug treatment at baseline

Table 4 shows the incidence of mortality among patients across different score categories. From this table one can directly
obtain the observed mortality per score category (reading horizontally). For example, of 23 subjects with a score of > 11,
78% (n = 18) died, while only 12% (n = 3) of the 25 subjects with a score of < −15 died. Similarly, the positive and negative
predictive values for various cut off points can be calculated: the positive predictive value for a score ≥ 7 is 31/48 = 65%,
while the negative predictive value of a score < 7 is 84/104 = 81%. Reading the table vertically provides estimates of the
sensitivity and specificity at different thresholds. For example, 74 (26 + 25 + 23) subjects received a score of ≥ 1. Of these,
43 (12 + 13 + 18) indeed died, correctly predicting 84% of all deaths (that is, the sensitivity or true positive rate). As
31 (31%) of all subjects predicted as future deaths did not die, the specificity of a threshold of 1 was 100 − 31% = 69%.

DISCUSSION

This study shows that a combination of easily obtainable variables accurately predicts 18 months mortality in patients with
heart failure. Except for the use of β blockers, weight, and quality of life score, these predictors were also used in earlier
studies (table 5).

Several studies have used multivariate logistic regression to derive predictive models.1,2,5–7,13–15 Most of these studies, however, involved highly selected and mostly relatively young patient populations (for example, patients
referred for cardiac transplantation). Our study was undertaken in patients included in a randomised controlled trial. Although
there were few inclusion or exclusion criteria in this trial, patients with severe psychiatric problems or dementia, those
with a planned admission to a nursing home, those who did not manage their own drug treatment, and those with a life expectancy
of less than three months (in the opinion of their physician) were excluded. The exclusion of patients with a short life expectancy
mainly led to the exclusion of those with malignancies and other comorbidities, but not to the exclusion of those with moderate
or severe heart failure. It should be borne in mind that these patients were in a clinical trial designed to improve compliance.
Patients had to give their informed consent and this is likely to have led to the selection of a group of relatively motivated
patients. However, it is not plausible that this would have influenced the predictive value of objective parameters such as
the presence of diabetes or renal dysfunction.

Other studies have often included variables that are not widely available in heart failure patients. The determinants included
in our study, however, are usually routinely available and they turned out to be at least as predictive of mortality as those
more specialised variables included in other studies.6,7 Our predictive values were comparable with those in a recent study in which age, the presence of pulmonary crepitations,
a lower systolic blood pressure, and higher creatinine concentrations were most predictive of mortality.2 That study, however, did not report the prognostic value of diabetes, which was an important predictor in our study as well
as in another study.1

Although results of echocardiography (95%), chest radiography (87%), and electrocardiography (99%) were available in most
of our cohort, more specific information on these data, such as ejection fraction (54%) and diastolic function (35%), were
only available in a proportion of the participants and were subject to large intrahospital differences. These findings were
therefore not included in the analyses.

Finally, although other studies have composed logistic regression models, they did not assess the prognostic performance of
a scoring rule combining the individual predictors derived from the model. Although there is no consensus on sample size estimations
in studies deriving multivariable prognostic models, as a rule of thumb several investigators recommend 10 or more events
per variable to allow a robust estimation of the coefficients.16 As we included more variables in our models, the precision of some of our estimates may be limited. However, limitation of
our analyses to the seven determinants most likely to influence prognosis, using the clinical + drug treatment at baseline
model excluding age and sex, yielded the same ROC area: 0.80 (95% confidence interval 0.71 to 0.87).

We did not have data on a second cohort of patients with heart failure. Therefore we could not carry out an external validation
study. Our scoring rule will be validated in another ongoing study in our group. Ideally, such a validation should take place
before the model can be applied in practice.

Easily obtainable clinical data can identify a group of heart failure patients at increased risk of mortality. In their daily
practice physicians normally try to estimate the prognosis of their individual patients. Our scoring rule enables them to
do this in a more rational way, which should be helpful in decision making. For example, patients with a very poor prognosis
might be excluded from invasive treatments. On the other hand, a focus on the management of diabetes, improvement of renal
function, attention to cachexia, and optimisation of drug treatment in patients identified as at high risk of early death
may be of value. Although quality of life scores are independent predictors of mortality, their added prognostic value is
too small to warrant quality of life measurements for such a purpose in routine clinical practice. Additional research is
necessary to validate the proposed model in other populations.

Acknowledgments

We would like to acknowledge Dr Karel G M Moons from the Julius Centre for Health Sciences and Primary Care, University Medical
Centre Utrecht, the Netherlands, for his help in developing the statistical model.