This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

The objective of this work was to assess the convergent validity of a previously developed
rheumatoid arthritis medical records-based index of severity (RARBIS) by comparing
it with the 28-joint Disease Activity Score (DAS28). This study was conducted in subjects
within the Brigham and Women's Hospital Rheumatoid Arthritis Sequential Study (BRASS).
We selected 100 patients with rheumatoid arthritis (RA) from the BRASS with DAS28
scores equally distributed in four quartiles. The medical records were reviewed to
calculate the RARBIS, which includes indicators from the following categories: prior
surgical history, radiologic and laboratory findings, clinical and functional status,
and extra-articular manifestations. The Spearman correlation between the RARBIS and
the DAS28 was assessed in the total study population and in relevant subgroups. We
re-weighted on subscales and recalculated the RARBIS score. This was performed based
on findings of correlations between the DAS28 and subscales; and also the result from
a multiple linear regression with the DAS28 (as a dependent variable) and five subscales
(as independent variables). The mean RARBIS was 4.36 (range 0–11). Among the total
study cohort, the RARBIS was moderately correlated with the DAS28 (r = 0.41, 95% confidence interval [CI] 0.23–0.56). In subgroup analyses, including age,
gender, rheumatoid factor status, and disease duration, we found no statistically
significant differences in the correlations. After re-weighting, the correlation between
the RARBIS and the DAS28 was somewhat improved (r = 0.48, 95% CI 0.31–0.62). In conclusion, the RARBIS correlated moderately well with
the DAS28 in this population. The RARBIS has both face and convergent validity for
patients with RA and relevant subgroups and may have application for medical records
studies in patients with RA.

Introduction

Rheumatoid arthritis (RA) is a chronic inflammatory disease that can lead to long-term
joint damage resulting in chronic pain, loss of function, and disability [1]. It causes substantial morbidity in most patients and premature mortality in many
[2-5]. Some patients with severe RA may be at higher risk for complications such as infection,
gastrointestinal problems, heart disease, and cancer [6,7]. However, those complications may be related to adverse effects of drug therapies
rather than to the effect of RA.

Several studies have reported serious but rare adverse effects of RA medications.
For example, biologic agents that block the action of tumor necrosis factor-α have
been investigated as the cause of serious infections, hematological cancers, and demyelinating
disease [8-11]. Lymphoproliferative malignancies among users of disease modifying anti-rheumatic
drugs have also been reported [12,13].

Data on adverse drug events come predominantly from clinical trials, case reports,
case series, and epidemiologic studies. Randomized clinical trials are limited in
their ability to detect rare adverse effects because of small sample size, selection
of patients least likely to experience toxicity, and short duration of follow-up.
It is difficult to base causality assessment on case reports/series. Therefore, pharmacoepidemiologic
studies can play a pivotal role in evaluating safety of medications used in RA. However,
the severity of RA may affect the choice of medication and RA outcomes. Failing to
control for RA severity in epidemiologic studies may lead to biased estimates of the
association between RA drug treatment and RA outcomes. This type of bias, confounding
by indication, is an important potential bias in many pharmacoepidemiologic studies
[14,15].

To measure RA disease severity in medical records, we defined a set of indicators
of severe RA through an expert Delphi panel of rheumatologists [16]. On the basis of their findings, we developed an RA medical records-based index of
severity (RARBIS) [17]. The goal of this project was to test the convergent validity of the RARBIS with
another accepted RA measure. However, there is no standard criteria for RA severity
that can be assessed from medical records.

Although disease activity and disease severity are not synonymous, we decided the
correlation of the RARBIS with the 28-joint Disease Activity Score (DAS28), a widely
used instrument that affects treatment decisions by rheumatologists in daily clinical
practice. The DAS28 is a statistically derived index consisting of number of swollen
joints, number of tender joints, erythrocyte sedimentation rate (ESR), and general
health [18,19]. Thus, recognizing the methodologic limitations of this approach, we chose to assess
the convergent validity of the RARBIS by comparing it with the DAS28, a well accepted
RA disease index.

Materials and methods

Study population

The Brigham and Women's Hospital Rheumatoid Arthritis Sequential Study (BRASS) is
a cohort of patients with confirmed RA who receive care from rheumatologists in the
Division of Rheumatology, Immunology and Allergy at our hospital. Every six months,
patients are asked to complete questionnaires regarding general health information,
medications for RA, health status, and surgical history for RA. Clinicians complete
a similar questionnaire, including global assessment and joint counts. For our study,
patients were identified from the BRASS cohort between March and October 2003. Identified
patients during the study period were partitioned in quartiles based on their baseline
DAS28 scores. We randomly selected 100 RA patients with the baseline DAS28 equally
distributed in four quartiles.

RARBIS

The RARBIS was developed through an expert Delphi panel of six rheumatologists, and
convergent validity was assessed by comparing it with intensity of the actual RA treatments
that patients received [16,17] (Table 1.) The index includes indicators from five categories: prior surgical history (C1–C2
fusion and joint surgeries), radiologic findings (C1–C2 subluxation and erosions),
laboratory findings (rheumatoid factor status, ESR, C-reactive protein, and platelet
counts), clinical and functional status (arthritis flares, morning stiffness, physician
global rating, and functional status), and extra-articular manifestations (vasculitis
and pulmonary nodule). These indicators are available in medical records. The total
RARBIS score was calculated by summing the five subscales. Because joint destructions
and surgical histories were considered by the Delphi panel to be indicators strongly
associated with severe RA, the prior surgical and radiologic subscales were given
higher weights in the total RARBIS score. The surgical subscale and radiologic subscale
account for 60% of the total RARBIS score. The RARBIS score with medication use was
also calculated with the addition of data on medication use.

Data collection and processing

Electronic medical records were reviewed to collect information on demographic characteristics,
clinical and functional status, laboratory test results, and radiology reports. Data
on clinical status indicators and laboratory test results were collected for one year
prior to baseline. If a patient's clinical and functional status changed during that
period, the worst condition was recorded. When there was an expression indicating
disease activity such as "flare up," "ongoing," and "active" in medical records, we
counted it as having a flare. The number of flares in the last year was summed. If
there was no information regarding clinical and functional status on the medical chart,
we assumed a patient had no apparent clinical and functional manifestations during
that period. The value associated with the most severe disease activity in the preceding
year was recorded. Whether patients ever had any erosions and C1–C2 subluxation was
examined in radiology reports. We obtained information on surgical history, extra-articular
manifestations, physician's global assessment of arthritis activity, and medication
use from a questionnaire at baseline in BRASS, which was completed by patients and
physicians. If data were missing on radiology findings, laboratory tests, surgical
history, or extra-articular manifestations, we assumed that no clinical information
or radiologic/laboratory reports suggested that patients did not havesignificant RA
symptoms at that time. The study was approved by the Institutional Review Board of
Partners HealthCare (Boston, MA, USA).

Statistical analysis

To test the convergent validity of RARBIS as a measure of disease severity, the Spearman
rank correlation between the RARBIS and the DAS28 was assessed in the total study
population and in relevant subgroups, including age, gender, rheumatoid factor status,
and disease duration. To examine whether different subscale weights improved the performance
of the index, we re-weighted subscales and recalculated the RARBIS score. The original
weights of the RARBIS were determined based on the finding of the Delphi panel rating
[16]. The surgical subscale was weighted more than the clinical subscale. The new weights
were derived from the correlations between the DAS28 and subscales and the result
from a multiple linear regression with the DAS28 as a dependent variable. Because
clinical indicators and radiologic indicators were closely correlated with the DAS28
and the surgical history was not correlated with the DAS28, we up-weighted the clinical
subscale and down-weighted the surgical subscale. Three different weighting systems
included: exchanging the maximum scores of the clinical status subscale and the surgery
subscale; up-weighting the clinical status subscale by a factor of two and removing
the surgery subscale and the extra-articular manifestation subscale; multiplying each
subscale by the regression coefficients of a linear regression of all subscales on
the DAS28.

Results

Patients characteristics and the RARBIS score

Patient characteristics of the study cohort are summarized in Table 2. The mean age of patients was 59 years with the majority less than 65 years. Most
of the patients were female (81%) with mean disease duration of 16 years. The mean
value of the DAS28 was 4.2 (standard deviation [SD] 1.6) at baseline. The mean total
RARBIS score calculated for all patients was 4.36 (SD 2.37, range 0–11) (Figure 1). The distributions of each subscale and total score are shown in Table 3. The mean of the clinical subscale was 1.43 (SD 1.20) with a range of 0–3. Patients
in the study had few surgeries, few extra-articular manifestations, and low scores
on the radiology subscale.

Comparisons of the RARBIS with the DAS28

Table 4 shows the Spearman correlation with 95% confidence interval (CI) between the RARBIS
and the DAS28. The total score was moderately correlated with the DAS28 (r = 0.41, 95% CI 0.23–0.56). Figure 2 shows scatter plots of the paired RARBIS scores and corresponding DAS28 scores. Two
components of the total score, the clinical subscale and the X-ray subscale, were
significantly related to the DAS28. The laboratory subscale had a weak correlation
with the DAS28. The other components were not correlated with the DAS28. When the
subscale of medication use was added to the RARBIS, the score was also significantly
correlated with the DAS28, though the correlation was attenuated.

Table 4. Spearman's rank correlation of the total RARBIS score with the DAS28

Correlations between the DAS28 and the RARBIS among subgroups are shown in Table 5. In subgroup analyses, we found no statistically significant differences between
the subgroups in their correlations between the DAS28 and the RARBIS. However, the
sample size in each subgroup was small.

Table 5. Correlations between the DAS28 and the total RARBIS score among subgroups

Re-weighting

Up-weighting on the clinical subscale and down-weighting on the surgical subscale
slightly improved the correlation with the DAS28. Different weights did not make large
differences in correlation between the RARBIS and the DAS28 (Table 6).

Discussion

The results of our study provide support for the convergent validity of the RARBIS
with the DAS28. We are hopeful that the RARBIS can be used to help define RA severity
but recognize that there are no well accepted measures of RA severity, making validation
quite a challenge. RA severity is considered a complex measure determined by objective
components such as disease activity (for example, tender/swollen joints and acute-phase
reactants) and physical damage (for example, radiologic damage, and functional disability)
and subjective components (for example, global health assessment, pain, grip strength,
fatigue, and costs) [20,21]. We chose to use the DAS28, a measure of disease activity, because it is commonly
used for therapeutic decision-making.

Symmons and colleagues [22] developed a measure of overall status in RA composed of demographic details, activity
score, damage score, and drug treatments [23]. Bardwell and colleagues [24] reported a brief measure of severity for RA based on physician's rating of disease
activity, functional impairment, and physical damage. Navarro-Cano and colleagues
[25] measured RA disease severity using an RA component of the Duke Severity of Illness
Checklist through a physician's assessment. A measure of RA functional status based
on American College of Rheumatology standards [26] involves clinical assessment (for example, physical examination), laboratory tests
(for example, ESR), and imaging procedures (for example, X-rays and magnetic resonance
imaging), which may be the most accurate measure of RA severity. Unlike those measures,
the RARBIS does not require a new physician's assessment and the information on indicators
can be routinely collected from typical medical records. Thus, the RARBIS may have
great utility for researchers using medical records.

The present study has several limitations. As we fully recognize and have noted, because
there is no standard RA severity measure, we assessed the convergent, and not the
criterion, validity of our index by comparing it with the DAS28. We assumed that the
DAS28 should be highly correlated with RA severity because disease activity is deemed
one of the core determinants of RA disease severity and the DAS28 includes patient-reported
disease status. To enhance information for the RARBIS, we obtained information on
surgical history, extra-articular manifestations, and physician's global assessment
of arthritis activity from a questionnaire at baseline in BRASS. This modification
did not change the properties of the index score in the present study.

Conclusion

We have examined the convergent validity of a medical records-based index of RA severity,
the RARBIS. It is obtained simply by forming an arithmetic sum of indicators on a
medical record. We propose it as a practical and useful tool for measuring RA severity.
Because the RARBIS does not require a physician's assessment for its calculation,
it is easily applied retrospectively to medical records. All of the data for the RARBIS
are accessible from medical records. The ease of collecting these data is greatly
enhanced by an integrated medical record that contains all laboratory, radiologic,
and surgical information. In studies that assess associations between treatments and
RA outcomes, the RARBIS will be useful to control for confounding by RA severity.
Before our index can be recommended, it would be beneficial to further evaluate it
in other populations as well as use other gold standards.