The evaluation design of this initiative was to answer the following questions:

What were the results at the clinic level in increasing goal-setting performance?

Summary of Action Plan data.

Assessment scores of clinic progress during the initiative.

What were lessons learned at the clinic level in implementing this patient self-management intervention for each component of the Chronic Care Model? For the Hispanic population?

What is the potential for spread within clinics at the end of this initiative? Likelihood of changes being institutionalized; further data collection being conducted to demonstrate the impact of goal setting on the control of the disease; and continuation and spread within the clinic?

What are the plans and likelihood of regional/national spread of what has been learned?

What were the key lessons learned relative to conducting this learning network initiative for community health centers?

1. Clinic-Level Results for Intervention

1A. Increasing Goal-Setting Rates

Self-management of a chronic condition relies primarily upon the ability of patients to change those behaviors that can improve the control of the condition. Self-management is more than patient education and includes communication between the patient and provider that promotes interactive goal setting and problem solving. Through the goal-setting process the patient can often develop the self-confidence needed to achieve the selected goal.

Improving the goal-setting rate in patients with diabetes was the key measure of this intervention. Table 1 summarizes characteristics of the six clinics, differences in their implementation strategies, and their progress with the intervention.

No clinician was involved in goal setting and no one-on-one interactive goal setting occurred.

Not available (Clinic has since implemented electronic medical record [EMR]; plans to evaluate impact of goal setting in diabetes school.)

Not available

Clinic used Action Plan for goal setting only in diabetes school. Trainer was not able to provide one-on-one interactive goal setting. Instead, patients used form to select goals while in class. Copies of Action Plans were not placed in medical records to allow followup by clinicians.

Colorado Region

Clinic D

Yes

Yes

Diabetic patients of one physician assistant (PA)

PA and quality coordinator

PA, whose performance on many indicators exceeded that of other clinicians in clinic. PA achieved patient continuity and goal-setting rate of 76%. PA worked closely with medical assistant for intervention.

No significant change in rate (75%)

No change in 8.1% level

Other clinicians participated in on-site patient self-management training to facilitate spread. Willing to try interactive goal setting after training because they learned that process did not have to be time consuming.

Clinic E

Yes

Yes

All patients in clinic with diabetes

Quality improvement coordinator and medical director

Mid-level practitioners and physicians

Increased from 25.9% to 49%.

Decreased from 8.2% to 8.0%.

Did not implement Action Plan because similar tool was unsuccessful. Instead, redesigned processes to facilitate interactive goal setting by clinic team with clinicians providing one-on-one support. Also, on-site training was provided to clinicians on patient self-management and goal setting.

Clinic began implementation of EMR during latter part of intervention.

Clinic F

No

No

All patients in clinic with diabetes

Clinic director only participant

Volunteer physicians (clinic heavily dependent on volunteers)

Not available

Not available

Unable to implement Action Plan or any goal-setting strategy during intervention period.

Reviewing the data for Clinics D and E from July 2005 through January 2007 provided additional information. As shown for Clinic E in Table 2, the number of patients with diabetes in the registry has remained constant, but the goal-setting rate was higher a year before the intervention started, and after the intervention it is still almost 10 percentage points lower than in July 2005. However, the HbA1c has steadily declined from 8.3 to 8.0, raising concerns that the goal-setting rate had little to do with the outcome results or control of diabetes for this clinic. This same finding has been noted for other clinics with a registry, and one potential explanation offered by clinics not participating in this intervention is that more goal setting occurs than is documented in the registry. They doubt that the goal-setting rate truly falls from 60 percent or higher to 25 percent or lower in a year, but instead believe that clinics document goal setting better when they are participating in an intervention focusing on patient self-management.

Clinic D presents a different picture. For the same time period, 72 patients with diabetes have been added to the registry for a 52 percent increase. The percentage of these patients with a goal has gradually declined about 9 percentage points, which is understandable given the increase in patients. However, the HbA1c remained the same over the 19 months.

Table 2. Additional HbA1c and goal-setting data for Clinics D and E

Date

DM Patients in Registry

HbA1c

No. with Goals

% With Goals

Clinic D

July 2005

136

8.1

88

64.7

July 2006

193

8.1

118

61.0

January 2007

208

8.1

116

55.8

Clinic E

July 2005

356

8.3

209

58.7

July 2006

351

8.2

91

25.9

January 2007

359

8.0

176

49.0

Overall, clinics with strong team participation in the learning network were more successful. Having a strong clinic champion improved the likelihood for success. Furthermore, having a medical director as the clinic champion may be the best scenario for intervention support. Following are additional findings from the intervention:

Clinics with a diabetes educator (even part time) completed the greatest number of Action Plans for patients with diabetes during the intervention.

Clinics without registries are at a disadvantage in collecting data needed to assess progress with quality improvement interventions.

Clinics may use the same chronic disease registry but vary in the amount of data they enter into the system and use routinely for performance improvement.

A clinic can redesign its clinic processes and increase the goal-setting rate similar to a clinic using a tool such as the Action Plan. Clinic team acceptance of a tool is important for success. Implementing a paper tool in any intervention may meet more resistance as clinics move to electronic medical records (EMRs).

A high-performing clinician with a goal-setting rate already in the highest quartile may not benefit significantly from a tool such as the Action Plan.

An improvement in the process and outcome measures of an intervention may not provide evidence that the intervention was effective in achieving the outcome. Ongoing monitoring and analysis of clinic data in the context of the environment are necessary to know when the intervention is the primary reason for the outcome results. Patient goal setting is a common measure for community clinics, but few clinics have designed a methodology to determine the effectiveness of their goal setting in changing patient behaviors.

1B. Analysis of Action Plan and Activation Tool Data

Action Plan. During the intervention, 211 Action Plans were completed, with more than 96 percent by Hispanic patients. The three most frequently chosen goals are shown in Table 3.

Table 3. Most frequently identified Action Plan goals

Type of Goal

Number of Action Plans
With Goal Identified

% of Action Plans

Diet

89

42.2

Exercise

74

35.1

Medication

24

11.3

Total

187

88.6

Differences in patient goals by clinic. The diabetes educator in Clinic B with 91 Action Plans had 51 patients or 56 percent choosing diet as their goal. Other clinics varied from 22 percent to 42 percent for diet. Additional findings included:

Clinic C did not provide one-on-one interactive goal setting and had patients in diabetes school complete the Action Plan without assistance. Table 4 highlights the difference in rates for goals chosen in Clinic C compared with those for all clinics.

Clinicians in Clinic C referred newly diagnosed patients with diabetes and those having the most difficulty with control to the diabetes school classes. Without linking the Action Plans with the medical records, it was not possible to determine if this group was significantly different from the patients targeted in the other clinics. The higher rate for medication goal setting for this group raises the concern that patients may be more likely to acknowledge lack of medication adherence when completing the Action Plan alone than when completing with a clinic team member.

Comfort levels chosen by patients indicated confidence in achieving their selected goal. If a form had a goal specified, but no comfort rating was provided, this form was dropped from calculations on comfort level. Every clinic had a few Action Plans without a comfort level noted.

However, an outlier was one clinic with 40 percent of Action Plans having no comfort-level rating.

For the four clinics implementing the Action Plan, the percentage of forms with a "10" rating ranged from 44 percent to 70 percent with the latter in Clinic C without a one-on-one goal-setting process. The lowest was in the clinic where comfort level was not rated in 40 percent of the Action Plans.

No Action Plan had a rating less than 5; the second most frequent rating was 5, with the third and fourth being ratings of 7 or 8.

Activation Assessment Tool. All clinics agreed from the beginning of the intervention that although this tool had been validated in Spanish, it was above the literacy level of their clinic population. However, a greater concern was that the concepts contained within the survey were not familiar to their Hispanic population. One clinician explained that Hispanics tend to be emotional and rate their feelings and well-being at the extreme poles of any rating scale and rarely select the middle ratings.

The two Spanish Catholic clinics continued to use the Activation Tool for 3 months to give the instrument a trial.

At Clinic A, 26 Activation Assessment Tools were completed as follows: 8 by the diabetes educator, 13 by the nurse practitioner, and 5 by the medical director. Table 5 provides the percentage of total responses (4 x 26 = 104) for each rating on the 10-point scale under the four areas of patient confidence.

Table 5. Clinic A: Percentage of total responses for each scale rating

1-2

3-7

8

9

10

0

7.6%

10.5%

14.4%

67.3%

At Clinic B, 74 forms were completed primarily with the assistance of the diabetes educator. Table 6 provides the percentage of total responses (4 x 74 = 296) for each rating on the 10-point scale under the four areas of patient confidence.

Table 6. Clinic B: Percentage of total responses for each scale rating

1-4

5

6

7

8

9

10

4%

12.8%

1%

8.1%

6.4%

6.7%

60%

In addition, of the total responses for Clinic B:

51.4 percent answered 10 under all four areas of patient confidence.

64.8 percent rated all four areas of patient confidence with the same number.

81.2 percent rated all four areas of patient confidence between 7 and 10.

For Clinics A and B, Latinos were more likely to choose ratings demonstrating a high confidence level in managing their own care and understanding their treatment.

One drawback in having six clinics with such varying characteristics and intervention designs is that the findings may be interesting for consideration and further study, but may not be applicable to other clinics. For example:

The patients in a diabetes school that completed the Action Plan themselves without one-on-one interactive goal setting were more likely to choose medication for their goal than patients setting a goal with a team member. They were also more likely to rate their comfort with reaching their goal at a "10" (70 percent).

Having a diabetes educator in the clinic for clinicians to refer patients for goal setting might result in more patients having a goal.

The discipline of the team member performing interactive goal setting might influence the type of goal selected by the patient.

Hispanic patients' responses for the Action Plan on a scale from 0 to 10 relating their comfort or the likelihood of reaching their goal were predominately from 7 to 10. A concern might be that the patients were expressing confidence to please the clinic team.

Delmarva translated into Spanish, produced as triplicate forms, and delivered to clinics.

Most clinics felt Action Plan facilitated change process and decreased paperwork (copy for patient was important for motivation; copy to record for followup; copy to Delmarva for data collection).

Clinics would like Action Plan modified for use with all chronic diseases and obesity.

Some felt Action Plan would be easier to use with section to note progress on followup visits. Others preferred completing new Action Plan for each visit and sending copy home with patient each time.

Starting with small steps in goal setting avoids setting patient up for failure.

2. Use Activation Assessment Tool to assess patient's comfort level in making behavioral changes. Knowing this level helps provider understand where to begin in goal-setting process and in education.

Tool validated in Spanish by Stanford University and available to use without charge.

Delmarva made copies and provided them to clinics.

All clinics agreed tool was above reading level of their population. Low literacy level was considered for any materials produced. Simple drawings are effective. It was also felt that concepts embedded in tool were not understood by their Hispanic populations.

Suggestion made that AHRQ promote research to determine if using graduated tool to assess comfort of Latinos produces same distribution of responses as with other cultures.

Additional suggestion made that before assessing self-activation level for goal, provider should assess importance of goal to patient during goal-setting process.

3. Make clinic team aware of potential benefit of intervention and assure they have skills for patient self-management.

On-site training offered to clinics on patient self-management.

Texas Association of Community Health Centers video used during training.

Clinics found training helpful in convincing clinicians that goal setting did not have to be time consuming. Video felt to be good example of provider/patient interactive goal setting.

Medical assistants may not feel empowered to help with goal setting. May require training and regular meetings for discussions and support.

4. Identify clinic team members involved in intervention and define their roles and responsibilities to maximize resources and patient outcomes.

In one clinic medical assistant who routinely worked with one provider was involved in helping with goal setting.

In two clinics patients referred to DEd that divided her time between clinics for help with education and goal setting. This team member also held diabetes classes on Saturday.

Another clinic had diabetes school for educating their patients and used these classes for goal-setting process due to time constraints on clinicians.

Regardless of type of team approach used, goal-setting process should be documented in patient record and involve clinician in followup and support of patients' goal-setting efforts.

Every encounter with patients should be utilized to discuss challenges and progress with their goals.

If separate classes for diabetes education are used for goal setting, following must be considered:

One-on-one time needed for interactive goal setting.

Documentation of goal chosen must be incorporated into patient's record for clinician followup at subsequent visits.

Culture affects diet and lifestyle.

Diet is very important to Latinos; helping them prepare their favorite food in healthier way is important.

Many Latinos have shown interest in how other cultures prepare food and have been open to learning these methods.

Walking is easily accessible exercise activity, but Latinos do not routinely swim, play tennis, golf, or go to gyms for exercise. Dancing is also well received for exercise.

5. Followup for progress on goals should not wait until next scheduled visit but occur within 2 weeks.

Contacting patient by telephone may be difficult during day. After work hours is better.

Patient cell phone has proven to be better means of making telephone contact.

6. Invite family to participate in patient care.

Family is important to Hispanics. Invite family members to classes, specifically one that lives with patient. If patient is male and female in home is responsible for food preparation, it is important for her to attend class and clinic to learn about diet and how to prepare healthier foods.

Active recruitment of family members when care is not going well has been successful in two clinics.

Creating trust and respect in clinic population is critical and can often trump other efforts to help patients.

If clinic is not orderly and high tech in appearance, it is probably more comfortable. If clinic feels like part of Latino community, patients are more relaxed.

8. Patient and all members of provider team must accept responsibility for care and outcomes.

Lesson taught by DEd is that several unsuccessful attempts to make telephone contact do not mean fulfillment of responsibility to patient. Try again and/or use other methods. DEd tries again at different hour and after work or on Saturday, calls cell phone, and calls emergency number given. She has a high success rate for telephone contacts because she feels responsible for making followup contacts.

9. Debunk myth in Latino population that insulin can worsen symptoms and complications of diabetes.

Provide good explanation of how lack of control of disease produces complications that many relate to use of insulin. Explain that delaying insulin when diet, exercise, and medication do not succeed in controlling glucose can allow complications. Be careful not to imply that if patients adhere to care plan they will never have to use insulin. If they have the disease long enough, need for insulin is a real possibility.

Table 8. Lessons learned in decision support

Decision Support

Tools/Steps

Resources

Tips/Lessons Learned

Determine any changes that could support better decision making in clinic.

1. Develop standing orders for medical assistants to use in ordering blood tests, procedures, and referrals for patients with diabetes.