A Randomized Trial of Practice-Based Education to Improve Delivery Systems for Anticipatory Guidance

Author Affiliations: From the Robert Wood Johnson Clinical Scholars Program (Drs Rosenthal and Miller), Division of General Pediatrics and Adolescent Medicine (Drs Rosenthal, Lannon and Margolis), Center for Children's Healthcare Improvement (Drs Lannon and Margolis and Mss Stuart and Brown), and Department of Epidemiology, School of Public Health (Drs Miller and Margolis), University of North Carolina at Chapel Hill. Dr Rosenthal is now with the Robert Wood Johnson Clinical Scholars Program, Yale University School of Medicine, New Haven, Conn.

BackgroundCommunication between physicians and patients is an important component of medical encounters and has been identified by the Institute of Medicine as a point of intervention to increase quality of care. In pediatric health maintenance visits, substantial recommended communication, in the form of anticipatory guidance, is not provided.

ObjectiveTo examine the effectiveness of an intervention of continuing medical education and process improvement methods to implement “office systems” to improve the delivery of anticipatory guidance for parents of young children.

MethodsWe performed a randomized controlled trial of a practice-based quality improvement intervention among 44 pediatric practices in North Carolina. In the 22 intervention practices, project staff coached practice staff in auditing performance and identifying, testing, and implementing new care processes to improve delivery of anticipatory guidance. We surveyed parents of 1-month-olds and 6-month-olds regarding their recollection of anticipatory guidance as well as parent knowledge and parent behaviors and used cluster analysis to determine the effects of the intervention.

ResultsThe proportion of parents of 1-month-olds who reported that they received all age-appropriate anticipatory guidance changed from 15.9% (95% confidence interval [CI], 8.9%-26.7%) to 10.0% (95% CI, 5.1%-18.8%) in the control practices and from 7.3% (4.1%-12.9%) to 24.0% (95% CI, 14.6%-36.9%) in the intervention practices (difference between 2 differences, P = .002). The proportion of parents of 6-month-olds who reported that they received all age-appropriate anticipatory guidance changed from 8.2% (95% CI, 3.6%-17.8%) to 5.4% (95% CI, 2.8%-10.2%) in the control practices and from 2.2% (95% CI, 0.8%-5.9%) to 18.1% (95% CI, 10.3%-29.9%) in the intervention practices (difference between 2 differences, P = .001). There were no differences in self-reported parent knowledge and parent behavior.

ConclusionAn office system intervention improved parent reports of quantity of anticipatory guidance but did not change parent knowledge or parent behavior. Future research should examine how to use systems improvement methods to improve the impact of anticipatory guidance.

Physicians who perform well-child care work in partnership with parents. During well-child visits, physicians provide anticipatory guidance by educating parents on child development, injury prevention, nutrition, and other topics that will help the parent to become a more effective caregiver. Recognizing the importance of the home environment,1 national physician organizations advocate spending a significant proportion of the well-child visit on anticipatory guidance.2

These expectations of anticipatory guidance are often unmet in practice. Parents report limited anticipatory guidance from their child’s physicians: Schuster et al3 reported that only 41% of physicians discussed sleep patterns, only 23% discussed learning encouragement, and only 25% discussed discipline during age-appropriate well-child visits. Young et al4 found that 79% of parents would like more information on child-rearing.

Physicians spend less than 2½ minutes (<13%) of the well-child visit on anticipatory guidance.5- 8 Although barriers to delivering better anticipatory guidance include limited time and lack of confidence in counseling techniques and their benefits, some have argued that the most important barrier is a lack of systems within the office (ie, reminder systems, previsit patient checklists, chart flow sheets, and practice feedback reports) to improve effectiveness and efficiency.9- 13

In light of the growing evidence that process improvement methods may facilitate the implementation of office systems for preventive care, which may, in turn, be associated with improved care,14 we designed a study to assess these methods in primary care offices. The intervention was based on the plan-do-study-act cycle of the “model for improvement,” a framework for applying process improvement methods to implement change.15- 17 The 4 intervention steps include (1) identifying the performance of preventive services, (2) identifying evidence-based changes that could improve performance, (3) testing the changes, and (4) monitoring new health care delivery processes and making adjustments.

The objective of this study was to evaluate the impact of an office-based intervention on parents’ reports of physician discussion of anticipatory guidance, parent knowledge related to common anticipatory guidance topics, and parent behavior related to common anticipatory guidance topics. We hypothesized that the intervention would increase the percentage of parents reporting prevention-related anticipatory guidance, increase parent knowledge, and improve parent behavior.

METHODS

PARTICIPANTS, RECRUITMENT, AND RANDOMIZATION

This study was part of a larger trial examining the impact of office-based assistance to primary care practices that combined continuing medical education with process improvement methods to improve the delivery of immunizations and screening. The primary results of the larger trial have been reported previously.18,19

The recruitment strategy has been described previously.19 Briefly, we identified all 453 pediatric and family practices in two regions of North Carolina located near practice assistance teams at the University of North Carolina at Chapel Hill and the Charlotte Area Health Education Center. We created our eligibility criteria to obtain a diverse population of practices. Practices meeting the following criteria were eligible for the study: sufficient newborns enrolled each month to achieve sample size requirements; not part of an academic institution or a publicly funded health center; and, to increase the proportion of children receiving Medicaid, in the region near the University of North Carolina practices needed to have annual Medicaid billing in excess of $50 000.

We used a stratified randomization procedure to select practices from those that met the eligibility criteria. We stratified by type of practice (pediatric/family practice), number of newborns enrolled each month, and annual Medicaid billing. Within each stratum, we used a computerized random number generator to assign equal numbers of practices to either an intervention group, in which practices received assistance to establish office systems for prevention, or a control group.

We used previously developed methods to recruit practices.20 The recruitment team was unaware of a practice's treatment allocation until after consent to participate was signed by all physicians in each practice. As an incentive, all practices received a copy of publicly available materials designed to facilitate preventive care,21 and the intervention group received continuing medical education credit.

A trained research assistant visited all 44 practices to determine how to recruit parents to respond to our survey. In each office, a sign was posted in the waiting area noting that the practice was participating in the “Partners in Prevention” project to improve care for children. The research assistant helped the office identify a specific staff person who identified, recruited, and obtained informed consent from eligible parents. A survey research firm, masked to the parents’ intervention status, called parents of 1-month-olds and 6-month-olds to administer the survey by telephone in 1998 (baseline) and 2000 (follow-up). The parents interviewed in 1998 were not the same people interviewed in 2000. It took, on average, 10 minutes to complete the 31-question, multiple-choice survey.

All parents were eligible if they were English speaking, were not deaf, and had a working telephone and their child was being seen for either a 1-month or 6-month well-child visit. Parents were identified on the day of their child’s well-child visit and called within 7 days. The University of North Carolina Institutional Review Board approved the study protocol.

INTERVENTIONS

In step 1 of the plan-do-study-act cycle, as a part of the larger clinical trial, practices formed an improvement team of clerical, nursing, and physician staff members to review the data on immunizations, tuberculosis, lead, and anemia from the chart abstraction audit. In step 2, the improvement team selected performance improvement goals, identified evidence-based strategies for improvement, and considered how these strategies might be adapted to their practice. The project team used minilectures, handouts from authoritative sources,2,22,23 and tools to educate the improvement team about preventive care and effective preventive service delivery strategies.24 Recommended changes included identifying an office prevention coordinator,25 creating a prompting system, developing a preventive service summary,26 establishing a tracking/recall system,27 and implementing routine monitoring of the delivery of preventive services. The tools included customized preventive services flow sheets, standardized risk assessment tools, reminder postcards, preprinted notes for prompting missed services, and structured health maintenance records.

Using the plan-do-study-act cycle, we initiated the anticipatory guidance aspect of the intervention 1 year after the start of the initial project. For step 1, the project teams audited charts and gave the practices feedback on their anticipatory guidance. In step 2, the project teams created and presented a grid of anticipatory guidance topics by age group, prioritized by evidence base. The project teams identified options for the practice improvement teams to choose; examples included (1) designing new well-child forms that incorporated and prioritized anticipatory guidance topics, (2) developing practice standards for the timing of specific topic discussions, and (3) developing a questionnaire for a parent waiting for a child’s appointment that would jog the parent’s memory of topics to discuss in the appointment.

During step 3, the project staff helped practices undertake plan-do-study-act cycles to implement change in small samples of patients. The practice improvement team identified changes to implement without knowledge of the specific content of our survey. Based on the testing, changes that improved the process of preventive care within current office routines were disseminated throughout the practice in step 4. The control group received only step 1 of the intervention, measurement and feedback. The focus of the intervention was on organizing materials and decreasing variability; there was no training in counseling techniques.

MEASURES

To assess parent reports of anticipatory guidance, knowledge, and behavior, we created a survey for parents of 1-month-olds and another for parents of 6-month-olds. We selected specific counseling topics because they address prevalent, significant health risks for infants,28,29 and an infant’s parent would thus be psychologically prepared for these discussions.30 We selected a group of topics, rather than a single topic such as smoking cessation, to identify overall practice performance rather than performance in a single area where practices might have a particular focus or interest. This approach of combining variables is a more sensitive measure, increasing the ability to detect changes. The survey was developed by investigators C.M.L., J.M.S., and L.B. and was based on the theory of reasoned action, which argues that anticipatory guidance precedes parent knowledge, which in turn precedes parent behavior change.31

For the 1-month-old survey, the anticipatory guidance questions consisted of one question on whether the parent recalled a discussion about each of the following topics: smoke detectors, owning a car seat, using a car seat, cigarette smoke exposure, and hot water temperature (eg, “At this visit, did your doctor [or the nurse] ask whether you had a smoke detector installed at home?”). The parent knowledge questions consisted of one question on each of the following topics: temperature for hot water heater, health problems with smoke exposure, and ways to prevent harm in a fire (eg, “Have you heard of any health problems that babies may have as a result of being around cigarette smoke?”). The parent behavior regarding prevention questions consisted of one question each on measuring hot water temperature and having a car seat, two questions on smoke detectors, and three questions on exposing the baby to cigarette smoke (eg, “Do you have a working smoke detector installed in your home?”). We asked two questions about car seat use to differentiate parents who did not use a car seat because they did not own one from parents who owned a car seat and might choose not to use it.

For the 6-month old survey, the anticipatory guidance questions consisted of one question on whether the parent recalled a discussion about each of the following topics: owning a car seat, using a car seat, cigarette smoke exposure, and using well water or city water. The parent knowledge questions consisted of one question on each of the following topics: smoke exposure and car safety. The parent behavior questions consisted of two questions about car safety and three questions about cigarette smoke exposure. All questions were answered yes/no/refused/don’t know.

OUTCOMES

The primary outcomes were the changes over time in the proportions of parents reporting receipt of 4 of the 5 anticipatory guidance items for 1-month-olds and receipt of 4 of the 4 anticipatory guidance items for 6-month-olds. The primary comparison was whether these changes differed between the intervention and control practices. We had originally planned to analyze how many parents reported receipt of all of the anticipatory guidance items, but because so few parents of 1-month-olds reported that they received all 5 anticipatory guidance items (3%), we chose to analyze how many received 4 of 5. Twenty-eight percent of the parents of 6-month-olds received 4 of 4 anticipatory guidance items, so we analyzed on that basis.

The secondary outcomes were change over time in the proportion of parents with improved knowledge and change over time in reports of parental behaviors.

STATISTICAL ANALYSIS

We conducted an intention-to-treat analysis in which all intervention and control practices were included. The estimated power of the study to detect a difference of 20% between intervention and control practices was 80%, with a type I error of .05 (2 tailed), using methods that accounted for within-practice clustering of the study data.32

In all analyses we used logistic regression with generalized estimating equation techniques and an exchangeable correlation matrix to account for within-practice clustering. For each outcome, we included terms for intervention status (interaction or control) and time (baseline or follow-up) and an interaction term between these two variables. We used an interaction term to provide a test of the effect of the intervention over time and to account for potential differences at baseline. We also used multiple logistic regression with generalized estimating equations to control for family demographics that might influence communication (number of children, maternal education, maternal age, and race/ethnicity). We present the results as adjusted proportions rather than odds ratios for clarity of interpretation. As the crude and adjusted proportions were so similar, we present only the adjusted proportions. We conducted data analyses using Stata 8.0 (Stata Corp, College Station, Tex).

RESULTS

OFFICE BASELINE CHARACTERISTICS

Of the practices screened for eligibility, 88 met the inclusion criteria (Figure 1). After contact for recruitment, 24 were found ineligible. Of the 59 practices recruited, 49 (83%) agreed to be randomized. In the intervention group, one did not participate in the intervention and three dropped out after they went bankrupt. In the control group, one practice went bankrupt during the study and dropped out. Randomization produced intervention and control practices with comparable baseline practice characteristics (Table 1). Control practices were twice as likely to be physician owned.

As reported previously,19 all 22 intervention practices developed improvement teams. The project teams met with the improvement teams a median of 8.5 times (range, 5-14 times) over an average of 24 months. Implementation among the intervention practices included the following: 18 (82%) implemented preventive services summaries in patient charts, 17 (77%) used tools to support risk assessments, 15 (68%) used clinician prompting, and 7 (32%) revised health maintenance records for well-child visits.

PARENT BASELINE CHARACTERISTICS

Prior to the intervention we surveyed 416 parents from the intervention offices (312 parents of 1-month-olds and 104 parents of 6-month-olds) and 326 parents from the control offices (284 parents of 1-month-olds and 42 parents of 6-month-olds). After the intervention we surveyed 357 parents from the intervention offices (182 parents of 1-month-olds and 175 parents of 6-month-olds) and 314 parents from the control offices (154 parents of 1-month-olds and 160 parents of 6-month-olds). Our response rate was 90% for baseline surveys and 78% for follow-up surveys (Figure 1).

When we compared the control parents with the intervention parents, the baseline characteristics of the 1-month-olds and the 6-month-olds were similar (Table 2 and Table 3).

The adjusted proportion of families of 1-month-olds who received 4 of the 5 age-appropriate anticipatory guidance items changed from 15.9% (95% confidence interval [CI], 8.9%-26.7%) to 10.0% (95% CI, 5.1%-18.8%) in the control practices and from 7.3% (95% CI, 4.1%-12.9%) to 24.0% (95% CI, 14.6%-36.9%) in the intervention practices (difference between two differences, P = .002) (Figure 2). Parents in the intervention practices (Table 4) reported a significantly greater adjusted change over time for anticipatory guidance about smoke detectors (from 24.2% [95% CI, 15.0%-36.7%] to 23.4% [95% CI, 15.5%-33.6%] in the control practices and from 15.6% [95% CI, 8.6%-26.5%] to 40.9% [95% CI, 22.3%-62.4%] in the intervention practices; difference between two differences, P = .046) and for anticipatory guidance about cigarette smoke exposure (from 36.4% [95% CI, 26.9%-47.1%] to 32.0% [95% CI, 20.5%-46.2%] in the control practices and from 34.1% [95% CI, 24.7%-45.0%] to 59.9% [95% CI, 43.3%-74.5%] in the intervention practices; difference between two differences, P = .02). There was no difference between the control and intervention practices in change over time for anticipatory guidance about hot water temperature, owning a car seat, or using a car seat.

Percentages of Parents of 1-Month-Olds and 6-Month-Olds Who Reported That Their Physician Discussed Topics of Anticipatory Guidance*

The adjusted proportion of families of 6-month-olds who received all of the age-appropriate anticipatory guidance changed from 8.2% (95% CI, 3.6%-17.8%) to 5.4% (95% CI, 2.8%-10.2%) in the control practices and from 2.2% (95% CI, 0.8%-5.9%) to 18.1% (95% CI, 10.3%-29.9%) in the intervention practices (difference between two differences, P = .001) (Figure 3). Parents in the intervention practices (Table 4) reported a significantly greater adjusted change over time for anticipatory guidance about owning a car seat (from 51.5% [95% CI, 32.3%-70.2%] to 31.6% [95% CI, 19.2%-47.5%] in the control practices and from 21.3% [95% CI, 12.2%-34.6%] to 43.7% [95% CI, 30.0%-58.5%] in the intervention practices; difference between two differences, P = .02) and for anticipatory guidance about cigarette smoke exposure (from 25.8% [95% CI, 10.7%-50.0%] to 22.2% [95% CI, 14.0%-33.3%] in the control practices and from 14.9% [95% CI, 5.5%-34.8%] to 58.6% [95% CI, 41.8%-73.6%] in the intervention practices; difference between two differences, P = .006). There was no difference between the control and intervention practices in change over time for anticipatory guidance about using well water or using a car seat.

Change in percentage of parents of 6-month-olds who reported discussion of 4 of 4 age-appropriate anticipatory guidance guidelines.

Parent Reports of Knowledge

The control and intervention parents did not differ in the adjusted change in the proportion of parents reporting parent knowledge about topics related to the 1-month-olds (from 13.7% to 8.6% in the control practices and from 14.1% to 11.0% in the intervention practices) or topics related to the 6-month-olds (from 38.1% to 61.3% in the control practices and from 42.3% to 62.9% in the intervention practices).

Parent Reports of Preventive Health Behaviors

The control and intervention parents did not differ in the adjusted change in the proportion of parents reporting parent behavior regarding topics related to the 1-month-olds (from 4.4% to 5.3% in the control practices and from 3.8% to 3.8% in the intervention practices) or topics related to the 6-month-olds (from 76.2% to 85.0% in the control practices and from 79.8% to 90.9% in the intervention practices).

COMMENT

In this randomized clinical trial, we demonstrated that a practice-based quality improvement intervention in physicians’ offices that led to the integration of process improvements in clinical practice improved parental reports of anticipatory guidance. In this trial, however, we did not find improved parental knowledge or improved self-reported parenting behavior.

This project improved anticipatory guidance; however, the majority of parents reported that they did not receive all of the recommended counseling. This reflects the findings of national studies in which parents have reported that they did not receive all of the recommended counseling4,33- 35; these findings may argue for the importance of this work. Moreover, the small magnitude of the improvement in anticipatory guidance was related to some extent to the strict criteria we used to analyze anticipatory guidance (4 of 5 topics for the 1-month-olds and 4 of 4 topics for the 6-month-olds); the magnitude of the improvement in individual counseling activities was larger.

We hypothesize that the increased anticipatory guidance without a detectable change in parental knowledge or behavior may have occurred for a number of reasons. First, we started the office-based intervention in 1998 and did not fully integrate the counseling aspect of the intervention for a year. Because of this delay, some intervention practices had less time to integrate the anticipatory guidance component and also had fewer interactions with the project team than originally planned. Had all 22 of the intervention practices received a full 2 years of counseling intervention, then we might have seen greater differences in the knowledge and behavior outcomes. In addition, because each office team determined which counseling interventions and anticipatory guidance topics it would emphasize, the specificity and number of topics addressed varied among practices. Second, our intervention did not include training in counseling techniques, which may be necessary to effect behavior change. Third, although parents may have the best intentions, parents with a low income may have difficulty influencing their children’s environment regarding those things controlled by a landlord (ie, smoke detectors, hot water heaters). The theory of reasoned action proposes that communication precedes knowledge, which in turn precedes behavior change.31 The growing evidence that recommendations from a child’s pediatric provider alter parental behavior4,36,37 may mean that improved physician discussion is a step along the pathway toward behavior change. Our intervention may be an important first but not sufficient step toward behavior change in parents.

Continuing medical education focusing on delivery of care and office systems can improve primary care,14 yet the results of interventions to help practices implement such systems have been mixed. Prior to beginning our intervention, we examined the reasons cited for the success or limited impact of other office system intervention studies and attempted to incorporate them into our intervention. For example, we noted that measurement and feedback for the office practices are essential first steps.38,39 We found that the limited impact of some office-based interventions is associated with an improvement model that focuses on planning and not change, underdeveloped content materials, inexperienced improvement team leaders, and lack of time for improvement activities.40 We found that behavioral outcomes may be especially difficult to assess, as the full impact may occur after the study ends.41 We thus encouraged practices to try small-scale changes, provided materials, gave technical support to the improvement leaders throughout the implementation of change, and chose outcomes that were both proximal (anticipatory guidance) and distal (behavior change). Our partial success may be due, at least in part, to the creation of office-based improvement teams: we gave some practices, for the first time, a forum for discussion of counseling and prevention services.

LIMITATIONS

Our data are potentially limited because they are based on parental reports. First, a parent report of a physician discussion is a subjective description of an event and may be altered by the parent’s biases. However, how a parent perceives or remembers a physician discussion may ultimately be more important than how a truly objective measurement, ie, an audiotape, would record it. Second, parents may attempt to give socially desirable answers. Since the intervention and control parents were all told that they were involved in a study and were masked as to which arm, this bias should have been nondifferential and therefore may have diminished our observed effects. Third, all 44 practices were audited and received feedback on their preventive practices. This may have inadvertently improved anticipatory guidance among control practices and thus affected our findings.

Generalizability may be limited, in part, because of the practices that participated and the parents we surveyed. All 44 practices volunteered to participate in a quality improvement intervention, and this could limit generalizability because both the intervention and control practices may have been more motivated to improve than average practices.42 We surveyed only parents who reported that they had access to a telephone. Families of low economic means may be less likely to have a telephone, and we may therefore be unable to generalize our results to them.

The parents we interviewed before the intervention were different individuals than those interviewed after the intervention. Although this may be construed as a limitation, since an individual parent is likely to increase the amount of anticipatory guidance received and knowledge accrued over time, interviewing the same set of parents would not be useful. In our study, all parents categorized as parents of 1-month-olds, at both baseline and follow-up, were interviewed while they had a 1-month-old child.

CONCLUSIONS

Quality improvement interventions within physicians’ offices may be able to meet many of the current challenges of our health care system. Parents want more information and more targeted information at their young child’s well-child visit,3,4,35,43 the Institute of Medicine recently proposed that the federal government “reward providers who achieve higher levels of quality,”44 and an American Academy of Pediatrics special article recently called for pediatricians to become “more involved in implementing quality-improvement methods into their practices.”45

Our study demonstrates that parents perceived increased communication when we implemented an intervention targeted to improve office systems to achieve better results for prevention. Better communication may be construed as a step in the process toward the ultimate goal of improved parental behavior. We hypothesize that the system improvements provided the physicians with a structured approach for discussing prevention and improving the quantity of anticipatory guidance. Future studies should delineate which aspects of the intervention are most important for improvement and should determine how to create an intervention robust enough to improve parents’ knowledge and behavior as well.

Funding/Support: This study was supported by grant RO1-HS08509 from the Agency for Healthcare Research and Quality, Rockville, Md; by grant RO1-HS08509 from the Maternal and Child Health Bureau, Rockville, Md; by the North Carolina Division of Medical Assistance, Raleigh; by the North Carolina Area Health Education Centers Program, Chapel Hill; by the Robert Wood Johnson Foundation Generalist Faculty Scholars Program, Princeton, NJ; by grant 5-T32-HP-14001-14 from the Health Resources and Service Administration, Rockville, Md; and by the Robert Wood Johnson Clinical Scholars Program, University of North Carolina at Chapel Hill.

Acknowledgment: The trial was a collaboration involving the University of North Carolina at Chapel Hill; the North Carolina Division of Medical Assistance, Raleigh; the North Carolina Office of Research, Demonstrations, and Rural Health Development, Raleigh; the Carolinas Medical Center, Charlotte, NC; and the North Carolina Area Health Education Centers Program, Chapel Hill. We recognize the leadership of the North Carolina Office of Research, Demonstrations, and Rural Health Development; the North Carolina Area Health Education Centers; and the North Carolina Division of Medical Assistance, without whose vision and persistence this study would not have been possible. We thank the practice assistance teams at the Charlotte Area Health Education Center and the North Carolina Office of Research, Demonstrations, and Rural Health Development, who provided assistance to the participating practices. We also thank the staff of The Center for Children's Healthcare Improvement at the University of North Carolina at Chapel Hill and the faculty, fellows, and staff of the Robert Wood Johnson Clinical Scholars Program at Chapel Hill. Most importantly, we recognize the dedication of the primary care practices, without whose participation this study would not have been possible.