Abstract

Background

Primary care addresses obesity through physician oversight of intensive lifestyle interventions or referral to external programs with demonstrated efficacy. However, limited information exists on community program reach, effectiveness, and costs across different groups of participants.

Objective

To evaluate a scalable, community weight loss program using reach, effectiveness, and cost metrics.

Main Measures

Key Results

A total of 40,308 adults (79% women; 73% white; BMI = 32.3 ± 7.44, age = 43.9 ± 13.1 years) enrolled in WAW. Women were more likely than men to enroll in the program and continue engagement beyond an initial weigh-in (57% vs. 53%). Based on census data, African Americans were over-represented in the sample. Among participants who engaged in the program beyond an initial weigh-in (n = 19,029), 47% and 34% of participants lost 3% and 5% of their initial body weight, respectively. The average duration for those who achieved 5% weight loss was 1.7 ± 1.3 years. African American participants were more likely to achieve 5% weight loss and remain enrolled in the program longer compared to non-African American participants (2.0 ± 1.3 vs. 1.6 ± 1.2 years). Implementation costs were $2,822,698. Cost per clinically meaningful weight loss for African Americans ($257.97/3% loss; $335.96/5% loss) was lower than that for Hispanics ($318.62; $431.10) and Caucasians ($313.65; $441.87), due to the higher success rate of that subgroup of participants.

Conclusions

Weigh and Win is a scalable technology-supported and community-based weight loss program that reaches a large number of participants and may contribute to reducing health disparities.

KEY WORDS

behavioral weight Loss public health impact incentives cost

INTRODUCTION

Approximately 64% of primary care patients are overweight or obese,1,2 and two-thirds of those patients would like to discuss weight loss with their physician.3 Lifestyle weight loss approaches, overseen by physicians, are recommended for overweight patients with comorbidities and for obese patients,4 with the goal of facilitating clinically meaningful weight loss (i.e., 3–5% of initial body weight). This proportional reduction in body weight reduces triglycerides, blood glucose, and hemoglobin A1c levels, and is associated with reduced risk of cardiovascular disease, morbidity, and mortality.4 However, physicians often report insufficient time and resources as well as uncertainty related to facilitating these weight loss interventions. Therefore, guidelines recommend that healthcare systems support the referral of these patients to efficacious electronic or commercial weight loss programs delivered in community or clinical settings.4–8

The evidence underlying these referral guidelines is primarily from efficacy trials that emphasize internal rather than external validity. However, representativeness is rarely reported in efficacy trials and limits an interpretation of external validity.9 Similarly, and fundamental to clinical and community decision making, program implementation costs are rarely reported.9 The lack of information on the representativeness of participants and costs makes it difficult for physicians and healthcare systems to compare across weight loss intervention options.

In contrast to interventions tested in efficacy studies, publicly available interventions that promote physical activity, healthful eating, and behavioral strategies typically have broader inclusion criteria (e.g., no requirements for study run-in period or exclusion of participants with comorbidities), use program data for evaluation rather than intensive research assessments, and encourage potential participants to try out the program to determine whether it is a good fit for their lifestyle. These programs are appealing because they can reach a large population—and theoretically at a low cost. Unfortunately, little is known about community-based interventions that do not depend on interactions with healthcare providers to encourage weight loss, and few studies combine metrics of representativeness, effectiveness, and cost in a way that could be used to compare outcomes across programs and allow for informed decisions related to program uptake for health organizations or clinical referral for primary care physicians.10,11

The primary objective of this study was to report on the outcomes of a scalable, community-based weight loss program using reach, effectiveness, and cost metrics important for healthcare systems interested in referring patients to, and underwriting, interventions for community implementation. Outcomes were operationalized to include the proportion of participants who (a) lost weight (b) achieved a 3% weight loss, and (c) achieved a 5% weight loss. Program implementation costs were examined as cost per participant as well as cost per participant who achieved a 3% or 5% weight loss for those enrolled between 2011 and 2014. Finally, an exploratory aim was to determine differences in the representativeness of participants to the regional population and robustness of effects and costs per participant achieving a 3% or 5% weight loss within populations that typically experience disparities in obesity and weight-related chronic conditions based on racial and ethnic status.

METHODS

This study was a longitudinal pre–post, quasi-experimental study designed to evaluate the reach, effectiveness, and costs associated with Weigh and Win (WAW). WAW is a technology- and community-based weight loss program supported by Kaiser Permanente Colorado as a community benefit with the goal of providing a scalable, accessible evidence-based program that community residents could participate in for free. This study was part of an ongoing integrated research–practice partnership intended to improve weight loss interventions through externally validated research methods, while also providing practical information for program development and impact in worksite and community settings.12–20 The goal of integrated research–practice partnerships is to concurrently work towards moving the scientific field forward through ecologically valid research and improving practice through more rigorous evaluation. This partnership included scientific partners with expertise in translational obesity research and practice partners from incentaHEALTH™—the WAW delivery organization—and from Kaiser Permanente Colorado Community Benefit and Relations. The extant de-identified data and invoices for program delivery were provided to the research partners, who conducted all analyses independently of practice partners. The RE-AIM framework21 was used to guide assessment based on the individual-level indicators (reach, effectiveness, maintenance) and implementation costs. All study procedures were approved by the Virginia Tech institutional review board.

Intervention

The 12-month WAW program includes a website, objective weight assessment, daily social cognitive theory-based email and text message support, online access to health coaches, and modest financial incentives intended to increase program reach and retention.19 Incentives were provided based on a minimum of 5% of initial body weight loss ($15 per quarter) and up to 30% reduction ($150 per quarter). The intervention encouraged increased intake of fruit, vegetables, lean protein, and complex carbohydrates while monitoring portion sizes. Physical activity recommendations were graduated from walking to combined strength and cardiovascular training, with approximately 30 min of physical activity 5 to 6 days a week, beginning with a recommendation of 10 min of low-intensity activity 4 days per week. Community-based kiosks (n = 83) with a scale and a camera to document the weight loss process were set up in community centers, local retailers, libraries, and other settings easily accessible to community residents (defined as locations that were open to the public and included hours beyond 9:00 to 5:00). Participants were encouraged to weigh in at least once each quarter.

Participants

Participants from the initial delivery year (2011) through 2014 were included in the evaluation (Fig. 1). Participants were recruited through a combination of advertising at the kiosk site, opportunistic media within the communities (e.g., newspaper and television coverage), and online promotion. Of note, the top response from participants regarding where they had heard about the program was that they had learned about it when walking by a kiosk. Participants could enroll by visiting the program website, completing a registration which included self-reported age, gender, race, and ethnicity, and then completing an initial weigh-in at a program kiosk. Alternatively, participants could initiate enrollment at a program kiosk and then complete enrollment by filling in the registration at the program website. De-identified demographic, program engagement, and weight data were used for analyses. Participant records were excluded if the participant was <18 years old, or if they did not complete the enrollment process by weighing in at a kiosk. Because of the general benefits of regular physical activity and healthful eating, regardless of weight status, participation in WAW was not restricted to overweight and obese individuals; however, participants with a BMI < 25 were not eligible for weight loss incentives. Therefore, we report the overall program reach including all adult participants, but limit study outcome analyses to participants with a BMI ≥ 25.

Measures

Program reach was operationalized as the number of adult participants and representativeness of the sample compared to census gender, race, and ethnicity data from the counties that included a WAW kiosk. Enrollment duration was measured as the length of time between participants’ initial weigh-in and most recent weigh-in. Weight (kg) was measured objectively using a calibrated medical-grade 600-lb-capacity Tanita WB-110A scale (Class III NTEP certified) at each kiosk. Quarterly weight was defined as the weigh-in recorded closest to the quarter-end date relative to the initial weigh-in date, and was recorded as missing when no weigh-ins were completed in a given quarter. Percentage weight loss was calculated using initial body weight. Once a participant had achieved a clinically meaningful weight loss (i.e., 3% or 5%), maintenance was defined as maintaining that magnitude of weight loss for 6 or 12 months. However, participants were not eligible for incentives beyond the initial 12 months of participation, though they could continue to weigh-in, visit the website, and receive email and text message support. Weight data through 2015 were collected to allow for 12-month data on participants who enrolled at the end of 2014 and to capture additional maintenance data from those who enrolled in 2013 or earlier.

Costs

Costs were gathered from all program invoices that occurred between 2011 and 2014 related to program implementation. The invoiced costs were grouped into technical systems support, program delivery personnel, kiosk leasing, marketing personnel and activities, and program incentives.

Analyses

Program reach was examined using odds ratios (OR; 90% CI) comparing the WAW sample to relevant census data for the region. The sample remainder was used as the denominator for calculating the odds relative to each race category in the respective sample (e.g., the odds of being African American in the WAW sample vs. the respective odds of being African American in the census data). Univariate analysis of variance (ANOVA) was used to test WAW reach sample for baseline BMI differences according to gender, race, and ethnicity. Pairwise comparisons were used to identify the source of significant differences in the case of a race effect.

Participant information was reported for the total sample as well as stratified by those who completed only one weigh-in versus those who completed more than one weigh-in. Chi-square tests (χ2) and univariate ANOVA were used to test for race, ethnicity, gender, and age differences between adult WAW enrollees who weighed-in more than once vs. those who weighed-in only once. Adjusted residuals (resadj) were examined to identify the source(s) of significant χ2, and OR (90% CI) were calculated for race categories with the greatest absolute values of resadj using the remaining sample within the respective subgroup analysis as the reference group (e.g., OR for African Americans relative to all other participants collapsed as the reference group).

Effectiveness assessments among adult participants with a BMI ≥25 kg/m2 were conducted using a baseline-value-carried-forward procedure to account for missing data. Univariate ANOVA was used to compare baseline values for weight and BMI within sample subgroups stratified by gender, weight status (i.e., overweight vs. obese participants), race, and ethnicity for those who a) weighed-in only once, b) weighed-in more than once, c) lost 3% of their weight, and d) lost 5% of their weight. A series of chi-square tests and univariate ANOVAs were used to compare proportional outcomes within the sample overall, as well as stratified by gender, weight status, race, and ethnicity. In the case of a significant chi-square resulting from a test involving a categorical variable with more than two levels (e.g., race), OR (90% CI) were used to assess effect size for the group(s) with the greatest absolute values of resadj. The remaining sample within the respective subgroup analysis was used as the reference group (e.g., OR for losing 3% body weight among African Americans was calculated using all other race groups collapsed as the reference group). Significant F-statistics were decomposed using pairwise comparisons. The probability value of p < 0.05 was adjusted to p < 0.001 using the Bonferroni method as an indicator of statistical significance to control family-wise error resulting from multiple comparisons. Effect sizes or significant test statistics were reported as η2.

The cost per participant was calculated by dividing the total implementation costs by the total number of unique participants who were overweight or obese. While healthy-weight participants are able to participate in the community program, we did not attribute costs to participants with a BMI < 25. The cost per participant who lost 3% or 5% body weight was calculated by dividing the total implementation costs by the total number of participants in each group who met the criteria. To determine total costs per subgroup, the total number of participants within that subgroup was multiplied by the cost per participant who was overweight or obese. The costs of 3% and 5% weight loss per subgroup were calculated by dividing the total cost per subgroup by the total number of participants within the subgroup who met the criteria.

RESULTS

Reach

Figure 1 displays the flow of participants for inclusion in the present analysis. A total of 40,308 adults (79% women; 73% white) enrolled in WAW between 2011 and 2014. The mean (SD) participant age and BMI were 43.9 (13.1) years and 32.3 (7.4), respectively, with the majority (58%) of participants being obese. Table 1 presents descriptive statistics for the sample of WAW enrollees ≥18 years old. Participants completed a mean (SD) of 4.9 (12.2) weigh-ins over 0.44 (0.78) years and received $10.54 (34.48) in monetary incentives.

Table 1

Mean (SD) Initial Age, Weight and BMI for All 2011–2014 Enrollees ≥18 Years Old

Histogram of participant weight loss (kg) for those who completed a second weigh-in.

The WAW sample had a significantly higher proportion of women [OR (95% CI) = 4.01 (3.91, 4.10); p < 0.001] and African Americans [OR (95% CI) = 4.78 (4.61, 4.96); p < 0.001] and a significantly lower proportion of Caucasians [OR (95% CI) = 0.42 (0.41, 0.42); p < 0.001] and Hispanic/Latinos [OR (95% CI) = 0.93 (0.90, 0.95); p < 0.001] than the census population (Table 2). A greater proportion of participants who weighed-in more than once were African American [resadj = 7.9; OR (90% CI) = 1.33 (1.24, 1.44); p < 0.001] than those who weighed-in only once, and a significantly greater proportion of those who weighed-in only once self-reported their race as “other” [resadj = −9.9; OR (90% CI) = 0.69 (0.65, 0.75); p < 0.001] than those who weighed-in more than once [χ2 (df) = 153.56 (5); p < 0.001; η2 = 0.062]. Further, a greater proportion of women weighed-in more than once than did men [χ2 (df) = 42.78 (1); p < 0.001; η2 = 0.033]. Differences in age and ethnicity distributions between those who weighed-in more than once versus those who weighed-in only once did not reach statistical significance.

Effectiveness

Table 3 presents overall descriptive statistics for the sample of participants with a BMI ≥ 25 kg/m2, as well as statistics stratified by gender and weight status. The mean (SD) weight loss was 2.1 (6.5) kg, and approximately 46% of the participants lost weight. Fifty-seven percent of participants weighed-in more than once, and 70% of these lost weight (Fig. 2). Twenty-seven percent and 19% of participants lost 3% and 5% of initial body weight, respectively. There were no gender differences in total weight lost or the proportion of participants losing 3% and 5% of body weight.

Values in bolded rows used as denominator for subsequently reported proportions within the column in the respective table section. Italicized numbers reflect the total sample in column and are used as denominator for subsequently reported italicized proportions within the column.

*Overweight vs. obese p ≤ 0.001 †Men vs. women p ≤ 0.001

Table 4

Impact Assessments by Race and Ethnic Group

Race

Ethnicity

African American

Asian

Caucasian

Native American

Other

Hispanic

Non-Hispanic

N Adults ≥25 kg/m2

2796

386

24,267

579

3034

6785

24,178

N (%) 1 weigh-in only

1037 (37)

181 (47)

10,538 (43)

269 (47)

1556 (51)

3037 (45)

10,520 (44)

M (SD) initial kg

99.8 (23.8)*

82.7 (14.5)

94.6 (20.4)

94.5 (20.6)

91.5 (20.7)

92.0 (20.9)

95.1 (20.5)‡

M (SD) initial BMI

35.5 (6.9)

30.9 (4.7)

33.5 (6.4)

34.4 (6.6)

34.2 (6.7)

34.3 (6.8)

33.6 (6.4)

N (%) ≥1 weigh-in

1759 (63)

205 (53)

13,729 (57)

310 (54)

1478 (49)

3748 (55)

13,658 (56)

M (SD) initial kg

98.4 (22.2)*

83.0 (17.0)

95.1 (21.0)

97.9 (23.7)

91.8 (21.3)

92.9 (21.3)

95.6 (21.2)

M (SD) initial BMI

35.3 (7.7)

31.2 (5.6)

33.7 (6.6)

35.7 (7.3)

34.4 (6.8)

34.6 (7.1)

33.8 (6.6)

N (%) lost weight

1182 (67.2)

150 (73.2)

9753 (71.0)

221 (71.3)

1001 (67.7)

2555 (68.2)

9692 (71.0)

M (SD) kg lost

3.5 (8.8)

2.9 (8.1)

3.3 (7.6)

3.5 (8.0)

2.8 (7.6)

3.2 (7.9)

3.3 (7.7)

N (%) 3% weight loss

909 (33)*

98 (25)

6489 (27)

141 (24)

627 (21)

1786 (26)

6444 (27)

M (SD) initial kg

99.7 (22.8)

83.2 (17.1)

95.7 (20.8)

99.2 (22.9)

94.1 (22.6)

94.2 (21.9)

96.5 (21.1)

M (SD) initial BMI

35.8 (8.3)

31.3 (6.0)

33.9 (6.5)

35.8 (7.4)

35.0 (7.1)

34.9 (7.3)

34.0 (6.6)

M (SD) kg lost

7.2 (9.9)

6.5 (7.9)

6.9 (8.7)

7.4 (9.6)

6.9 (8.8)

6.9 (8.8)

6.9 (8.8)

M (SD) years enrolled

1.9 (1.3)*

1.4 (1.2)

1.5 (1.2)

1.5 (1.2)

1.4 (1.2)

1.7 (1.3)†

1.5 (1.2)

M (SD) incentives ($)

40.2 (60.7)

29.1 (46.9)

33.9 (54.6)

33.0 (60.0)

27.8 (51.0)

35.1 (53.3)

34.0 (56.0)

N (%) maintained 6 mos.

397 (43.7)

32 (32.7)

2104 (32.4)

54 (38.3)

165 (26.3)

375 (37.8)

2087 (32.4)

N (%) maintained 12 mos.

248 (27.3)

13 (13.3)

1153 (17.8)

20 (14.2)

82 (13.1)

376 (21.1)

1145 (17.8)

N (%) dropped after goal

381 (41.9)*

48 (49.0)

3181 (49.0)

71 (50.4)

340 (54.2)

832 (46.6)

3167 (49.1)

N (%) 5% weight loss

698 (25)*

66 (17)

4606 (19)

109 (19)

424 (14)

1320 (20)

4561 (19)

M (SD) initial kg

98.6 (21.9)

82.8 (16.0)

94.5 (20.7)

95.5 (22.2)

91.0 (20.6)

95.1 (22.3)

97.1 (21.2)

M (SD) initial BMI

35.8 (8.4)

31.2 (4.5)

34.2 (6.6)

36.1 (7.1)

35.3 (7.3)

35.2 (7.6)

34.3 (6.7)

M (SD) kg lost

8.7 (10.4)

8.6 (8.6)

8.5 (9.4)

8.8 (10.3)

8.5 (9.7)

8.3 (9.5)

8.6 (9.6)

M (SD) years enrolled

2.0 (1.3)*

1.6 (1.2)

1.6 (1.2)

1.6 (1.1)

1.6 (1.3)

1.9 (1.3)†

1.6 (1.2)

M (SD) incentives ($)

51.5 (64.9)

43.0 (51.8)

47.0 (59.7)

42.1 (64.9)

40.3 (57.4)

46.6 (57.4)

47.3 (61.5)

N (%) maintained 6 mos.

319 (46)

25 (38)

1610 (35)

36 (33)

127 (30)

662 (50)

1996 (44)

N (%) maintained 12 mos.

195 (28)

11 (17)

822 (18)

16 (15)

55 (13)

371 (28)

1114 (24)

N (%) dropped after goal

259 (37)*

26 (39)

1994 (43)

44 (40)

204 (48)

519 (39)

1911 (42)

Values in bolded rows used as denominator for subsequently reported proportions within the column in the respective table section. Italicized numbers used as denominator for subsequently reported italicized proportions within the column

*African American vs. all others p ≤ 0.001

†Hispanic vs. non-Hispanic p ≤ 0.001

Costs

Total implementation costs over the study period were $2,822,698. The majority of costs were related to annual fees ($1,758,766), which included the maintenance and oversight of technical system support ($1,124,803), kiosk leasing ($349,500), and participant-related prizes and activities unrelated to weight loss ($248,151). Program implementation personnel costs were $383,119. Marketing personnel and activity costs were $344,054, and weight loss incentives distributed to participants with a BMI > 25 totaled $300,000. Finally, monthly fees that were primarily related to internet and short message service (SMS) use were $36,759.

Cost per enrolled participant who was overweight or obese (n = 33,656) was $83.87. The cost per participant achieving a 3% weight loss (n = 9027) was $312.70, compared to $436.68 for those who lost 5% of their initial body weight. Costs for men losing 3% and 5% body weight were $321.15 and $459.83, respectively, whereas costs for women were $314.28 and $437.45, respectively. Costs per African American participant achieving a clinically meaningful weight loss were $257.97 (3% loss) and $335.96 (5% loss). Costs for Hispanic ($318.62; $431.10) and Caucasian ($313.65; $441.87) participants were similar between those with 3% and 5% weight loss.

DISCUSSION

Approximately 19,000 WAW participants engaged in the program beyond enrollment tasks over the 4-year period. African American participants were more likely to enroll, extend their engagement beyond enrollment, lose a clinically meaningful amount of weight, and remain engaged longer than participants of other races. WAW also recruited four times the average number of participants who identified as Hispanic when compared to RCT weight loss trials22. These data are promising and may support the call for healthcare systems to provide interventions that attract populations that experience health disparities and can be scaled broadly.23,24

The average participant weight loss was lower than interventions tested in RCTs,25 but appears to compare favorably when considering the proportion of participants who achieved a clinically meaningful weight loss. The weight loss trial by Tate and colleagues reported that 22% of the participants who completed follow-up achieved a 5% weight loss with an internet-based educational program—which is similar to the WAW baseline-value-carried-forward proportion of 19%. However, when a doctoral-level therapist provided personalized behavioral therapy, the proportion of participants achieving 5% weight loss increased to 45%.26 A more intensive intervention, the Diabetes Prevention Program, showed that a lifestyle intervention resulted in 49% of the 1079 participants and 35% of the 204 African American participants achieving a 7% weight loss.27 Using only those who engaged beyond enrollment, 34% of the 19,029 WAW participants and 40% of the 1759 African American participants achieved a clinically meaningful weight loss—making comparisons much more favorable. Furthermore, the average percentage weight loss for those who exceeded the 5% criterion was 8.5% of initial body weight. Finally, while it is challenging to compare the cost per clinically meaningful weight loss due to the paucity of this information in the literature, the ∼ $84 cost per participant for the 12-month program appears to be low when compared to $258 for 6 months of Weight Watchers participation.8

This study is limited by a lack of a control group and ability to generalize beyond the program region. In addition, the assessment of maintenance was based on participants maintaining participation 6 to 12 months post achieving a 3% or 5% weight loss. To address this, we considered the 43% of the participants who lost 5% but did not have additional follow-up assessments as unsuccessful at maintenance. Finally, costs were included for 4 years of implementation, but some participants from 2014 received the portions of the program in 2015, indicating that the costs may be somewhat underestimated. However, we also did not attribute costs to participants with a BMI under 25, which may have contributed to an overestimation of costs.

CONCLUSIONS

The primary objective of this study was to examine reach, effectiveness, and cost metrics associated with WAW as a way to compare weight loss interventions and inform healthcare system decisions on the appropriateness of community-program referral. WAW (a) reached a large number of participants and may contribute to reducing health disparities experienced by African Americans based on our exploratory analyses, and (b) had a cost per participant that rates favorably against other publicly available weight loss programs,6,8 and (c) the cost per participant who achieved a clinically meaningful weight loss appears to be modest,28 though comparisons across weight loss interventions remain difficult due to a lack of information on costs across RCT-tested interventions.9–11

Notes

Acknowledgements

Funding for this project was provided by the Harold M. Maurer Distinguished Chair in Public Health Research Endowment at the University of Nebraska Medical Center and the Virginia Tech Fralin Center for Translational Obesity Research. Thanks to Dr. Wen You who provided feedback on early drafts of the manuscript to review conceptual clarity and appropriateness of statistical methods, and to Kaiser Permanente Colorado for allowing access to all de-identified participant data and invoices for program implementation costs.

Compliance with Ethical Standards

Conflict of Interest

Lia Schoepke is employed by IncentaHEALTH LLC. Todd McGuire is employed by, holds the patent to, and is a part owner of IncentaHEALTH LLC. All other authors declare that they have no conflict of interest.

References

1.

Ahmed NU, Delgado M, Saxena A. Trends and disparities in the prevalence of physicians’ counseling on diet and nutrition among the U.S. adult population, 2000-2011. Prev Med. 2016;89:70–5.CrossRefPubMedGoogle Scholar

Jensen MD, Ryan DH, Apovian CM, et al. 2013 AHA/ACC/TOS Guideline for the Management of Overweight and Obesity in Adults: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and The Obesity Society. Journal of the American College of Cardiology 2013.Google Scholar