Volume 32, Issue 2, Fall 2008 - Forrestal;Issel;Kviz;Chávez

Validating a Children’s Self-Report Plate Waste Questionnaire

ABSTRACT

Purpose/ObjectivesThe National School Lunch Program is well situated to address the vulnerability of lower income children at increased risk for both under- and overnutrition. Evidence suggests, however, that a significant amount of food served in the program goes uneaten. One way to monitor this problem is through children’s self-reported plate waste (PW), but no validated measure exists. Therefore, the purpose of this study was to develop and validate a children’s self-report PW questionnaire.

MethodsLunch trays were collected from 54 sixth grade students after lunch. Remaining foods were weighed to determine the percent wasted. The same children completed a PW questionnaire later in the day. To assess validity, students’ self-reported PW was compared to the weighed quantities. The questionnaire also asked about a series of 15 potential reasons for PW. Reliability of these items was analyzed to assess internal consistency. Twenty-four participants completed the procedures a second time so that test-retest reliability also could be determined.

ResultsSignificant correlations were found between weighed and self-reported PW (p<.05), indicating criterion validity. Test-retest reliability on eight of the 15 PW reasons was significant (p<.1); an additional four were invariant for at least one of the two timepoints. The series of items was also internally consistent (Cronbach’s alpha = .71). Further research is needed to evaluate the instrument across a wider variety of school settings and lunch menus.

Applications to Child Nutrition ProfessionalsChild nutrition professionals may use this valid and reliable self-report measure of children’s PW to monitor menu performance in federal child nutrition programs. This instrument may be used to assess changes made to the meals served, which can assist in reaching Healthy People 2010 goals for school nutrition.

INTRODUCTION

The National School Lunch Program (NSLP) is designed to provide low cost, nutritious lunches and snacks to children in over 95,000 schools (United States Department of Agriculture [USDA], 2007). The NSLP is mandated to provide one-third of children’s Recommended Dietary Allowance of vitamins A and C, iron, calcium, and protein (USDA, 2007) – nutrients that support physical and cognitive development (Birch & Fisher, 1998; Bryan et al., 2004). In 2007, 30.5 million lunches were served each day, 59% of which went to children whose family income qualifies them to receive the meals for free or at a reduced price (USDA, 2008).

The NSLP is especially important for low-income children, who may have limited access to nutritious foods. These children are at increased risk for both over- and undernutrition. In the past three decades, the number of overweight children in the United States has risen dramatically, with prevalence doubling among children age 6 – 11 and tripling among 12- to 19-year olds (National Center for Health Statistics, 2007). Currently, 17% of children nationwide are overweight or obese (Ogden et al., 2006), and lower income children are 2.9 times more likely to be overweight than their higher income peers (Strauss & Knight, 1999). Lower income children are also more likely to be food insecure. Compared to children from higher income families, low-income children are five times as likely to experience food insecurity (Nord, Andrews, & Carlson, 2007).

Despite the importance of the NSLP, evidence suggests that the meals may not be meeting children’s nutritional needs. In a national survey of school cafeteria managers, respondents estimated that up to 12% of the calories served in the meals were wasted, with nutrient-dense fruits and vegetables being discarded the most (Buzby & Guthrie, 2002). Increased attention is being given to the overall nutritional quality of foods and beverages available at schools. The Institute of Medicine (2007) recently published a report identifying standards to ensure that school foods are consistent with the 2005 Dietary Guidelines for Americans.

One criterion to evaluate menu performance in the NSLP is plate waste (PW), defined as the amount of edible food served that goes uneaten. PW can be measured a variety of ways, including direct weighing, visual estimation, and self-reports. While direct weighing is the most accurate approach, setting up and using a scale in a cafeteria can be disruptive to foodservice operations, and data collection can be messy and time-consuming. In contrast, self-reported measures of PW are quick and easy to administer. However, to be useful substitutes for direct weighing, valid and reliable self-report measures are needed. A literature search by the authors did not find any descriptions of self-reported PW measures validated with children.

The purpose of this study was to develop and validate a children’s self-report PW questionnaire. Specifically, the study was designed to answer three questions: (1) are children’s self-reports of PW consistent with actual behavior?, (2) are self-reports consistent over time?, and (3) are self-reports internally consistent?

METHODOLOGY

The study was conducted in two phases with sixth grade students at one public middle school. The study targeted sixth graders because their level of cognitive development is likely sufficient for handling the response tasks required by the questionnaire, especially portion size estimation (Livingstone, Robson, & Wallace, 2004). In Phase I, a questionnaire draft was developed and revised based on feedback obtained in a focus group. In Phase II, the revised questionnaire, pre-coded with the day’s lunch menu, was validated by comparing children’s self-reports to their weighed PW. The Institutional Review Board of the University of Illinois at Chicago approved all materials and study procedures, including parental consent and child assent, prior to data collection.

Phase I: Questionnaire DevelopmentKey concepts from the school lunch PW literature were organized into a framework that guided questionnaire development. Items inquired about school, cafeteria, and individual factors. Children were asked: (1) when school begins and ends, and when they eat lunch, (2) how often they eat school lunch, (3) how much of each school lunch item they ate on the target day, (4) how much school lunch they would eat under several conditions that may contribute to PW, and (5) demographic questions. Twelve items asked about the reasons for PW. Respondents were asked how much they agreed that each contributed to wasting food. The items were drawn from surveys of upper elementary students (Meyer, 2005) and cafeteria managers (United States General Accounting Office, 1996).

A draft questionnaire was presented to five students in a focus group. The children were a convenience sample drawn from one of the three groups of sixth graders in the school. Participants completed the instrument and then discussed it. They were presented with three alternatives to measure school lunch consumption: bars to be shaded up to the appropriate fraction (Figure 1a), semi-quantitative amounts arrayed in a grid (Figure 1b), and pie charts representing fractions eaten, ranging from none (100% shaded) to all consumed (0% shaded; Figure 1c). The first approach resembled one used in a Summer Food Service Program PW study that did not use self-reports (Cotugna & Vickery, 2004). The pie chart was adapted from a hospital menu performance study (Connors & Rozell, 2004).

Phase II: ValidationFifty-four children, none of whom had been in the focus group, participated in Phase II, in which the questionnaire was validated against weighed PW. The convenience sample consisted of students in the other two groups of sixth graders in the school. To minimize reactivity, participants were not told the study purpose. Prior to lunch, each was given an identification number. After lunch, participants were asked to return their trays to the researchers without discarding anything. As children returned their trays, they provided their identification number so that weighed PW could be matched to the appropriate questionnaire.

The school served pre-packaged foods. To determine the quantity served, five of each item was weighed to the nearest gram on a digital scale (Ohaus Model CS-2000, Pine Brook, NJ), with the weights averaged. To determine the quantity wasted, uneaten items left on students’ trays were weighed. The observed PW percentage was calculated by dividing the amount left on the tray by the average amount served. Self-reported PW was calculated by subtracting the fraction reported eaten from 100%.

Two hours after lunch, participants completed the questionnaire in class while questions were read aloud; administration required 15 minutes. All students, whether or not they had eaten school lunch, were asked to complete the questionnaire. A subset of 24 participants repeated the procedures one month later when the menu was served again. Lunch was identical except that apple were served the first time, and bananas the second time.

Table 1 summarizes Phase II participants’ characteristics. The sample was multi-ethnic and predominantly male at both assessments. Eighty-five percent of the children reported receiving free or reduced price lunch, which exceeds the national average of 59% (USDA, 2008). Forty-four percent reported eating school lunch on the day of the first assessment; for the second assessment, 53% of the children ate school lunch.

Table 1. Demographic characteristics of the full sample and the subset who underwent repeat assessment.

Data AnalysisSPSS (version 15) was used for all analyses. Validity was assessed with Pearson correlations between self-reported and weighed PW. Reliability of the PW reasons was assessed using two measures. Cronbach’s alpha was calculated to determine internal consistency among the statements. Kendall’s tau-b was calculated to determine test-retest reliability among the children who completed the instrument twice.

RESULTS AND DISCUSSION

Focus Group FeedbackParticipants’ feedback guided revisions to the instrument prior to Phase II. Specifically, the children preferred shading bars to report PW (Figure 1a), saying the method was easiest to understand. The PW reasons were revised to include three additional considerations the children identified (dietary restrictions, open food packaging, and foods past their expiration dates), and the response options were changed to ask about quantities eaten: all, some, a few bites, none, and not applicable.

Questionnaire ValidityThe children wasted large amounts of the school lunch, likely exceeding the 12% of calories found in previous studies (Buzby & Guthrie, 2002). The entrée, pepperoni pizza, was wasted the most (78% of the amount served), and milk the least (19%). The children were asked to identify their least liked school lunch food, and pizza was the most frequently named (52%), though this may at least partially reflect the item’s salience rather than its true unpopularity. Thus, participants’ observed behavior is consistent with their self-reported preferences. The unpopularity of the pizza was serendipitous, as it provided a good test of the questionnaire. The substantial amount of waste available for measurement made comparisons to self-reports feasible; better liked foods may have been eaten entirely, making PW validation impossible due to a lack of any leftover foods for weighing.

Pearson correlations between self-reported and weighed PW ranged from .55 to .98, and were significant (p<.05) for all items (Table 2). This supports the questionnaire’s validity as a measure of children’s PW. Bananas had the largest correlation. The children generally either ate all or none of the fruit, making it easy to report accurately. Milk had the smallest correlation. Because it was served in an opaque carton, the packaging likely prohibited an accurate assessment of its consumption.

Questionnaire ReliabilityReliability was evaluated with two different measures. Test-retest reliability was calculated on the series of PW reasons, using data from children whose reports could be matched from both assessments. With alpha set at .10 due to the small sample size, Kendall’s tau-b was significant for eight of the 15 items; five of the eight were significant at p<.05. Four items were invariant for one or both of the assessments (Table 3). Further research is needed to assess whether the invariant or non-significant items should be deleted, or if the lack of significance for some items may be related to the small sample size achieved in this study.

Scale reliability was also examined. For the same series of items, Cronbach’s alpha was calculated to determine whether the questions were reliable across children. With a value of .71, the series demonstrates moderate internal consistency.

Study StrengthsThe qualitative and quantitative procedures in the two study phases provided different types of evidence for validity. In Phase I, children’s feedback provided evidence of face validity; that is, their comments indicated that the questionnaire was generally relevant and comprehensible. After their suggestions were incorporated, Phase II offered further evidence of validity and reliability. Significant correlations between reported intakes and weighed PW for all tested menu items demonstrate criterion validity. A measure of test-retest reliability suggested that children’s reports of their potential eating behavior under a variety of circumstances were stable over time for two-thirds of the statements tested. Finally, Cronbach’s alpha was high for these same PW reasons, providing evidence for internal consistency. It is noteworthy that findings were significant despite low statistical power due to the small sample size available for the study.

Four additional methodological aspects of the study contributed to its strength. First, students were asked about what they ate for lunch shortly after the meal, minimizing the difficulty of the recall task. Baxter et al. (2004) documented a similar effect. Fourth grade students were observed eating school lunch and were later interviewed about their intake during the preceding 24 hours. Students who were interviewed in the afternoon had fewer omissions and intrusions for school lunch items, as well as better overall accuracy compared to students interviewed later in the evening or the following morning (Baxter et al., 2004). Querying students shortly after the target meal guards against fading memory.

Second, the school served pre-packaged foods. Students were asked to report only the amount eaten, not the amount served. Estimating portion size is a difficult cognitive task even for adults, and may be beyond the ability of children younger than 11 (Livingstone, Robson, & Wallace, 2004). Thus, asking students only about the fraction they consumed, rather than an absolute quantity, is less challenging. However, many schools do not serve pre-packaged lunch items, so using the questionnaire elsewhere may require students to estimate both the amounts served and eaten, due to greater variability in the portion sizes served.

Third, sixth grade students were deliberately chosen for the validation study because their level of cognitive development was likely sufficient for handling the response tasks (e.g., memory recall, portion size estimation, and abstract reasoning). Borgers, de Leeuw, and Hox (2000) reviewed children’s capabilities as survey respondents within the framework of Piaget’s cognitive developmental stages. By age 12, children are generally in the formal thought stage, where they are competent in logical operations and abstract reasoning. Participants in this study were 12 years old, and therefore likely had sufficiently developed cognitive skills to provide valid self-reports, though this was not directly assessed. It is likely that the questionnaire would perform at least as well, if not better, with older subjects, given their greater cognitive development.

Finally, the recall task was simplified by pre-coding the questionnaire with the day’s lunch menu. Recognizing foods from a list is a superior recall strategy compared to having to remember on one’s own. In a study eliciting children’s dietary recall strategies, participants reported reading the lunch menu facilitated memory retrieval (Domel, 1997). Listing the food items on the PW questionnaire may have served a similar function, making it easier for the children to recall additional information about quantities consumed.

Study LimitationsAlthough the findings are promising, the study has two limitations. Participants comprised a convenience sample from one school. The study was not intended to be generalizable to the national population of sixth grade students. Further replication is needed in a variety of school settings to draw stronger conclusions about the instrument’s validity. Second, only one menu was tested. Although this simplified the validation procedures, more work is needed to assess whether the questionnaire is reliable with other foods. It is possible that children may have a more difficult time reporting their consumption of entrees such as hamburgers or chicken sandwiches, which can be more easily separated into their component parts (e.g., bun, patty, etc.) than pizza.

Applicability of the questionnaire itself may be limited by the particularities of the foodservice setting. Cafeterias implementing the offer versus serve provision pose a greater challenge to its utility. Students’ freedom to choose or decline items may hamper the usefulness of a pre-coded instrument, given the potentially greater variability in items selected. On the other hand, the offer versus serve provision was implemented to reduce PW (Buzby & Guthrie, 2002), so the need to reduce waste in these settings may not be as pressing.

CONCLUSIONS AND APPLICATIONS

This study has demonstrated that: (1) 12-year-old children may accurately report what they consumed in the school lunch when queried in a developmentally appropriate manner, (2) the questionnaire is a valid and reliable measure of PW in the NSLP, and (3) it represents an easy way to assess children’s PW without resorting to more resource-intensive methods like weighing.

Although the questionnaire requires further evaluation, it has the potential to be used in both basic and applied research in different settings, so long as respondents are cognitively able to complete the response task and standardized portions are served. Such settings may include the Summer Food Service Program or for afterschool programs participating in the Child and Adult Care Food Program.

The questionnaire could be used empirically to identify additional factors that contribute to PW in child nutrition programs. For example, the role of competitive foods in children’s PW patterns could be explored (neither vending machines nor a snack bar were present in the study school, so these factors could not be assessed). Anthropometric measures could be combined with self-reports to examine the relationship between adiposity and underreporting food intake, a phenomenon that has been documented previously, including recently by Savage and colleagues (2008). Child nutrition professionals, including foodservice personnel, nutritionists, and researchers, may use this questionnaire as a rapid, low-cost assessment of menu performance or an evaluation tool in school foodservice programs. Thus, the questionnaire may be used to help reach Healthy People 2010 Objective 19-15: “Increase the proportion of children and adolescents aged 6 to 19 years whose intake of meals and snacks at school contributes to overall dietary quality” (United States Department of Health and Human Services, 2000).

ACKNOWLEDGEMENTS

The authors thank the faculty, staff, and students at the participating school for their assistance and participation. Thanks also to Meghan Benson and Moji Owolabi, who aided in data collection. The study was funded by a University of Illinois at Chicago Provost’s Award for Graduate Research.

United States Department of Health and Human Services (2000). Healthy People 2010: Understanding and improving health (2nd ed.). Washington, DC: United States Government Printing Office.

United States General Accounting Office (1996). School lunch program: Cafeteria managers’ views on food wasted by students (Report No. GAO/RCED-96-191). Washington, DC: United States General Accounting Office.

BIOGRAPHY

Forrestal, Issel, Kviz, and Chávez are, respectively, Doctoral Candidate, Clinical Associate Professor, Professor, and Associate Professor at the University of Illinois at Chicago School of Public Health.