I don't usually delve into K-12 classroom-based research to any great extent. However, as my daughter's elementary school is starting a "redesign" process—and I heard arguments on both sides of the multigrade-classroom issue—I thought perhaps I'd give a quick look at the research available.

I had hoped that this would be a quick review, where I would find one or two definitive research reviews in scientific refereed journals, but unfortunately, the research base is rather frail and unclear.

Still, I think the following review does provide some wisdom about how to think about multiage classrooms.

————————————————-

Extent of Review

I did a moderately quick review—not an exhaustive review—of a couple dozen recent research articles on multigrade classrooms.

This review was conducted in September and October of 2010.

————————————————-

Major Conclusions (See More Specific Recommendations Further Below):

The research, although being too scant and too difficult to interpret to make definitive recommendations, generally suggests that multigrade classroom approaches are not likely to produce results that differ substantially from single-grade classrooms.

Specifically, it is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized.

Multigrade teaching is generally considered more difficult and onerous than single-grade teaching.

If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels), and so forth.

————————————————-

Context:

Researchers and practitioners use many labels for multigrade education, including the following: multigrade, multiage, mixed-age, vertical grouping, combination, composite, double-grade, split-grade, dual-age, hyphenated, nongraded, etc. These terms are often confused, redundant, etc., making researching and thinking about related issues difficult. In this report, I will use the following terms:

Single-grade classrooms are comprised of one grade.

Multigrade classrooms are comprised of two (or more) grades.

Combination Multigrade classrooms are multigrade classrooms that are utilized for logistical reasons.

Pedagogic Multigrade classrooms are multigrade classrooms that are utilized for pedagogic (learning-related) reasons.

Sometimes multigrade classrooms are created because of logistical reasons such as declining or uneven enrollments (Veenman, 1996; Burns & Mason, 2002; Mulryan-Kyne, 2007).

Sometimes multigrade classrooms are created for philosophical and pedagogical rationale. There are strong advocacy groups for pedagogic multigrade classrooms. Some of this advocacy is at odds with the research—in the sense that they claim overwhelming benefits for multigrade classrooms when the research is more balanced and uncertain (I noticed this myself in reviewing the research, but also see Burns & Mason, 1997).

It can be helpful to view classes on a continuum from single-grade classes, to combination multigrade classes, to pedagogic multigrade classes (See Mason & Burns, 1997). While many different arrangements are possible, we can talk in generalities as follows:

Single-grade classes have students only from one grade.

Combination multigrade classes have students from two (or more) grades and one teacher who teaches them.

Pedagogic multigrade classes have students from two (or more) grades but are taught by a team of teachers. Pedagogic multigrade classes also tend to be more focused on providing learners individually-tailored learning content than either single-grade or combination multigrade classes.

Teachers in combination multigrade classes are less likely to receive the support and structure they need than teachers in pedagogic multigrade classes (Mason & Burns, 1997).

Multigrade classrooms are not aberrations, but are used throughout the world, and are likely to continue and grow in use in the future (Mulryan-Kyne, 2007).

————————————————-

Quality of Available Research:

Unfortunately, it seems that much of the available research is tainted by methodological weaknesses. Specifically, it appears that more experienced teachers tend to teach multi-age classrooms—so that when advantages appear in the educational results, those advantages may be due to teacher experience as opposed to the multi-age classroom itself. In addition, more affluent students are the ones who tend to be taught in multigrade classrooms—so that when research results show advantages, those could be due to socio-economic and educational advantages of parents as opposed to the multigrade classrooms.

Quotes from the Research:

“Effective research in the area of multiage education is still in its infancy.” From Kinsey (2001).

“The literature on multigrade teaching is relatively sparse, some of it anecdotal in nature and/or of poor quality.” From Mulryan-Kyne (2007).

“There is considerable evidence that principals, in an effort to reduce the burden on multigrade teachers, place more able, more independent, and more cooperative students in multigrade classes.” From Mason and Burns (1996). Note: This is relevant in that good results in comparison to single-age classrooms may be due to having better students in the class.

“No studies in which students were randomly assigned to experimental and control groups were found.” From Veenman (1995). Note: While such random assignment is the gold standard in research, it is difficult to implement in the classroom.

“Because of the lack of distinction between combination and multiage/nongraded classes and the omission of important studies and methodological considerations, it appears that researchers have drawn overly optimistic and erroneous conclusions about the effects of combination classes.” From Mason & Burns (1997).

Conclusion based on Quality of Available Research:

These methodological weaknesses make firm conclusions difficult. Tentative conclusions are still possible.

————————————————-

Teaching is Often More Difficult in Multigrade Classrooms:

It appears that teaching in a multigrade classroom is more difficult than teaching in a single-age classroom. This conclusion comes from those on all sides of the debate, so it is a fairly strong conclusion.

Quotes from the Research:

“It is commonly stated in the literature that multigrade teaching is more difficult than single grade teaching” From Mulryan-Kyne (2007).

“Multigrade classes hold instructional potential for some, but they are potentially onerous for most. Indeed, we argue that multigrade classes lead to a negative instructional effect and that they increase teachers' stress and may jeopardize teachers' motivation and commitment to teaching.” From Mason and Burns (1996).

“Most teachers, when asked about their feelings toward and organizational and teaching strategies for combination classes, responded negatively and preferred not to teach them.” From Mason and Burns (1995).

“The professional knowledge and skills that are relevant and necessary to teaching effectively in single-grade contexts are also relevant and necessary for effective multigrade teaching… However, many of these skills need heightened emphasis in the context of the preparation of teachers for multigrade teaching.” From Mulryan-Kyne (2007).

“[Teachers] report that these classes require more planning, are more difficult to teach, and diminish instruction and curriculum coverage.” From Mason & Burns (1997).

“We concluded that the difficulties teachers face in multigrade classes are centered around five problem areas: (a) the efficient use of instructional time, (b) the design of effective instruction, (c) classroom management, (d) the organization of independent practice or learning, and (e) the formulation of clear and collectively agreed-upon goals for making the multigrade school work.” From Veenman (1996).

“…multigrade teachers pressed for instructional time and the mastery of basic skills (e.g., reading, writing, and mathematics) might neglect science, social studies, and other subjects, which would lead to negative achievement effects in these areas.” From Mason & Burns (1996).

————————————————-

Academic Achievement Results from Multigrade Classrooms:

In the two most credible recent reviews of the research, the findings have shown no differences in academic achievement between multigrade classrooms and single-grade classrooms (Veenman, 1995; Mason & Burns, 1997). However, because of the likelihood that this research is tainted in having better students and teachers in multi-grade classrooms, it is possible that multigrade classrooms produce “at least small negative effects” (Mason & Burns, 1997).

Separating the results for combination multigrade classrooms and pedagogical multigrade classrooms is difficult because researchers haven’t always noted this difference. In Veenman’s (1995) review, both combination and pedagogic multigrade classrooms produced non-significant results—in other words, they were found to produce the same academic results as single-grade classrooms. However, as Mason and Burns (1996) pointed out, Veenman didn’t take into account some important potential biases. Mason and Burns write: “We conclude that multigrade classes have at least a small negative effect on achievement as well as potentially negative effects on teacher motivation.”

Unfortunately, in Mason and Burns (1997) follow-up research review, they only focused on combination multigrade classrooms—NOT pedagogic multigrade classrooms. Their results on combination multigrade classrooms were consistent with Veenman’s in finding no statistical differences, but they warned that “all things being equal, combination classes have at least small negative effects.” Such a negative outcome was found in a 2008 study of combination multigrade classrooms in California (Sims, 2008).

Conclusions on Academic Achievement:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of academic results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes, but if they are different from single-grade classes, they may be slightly worse.

————————————————-

Social/Emotional Results from Multi-age Classrooms:

Veenman’s (1995) research found that a majority of combination multigrade classes performed no better than single-grade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Mason and Burns (1997) examination of combination multigrade classes found similar non-significant affective benefits.

For pedagogic multigrade classes, however, Veenman (1996) found a “very small” effect, showing slight benefits for pedagogic multigrade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Again, because of the potential biasing effects in terms of student and teacher selection, these results could be due to bias instead of any benefits for pedagogic multigrade classes.

Conclusions on Social/Emotional Results:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of social/emotional results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes. If there are very small benefits, they might be obtained in well-designed pedagogic multigrade classrooms—as opposed to logistically-driven combination multigrade classrooms.

————————————————-

Do some Students Benefit More than Others from Multigrade Classrooms?

Although this was beyond the scope of my review—I did come across one recent study that dealt specifically with the question of how different types of students might be affected by a multigrade approach (Ong, Allison, & Haladyna, 2000).

First, it should be noted that in this one study, multigrade classrooms tended to outperform single-grade classrooms for all students, regardless of their background. Given that as background, the study found the following:

It found no differences between boys and girls.

It found that regular students seemed to benefit more from multigrade classrooms than disadvantaged students (that is, Title 1 students).

It found that overall non-Hispanic students seemed to benefit more from multigrade classrooms than Hispanic students.

The scarcity, frailty, and equivocation in the research make strong recommendations impossible. Instead, I offer these tentative ideas for consideration:

1. It is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized. Given this, it appears that choosing a multigrade approach would be acceptable, though the following points should also be kept in mind.

2. If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels*), and so forth.

* Recent research indicates that teachers, tutors, and other learning professionals tend not to be very good at providing instructional explanations at learner’s appropriate levels—but that they can improve on this by specifically being informed of their learners’ level of understanding (Wittwer, Nückles, & Renkl, 2010).

3. Multigrade teaching seems to offer the possibility for alternative methods of learning, including group work targeted to different developmental levels. On the other hand, multigrade teaching by itself is no guarantee of good teaching methodology.

4. It should be recognized that multigrade teaching probably has costs associated with its use. Specifically, teachers may have to invest more effort and care in the process and may have a tendency to tradeoff other desirable educational goals.

5. In some forms of multigrade teaching, especially those that utilize team teaching, teachers have to learn the skill levels (in each discipline) of many more students. Because one of the most important aspects of teaching is providing learners with just the right level of instruction, this may cause students to be underserved at first as the teacher learns their students' skill levels. One way to ameliorate this problem is for students entering a multigrade cycle to be diagnosed through testing, problem solving, and other performance metrics early on. This “slow-start” issue has a flip side—because multigrade classes stretch into two (or more) years, students in subsequent years will probably experience accelerated learning due to deeper understanding of each student by teachers and hence better instructional scaffolding.

6. Teachers with less experience may be especially unprepared for multigrade teaching. With experience, teachers learn how to automate aspects of their classroom performance so that they can utilize their limited working memory capacity to focus on supporting their learners in learning (for example, differentiating their instruction, etc.). Newer teachers will be unlikely to provide the necessary instructional scaffolding to perform at a high level.

7. Students who have behavioral problems or who are less competent in learning may make the task of multigrade teaching more difficult. This may suggest that extra effort and guidance in the early grades is warranted or that a multigrade approach should be delayed until a time when most students are ready to engage fully in the process.

Song, R. Spradlin, T. E., & and Plucker, J. A. (2009). The Advantages and Disadvantages of Multiage Classrooms in the Era of NCLB Accountability. Education Policy Brief, 7, 1-7. Published by the Center for Evaluation and Education Policy of Indiana University.

Why not cited: This article is not from refereed journal so it may not have been fully vetted. Also, there are hints of bias in the article, for example, (a) providing invited proponents of multiage classrooms to describe its value, without providing a similar counterpoint, and (b) talking about the lack of good research but then plowing ahead with a list of the “benefits (perceived and real)” of multiage classrooms—so in a real sense the article ignores the research by plowing ahead to benefits. Still, I wouldn’t discount this article completely. Its research review seems good when it is limited to reviewing the actual research. Also, it makes recommendations that are mostly consistent with the findings contained in this review, so I can’t dismiss their conclusions. The bottom line is that this review does not add anything to a review of the research, and, because it is not vetted in a refereed journal and appears slightly biased, I think it safer not to cite it as separate evidence.

Why not cited: This article looked at only 4 teachers, all who had chosen to be teaching in multiage classrooms. It was really a descriptive research project and did not look at actual learning outcomes. No single-grade teachers were examined so we don’t really know how different results with single-grade teaching might be.

Why not cited: This article utilized an incredibly biased research design. It took one classroom at one school that used multiage grouping and compared it on standardized exams with the national average.

Mariano, S., & Kirby N. (2009).Achievement of Students in Multigrade Classrooms Evidence from the Los Angeles Unified School District. From the RAND Education Working Paper Series (WR-685-IES).

Why not cited: This article is not from refereed journal so it may not have been fully vetted. It also uses somewhat opaque statistical methods to estimate findings, not looking directly at actual comparisons between multigrade and single-grade classrooms. I must admit that I don’t fully understand all the statistical employments utilized (for example, “doubly robust regression,” “non-parametric generalized boosting,” “the Kolmogorov-Smirnov (K-S) statistic”). Thus, I may be missing the full implications of the research employed.

I don’t usually delve into K-12 classroom-based research to any great extent. However, as my daughter’s elementary school is starting a “redesign” process—and I heard arguments on both sides of the multigrade-classroom issue—I thought perhaps I’d give a quick look at the research available.

I had hoped that this would be a quick review, where I would find one or two definitive research reviews in scientific refereed journals, but unfortunately, the research base is rather frail and unclear.

Still, I think the following review does provide some wisdom about how to think about multiage classrooms.

————————————————-

Extent of Review

I did a moderately quick review—not an exhaustive review—of a couple dozen recent research articles on multigrade classrooms.

This review was conducted in September and October of 2010.

————————————————-

Major Conclusions (See More Specific Recommendations Further Below):

The research, although being too scant and too difficult to interpret to make definitive recommendations, generally suggests that multigrade classroom approaches are not likely to produce results that differ substantially from single-grade classrooms.

Specifically, it is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized.

Multigrade teaching is generally considered more difficult and onerous than single-grade teaching.

If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels), and so forth.

————————————————-

Context:

Researchers and practitioners use many labels for multigrade education, including the following: multigrade, multiage, mixed-age, vertical grouping, combination, composite, double-grade, split-grade, dual-age, hyphenated, nongraded, etc. These terms are often confused, redundant, etc., making researching and thinking about related issues difficult. In this report, I will use the following terms:

Single-grade classrooms are comprised of one grade.

Multigrade classrooms are comprised of two (or more) grades.

Combination Multigrade classrooms are multigrade classrooms that are utilized for logistical reasons.

Pedagogic Multigrade classrooms are multigrade classrooms that are utilized for pedagogic (learning-related) reasons.

Sometimes multigrade classrooms are created because of logistical reasons such as declining or uneven enrollments (Veenman, 1996; Burns & Mason, 2002; Mulryan-Kyne, 2007).

Sometimes multigrade classrooms are created for philosophical and pedagogical rationale. There are strong advocacy groups for pedagogic multigrade classrooms. Some of this advocacy is at odds with the research—in the sense that they claim overwhelming benefits for multigrade classrooms when the research is more balanced and uncertain (I noticed this myself in reviewing the research, but also see Burns & Mason, 1997).

It can be helpful to view classes on a continuum from single-grade classes, to combination multigrade classes, to pedagogic multigrade classes (See Mason & Burns, 1997). While many different arrangements are possible, we can talk in generalities as follows:

Single-grade classes have students only from one grade.

Combination multigrade classes have students from two (or more) grades and one teacher who teaches them.

Pedagogic multigrade classes have students from two (or more) grades but are taught by a team of teachers. Pedagogic multigrade classes also tend to be more focused on providing learners individually-tailored learning content than either single-grade or combination multigrade classes.

Teachers in combination multigrade classes are less likely to receive the support and structure they need than teachers in pedagogic multigrade classes (Mason & Burns, 1997).

Multigrade classrooms are not aberrations, but are used throughout the world, and are likely to continue and grow in use in the future (Mulryan-Kyne, 2007).

————————————————-

Quality of Available Research:

Unfortunately, it seems that much of the available research is tainted by methodological weaknesses. Specifically, it appears that more experienced teachers tend to teach multi-age classrooms—so that when advantages appear in the educational results, those advantages may be due to teacher experience as opposed to the multi-age classroom itself. In addition, more affluent students are the ones who tend to be taught in multigrade classrooms—so that when research results show advantages, those could be due to socio-economic and educational advantages of parents as opposed to the multigrade classrooms.

Quotes from the Research:

“Effective research in the area of multiage education is still in its infancy.” From Kinsey (2001).

“The literature on multigrade teaching is relatively sparse, some of it anecdotal in nature and/or of poor quality.” From Mulryan-Kyne (2007).

“There is considerable evidence that principals, in an effort to reduce the burden on multigrade teachers, place more able, more independent, and more cooperative students in multigrade classes.” From Mason and Burns (1996). Note: This is relevant in that good results in comparison to single-age classrooms may be due to having better students in the class.

“No studies in which students were randomly assigned to experimental and control groups were found.” From Veenman (1995). Note: While such random assignment is the gold standard in research, it is difficult to implement in the classroom.

“Because of the lack of distinction between combination and multiage/nongraded classes and the omission of important studies and methodological considerations, it appears that researchers have drawn overly optimistic and erroneous conclusions about the effects of combination classes.” From Mason & Burns (1997).

Conclusion based on Quality of Available Research:

These methodological weaknesses make firm conclusions difficult. Tentative conclusions are still possible.

————————————————-

Teaching is Often More Difficult in Multigrade Classrooms:

It appears that teaching in a multigrade classroom is more difficult than teaching in a single-age classroom. This conclusion comes from those on all sides of the debate, so it is a fairly strong conclusion.

Quotes from the Research:

“It is commonly stated in the literature that multigrade teaching is more difficult than single grade teaching” From Mulryan-Kyne (2007).

“Multigrade classes hold instructional potential for some, but they are potentially onerous for most. Indeed, we argue that multigrade classes lead to a negative instructional effect and that they increase teachers’ stress and may jeopardize teachers’ motivation and commitment to teaching.” From Mason and Burns (1996).

“Most teachers, when asked about their feelings toward and organizational and teaching strategies for combination classes, responded negatively and preferred not to teach them.” From Mason and Burns (1995).

“The professional knowledge and skills that are relevant and necessary to teaching effectively in single-grade contexts are also relevant and necessary for effective multigrade teaching… However, many of these skills need heightened emphasis in the context of the preparation of teachers for multigrade teaching.” From Mulryan-Kyne (2007).

“[Teachers] report that these classes require more planning, are more difficult to teach, and diminish instruction and curriculum coverage.” From Mason & Burns (1997).

“We concluded that the difficulties teachers face in multigrade classes are centered around five problem areas: (a) the efficient use of instructional time, (b) the design of effective instruction, (c) classroom management, (d) the organization of independent practice or learning, and (e) the formulation of clear and collectively agreed-upon goals for making the multigrade school work.” From Veenman (1996).

“…multigrade teachers pressed for instructional time and the mastery of basic skills (e.g., reading, writing, and mathematics) might neglect science, social studies, and other subjects, which would lead to negative achievement effects in these areas.” From Mason & Burns (1996).

————————————————-

Academic Achievement Results from Multigrade Classrooms:

In the two most credible recent reviews of the research, the findings have shown no differences in academic achievement between multigrade classrooms and single-grade classrooms (Veenman, 1995; Mason & Burns, 1997). However, because of the likelihood that this research is tainted in having better students and teachers in multi-grade classrooms, it is possible that multigrade classrooms produce “at least small negative effects” (Mason & Burns, 1997).

Separating the results for combination multigrade classrooms and pedagogical multigrade classrooms is difficult because researchers haven’t always noted this difference. In Veenman’s (1995) review, both combination and pedagogic multigrade classrooms produced non-significant results—in other words, they were found to produce the same academic results as single-grade classrooms. However, as Mason and Burns (1996) pointed out, Veenman didn’t take into account some important potential biases. Mason and Burns write: “We conclude that multigrade classes have at least a small negative effect on achievement as well as potentially negative effects on teacher motivation.”

Unfortunately, in Mason and Burns (1997) follow-up research review, they only focused on combination multigrade classrooms—NOT pedagogic multigrade classrooms. Their results on combination multigrade classrooms were consistent with Veenman’s in finding no statistical differences, but they warned that “all things being equal, combination classes have at least small negative effects.” Such a negative outcome was found in a 2008 study of combination multigrade classrooms in California (Sims, 2008).

Conclusions on Academic Achievement:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of academic results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes, but if they are different from single-grade classes, they may be slightly worse.

————————————————-

Social/Emotional Results from Multi-age Classrooms:

Veenman’s (1995) research found that a majority of combination multigrade classes performed no better than single-grade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Mason and Burns (1997) examination of combination multigrade classes found similar non-significant affective benefits.

For pedagogic multigrade classes, however, Veenman (1996) found a “very small” effect, showing slight benefits for pedagogic multigrade classes in terms of attitudes towards school, self-concept, and personal and social adjustment. Again, because of the potential biasing effects in terms of student and teacher selection, these results could be due to bias instead of any benefits for pedagogic multigrade classes.

Conclusions on Social/Emotional Results:

The research is not definitive because there are factors that have not really been teased out as of yet.

In terms of social/emotional results, the tentative conclusion is that multigrade classes on average are probably no better and no worse than single-grade classes. If there are very small benefits, they might be obtained in well-designed pedagogic multigrade classrooms—as opposed to logistically-driven combination multigrade classrooms.

————————————————-

Do some Students Benefit More than Others from Multigrade Classrooms?

Although this was beyond the scope of my review—I did come across one recent study that dealt specifically with the question of how different types of students might be affected by a multigrade approach (Ong, Allison, & Haladyna, 2000).

First, it should be noted that in this one study, multigrade classrooms tended to outperform single-grade classrooms for all students, regardless of their background. Given that as background, the study found the following:

It found no differences between boys and girls.

It found that regular students seemed to benefit more from multigrade classrooms than disadvantaged students (that is, Title 1 students).

It found that overall non-Hispanic students seemed to benefit more from multigrade classrooms than Hispanic students.

The scarcity, frailty, and equivocation in the research make strong recommendations impossible. Instead, I offer these tentative ideas for consideration:

1. It is likely that the quality of the learning methods utilized and the teacher’s performance in the classroom makes more of difference than whether a multigrade or single-grade approach is utilized. Given this, it appears that choosing a multigrade approach would be acceptable, though the following points should also be kept in mind.

2. If a multigrade approach is utilized, then it should be utilized with due diligence—providing teacher support and development, utilizing team teaching, encouraging many diagnostic opportunities (so that learning can be tailored to learner’s current levels*), and so forth.

* Recent research indicates that teachers, tutors, and other learning professionals tend not to be very good at providing instructional explanations at learner’s appropriate levels—but that they can improve on this by specifically being informed of their learners’ level of understanding (Wittwer, Nückles, & Renkl, 2010).

3. Multigrade teaching seems to offer the possibility for alternative methods of learning, including group work targeted to different developmental levels. On the other hand, multigrade teaching by itself is no guarantee of good teaching methodology.

4. It should be recognized that multigrade teaching probably has costs associated with its use. Specifically, teachers may have to invest more effort and care in the process and may have a tendency to tradeoff other desirable educational goals.

5. In some forms of multigrade teaching, especially those that utilize team teaching, teachers have to learn the skill levels (in each discipline) of many more students. Because one of the most important aspects of teaching is providing learners with just the right level of instruction, this may cause students to be underserved at first as the teacher learns their students’ skill levels. One way to ameliorate this problem is for students entering a multigrade cycle to be diagnosed through testing, problem solving, and other performance metrics early on. This “slow-start” issue has a flip side—because multigrade classes stretch into two (or more) years, students in subsequent years will probably experience accelerated learning due to deeper understanding of each student by teachers and hence better instructional scaffolding.

6. Teachers with less experience may be especially unprepared for multigrade teaching. With experience, teachers learn how to automate aspects of their classroom performance so that they can utilize their limited working memory capacity to focus on supporting their learners in learning (for example, differentiating their instruction, etc.). Newer teachers will be unlikely to provide the necessary instructional scaffolding to perform at a high level.

7. Students who have behavioral problems or who are less competent in learning may make the task of multigrade teaching more difficult. This may suggest that extra effort and guidance in the early grades is warranted or that a multigrade approach should be delayed until a time when most students are ready to engage fully in the process.

Song, R. Spradlin, T. E., & and Plucker, J. A. (2009). The Advantages and Disadvantages of Multiage Classrooms in the Era of NCLB Accountability. Education Policy Brief, 7, 1-7. Published by the Center for Evaluation and Education Policy of Indiana University.

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. Also, there are hints of bias in the article, for example, (a) providing invited proponents of multiage classrooms to describe its value, without providing a similar counterpoint, and (b) talking about the lack of good research but then plowing ahead with a list of the “benefits (perceived and real)” of multiage classrooms—so in a real sense the article ignores the research by plowing ahead to benefits. Still, I wouldn’t discount this article completely. Its research review seems good when it is limited to reviewing the actual research. Also, it makes recommendations that are mostly consistent with the findings contained in this review, so I can’t dismiss their conclusions. The bottom line is that this review does not add anything to a review of the research, and, because it is not vetted in a refereed journal and appears slightly biased, I think it safer not to cite it as separate evidence.

Why not cited:
This article looked at only 4 teachers, all who had chosen to be teaching in multiage classrooms. It was really a descriptive research project and did not look at actual learning outcomes. No single-grade teachers were examined so we don’t really know how different results with single-grade teaching might be.

Why not cited:
This article utilized an incredibly biased research design. It took one classroom at one school that used multiage grouping and compared it on standardized exams with the national average.

Mariano, S., & Kirby N. (2009).Achievement of Students in Multigrade Classrooms Evidence from the Los Angeles Unified School District. From the RAND Education Working Paper Series (WR-685-IES).

Why not cited:
This article is not from refereed journal so it may not have been fully vetted. It also uses somewhat opaque statistical methods to estimate findings, not looking directly at actual comparisons between multigrade and single-grade classrooms. I must admit that I don’t fully understand all the statistical employments utilized (for example, “doubly robust regression,” “non-parametric generalized boosting,” “the Kolmogorov-Smirnov (K-S) statistic”). Thus, I may be missing the full implications of the research employed.

Research Benchmarking is the process by which your learning interventions are benchmarked against research-based best practices.

While random-assignment between-group research is likely to be too expensive and time-consuming for most of us, and benchmarking our work against other industry players is likely to push us toward mediocrity, Research Benchmarking offers a potent alternative.

Learning programs are examined to determine how well they (a) create understanding, (b) support long-term remembering (minimize forgetting), and (c) motivate on-the-job performance. They will be research-benchmarked against the 12 most decisive factors in learning design.

If you’d like to discuss research benchmarking further, contact me, Dr. Will Thalheimer, at 1-888-579-9814 or email me.

If you really want to know what Will Thalheimer’s up to—if you want to be the first to know when new research and information is revealed, if you want to be the first to get involved in Will’s Book Project—sign up here.

The following list of citations from the world’s best scientific refeered journals and publications are representative of the research articles that inform our work. The researchers cited below—although often unknown to practitioners in the learning-and-performance field—are heroes on the front line of learning and educational improvement.

Anderson, R. C., & Biddle, W. B. (1975). On asking people questions about what they are reading. In G. H. Bower (Ed.) The psychology of learning and motivation: Advances in research and theory (Vol. 9). New York: Academic Press.

Ericsson, K. A. (2009). Enhancing the development of professional performance: Implications from the study of deliberate practice. In Ericsson, K. Anders (Ed), Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments. (pp. 405-431). New York, NY, US: Cambridge University Press.

Guéguen, N., & Pascual, A. (2005). Improving the Response Rate to a Street Survey: An Evaluation of the “But You Are Free to Accept or to Refuse” Technique. The Psychological Record, 55, 297-303.

Gueguen, N., Pichot, N., & Le Dreff, G. (2005). Similarity and helping behavior on the Web: The impact of the convergence of surnames between a solicitor and a solicitee in a request made by e-mail. Journal of Applied Social Psychology, 35, 423-429.

Guéguen, N., Pichot, N., & Le Dreff, G. (2005). Similarity and Helping Behavior on the Web: The Impact of the Convergence of Surnames Between a Solicitor and a Subject in a Request Made by E-Mail. Journal of Applied Social Psychology, 35, 423-429. doi: 10.1111/j.1559-1816.2005.tb02128.x

Hamilton, R. J. (1985). A framework for the evaluation of the effectiveness of adjunct questions and objectives. Review of Educational Research, 55, 47-85.

Harp, S. F., & Mayer, R. E. (1997). The role of interest in learning from scientific text and illustrations: On the distinction between emotional interest and cognitive interest. Journal of Educational Psychology, 89(1), 92-102.

Dr. Will Thalheimer, President of Work-Learning Research, announces the winner of the 2009 Neon Elephant Award, given this year to Ruth Clark for her many years in leading the workplace learning-and-performance field with research-based insights and recommendations, and—by so doing—helping to professionalize our field.

Ruth Clark, EdD, a recognized specialist in instructional design and technical training, holds a doctorate in Educational Psychology and Instructional Technology from the University of Southern California. Prior to founding CLARK Training & Consulting, Dr. Clark served as training manager for Southern California Edison. She is past president of the International Society for Performance Improvement and author of six books and numerous articles. Dr. Clark is the 2006 recipient of the Thomas F. Gilbert Distinguished Professional Achievement Award from ISPI. Ruth is the author of numerous books, searchable at Amazon.com under her full name Ruth Colvin Clark, including:

The New Virtual Classroom: Evidence-based Guidelines for Synchronous e-Learning,

In addition to her lifetime of work, she is honored this year for the 3rd edition of her excellent book, published just a little over a year ago, Building Expertise: Cognitive Methods for Training and Performance Improvement. Although this is said to be the 3rd edition, the research cited is fresh and up-to-date. This book may be Ruth’s masterwork. It covers a wide swath of the learning research. It’s written by a research translator at the height of her powers. It’s a must-read (and must-study) for everyone in the field of workplace learning-and-performance.

It’s not easy to examine learning research from refereed scientific journals and compile it so that it is practical for others. The time commitment is incredible, the research skills must be of the highest caliber—and it requires guts and gusto. Some of what the research reveals cuts against the common wisdom. Sometimes it chaffs and brings angst and heat. Ruth’s continuing perseverance over the last three decades is testament to her passion and tenacity. Her work itself is testament to her integrity and skills.

I would imagine that over the last two decades there is no one in our field who has improved the work of as many instructional designers, trainers, and e-learning developers as Ruth Clark. For me, she continues to be a beacon—proof that research-based work is valued by our profession. For our field, Ruth’s work is simply indispensible.

Using evidence-based reasoning and recommendations is not just useful in practice. It is what respected, successful professions are based on. We owe Ruth Clark our most grateful thanks.

Dr. Will Thalheimer, President of Work-Learning Research, announces the winner of the 2008 Neon Elephant Award, given this year to Robert Brinkerhoff for developing the Success Case evaluation method and for advocating that learning professionals play a more “courageous” role in their organizations.

Robert O. Brinkerhoff, EdD, professor emeritus, Western Michigan University is planning his retirement from his full-time work as a principal consultant and partner at Advantage Performance Group, where he has worked since 2005. His clients include Anglo-American Corp., Bank of America, Pitney Bowes, the Federal Aviation Administration, Dell, and the World Bank. He is the author of numerous books, including Courageous Training (with Tim Mooney), Success Case Method, High Impact Learning, and Telling Training’s Story.

In addition to his lifetime of work, he is honored this year for the development of the Success-Case Method and for his advocacy that learning professionals play a more “courageous” and integral part in organizational performance. Too many of us play order-taker roles when we should be partners in helping our organization/business get results. Brinkerhoff’s insight that overly-complex methodologies are generally ineffective because they can’t be understood easily by stakeholders is one that more thought leaders should embrace.

The Success Case Method, while it can’t provide a complete picture of the training-impact landscape, is an important tool in any training-evaluation toolkit. It embodies two key insights. First, training doesn’t have to prompt all trainees to utilize training successfully to have a major impact on the organization. If one person implements one insight that nets the organization millions of dollars, the overall impact of the training may hinge on that one result—not the average or median result from all the learners. The second key insight embodied in the Success Case Method is the understanding that we ought to be diagnosing the cause of training failures and then working to fix those failures. By diving into deep case analyses of failure instances, we can uncover obstacles and forces that are limiting training impact. To learn about the Success Case Method, see Brinkerhoff’s book “Telling Training’s Story: Evaluation Made Simple, Credible, and Effective.” You can purchase it by clicking here: Telling Training’s Story: Evaluation Made Simple, Credible, and Effective

Brinkerhoff’s latest book, entitled “Courageous Training: Bold Actions for Business Results,” and written with Tim Mooney, build’s on Brinkerhoff’s years of experience in evaluating learning. He has seen how training succeeds and how it fails. He uses wisdom gained from these evaluations to lay out a comprehensive and practical process for going from needs-identification to results. Mooney and Brinkerhoff’s book challenges us as learning professionals to go outside our comfort zones to make true performance impacts. It is an important book in the mold of Wick, Pollock, Jefferson, and Flanagan’s “Six Disciplines of Breakthrough Learning.” You can purchase it by clicking here: Courageous Training: Bold Actions for Business Results (Bk Business)

Dr. Will Thalheimer, President of Work-Learning Research, announces the winners of the 2007 Neon Elephant Award, given this year to Sharon Shrock and Bill Coscarelli for advocating against the use of memorization-level questions in learning measurement and for the use of authentic assessment items, including scenario-based questions, simulations, and real-world skills tests.

Sharon Shrock and Bill Coscarelli have worked together for over 25 years building world-class expertise in the area of learning measurement and criterion-referenced test development. Their lifetime of work embodies the values of the Neon Elephant Award.

The third edition of their book Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training, released late this year, is a standard reference in the learning-and-performance field, having won two professional society awards as the year’s outstanding publication. Their workshops and consulting on Level 2 Assessment and Certification are based on sound methodology and real-world expertise.

In addition to their lifetime of work, they are honored this year for their paradigm-altering recommendation that memorization-level questions are inadequate for use in measuring learning as it relates to workplace competence. This recommendation is absolutely stunning given that current practice in evaluating learning relies, almost without exception, on memorization-level questions. That the most highly-regarded text on criterion-referenced testing advocates for this change—especially given its focus on legal liabilities—should send shockwaves through our industry.

Dr. Will Thalheimer, President of Work-Learning Research, announces the inaugural edition of the Neon Elephant Award, awarded for 2006 to Cal Wick of the Fort Hill Company for leading the development of the first commercially-viable training-follow-through e-learning software and his work as co-author of the book, The Six Disciplines of Breakthrough Learning: How to Turn Training and Development into Business Results.

Cal Wick embodies the values of the Neon Elephant Award. In 1999, he founded the Fort Hill Company because of his frustration with the way typical management-training programs produced virtually no impact when the learners returned to their jobs. Cal’s answer to this problem was the development of Friday5s—an e-learning platform that enables training results to be channeled to on-the-job implementation. By tracking learners’ progress toward goals and connecting the learners’ managers and learning administrators to the progress of implementation, Friday5s enables a revolution in how we think about training. No longer is training about information. No longer is it an event. With an online training-follow-through platform, implementation can become part of the training contract. Since the 1980’s the field has moved from training to performance improvement. Cal and his team’s invention provides a bridge between training and performance. It’s not one or the other. The goal is on-the-job performance. Now training can be outfitted to support that performance.

Cal and his team at the Fort Hill Company do more than create and market their products and services. By utilizing results from both research and practice and by honestly evaluating their own on-the-job results, they have been able to build a continuing cycle of improvement in their own work efforts. They’ve also compiled their own learning in The Six Disciplines book, published this past April (2006), making it easy for the rest of us to improve what we’re doing. In the book, which I’ve previously reviewed as “nothing short of revolutionary,” they’ve laid out a coherent and well-tested system for getting training results. It’s not just about training-follow-through software. It’s about an attitude and a complete methodology for getting business results. To get the 2nd edition of the book, click here: The Six Disciplines of Breakthrough Learning: How to Turn Training and Development into Business Results (Pfeiffer Essential Resources for Training and HR Professionals)

The Neon Elephant Award is awarded to a person, team, or organization exemplifying enlightenment, integrity, and innovation in the field of workplace learning and performance. Announced in December—during the time of year when the northern hemisphere turns away from darkness toward the light and hope of warmer days to come—the Neon Elephant Award honors those who have truly changed the way we think about the practice of learning and performance improvement. Award winners are selected for demonstrated success in pushing the field forward in significant paradigm-altering ways while maintaining the highest standards of ethics and professionalism.

Why “Neon Elephant?”

The elephant represents learning, power, strength, and the importance of nurturing the community. The glow of neon represents enlightenment, illumination, and a spark of something unique and alluring.

Selection Methodology

The award is based purely on merit and the criteria detailed above. Proposals are not accepted, nor are any entrance fees solicited or accepted. While advice on the selection is sought from industry thought leaders, Dr. Will Thalheimer of Work-Learning Research is the final arbiter. Awards will only be made in years when exceptional contributions to the workplace learning and performance field are apparent.

Winners

The 2019 Neon Elephant Award, given this year to David Epstein for writing the book Range: Why Generalists Triumph in a Specialized World, and for his many years as a journalist and science-inspired truth teller.

The 2018 Neon Elephant Award, given this year to Clark Quinn for writing a book debunking the learning myths, Millennials, Goldfish & Other Training Misconceptions: Debunking Learning Myths and Superstitions—and for his many years advocating for research-based practices in the workplace learning field.

The 2017 Neon Elephant Award, given this year to Patti Shank for writing and publishing two research-to-practice books this year, Write and Organize for Deeper Learning and Practice and Feedback for Deeper Learning—and for her many years advocating for research-based practices in the workplace learning field.

The 2016 Neon Elephant Award, given this year to Pedro De Bruycere, Paul A. Kirschner, and Casper D.Hulshof for their book Urban Myths about Learning and Education—a book that provides a research-based reality check on the myths and misinformation that float around the learning field.

The 2015 Neon Elephant Award, given this year to Julie Dirksen for her book, Design for How People Learn—a book that wonderfully conveys practical, research-based wisdom through the authentic voice of an experienced instructional designer and strategist.

The 2014 Neon Elephant Award, given this year to Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel for their book, Make it Stick: The Science of Successful Learning—a book that brilliantly conveys scientific principles of learning in prose that is easy to digest, comprehensive and true in its recommendations, highly-credible, and impossible to ignore or forget.

The 2013 Neon Elephant Award, given this year to Gary Klein for his many years doing research and practice in naturalistic decision making, cognitive task analysis, and insight learning–and for reminding us that real-world explorations of human behavior are essential in enabling us to distill key insights.

The 2012 Neon Elephant Award, given this year to K. Anders Ericsson for his many years conducting research on expertise and creating a body of knowledge that has inspired many others to translate his research into recommendations for use by performance-improvement professionals.

The 2011 Neon Elephant Award, given this year to Jeroen van Merriënboer for his many years conducting research on learning and translating that research into practical models for use by learning professionals.

The 2010 Neon Elephant Award was awarded to Richard E. Clark for his many years in leading the workplace learning-and-performance field by bridging the gap between academic research and practical application.

The 2009 Neon Elephant Award was awarded to Ruth Clark for her many years in leading the workplace learning-and-performance field with research-based insights and recommendations, and—by so doing—helping to professionalize our field.

The 2008 Neon Elephant Award was awarded to Robert Brinkerhoff for developing the Success Case evaluation method and for advocating that learning professionals play a more “courageous” role in their organizations.

The 2007 Neon Elephant Award was awarded to Sharon Shrock and Bill Coscarelli for advocating against the use of memorization-level questions in learning measurement and for the use of authentic assessment items, including scenario-based questions, simulations, and real-world skills tests.

The 2006 Neon Elephant Award was awarded to Cal Wick of the Fort Hill Company for his work developing methodologies and software to support learning transfer.