Blended Learning Improves Test Scores: Study

High school students who used algebra software to supplement traditional classroom lectures did better on tests than those who didn't in two-year study by Department of Education, RAND.

8 MOOCs Transforming Education

(click image for larger view and for slideshow)

A new study by the RAND Corporation and the Department of Education gives new credibility to the popular notion that blended learning -- a combination of traditional classroom methods with computer-mediated activities -- can improve students' test scores.

The study was one of the largest of its kind, involving 18,000 students in 147 urban, suburban and rural high schools and middle schools in seven states. It demonstrated the success of blended learning in a high school algebra curriculum.

To randomize the study, schools were matched into similar pairs based on achievement profiles and test scores. Half were assigned to retain their normal classroom-based algebra curricula. The other half used software from Carnegie Learning, Inc., called Cognitive Tutor Algebra I (CTAI), to supplement their teachers' instruction.

The program is designed to develop proficiency in algebraic concepts such as linear equations, linear and quadratic expressions, proportional reasoning and data analysis. As students solve multi-step problems, the software tracks their strategies and provides feedback for improvement.

Teachers in experimental classrooms received four days of CTAI program training and were given time to prepare curricula. To measure the effectiveness of the study, students were given a pretest and post-test at the beginning and end of each school year.

Over the two-year study, students in CTAI-based classrooms spent two days per week using the software, while teachers worked with individuals as needed. Three days involved group work and problem-solving without the software. Teachers made the software available to students for out-of-school use, though the study didn't record this use.

In its first year of implementation, CTAI did not prove successful among middle school or high school students. In fact, the high school study showed negative progress; the middle school study showed no progress.

After the second year, however, test scores improved significantly. To understand the progress of CTAI in its second year of implementation, consider a high-school student who would typically score in the 50th percentile after a year of a typical algebra curriculum. Given the results of the study, this same student would score in the 58th percentile after a year using CTAI.

That eight-point jump is significant; however, RAND cautions that it can't be determined whether it's due to the software or the overall program structure. Teachers in the experimental group, who used CTAI, reported using more progressive practices such as group work and student presentations. Instructors in the control group, who did not use CTAI, continued using traditional methods such as lecture-based sessions.

"Teaching was different in Carnegie Learning classrooms than in the control group classrooms; we can say that," said John Pane, senior scientist at RAND and a lead researcher in the study. "We can't tell if it's the software or something happening in the classroom."

RAND hopes to conduct another study in the future to focus on the direct impact of CTAI software.

Recent test scores in the U.S. prove the need for an improved math curriculum. In 2011, the report states, only 35% of eighth-grade students demonstrated basic mathematical proficiency or higher. Algebra is an area of math that often causes students to fall behind, yet is also a gateway course to advanced math and science courses.

Sorry, David, I just saw this. Based on the results of the study, I would say that it was a combination of the Carnegie software and the blended learning technique that boosted test scores. While the software helped, it's also worth noting that teachers in blended learning classrooms encouraged more student collaboration and hands-on work, which could have also contributed to students' success. The teachers in traditional classrooms maintained a lecture-based instruction model in which students primarily took notes. It will be interesting to keep tabs on the upcoming study that will focus solely on the benefits of the software, to see exactly how much it benefits students.

Among 688 respondents, 46% have deployed mobile apps, with an additional 24% planning to in the next year. Soon all apps will look like mobile apps – and it's past time for those with no plans to get cracking.