Adaptive Learning

Can industry’s most provocative software make a difference in the classroom?

Yes! A detailed report of insights, data and frameworks. We're putting the finishing touches on this PDF wonder. Add your info here and we will send it to you as soon as it's live. Get ready!

Meet Aaron Cheng, my daughter’s sixth-grade math teacher. He’s a smart, technically savvy 28 year-old at the Alameda Community Learning Center, a progressive charter school just fifteen minutes from the tech mecca of San Francisco. I asked him the other day if ACLC was thinking of using any adaptive learning software.

“What’s that?” Cheng asks.

PETER BURROWS

journalist

Why is adaptive learning important?

90 sec

Thirty five miles south at Joseph Weller Elementary School in Milpitas, everyone knows about adaptive learning. When EdSurge reporter, Paty Gomes, and I visit, third graders are sitting on bright red plastic chairs in an expansive, airy learning lab, each quietly reading a book they selected from Reading Counts, an adaptive program that suggests titles that can help them improve in a certain area—say, vocabulary. So many reporters and educators have visited this cutting-edge lab that teacher Diane Semrau doesn’t bother to introduce the camera-laden visitors, and the kids could hardly care less. After they go off to lunch, district superintendent Cary Matsuoka and director of technology Chin Song sit for an interview, reeling off information about the program.

For us, the decision to use adaptive technology was about helping underachievers catch up.

Of the two, Joseph Weller is simultaneously more—and less—like most schools throughout the US. As a traditional public school, it serves a staggering array of students with diverse needs. Forty percent are English language learners, compared to just 11 percent at ACLC, where students come from a more homogenous upper-middle class community. Most of Joseph Weller students, who between them speak 10 different languages at home, live in cookie-cutter 1,100 square foot homes built by the Ford Motor Company early last century. Roughly 60 percent of the students perform at proficient or better levels, according to California standards, compared to 80 percent at ACLC.

“For us, the decision to use adaptive technology was about helping underachievers catch up,” says Matsuoka. “And it was about helping kids take responsibility for their own learning. It’s about student agency.”

At schools like ACLC, where most of the kids are doing just fine, there’s less pressing need for the technology—but plenty of nervousness about the problems it could create. When I describe the software to ACLC’s Cheng, he stops to consider. “Well, I’ve only been teaching for three years, so I’d be willing to try. But I think a lot of teachers wouldn’t want to change their ways.” Pausing again, he adds, “And I’d want to see evidence that this stuff really works.”

Changing teaching practices. Lack of evidence. Throw in tight budgets, privacy concerns about software that creates a digital record of kids’ performance, and questions about the financial viability of some of the companies that make these tools.

These are some of the issues at the core of whether what’s arguably the most controversial and yet the most tantalizing of the software to emerge in the past decade’s pre-Cambrian explosion of education technology will make a difference in the lives of teachers and students.

Director of technology Chin Song (left) and superintendent Cary Matsuoka of Milpitas Unified School District | Photo Credit: Paty Gomes/EdSurge

So far, adaptive technology (which we define here) has touched only a fraction of America’s K-12 students—maybe 20 percent, based on an informal poll of educators and entrepreneurs. Yet it attracts attention because it takes aim at several fundamental questions: Can we create a way to deliver content that keeps kids more engaged than the classic textbook? How much does the order in which concepts or skills are taught, or “sequenced”, matter? How do we use testing—or assessment—not simply to rank students but as meaningful windows into why they struggle to learn? And the big one: Can changes in digital curriculum help close the aching achievement gap?

The need to tackle these issues cuts deeper daily: The students entering America’s classrooms come from more diverse backgrounds and bring a wider set of needs and abilities than ever in history. By contrast, funding for schools grows modestly at best. In most segments of life, when we’ve tried to do more with the same (or fewer) resources, we’ve invented tools to help.

Well, I’ve only been teaching for three years, so I’d be willing to try. But I think a lot of teachers wouldn’t want to change their ways.

But like so many bright and shiny technology promises, adaptive learning has yet to offer any definitive answers, despite decades of work. Both industry and teachers are even wrestling with exactly what will constitute the “evidence” that so many educators crave. If there’s scant proof that these tools raise test scores, is it worth doing if it makes students more enthusiastic learners, or if it frees up teachers to spend more time teaching to smaller groups? These questions unnerve many, including parents who don’t want their children to get an inferior education as schools work out the kinks in new technology, and school district leaders, who are loath to champion risky projects that could get them in hot water with the school board or on the front page of the local paper.

No one, even that most evangelical proponent of technological change, former Harvard Business School professor Clayton Christensen, promises change will be easy. New technologies, Christensen has observed, are typically inferior to existing ones—until people change the way they work. And so exploring what “adaptive learning” might mean in education has pulled a small group of educators, business and philanthropists into a tentative and at times, awkward dance. Here’s what they are doing—and what they’re learning along the way.

Changing practices

Some of the most widespread education technologies did not demand much change in teachers’ practices. Digitized books were, after all, just books in a different medium. PCs and now Chromebooks provided an electronic replacement for typewriters and paper and pencil, but didn’t immediately revolutionize the classroom. Adaptive learning does not fit easily into the status quo. Besides having to use a blended learning model, in which class-time is divvied up between traditional and electronic learning, teachers must be willing to let students progress at their own pace. They need to be comfortable letting software make real decisions about what students should learn next and use quantitative data on student performance gathered by the software along with their own qualitative gut instincts. They need to be willing to trade the stand-in-the-front-of-the-room-and-lecture model, and instead provide more intimate, personalized instruction to whichever kids aren’t on computers at the moment.

You have to trust that each student is working hard, and working at their top level.

At Aspire ERES Academy in Oakland, CA, students spend up to a quarter of their day—50 to 80 minutes in total—using online tools including ST Math and i-Ready. Like the Milpitas public schools, Aspire Public Schools, which operates 38 schools in California and Tennessee, saw adaptive technology as the most efficient way to achieve its goal of college-readiness for kids in low-income populations. At ERES Academy, 99 percent of the kids are English learners.

Besides the obvious logistical challenges of a blended classroom, such as setting up rotations for kids to cycle from teacher time to computer time, using adaptive learning tools require other changes. Every Friday, second-grade teacher Mark Montero has fifteen to thirty minute “data talks,” when the kids talk about their progress and the problems they ran into using the adaptive products. Kids who are doing particularly well are named “student coaches.” Montero makes a list of who is struggling with what, and assigns one of the coaches to spend the last ten minutes of their 30-minute rotation helping one of their classmates get past the hurdle. “Kids need to discuss what they’re doing on the computer,” he says.

Adaptive technology requires a different sort of trust between teacher and student. “You have to let go of some of the micromanagement,” says Montero.

“You have to trust that each student is working hard, and working at their top level,” rather than idly clicking to sandbag the time away. To keep them honest, there’s a drawing of a traffic light in front of every student’s Chromebook. If Montero thinks a student is slacking off, he shines a laser pointer at the yellow light. If he has to come back and shine it on the red light, the child has to get off the computer. The “Are You Still There” screen saver in i-Ready that pops up after a few inactive minutes is also a mark of shame for a student. And of course, Montero can always check the extensive data stored by these tools, including how many lessons students have passed and how much time they put in.

Getting ready to use an adaptive learning tool

First: all students participate in a reading sessionSecond: they are organized into two groupsThird: traditional and digital instruction coexist

Getting ready to use an adaptive learning tool

First: all students participate in a reading sessionSecond: they are organized into two groupsThird: traditional and digital instruction coexist

1/3

At the same time, teachers can’t get too enamored with the technology. When Weller Elementary first rolled out i-Ready, “we made the mistake of thinking data was the holy grail,” says superintendent Matsuoka. At first, the plan called for teachers to stay in the classroom while their students went to the learning lab. This would work because the data generated by the adaptive tools would inform the teacher’s lesson plans. “Let’s just say that was a bad assumption,” Matsuoka says. He won’t share details, but admits the district learned the hard way that results don’t improve when teachers spend too much of their lesson-planning time mining data. “Data is important, but it’s not the most important thing. The most important thing is quality teaching,” he notes.

And quality learning. While there isn’t definitive proof of a link between adaptive learning and better mastery, Matsuoka says he has no doubt it has helped close the achievement gap for many struggling students. It’s certainly helped with behavior: the number of suspensions at Weller fell from 50 to zero in the year adaptive learning technology was rolled out. Just as gratifying, Matsuoka says, is watching gifted students race ahead, unshackled for the first time in their school careers. “Some kids go slower and some kids go faster,” says nine-year-old Arielle Talucod, a calm, confident third grader who was tapped to give demos of the various programs she uses. She’s clearly in the faster group. While many of the kids use points won for the number of books they read to unlock little video games, she evidently doesn’t have time to waste. “I’m saving up,” says Arielle, who is clearly a long-term planner. She hasn’t decided yet whether to be an astronaut, or President of the United States.

Can Districts Adapt?

Baltimore County Board of Education building | Photo Credit: Austin Kirk/Flickr, licensed under CC BY 2.0

PETER BURROWS

The challenge of implementing adaptive tools in big districts

80 sec

Adaptive learning also requires major changes from the districts and school administrators that want to go down that path. One of the largest deployments currently underway is in Baltimore County, where the public school system began rolling out adaptive tools including DreamBox and i-Ready last autumn after four years of exhaustive planning.

With 175 schools and 111,000 students, the BCPS team spent 18 months doing hundreds of interviews with teachers, parents, local businesses, community groups and others.

The decision to make adaptive learning technology a key part of Baltimore’s Students and Teachers Access Tomorrow, or STAT initiative, was driven to a large degree by the desire to ensure an equitable education to children from economically-diverse communities. “Adaptive technology can help ensure that kids aren’t penalized because of their zipcode or their race or what school they happen to go to,” says Christina Byers, executive director of leadership development for elementary schools in the district.

Preparing the district for the new approach also took time. First, the county restructured the curriculum so it would work in the new blended model, and set the groundwork for the project by defining a lexicon of terms, to fill the gap left by an industry that tends to lump everything from narrow assessment tools to sweeping “intelligent platforms” under the term “adaptive learning.” The district then introduced the products at a few “lighthouse” schools, starting with lower grades, before rolling anything out too broadly. Meanwhile, there were many levels of training, starting with administrators and principals and moving down to teachers. One teacher at each school is trained to be a STAT Teacher, to be a local resource when problems arise.

We may have first graders and fifth graders who all need access to third grade curriculum.

Adaptive learning—in spite of the buzziness of the term—is just a sliver of what it means to add technology to the classroom. BCPS plans to have a digital device—a hybrid laptop/tablet from Hewlett-Packard—for every student by the 2018-2019 school year. The district also created a steering committee to manage and coordinate the progress of eight different “conversions” that all need to happen in lockstep, including new kinds of curricula and the computer networks and teacher training to make it effective. While the goal is to give more control to the students in how they want to learn, adaptive tools had to integrate cleanly with the overall learning management system, so teachers would be able to use them in combination with other digital and non-digital resources to help a specific student. The county also implemented ways that teachers could easily let the entire district know when bugs crop up, or recommend ways to change a program’s user interface to make it easier to use.

If Baltimore County is any guide, providers of adaptive technology products will undergo some big changes as well. The district developed a detailed process for would-be vendors. Baltimore County hired the Center for Research and Reform in Education at Johns Hopkins University to do an extensive evaluation of the STAT initiative. As a part of the assessment, the Center would appraise appraise the effectiveness of the digital instruction tools as well as the changes they bring to teaching. “We’ll sit through the sales presentations, but we know that building credibility in our district means showing our schools and teachers that something actually works,” says Dr. Renard Adams, executive director of performance management and assessment at BCPS.

There was an era in edtech when technologies were difficult to incorporate into classroom practice. We’re going very quickly past that.

That was just the start. Big established players wanted the district to buy their full suites of tools and courseware, “but we wanted to take an ‘iTunes’ approach. We didn’t want to buy the whole album—just the songs we wanted. That blew their minds,” says Jeanne Imbriale, director of enterprise applications in the BCPS IT department. Many smaller companies couldn’t—or wouldn’t—meet the district’s demand for extra staffing of support desks in the peak hours before and after lunch-time, or to have staffers on call after hours and on weekends, when many teachers are going through the data collected on student activity. “There was a lot of inflexibility on what many of the companies would and would not do,” says Byers.

DreamBox is one of the companies that ran that gauntlet. It agreed to provide extra support and tweaked its licensing model. Rather than sell products on a per-grade basis, it agreed to a more flexible approach. After all, the point of adaptive learning is that every student can progress at his or her own pace. “We may have first graders and fifth graders who all need access to third-grade curriculum,” says Byers.

DreamBox CEO Jessie Woolley-Wilson confirms that the company made these modifications and others to boot. Since the company first started selling to schools in 2011, it has added more and more features as teachers have accustomed themselves to the program and aligned their instruction with it. “Now we call it an ‘intelligent adaptive platform,’” Woolley-Wilson says, emphasizing that teachers can use DreamBox to create their own lesson plans. “There was an era in edtech when technologies were difficult to incorporate into classroom practice. We’re going very quickly past that.’”

From Practice To Evidence?

But how can a district or teacher know if software is effective or not? There is very little, if any, definitive academic research proving that adaptive software improves grades or mastery.

Even districts that want to move forward to get other benefits—say, to off-load some lessons so teachers can spend more time with smaller groups of kids—are hard-pressed to find credible product evaluations or consultants that can advise them on how to procure, deploy and make use of the software.

Chicago-based LEAP Innovations was founded in 2014 to help fill these gaps. The nonprofit is staffed with plenty of edtech veterans, including former teachers frustrated with how difficult it was for them to find and get results from adaptive and other edtech products. It has a 5,000 square foot “Collaboratory” at its office where educators, academic researchers and company representatives can work together.

LEAP also runs the Pilot Network Program—the nonprofit is now recruiting 20 schools for the third class of participants—designed to pilot and evaluate personalized learning technologies and practices in real classrooms, with the goal of sharing and scaling what works. “There were a lot of products out there, but without any research behind them,” says Chris Liang-Vergara, LEAP’s chief of learning innovation.

There were a lot of products out there, but without any research behind them.

To be included in the Pilot Network, companies have to complete a lengthy application with details on everything from the design of the product to the IT requirements to existing efficacy research. Schools that want access to promising products must also show they have assigned the right people and have the right philosophy to be successful. They also must take a series of workshops in the Spring to prepare them to implement personalized learning with and without technology. As schools become more familiar with the new concepts, LEAP introduces them to the types of available products. All the accepted products are invited to a Match Day in May, where schools pick tools they want to investigate and possibly pilot.

So far, LEAP has hosted two Match Days, that have included 17 vendors and 29 schools or districts. From these, schools have wound up working with 13 vendors on pilot projects that last through the following academic year. There are two escape hatches for schools, at the ninth week and eighteenth week of the program, when they can back out if they think the product is too far from prime time.

Vendors tend to learn a lot through the piloting process. For instance, some fledgling tool providers may make arbitrary recommendations for how long their product should be used each week to help students. “A lot of [products] were developed in a very closed environment,” says Liang-Vergara.

Building A Business

Such matchmaking is key in a market that’s been slow to get off the ground. Despite decades of work, industry revenues for adaptive learning software were only around $200 million as of 2012, the last time Kate Worlock, an analyst with the UK-based market research firm Outsell, tried to size the market. The biggest portion of that has gone to New York City-based, Knewton, which has aggressively marketed its adaptive learning platform to deliver math, English and biology courseware from a variety of publishers.

Our growth is a hopeful sign that there’s more interest from school districts and more willingness to try new things.

Other firms say they are starting to get traction. Boston-based Curriculum Associates, maker of i-Ready, has three million active users. Pearson’s SuccessMaker also has approximately three million users. DreamBox, a ten-year-old company based in Bellevue, Wash., was used by 1.5 million students last school year, up from zero when it began selling to schools in 2011. “Our growth is a hopeful sign that there’s more interest from school districts and more willingness to try new things,” says Woolley-Wilson.

Some of edtech’s biggest players are evidently hearing the same thing. For years, companies like Houghton Mifflin, Pearson and McGraw-Hill used their market dominance to protect multi-billion dollar textbook businesses. Now, they are moving more aggressively to embrace innovative technologies, including adaptive learning. Houghton Mifflin bought Scholastic’s tech unit, and Pearson has sold off the Financial Times and the Economist to focus more on education technology.

In a market that’s been characterized by a few slow-moving giants surrounded by swarms of under-sized small fry, McGraw-Hill’s recently spun-off education unit seems well-placed. It’s been selling adaptive products, some developed in-house and some via acquisition, for years. More than four million students use products such as SmartBook and ALEKS, a math product that has been in development for 20 years, says chief digital officer Stephen Laster.

1/7

With $1.4 billion in revenues and $1 billion in profit—and an initial public offering probably in its future—McGraw-Hill can afford to make the heavy investments needed to help schools successfully implement adaptive learning, contends Laster.

“It’s important to have companies with sustainable business models, who can afford to attract and compensate great talent,” he says. “We’re hiring a lot of people who are tired of chasing ad revenue for Google or Facebook, and want to come here and push the advancement of learning. But it’s expensive.”

This is a ‘some profit’ business, somewhere between ‘non-profit’ and ‘for-profit.’

McGraw-Hill’s technology philosophy is also likely to be attractive to parents, educators and others who worry about students’ privacy. Some companies—notably Knewton—collect vast amounts of data, including not only answers to questions but information on how often they clicked and where. McGraw Hill uses a “small data” approach that employs algorithms that analyze only the student’s most recent clicks to determine which question to ask next.

“Let’s not overcomplicate this—and try to do no harm,” says Laster, who says the goal is to share no more information than a teacher would get in the analog world. “I don’t need to know everything about your child.”

McGraw-Hill is not the only company paying attention to data privacy. “Having access to data brings responsibility,“ says Johann Larusson, who leads Pearson's Center for Digital Data, Analytics & Adaptive Learning. "Everything we build at Pearson is focused on delivering the best outcome for a student, and we only use the data needed to achieve that.“

Maybe the best news of all is a new realism that’s starting to infuse the adaptive learning community. There have been no break-out hits, from an investor perspective. That’s weeded out many potential investors and entrepreneurs, as the realization hits that this promising type of technology isn’t likely to spawn the next Silicon Valley “unicorns” with $1 billion valuations.

And nobody talks about technology replacing teachers anymore, or even about the ability of technology to raise test scores on its own.

“It all boils down to good teachers, good students, good parents and good principals,” says Dr. Steven Ross, senior researcher and professor at the Center for Research and Reform in Education at Johns Hopkins. “Without that, some little software project isn’t going to make that much of a difference.”