Steve Jobs didn’t think that technology alone could fix what ails American education. It’s worth remembering that in the wake of last week’s breathless coverage of Apple’s new iBooks platform, which the company promises will radically change how students use and experience textbooks. Under Apple’s plan, companies and individuals will be able to self-publish textbooks, ideally creating a wider array of content. Students will be able to download and use these books on their iPad much like they would use a regular textbook — including highlighting passages, making notes and pulling out passages or chapters that are especially important to them. Apple says it also plans to cap the price of textbooks available through iBooks at $14.99, a significant departure from the price of many textbooks now.

Critics were quick to pounce that Apple wasn’t being revolutionary enough. Former school superintendent and current ed-tech investor Tom Vander Ark chided Apple for not thinking past textbooks, which he considers hopelessly 20th century. Others worried that Apple’s real goal wasn’t to open up the textbook industry but to control it and profit from it through restrictive licensing agreements and a platform that dominates the market. I’m sure the for-profit company’s shareholders will be horrified at that news.

Let’s slow down. Textbooks or tools that look a lot like textbooks aren’t going anywhere anytime soon. And since high quality educational material isn’t cheap to generate, simply tearing down distribution barriers will only go so far in reducing the costs of producing good content. Lost in the heated claims, however, is a more fundamental question: what have educational technology efforts accomplished to date and what should we expect?

As a field, education is easily seduced by technological promises. Textbooks? Thomas Edison saw movies as way to replace them. In a prelude to today’s debates, the phonograph and film strip were lauded as technologies that could replace live teaching. These days, conservatives are in love with the idea that technology will not only shrink the number of in-classroom teachers but render the teachers’ unions obsolete.

The experience to date is less grandiose and more worrisome considering the billions that have been spent on technology in schools in the past few decades. Interactive whiteboards have been around since the early 1990s and done little to transform how teachers teach, and computers are often unaligned with classroom instruction, even though 90% of classrooms around the country have them. Still, according to Department of Education data from 2009, just 61% of students use computers to prepare texts “sometimes or often” and just 45% do more complicated tasks, for instance to “solve problems, analyze data, or perform calculations” on a regular basis.

Usage aside, there is scant evidence that technology is improving learning — even the cheerleaders are reduced to arguing that various education technology tools are obvious rather than supported by much evidence. And when you watch, say, high school students use the Internet to prepare research papers, it’s questionable whether technology — especially when coupled with poorly trained teachers — isn’t doing more to enable the superficial rather than open up richer veins of information for students.

The reasons for the slow pace of change are as obvious as they are stubborn. Altering classroom and school practice in our wildly decentralized education system is always a slow process. Many teachers are not familiar with technology or how to use it in the classroom, and high-quality training programs — either in schools of education or as part of a teachers’ ongoing professional development — are rare. As always, there are few guides for educators to determine which products are any good.

There is, of course, still promise in education technology. When Dreambox Learning, an online math program for elementary-aged students, offered me a free trial to check it out, I did what I usually do with new educational tools — I put it to the ultimate panel of critics: my kids. Dreambox, which just this week announced a new series of lessons aligned to the nascent Common Core standards and free licenses for every school in the country, combines real content with an interactive format so kids are learning even when they think they’re just playing games. I’ve looked at a variety of products, and it’s one of the best in terms of powerful instruction. In a short time, it substantially boosted my kids’ math achievement. (They have a great teacher, too.) As for engagement? Maybe too much. One of my daughters woke me up at 5 am the other day because she wanted to do math.

Yet even a top-shelf product can only augment live teaching. Despite Dreambox’s overall good functionality, there are places where students can become frustrated — not because they don’t know how to do the underlying math, but because the directions for the online activity are confusing. Likewise, technology is bringing back in vogue the idea of the “flipped classroom” with the teacher acting as a “guide on the side” rather than the primary source of instruction. I say back in vogue because, ironically, talk of devaluing the teacher as content provider has been a fixture of progressive education thought for a century. Another variation of the flipped-classroom idea is to use technology to explain concepts at home and use classroom time differently. Again, a lot of potential, but only with keen attention to instructional quality. Much of the online content available today merely replicates the lame instruction already available in too many of our nation’s schools.

As a parent and an analyst, I want technology that includes rich content or enables students to access it. And I want technologies that are engaging for students but actually teach them something. Plenty of applications err on one side or the other. And as with lots of offline schoolwork, there are time wasters that aren’t helping anyone learn much of anything. If anyone tells you an ed tech tool has “gaming elements,” make sure it’s not just a game.

American education desperately needs an overhaul that goes far beyond upgrading computers in the classroom. It’s the last major American field relatively untouched by technology. But Jobs was right: technology by itself won’t fix what ails our schools. He saw teachers’ unions and archaic practices as the big barriers. Perhaps, but I’d argue they are symptoms of our larger inattention to instructional quality. The bells and whistles of technology, for all its promise, are distracting us from this mundane but essential reality.

Rotherham is a co-founder and partner at the nonprofit Bellwether Education, a national non-profit organization which, among other activities, does paid consulting work for clients including school districts, charter schools, and educational organizations around the country. The views expressed are solely his own.