A job to do: Much as the Luddites feared mechanical looms 200 years ago, today’s middle-class workers have reason to worry that information technology erodes their employment prospects.

The United States faces a protracted unemployment crisis: 6.3 million fewer Americans have jobs than was true at the end of 2007. And yet the country’s economic output is higher today than it was before the financial crisis. Where did the jobs go? Several factors, including outsourcing, help explain the state of the labor market, but fast-advancing, IT-driven automation might be playing the biggest role.

Since the beginning of the Industrial Revolution, people have feared that new technologies would permanently erode employment. Over and over again, these dislocations of labor have been temporary: technologies that made some jobs obsolete eventually led to new kinds of work, raising productivity and prosperity with no overall negative effect on employment.

There’s nothing to suggest that this dynamic no longer operates, but new research is showing that advances in workplace automation are being deployed at a faster pace than ever, making it more difficult for workers to adapt and wreaking havoc on the middle class: the clerks, accountants, and production-line workers whose tasks can increasingly be mastered by software and robots. “Do I think we will have permanently high unemployment as a consequence of technology? No,” says Peter Diamond, the MIT economist who won a 2010 Nobel Prize for his work on market imperfections, including those that affect employment. “What’s different now is that the nature of jobs going away has changed. Communication and computer abilities mean that the type of jobs affected have moved up the income distribution.”

Erik Brynjolfsson and Andrew McAfee study information-­supercharged workplaces and the innovations and productivity advances they continually create. Now they have turned their sights to how these IT-driven improvements affect employment. In their new book, ­Brynjolfsson, director of the Center for Digital Business at MIT’s Sloan School of Management, and McAfee, its principal research scientist, see a paradox in the first decade of the 2000s. Even before the economic downturn caused U.S. unemployment to rise from 4.4 percent in May 2007 to 10.1 percent in October 2009, a disturbing trend was visible. From 2000 to 2007, GDP and productivity rose faster than they had in any decade since the 1960s, but employment growth was comparatively tepid.

Things Reviewed:

Race Against the Machine: How the Digital Revolution Is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the EconomyErik Brynjolfsson and Andrew McAfee October 2011

“The Growth of Low Skill Service Jobs and the Polarization of the U.S. Labor Market"David Autor and David Dorn June 2011

Brynjolfsson and McAfee posit that more work was being done by, or with help from, machines. For example, Amazon.com reduced the need for retail staffers; computerized kiosks in hotels and airports replaced clerks; voice-recognition and speech systems replaced customer support staff and operators; and businesses of all kinds took advantage of tools such as enterprise resource planning software. “A classically trained economist would say: ‘This just means there’s a big adjustment taking place until we find the new equilibrium—the new stuff for people to do,’ ” says McAfee.

We’ve certainly made such adjustments before. But whereas agricultural advances played out over a century and electrification and factory automation rolled out over decades, the power of some information technologies is essentially doubling every two years or so as a consequence of Moore’s Law. It took some time for IT to fully replace the paper-driven workflows in cubicles, management suites, and retail stores. (In the 1980s and early 1990s productivity grew slowly, and then it took off after 1996; some economists explained that IT was finally being used effectively.) But now, Brynjolfsson and McAfee argue, the efficiencies and automation opportunities made possible by IT are advancing too fast for the labor market to keep up.

More evidence that technology has reduced the number of good jobs can be found in a working paper by David Autor, an economist at MIT, and David Dorn, an economist at the Center for Monetary and Financial Studies in Madrid. They too point to the crucial years of 2000–2005. Job growth happened mainly at the ends of the spectrum: in lower-paying positions, in areas such as personal care, cleaning services, and security, and in higher-end professional positions for technicians, managers, and the like. For laborers, administrative assistants, production workers, and sales representatives, the job market didn’t grow as fast—or even shrank. Subsequent research showed that things got worse after 2007. During the recession, nearly all the nation’s job losses were in those middle categories—the positions easiest to replace, fully or in part, by technology.

Brynjolfsson says the trends are “troubling.” And they are global; some of the jobs that IT threatens, for example, are at electronics factories in China and transcription services in India. “This is not about replacing all work, but rather about tectonic shifts that have left millions much worse off and others much better off,” he says. While he doesn’t believe the problem is permanent, that’s of little solace to the millions out of work now, and they may not be paid at their old rates even when they do find new jobs. “Over the longer term, they will develop new skills, or entrepreneurs will figure out ways of making use of their skills, or wages will drop, or all three of those things will happen,” he says. “But in the short run, your old set of skills that created a lot of value are not useful anymore.”

This means there’s a risk, unless the economy generates new high-quality jobs, that the people in the middle will face the prospect of menial jobs—whose wages will actually decline as more people compete for them. “Theory says the labor market will ‘clear.’ There are always things for people to do,” Autor says. “But it doesn’t say at what price.” And even as it gets crowded and potentially even less rewarding at the bottom, employees at the top are getting paid more, thanks to the multiplier effects of technology. Some 60 percent of the income growth in the United States between 2002 and 2007 went to the top 1 percent of Americans—the bulk of whom are executives whose companies are getting richer by using IT to become more efficient, Brynjolfsson and McAfee point out.

Dramatic shifts have happened before. In 1800, 90 percent of Americans were employed in agriculture. The figure was down to 41 percent by 1900 and stands at 2 percent today. People work, instead, in new industries that were unimaginable in the early 19th century. Such a transformation could happen again. Today’s information technologies, even as they may do short-term harm to some kinds of employees, are clearly a boon to entrepreneurs, who now have cheaper and more powerful tools at their disposal than at any other time in history. As jobs are lost, Brynjolfsson says, “we will be running an experiment on the economy to see if entrepreneurs invent new ways to be productive equally quickly.” As examples, he points to eBay and Amazon Marketplace, which together allow hundreds of thousands of people to make their living hawking items to customers around the world.

The problem, he says, is that not enough people are sufficiently educated or technologically savvy to exploit such rapid advances and develop as-yet-unimagined entrepreneurial niches. He and McAfee conclude their book by arguing that the same technologies now making industry far more productive should be applied to updating and improving the educational system. (In one promising example they cite, 58,000 people went online to take an artificial-intelligence class offered by Stanford University.)

IT-based entrepreneurship isn’t the only potential technological driver of new jobs. Revitalizing manufacturing (see “Can We Build Tomorrow’s Breakthroughs?”) could also help. But automation has made manufacturing far less labor intensive, so even a manufacturing revival is not likely to mean a great many new jobs on balance. Likewise, anyone whose hopes are pinned on “green jobs” may be disappointed. Though jobs will be created in the switch to cleaner energy sources, jobs tied to traditional energy will be lost in the same process. Many economists are not certain what the net effect will be. And in any case, these days manufacturing and energy account for small slices of the U.S. economy, which is now driven much more by the service sector. That’s why fast-advancing information technologies, with their pervasive reach and their potential to create new services and satisfy new niche markets, may be a better bet for job creation—though the tumult IT is causing in the labor market isn’t necessarily going to resolve itself quickly.

Peter Diamond says that one of the most important things the government can do for employment is to take care of basics, like infrastructure and education. “As long as we have so many idle resources, this is the time when it’s advantageous—and socially less expensive—to engage in public investment,” he says. Eventually, he believes, the economy will adapt and things will work out, once again. “Jobs have been changing and moving around—within the country, out of the country—for a very long time,” he says. “There will be other kinds of jobs that still require people.”