Next to the M56, on the outskirts of Manchester, the future has landed. A cluster of huge distribution centres sits at the heart of Airport City, a new development part-funded by the Beijing Construction Engineering Group (two years ago, it was visited by president Xi Jinping of China). Among the biggest buildings is one of Amazon’s self-styled “fulfilment centres”. Known within the company as MAN1, it opened in September last year, but everything inside, from the chairs to the wall-mounted screens, looks as if it has just come out of a box. Deeper within the centre, beyond the reception area and meeting rooms, there is something else just as new: a great expanse of space behind a metal cage, where dozens of robots, finished in Amazon orange and each emblazoned with its own number, glide across the floor, gracefully avoiding collisions and sprinting to their next task.

Amazon employees call them “drives”, but to all intents and purposes these are droids, summoned from the dreams of science fiction and put to work. In some Amazon warehouses, workers – or, in the company’s parlance, “associates” – still pace up and down huge aisles, picking out goods and preparing them for shipment; these shifts are said sometimes to involve hikes of 11 miles. But here everything moves much more quickly. The humans in charge of the process known as “picking” now remain in closed workstations, built around a screen that tells them what they need to get next, while the robots bring the shelves – reinvented as four-sided fabric towers, full of pouches that contain everything from DVDs to dolls – to them.

There are tasks only a human can do, such as the careful packing of boxes. But for how long?

The robots lift the towers using their most obvious feature: a big black cylinder that can withstand a serious amount of weight and which locks on to the shelves using a corkscrew action. They navigate by following barcodes stuck to the warehouse floor, and have a front-mounted camera that ensures they don’t collide with each other. They will pause for five minutes of recharging every hour, and get their instructions from software that runs on a cloud server.

Today, these robots are under the command of Sean McFadzean, a 26-year-old with a background in electrical engineering. He has worked here for a year: his corner of the building is effectively a robot garage, replete with a mixture of screens and tools. “The whole thing has just gripped me,” he says. “It’s fascinating.”

Does he have a favourite robot? “I actually do. 64117. There’s a kind of leaderboard system that tracks each drive unit, and I follow them all. 64117 has travelled only 164 metres the whole time it’s been here. It’s the laziest drive we’ve got. It’s got the work/life balance worked out.”

The introduction of robots to Amazon warehouses in the US and Europe has released huge amounts of space once taken up by the storage facilities built around people; it is also said to have cut the time it takes to complete an order from more than an hour to 15 minutes. It has been estimated that when Amazon customers now receive a package from a centre such as this, its preparation will have involved no more than a minute of human work. A great deal of Amazon’s nitty-gritty operations are increasingly done by software and machines – and the company is by no means alone.

In his bestselling book Homo Deus, the Israeli historian Yuval Noah Harari makes a simple point about why the future of work may differ drastically from its past. Since the Industrial Revolution, he writes, “As old professions became obsolete, new professions evolved, and there was always something humans could do better than machines.” People in developed countries went from “fields and flocks” to industrial jobs, and then into service industries. “Yet this is not a law of nature,” he points out, “and nothing guarantees it will continue to be like that in the future.” With many jobs in service industries on the verge of being automated away, what new work will there be for the millions of people who currently do them?

There are two broad schools of thought about what is happening to paid employment in the 21st century. One is essentially optimistic. The optimists cite the way that, although technology always does away with some jobs, it usually creates others. If the Industrial Revolution was bad news for weavers, it also led to millions of people working in hitherto unimagined occupations, from train driving to rocket science. With some justification, the optimists also tend to evangelise about the liberating prospect of people being freed from some of the most monotonous, soul-destroying work.

The other view is more downbeat, and points to the possible dawn of what Harari describes as “the useless class”: large numbers of people who will have no economic value. In this view of things, we need to accept that if paid employment will soon be in much shorter supply, we have to come up with radical answers – starting with the introduction of a universal basic income.

In March this year, a report by the consultancy firm PricewaterhouseCoopers (PwC) found that more than 10 million workers in the UK are at risk of having their jobs rendered obsolete by automation in the next 15 years; 2.25m of them currently work in retail and distribution. “For those with just GCSE-level education or lower, the estimated potential risk of automation is as high as 46% in the UK,” said the report, “but this falls to only around 12% for those with undergraduate degrees or higher.” Clearly, automation is tangled up with inequality, and if we’re not careful, the gaps that increasingly separate different kinds of worker will soon get even bigger.

It’s not that the robot replaces the person. New possibilities open up

Amazon’s senior staff tell me the company now employs more than 500,000 people around the world (up from just over 300,000 a year ago, and a figure that excludes seasonal workers). They also insist human beings will always be central to what they do. Further up the hierarchy, software engineering, machine design and all kinds of innovation depend on human ingenuity and hard graft. Meanwhile, plenty of tasks in fulfilment centres, they say, are still too complex for robots: checks on quality, the quick assembly of cardboard boxes, the careful packing of items.

But for how long? For the past three years, the company has organised a competition, the Amazon Robotics Challenge, in which entrants have to come up with a robot that “identifies objects, grasps them, and then safely packs them in boxes”. This year, £60,000 in prize money was won by a team from the Australian Centre For Robotic Vision in Brisbane, who invented Cartman, which uses “suction cups and a two-fingered claw to grasp and manipulate items”; one of the team was swiftly hired. Meanwhile, Amazon’s experiments with drones are well known (it officially launched a drone-based delivery method called Prime Air in 2013, though it is unlikely to become a reality until 2020 at the earliest). The company is also said to be researching the use of driverless vehicles.

“We’re taking the kinds of techniques and processes that have been long established in manufacturing and applying them to a service industry,” says Roy Perticucci, Amazon’s European vice-president for customer fulfilment. “That causes a reallocation of resources, and it’s really a third or fourth wave of industrialisation: it started in manufacturing, and it continued through to everything else.”

Amazon got its droids, Perticucci tells me, by paying $775m for the acquisition of Kiva Robotics, the Massachusetts-based firm that invented them, in March 2012. “We were so excited by the idea that we bought the company,” he says. The system they sit at the heart of is now in operation at 25 US distribution centres, along with a handful in the UK, Poland and Germany.

It is all about the elimination of waste and “dead time”, Perticucci explains, and the importance of flawless customer service. He traces what Amazon is doing to the kind of factory automation with which we have all been familiar for years: robot arms picking and prodding at assembly lines, while human beings see to the tasks that require a bit more thought.

Like Harari, Perticucci uses the example of the Industrial Revolution, but with a positive spin: “In the end, almost everyone involved was far better off, and the quality of whatever was being provided was better than what had been there before.”

Clearly, though, if I now buy a book or a record from Amazon, rather than a shop, there is much less human effort involved.

“Er, I would say that, in the most valuable portion of their creation, which is writing the book or making the music, I think that’s about the same.”

Of course. But what about production, and getting the book or the record to people?

“Sure. There are probably fewer people involved than there were before. But this is not a zero-sum game. And industrial history over the last 100 years or so has proved that this creates more opportunities, not fewer.”

What does he make of the increasingly influential idea that a great wave of robotics, artificial intelligence and automation is about to make millions of us unemployed?

“Look, let’s go back,” Perticucci says. He mentions “the thresher rebellion”, which I think is a reference to the Swing Riots of the 1830s, when farm workers in the south of England violently protested against the mechanisation of agriculture. “They taught me all that in history class,” Perticucci says. “There were people who would no longer be threshers because they invented the threshing machine. But those people ended up doing other things. And by the way, in the process, instead of dying at 35, they made it to 45 or 55. I think that’s true now as well. Very few of us are still manual workers. What is happening now is just a natural continuation of something that’s been happening for a long time.”

***

In Hatfield, the online supermarket Ocado runs two very different operations on either side of a ring road. One is a vast distribution centre, where many of the workers seem to arrive either on foot or by bike, dressed in voluminous clothing to guard against the elements. The other is the HQ of Ocado’s tech division, staffed by people in their 20s and 30s dressed in casualwear, who clutch takeaway coffees and fidget with their phones.

As well as its own brand, Ocado manages orders for Waitrose and Morrisons. A lot of online shopping is still based on distribution centres designed along the same lines as a conventional supermarket, so workers pace up and down endless aisles and fill trolleys. Here, things could not be more different. The systems that run the centre have been in operation since 2012 – and, as with Amazon, have steadily moved towards a model in which human “pickers” stay put, and machines move everything around.

“We are better than a supermarket,” reads a sign on one of the walls. “We have no stores. We have no limits.” Everything is based around a labyrinth of conveyor belts and storage towers, which ferry three different kinds of crate around the building. Green and yellow crates contain the “ambient and chilled” products that makes up most of Ocado’s stock (frozen items are dealt with by a completely different system in which humans take items out of huge freezers). Red crates contain customer orders. The software that keeps a great deal of the system running works through machine learning: it draws conclusions from mistakes, and also notices if particular manoeuvres speed things up.

Most of the pickers, who are paid between £8 and £10.65 an hour, are women. I meet Ophelia, from Romania, and watch her work; she rapidly takes items – Schwartz spices, Rich Tea biscuits, Colgate toothpaste – out of green and yellow crates, and transfers them to red ones, following orders that flash on to a screen. She tells me she started work today at 4.45am and will knock off at 2.45pm. What’s it like working here, I ask her.

“It’s OK,” she shouts over the sound of the machinery. “It’s better than before. Less walking.”

This is not the most advanced of Ocado’s distribution centres. In another building in Andover in Hampshire (where journalists are not permitted), the company has launched a new system in which products are kept in a huge, floor-to-ceiling “hive”. They are pushed to the top as required, and robots not unlike the Amazon droids scoop them out and ferry them to pickers. The impact on efficiency is remarkable: whereas Hatfield starts and finishes the average customer order of 50 items in around two hours, the Andover system can do it in around 15 minutes.

A lot of people who are highly skilled will gain from automation. Lower-skilled workers are likely to lose out

All of this is explained to me in a meeting room by two senior Ocado employees: Alex Voica, the head of technology PR and communications, and Matt Soane, general manager of the tech division, who has been central to most of Ocado’s innovations. We talk for 90 minutes, and the conversation is regularly interrupted by Voica turning to his laptop and showing me videos and photographs of cutting-edge inventions. The system in Andover is particularly mind-boggling: everything looks so pristine and smooth-running that it appears to be a computer simulation but is actually a film, shot earlier this year by a drone flown through the warehouse.

In partnership with a British firm called Oxbotica, Ocado has trialled driverless delivery vans, which earlier this year did two-mile loops around Greenwich in south London. Together with the Disney corporation, the company is involved in robotics work aimed at approximating the dexterity of the human hand – trying to crack the same problem Amazon has: how to automate the job of picking, particularly fruit and vegetables, without causing damage. Its robotics teams have worked on a suction-based picking robot that can move tins, boxes and other products that have a uniform shape. “It uses a camera to look into the bin and figure out what and where to pick, and then where to place it,” Voica says. “It can do part of our range, and it’s ready to be deployed.”

In September, in an analysis of Ocado’s prospects aimed at potential shareholders, financial services giant Credit Suisse looked at the patents Ocado has recently filed and concluded that the company was pushing into “an automated future”. Half of its staff, it said, could be gone within a decade. What, I wonder, did Voica and Soane make of that?

Voica exhales. “I think what we’ve tried to show you is that robotics and automation and humans can coexist and work better together.” The human element, he insists, is crucial, as evidenced by some people’s emotional attachment to the ritual of an Ocado driver knocking at their door. “For some customers, those interactions are incredibly important,” he says. “If you remove that completely, which I think is what they’re suggesting, you lose some of what makes you human, as a company.”

“The change is fairly incremental,” Soane says. “You’re not going to do all of your picking with robotics suddenly.”

We very quickly arrive back at the same question highlighted by my visit to Amazon. Clearly, in the supermarket business, there is a big shift under way. The amount of human effort it takes to get goods into people’s homes is dwindling as machines take over much of the work. That’s incontrovertibly true, isn’t it?

“But I still believe that automation gives the possibility of people doing other things and providing those as part of the service,” Soane says. “It’s not that the robot replaces the person. New possibilities open up.”

What form those possibilities will take, however, and exactly whom they will be open to, is a question that consumes Swedish academic Carl Benedikt Frey. Working at Oxford University’s Martin School, which describes itself as “a world-leading centre of pioneering research that addresses global challenges”, Frey has been intensively researching the relationship between automation and human employment since 2011. He now spends 90% of his professional life working on the subject, and his findings do not make for comforting reading.

Four years ago, he co-authored a watershed paper titled The Future Of Employment: How Susceptible Are Jobs To Computerisation?, which put the proportion of US workers threatened by technology at 47%. An update will be published next year, and among Frey’s recent pronouncements is the claim that “retail is one industry in which employment is likely to vanish, as it has done in manufacturing, mining and agriculture”.

Frey tells me the lack of manufacturing jobs in the UK, relative to the US, puts the share of British jobs that could be automated at around 35%; but he gives the impression that this is cold comfort. Automation, he explains, first hit manufacturing. Then it began to affect secretaries and bookkeepers in the “white collar” parts of the economy, where AI and new digital innovations are now advancing even further. Although the technology is only just starting to be implemented, the same effect is set to tear through unskilled or semi-skilled work in services.

This, Frey insists, may not entail mass unemployment. “There will always be jobs if people are prepared to work for sufficiently low wages,” he says, matter-of-factly. “But the big question is whether people are going to be better off as a result of automation in the future.” He pauses. “Some will. Some won’t.”

His research points to widening inequality. “A lot of people who are highly skilled will gain from automation. Lower-skilled workers are likely to lose out.” In time, he says, generations with more tech skills and an adaptable attitude to work may ease the birth pangs of this new world; but if you’re a 55-year-old who has lost their job to a robot, things are likely to be bleak. There may be work in fields that require “complex social interactions” – he mentions fitness trainers, beauticians and carers – but it may well be poorly paid and not suited to everybody.

Aviva insurance has 33 million customers spread across 16 countries. Its future is being planned from what it calls “a digital garage” in Hoxton Square, one small part of the expanding cluster of tech businesses centred around Old Street, London’s Silicon Roundabout. It is, inevitably, not a garage at all, but a wood-lined warren of desks, benches, meeting rooms and cafe spaces populated by young people in casualwear.

I meet people working on insurance advice that will be piped through the Amazon Echo (ask it, for example, what a pension annuity is, or how to take advantage of a no-claims bonus, and an answer instantly comes back). Others are busy automating the way Aviva pays the companies that fix the cars and do the household repairs that insurance pays for. I’m also introduced to Neil Dodd, the man in charge of the visual side of Aviva’s online operations, notably its MyAviva app. Until recently, he worked for Activision, creator of such games as Call Of Duty and Guitar Hero.

Walking around, you get a clear sense that the human/machine balance in insurance, pensions, banking and other financial services is changing fast. Software can now process routine insurance claims and spot probable fraud. Aviva’s big idea is called Ask It Never, built around a vision of people applying for insurance and not having to be asked a single question – because software will do most of the work, finding out who you are and what risks you might present from the huge trail of data we all create.

We’re all going to have many careers now. We will need good basic education, plus resilience, curiosity and adaptability

Obviously, this is not the kind of automation you can see. It is based around an endless array of computer applications – in Aviva’s case, as many as 40 – which crunch numbers and process information behind a wall of apps and websites. Most of us tend to think of the insurance industry as being awash with forms, bills and people processing applications and claims, but that is not how the future is going to look at all.

This year’s PwC study found that 32% of UK jobs in finance and insurance might soon be automated. In Frey’s 2013 study, top of the list of roles ranked susceptible to automation were claims and policy processing clerks, claims adjusters, examiners and investigators, insurance underwriters – jobs central to this industry.

Aviva’s chief digital officer is Andrew Brem: a floral-shirted fortysomething who talks about the future with a breathless zeal. Sitting in an alcove near Aviva’s meeting space – where it also hosts yoga sessions – he explains how he and his teams want insurance to work. Automation, he says, holds out the prospect of dealing with your insurance policies or pension plan in a much more agreeable way, but that requires a top-to-bottom transformation. “We talk about being 2% done. Partly because our aspirations are really far ahead. We want the experience of engaging with Aviva to be as good as Facebook, or Amazon, or Airbnb.

“Fifteen years ago,” he continues, “if you wanted insurance, you would still have gone to talk to someone, probably in a branch. They’d fill in a load of forms, compare a load of prices, get a quote, and eventually make it all happen. How do you do it today? We’ll have to ask you a few questions, but we’re literally getting it down to two or three. I don’t know why I get asked what kind of house I live in, because I only need to look at Google Street View to know that. There are lots of different places to get information in an automated way. We want to make it totally frictionless.”

The company, he says, has investments in tech startups which are using AI to speed up how Aviva might amass information about people. He says he can’t give away the details, but one of them is working on the application of AI to selfies. “From the selfie, the software can, for example, make judgments about your body mass index,” he says. “So you don’t have to get someone to measure you. These are experimental things, but we want to experiment.”

This kind of talk will jangle some people’s nerves. Might automation extend to software that trawls people’s Facebook profiles? “Not yet, no. We’re not doing that at the moment.”

What about a newspaper report in February that Aviva planned to ask each of its 16,000 employees whether their job could be automated? “Employees who answer ‘yes’ will be retrained for another role at the firm,” the report suggested. The scheme was reportedly discussed at the highest level of the company.

“I do remember hearing about that,” Brem says. “I never found out what was said to whom.” A later exchange of emails does not result in any official denial or confirmation; the company line is that “conversations are taking place – and encouraged – at all levels across Aviva… If there are parts of roles that can be automated, it’s important that we identify them and look at whether this could enable colleagues to spend more time putting their skills to better use.”

What will all these changes mean for the kind of jobs the company can offer people, I ask Brem.

“Repetitive, menial tasks are less relevant, right?” Brem says. “Here, we’re talking internally about the organisation of the future and what it means for work: the gig economy, and all that kind of stuff. My view is that, let’s face it, we’re all going to have many careers and jobs over our long working lives now. We’re going to be employed and self-employed – that distinction’s going to go, I think. But the important thing is to have really good basic education, plus some cultural traits like resilience, curiosity and adaptability… and at various points, we’re all going to have to retrain.”

Hapless Boston Dynamics robot in shelf-stacking fail - video

There are echoes here of the future mapped out by Frey and his colleagues. Clearly, a lot more demands will be made of people.

“I don’t know if it’s more or less demanding,” Brem says. “To me, it’s a lot more interesting.”

It may well be, but in an industry traditionally as labour-intensive as insurance, all this means fewer jobs, doesn’t it?

“Everybody seems to say that of every industry,” he says. “The truth is, generally speaking, what every company is trying to do is grow their business and have people doing different jobs. What I do has got nothing to do with job reduction. That is not the big prize here.”

But will it be the end result? Upstairs, in a loft space at the top of the building, is the cramped HQ of the Aviva offshoot Quotemehappy, an online-only insurance operation that sells itself on the promise of low cost. On one wall is a plasma screen that tracks the number of people referred to it on a minute-by-minute basis by such online services as Moneysupermarket and Comparethemarket. On a nearby shelf is a collection of toy meerkats.

“We test new things here,” says the run manager in charge. These include automated chatbots on Facebook Messenger, which entice people into buying insurance by engaging them in algorithm-enabled conversation, and software that processes claims with no need for human work. An in-house “fraud guru” keeps watch over everything – but, I’m told, “the data identifies claims that don’t require intervention, and we pay them”. These are tasks that once required the labour of whole departments; now, digital processors take the strain.

Looking around, I get the same feeling of future-shock I experienced at Amazon and Ocado: a sense of hardware and software whirring away, and all those people clutching phones and coffee cups planning more of the innovations that are disrupting the world of work as never before.

Some of Quotemehappy’s operations are dealt with by Aviva staff in Perth and Norwich, but to all intents and purposes, it is run from here. It has one million customers, and is growing fast. Its workforce totals 25.

• Commenting on this piece? If you would like your comment to be considered for inclusion on Weekend magazine’s letters page in print, please email weekend@theguardian.com, including your name and address (not for publication).

This article contains affiliate links, which means we may earn a small commission if a reader clicks through and makes a purchase. All our journalism is independent and is in no way influenced by any advertiser or commercial initiative. By clicking on an affiliate link, you accept that third-party cookies will be set. More information.