We Built A Powerful Amazon Facial Recognition Tool For Under $10

Security
I cover crime, privacy and security in digital and physical forms.

The democratization of mass surveillance is upon us. Insanely cheap tools with the power to track individuals en masse are now available for anyone to use, as exemplified by a Forbes test of an Amazon facial recognition product, Rekognition, that made headlines last month.

Jeff Bezos’ behemoth of a business is seen by most as a consumer-driven business, not a provider of easy-to-use spy tech. But as revealed by the American Civil Liberties Union (ACLU) last week, Amazon Web Services (AWS) is shipping Rekognition to various U.S. police departments.

And because Rekognition is open to all, Forbes decided to try out the service. Based on photos staff consensually provided, and with footage shot across our Jersey City and London offices, we discovered it took just a few hours, some loose change and a little technical knowledge to establish a super-accurate facial recognition operation.

“This underscores how easily a government can deploy Amazon’s face recognition to conduct mass surveillance,” ACLU technology and civil liberties attorney Matt Cagle said of Forbes' project. “Now it’s up to Amazon. Will it stop selling dangerous technology to the government? Or will it continue compromising customer privacy and endangering communities of color, protesters and immigrants, who are already under attack in the current political climate?”

Amazon didn’t provide comment for this article, but pointed Forbes to a blog post from last week, in which the company noted there has been “no reported law enforcement abuse of Amazon Rekognition.” Dr. Matt Wood, general manager of artificial intelligence at AWS, wrote that the company's Acceptable Use Policy (AUP) prohibits the use of services for “any activities that are illegal, that violate the rights of others, or that may be harmful to others.”

“This includes violating anybody’s Constitutional rights relating to the 4th, 5th and 14th Amendments—essentially any kind of illegal discrimination or violation of due process or privacy right. Customers in violation of our AUP are prevented from using our services,” Wood added.

A recipe for facial recognition

To get things started with Rekognition, we enlisted the help of independent researcher Matt Svensson. He set up an AWS database (known as an S3 bucket) into which we poured a mix of stock photos and Forbes staff mugshots. As Amazon didn’t have a straightforward tool to visualize a face match and simply sent back results in text form, Svensson quickly coded up a program that put a red square around our “targets” and green for “innocents,” giving the system an air of professional surveillance.

Our video teams in Jersey City and London took some simple footage mimicking CCTV footage, shots still or pivoting slightly. This meant employees might be at a distance or at potentially difficult angles for Rekognition to recognise.

As we’d expected, though, Amazon’s tech didn't struggle. It had little trouble picking up people’s faces as soon as we put the footage through it. In every case where a Forbes employee was included in the database and a filming, a successful match was made, as shown by the little red squares drawn around their faces.

Using a simple six-step recipe from Svensson, I went through the process of setting up a text-only version of Rekognition to prove even those with little technical nous can quickly stand up a facial recognition operation.

In less than a few hours, I’d registered an Amazon account and created an AWS S3 bucket full of images. Then, using the Terminal on an Apple Mac, I tested a quick selfie against the collection. A 99% match was found.

The text version of Amazon's Rekognition tool in action.

Forbes

I then asked Amazon to look for faces in a short selfie video, then requested find a photo match. Again, quick success with another 99% match. It all worked with little effort, even for a dummy.

Cheap and cheerful facial recognition

This small-scale test was essentially free, largely thanks to Svensson not charging. In a professional deployment the cost would still be minuscule. “Even if we include costs of testing, figuring out AWS and actually running the facial recognition on our scenario, it’s going to be under $10,” Svensson added.

Law enforcement are already enjoying the low cost: the ACLU found the Orlando Police Department spent just $30.99 to process 30,989 images.

Compared to other facial recognition projects currently being run by the federal government, the Amazon service is staggeringly cheap. For instance, Forbes found one $10.8 million deal between the U.S. Customs & Border Protection and contractor Government Acquisitions in December 2017 for tech that included “unlimited facial recognition matching algorithms.”

The scale for surveillance

Amazon isn’t the only consumer tech giant with uber scale dabbling in surveillance. Both Google and Facebook have their own facial recognition arms, though there’s no evidence they’ve sold such services to the U.S. government or local law enforcement agencies.

While we had the permission to use the images of Forbes’ staff, it’s unlikely the average creep who wanted to set up their own facial recognition would bother. They could just scrape the websites, Facebook and Google for instance, for all manner of images to upload to their database. Israeli surveillance giant Verint is already sitting on a large facial recognition database of faces scraped from Facebook and YouTube. For instance, in early tests, Svensson simply grabbed my Google profile image and matched it to my Forbes photo with a 97.4% similarity.

Another interesting moment in those incipient tests came when Svensson tried to combine images shot with Ring, the $1 billion Amazon-owned “smart” door buzzer that comes with a camera for extra home security. Initially, using the single image that he’d trained the Amazon system on, Rekognition failed to recognize him from the Ring video. All it needed, though, was a few more images for training and it was getting matches, even if there were some “false negatives” where Amazon simply recognized a face rather than Svensson, as evidenced by green boxes flashing up around his head before the red.

“Real world matching would be the same, requiring multiple angles of someone's face to be able to match well. For Facebook and Google, they have this in spades,” Svensson said. Google, of course, already has facial recognition software inside its Nest Hello doorbell.

Amazon better than open source?

While open source facial recognition tools are available, Amazon’s platform is different. As with its other products, it has the scale and quality of service to deliver facial recognition at incredibly low cost and to anyone with a computer.

Svensson found Amazon was faster than one of the more popular open source tools, found on Github. While both were accurate in detecting the four faces of Forbes staff, AWS took an average of 0.3 seconds to index a picture, compare to the open source system's 4.5 seconds. To suck in the video, AWS took 0.2 seconds per frame compared to the open source’s 6.45 seconds. Needless to say, Amazon was considerably faster.

So cheap, simple and speedy is Rekognition that it “will likely transform the way we view our privacy online and in the ‘real world,’” Svensson said. Open source tools require the user to have a significant amount of computer power and be able to handle the complexity of the task. Rekognition gets rid of those barriers by using Amazon's immense infrastructure, Svensson said.

Watching the watcher

Amazon says it won’t allow anyone to break the law with Rekogntion and it'll respect citizens' privacy rights. But just how will it monitor the mass of customers it can expect? It hadn’t responded to requests for specifics at the time of publication.

Amazon also hasn't openly explained just how it’s made Rekognition so effective, other than to note it uses artificial intelligence and machine learning to train the software. “The more faces they have access to, the better they can make their system,” explained Adam Geitgey, developer of an open source facial recognition program. “A company like Facebook has an advantage here because they have so many tagged photos of real people. But I’m sure Amazon has a good solution in place too. I’m also assuming that Amazon is also using the data from all the Amazon Rekognition users to help retrain and improve the system for everyone else over time.”

But Geitgey has anxieties about poorly trained facial recognition. “It's also important to remember that any face recognition system is only as good as the data used to train it. Any ethnicity, appearance, age group, etc., that is underrepresented in the training data will not be recognized as well in the final system. So for example, if someone builds a face recognition using pictures of adult western celebrities as training data, it will tend to perform poorly with Asians and young children.”

There are, of course, what some might consider non-surveillance uses of facial recognition. Sky News recently used Rekognition to spot celebrities at the royal wedding earlier this month. Such tools are also used for simple biometrics, allowing only authorized individuals into buildings, for example.

Security concerns

But alongside the almost-palpable privacy concerns, there are security anxieties around the use of Amazon Rekognition too. That's largely because of lapses on myriad occasions when Amazon Web Services’ customers have left their databases open to the public. For instance, in December 2017 marketing analytics company Alteryx left a massive bucket of personal information on individuals across 120 million American households open on an AWS server. Imagine the fallout if faces of law enforcement targets (and supposed innocents) were spilled on the web.

“It only takes one misconfiguration to expose millions of people,” Svensson added.