Editor's note: This story is part of a months-long investigation into cashless tolls to find out why drivers are getting escalating fines, who's running the system and where the system is breaking down. Read more here.

The company behind the camera at New York's tolling gantries is developing technology to enable those same lenses to read emotions on motorists' faces as they cross bridges and tunnels.

"Eyes are the window to the soul, after all," Conduent wrote in a Facebook post in August of 2017.

The advances in this kind of technology is part of the value Conduent, the cashless tolling giant, places on data, according to its own website.

“Today, many consider data a new class of currency,” a statement on its site reads.

The corporate engines driving the dark future find their fuel in personal data, a fact not lost on the American Civil Liberties Union (ACLU).

“You have companies who only just care about the monetary bottom line and not necessarily who is accessing it, and how this information can be misused,” Rashida Richardson, Legislative Counsel for the New York Civil Liberties Union, said.

The labs

Formerly nestled in Webster, N.Y., a suburb of Rochester, Conduent's experimental research facility also has a sister lab in Bangalore, India. Together, these labs develop Conduent's latest systems, according to Conduent's website.

Conduent data collection comes in many forms. With research into machine learning and facial recognition, Conduent is at the forefront of new imaging technologies.

Out of these labs, Conduent has filed nearly 20patents since its spin-off from Xerox at the end of December in 2016. Among their developments are:

Facial-recognition advances that will be used in retail and transportation to help Conduent judge customer satisfaction, Conduent claimed in its press release for the new system.

Programs that can see if drivers are using cell phones.

Plans to combine speeding and red-light cameras into a single system. While the system plans on using three humans to verify the numbers before a ticket is sent out, computers will do the initial matching.

Necessity births research

Conduent's contracts with governments and businesses create immense amount of work for it, more than human labor could ever do.

Adding that to the rest of the Thruway Authority’s toll roads, the Port Authority and the MTA’s toll roads makes for a number that would require massive amount of human labor to sort through everyday.

Conduent operates cashless systems in Los Angeles, San Francisco, Texas, Florida, Maryland, New Jersey, North Carolina and South Carolina, as well as New York. It found a way to sort the millions of tolls and billing address matches without using humans.

Conduent's Chief Information Officer, Carol Kline explained at a Conduent hosted conference last December in New York.

“When we created our application to be able to do a screen capture of a license plate at high fidelity, and then be able to correlate that back to who you are, the team used machine learning,” Kline said.

Computers do the heavy lifting for cashless tolling, a fact not visible in the contract with the Thruway Authority.

Glitches in the software

The Consumer Financial Protection Bureau fined Conduent $1.1 million for providing faulty software to auto loan lenders, in November of 2017 Conduent's software sent incorrect data for more than 1 million consumers to credit reporting agencies, according to a statement from the CFPB.

Part of the $1.1 million fine wasn't just for the faulty software. The CFPB alleged that Conduent knew about the problem and didn't tell its customers about it. Conduent did

One of Conduent's other jobs is managing high occupancy vehicle lanes, commonly called carpool lanes.Conduent developed technology to read the amount the passengers in the cars in the designated lane.

That system has E-ZPass-like transponders with with settings for the number of people in the car. If the car's transponder setting doesn't match what the camera identifies, the system notifies the police.

What data does for Conduent

When Kline announced Conduent's advances with machine learning, she was referencing the work of the research labs, Conduent Labs USA.

Lead by Head of Conduent Technology Innovation, Armon Rahgozar, Conduent’s research labs found a way to use cashless tolling cameras for surveillance.

“It’s not only just the license plate, we can recognize the number of people in the car, the type of the automobile and all that data,” Rahgozar said at a Conduent hosted conference last December in New York.

Rahgozar's reference to "all that data" was a response to part of an earlier statement made by Kline at the event, where she spoke about how much companies know about people's habits.

“We know where you are, we know what you’re buying, we know what you’re doing,” Kline said at a Conduent conference in 2017. “We know your habits, it’s creating quite a capability.”

Conduent's cashless tolling systems are able to track people's movements throughout the system. Every time drivers pass under a gantry, or through an E-ZPass plaza their location, time and date gets logged.

With the data from millions of car trips, Conduent could find itself with a powerful amount of data about drivers' choices. However Conduent's contract with the Thruway Authority and others forbids it from using that data, a clause Conduent's allies seek to change.

D’Artagnan Consulting — a firm that bills itself as a pioneer in the transportation world — is one of them.

“We have to build an open network of transportation information, and connect everything to it to get near real-time updates,” said former President and current partner at D’Artagnan Consulting, Jack Opiola, on a Conduent-hosted podcast.

Cashless tolls: A lohud investigation(Photo: lohud)

About this project

Reporters at lohud and The Journal News have spent five months investigating cashless tolls to find out why drivers are getting fees and escalating fines for tolls for which many say they were never billed, who's running the system and where the system is breaking down. Read the full project here.

The reporting so far has prompted changes, including:

an amnesty program forgiving thousands of dollars from individual bills;

a bill introduced in Albany to help toll payers;

a new web page for the amnesty program instead of using the faulty Tolls By Mail site;

more distinct envelopes so drivers know they've received a bill;

new toll signs on the Gov. Mario M. Cuomo Bridge;

more responsiveness from Thruway officials, two of whom attended a lohud forum on cashless tolling and personally helped drivers with their individual cases.

legislation drafting a tollpayer’s bill of rights;

and an apology from the lieutenant governor.

Tell us your story

Has this happened to you? Tell us your story. Email digital@lohud.com with the subject line "cashless tolls" or call 914-510-2181 and leave a message.