How AI Could Tackle City Problems Like Graffiti, Trash, And Fires

The trash truck rumbles down the street, and its cameras pour video into the city’s data lake. An AI-powered application mines that image data looking for graffiti—and advises whether to dispatch a fully equipped paint crew or a squad with just soap and brushes.

Meanwhile, cameras on other city vehicles could feed the same data lake so another application detects piles of trash that should be collected. That information is used by an application to send the right clean-up squad. Citizens, too, can get into the act, by sending cell phone pictures of graffiti or litter to the city for AI-driven processing.

Applications like these provide the vision for the Intelligent Internet of Things Integration Consortium (I3), a new initiative launched by the University of Southern California (USC), the City of Los Angeles, and a number of stakeholders including researchers and industry. At USC, I3 is jointly managed by three institutes: Institute for Communication Technology Management (CTM), Center for Cyber-Physical Systems and the Internet of Things (CCI), and Integrated Media Systems Center (IMSC).

“We’re trying to make the I3 Consortium a big tent,” says Jerry Power, assistant professor at the USC Marshall School of Business’s Institute for Communication Technology Management (CTM), who serves as executive director of the consortium. “Los Angeles is a founding member, but we’re talking to other cities and vendors. We want lots of people to participate in the process, whether a startup or a super-large corporation.”

As of now, there are 24 members of the consortium, including USC’s Viterbi School of Engineering and Marshall School of Business. And companies are contributing resources. Oracle’s Startup for Higher Education program, for example, is providing $75,000 a year in cloud infrastructure services to support the I3 Consortium’s first three years of development work.

The I3 Consortium needs a lot of computing power. The consortium allows the cities to move beyond data silos where information is confined to individual departments, such as transportation and sanitation, to one where data flows among departments, can be more easily managed, and also lets cities use data contributions from residents or even other governmental or commercial data providers. That information is consolidated into a city’s data lake that can be accessed by AI-powered applications across departments.

The I3 Consortium will provide a vehicle to manage the data flow into the data lake, explains Cyrus Shahabi, a professor at USC’s Viterbi School of Engineering, and director of its Integrated Media Systems Center (IMSC). IMSC is using Oracle Cloud credits to create advanced computation applications that apply vast amounts of processing needed to train AI-based, deep learning neural networks and use real-time I3-driven data lakes to recognize issues, such as graffiti or garbage, that drive action.

IMSC is developing a suite of open source software that can run in the cloud or in a city’s own data centers. The code and documentation will provide the base needed for cities, states, or even large college campuses to create data lakes and invent new applications of IoT technology. Initially, IMSC is focusing on collecting and analyzing images from still and video cameras.

IMSC’s software will be ported to I3 as a data broker, which will ingest streams of camera information, using deep learning to label the data (such as “area with graffiti”) in the cloud and make that data available to other applications through I3. The data will also contain valuable metadata, such as the GPS location of the camera, the direction the camera was facing, and the image’s environment, such as urban, suburban, rural, parkland, or wilderness—all of which are possible in Los Angeles’ 500-plus square miles.

“Labeling is the part that’s very computational-intensive,” he says, “and in a streaming environment, it’s difficult to do this labeling in real time. That’s where the Oracle Cloud credit is very useful. It significantly improved the speed at which we can train the machine-learning systems to efficiently perform the data-intensive classifications so we can do labeling very quickly.”

The I3 research is important because it is “taking us closer to the reality of an IoT-enabled smart city ecosystem,” says Patrick Mungovan, Oracle group vice president. And the research could be applied in the future well beyond municipalities to other connected community efforts, Mungovan says.

Power sees many future applications of the I3 Consortium’s work, some of which are in development with the City of Los Angeles. One system in the works could integrate video feeds from city-owned and private parking lots and parking structures, and work with a mobile app to guide drivers directly to open spaces. Another project would rapidly spot and classify fires burning within the city.

The fire use case is very complex, as it requires accurately understanding whether a fire poses a real danger: The Los Angeles Fire Department doesn’t want to miss uncontrolled fires, but it doesn’t want to deploy trucks and firefighters for false alarms or controlled burns. That’s where the AI comes in, plus USC’s own computer scientists, who are always on the lookout for really tough problems to solve. “We may put in our own resources because fire classification is a challenging research project,” says Shahabi.

The I3 Consortium, announced in November, is still in its formative stages, raising money, signing up partners, conducting feasibility studies, and working on intellectual property agreements.

Once the foundational work is complete, the real work begins: Getting data into the data lake, processing it to add value, and then using that information to drive action. “If you’re not driving action, there’s no value,” says Power. “We want lots of IoT data to go into the lake, and we want to make that data actionable. That’s our vision.”