In an age of computers, GPS devices and fleet GPS transceivers, RFID tag readers, smart meters, embedded microprocessors and sensors, we are now confronted with a tsunami of data.

Software applications allow people to conduct more and more from their smartphones, where geospatial applications like Google Maps generate vast quantities of transient data every day, every transaction is now recorded as well as every potential buying decision, and digital video technology tracks people’s behaviour. It’s pretty clear big data is a game changer, particularly for the way we build cities and create sustainability.

Parking

Fortune magazine describes how big data is changing American cities. In Los Angeles, for example, they have embedded sensors in parking spots. These sensors transmit the data created to an app that reveals which parking spaces are empty. The result: traffic congestion has been reduced by 10 per cent.

Crime prediction

Also, we learn that Los Angeles police are using PredPol, which uses 10 years of police data to predict where the next crimes will break out. The crime rate dropped by 13 per cent. In Seattle police have used highlighted “red boxes” created by the software to initiate more intensive patrols.

Theoretically, as explained by specialists at Duke University, expansion of public transport means it’s easier for everyone to get around, including criminals. This results in the shift of crime from neighbourhoods in which criminals live to other neighbourhoods. It can also result in a shift from crime to legitimate employment as the travel costs associated with having a job decrease. Urban planners and architects can simulate the energy and infrastructure needs of new, large-scale developments with unprecedented depth of detail before construction begins.

Garbage and rodents

According to Dell, big data is already being used in American cities like Chicago where officials are mining data about inspection records, census data and even complaint calls about uncollected garbage.

Dell notes: “For example, by tracking complaints about uncollected garbage and calls to animal control about rodent infestations, city leaders discovered a link: in areas where garbage collection lagged, rodents flourished. Pretty commonsensical, right? However, in 2013, by deploying trucks more frequently to neighbourhoods that historically have more garbage complaints (as well as reacting more timely to complaint calls), the city saw a decrease in requests for rodent control by 15 per cent.

Lead contamination

“Additionally, the city is working to limit potential lead poisoning in children through the predictive analysis of everyday building inspection records crossed checked against census data. By doing so, they are better able to predict which buildings may have a lead issue and have children living in them.”

Traffic

The city is also putting in place sensors to register WiFi and Bluetooth devices to help with everything from pedestrian signals and the assignment of police officers. Potentially, it would enhance the flow of traffic at intersections based on work and holiday schedules.

Big data will affect urban planning by tracking the movements of people. As reported here, MIT professor Marta Gonzalez and second year PhD student Jameson Toole found that a small number of Boston-area drivers contribute disproportionately to traffic jams and gridlock. They worked this out by using location information gathered from mobile phones.

“Mobile phones are amazing data sensors,” Toole says. “Every time you use your cellphone, there is a little breadcrumb that’s stored that can be used in a lot of different ways to help improve human lives.”

The implications of this are exciting. Just think of what would happen if urban planners could predict traffic flows on a given day and devise ways to encourage drivers to take a different route?

“We know people travel for work, but what about other locations?” Toole says. “If we can trace how people move for other social reasons, we can help urban planners determine which areas receive the highest traffic and might be good places for businesses to invest in.”

Illegal subletting

And in New York, Fortune tells us, big data is used to mine information about illegal sublets. In the era of Airbnb and growing concern from councils and neighbours worried about short-term letting, this could be the case of one new technology used to stifle another.

Weather forecasting

IBM is using big data tools like Deep Thunder to create accurate, hyperlocal weather forecasts. The technology is built around a parallel processing supercomputer. All this is designed to help organisations forecast weather down to a square kilometre, and even smaller. IBM sells the technology to utility companies, city governments and others as a cloud-based offering that’s used for precision weather forecasting.

Water waste

HydroPoint Data Systems, a water management company in Petaluma, California, uses big data analytics and the cloud to create an automated system that eliminates water waste while monitoring and protecting against damages caused by leaks and runoff. The system is called WeatherTRAK. A smart irrigation controller, it replaces existing timers with an internet-enabled controller. It takes in data, like weather data, to water supply maintainers via a website and mobile app. It then automates water efficiency and smart buildings with realtime weather analytics, leak detection and wireless applications.

Housing and urban research

Big data has also helped it create an Employment Clustering Tool designed to identify target geographical areas for investment, development and job creation. It does this by drawing on business, economic, industry and land use data.

And in Townsville, AURIN is using big data to bring together energy and water consumption information, weather records and climate data to make Townsville a liveable city.

Professor Robert Stimson, director of AURIN, says big data will play a critical role in urban development as the population increases.

“Australia’s population is likely to top 40 million by mid-century and we are already one of the world’s most urbanised nations,” Professor Stimson says.

“Coping with this growth and achieving the sustainable development of our diverse cities is a major challenge. It needs evidence-based research and policy. The AURIN initiative is providing the data, integrated from multiple sources, and the analytical tools needed to make this happen.”

Michael Batty from University College in London says big data will change city planning.

“In fact, what has really spurred the rise of big data is the collection of data pertaining to activities that humans are intimately involved with,’’ Batty writes. “With seven billion people on the planet, who access about 1.2 billion personal computers, there is now [mid-2013] more than this number of smart phones, some 1.5 billion, growing at around 30 per cent annually. The scale of data being generated by these devices is daunting. In fact, sensor technologies have become ubiquitous with almost plug-and-play like qualities, thus enabling anyone to monitor and collect data from objects with motion that can be sensed by these devices. In our domain of the city, for example, fast automated travel data now records demand and supply, the performance of the various devices, the costs involved, usage of fuel, energy and so on.

“The data we have for public transit in London where some eight million trips a day are made on tubes, heavy rail and buses are taken from the smart card that 85 per cent of all passengers use. This yields about 45 million journeys a week, 180 million a month, a billion or so every half year and so on. The data set will be endless until the technology is changed and even when this occurs, the data being generated will at least in principle still remain a continuing stream. This is the kind of data that are of concern here.

“Big data is certainly enriching our experiences of how cities function, and it is offering many new opportunities for social interaction and more informed decision-making with respect to our knowledge of how best to interact in cities.”

It’s clear that how our cities are built and function will be changed by data analytics. This is as monumental as the Industrial Revolution. It’s early days, but if big data can power future progress in the same way, the 21st century may become a much more sustainable period than has been thought possible.