Chainsaws, Gunshots and Coughs: Our Smartphones Are Listening

From chainsaws whirring in rainforests to snoring that sounds like chainsaws, entrepreneurs are finding all sorts of creative ways to detect sounds using smartphones.

App makers have long focused on detecting speech and music, but some upstarts are turning to a wider variety of sound-detection tasks. They are taking advantage of more sophisticated mobile hardware and software to recognize distinct audio patterns.

In one of the quirkier ideas around sound detection, a company called Rainforest Connection wants to mount smartphones in trees to detect chainsaw noise and quickly notify local authorities about illegal logging. The company, founded last year, has launched a $100,000 Kickstarter crowdfunding campaign and partnered with the Zoological Society of London to kick off the project in Cameroon.

As of this post, Rainforest has raised $36,000 toward the campaign with 26 days to go. The company retrofits recycled Android-powered phones with custom-made solar panels – to provide an ongoing power source – and hides them in the canopy of trees.

The phones can detect the noise within one square mile and send it to the cloud for analysis. Specific sound signatures from logging, gunshots or distressed animals trigger text notifications to local enforcement officers. Rainforest Connection said the system worked in a months-long test in a rainforest in West Sumatra, Indonesia. But the technology does have its limitations, such as the system’s durability in harsh conditions and weak cellular coverage in rainforest areas.

“We still have a lot to prove in terms of technology,” said Topher White, co-founder and CEO of the San Francisco startup.

The project is one of several new mobile initiatives designed to detect sound patterns.

A crude prototype of BodyBeat, revealed in mid-June, uses an external custom-made microphone to track body sounds, such as breath or cough, with the ambitious aim to detect illnesses or record food consumption.

The microphone is placed on the neck with a 3D-printed neckpiece, which is plugged into a small audio processing device that is wirelessly connected to a smartphone. BodyBeat authors plan to redesign the system for better usability in commercial applications.

It may sound far-fetched. But there could be plenty of market opportunities for systems like BodyBeat. Breathing sounds are indicative of lung conditions, and data on what users consume – say, how often do they drink or eat certain products – can provide important data for diet tracking apps.

There are certainly limitations to sound-detection technology. The quality of embedded microphones remains a concern, for one. “The problem is you can’t create a robust app because everyone is using different microphones,” said Alexander Adams, who helped develop BodyBeat.

The differences in microphone characteristics may prevent devices from capturing subtle audio details, essential for robust detection of some sounds. Applications that record and analyze audio may also pose privacy risks, as they have to keep microphones turned on at all times.

“Privacy is critical both to adoption and to the public’s perception of all of these technologies,” said James Beldock, a senior vice president of ShotSpotter, a company with about 6,000 custom-made sensors deployed nationwide to detect gunshots. These sensors use the hardware similar to cellphones.

He adds that these apps are not generally streaming audio anywhere, but instead are exchanging small portions of sound-describing data, with the processing systems sitting in the cloud.

BodyBeat’s makers emphasize that developers can go numerous ways to make sure user privacy isn’t violated. They can immediately filter out speech and focus not on what was said, but on how it was said. “Depending on what you are trying to learn from the data, more times than not you can do it in real time without any recordings,” Adams said.

For White’s rainforest project, privacy doesn’t seem to be an issue: “We’re addressing it simply by not focusing so much on the people themselves, but on the sounds of the forest.”

Correction: An earlier version of this story incorrectly spelled James Beldock’s surname as Bedlock.