Understanding collective animal behavior may be in the eye of the computer

Jan 16, 2014

This shows snapshots of video data of collective behavior (collected by NYU undergraduate student Gozde Ustuner) from experiments with (a) ants, (b) fish, (c) frogs, (d) chickens, and (e) humans. Human faces have been obscured to protect privacy. A machine learning system nearly matched human observation of the video in its assessment of collective behavior of each of the species. Credit: NYU School of Engineering

No machine is better at recognizing patterns in nature than the human brain. It takes mere seconds to recognize the order in a flock of birds flying in formation, schooling fish, or an army of a million marching ants. But computer analyses of collective animal behavior are limited by the need for constant tracking and measurement data for each individual; hence, the mechanics of social animal interaction are not fully understood.

An international team of researchers led by Maurizio Porfiri, associate professor of mechanical and aerospace engineering at NYU Polytechnic School of Engineering, has introduced a new paradigm in the study of social behavior in animal species, including humans. Their work is the first to successfully apply machine learning toward understanding collective animal behavior from raw data such as video, image or sound, without tracking each individual. The findings stand to significantly impact the field of ethology— the objective study of animal behavior—and may prove as profound as the breakthroughs that allowed robots to learn to recognize obstacles and navigate their environment. The paper was published online today in Scientific Reports.

Starting with the premise that humans have the innate ability to recognize behavior patterns almost subconsciously, the researchers created a framework to apply that instinctive understanding to machine learning techniques. Machine learning algorithms are widely used in applications like biometric identification systems and weather trend data, and allow researchers to understand and compare complex sets of data through simple visual representations.

This video is not supported by your browser at this time.

Human interaction captured on video: People exhibited more social interaction than frogs but far less than ants, whether measured by ISOMAP's machine learning system or humans viewing the videos of 10-day experiments. Credit: NYU School of Engineering

A human viewing a flock of flying birds discerns both the coordinated behavior and the formation's shape—a line, for example—without measuring and plotting a dizzying number of coordinates for each bird. For these experiments, the researchers deployed an existing machine learning method called isometric mapping (ISOMAP) to determine if the algorithm could analyze video of that same flock of birds, register the aligned motion, and embed the information on a low-dimensional manifold to visually display the properties of the group's behavior. Thus, a high-dimensional quantitative data set would be represented in a single dimension—a line—mirroring human observation and indicating a high degree of organization within the group.

"We wanted to put ISOMAP to the test alongside human observation," Porfiri explained. "If humans and computers could observe social animal species and arrive at similar characterizations of their behavior, we would have a dramatically better quantitative tool for exploring collective animal behavior than anything we've seen," he said.

The team captured video of five social species—ants, fish, frogs, chickens, and humans—under three sets of conditions—natural motion, and the presence of one and two stimuli—over 10 days. They subjected the raw video to analysis through ISOMAP, producing manifolds representing the groups' behavior and motion. The researchers then tasked a group of observers with watching the videos and assigning a measure of collective behavior to each group under each circumstance. Human rankings were scaled to be visually comparable with the ISOMAP manifolds.

This video is not supported by your browser at this time.

Whether measured by the ISOMAP machine learning system or by humans viewing videos like this of 10 days of experiments, frogs exhibited the least collective behavior of the 5 species studied. The researchers' findings were in line with known qualities of each species. Credit: NYU School of Engineering

The similarities between the human and machine classifications were remarkable. ISOMAP proved capable not only of accurately ascribing a degree of collective interaction that meshed with human observation, but of distinguishing between species. Both humans and ISOMAP ascribed the highest degree of interaction to ants and the least to frogs—analyses that hold true to known qualities of the species. Both were also able to distinguish changes in the animals' collective behavior in the presence of various stimuli.

The researchers believe that this breakthrough is the beginning of an entirely new way of understanding and comparing the behaviors of social animals. Future experiments will focus on expanding the technique to more subtle aspects of collective behavior; for example, the chirping of crickets or synchronized flashing of fireflies.

Related Stories

Recent studies from two research teams at the Polytechnic Institute of New York University (NYU-Poly) demonstrate how underwater robots can be used to understand and influence the complex swimming behaviors ...

Female cowbirds incapable of recognizing high-quality male songs can alter the behavior of flock-mates of either sex and disrupt overall social structure, according to research published May 1 in the open ...

(Phys.org) —A team of researchers from several facilities in France has devised a means for modeling coordinated swarming behavior in a controlled environment. In their paper published in the journal Nature, the te ...

Maurizio Porfiri, an associate professor of mechanical engineering at the Polytechnic Institute of New York University (NYU-Poly), is best known for his work on biologically inspired robots that mimic the movement of schooling ...

The adulthood of a midge fly is decidedly brief—about three days. But a new study of its mating swarm may yield lasting benefits for analyses of bird flocks, fish schools, human crowds and other forms of ...

Swarming is the spontaneous organised motion of a large number of individuals. It is observed at all scales, from bacterial colonies, slime moulds and groups of insects to shoals of fish, flocks of birds and ...

Recommended for you

While studying a ground-nesting bird population near El Reno, Okla., a University of Oklahoma-led research team found that stress during a severe weather outbreak of May 31, 2013, had manifested itself into malformations ...

(Phys.org)—A pair of researchers with Harvard University has uncovered one of the secrets behind pigeons' impressive flight abilities. In their paper published in Proceedings of the National Academy of ...

The International Society of Feline Medicine (ISFM), the veterinary division of International Cat Care, has convened an expert panel of veterinary clinicians and academics to produce practical guidance to ...

(Phys.org)—A trio of researchers has found that one species of flower is able to pick and choose when it comes to accepting pollen from a variety of pollinators. In their paper published in Proceedings of ...

User comments : 0

Please sign in to add a comment.
Registration is free, and takes less than a minute.
Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.

Javascript is currently disabled in your web browser. For full site functionality, it is necessary to enable Javascript.
In order to enable it, please see these instructions.