Packs of hunting dogs, troops of baboons, herds of antelope: when people observe social animals, they are often struck by how intelligent they seem, and recent studies suggest that sociality has played a key role in the evolution of larger brain size among several orders of mammals. But new research from two evolutionary biologists, John Finarelli of the University of Michigan and John Flynn of the American Museum of Natural History, calls this hypothesis into question--at least for the Carnivora. After a sweeping analysis of many living and fossil carnivore species that places relative increases in brain size in an evolutionary context, Finarelli and Flynn found that increased brain size is not routinely associated with sociality. Their new research paper is being published in this week's Proceedings of the National Academy of Sciences.

"The universality of the Social Brain Hypothesis does not apply," says Finarelli. "When you look at relative brain size from the point of view of the entire evolutionary history of the clade, the story starts to fall apart--at least in carnivores. This study shows that, almost assuredly, brain size is increasing for different reasons in different groups of carnivores."

Flynn adds "When you analyze carnivores group by group, canids alone are responsible for the pattern seen in the recent analysis of the Social Brain Hypothesis." Flynn is referring to a 2007 paper in Evolution that tested the Social Brain Hypothesis, which proposed that sociality has driven the relative increase in brain size among mammals in three speciose orders: carnivores, primates, and ungulates. The evolution of relative brain size is of broad interest in biology, with important implications for ecology, energetics, and life history, and the previous study found correlations between sociality and relative increase in brain size to body size for all three groups.

As part of their broader study of how brain size evolved throughout the Carnivora, Finarelli and Flynn tested the idea in further detail by analyzing 289 terrestrial carnivores, about half of which were fossil species. The fact that so many fossils were included makes this the first study to reconstruct relative brain size across the full span of the evolutionary tree for this group of mammals. Extant carnivores span 15 families and include bears, weasels, cats, dogs, and related species. For all terrestrial carnivore groups, the authors compiled data on endocranial volume (brain size) and body mass, to estimate relative brain size or encephalization. Encephalization data was then used to map changes in relative brain size within specific clades (known as reconstructing a scaling allometry).

Their detailed analysis of the evolutionary history of carnivores documents at least six separate changes in brain sizes for the group, suggesting that the story of brain size increase is far more complex than previously assumed. Some lineages of carnivores have been remarkably stable in relative brain size (for example, one of the two major groups of living carnivores, the feliforms, except for small cats), while others like the extinct bear-dogs (Amphicyonidae) got progressively smaller brains though time when compared to their ancestors. Dogs, on the other hand, have undergone a relatively recent increase in brain size. Finarelli and Flynn determined that this clade skews the data for the modern carnivores that were analyzed in the previous test of the Social Brain Hypothesis; deleting them from the analysis removes any correlation between brain size and sociality in other carnivores. But even though modern canids have large brains, the reason for the relative increase remains unclear: was larger brain size co-opted for sociality, or did sociality drive the brain size increase? The answer might partly lie in previous research by Finarelli analyzing evolutionary changes among dogs. That 2008 study found that the increase in brain size began around 10 million years ago with the appearance of the first representatives of modern dogs.

The relationship between brain size and sociality is variable among living carnivores as well. If social living is the cause of brain size increase for the carnivore order, or evolution of large brains fosters sociality, then the large-brained bears, small cats, and weasels should be social--but they are not. Carnivores retaining the ancestral condition also do not fit within the picture that the Social Brain Hypothesis would paint; relatively small brained hyenas and mongooses both have social and nonsocial taxa.

"This is a sophisticated and powerful analysis that integrates fossils with extant species of carnivores," says Flynn. "If you only analyze living forms, you often don't correctly reconstruct the evolutionary transformations. Our research shows another example of this, and indicates that the Social Brain hypothesis does not hold for all Carnivora."

John Finarelli is an Assistant Professor in the Department of Geological Sciences at the University of Michigan, and John Flynn is the Frick Curator of Fossil Mammals and Dean of the Richard Gilder Graduate School at the American Museum of Natural History. The research was funded by the National Science Foundation, an AMNH Collections Study Grant, the Brown Family Foundation Graduate Fellowship, and the University of Michigan's Society of Fellows.