Male mice are not natural-born fathers. Males that have never mated respond with aggression to chemical signals from newborn mice pups, whereas those that have fathered pups are more nurturing, a new study finds.

In addition to their normal sense of smell, mice and some other animals have a sensory system in their brain, known as a vomeronasal organ, that responds to chemical signals, or pheromones. The study, detailed in Wednesday's issue of the Journal of Neuroscience, showed that after male mice spent some time around baby mice, neurons in this sensory organ were more active in virgin males than in mouse fathers. Suppression of the vomeronasal system in mice might be important in the transition from attack behavior to parenting, the researchers say.

"The male mouse is normally aggressive, and this seems to be elicited by cues in the vomeronasal system," behavioral neuroscientist Alison Fleming of the University of Toronto at Mississauga, who was not involved in the study, told LiveScience. "If you remove those cues, the animal stops being aggressive and becomes parental. So there's a shift-over that happens." [Countdown: History's Top 12 Doting Dads]

Researchers at the RIKEN Brain Science Institute in Japan wanted to understand how that shift occurs at the cellular level in the brain. They observed the behavior of virgin male mice and mouse fathers that had lived with a female and her babies when placed in a cage with babies. The babies were kept in a mesh ball to prevent any harm from hostile males.

The majority of virgin males were aggressive toward the pups, the researchers found. But after the males mated, their aggressive behavior steadily decreased as they spent more time living with their mate and babies. In other words, after the males experienced fatherhood, they became much more nurturing.

Next, the researchers examined differences between virgin males and mouse dads at the cellular level. Spending time with babies activated certain types of cells in the mice's vomeronasal systems.

The scientists confirmed that the vomeronasal organ was involved by surgically removing it from virgin males and then watching how the mice responded to babies. Now, mice that were formerly hostile toward babies suddenly lost their aggressiveness and became more nurturing. The findings provide a basis for understanding the shift to parental behavior in mice.

This study confirms earlier studies linking the aggressive behavior of male mice to the vomeronasal system, said neuroscientist Peter Brennan of the University of Bristol, U.K., who was not involved in the work. But the findings are not really applicable to humans, who don't have this kind of vomeronasal system, Brennan said.