Modern humans, along with environmental factors, have long been suspects in the sudden extinction of our thick-browed relatives, but the nature of their decline is still under debate.

Mellars and French analysed archaeological evidence in Périgord, a former province of southwestern France, which is renowned for its Neanderthal and early human sites. They found that the population of H. sapiens that arrived in the region was at least ten times larger than that of the Neanderthals already settled there.

In particular, the area saw a sharp rise in the number and size of early human sites and the detritus of life they left behind, such as stone tools and the remains of animal carcasses, according to a report in Science.

The researchers believe the sheer pressure of being outnumbered was exacerbated by the social and technological advantages that modern humans displayed, from long-range hunting spears to stronger cooperation and communication. The arrival of modern humans coincided with the appearance of elaborate cave paintings, decorative stones and beads, and imported shells, suggesting H. sapiens had a more complex society than the Neanderthals.

"It was clearly this range of new technological and behavioural innovations which allowed the modern human populations to invade and survive in much larger population numbers than those of the preceding Neanderthals across the whole of the European continent," said Mellars.

"Faced with this kind of competition, the Neanderthals seem to have retreated initially into more marginal and less attractive regions of the continent and eventually, within a space of at most a hundred thousand years, for their populations to have declined to extinction – perhaps accelerated further by sudden climatic deterioration across the continent around 40,000 years ago."

• This article was amended on 29 July 2011. The original picture was a reconstruction of Homo ergaster not H. neanderthalensis. This has been corrected.