New research has revealed that Neanderthals died out within a few thousand years of the appearance of Homo sapiens. Photograph: BBC

The puzzle is one of the greatest surrounding our species. On a planet that bristled with different types of human being, including Neanderthals and the Hobbit-like folk of Flores, only one is left today: Homo sapiens.

Our current solo status on Earth is therefore an evolutionary oddity – though it is not clear when our species became Earth's only masters, nor is it clear why we survived when all other versions of humanity died out. Did we kill off our competitors, or were the others just poorly adapted and unable to react to the extreme climatic fluctuations that then beset the planet?

In particular, researchers have focused on the Neanderthals, a species very close in physique and brain size to modern humans. They once dominated Europe, but disappeared after modern humans emerged from our African homeland around 60,000 years ago. The question is: why?

"A major problem in understanding what happened when modern humans appeared in Europe has concerned the dates for our arrival," said Professor Chris Stringer, of the Natural History Museum, London. "It was once thought we appeared in Europe around 35,000 years ago and that we coexisted with Neanderthals for thousands of years after that. They may have hung on in pockets – including caves in Gibraltar – until 28,000 years ago, it was believed."

In other words, there was a long, gradual takeover by modern humans – an idea that is likely to be demolished at this week's conference, Stringer said. Results from the five-year research programme, Reset (Response of humans to abrupt environmental transitions), will show that modern humans arrived much earlier than previously estimated and that Neanderthals expired earlier than we thought. Careful dating of finds across Europe suggest Homo sapiens could have reached Europe 45,000 years ago. Five thousand years later, Neanderthals had largely disappeared.

"Previous research on Neanderthal sites which suggested that they were more recent than 40,000 years old appears to be wrong," said Stringer. "That is a key finding that will be discussed at the conference."

Using radiocarbon technology to date remains that are 40,000 years old has always been tricky. Radioactive carbon decays relatively quickly and after 40,000 years there will only be a tiny amount left in a sample to measure. The tiniest piece of contaminant can then ruin dating efforts.

However, scientists have set out to get round these problems. At Oxford University, scientists led by Tom Higham have developed new methods to remove contamination and have been able to make much more precise radiocarbon dating for this period. In addition, Reset researchers have used evidence of a devastating eruption of the Campi Flegrei volcano west of Naples 39,000 years ago.

Recent studies have shown this eruption was much more destructive than previously recognised. More than 60 cubic miles of ash were blasted into the atmosphere and covered a vast area of eastern Europe, North Africa and western Asia. This layer gives scientists a precise means of dating for this period and, combined with the new radiocarbon dating, shows there seem to be no Neanderthal sites anywhere in Europe 39,000 years ago, a date 10,000 years earlier than previous estimates. It is a significant shift in our thinking about our nearest evolutionary cousins.

Some researchers have even suggested that Campi Flegrei – the biggest volcanic eruption in Europe for more than 200,000 years – would have had a catastrophic impact. Vast plumes of ash would have blotted out the sun for months, or possibly years, and caused temperatures to plummet. Sulphur dioxide, fluoride and chloride emissions would have generated intense falls of acid rain. Neanderthals may simply have shivered and choked to death.

The Campi Flegrei eruption not only gives us a date for the Neanderthals' disappearance, it may provide us with the cause of their extinction, though Stringer sounds a note of caution.

"Some researchers believe there is a link between the eruption and the Neanderthals' disappearance. But I doubt it," he said. "From the new radiocarbon dating and the work carried out by Reset scientists, it looks as if the Neanderthals had probably already vanished. A few may still have been hanging around, of course, and Campi Flegrei may have delivered the coup de grace. But it would be wrong to think the eruption was the main cause of the Neanderthals' demise."

So what did kill off the Neanderthals? Given the speed at which they seem to have disappeared from the planet after modern humans spread out of Africa, it is likely that Homo sapiens played a critical role in their demise. That does not mean we chased them down and killed them – an unlikely scenario given their muscular physiques. However, we may have been more successful at competing for resources, as recent research has suggested.

Eiluned Pearce and Robin Dunbar of Oxford University recently worked with Stringer and compared the skulls of 32 Homo sapiens and 13 Neanderthals, finding the latter had eye sockets that were significantly larger. These larger eyes were an adaptation to the long, dark nights and winters of Europe, they concluded, and would have required much larger visual processing areas in Neanderthal skulls.

By contrast, modern humans, from sunny Africa, had no need for this adaptation and instead they evolved frontal lobes, which are associated with high-level processing. "More of the Neanderthal brain appears to have been dedicated to vision and body control, leaving less brain to deal with other functions like social networking," Pearce told BBC News.

This point is stressed by Stringer. He said: "Neanderthal brains were as big as modern humans' but the former had bigger bodies. More of their brain cells would have been needed to control these larger bodies, on top of the added bits of cortex needed for their enhanced vision. That means they had less brain power available to them compared with modern humans."

Thus our ancestors possessed a fair bit of enhanced cerebral prowess, even though their brains were no bigger than Neanderthals'. How they used that extra brain power is a little trickier to assess, though most scientists believe it maintained complex, extended social networks. Developing an ability to speak complex language would have been a direct outcome, for example.

Having extended networks of clans would have been a considerable advantage in Europe, which was then descending into another ice age. When times got hard for one group, help could be sought from another. Neanderthals would have less backup. This point is supported by studies of the flints used for Neanderthal tools. These are rarely found more than 30 miles from their source. By contrast, modern humans were setting up operations that saw implements transported 200 miles.

Cultural life became increasingly important for humans. Research by Tanya Smith of Harvard University recently revealed that modern human childhoods became longer than those of Neanderthals. By studying the teeth of Neanderthal children, she found they grew much more quickly than modern human children. The growth of teeth is linked to overall development and shows Neanderthals must have had a much reduced opportunity to learn from their parents and clan members.

"We moved from a primitive 'live fast and die young' strategy to a 'live slow and grow old' strategy and that has helped make humans one of the most successful organisms on the planet," said Smith. Thus Neanderthals – who already lived in sparse, small populations across Europe – were fundamentally ill-equipped to deal with the newcomers who had arrived from Africa.

"There may not have been a single cause of Neanderthal extinction," said Stringer. "They may have disappeared in different regions for different reasons, but the background cause is clear. They didn't have the numbers."