The murder of the journalist James Foley is sickening. As his grieving parents said, this courageous man had his life terminated for trying to expose the suffering of the Syrian people. And among those causing that awful pain and devastation, as in Iraq, are the very same bunch of zealots who carried out his barbaric slaying.

Yet however callous, however cruel, however utterly contemptible, the beheading was far from senseless for the Islamic State jihadists. Such well-publicised savagery produces the panic that makes fearful infidels flee their advance, as I saw these past 10 days in Iraqi Kurdistan. What other group could empty a refugee camp packed with Christians in less than an hour, or a sizeable Yazidi town on a Sunday morning? This fear factor helps a comparatively small force – perhaps just 25,000 strong – to conquer and control a slab of land the size of Britain.

The killing highlights the group’s skilled use of social media to spread their warped message – such a strange but effective fusion of medieval beliefs and modern technology. I clicked on the video following a link from Twitter, then cursed myself and turned off the computer. We do not need to see those dreadful images to appreciate the disgusting nature of the deed; they offer only a voyeuristic glimpse into the corrosive and often-choreographed pornography of conflict in our age. Watch them, and you demean the legacy of a brave reporter.

Last week I spoke with an Islamic State official, a man inflicting his corrupted religious beliefs on hundreds of kidnapped Yazidi women and children. He calmly explained his organisation’s aims: to drive non-Muslims out of its self-declared caliphate and fight the US because it was using Arab rulers to attack them. He was polite, refusing only to discuss the foreign fighters thought to make up one-third of their forces, but chilling as he discussed the sale of women as slaves and the theft of children for conversion.

The key question is how to defeat such frightening fanaticism. This is a group that alarmed even al-Qaida, exploiting the Syrian civil war and sectarian rule of Iraq to amass arms, wealth and influence at immense speed. It is thought 500 Yazidi people were slaughtered in response to the first US air raids. Clearly the aim of this latest outrage – combined with the threat to kill another of the 20-odd journalists who have disappeared in Syria – is to terrify and goad the west.

The beheading of an American reporter, apparently by a British jihadist, is horrific. It is also a deliberate provocation. Foley would have been among the first to remind us that there have been many more deaths in this bloodstained and increasingly divided part of the world. It was disturbing to hear repeated claims from refugees of Arabs turning so rapidly on Christian and Yazidi neighbours, readily joining jihadists in carrying out atrocities in their towns and villages. Similar forces can be seen in west Africa and elsewhere.

Tragically, it appears that the kaleidoscope of cultures in the Middle East has been so badly shaken it may be broken for ever. Britain and the US bear some responsibility; such was the arrogance that when George Bush and his senior advisers pushing the 2003 war first met with Iraqi exiles, much of the time was spent explaining the difference between Shia and Sunni Muslims – and that both were found in Iraq. But given the extreme nature of today’s crisis, both sides of this debate need to stop fighting past battles.

We must accept our own impotence. Western nations should offer sanctuary to many of those devastated refugees and collectively ban for life the return of any young fools joining the jihad. But we cannot solve this explosion of complex economic, social and political issues – let alone heal religious fissures dating back centuries. This is not to advocate isolationism, merely to suggest any fresh interventions are laced with realism.

Yes, it makes sense to use western special forces on the ground, not least to shore up the beleaguered Kurds whose accidental state is the one beneficial thing to have emerged from recent incursions. But we must banish for ever the idea that we can impose simplistic solutions.

Britain should reflect on foreign policies that seem more confused by the day, sending out conflicting messages exploited easily by extremists. So we support democrats in one place, yet despots next door: we look at banning a party that won an election in Egypt, and back a repressive general who mounts a coup; we back Israel regardless of its lethally disproportionate response to rocket attacks; and we arm a feudal regime in Saudi Arabia that exported religious extremism around the globe with devastating consequences. It is even reported that our own allies aided the funding of Islamic State.

One thing is clear from this latest evil act: the connected nature of a planet in which a possibly British man beheads an American journalist to advance religious insurgency in the Middle East, and the video is posted online. The idiocy of those who claim they have all the answers is equal only to the stupidity of those who think we can ignore global problems.