The story that isn't being told

In the past five years, Iraq has never stopped being the most important international news story and the biggest challenge for news organisations and journalists from all over the world. Yet at the same time, our ability to report what must be the most consequential war of the past 30 years has been eroded, to the point where journalists, however large and well-funded their news organisation, can only try to provide a snapshot of the war's impact on Iraqi society.

I returned to Iraq recently for the first time in 18 months to make a documentary about the ordinary Iraqis I have known over 10 years of reporting from the country. But operating there as a journalist has never been harder.

It is, and always has been, very different when it comes to telling the story from the British and American military viewpoint. If there is one thing that has not changed since 2003, it is the system that has come to be known as "embedding". The phrase was invented for the Iraq war, and it is where journalists eat, sleep, travel and come under fire with British and American troops, and thus see the war exclusively from that perspective.

For the first year and a half of the occupation, Iraq was a story that was accessible and possible to cover comprehensively from all angles - from the lack of any effective post-invasion planning, to the impact of the economic collapse in the country, and the daily challenges facing British and American soldiers utterly unprepared to act as military occupiers and nation builders. Journalists could travel and tell all these stories and more.

The critical thing was that Iraq was being reported by a huge variety of media organisations and correspondents. Independent freelance journalists were able to operate alongside staff reporters, and not just westerners. Arab journalists from smaller and lesser-known publications and satellite news channels were telling the story. This made journalism not only possible, but also an important career for a huge number of young Iraqis who would never have considered it under the dictatorship of Saddam Hussein.

There was also an extraordinary diversity of views about the war and the occupation: independent bloggers such as the excellent Arab-American writer Dahr Jamail operated alongside reporters from the New York Times, ITV and al-Jazeera. But as insecurity, violence and political instability became inexorably worse from the end of 2004, the media's ability to tell all sides of the story began to close down.

Increasing numbers of Iraqis began to resent and hate the presence of western influence in the country. Although thousands of Iraqis had been kidnapped, largely for financial reasons, kidnappings only began to be reported when western contractors and journalists were taken.

Getting around Baghdad, or speaking to people on the streets, became incredibly dangerous. It's now almost impossible unless you are surrounded by armed bodyguards or you observe "the 20-minute rule" - that you allow yourself no more than 20 minutes to get out of a car, speak to Iraqis and then leave. Any longer and onlookers will phone local militia to say they've seen westerners on the street.

It's now lethal to be associated with anything western. Just before flying to Baghdad, I called a colleague from al-Jazeera Arabic. He was walking in the centre of town. I greeted him in English, but he replied in Arabic and kept speaking, pretending I was his mother and saying he'd be home soon. He called me later to say that it would've been too dangerous to speak English.

This fear of being associated with anything western has appalling consequences. Syria is home to nearly 1.5 million largely middle-class Iraqi refugees, but even in exile Iraqis are terrified to have their faces shown. Officials from the UN High Commission for Refugees told me there have been four cases of Iraqis being killed specifically because they had been interviewed by western journalists. In Baghdad, it's hard to find Iraqis willing to go on camera for exactly this reason.

One of the least reported or acknowledged aspects of the dangers of reporting Iraq is that it is now only the richest news organisations that can remain there. The reason is simple - insurance. The costs of sending staff to Baghdad and keeping them there are astronomical: western "security advisers" ("mercenaries" is more accurate) employed by television organisations get paid around £400-700 each a day. Insurance brokers have more of a say than foreign editors on where reporters go. The more high-profile the reporter, the more expensive, and therefore the less likely it is that he or she will be allowed to leave military bases or the green zone unless escorted by armed guards or the military. (A complaint I have heard from John Humphrys, Huw Edwards and Jon Snow.)

I was criticised by some colleagues when I said I believed TV news organisations were perpetrating a small fraud on British viewers by not explaining our severely limited capacity to report comprehensively from Iraq. I stand by that, and believe it even more today. It would have been easy to give the impression when I returned to Baghdad that I was walking around the city freely, which is the impression you get from pieces to camera. What you don't see are the three very heavily armed British guards with walkie-talkies and back-up cars surrounding us.

Living outside the green zone - the five square miles of central Baghdad colonised by the US military and security forces - used to be a badge of honour for many journalists. But the green zone has come to them. All the major bureaux - whether al-Jazeera, BBC, CNN or ITV - in effect have to be armed compounds, with turrets, concrete blast walls and security checks. The same goes for the Hamra hotel, used mostly by newspaper reporters.

Journalists have to keep reporting in Iraq, and many continue to do so courageously and with commitment. Any amount of reporting is better than none. But five years on from the invasion I find it impossible to escape the conclusion that Iraq is a contradiction - it is still the most important international news story, but the continuing violence and insecurity have also made it an information abyss.