Two days into the new year of 2009 an Afghan man paid the ultimate price for straying too close to the main military training centre on the outskirts of Kabul.

He was out collecting "rocks and grass" when he was shot in the head with a single bullet, according to the Ministry of Defence. He was taken to hospital but died that day.

The MoD concedes that the fatal shot could have been fired by British forces who were training there. Five months later another Afghan man died in similarly tragic circumstances – he was shot in the neck while praying in a field. Local witnesses said the shots were fired by soldiers from the Welsh Guards or the Mercian regiment.

These are just two incidents among nearly 100 that developed into sensitive matters for British troops in Afghanistan.

Documents obtained by the Guardian show that the Royal Military Police (RMP), the army's police force, launched investigations into 99 incidents in which British soldiers were alleged to have killed or wounded 72 Afghan civilians between January 2005 and March 2011.

Detectives from the RMP, known as the Redcaps, sought to establish if the soldiers had committed criminal acts, including murder or manslaughter.

Commanders on the battlefield had examined each incident and determined that further investigation by the RMP was required.

The actual toll of those who died or were wounded in the incidents is likely to be much higher as the MoD has disclosed only bare descriptions of about half of the investigations.

Twenty-one investigations centred on what the MoD calls "shooting incidents". The highest number of civilian casualties was in an incident on 30 December 2009, in which seven people, including two children, were allegedly killed during an air strike on a group of suspected Taliban insurgents.

The MoD said "it would appear" that five suspected insurgents were planting bombs and firing at a base with "a long-barrelled weapon".

"It was initially believed that four insurgents had been killed and two wounded. However, claims have been made by local elders that two insurgents were killed and seven Afghan civilians had been killed, including two children and one wounded," added the MoD.

In another incident up to nine civilians were killed following a battle on 4 November 2009.

According to the MoD: "Nine people were seen to be acting suspiciously in a field. During continued surveillance it was assessed the Afghans were in the process of laying an improvised explosive device." A rocket was fired at them, but later inquiries by Nato and officials from the Helmand local authority "established that it is highly likely that all of those killed in the strike were civilians who were working in the field at the time".

In a third incident, on 11 August 2009, mortar rounds were fired over 10 minutes at group of what the MoD calls "positively identified insurgents who it was believed were engaging a [British military base] with small arms fire". Later that day British forces received "unverified" reports that the mortars had killed two women and two children.

Another of the investigations appears to have been initiated after a Royal Marine blew the whistle. The commando serving with 42 Royal Marines alleged that on 1 April 2008 mortars had been fired at unarmed Afghans, killing one and injuring a number of others. "He alleged that the order to fire was given despite his protestations that no enemy forces had been positively identified and claimed the only people in the area were unarmed agricultural workers," the MoD said.

In Afghanistan the normal fog and confusion of war is made far worse because of the nature of the conflict. The enemy does not wear uniform, they mingle with the civilian population as they plant their favoured weapon – the home-made improvised explosive device, or IED. The growing threat of suicide bombers, combined with a mistrust of Afghan troops, add to the nervousness and occasional "trigger-happiness" of British and other Nato soldiers. Colonel Richard Kemp, former commander of British troops in Helmand, said: "British troops had decades of experience dealing with an enemy that deliberately attacked from the cover of the civilian population in Northern Ireland. Whole training regimes were developed there to help young soldiers take that incredibly tough, split second decision on whether or not to open fire. The job is even harder in Afghanistan today, where the Taliban not only hide behind civilians while conducting attacks, but actively try to lure our troops to kill innocent men, women and children. Such tactics work well for them, and have been refined over many years".

He added: "If the soldiers shoot and kill civilians, the Taliban have scored a strategic victory. If the soldiers hold fire, they live to fight another day. But for our troops the dilemma is greater still – hesitation at the critical moment can easily cost them their own life. The aptly-named doctrine of "heroic restraint", imposed on Nato troops in Afghanistan by General Petraeus, forces our soldiers to accept personal risk at an unprecedented level – greater even than the horrific dangers they already face in this lethal conflict where so many of our brave men have been killed and maimed."

But for a force whose mission is to protect and help rebuild a broken country, every civilian who dies at the hands of British soldiers is, at the very least, a severe embarrassment as well as a personal tragedy for the victim's family.

The MoD does not routinely count the number of Afghan civilians killed or wounded by British troops. From time to time, however, details do emerge.

This month the defence secretary, Liam Fox, told Labour MP Paul Flynn that none of the incidents revealed in the WikiLeaks documents were reported to the military police to be investigated, as commanders concluded that British forces had not broken any rules.

It took Fox 10 months to respond to Flynn's questions in the House of Commons asking if the deaths had been investigated. Such reticence to shed light on civilian casualties is mirrored in the MoD's foot-dragging in disclosing the latest documents to the Guardian, which submitted its freedom of information request last November.

Even then, details of 49 of the 99 investigations listed have been suppressed by the MoD. The information has been withheld, the MoD says, to protect national security and Britain's international relations, to conceal "personal data" and to avoid scaring off witnesses in future prosecutions. Incidents involving British special forces, who have been engaged for years in hunting Taliban fighters and commanders, mainly in night raids, have been withheld, the Guardian was told.

Army view

Colonel Richard Kemp is the former commander of British troops in Helmand

British troops had decades of experience dealing with an enemy that deliberately attacked from the cover of the civilian population in Northern Ireland. Whole training regimes were developed there to help young soldiers take that incredibly tough, split-second decision on whether or not to open fire.

The job is even harder in Afghanistan today, where the Taliban not only hide behind civilians while conducting attacks, but actively try to lure our troops to kill innocent men, women and children. Such tactics work well for them, and have been refined over many years by Shia militias and al-Qaida fighters in Iraq, and against the Israelis by Hamas and Hezbollah in Gaza and in Lebanon.

If the soldiers shoot and kill civilians, the Taliban have scored a strategic victory. If the soldiers hold fire, they live to fight another day. But for our troops the dilemma is greater still – hesitation at the critical moment can easily cost them their own life.

The aptly-named doctrine of "heroic restraint", imposed on Nato troops in Afghanistan by General Petraeus, forces our soldiers to accept personal risk at an unprecedented level – greater even than the horrific dangers they already face in this lethal conflict, where so many of our brave men have been killed and maimed."