When General Odierno returned to Iraq in 2006, he asked Emma Sky to accompany him as his political adviser. Photograph: Linda Nylind for the Guardian

Emma Sky was at her home in Wandsworth, south-west London in September 2006, when she received an email from a friend in the US. At first she tried to ignore it. But Sky knew she wouldn't refuse him his unusual request.

The author was General Raymond T Odierno, one of the US army's most senior officers. He was about to return to Iraq to head "Phantom Corps" in a last ditch attempt to stop violence tearing the country apart.

And he wanted Sky to go with him as his political adviser.

"I hadn't been in Iraq for two years and had just done a six-month tour in Afghanistan, so the email came as something of a surprise. When he asked me to return I was flattered. I also felt that if anyone could make a difference in Iraq it was Odierno. The general is a good listener, he doesn't think he knows the whole truth, he is intellectually curious. He is prepared to take in ideas, and then make decisions. That's why I was prepared to return at the worst of times."

The presence of a British woman at his side would prove controversial and unpopular in some quarters, particularly at the US state department, but the stakes were high and Odierno was evidently prepared to take a risk.

The general had been criticised for his aggressive approach to security in the months after the invasion, though Sky says he took the blame for circumstances beyond his control, and she did not find him to be "some brutal unthinking monster who suddenly had a complete change of personality".

Sky believed he wanted her to help challenge the army's punch first instincts, raise with him things he might not want to hear, as well as offer advice he couldn't get "in-house". "He didn't want me to comply and he didn't pigeonhole me."

The situation in Iraq at the time was desperate. The violence in Iraq had morphed from an insurgency into sectarian conflict. The al-Qaida leader Abu Musab al-Zarqawi had provoked a civil war between Sunni and Shias that would take the country close to collapse.

In 2006, 16,564 Iraqi civilians died, including 3,389 in September, the highest amount for any month during the conflict. Coalition casualties were also high; 873 troops were killed that year, 823 of them American. Inevitably, political support for continuing the military campaign was ebbing away in Washington and London.

Nevertheless, the US president George Bush was poised to disregard the advice of some of his closest advisers – and most commentators too – to announce he was sending an extra 20,000 troops to Iraq, most of them into the cauldron of Baghdad.

The surge was a gamble. It seemed then, and with hindsight remains, an astonishing risk taken by a president who had stopped believing those people who said the violence was being provoked solely by the presence of US forces.

With thousands of extra troops heading for Iraq, Odierno set up headquarters in the vast US military base outside Baghdad near the airport, the unfortunately named Camp Victory.

Sky was given her own basic accommodation and was expected to accompany the general everywhere he went.

Emma Sky became a core member of General Odierno's team and went everywhere he went. Photograph: Linda Nylind for the Guardian

She became a core member of Odierno's handpicked team, which included of some of the best officers in the military, all of them Iraq veterans.

Specifically, Odierno wanted Sky to help him work out an operational plan. A process, she said, that could only begin with a brutal acknowledgment of previous tactics.

"During one of our first discussions, I told him that the situation in Iraq was a disaster and perhaps the biggest strategic failure in the history of the US," said Sky.

"His response was, 'what are we going to do about it? We cannot leave it like this'. There was no denial about the extent of the problem."

"We spent many hours discussing the depth of the problem and what needed to be done. Sometimes it was just me and him, at the end of the day, sitting at Camp Victory on his balcony, and he's smoking cigars. Sometimes we are at his office and he's brought in a small team of people. But every day we would be up late talking about why people are using violence."

"There was a power struggle going on at every level, a communal struggle for power and resources. I knew from my time in Kirkuk that politics drives this kind of instability, and that politics needs to be managed to bring down violence. I believed Iraqis were using violence to achieve political goals. We had to stop stigmatising these people. We had to stop calling these people the enemy. We needed to identify all the different the groups and ask, 'why are they fighting? What are the drivers of instability?'"

Implementing Fardh al-Qanoon

The overall strategy was masterminded by General David Petraeus, who had spent months in the US developing a new counter-insurgency doctrine.

In February 2007, he arrived in Baghdad to assume command of all coalition forces in Iraq, and reviewed the plans drawn up by Odierno's team about where and how the extra troops should be deployed.

"The operational details for the surge were left to General Odierno," said Sky.

An important part of the new campaign involved separating the people who might be persuaded to abandon violence, the so-called "reconcilables", from those who were not. The former would not be targeted by Special Forces operations, the latter could be.

The men in charge of this were General Petraeus' deputy, Graeme Lamb, a former director of UK special forces, and the American General Stanley McChrystal.

"The irreconcilables were those people who essentially believed that you have to destroy the nation-state to build the caliphate. But you have to be really careful deciding who can be won over, and who can't. It meant we would have to start dealing with people we had been fighting and for any commander that is a very difficult thing to do. We couldn't afford to say 'we'll only deal with people as long as they haven't got blood on their hands'. We've all got blood on our hands," Sky says.

Referring to where he was going to put the "wedge", and who could be put in his "squeeze box", Lamb drew up "Restricted Target Lists" – the names and details of those Iraqis that could not be targeted in operations because they were talking to the military. McChrystal dealt with those who refused to compromise.

Once Odierno's plans had been endorsed by Petraeus, he and Sergeant Major Neil Ciotola travelled the length and breadth of Iraq to visit the troops and explain the new tactics. Sky was always at Odierno's side.

The campaign was given an Arabic name, Fardh al-Qanoon – imposing the law. As an important first step, US troops began to move out of their bases to live among the local population.

US soldiers in southern Baghdad, March 2007: hundreds of soldiers were killed during the first few months of the 'surge'. Photograph: David Furst/AFP/Getty Images

And they had to do two things which were fundamentally counter-intuitive; prioritise protecting the population rather than trying to defeat the enemy; secondly, reach out to the armed groups which were killing civilians and soldiers.

"The general challenged his soldiers to understand the causes of instability, to understand the 'why' not just describe the 'what'." He would tell the soldiers, 'the average Iraqi is just like you and me, they want to have their breakfast, take their kids to school and go to work. They are good people they are not our enemy'. People were using violence to achieve political objectives, so we had to create a process where they could achieve their objectives without violence. I had confidence in our analysis. But I was not sure the strategy would work. Not because I thought it was wrong, but because I worried the situation in Iraq was so out of control our extra forces might only exacerbate the violence, not lessen it."

In those first months, there were few signs of progress and there was violence everywhere they went.

"You can hear it, you can smell it, it is all around. We would go to the hospitals to visit the wounded. We would attend memorial and ramp services for the dead. Every day, the general would be slipped a note with details of casualties which went up and up. We lost over a hundred soldiers a month in April, May and June 2007. In the past, I had been a spectator, an observer. I had never been involved in the decision-making to send our soldiers somewhere. It's not like being a politician sitting in London. We were living among these men. People I knew died out there, and I am asking myself, 'what have we sent them out to die for?'

"For weeks and weeks this went on. And every day, the general would talk to commanders and troops, explain the strategy, listen to their concerns, boost their morale. He would tell them that he knew it was so tough in this gruelling heat to put on body armour and go out day after day on raids. And the general continued telling them that they were making a difference, and all the little tactical successes were helping the strategy."

Sky said she never felt in danger herself, though with hindsight, she accepts her confidence may have been misplaced.

"We were on our way to Mosul when our plane got shot at and we started to take evasive action. Then the door at the back of the plane fell open and we had to get it closed, and on the ground there was shooting, and when we got in a vehicle and it was hit by an IED. But I never had a sense that I was going to die, and I was sure the General could not die. I thought, this is not where the story ends."

Sky said she found many of the daily security briefings distressing.

"We'd have power point presentations with pictures of men who've had half their brains blown out. Some things you never forget … the smell of burning bodies. I didn't want to learn to cope with these images. The military talk about KIAs (killed in action). That's how they cope. They don't say, the victims were women and children. There was so much violence that it was almost too big to comprehend. The military has a language that is not accidental, it is used to quarantine emotion. Everyday we would hear reports that another 60 or 70 bodies had turned up, heads chopped off or drilled through. It was absolutely horrific. We could tell which groups had been behind the attacks by the way the victims had been killed."

"It can be very lonely being in command and the general appreciated having a confidante. As commander you have to show leadership, you can't show you have doubts, you have to be strong. But I was a civilian outside the chain of command who could say 'how are you feeling, are you alright, has it been a bad day? We were not peers and he was always in charge. But I could be more of a friend to him."

The awakening

Within two months of the launch of the new campaign, al-Qaida militants had claimed responsibility for an audacious suicide bomb attack on the Iraqi parliament in the heart of the fortified Green Zone; two of the bridges in the capital were also hit by truck bombs. These "spectaculars" inevitably raised further doubts about the surge among Iraqi politicians and, privately, among military commanders.

But these incidents proved to be the high-water mark. "When the insurgents blew up the parliament, everyone in Iraq was probably thinking 'this isn't going to work'. Of course there were nights when I thought, we are bringing more violence and it is causing more violence, but is it actually going to break the violence. Everything had just escalated and escalated … there were occasions when I doubted whether we were ever going to break the back of it, and whether we should call it quits.

"But by July we started to feel things were changing. We heard it first from the battalions who described how more and more Iraqis were coming forward to give information about 'bad guys', and how insurgents were reaching out to do deals. There were ceasefires everywhere, local agreements, because more and more Iraqis were coming forward wanting to work with us. The intelligence we were getting improved, and the number of Iraqi casualties started to go down."

Separately, the "awakening" in Anbar, which had begun a year earlier, began to have its own important effect. Anbar had been the most violent of all Iraq's provinces, a place where Sunni tribal leaders had joined forces with al-Qaida to fight American forces. That was until those same tribal chiefs began to see al-Qaida as a greater threat to them, and turned to the US military for help to drive the insurgents out of the region.

This process had begun before the surge, but the Fardh al-Qanoon programme put the US in a better position to work with, and build trust between, sheiks who had spent the previous four years waging vicious conflict against American forces.

"The Sunnis could see we were trying to push back on the Shia extremists, and I think that had a huge affect," said Sky. "With the awakening happening and spreading, it created the environment for the Sunnis to come back into society. This started before the surge when the Anbaris became sick of al-Qaida. In that wonderful way people in the region can switch alliances, they just changed side. One minute they are wearing al-Qaida patches on their sleeves, and the next they are taking them off and calling themselves 'Sahwa' (Awakening). They saw they could get American help, and they regarded Iran, and the Shia militias it supported, as the bigger threat, and decided to align with the US to fight them."

Talking to Bassima

While tentative progress was being made out on the ground by the military, Sky was tasked with talking to the Iraqi government and assuaging some of their fears.

One unexpected consequence of the campaign was that Shia leaders had begun to worry that through the ever-increasing awakening the US was creating a Sunni army that would eventually overthrow them.

Al-Jaidra was remarkable in many ways. She was a young Shia, in her late 30s. She had been a rocket engineer. And she was tough. Sky admits that some in the military suspected she was a "leader of the Shia death squads across Baghdad". Such criticism didn't seem to faze her at all.

When she was denounced by the US for her unwillingness to include Sunnis in the higher echelons of the new Iraqi security forces, she said: "I have had a long struggle with men … I can handle the American officers."

Over the summer and autumn, Sky made regular helicopter trips into the Green Zone to speak to Al-Jaidra, who was known for wearing the striking combination of stiletto heels and a veil.

The meetings would take place in her office which was part of the prime minister's office.

"I thought, I cannot go to speak to Maliki directly, so the best way to influence him is through Bassima. I think it would be fair to say she is not an easy woman. I would try to explain to her what we were doing and why.

"The Iraqi government could not accept some of the people we were doing deals with. To them they were bad Ba'athists, terrorists, and the awakening was creating a militia which could be a danger to the state.

"They were so suspicious of our motives … and they could not believe that the US had gone into Iraq without a grand plan. They assumed that this was all part of a conspiracy by the US to purposefully destroy Iraq, keep it weak and humiliate its people. I tried to get her to understand our position and how we had got there, and vice-versa."

To encourage Iraqi government support for the awakening, Odierno had been relaying to the prime minister "good news" stories he had received from his commanders about the Sons of Iraq, the term the US used to described the awakening.

"But Maliki was only hearing bad news from his people on the ground. He therefore assumed the US was plotting a coup against him using the Sons of Iraq! When you ask your commanders for good news, you get good news. If you ask for bad news, you get bad news."

Sky said it took "weeks and weeks" to earn Al-Jaidra's trust. It helped that they were women in similar positions. "We were both working for big men. We were the same age, and neither of us had married. And we were both trying to bring our bosses closer together.

Sky persuaded Al-Jaidra that it would be better, and safer, for the government to integrate the new groups emerging around the country into the Iraqi security forces, rather than ostracise them.

In December 2007, Odierno and Maliki were at a meeting of the National Security Conference in Baghdad. When Odierno set out why the awakening needed to be integrated into Iraq's security and the plan to do so, Maliki commented: "I agree with the general 100%."

"Some people in the room gasped," said Sky. "It was a hugely important moment. That year we went from being in hell to bringing the violence down."

Tony Blair in Iraq in May 2007: when the prime minister met Emma Sky he asked if she really was British and why she was working with the US military. Photograph: Christopher Furlong/EPA

In 2007, 15,960 Iraqi civilians were killed in violence. In 2008, the number had come down to 4,859. US casualties went from 904 in 2007 to 314 in 2008.

The British

Sky was at the heart of the US military machine and her advice was being sought at the top of the political pyramid. But she says she only ever met British diplomats when she accompanied Odierno to embassy meetings.

When Tony Blair made his last visit to Iraq in May 2007, Sky was introduced to him by Petraeus and Odierno. They told the prime minister their senior adviser was from the UK.

He said: "Are you really British? I assured him that I was British born and bred. He then asked, 'so how come you are working with the US military?' I replied, 'Stockholm Syndrome'."

To end any suspicions, Sky says she was not and never has worked for MI6.

Sky saw what the British were doing from the US side of the fence. More than 40,000 British troops took part in the 2003 invasion but, by 2007, it seemed the UK was losing control of the south to the Iran-backed Shia militias of the cleric, Muqtada al-Sadr.

And there was little political appetite to win back this territory. The early confidence that led senior members of the British military to boast to the Americans about their experience in counter-insurgency had evaporated.

"This was a time when the British were saying, 'the problem in Basra is the British presence', so the Brits were intending to pull out." Sky remembers one conversation with an American general. "He said to me, 'we are surging and the Brits are de-surging'. He didn't know the opposite of surge."

Sky added: "The British public support for this war was always very low. In America they are much more supportive of the military and even though you saw public opinion turning against the Iraq war, it wasn't to the level that it was in Britain. Of course the Americans wished the British forces were bigger and had more resources, but to be perfectly honest, the British think far more about what the Americans think of them than the Americans think about them." At the end of the day, the Americans were grateful to have the British as a close ally.

In March 2008, 30,000 troops from the Iraqi army surged into Basra to clear the city of Shia militias; the operation was called the Charge of the Knights. The British were peripherally involved, mostly giving medical and logistical help.

Brigadier Julian Free, commander of British troops in Basra at the time, admitted the UK could do little more. "We didn't have enough capacity in the air and we didn't have enough capability on the ground."

General Petraeus contacted Sky to ask how they could persuade General Odierno to replace him as commanding general of all coalition forces in Iraq. Photograph: Dusan Vranic/AP

All of which meant the British inevitably left Iraq under a cloud. "The Sadrists will always claim that they are the ones who won in the south, and pushed the British out," said Sky. "And I think the Iraqi government will claim that the British didn't stand there and fight."

With the British hamstrung by lack of numbers, and with Prime Minister Maliki overestimating the capabilities of his own forces, the US had to intervene to stop the Charge of the Knights turning into another disaster.

"The risk of failing in Basra would have been catastrophic for the country," Sky said.

The end game

AT the end of 2007, Sky left Iraq for what she thought was the last time.

But three months later there was an unexpected reshuffle at the top of the US military. The officer in charge of US Central Command (Centcom), Admiral William Fallon, was forced to resign after an article in Esquire magazine, written with his cooperation, claimed he was opposed to President Bush's approach to Iran. In the rearrangement, Petraeus was to leave Iraq to take command from Centcom and Odierno was asked to return to replace him as the commanding general of all coalition forces in Iraq.

"I was walking in the hills in France when I got this email from Gen Petraeus saying, how can we persuade Odierno to accept to come back to replace me in Iraq. General Odierno had been separated from his family for so long and had been so looking forward to going home. Within months, he was told he was being sent back to Iraq. For senior commanders, they get little choice. The poor guy, I felt so sorry for him. But General Odierno was going to go regardless. For him it was duty. And if he goes, and he wants my help, I go. That's a given."

Sky spent two months working for Petraeus in Baghdad in May and June, and then returned to Iraq as Odierno's adviser shortly before he arrived in September. This time, with broader responsibilities, she was based in the US embassy in Baghdad, but still accompanied Odierno to all his meetings.

Not everyone was pleased.

"One of the general's staff told me that everybody hated me. Someone else said to me 'if you send anymore emails to the general we will destroy you, get rid of you'. Staff like to feel they are controlling the general and they did not like him getting different ideas from me. It was upsetting, but I felt the mission was important. If I'd thought the general didn't value me there is no way I would have put up with that shit. I didn't tell the general about it. He had enough things going on. You certainly need thick skin to work with some in the military."

But such incidents were isolated, and most of Odierno's staff accepted her.

The key initial task was on negotiating a status of forces agreement, the legal basis that allowed the US to remain in the country, and for how long. Sky, the Englishwoman, was asked to represent the US military during the talks.

With a UN resolution due to expire, getting an agreement was essential before the end of 2008. "I was on of a small team under the US ambassador Ryan Crocker. If we didn't get it, the US would have to withdraw 150,000 troops within two or three months, they'd have to pack up and go home. And if the US went home, the Iraqis wouldn't get their help anymore."

"There were times when I really thought this isn't going to happen, it really came down to the wire. Some of the Iraqis were scared the agreement made the prime minister too strong and wanted reassurances. We had already done a contingency plan on the basis we'd have to leave. But, at the last moment, an agreement was signed. It specified that the military had to be out of the cities by the end of June 2009, and out of Iraq completely by 2011."

As Sky wasn't American or in the military chain of command, she was allowed to leave the Green Zone and meet Iraqis, picking up an on-the-ground understanding she fed back to the general. Photograph: Stefano Rellandini/Reuters

After so many years of fighting in Iraq, it was natural the military would find it difficult to let go.

"General Odierno would go out visiting troops and they would always say, 'security isn't good enough, there is still a risk, we cannot leave'. But by letting go, our relationship with Iraqis would improve. So the general had to get them to understand that success was something different now. We were shifting from counter-insurgency to stability, and putting Iraqis in the lead was the priority. When you do counter-insurgency the focus is protecting the people. In stabilisation, the priority is building up the institutions."

As the change in military posture and preparations for withdrawal continued, Sky remembers tensions between the military and the state department. Some of the embassy officials were on their first tours to Iraq and didn't seem as committed as their predecessors or the soldiers.

"One of the diplomats told me it was like being handed a bus with no wheels on, and I said, at least you recognise it as a bus. In the last few years you couldn't even recognise it as a bus."

Secret trips into Baghdad

Because Sky wasn't in the military chain of command, and because she wasn't an American, nobody could actually stop her leaving the confines of the Green Zone to get out among Iraqis.

These trips gave Sky a chance to speak to Iraqis and see places for herself, picking up valuable on-the-ground understanding she could feed back to the general and his staff.

"Everyone was under all these regulations. I was supposed to be as well, but being a non-American, and not coming under the British either, I was in a unique situation and Odierno trusted my judgment. I would travel at night around Baghdad to get a sense of what it was like so I could report back on different areas. I was going out with and among Iraqis. I could see if the Iraqis were working the checkpoints properly, if the electricity was on. Things like that can help give commanders the confidence to let go.

"In some places, I'd get people from the area to take me around. I was going in and out of Sadr city (a district of Baghdad), which the Americans regarded as one of the most dangerous places on earth at the time."

The year before, Sky had helped work on the ceasefire of Jaish al-Mahdi (JAM), an Iraqi paramilitary group created by Al-Sadr, so she already knew some of its members.

"I knew some of them, and I had built up a relationship with them. They had their own lives and their own motivations. Iraqis are the most extraordinary people, they might distrust each other but they can be remarkably open to an outsider."

Sky said she did not feel in danger – the people she relied upon to keep her safe on her trips into the city's underworld were taking high risks too.

"I think they felt responsible for me. I was a woman on my own, and they took good care of me. The people who would have done me harm, would have done them harm too. So if the security was good enough for them, it was good enough for me. Although the risk of kidnapping was real, I was not worried that I would be taken. I trusted the Iraqis with my life, I trusted them completely."

Sky would travel from neighbourhood to neighbourhood. "In some areas there was still something sinister, completely dark. And in others, you didn't get that at all. You could see areas coming back to life. When women and children are in the streets you know they must feel safe. Even Sadr city started to buzz, and that was very exciting."

During the day, Sky would occasionally have meetings with Iraqis at the Rashid Hotel in the Green Zone. One meeting made a particular impression. "I thought this man was just an angry Sunni, and we were trying to find common ground. We had tea together. A little later I discovered he had been arrested and was the al-Qaida emir for northern Iraq. I don't know how he managed to get into the Green Zone."

Sky still keeps in touch with some of the Iraqis she knew then, including one member of JAM, who sends her a Valentine's card every year.

Obama's first Iraqi trip

When Obama was unable to go to the Green Zone to meet Maliki, Sky played a key role in getting the Iraqi prime minister to go to Camp Victory to meet the US president. Photograph: the Guardian

The election of Barack Obama didn't change US plans to pull back from Iraq according to the timetable set by George Bush.

But Obama-mania was still very much alive when he made his first visit to Baghdad in April, 2009. He was mobbed by US troops, and Iraq's senior politicians and tribal leaders were enthusiastic to meet him too.

Though not reported at the time, Sky says the trip so nearly ended in acute embarrassment for all sides.

The problem was something even the leader of the free world could not control; the weather.

"Obama was supposed to land at Camp Victory and then go by helicopter to the Green Zone to meet the Iraqi prime minister and other Iraqi politicians. But the weather was so bad the helicopters couldn't fly. The president's security people were saying there's no way he will travel by road to the ceremony, and the US embassy was saying there's no way the Iraqi politicians will come to Camp Victory, the seat of the occupation. And I am saying, there's no way the president can come to Iraq and not see Iraqis. It is their country, he has to meet them. It would be a disaster if he didn't." Odierno told Sky to try to persuade Prime Minister Maliki to drive to Camp Victory.

"So I go over to see the prime minister, who is having his afternoon siesta. I had to wake him up. I said 'I am terribly sorry but President Obama cannot come to the Green Zone because of the weather and I hate, hate to ask of you, is there any chance you can come to Camp Victory?' Obama was new. Everyone was excited about him, and Maliki agreed. And if Maliki agreed, then the others would probably come too."

In the Green Zone, nobody else knew about the looming crisis. "President Talabani had got the band playing and was waiting for Obama to arrive, and I am trying to focus on getting Maliki to Camp Victory. You have to remember that a lot of these politicians don't get on at all, and we still had to decide the order of who sees Obama, when and where."

Odierno's residence in Camp Victory became the emergency reception area and Sky travelled with the prime minister's convoy on the way out to the base. There were myriad security check-points along the route and Sky knew the prime minister would take umbrage if he was stopped anywhere along the drive, and U-turn back to the Green Zone.

"I was in the first car, sending messages to the military to open the checkpoint gates. At every one I jumped out, waved my military badge and shouted. 'Prime Minister of Iraq, open the gate'. It was a miracle that we got him in without a major diplomatic incident." President Talabani arrived soon after, but there was nowhere for him to wait before his audience with Obama. "We ended up putting him the bedroom of Odierno's bodyguard. There was laundry all over the bed."

Sky attended all the meetings between the Iraqis and Obama, and Odierno introduced them. Despite the chaos, and the opportunities for bruised egos, the visit ended without any major diplomatic incidents.

To Sky's surprise, Maliki was so impressed with his tour around Camp Victory that he thought it would make a good site to hold the Arab Summit in 2010.

"The next day in our staff meeting General Odierno told his chief of staff to come up with a feasibility study to get all US soldiers out of Camp Victory in 2010 just in case the prime minister asked about it again. The chief of staff almost had a heart attack."

Hostages

Although the ceasefires between Sunnis and Shia were holding, tensions in the north had increased between Kurds and Arabs.

The president of the semi-autonomous Kurdish region, Massoud Barzani, and Iraq's Shia prime minister Nouri al-Maliki, did not get on well, which didn't help matters when, as Sky put it, "things began to get a bit dodgy in the north" – Kurdish peshmerga forces were squaring up to Iraqi security forces.

One episode reflected the difficulties; there had been a spate of bomb attacks close to the town of Hawija, just south of Kirkuk, which had been blamed on al-Qaida. Sky suspected it wasn't insurgents, but local Arabs fearful that their town was about to be overrun by the Kurdish peshmerga.

"I was sitting in the office in Baghdad when someone showed me a map of where all the different forces were due to be stationed, including peshmerga south of Kirkuk. I thought this upsurge in violence isn't al-Qaida, it is the Hawija Arabs. They are angry. So Gen Odierno told me to accompany one of his generals to speak to the sheiks.

"The sheikhs are not an easy lot but I had known them since 2003. I told them the peshmerga would not be positioned south of Kirkuk. And they said, 'thank God, we had to put plant all these road side bombs because we were worried you were letting them in'. This is how they saw things so they took their own defensive action."

Matters came to a head in Nineveh in February 2010, when the province's new Arab governor, Atheel Najafi, decided he was going to test his freedom of movement by taking a trip into an area of his province which was predominantly Kurdish.

"The governor is supposed to have freedom of movement, but the Kurds said he can't go in there. The Americans said he could, as part of an agreement that we had brokered.

"So the Americans escort the governor and the Kurds send reinforcements and things begin to escalate, and then shots are fired at the governor.

"The US brought tanks to a Kurdish village, and are flying F16 fighters overhead to try to calm the situation. And then the Iraqi security forces arrested some Kurds for trying to assassinate the governor."

It was not an end to the affair.

"I was woken up at 2am by the Turkish ambassador in Baghdad, who had received a report from Ankara that the Kurds had invaded Mosul. I didn't know what he was talking about and was desperate to find out what was going on.

"This was really very bad, definitely up there in the list of the most stressful events I have ever had to deal with. There hadn't been an invasion, but the Kurds had kidnapped a number of Arabs in Nineveh in response to the arrests. So we had a group of Kurds detained in Mosul, and an group of Arabs had been taken in retaliation."

Sky said the US embassy insisted that men accused of attempting to assassinate the governor should be put on trial, in accordance with the rule of law.

"When I mentioned this to the Kurds, they screamed at me 'there is no rule of law in Iraq'. Every time Barzani turned on his TV, they were showing the American tanks and the F16s. He was furious…"

Odierno told Sky to find a pragmatic solution to the crisis; realistically, it could only be solved one way – an exchange of hostages.

"I tried to organise a deal to swap the detained Kurds with the Arabs. But to do this, I needed to get proof of life of the Arab detainees. So I had to fly up to Kurdistan on the general's plane. The weather was absolutely terrible. There was thick fog, the airport was closed and the pilots couldn't see the runway. But they were determined to get me to my meeting and managed to land on the second attempt. The Kurds were amazed I'd manage to fly in.

The Kurds took Sky to a presidential guest house, but before addressing the critical security situation, her hosts said she had another appointment – with a beautician.

"They got a young Kurdish girl to look after me. I had my hair cut and my legs waxed. It was quite nice but rather bizarre. Then they said they wanted to take me to a new mall. They love their malls."

This was partly a deception; on the way, Sky was diverted to meet members of the Asayesh, the Kurdish intelligence service.

"They were holding three of the Arab hostages. I saw they were alive and well. So I called the deputy prime minister (Rafi al-Issawi) and told him I had proof of life."

Sky flew down to Baghdad to pick up Issawi and his adviser, Jaber al Jaberi, and then they all flew back to Mosul to seal the deal.

There was a further twist; the three Kurds suspected of attempting to assassinate the governor had to be taken before a court so an Iraqi judge could formally release them from custody.

The Kurds were suspicious.

"So we are sitting at the airport trying to do the deal. The Kurds have informers everywhere and there was no way they wanted the prisoners taken before a judge without having some way of ensuring they came back again.

"So we had to give up Jaber as a hostage to the Kurds. He wasn't very pleased about that!"

Two American military helicopters went to pick up the 15 kidnapped Arabs.

"The Kurdish negotiation side wouldn't let the Arabs get off the helicopters until the Kurds were back from the judge. All this time they were saying, we are going to call off the deal, we are going to call off the deal. This went on for about four or five hours … it was incredibly stressful. The mobile reception was terrible. It was on, then off, then on then off." Eventually, the Kurds and the Arabs were released.

"Issawi hugged them and gave them each some money. The Arabs had had no idea why they had been detained. Then we held a press conference in which Issawi went on about national reconciliation and on the flight back to Baghdad he was saying how great it was to do something that made all sides happy."

Conclusion

Emma Sky left Iraq, along with Odierno in September 2010, at the end of combat operations. In total, she had been in the country for 50 months, completing more tours than most military commanders.

By nature she was suspicious of armed forces, and she was no supporter of America either. So Sky was probably the last person US commanders wanted at their side pointing out where they were going wrong. Which is one of the reasons she came to like and respect them. They were brave enough to take her in, and braver still to listen to what she was saying. The British would not have dared be so bold.

Sky has thought long and hard about what went wrong in those early days, and whether enough was done in the later years to give Iraq a chance for stability.

She is angry that no one has been held accountable for a war fought over false claims of WMD which had such high costs; more than 100,000 Iraqis were killed, along with 4,486 US soldiers and 179 British soldiers.

She believes the surge helped reduce the violence and allowed US forces to withdraw in 2011 with dignity – something that would have been inconceivable years earlier.

Sky says it is probably too early to judge whether Iraq can evolve into a democracy and become a force for regional stability: "People tend to be critical of the military, but the criticism needs to be more focused on the politicians and civilian leaders who failed to set an overall strategy. No one has been held accountable. They do not understand what the military is capable of, what it can and cannot do. Success in Iraq was always going to be defined by politics. We needed a political solution, a pact, a peace. The military had been asked to fight the war and then to deal with the consequences of it, without anyone in political authority having a plan or understanding Iraq well enough to appreciate the consequences of some of their decisions.

"I don't want to live in a world where we see the killing of innocent civilians and don't yearn to stop it. However, the Iraq war should have taught us, if nothing else, about the limitations of our own power."

She is also unashamed of her conversion regarding the US military. As a self-confessed Guardian reader, she had prejudices that were challenged, and ultimately reshaped, by her experiences.

"They made me feel part of the team, and were as driven as I was to find a way of improving the situation in Iraq. I went on patrol with them, and spent hours in humvees and helicopters. I built up a camaraderie with soldiers that only people who go to war experience. Some of them remain close friends." Odierno was the best of the lot, she says.

"I would have followed him anywhere."

Sky still keeps in touch with many Iraqis – including a few who were once insurgents.

"If I had never volunteered and stepped on that plane in 2003 I would never have known that Iraq is such an amazing society. I think Iraqis are some of the most warm, generous, kind and funny people."

"Nothing in my life will ever compare to the experience I had in Iraq. I had a real sense of purpose and I don't regret going there for a single moment. People sometimes ask me, why did you go to Iraq, and I respond, why wouldn't you go?" It was the best decision of my life."