On a recent Friday morning in west Baghdad, 20 of Hajji Abu Abed's men were shifting their feet nervously in the dusty yard outside his house as they waited for their leader to emerge.

The men, young and well armed with Kalashnikovs, pistols and hand grenades, were wearing the favoured dress for militiamen in Iraq these days: green camouflage commando uniforms decorated with bits of US army kit - a pouch on one man, webbing on another, a cap here, sunglasses there, a few flak jackets between them. Some bore the insignia of Iraqi army officers.

Around noon, a fighter came running from the large house across the street and shouted: "The Hajji is coming!"

A pick-up truck came speeding into the yard, followed by several saloon cars packed with fighters. In the back of the pick-up, a man with a bandanna swung a big machine gun on its mounting. The great iron gate opened and Hajji Abu Abed emerged - a squat, chubby fellow with close-cropped hair and a thin goatee and moustache. Half his face was covered with large wraparound sunglasses, a pistol was tucked into his belt and a short machine gun dangled in his hand. Three guards ran in front of him and jumped into a new Toyota saloon. With sirens wailing and men brandishing their guns in the air, the convoy drove the 50 metres from Hajji Abu Abed's house to his headquarters.

Abu Abed, a member of the insurgent Islamic Army, has recently become the commander of the US-sponsored "Ameriya Knights". He is one of the new breed of Sunni warlords who are being paid by the US to fight al-Qaida in Iraq. The Americans call their new allies Concerned Citizens.

It is a strategy that has worked well for the Americans, on paper at least. This week, the US military claimed it had forced the extremist group al-Qaida in Mesopotamia out of Baghdad altogether, and cut the number of murders in the city by 80%. Major General Joseph Fil, commander of US forces in Baghdad, said: "The Iraqi people have decided that they've had it up to here with violence."

Critics of the plan say they are simply creating powerful new strongmen who run their own prisons and armies, and who eventually will turn on each other.

A senior Sunni sheikh, whose tribe is joining the new alliance with the Americans against al-Qaida, told me in Beirut that it was a simple equation for him. "It's just a way to get arms, and to be a legalised security force to be able to stand against Shia militias and to prevent the Iraqi army and police from entering their areas," he said.

"The Americans lost hope with an Iraqi government that is both sectarian and dominated by militias, so they are paying for locals to fight al-Qaida. It will create a series of warlords.

"It's like someone who brought cats to fight rats, found himself with too many cats and brought dogs to fight the cats. Now they need elephants."

A former intelligence officer and a pious Sunni, Hajji Abu Abed has the aura of a mafia don. And for Abu Abed, like a don, connections are everything. His office is decorated with pictures of him hugging US officers, including the senior commander in Iraq, General David Petraeus, and a Captain Cosper.

On Abu Abed's desk stands a glass box containing a black suede cavalry hat and a letter proclaiming him an honorary US cavalryman. In a silver frame is a picture of him with a female interpreter in military uniform.

As the Hajji settled into his office, a long line of men formed at the door. From a small purse tucked into his belt he dispensed handfuls of Iraqi dinars to his followers as they filed through. He is the only figure of authority many of them have seen for several years.

One old man asked him for an electricity generator; another, carrying a large file, asked him about a US construction contract that he was promised. Two young boys were seated next to him. One had brought him a leather ammunition belt, and the other handed him the keys to a new pick-up truck Abu Abed had ordered.

Neurotic

The Americans pay him $400 (£200) a month for each fighter he provides, he said, and he had 600 registered. His men are awed by his courage, his piety and his neurotic rages.

Like many other insurgent groups, the Islamic Army had an uneasy alliance with al-Qaida. On one hand they needed financial support; on the other, al-Qaida became a burden, bringing upon the Sunnis the wrath of Shia militias and death squads who started an organised campaign of sectarian cleansing against the Sunnis in retaliation against al-Qaida's mass killing of Shia.

"We lost our area," Abu Abed said. "It became a battle zone between al-Qaeda and the Shia militias."

So when a prominent Iraqi Sunni politician who had lived in the US returned to Iraq last year and started direct talks between the Islamic Army commanders from his tribe and the Americans, Abu Abed was prepared to listen. "A year ago we reached the decision that we needed to fight al-Qaida," he said. "I knew I couldn't fight them face to face - they had more men and weapons. So I started gathering intelligence on their commanders. I knew them all very well."

The turning point came last year, when al-Qaida declared the establishment of the Islamic State of Iraq and attempted to impose itself on other insurgent groups. In one instance in west Baghdad, they demanded 25% of all the loot from other insurgent groups' operations. The Islamic Army refused to pay and direct confrontations ensued.

"The bodies piled up in the streets," Abu Abed said. "Most of the people had to leave the area and flee."

The Hajji and his men used the same techniques they mastered as insurgents against their former allies. Sitting on a big sofa in his office, he recounted the events. "When we decided to attack we started with assassinations. We killed six [al-Qaida] commanders in the first week of fighting," he said. "We would drive in unmarked cars, shoot a commander dead and then flee. At first, no one knew who was killing them."

Soon an open war started. Of the hundreds who pledged to fight al-Qaida, only 13 actually stuck with Abu Abed. These days, almost all his followers claim to have been one of the 13. "When the Americans intervened, we went out with them on missions, leading them to the Qaida fighters," he said.

He pulled his pistol out and showed it to me. It was a Glock, supplied by the US to Iraqi security forces. "This belonged to the commander of al-Qaida here," he said. "They called him the White Lion. I killed him and got his gun."

Our conversation was interrupted by the arrival of a big man named Bakr with a bandolier of bullets over his chest. He squatted next to Abu Abed, laid his big BKC machine gun down and spoke to him conspiratorially, covering his mouth with his hand like a schoolgirl.

Bakr was Abu Abed's head of intelligence. "I was told that someone from al-Qaida is in the area," Bakr said. "We will go out, develop some intelligence and then raid the house."

The only vehicles in the streets belonged to our screeching convoy. A few shops were open and people walked past carrying plastic shopping bags. All around us were the traces of battle: craters in the road from improvised bombs, facades pockmarked with bullet holes, a pile of rubble that had once been a building.

Ameriya is a closed zone, surrounded by high concrete walls. Only pedestrians are allowed through the two Iraqi army checkpoints out of the suburb. The "knights" are the only authority inside.

When we arrived at the house where the alleged al-Qaida commander was hiding, Bakr was already in action. He was dragging a plump man into a car, grabbing his neck with one hand and his BKC machine gun with the other.

The horrified man begged them not to take him. "By Allah, I didn't say Qaida is better than you, you are our brothers, just let me go!" A gunman kicked the man and pushed him into a car.

The suspect's brother, still in his pyjamas, pleaded, and women in nightgowns stood in the street wailing and begging the gunmen to release him.

The gunmen pointed their guns at the people and pushed them back. A young fighter carrying an old British sub-machine gun fired a burst into the air.

Abu Abed walked into the scuffle. The detained man was not the target. Someone had overheard him saying Abu Abed's men were "worse than al-Qaida" after Bakr's men raided the house.

Furious at the insult, Abu Abed aimed his gun at the brother. "Al-Qaida is better than us, huh? Did you forget when the bodies were piled in the streets?"

Some neighbours intervened, and the man was released. His brother grabbed him by the arm and pushed him inside.

Abu Abed, shaking his head and waving his gun, walked back to his car, murmuring "Al-Qaida, better than us..."

He stopped in mid-stride and turned to charge with his men back into the house. They pushed the gate open and ran inside firing their weapons in the air. In the dark kitchen, they grabbed the man again, pushed him to the floor and kicked him. The women were screaming and crying. One of them pulled away her headscarf and wailed, holding on to the man's ripped shirt as Abu Abed and the gunmen dragged him out, kicking and slapping him. Other fighters fired their Kalashnikovs in the air. The man was shoved into a car, as was his brother.

Abu Abed, screaming and pointing his gun, charged at the crowd. "Qaida is better than me? I will show you!"

He held his gun high and quoted al-Hajjaj, a 7th-century ruler of Iraq, in a hoarse voice: "Oh, people of Iraq, I had come to you with two swords, one is for mercy which I have left back in the desert, and this one" - he pointed his gun at the crowd -"is the sword of oppression, which I kept in my hand."

The convoy drove off, sirens blaring, fighters hanging out of the car windows.

After we had settled again in his office, Abu Abed told me of his grand dreams. "Ameriya is just the beginning. After we finish with al-Qaida here, we will turn toward our main enemy, the Shia militias. I will liberate Jihad [a Sunni area next to Ameriya taken over by the Mahdi army] then Saidiya and the whole of west Baghdad."

Rumours

Hours later the Ameriya Knights were on the streets again. There were rumours that Iraq's Sunni vice-president, Tariq al-Hashemi, was visiting Ameriya for the first time in two years. As we approached the mosque where he was believed to be praying, the street was blocked by his guards.

"Open the road for the Ameriya Knights," yelled one of Abu Abed's men.

"I can't, I don't have orders," replied a gunman. "Do you know who I am? I am the commander of Ameriya," Abu Abed screamed at the vice-president's commander of guards. "Who are you? Did you dare to show your faces here before I kicked al-Qaida out? Even the Americans with their tanks couldn't come before I liberated Ameriya." Bakr pointed his gun at the entourage. Guns were cocked on all sides.

"Abu Abed, we all know who you are, but this is the vice-president of Iraq."

"This is Ameriya, not Iraq! Here I rule, I am the commander, I can make sure that you won't show your faces here!"

"We are all Sunni brothers. The Shia militias will be happy to see us fighting; we have the same enemy," said the man.

"You are trying to claim my victory. I will show you!" Abu Abed pushed the officer and went back to his car.

That night, Abu Abed decided to attack another group of Ameriya Knights under his general command. He suspected their commander, Abu Omar, was allied with the vice-president's Islamic party, which has been trying to control the Sunni area.

"I have to show them there is one commander. If the Americans don't like it, I will withdraw my men," he told me. "Let's see if they can fight al-Qaida alone." By sunset, his men were gathered in front of the house again. He distributed extra guns and he carried an extra shotgun with his machine gun.

All the way to Abu Omar's HQ he was humming an Islamic verse in a beautiful voice. "Oh prophet, how beautiful your light is, oh prophet of God."

Abu Omar's gunmen, thinking Abu Abed was there for an inspection, took away the coils of razor wire and opened the gates. Then Abu Abed's Knights charged for the third time that day, this time accompanied by gunfire. Bullets whizzed in their confused way and red tracers flashed against the dark blue sky.

Abu Omar's men were rounded up. Some were put in pick-up trucks, others were squeezed in car boots. By the light of headlamps, Abu Abed's men looted weapons, ammunition boxes and radios.

"Look, this head of yours, I will cut it off and put it on your chest if you don't tell where the guns are by tomorrow." He tried to put his shotgun in the boy's mouth but his men restrained him.

Humming

Back at Abu Abed's HQ, the men were put into cells. Men in US-supplied blue uniforms were being jailed by men in US-supplied green uniforms.

An American officer, Captain Cosper, visited Abu Abed that night. He sat in the office trying to make sense of what was going on. "They [the Concerned Citizens] are not allowed to detain people or conduct raids," he told me.

In a nearby room, two blindfolded men were being questioned by Abu Abed's men. An American soldier put his head inside, watched for a few seconds and left. "They won't do anything to them while we're here," he said.

When Capt Cosper had gone, the men were beaten up and taken to the cell. Later, one of Abu Abed's men drove up and shouted: "I brought another one." His face was shining with happiness.

"Where is he?" asked a captain.

"In the boot," replied the gunman. "I found him standing in the street behind Abu Omar's building."

"Are you sure you didn't capture Mudhar? I asked him to guard the back."

"No, no, I am sure he is one of them," said the fighter. The captain pulled out a shaking man from the boot like a magician pulling out a rabbit. "Ah Mudhar, I am sorry," said the captain. "I told you he is one of us."

The fighter kissed Mudhar twice and said he was sorry but Mudhar should try not to look so suspicious in future.