Libyan military guards check one of burnt-out buildings of the US consulate in Benghazi. Photograph: Mohammad Hannon/AP

The black flag of the Islamist Ansar al-Sharia militia continued to flutter over its base in downtown Benghazi, but the garrison was nervous, braced for reprisals after the killing of the US ambassador to Libya on Tuesday night. Many in Benghazi say Sharia played a part in the storming of the US consulate that left four Americans dead.

At the gate of the militia's compound, a bearded commander dressed in black from head to toe said the talk inside was of two US warships that had been deployed off the Libyan coast. "There are two military boats," he said. "Everybody is talking about it. I know they are there, what do I need to do to prove it, swim?"

He refused to give his name or to allow journalists entry. When asked about the death of the ambassador, Chris Stevens, he terminated the interview, ducked back into the base and slammed the gate shut.

Sharia has been blamed for string of recent attacks on western targets, including the destruction of Commonwealth war graves and a rocket attack on the British ambassador in June. The group was formed early in last year's uprising and its members did much of the early fighting that stabilised the frontline in March 2011 when Muammar Gaddafi's forces threatened to capture Benghazi.

Its main base is 100 miles north-east, in the coastal city of Derna, known as the centre of Islamic conservatism in Libya. It was here that the Libyan Islamic Fighting Group was formed to combat Gaddafi in the 1990s. When it was crushed, some members fled to join the ranks of Islamists in Afghanistan.

The man with responsibility for security in Benghazi is Fawzi Yunis Gaddafi, who is from the same tribe as Libya's former dictator but was imprisoned by the former regime for his political views. Chief of the Benghazi branch of the supreme security committee, the only national military force, Yunis Gaddafi said Sharia sat apart from the security apparatus.

"There is the SSC, and [in Benghazi] there are other brigades that are not with the SSC. Ansar al-Sharia is separate," he said. But he was reluctant to close down Sharia without direct orders, despite evidence that Sharia personnel were involved in the night of violence.

"We saw some individuals there [at the consulate attack]," he said. He even knew where some of the suspects were: Benghazi's al-Jala hospital was treating several of the mob wounded in the fighting. But security forces did not dare enter the hospital because it was guarded by the Sharia brigade. "There are some who have been wounded, but if you go there the Sharia are around the hospital," he said.

On Tuesday night Yunis Gaddafi had taken panicked calls from American diplomats pleading for help as they were slaughtered. "I spoke to the Americans, they were saying please help us," he said. His units arrived too late to save Stevens and his colleagues.

He said he would have had more men on duty if he had known the ambassador was inside the embattled compound. "We didn't know the ambassador was there. It was the Americans' fault. If we had known he was there, we would have been earlier."

Diplomats have in the past muttered about Islamist units such as Sharia enjoying funding and support from Qatar and Saudi Arabia, powerful allies of the new Libya whom the government hesitates to oppose. But Yunis Gaddafi denied this was a factor. He said Sharia could only be blamed if conclusive proof emerged from an investigation.

That seems unlikely. Libya's interior ministry said it had arrested four men, refusing to confirm whether they were Sharia members, but there were no signs of an investigation at the burnt-out consulate. Police were conspicuous by their absence and journalists were allowed free access, trampling possible clues into the dust.

"You want to know who controls Benghazi?" asked a worshipper, who did not give his name, outside the Mishal al-Harem mosque after Friday prayers. "Nobody controls the streets of Benghazi."

Loudspeakers outside the mosque broadcast a sermon in which the imam, Suheib, condemned the attack. "The burning of buildings is not the path of the prophet Muhammad," the imam said.

It was a sentiment shared by worshippers mingling outside in the hot afternoon sun, and anger was growing towards a government unable or unwilling to take control. The fear across Benghazi is that the jihadists want to decouple Libya from western support, creating chaos in which they will hope to seize power.

"We have the armed forces. We should tell them [Sharia brigade] to disarm or get out, yet the people in charge [of Libya] can't give them a single ultimatum," said Khalid Faraj, a Libyan-American businessman. "What if these idiots shoot a plane down, who is going to listen to us making excuses?"