Fighting groups that battled Gadhafi have stepped in to maintain law and order after the fall of the regime, an expert on post-Gadhafi Libya told CNN.

Most of the groups are simply neighborhood watch entities. But some include hard-line Muslim Salafis and have "a very Islamist orientation," said Frederic Wehrey, a senior associate in the Middle East Program at the Carnegie Endowment for International Peace.

The group accused of being behind the consulate assault, the Imprisoned Omar Abdul Rahman Brigades, is said to be pro-al Qaeda.

JUST WATCHED

Watch how consulate attack unfolded

MUST WATCH

JUST WATCHED

Aujali: Libyans 'shamed' by attacks

MUST WATCH

Aujali: Libyans 'shamed' by attacks02:02

Another analyst, Andrew Lebovich, a Washington-based researcher focused on security issues in North Africa and the Sahel, said the militias, criminal groups, and hard-line Islamist groups in Libya make up a "somewhat diffuse" environment.

"Some groups, such as the Ansar al-Sharia Brigades in Benghazi and Derna and the Imprisoned Omar Abdel Rahman Brigades have been involved in increasing shows of force and outright attacks against Western and other targets in Libya in recent months," he said.

Wehrey, in speaking to CNN, cited two of his recent essays about security in Libya. One was in Foreign Affairs in July and the other appeared Wednesday on the Carnegie Endowment for International Peace website.

He said the fledgling government is in a bind.

Officials are trying to demobilize and reintegrate the militias and bring these groups into the government security forces, he said.

But the militia members across Libya remain loyal to their groups and distrust the new government's authority, in part because of the "taint" of a link to the Gadhafi regime, Wehrey said.

The government has used militia commanders to quell tribal fighting, subcontracted border control and defense of oil installations to small brigades, and used armed groups to provide security during elections.

In Benghazi, he said, ballots for an election were stored and counted at the headquarters of the city's strongest militia.

"The strategy of trying to dismantle the regional militias while simultaneously making use of them as hired guns might be sowing the seeds for the country's descent into warlordism," he warned.

"It has also given local brigades and their political patrons leverage over the central government. Emboldened by the writ of state authority, brigade commanders have been free to carry out vendettas against rival towns and tribes, particularly those favored by ... Gadhafi," Wehrey said.

Violence between warring militias and attacks against Western and moderate Sufi Muslim targets erupted in recent months, Wehrey said. In Benghazi, there was a "rapid deterioration" of security before the U.S. Consulate attack.

A strike on a British consulate vehicle in Benghazi in June wounded a diplomat.

The International Committee of the Red Cross said its workers endured attacks at least five times in less than three month in Benghazi and Misrata. The group announced the suspension of its activities in August.

The Imprisoned Omar Abdul Rahman Brigades, in fact, first surfaced in May when it claimed responsibility for an attack on the ICRC office in Benghazi. The next month it claimed responsibility for detonating a blast outside the U.S. Consulate in Benghazi. It also released a video of the attack.

Sufi mosques and tombs were among the sites targeted. Sufism is considered a more moderate form of Islam.

Libyan Interior Minister Fawzi Abdel A'al was quoted by Human Rights Watch as calling the attackers "groups that have a strict Islamic ideology where they believe that graves and shrines must be desecrated."

That comment, Human Rights Watch said, refers to Salafists, the name for those Muslims who want a "return to Islam as they believe it was practiced in the days of the Prophet Mohammed."

Salafists have increasingly asserted themselves in eastern Libya. In June, hundreds of fighters wielding AK-47s and black Islamist banners converged on Benghazi to call for the imposition of sharia law.

This spring, an associate of al Qaeda leader Ayman al-Zawahiri addressed a large gathering in the town square of Derna, in eastern Libya. An online video of his address has been seen by CNN.

"Salafi militias have reportedly carried out assassinations of Gadhafi-era officials, taken over radio stations and shut down beauty parlors," Wehrey said.

Derna has for years been a recruiting ground for al Qaeda.

A 2008 diplomatic cable from Christopher Stevens, the U.S. ambassador to Libya who was slain in the consulate attack, described the area as "a wellspring of Libyan foreign fighters" for al Qaeda in Iraq.

In some cases, Wehrey said, the revolutionary brigades have become "very sophisticated." They have checkpoints, security headquarters, ID cards, and their own payroll. Some militia coalitions have made handshake deals to police areas.

"These are militias that operate openly," and operate blogs and Facebook pages.

With that Salafi foothold in eastern Libya, Wehrey said, it wouldn't be hard for armed militia members to "converge on a target" in Benghazi with an hour's notice.

Was al Qaeda involved? Al Qaeda in the Islamic Maghreb, the al Qaeda branch that operates in Africa, would have connections in the country's far south, if it indeed has any network in Libya, Wehrey said.

And Lebovich said he's seen no evidence of "overt" al Qaeda in the Islamic Maghreb involvement in Libyan violence, despite reports of arrests, smuggling and weapons purchases linked to the group.

As for other militia groups, he said, "There is little public evidence linking their actions to al Qaeda, even if some of these groups' leaders have extensive militant pasts and ties with the group."

"However," he added, "we know very little about the exact compositions of Libya's hard-line militias beyond the top public leadership, making it impossible to say definitively whether or not links or relationships with other jihadist organizations like al Qaeda, or members of these organizations, are playing a role in militia-led violence in Libya."

Regarding the consulate attack, Wehrey said he thinks the strike "was a much more local affair" than one that involved al Qaeda in the Islamic Maghreb. He called it "the latest in a series of attacks by the country's increasingly active Salafis."

"Libyans' public reaction to such strong-arm tactics has been vociferous and damning," Wehrey said. "Tribes, women's groups, and civil society -- as well as the country's increasingly active social media community -- have all mobilized to condemn the recent attacks on Sufis, while mounting demonstrations of their own against the Salafis' shows of force."

Wehrey said that "much of the violence suggests a movement in search of a cause."

Salafis "are now grasping at foreign causes they believe will excite Libyans' emotions," such as the Israeli-Palestinian conflict, Syria, and anti-Americanism, he said. Meanwhile, the country's provisional government has had to deputize revolutionary brigades for security work.

"Invariably, these poorly trained bodies contain a number of Salafi militias who have used their warrant from the government to enforce draconian social mores, conduct vendettas against Gadhafi-era intelligence officers, and attack Sufis," Wehrey said.

Citizens have voiced outrage over Salafi shows of force and have mobilized against their "strong-arm tactics against, for example, the Sufis."

"For the citizens of Tripoli, Benghazi and other cities, all this is a stark and tragic reminder of the perennial problems of poor governance and the security vacuum," Wehrey said.