Two weeks after the Obama administration announced charges against suspects in the Benghazi attack, a large portion of the U.S. team that hunted the suspects and trained Libyans to help capture or kill them is leaving Libya permanently.

Special operators in the region tell Fox News that while Benghazi targets have been identified for months, officials in Washington could “never pull the trigger.” In fact, one source insists that much of the information on Benghazi suspects had been passed along to the White House after being vetted by the Department of Defense and the State Department — and at least one recommendation for direct action on a Benghazi suspect was given to President Obama as recently as Aug. 7.

Meanwhile, months after video, photo and voice documentation on the Benghazi suspects was first presented to high-level military leaders, the State Department and ultimately the White House, prison breaks in the country have eroded security. U.S. special forces have now been relegated to a “villa,” a stopover for the operators before they’re shipped out of the country entirely.

“We put American special operations in harm’s way to develop a picture of these suspects and to seek justice and instead of acting, we stalled. We just let it slip and pass us by and now it’s going to be much more difficult,” one source said, citing 1,200 prisoners escaping two weeks ago. “It’s already blowing up. Daily assassinations, bi-weekly prison escapes, we waited way too long.”