There is no surer sign of a liberated city than the release of prisoners. But in the case of Raqqa, in northern Syria, the 50 prisoners in question were not captives of the government, but of the Islamic State of Iraq and Syria (ISIS), an al-Qaeda franchise ostensibly aligned with rebels pushing for the fall of Syrian President Bashar Assad. Even as ISIS, which got its start as al-Qaeda in Iraq back in the days of the American war, regains territory in the Iraqi province of Anbar, it is slowly being pushed out of its northern Syrian strongholds by a broad coalition of moderate and Islamist groups fed up with its draconian interpretations of Islamic law and its abuses of power.

Days of fierce infighting between insurgent factions have led to a victory of sorts: by Monday, ISIS had largely been forced from its Raqqa stronghold, the biggest city an al-Qaeda affiliate has ever held, according to activists and fighters on the ground. The violent schism between ISIS and other insurgents within the rebellion may lead to a recalibration of Western attitudes toward the Syrian opposition, which has long been tainted by its links to terrorist groups. But even if the Western-backed rebels succeed in driving ISIS from its Syrian strongholds, the extremist threat is likely to remain. “This is not the end of ISIS by any means,” says Charles Lister, a visiting fellow at the Brookings Doha Center in Qatar, an international think tank with a special focus on conflict in the Middle East. “Even though ISIS has suffered territorial losses in past few days, it hasn’t lost the capability to inflict damage on any target they please.” ISIS has already launched at least one suicide car-bomb attack in the contested northern town of Darkoush, killing at least 17 fighters from a rival brigade, and ISIS commanders in Aleppo are accused of killing some 50 detainees — many of them anti-ISIS activists, fighters, journalists and relief workers — in a local eye hospital Monday night, according to activists. With these precedents, the threatened ISIS counteroffensive is likely to be grim, warns Lister.

Although rival militia groups have clashed throughout the three-year revolution, the uprising against ISIS is unprecedented. According to activists, it was sparked by ISIS’ abduction, torture and killing of a popular doctor, who was also member of a rival militia, near Aleppo on Jan. 1. Widespread protests and clashes broke out following Friday prayers two days later, as rebel forces began attacking ISIS fighters and positions. In response, ISIS started pulling out of several towns over the weekend, including Raqqa, though many analysts suspect it is simply a tactical withdrawal while ISIS commanders call in reinforcements from Iraq and western Syria, where it still has a strong presence. ISIS is not likely to go down without a fight. Raqqa, which fell to the insurgents in April, is the only Syrian provincial capital in rebel hands, and for ISIS commander Abu Bakr al-Baghdadi, who took control of the city almost immediately, it is the cornerstone of his future Islamic caliphate. Losing the capital at this point could be perceived as an ideological blow to an organization that says it has a divine mandate to rule over a borderless Islamic state encompassing Syria, Iraq and Lebanon.

The problem is, most groups fighting in Syria don’t agree with al-Baghdadi’s vision — at least not immediately. Their priority is the overthrow of Assad, a goal that has often been overlooked by ISIS in its attempts to consolidate territory and enforce its draconian interpretation of Islamic law on populations under its control. There are three major rebel coalitions aligned against ISIS: the Islamic Front, Jaish al-Mujahideen and the secular Syrian Revolutionaries Front, which fights on behalf of the Western-backed opposition Syrian National Coalition. The official al-Qaeda franchise in Syria, the Nusra Front, has also joined in the fight against ISIS. In April, al-Baghdadi attempted to subsume the Nusra Front, which has been designated a terrorist group by the U.S. State Department, under ISIS control, but al-Qaeda leader Ayman al-Zawahiri stepped in and ordered al-Baghdadi back to Iraq. He refused, and the two groups have managed to maintain an uneasy alliance until now.

The real difference between the two isn’t so much ideological as ethnic: the Nusra Front is largely made up of Syrians, whereas foreign fighters dominate ISIS. Abu al-Hasan, a 36-year-old spokesman for one of the Islamist brigades fighting against ISIS, worries that conflict between rebel groups sets a bad precedent. ISIS, he says, speaking via Skype on a personal basis because his brigade has yet to make a formal statement about its position, was “stabbed in the back by the other factions. When those foreigners came to Syria to help us, we all welcomed them unconditionally. Now that they’ve helped us liberate towns and seize major bases, we tell them, ‘Turn around and go back to where you came from?’” Abu al-Hasan, who goes by his nom de guerre, calls the clashes a gift for Assad. “The situation only serves the regime,” he says. “There are regime soldiers who have not had the chance to take a shower in 20 days. Now they can lay back, relax and wait to see which side is weakened and which side is eradicated completely before they make their move.”

Some Syrians are starting to believe that ISIS is in fact secretly aligned with the regime, part of a government conspiracy to undermine the rebels by making them all look like radical, anti-Western jihadists so that the West will abandon its demand that Assad step down. “The Assad regime is championing itself as a partner in the fight against extremism as a pretense to stay in power,” says Oubai Shahbandar, an adviser to the Syrian political opposition and the Syrian Revolutionaries Front, speaking by telephone from his base in Turkey. The regime, he says, turned a blind eye when al-Qaeda forces traversed Syria on their way to battle American soldiers in Iraq. “With Assad’s help, al-Qaeda established a beachhead in the Euphrates River Valley. Only the opposition is qualified to rid Syria of the extremists that threaten Syria and the West.” Success, he says, requires material and financial support for the rebels. “If you want to fight al-Qaeda, you have to double down on assistance to the SRF,” he says, using the initials for the Syrian Revolutionaries Front.

But ridding Syria of ISIS is unlikely to quash the al-Qaeda threat, notes Lister of the Brookings Doha Center. If anything, it is likely to bring even more violence and suffering, as Iraq witnessed in 2004–08 when al-Qaeda in Iraq was fighting for its survival. “ISIS is even more paranoid and defensive than ever,” he says, noting a steady stream of ISIS social-media messaging that features car bombs ready to detonate and suicide bombers in training. Nor is ISIS the only threat to Western interests in Syria.

With ISIS in retreat, the Nusra Front will seek to consolidate power among Syrian supporters, who see the organization as a legitimate Syrian rebel force loyal to the revolution, says Lister. “The ultimate irony is that Nusra, far more than ISIS, is the group that is most loyal to al-Qaeda Central.” ISIS’s defeat in Syria could conceivably allow it to strengthen its presence in Iraq, while the Nusra Front consolidates power in Syria. Either way, it’s a victory for al-Qaeda.

Aryn Baker is the Middle East Bureau Chief for TIME, covering politics, society, culture, religion, the arts and the military in the greater Middle East, including Pakistan and Afghanistan. She currently resides in Beirut, Lebanon.