Some U.S. and Western officials say Islamic State is having trouble entrenching itself in Libya despite a concerted effort over the past year, but they warn that the terror group's persistence will pay off if left unchecked.

The most recent defense and intelligence estimates put the number of IS fighters in Libya at 5,000, roughly twice what it was a year and a half ago. The number is expected to grow as IS officials in Syria and Iraq continue to encourage foreign fighters to head to Libya, many trickling in from across the Middle East and Africa.

But while Libya has become perhaps the most dangerous place outside Syria and Iraq in IS’s self-declared caliphate, skepticism is growing that the group will be able to replicate the surge that propelled it across large swaths of Syria and Iraq.

“The presence of ISIS in Libya is almost exaggerated,” a European diplomat told VOA on condition of anonymity, using an acronym for the terror group. “It’s really a base of operations.”

More difficult in Libya

There are key differences between Libya and IS's core territory in Iraq and Syria, and those, officials contend, are helping to hold back the group’s progress.

It is "significantly harder" for IS to operate inside Libya, "because they don't have the homegrown people that know as much about Libya like they did in Iraq and Syria,” General David Rodriguez, the head of U.S. Africa Command, told Pentagon reporters Thursday.

Rodriguez said another factor is that the Libyan military and several Libyan militias have been able to pressure IS, making it more difficult for the terror organization to operate in places like Benghazi, Derna and Sabratha.

For now, some feel the pressure may be enough to force IS leaders to view Libya essentially as a launching pad for attacks against the West, a capability Rodriguez described as “aspirational.”

“They have not to this point been able to project that power toward Europe,” the AFRICOM commander said.

This picture released online by the Sabratha Municipal Council on Feb. 19, 2016, shows the site where U.S. warplanes struck an Islamic State training camp in Sabratha, Libya near the Tunisian border.

Fearing IS

Still, there is ample concern IS's status in Libya will change, especially if the country’s political struggles continue.

While the U.S., France, Italy and several other European nations already have some forces on the ground in Libya, all have said they will not take action against IS until they get an official request from the Government of National Accord — a government that is still not in full control of the country.

“We are still very cautious about Libya,” the European diplomat said.

"They [IS] are increasing their means there," the diplomat added. "In one year, we can expect their base will be much bigger, and then it can be used for something else.”

Libyans themselves fear IS’s potential.

“We do not know the exact numbers of those fighters,” Wafa Bugaighis, the charge d'affaires at the Embassy of Libya in Washington, warned an audience at the National Press Club late last month.

“Intelligence assessments say they are between 5,000 and 7,000, but the number is on the increase, especially after what Daesh has suffered in Syria and Iraq,” she said, using the terror group’s Arabic acronym.

Expanding the caliphate

The intelligence community believes that IS views Libya as much more than a base of operations for attacks on the West.

“I think that train has already left the station. Those cells are already in play,” said Patrick Skinner, a former U.S. intelligence officer now with the Soufan Group, a strategic security intelligence consultancy.

“I think ISIS senior leadership sees Libya as their best shot of continuing the self-proclaimed caliphate,” he said. “Fighters and families need a physical place to go to.”

And intelligence officials believe that despite the obstacles IS has encountered as it tries to expand, its operation in and around the group’s stronghold in Sirte is advanced, with consistent communication with IS leaders in Raqqa and Mosul.

“It is the most important holding for ISIS outside of Syria and Iraq,” said Jason Pack, founder of EyeOnISISinLibya.com.