The Syrian government has withdrawn large numbers of troops from the Golan Heights in a move that has cast doubt over the future of a UN peacekeeping force on the strategically vital plateau and increased the risk of an intervention by Israel in the conflict.

Western diplomats said the Syrian redeployments near the Golan ceasefire line were the most significant in 40 years, with at least several thousand soldiers thought to have been moved in recent weeks to battle fronts closer to Damascus.

Rebel groups have moved into the vacuum, and Israel fears that jihadists will use the area as a staging ground for attacks on territory it controls.

Meanwhile, the United Nations observer force on the Golan Heights, Undof, finds itself in an ever more vulnerable position, with states whose peacekeepers comprise the mission known to be reconsidering their commitment, including the Austrians, who provide the largest individual contribution of troops.

The eastern border of the Golan Heights, an area of high ground that was seized by Israel in 1967 during the six-day war, was until recently thought to be occupied by four Syrian army divisions whose positions helped make the Golan the safest of Israel's four borders for more than four decades.

"They [the Syrian government] have moved some of their best battalions away from the Golan," said a western diplomatic source of the Syrian changes. "They have replaced some of them with poorer-quality battalions, which have involved reducing manpower. The moves are very significant."

Separate media reports in Israel suggest the Syrian redeployments could amount to as many as two divisions – up to 20,000 soldiers.

"Undof is of the highest importance, now more than ever," said one senior Israeli government official. "We know some participant countries are having second thoughts and we're concerned about that. We are talking to them to try to understand what they plan on doing if the going gets rougher. We know some are hesitating, and it's a problematic situation.

"We are also talking to New York [the UN headquarters] about whether there could be a replacement in case one contingent pulls out. We don't envision a scenario in which Undof dissolves but we are very aware of the fragility of the situation."

Croatia withdrew its troops from Undof in February, putting extra onus on the Austrians to stay.

Another senior Israeli official said: "It's clear Undof is having very serious problems in meeting its challenges. But Israeli national security figures are very sceptical as to the real utility of international forces in dealing with our security issues.

"We are very concerned [about the Golan]. Since 1974 the Golan has been remarkably quiet. That has now changed, and we are following the situation very closely. As you know, we are building a fence along the border and monitoring matters very closely. We are aware of different actors in close proximity to the border, and we are watching them very closely."

The former commander of the Israeli Defence Force liaison unit responsible for relations with peacekeeping forces, Brigadier General Baruch Speigel, said: "It's a very sensitive situation [with Undof]. It's important to find a mechanism to allow them to stay, but I'm not sure if it's possible because of the situation in Syria.

"If the UN is unable to fulfil its mission, this is a big, big dilemma. No one can tell you the bottom line. We have never faced this situation, but we have to act very responsibly. But worst-case scenarios can bring us worst-case answers."

Austrian soldiers comprise close to a third of the 1,000-strong Undof contingent and are considered essential to the viability of the mission. Vienna has made no decision to pull out its troops, but the cabinet is known to have held recent discussions about the issue. The Austrian foreign ministry did not return calls.

The area from the southern Syrian town of Deraa – where the uprising started in March 2011 – to the Jordanian border has been the most active battleground in the country since early January. Rebel groups and jihadists there have made significant territorial gains since late January, including in the Quneitra governorate near Israel, where they took over an artillery base two weeks ago near the demilitarised Golan buffer zone.

Since then a key border crossing to Jordan has been closed and Syrian opposition figures claim that it is now being used as a resupply route to rebel groups.

The CIA is known to have trained a small number of Syrian rebels at bases in Jordan in an attempt to drive a wedge between jihadist groups such as the al-Qaida-aligned Jabhat al-Nusra, which continues to gain prominence in key Syrian battlefronts. Informed sources, who refused to be identified, said some of the US-trained rebels were deploying to the Golan area to act as a buffer between the jihadists and Israeli units.

Israel has reported a number of incidents of small arms fire directed at its units near the ceasefire line. It has three times fired back across the border with missiles, but has not publicly blamed either rebel groups or loyalist forces for the attacks.