Another vacancy has opened up in Downing Street with the departure of the man hired by the prime minister to manage the Scottish media.

Paul Sinclair, the former special adviser to international development secretary Douglas Alexander, will leave his position as communications adviser at No 10 in November to become the UK managing director of the public affairs firm Hill & Knowlton.

Downing Street confirmed there were no plans to immediately replace the adviser, who was responsible for briefing the "commentariat" and the Scottish media, bringing to three the number of people on Gordon Brown's communications team whose posts remain vacant after their departure in the last six months.

Reports persist that the man brought in by Brown six months ago to revamp the communications programme, Stephen Carter, has been moved sideways.

As an aide to Alexander in his post as Labour's election coordinator, Sinclair played a key role in the decision not to hold an election last autumn. It is understood that he agreed to try working at Downing Street for six months, and one colleague said the intense political infighting at No 10 left him feeling "it was not the place for him". He was also said to believe his billing as the man briefing the commentariat eroded his influence with political commentators.

The government loses Sinclair just weeks before a key byelection in Fife. The Scottish press has been increasingly hostile to the prime minister, with the SNP rising in favour, and though Sinclair will not leave until after the tricky byelection it is not expected he will play a key role in the Glenrothes campaign.

Glenrothes is next to Gordon Brown's constituency and the party fears a repeat of the 22% swing to the SNP seen in Glasgow East when Labour lost the seat. Its majority there was some 3,000 larger than that being defending at Glenrothes. It is thought the party will aim to schedule the byelection around the time of the US presidential election to limit any publicity the SNP may attempt to drum up.

At his monthly press conference yesterday, Brown admitted that he was responsible for Labour's poor performance in the opinion polls, but said the economic crisis was not confined to the UK. "I'm the person who is in charge and I have got to take responsibility for what happens. But if you look around the world, you will see that every country is affected."

A Channel 4 News/YouGov poll yesterday showed a 12% swing to the Tories in 60 Labour-held marginals that Brown must retain to gain another term. The poll also showed that only 1% of respondents rated Brown's performance as excellent.

Other departures from Brown's team include Martin Sheehan, who quit as head of strategic communications to join Portland PR, and chief speech writer Beth Russell, who left for the Treasury in June. Downing Street would not confirm reports that Russell was to be replaced by Tim Kiddell, a civil servant who worked on health and wellbeing policy, but said the process to replace Sheehan was in place.