Yeovil is pretty much Lib Dem central. Paddy Ashdown used to be MP here and the district council has been orange for decades. Twenty-three of the 24 town councillors are Lib Dems and even the bus stops seem to be branded in the party's colours.

So there was a sense of shock on Friday when the Somerset town woke up to find that the new incumbent of the Yeovil Central county council seat was sporting the purple of Ukip.

"It is a surprise, a disappointment," said the Lib Dem town council mayor, Clive Davis, after finishing his morning surgery at the baptist church. "I think the voters are trying to tell us something about what is happening in central government."

Davis said that when he went out canvassing, voters were complaining about the coalition government and about Westminster cuts. "It's a shame – local politics should be about the local community, what is happening here. I'm not much of a political man myself. I want to help people. I hope the new Ukip person does too."

Led by the Tories since 2009, Somerset county council has struggled with £80m of cuts to its budget. Unpopular policies have included the downgrading of the library service – modified after being challenged in the high court – and reduced access to tips.

In the event the Conservatives just about clung on to power, winning 28 of the 54 seats up for grabs on Thursday. Ukip and Labour won three each and independent candidates two. The Lib Dems lost four seats, ending up with 18, but will be fighting hard for another when the election for the 55th seat – the Coker division – takes place later this month.

Nick Clegg's party lost badly in some places. In the town of Ilminster in the south-west of the county, which the party had held, its candidate finished third behind the Tories and Ukip. In Chard North, 15 miles from Yeovil, it lost a seat to Ukip and was squeezed into third place by the Tories.

Certainly, it was close in Yeovil Central. The Lib Dem Peter Gubbins, a town, district and – until Friday morning – county councillor was pipped by 17 votes.

The victorious Ukip candidate, Alan Dimmick, said immigration was the key issue. He said that when he started work at a local food processing plant in the late 80s, "98%" of the workforce was English. Now, he said, it was 50-50 Polish and English. "This is putting pressure on infrastructure, on schools," he said. "People are worried."

Sam Crabb, the leader of the Lib Dem group on the county council, said it looked as if his party had been hit by a protest vote and had been particularly affected in urban (or as urban as Somerset gets) areas such as Yeovil Central.

Still, Crabb was puzzled. Lib Dem activists had found it hard to find the hotbeds of Ukip supporters. "When we knocked the doors there were few people who said they would vote for Ukip."

Crabb insisted the grassroots workers, famously industrious, put everything into the campaign. "We knocked on thousands and thousands of doors, we dropped vast numbers of leaflets, we used a new data analysing system that worked very well." But it was hard to try to win over potential Ukip supporters because they had not found them.

Already Crabb is thinking of what the party needs to do in Somerset before the 2014 European elections. After a few hours in bed after his all-nighter at the count he was planning to leap up and hit the campaign trail in Coker to try to win that 55th seat. And when Yeovil Central is next contested, Crabb said, the Lib Dems would go after it with more tenacity than ever.