Many voters who switched to Labour fear that a Labour government could 'spend and borrow more than the country could afford'. Photograph: Ray Tang /Rex Features

Labour's strong poll lead is softer than it appears, because of swing voters' concerns for the country's finances if Ed Miliband became prime minister, according to new polling conducted by Lord Ashcroft, the former Conservative deputy chairman.

Four in 10 of the voters who have switched to Labour since 2010 fear that another Labour government could "spend and borrow more than the country could afford", according to the poll. And 43% of these agree Labour has failed to made clear what it would do to improve things.

Those who would consider Labour but have not yet switched are more likely than others to think the party has failed to learn its lessons, especially on spending and borrowing.

More than half of all voters polled said Labour had failed to learn the right lessons from what went wrong during its time in office.

However, many of those who supported a different party at the last election but now say they would vote Labour – especially those who have lost out through austerity – believe that a Labour government would reverse many of the cuts.

The poll findings came as the Fabian Society launched an inquiry into future spending choices, chaired by the former Treasury select committee chairman Lord McFall.

The Fabians' general secretary, Andrew Harrop, said there was every chance voters in 2015 would reject David Cameron's and George Osborne's plans for austerity, but politicians of the left had been slow to say what they would do instead on public spending. The commission would look at spending choices that could bring the deficit under control but also protect vulnerable families and safeguard the future of the welfare state," he said.

Labour may dismiss the Ashcroft poll because of its origins, but Ashcroft has built a reputation for conducting fair and accurate polling that sometimes delivers unwelcome messages to all political parties.

In better findings for Labour, the poll suggests nearly 50% of "Labour considerers"– those who have not yet switched to Labour – describe themselves as moving away from the Conservatives, indicating that the trend is still away from Cameron. One in 10 voters describe themselves as Labour considerers; more than half of these are former Liberal Democrats.

A quarter of those who have switched to Labour since the last election say they have not finally decided and may well change their minds. More than two-thirds of these Labour joiners voted Liberal Democrat at the last election, and only 24% voted Conservative; 40% of these say they are concerned that a Labour government would spend and borrow more than the country could afford.

Half of all voters, including more than a fifth of those switching to Labour, say Britain's overall level of debt would be worse today if Labour had won in 2010.

Fifty-two per cent of voters say Labour have "not yet learned the right lessons from what went wrong during their time in government, and cannot yet be trusted to run the country again".

In a large glimmer of hope for Osborne, the poll said: "Many of those who would vote Labour in an election tomorrow said they would have to reassess their voting intention if there were a real economic recovery in which their living standards improved."

Labour considerers think that if Labour had been elected the crime rate, the state of the economy, management of immigration, welfare dependency, the level of debt and the state of the economy would all be worse.

The polling was conducted among 8,103 adults between 17 and 28 October.