Gordon Brown today sought to minimise the damage to Labour's election campaign when he personally apologised to a Labour supporter he had earlier called a "bigoted woman".

The prime minister sent the Labour campaign into turmoil when he insulted 65-year-old Gillian Duffy to aides immediately after speaking to her while electioneering in Rochdale, Lancashire.

Brown was still wearing a Sky News microphone and his comments were recorded.

The encounter came on a day Labour had hoped would focus on policy issues, but things began to go wrong when Duffy tackled Brown about Labour's plans to cut the budget deficit.

The prime minister stopped to talk to her for several minutes – after which Duffy told reporters he was a "very nice man", that she had voted Labour all her life and that she intended to do so again next week.

But as Brown got into his car, he was still wired up to the microphone, which recorded him rebuking his advisers.

He said: "That was a disaster – they should never have put me with that woman. Whose idea was that? Ridiculous."

Asked what she had said, he replied: "Everything, she was just a sort of bigoted woman."

After relentless TV reruns of the episode, the prime minister travelled back to Rochdale to see Duffy, emerging 40 minutes later from her house – which was besieged by reporters – to say he was a "penitent sinner".

"I am mortified by what has happened," he added. "I have given her my sincere apologies.

"I misunderstood what she said. She has accepted that there was a misunderstanding, and she has accepted my apology. If you like, I am a penitent sinner."

Brown said he told Duffy he "understood the concerns she was bringing to me and I simply misunderstood some of the words that she used".

Duffy, who said she was "very upset" and would not be voting Labour after learning of the prime minister's earlier description of her, did not comment after his visit.

Before going to see her in person, Brown had apologised for his comments on BBC Radio 2's Jeremy Vine show, cutting a dejected figure as he sat with his head in his hands when questioned about the incident.

He told Vine: "I apologise if I have said anything like that. What I think she was raising with me was an issue about immigration and saying that there were too many people from eastern Europe in the country."

The shadow chancellor, George Osborne, said the comments "speak for themselves, and the prime minister's got a lot of explaining to do".

"The thing about general elections is that they reveal the truth about people," he added.

"What people will see is the contrast between what he was saying publicly and what he was saying privately."

The Liberal Democrat leader, Nick Clegg, said the prime minister was "right to apologise because, when people ask you questions, whatever you think of the questioner or the question, you have got to treat them with respect and give a straight answer".

The business secretary, Lord Mandelson, said the remarks were regrettable and that Brown was "mortified" by what had happened.

"It has been a very long campaign, but that is not excuse for letting off steam as he did ... but, you know, that is what all of us do sometimes, we are all human. We do say things we don't believe.

"You don't expect them to be picked up by a microphone, but the fact is that he doesn't believe this of Mrs Duffy – he doesn't believe it either publicly or privately – he did let off steam, it is very regrettable and there is no justification for it."

When approached by Duffy, Brown had initially ignored her intervention but was then asked by senior aides to meet her.

During the exchange, she questioned him on pensions, the deficit and tuition fees, as well as mentioning the presence of eastern Europeans in Britain.

The Labour party released a transcript of the exchange.

Duffy said: "We had it drummed in when I was a child ... it was education, health service and looking after the people who are vulnerable.

"But there's too many people now who are vulnerable but they can claim and people who are vulnerable can't get claim, can't get it."

Brown replied: "But they shouldn't be doing that, there is no life on the dole for people any more. If you are unemployed you've got to go back to work. It's six months ..."

Duffy interjected: "You can't say anything about the immigrants because you're saying that you're ... but all these eastern European what are coming in, where are they flocking from?"

Brown said that although there were one million immigrants to Europe from Britain, there were also one million Britons who had moved to Europe.

At the end of the exchange, he said: "Good to see you. Thanks very much."

Duffy has lived in Rochdale all her life, working with disabled children for the council until her retirement five years ago.

She is a widow and has a daughter and two grandchildren. Her husband, a painter and decorator, died of cancer four years ago.