The chairman of HSBC has admitted his shame at the “horrible reputational damage” the bank has suffered following the revelations of the systematic aiding of tax avoidance at its Swiss subsidiary, but has refused to take personal responsibility for the failings.

Douglas Flint, who was finance director at the time HSBC took over the Swiss subsidiary, infuriated members of the Treasury select committee on Wednesday by blaming the failings at the Swiss unit on local managers and said that the secrecy surrounding banking in the country made it difficult for him to have a direct line of sight of what has happening at the bank.

Flint, who has been chairman of the bank since the end of 2010, said: “I believe in personal accountability and I do believe people should be held responsible for what they have direct oversight over when they have failed”.

While he said he felt “very ashamed” of events at the bank, he said he would not forfeit past bonus payments in response, telling MPs: “I don’t feel that proximate to what was happening in the private bank.”

Green, an ordained Tory minister, has steadfastly refused to answer questions in public about the scandal – citing a “point of principle”. He has also, to the anger of some MPs, managed to avoid facing questions on his role either from the media or from the two select committees that have been examining the fallout from the scandal. There is some suggestion that Tories are eager to keep Green from attending a select committee hearing.

On Wednesday, Flint largely blamed the directors of HSBC Group private banking, Chris Meares and Clive Bannister, saying they “certainly bear fairly direct responsibility for what went on in the private bank during their stewardship”.

He added: “Most accountable, I think, are the management in Switzerland. It’s very difficult for people outside Switzerland to get any access to the detailed account-level information in Switzerland. That’s something only the management on the ground can have access to for all the privacy and secrecy reasons.

“The individuals that I think are most accountable both for the data theft and the weakness that allowed that to happen, and for the behaviour that was unacceptable in relation to our standards, were the management on the ground in Switzerland.”

It was clearly unacceptable. We very much regret this

Stuart Gulliver on HSBC Suisse practices

In his most reflective remarks, Flint admitted he had a shared responsibility, adding: “One of the most humbling things that has happened in my career is a recognition of all the things you did not know, and you go and say: ‘What could I have known or what should I have known?’”

Anger at the stance of HSBC was not just confined to the activities of the bank a decade ago, but also its systemic culture of failing its responsibilities.

Stuart Gulliver, HSBC’s chief executive, was also asked about his pay arrangements, but said they had not been intended to avoid tax. He conceded it looked strange that his salary while at HSBC in Hong Kong had been placed into a Panamanian account, but he insisted he had made no tax benefit as a result. He said he had made the arrangement in 1998 purely to hide his salary from HSBC staff who might have been able to see it. “I can understand how people find these kind of arrangements unusual and rather strange,” he said.

Both bankers insisted they had not discussed the activities at HSBC’s Swiss arm with anyone at the Treasury, but said they had held two meetings with HM Revenue and Customs.

Stuart Gulliver, HSBC chief executive, apologises for ‘unacceptable events’ at the Swiss arm of the bank

Gulliver also said the bank had not used its UK newspaper advertising budget to manipulate editorial coverage, but he said some adverts due to be in the Daily Telegraph had been postponed, not cancelled, when the paper highlighted HSBC’s activities in Jersey.