David Cameron has rejected a call by the Labour party for an independent inquiry into Britain's banking industry after the £290m fine imposed on Barclays for manipulating interest rates.

The prime minister called for a change in culture in Britain's banks but said this could not be achieved solely by amending laws and regulations.

Cameron, who was speaking at the end of the EU summit in Brussels, launched his strongest attack yet on Bob Diamond, the Barclays chief executive.

Asked whether Diamond was the right man to lead Barclays, the prime minister said: "I can't say that. He has questions to answer."

But Cameron declined to endorse the call by Ed Balls for a "proper, independent, arm's length inquiry" to be held into Britain's banking industry. "I never reject anything Ed Balls says," he joked after he was asked by the Guardian whether he endorsed the shadow chancellor's view.

The prime minister said that laws and regulations were not the only way to reform the banks as he wholeheartedly endorsed the remarks by Sir Mervyn King, the governor of the Bank of England, who called for a change in the culture of Britain's banking industry.

The prime minister said: "We know what needs to be done. The most important thing people want to see is a really concrete set of actions that will help change the culture. You don't change culture by changing laws and changing regulations alone. But if we can do this in a very clear and consistent way we have a chance of getting this right."

Cameron said King was right to make "stinging criticisms" of Diamond as he then read out the governor's main remarks a few hours earlier in London. The prime minister highlighted King's attack on excessive levels of compensation, shoddy treatment of customers and deceitful manipulation.

The prime minister said: "What [the governor] says is vitally important. We need a change of culture and we know what's gone wrong and largely we know what needs to be done to put it right. What you are going to see from the government is an incredibly methodical series of actions to deal with all of these problems."

Cameron listed a series of actions the government has announced over the last two years:

• Endorsing the Vickers report, which called for the separation of "risky" investment banking from retail banking.

• Scrapping the failed tripartite system of bank regulation.

• Handing more powers to the Bank of England on regulation and supervision of banks.

• Calling time on excessive levels of debt in banks.

• Increasing taxes that banks must pay, including the permanent bank levy.

• Introducing the toughest rules of remuneration in the world.

The prime minister also reiterated a series of measures outlined by George Osborne in his commons statement on Thursday. These were:

• New legislation to prevent the manipulation of interest rates.

• Examining whether changes in the criminal sanctions regime are needed "so that just as crime on our streets are punished crimes on the banking sector are punished too".

• Consulting on whether new criminal sanctions should be introduced so directors of failed banks are held to account.

• Changing the rules so that fines, such as those imposed on Barclays, cannot be used to reduce banks' levies.

The prime minister said: "People are rightly angry about the behaviour of the banks and so am I. People want to see real accountability for what has happened. When people have broken the rules they should face the consequences."