Controversy surrounding the pay of the chief executive of Barclays, Bob Diamond – and his £5.7m tax bill – intensified on Tuesday night when a leading group of shareholders were warned about problems with his remuneration package.

The Association of British Insurers, whose members control around 15% of the stock market, issued an "amber top" warning over possible breaches of corporate governance codes.

The move by the influential ABI follows a call by investor advisory body Pirc that Diamond should not receive a bonus on top of his £1.3m salary, because of the underperformance of the bank. Some shareholders, such as Standard Life, Aviva and Scottish Widows, are already believed to be considering voting against the remuneration report.

The ABI is understood to be highlighting two issues with Diamond's pay, which reached £17m in 2011 when deals from previous years were cashed in.

First, it points out that Diamond himself described the bank's performance in 2011 as "unacceptable" when it failed to reach targets for returns to shareholders. Second, the ABI highlights the bank's decision to pay £5.7m to the exchequer on behalf of Diamond.

The bank paid the tax bill as it was incurred when Diamond relocated from New York to London after his promotion to chief executive in January 2011.

While the ABI's guidelines on pay state that companies "should not seek to make changes to any element of executive remuneration to compensate participants for changes in their personal status", the payment on behalf of Diamond is not thought to be in direct breach of this guideline, which was written amid fears that companies would try to compensate directors for any change in tax rates within the UK.

Even so, shareholders admit to being surprised at the size of the tax bill. Guy Jubb, head of corporate governance at Standard Life, has said: "I don't believe anyone had an inkling that this payment, which is almost the largest figure in the remuneration report, was going to surface."

Diamond's contract, which was rewritten when he took the helm after more than 10 years running the Barclays Capital investment bank, states that the bank would pay his tax if he was being taxed twice – in the UK and in the US. The contract states: "Barclays will consult with you in good faith and give consideration as to whether and in which ways Barclays can hold you harmless against any incremental tax liability incurred as a result of your return to the UK to perform your assignment."

Barclays made clear it had been holding meetings with its shareholders over the pay deals and described the ABI report as "carefully considered". Barclays said: "We note the concerns and questions that they have raised in their report… and look forward to continuing to work with them on those."

A vote against the chair of the remuneration committee, Alison Carnwath, is also possible as Pirc has recommended voting against her re-election to the board.

Pirc pointed to the fact that Barclays shares are trading below their net asset value, and also states that the bank should consider clawing back bonuses from executives because of the £1bn provision for payment protection insurance mis-selling.