The comedian Jimmy Carr apologised for making a "terrible error of judgment" by investing in a Jersey-based tax avoidance scheme, after facing a torrent of criticism.

Carr, who was due to film an episode of the panel show Eight Out Of Ten Cats for Channel 4 on Thursday evening, issued his apology in a series of messages on Twitter amid a growing political storm.

But prime minister David Cameron, who had led the criticism of Carr, declined to do the same to the Tory-supporting Take That singer Gary Barlow, who has also been accused of using another complex scheme to avoid paying millions of pounds to HM Revenue and Customs.

"I am not going to give a running commentary on different people's tax affairs. I don't think that would be right," Cameron said at a press conference in Downing Street. "I made an exception yesterday [with Carr] because it was a very specific case where the details seemed to have been published and it was a particularly egregious example of an avoidance scheme that seemed to me to be wrong and I made that point."

On a day of fast-moving developments, Downing Street also raised eyebrows by suggesting it may now abandon plans for David Cameron and senior ministers to disclose their tax returns.

Cameron said in April that he expected information about his finances to be made public, and was relaxed about the prospect. But on Thursday No 10 said there were no current plans to do so.

The prime minister's spokeswoman said the proposal for ministers to reveal their tax details was still being "looked into". Ministers were "not closed to the idea", but it was "not a very near-future thing", the aide said.

A Times report suggested Carr was one of more than 1,000 beneficiaries who sheltered £168m from the taxman each year using a tax scheme called K2. In a series of tweets the comic said: "I appreciate as a comedian people will expect me to make light of this situation, but I'm not going to in this statement as this is obviously a serious matter.

"I met with a financial advisor and he said to me: 'Do you want to pay less tax? It's totally legal.' I said: 'Yes.' I now realise I've made a terrible error of judgment … Apologies to everyone. Jimmy Carr."

Privately, senior Tories are concerned that Cameron's comments on Carr were a tactical mistake because they gave journalists a green light to investigate the tax affairs of Conservative ministers, MPs and donors.

On Tuesday, the PM described Carr's tax arrangements as "straightforward tax avoidance". He said it was unfair on the people who pay to see Carr perform that he is not paying his taxes in the same way that they do. The K2 tax-avoidance scheme Carr is said to have used enables members to pay income tax rates as low as 1%.

The comic, who has famously lampooned fat cat bankers, reportedly protects some £3.3m a year by channelling cash through the Jersey-based company.

But on Thursday, the prime minister and his staff were careful to hold back from criticising Barlow, who received an OBE last week. It has been alleged that Take That stars Barlow, Howard Donald and Mark Owen, and the band's manager Jonathan Wild, invested at least £26 million in a scheme run by Icebreaker Management Services – which says it works within the law.

The singers and their manager are among 1,000 people who the Times says contributed £480m to 62 music industry partnerships that HMRC claims act as tax shelters.

HMRC is attempting to shut down the partnerships, which are overseen by Icebreaker Management Services, according to the Times.

Take That's lawyers say the bandmates believed the investments were legitimate enterprises and not schemes designed to avoid tax, and that all four named pay "significant tax". There has been no suggestion of any illegality on the part of the scheme or investors. The two other Take That members, Robbie Williams and Jason Orange, are not involved in the investments.

A spokesman for Icebreaker denied the partnerships were designed to avoid paying tax, saying they were created to invest money in the music industry for taxable profit. Icebreaker's website says the organisation is "an exciting opportunity to make money from a broad range of knowhow and creative material".

Cameron's spokeswoman later said the Carr and Barlow cases are different because the K2 scheme was seen as more "aggressive".

Cameron's complicated position on tax resulted in ridicule from political opponents. Shadow leader of the house Angela Eagle questioned Cameron's decision to speak out about Carr's tax arrangements while remaining silent about others.

"Oddly, [Cameron] did not take the opportunity to condemn as morally repugnant the tax avoidance scheme used by Conservative supporter Gary Barlow, who has given a whole new meaning to the phrase Take That.

"If he is also morally repugnant, why has he been given an OBE in the Birthday Honours?

"Why is the Prime Minister's view of what's dodgy in the tax system so partial? Sir Philip Green has interesting tax arrangements but far from being labelled morally repugnant in a Mexico TV studio, he has got a government review to head up," she said.