The Culture Secretary, Jeremy Hunt, said yesterday that the Government could take action to force newspapers to sign up to a new regulatory system and subject them to severe financial penalties if they refused.

Mr Hunt said he was prepared to consider "statutory underpinning" of any body that succeeded the Press Complaints Commission (PCC) so the new system had the confidence of the British public.

He stressed that he was not in favour of any statutory involvement in press content and that he was hopeful the newspaper industry would put forward its own solutions for the structure of its future regulation. The Government favoured a "light touch" approach, he said.

Mr Hunt's evidence, given to the Joint Committee on Privacy and Injunctions, has important implications for newspaper publishers, particularly Richard Desmond's Northern & Shell group, which has taken its four newspapers – the Daily Express, Sunday Express, Daily Star and Daily Star Sunday – out of the PCC. In evidence to the Leveson Inquiry last week, Mr Desmond ridiculed the body, describing it as a "tea and biscuit brigade".

Mr Hunt said one possible penalty for press organisations that refused to join the new regulatory body would be to deny them legal status as newspaper companies, thereby forcing them to charge VAT on their products. "At an extreme you could expel an organisation from an industry-led body," he said. "There's a lot of different varieties of statutory underpinning – it's about incentives to comply with the code [of practice]." The Culture Secretary said the existing PCC editors' code of conduct was "excellent" but that it lacked "effective enforcement".

Although a succession of national newspaper editors have told Lord Justice Leveson of their opposition to state regulation of the press, Mr Hunt suggested there may be support for the new regulator to be legally backed.

At the same hearing, Mr Hunt expressed reluctance to extend the remit of a future press regulator to include bloggers. A member of the committee, Lord Gold, told him and Ken Clarke, the Justice Secretary, that some bloggers were operating outside the jurisdiction and making "lots of money". Mr Clarke responded by saying that a "high proportion" of bloggers were "nuts and extremists".

Mr Hunt said the press played a valuable role in society and that it was important the public had confidence in it. During discussion of the laws on privacy, Mr Clarke also said he believed the use of super-injunctions had sharply declined.