The internet industry has refused to sign up to plans to give law enforcement and intelligence agencies access to the records of British web and email users, throwing David Blunkett's post-September 11 data surveillance regime into fresh disarray.

In the latest of a long line of setbacks for the home secretary's data retention campaign, the Guardian has learned that internet service providers have told the Home Office that they will not voluntarily stockpile the personal records of their customers for long periods so that they can be accessed by police or intelligence officers.

Nicholas Lansman, secretary general of the Internet Service Providers Association, wrote to officials last month informing them that the industry had not been convinced that extending the length of time companies hold on to customer logs was necessary for the fight against terrorism and serious crime.

The letter, which has been seen by the Guardian, makes clear the depth of concern among web companies over the privacy and cost implications of retaining subscriber information.

Mr Lansman said that service providers were "rightly concerned" that retaining communications data beyond normal business practices may be unlawful. A paper produced by law enforcement agencies had failed to address these concerns or make a "compelling case" for data retention.

"The document fails to provide details of the number of investigations that are currently compromised through lack of available data and assess whether this is detrimental to the public interest and national security. The investigations citedrefer to cases in which officers sought data older than 15 months and where there was no national security consideration involved," he wrote.

Industry representatives and Whitehall officials have been struggling to agree terms of a voluntary code of practice introduced under the anti-terrorism legislation rushed through parliament last No vember in the aftermath of the attacks on the US. The apparent collapse of the negotiations may leave Mr Blunkett facing a choice between using his reserved powers under the legislation to force internet prov-iders to comply or dropping the measure in response to public and political opposition.

The data to be retained includes customers' names and addresses, source and destination of emails and addresses of websites visited, all of which would be available to the authorities without need for a judicial or executive warrant.

Telephone providers are also being asked to retain records of calls made and received as well as mobile phone location data.

In July, the information commissioner, the official privacy watchdog, warned the Home Office that data retention might breach the Human Rights Act because communications logs retained strictly for national security purposes could be accessed by police and intelligence officers investigating cases such as public health and tax collection.

The Home Office has refused to amend the legislation to resolve this conflict. As a result, Mr Lansman said, the association could not "recommend to members that they voluntarily comply with the proposed code of practice".

Mr Blunkett has the power to make the code mandatory. In the Guardian last month, John Abbott, director general of the national criminal intelligence service, said all communications companies should be compelled to stockpile customer logs.

Last night, Home Office insiders dismissed suggestions that the voluntary code was dead in the water. But human rights campaigners said Mr Blunkett now had little choice but to think again.

Ian Brown, director of the Foundation for Information Policy Research, said: "Civil society, Europe's data protection commissioners, and now internet service providers have all told the Home Office their data retention plans are an unacceptable invasion of privacy."

John Wadham, director of Liberty, said: "Service providers are right to raise concerns."