The government will carry on with plans to require communications service providers to store details of all communications flowing over their networks, despite serious concerns being raised over technological and civil liberties issues.

Details of all internet communications originating on a communications service provider's (CSP's) networks, including who is communicating with whom, when, and where, will have to be stored, the government said on Friday, in response to a consultation on web monitoring which began in April.

As part of its interception modernisation programme (IMP), the government wants CSPs to store details of all web communications, including those originating on social-networking sites such as Facebook; in gaming and virtual reality environments; and instant messaging. The aim is for public authorities to be able to see who is talking to whom, and build up a picture of an individual's social interactions, as an anti-crime and anti-terrorism measure. The programme will cost £2bn over 10 years, according to official estimates.

However, as part of the consultation, organisations raised serious privacy concerns. Data protection watchdog the Information Commissioner's Office (ICO) said the plans were not proportionate.

"Based on the evidence presented in the consultation, and further documentation available to us, the information commissioner believes that the case has yet to be made for the collection and processing of additional communications data for the population as a whole being relevant and not excessive," said the ICO.

In its response to the consultation, the government said CSPs would be required to store details of all web communications that originated on their own infrastructure. At present, CSPs are required to retain data relating to email and telephone conversations.

The government proposals have been criticised by numerous IT experts as being technologically unfeasible, in part due to the numerous different proprietary protocols used by different communications companies. For example, Facebook uses different protocols from Yahoo Instant Messenger, which in turn uses different code from Second Life, and so on.

In its response, the government addressed technological criticisms, saying its plans would work. "Whilst recognising the challenges, the government is confident its proposed solution is technically feasible and will continue to work with communications service providers in developing it," said the response.

On Monday, Home Office minister David Hanson said in a statement that the government would consult CSPs on how best they could store and present communications data.

"Communications data is crucial to the fight against crime and in keeping people safe. It is a highly technical area and one which demands a fine balance between privacy and maintaining the capabilities of the police and security services," said Hanson. "We will now work with communications service providers and others to develop these proposals, and aim to introduce necessary legislation as soon as possible."

A Home Office spokesperson declined to say whether the government would attempt to bring in communications data legislation before the next general election, in 2010. Upcoming legislation will be detailed in the Queen's Speech at the official opening of parliament next Wednesday, but draft legislation disclosed in the summer made no mention of communications data legislation.

The Internet Service Providers' Association (ISPA) told ZDNet UK on Tuesday that it had expressed "deep concern" about the amount of data the government expected ISPs to retain. "It seems the government is convinced of the need to increase data retention capabilities," said an ISPA spokesperson. "We'll be looking for more details from the government."

However, IT security expert Richard Clayton told ZDNet UK he suspected the proposals had been put on hold for the moment. "If you're going to spend £2bn on combating crime, there are better things you could spend the money on," said Clayton.