BT and Phorm secretly tracked 18,000 customers in 2006

Exclusive BT secretly intercepted and profiled the web browsing of 18,000 of its broadband customers in 2006 using advertising technology provided by 121Media, the alleged spyware company that changed its name to Phorm last year.

BT Retail ran the "stealth" pilot without customer consent between 23 September and 6 October 2006. The technology was approved, pending a further trial*.

Documents seen by The Register show that the companies used the secret profiles to target advertising at broadband customers when they visited certain popular websites.

Phorm had purchased commercial space on these websites, although their URLs are not included in the documents. The groups targeted included people interested in finance (for an Egg credit card campaign), weight loss (a Weight Watchers campaign), and jobs (a Monster.com campaign).

The technical report drawn up by BT in the wake of the 2006 trial states: "The validation was made within BT's live broadband environment and involved a user base of approximately 18,000 customers, with a maximum of 10,000 online concurrently.

"The customers who participated in the trial were not made aware of this fact as one of the aims of the validation was not to affect their experience."

The Regulation of Investigatory Powers Act 2000 (RIPA) makes intercepting internet traffic without a warrant or consent an offence.

BT claims that when it launches, Phorm's technology will be legal under RIPA, despite counter arguments from respected experts on the legislation. The ISP's and Phorm's claim is based on advice from the Home Office, which was recently published and disputed on the influential UK-Crypto mailing list.

The government advice was solicited by the ISPs and Phorm in the run up to the announcement of their partnership on 14 February. Written by civil servant Simon Watkin, it argues that the system will probably be legal if consent is obtained from users.

Watkin wrote: "Targeted online advertising services should be provided with the explicit consent of ISPs' users or by the acceptance of the ISP terms and conditions."

BT has said it plans to change its terms and conditions accordingly to comply with the law.

On the legality of the proposed opt-out system using cookies, the BT technical report states: "Whilst the... issue is not really a technical consideration of this report, it is mentioned since owing to the legal position, direct cookie dropping could not be trialed and should be verified once the legal position is clearer."

That means all 18,000 test subjects were always opted-in without their knowledge.

BT has not answered The Register's question, posed on Friday morning, over whether it believes intercepting and profiling the web traffic of 18,000 customers without telling them was a lawful act. A statement it sent us merely confirmed it performed the experiments on customer data, and repeated the party line that no personally identifiable information is used by Phorm technology. You can read the statement here.

BT also refused to reveal where in the national broadband network the thousands of guinea pigs were sourced from.

One senior source in the broadband industry we spoke to was appalled by BT's actions. "This is extremely serious," he said. "Data protection errors are generally viewed as a potentially bad thing by the industry, but not a real threat to an ISP's reputation. This seems like a breach of criminal law, which is much, much worse."

Even during the early phase of the BT/Phorm deal that the technical report describes, the pair were preparing to spin the technology to the public. "121Media [Phorm] will take action (both technical and public relations) to avoid any perception that their system is a virus, malware or spyware and to show that in effect it is a positive web development," BT wrote in the report.

Read on to see Phorm school El Reg on ethics.

A county in the south east of England

Phorm was calling its technology "PageSense" in 2006. The early iteration inserted JavaScript tags into every page users accessed in order to retrieve targeted advertising. This explains a series of strange postings containing JavaScript code, that appeared in web forums at the time.

The current version, being promoted to BT, Virgin Media, and Carphone Warehouse customers as "Webwise", does not use JavaScript in this way. BT's report identified that it makes consumers more likely to be aware that they are being profiled as they browse.

At the time of this newly-revealed first trial, Stratis Scleparis was the chief technology officer of BT Retail. He hopped across to occupy the same position at Phorm in January 2007. BT has not addressed our question over whether it is comfortable with the role Scleparis has played in the deal.

Before the controversy over Phorm began, City analysts estimated BT stands to trouser £85m annually in extra revenues.

However, an email written by Virgin Media director of corporate affairs Paul Richmond suggests the cable operator could yet pull out of its own deal with Phorm. He wrote: "We understand our legal position here [is that] we effectively have a MOU [memorandum of understanding]. We will work with this technology through trials and by sharing our understandings with the other large ISPs."

He goes on to suggest Virgin Media could back out of the plan if its brand is tarnished. "If at any stage we believe we cannot make this work for both our customers and our shareholders we will not proceed. We value our brand and our reputation enormously. Nobody knows the optimum way to implement this technology. We will trial this and find out," Richmond wrote. A Virgin Media spokesman said it has not performed any trials yet.

BT is set to conduct yet another experiment with Phorm technology, in the open this time, on 10,000 customers. Phorm itself emphasises that it is firing "a revolution in online privacy" and that consent is a key part of its proposition.

In a recent interview, CEO Kent Ertugrul told The Register: "It [the system] has got an on/off switch. There's a place consumers can go and say 'off'. This centralises control of the user's privacy in their hands. If we had anything to hide we wouldn't invite you in here."

We asked Phorm on Monday how it squares such claims with the fact that it participated in tracking and profiling 18,000 BT customers without their consent. 'Does Phorm believe its actions were ethical and if so, why?', we asked. Rather than answer the question, the company chose to send us this retort**:

We think it is unethical of the Register to seek to undermine a technology that enhances online privacy - Phorm's system ensures that ads are served with no data storage - something that will benefit readers of the Register and other websites.

In the interests of balance, we would like the Register to reflect the improved privacy environment Phorm provides over the other major online ad targeting companies detailed in the attached table.

You can view a JPEG of the Powerpoint slide Phorm is referring to here, with two important caveats: in the context of the secret trial we describe here, there was no opt-out. In the broader context of the final national deployment, a comparison with search engines is disingenuous.

Further in the interests of balance, here's this reporter's opinion of why such attempts to market ISP-level advertising targeting to web users as a privacy benefit are unethical and designed to deceive.

We'll let you judge what this all means for Liberal Democrat MP Don Foster's questions over BT's integrity and ISP trust. We've asked the UK's national telco for an interview with an executive who can account for its actions in relation to Phorm. ®

*As we've reported, that second experiment with customer data took place in July 2007 and was immediately denied by BT.