US internet providers hijacking users' search queries

Update:Since the practice of redirecting users' searches was first exposed by New Scientist last week, we have learned that all the ISPs involved have now called a halt to the practice. They continue to intercept some queries – those from Bing and Yahoo – but are passing the searches on to the relevant search engine rather than redirecting them.

Original story posted on 4 August 2011

Searches made by millions of internet users are being hijacked and redirected by some internet service providers in the US. Patents filed by Paxfire, the company involved in the hijacking, suggest that it may be part of a larger plan to allow ISPs to generate revenue by tracking the sites their customers visit. It may also be illegal.

Reese Richman, a New York law firm that specialises in consumer protection lawsuits, today filed a class action against one of the ISPs and Paxfire, which researchers believe provided the equipment used to hijack and redirect the searches. The suit, filed together with Milberg, another New York firm, alleges that the process violated numerous statutes, including wiretapping laws.

The hijacking seems to target searches for certain well-known brand names only. Users entering the term "apple" into their browser's search bar, for example, would normally get a page of results from their search engine of choice. The ISPs involved in the scheme intercept such requests before they reach a search engine, however. They pass the search to an online marketing company, which directs the user straight to Apple's online retail website.

More than 10 ISPs in the US, which together have several million subscribers, are redirecting queries in this way (see below for a complete list). None of the companies would comment on the redirection scheme, but evidence collected by Christian Kreibich and Nicholas Weaver at the International Computer Science Institute in Berkeley, California, who discovered the redirection and have been monitoring it for several months, suggest that the process generates revenue for the ISPs.

The Berkeley team has identified 165 search terms, from "apple" and "dell" to "safeway" and "bloomingdales", that are passed to marketing companies and then redirected to the appropriate retail website. The marketing companies include organisations like Commission Junction, a Santa Barbara, California, a firm that retailers pay to supply traffic to their websites.

Organisations that provide Commission Junction with traffic, which may include Paxfire and the ISPs the Berkeley team monitored, receive a cut of any purchase their users make. The cut is typically around 3 per cent. Commission Junction said that it was investigating the behaviour identified by the Berkeley researchers.

Buy, buy, buy

The process is highly contentious. A user who searched for "apple" would easily have found the company's store via a search engine, so Apple may be needlessly sharing revenue with Commission Junction and the ISPs. Search engines are also being deprived of traffic intended for them. The ISPs are understood to have stopped redirecting Google search traffic after the company complained to them earlier this year. All the ISPs identified by the Berkeley team redirect some Bing and Yahoo searches.

The redirection can also produce unwanted results. A user wanting to read an article in The Wall Street Journal, for instance, might search for "wsj"; the redirection system would take them to a page offering subscription deals for the paper. Searches for "kindle" are sent to Amazon, the company that makes the e-book reader of that name. A normal search for the term provides links to Wikipedia, reviews of the device and links to Kindles for sale on eBay.

"This interception and alteration of search traffic is not just your average privacy problem," says Peter Eckersley at the Electronic Frontier Foundation, a San Francisco-based internet advocacy group that helped the Berkeley team investigate the ISPs. "This is a deep violation of users' trust and expectations about how the internet is supposed to function."

It is not the first time that the desire of ISPs to monitor and monetise the traffic they carry has led to controversy. In 2008, service providers in the UK suffered a backlash after it emerged that they were working with Phorm, a company that developed techniques for tracking the interests and activities of internet users. Advertisers and publishers already track users' browsing, but ISPs are in a particularly powerful position because they can observe almost everything we do online. Many users complained about Phorm's data collection, prompting several ISPs to sever links with the company.

Paxfire connection

In this case, examination of the redirected traffic has led the Berkeley team to believe that the service is provided by Paxfire. The firm, based in Sterling, Virginia, has provided advertising services to ISPs since it was founded in 2003. As well as using Paxfire to redirect specific queries, the ISPs pass many, or perhaps all, searches on Google, Bing and Yahoo through Paxfire servers – a process that places Paxfire in a similar position to Phorm.

Paxfire executives did not reply to New Scientist's multiple requests for comment, but the patents that Paxfire has been awarded, as well as others it has applied for, provide hints of its plans. In March, for example, company CEO Alan Sullivan applied for a patent for a system that would allow ISPs to create a "database of information about particular users" based on the searches and website visits observed by the service provider. The patent says that ISPs could use the information to display relevant advertising.

Paxfire is named in the lawsuit filed by Reese Richman and Milberg, alongside RCN, based in Herndon, Virginia, one of the ISPs identified by the Berkeley team. The suit, which was filed in the district court for the southern district of New York, claims that the two companies violated privacy safeguards enshrined in the Wiretap Act, a 1968 law that regulates electronic communications.

Want to check whether your ISP is monitoring your searches? Try running this tool developed by the Berkeley researchers.

Shielding searches from prying eyes

Feel uneasy about the possibility of your internet search provider keeping tabs on your searches? A simple fix is at hand. Last year, Google launched a service that encrypts its search traffic, including the search term itself. To turn this encryption on, just use "https" instead of "http" at the beginning of the address that you have bookmarked for Google.

If you're a Firefox user and want to use encrypted communication on other sites, including Wikipedia, Twitter and Facebook, consider installing the HTTPS Everywhere extension developed by the Electronic Frontier Foundation. The extension automatically turns on encryption for around 1000 sites that offer it.

List of ISPs that are redirecting some search queries

Charter and Iowa Telecom were observed to be redirecting search terms, but have since ceased doing so. Iowa Telecom stopped its redirection between July and September 2010, and Charter stopped in March 2011.

If you would like to reuse any content from New Scientist, either in print or online, please contact the syndication department first for permission. New Scientist does not own rights to photos, but there are a variety of licensing options available for use of articles and graphics we own the copyright to.