Free Expression Principles

Major technology companies, including Google, Yahoo! and Microsoft, have agreed, in principle, to a voluntary set of principles designed to “guide businesses when they encounter laws and practices that may contravene international human rights standards or be at odds with law or culture in their home jurisdiction.” The objective is to protect and advance freedom of expression and privacy. Included in this initiative are mechanisms to provide for ongoing learning as well as the monitoring of compliance.

Google, Yahoo! and Microsoft sent letters to Sen. Durbin announcing the agreement. The letters re-state each company’s commitment to freedom of expression and highlight the core components of the initiative including the principles, the implementation guidelines and the accountability and learning framework.

Google’s letter draws on my report that compared Google, yahoo! and Microsoft’s search engines along with the domestic Chinese company Baidu. The most significant point centers on the impact of engagement. I found that the presence of foreign search engines resulted in an increased amount of information being available to Chinese Internet users. More specifically, I found that:

When the results from Google, Microsoft and Yahoo are combined, 20% of the sites censored by Baidu are available. However, individually they provide more information, especially Google and Microsoft which provide, on average, 51% and 55% more content (content not available in Baidu) while Yahoo! averages 25% more.

Since the search engines were censoring different content mixing searches across multiple search engine resulted in the ability to find sites censored by the other search engines.

Also, I noted that Baidu, the leading Chinese search engine, had introduced a censorship notification following the lead of the foreign search engines. Unlike foreign search engines under pressure from the home governments Baidu is not. While a still a small step, it shows that engagement can make a difference and that industry standards are important. that is why I think the principles for free expression and privacy are so important. They present a united effort and set an industry standard.

Engagement certainly presents a series of hard choices, but is a better choice than disengagement when it comes to information and communications technologies. These technologies build the bridges that connect diverse people and places, putting up barriers is what the censors do. I find it hard to believe that the promotion of free expression is served in Iran by denying Iranians access to the Java programming language.

The catch here is that this agreement and these principles are not an end point but a starting point. As I noted in my report the overall level of transparency is low — there is work to be done in this area. The process for determining what to censor is still unclear and supports the secrecy and unaccountability of China’s censorship policies. Even within a restrictive environment such as China I believe there is much more that can be done. (See below). I also showed that while the total amount of censorship may not be high, the significance of the censored sites is important.

These censored sites are often the only sources of alternative information available in the top ten results for politically sensitive search queries. Moreover, even the uncensored versions of these search engines highly rank content that is hosted in China or ends in the domain suffix .cn, both of which China retains control over and are thus unlikely to present alternative information.

China recently unblocked many censored web sites after intense international pressure and scrutiny after having promised uncensored access during the Olympics. Andrew Lih tested a sample of websites normally censored in China and found them to be accessible. The web sites of human rights groups such as Human Rights Watch, Reporters Sans Frontiers and Amnesty International are all now accessible.

Andrew posted his test results on August 1st, 2008, five days later search engines are still censoring sites that are not unblocked in China. For example, Yahoo! Microsoft and Baidu are still censoring www.amnesty.org while Google is not. Google, Microsoft and Baidu are still censoring www.hrw.org while Yahoo!s not. (Yahoo! has only one result, www.hrw.org/russian, I’m not sure how many Russian speakers there are in China, anyone know?) Only Microsoft is still censoring www.rsf.org — even Baidu is not. In fact, Microsoft is censoring more of these newly unblocked websites than the Chinese company Baidu! Another noteworthy observation is that Yahoo! is censoring the least of these newly unblocked sites.

DOMAINS

Google

Yahoo

Microsoft

Baidu

ip =
"203.208.39.99"
host = "www.google.cn"

ip =
"202.165.102.243"
host = "one.cn.yahoo.com"

ip =
"202.89.236.206"
host = "cnweb.search.live.com"

ip =
"202.108.22.43"
host = "www.baidu.com"

chinese.wsj.com

OK

OK

OK

OK

cn.reuters.com

OK

OK

OK

OK

news.chinatimes.com

OK

OK

CENSORED (0)

OK

olympics.scmp.com

OK

OK

OK

OK

udn.com

OK

OK

OK

OK

www.amnesty.org

OK

CENSORED (0)

CENSORED (0)

CENSORED (0)

www.atchinese.com

OK

OK

CENSORED (0)

OK

www.ftchinese.com

OK

OK

OK

OK

www.hrw.org

CENSORED (0, message)

OK

CENSORED (0)

CENSORED (0)

www.libertytimes.com.tw

CENSORED (0, message)

OK

OK

OK

www.mingpaomonthly.com

OK

OK

OK

OK

www.mingpaonews.com

OK

OK

CENSORED (0)

OK

www.rfa.org

CENSORED (0, message)

OK

CENSORED (0)

OK

www.rsf.org

OK

OK

CENSORED (0)

OK

www.scmp.com

OK

OK

OK

OK

www.voanews.com

OK

OK

CENSORED (0)

CENSORED (0)

www.yzzk.com

OK

OK

OK

OK

www1.appledaily.atnext.com

OK

CENSORED (0)

OK

OK

zh.wikipedia.org

OK

OK

CENSORED (0)

OK

* If at least one result was returned for a “site:” search on a domain, it was marked as OK.

To be fair, it does take time for search engines to respond. They have multiple servers, it may take time for them all to be updated. Also, there differences in implementation between those that crawl and index the web from behind China’s filtering system and those that do not and thus have to “de-list” results. (See the report for details on this.)

Still, I find it difficult to accept that sites that are unblocked in China remain censored in these search engines.