Two weeks ago, on 17th December 2014, the Government of India's Ministry of Communications and Information Technology issued an order to all licensed Internet Service Providers (ISPs) in the country to block access to 32 websites, effective immediately. Not only did the ban affect access to popular cultural sites such as archive.org, vimeo.com, dailymotion.com, but the order also blocked access to sites like github.com, pastebin.com, which are useful for all sorts of people but are especially popular with software developers.

A copy of the order that the MCIT's Department of Telecommunications sent to ISPs by email can be found here (356kB, compressed PDF) or here (2MB).

The Ministry's order was issued following a request from the Mumbai police's Anti-Terrorism Squad on 15th November 2014. The police request argued that the targeted web services were being used for "Jihadi Propaganda" by "Anti-National groups", and were encouraging youth in the country to join organisations like the Islamic State (ISIS/ISIL).

However, many of the blocked sites are large resources for general use by diverse communities which have no links to terrorism. Tools which are important in the daily work of India-based software developers are included in the banned sites, whose work in the IT sector is penalised by broad bans with the excuse of anti-terrorism measures.

As IT professionals in India attempt to continue to do their jobs, there has been a lack of information about the nature of the site bans. We thought it would be a good idea to do some research using free and accessible tools and to look at how censorship has been implemented, as well as the various circumvention techniques people are using.

A summary of our key findings

Between January 1st and 3rd 2015, we conducted censorship measurements from various Internet connections using seven different ISPs in India. These include TATA Communications and the state-run Mahanagar Telecom Nigam Limited (MTNL). The understanding we currently have is preliminary and draws on the browsing experience of several customers of different ISPs around India as well as information gained through the use of the open source censorship measurement toolkit provided by Open Observatory of Network Interference (OONI) and other manual tests we conducted.

The censorship order issued by the Ministry specifies what to block, but not how. Unsurprisingly, this has led to a situation where different ISPs are blocking sites using different techniques. Users of some ISPs may be able to circumvent the censorship by simply changing their DNS settings, while others will need to configure proxies or install circumvention software. In all cases we have observed, censorship can be bypassed using standard circumvention tools such as the Tor Browser.

We saw a variety of different block pages across multiple ISPs. Here are screenshots of the six we captured.

Besides finding that different ISPs use different methods of blocking, we also found that the same sites might be blocked with different methods even from the same ISP. The "not found" and "this is a comment" error pages appeared across multiple ISPs, which could indicate that there are multiple layers of blocking so that if the first one "fails open" another layer catches it. Even so, the blocking is unreliable--when requesting the same site many times, it sometimes loads and sometimes yields a censorship message or error page.

TATA appears to be using a proxy server to inspect and modify traffic to certain IP addresses. If the request is for one of the censored sites, a block page is returned instead. We can tell that the filtering is only being applied to certain IP addresses by sending HTTP requests for censored hostnames to the IP addresses of unrelated websites. Using some TATA connections, requests to some IP addresses are blocked based on the content of the request, while requests for those same hostnames sent to other IPs are not blocked. In particular, requests to google.com IPs containing host headers requesting blocked hostnames return the block page for those hostnames, while requests to yahoo.com IP addresses do not.

Instead of Deep Packet Inspection, MTNL appears to be using a combination of DNS-based and IP-based blocking approaches. Their DNS resolvers gives an incorrect answer (59.185.3.14) for the censored hostnames. It is possible to see the block page that MTNL users experience by browsing to http://59.185.3.14/ from anywhere in the world. Some MTNL customers were still able to connect to github's correct IP, while others were not.

Most of the reports collected using OONI are available here. These reports contain evidence of other sites being blocked in addition to the 32 websites specified in the December 17th order. The other sites are apparently being blocked using the same infrastructure, but we have not been able to determine under what authority their blocking has been ordered.

Other domains which appeared blocked on MTNL during testing included adult websites featuring Indian people although other adult websites listed in the alexa-top-1000 were not observed to be blocked. Censorship of advertisement, music sharing, and file hosting websites was also observed.

How can one circumvent this censorship?

In some cases, as ISPs are only blocking HTTP connections, while allowing access to sites over HTTPS, one could try to manually access the site using https:// instead of http:// in the URL. Regardless of whether the webpages you access are being censored or not, we recommend using the HTTPS Everywhere plugin in your web browser to automatically use the HTTPS version of many sites.

When the censorship is DNS-based, it can usually be circumvented by changing the DNS configuration on your device to use nameservers hosted outside of India. Two popular public DNS services are offered by OpenDNS and Google's public DNS.

We were also able to access the blocked websites using Tor Browser at all times.

Another option is to use a Virtual Private Network (VPN) hosted outside the country and a couple of services which offer this are The RiseUp Collective (Free) and iVPN (Paid). Mobile users using Android devices can also use Psiphon.

Additional resources

If you would like to understand more about censorship techniques or help collect more data, here are some useful resources that you might want to refer to:

• Open Observatory of Network Interference provides a set of open source tools that can be used to test and collect technical data about censorship and network tampering. We have made the reports generated from the data we collected using OONI here.

• Tor Browser is a free and open source software tool, which lets you securely circumvent censorship and surveillance and allows you to access resources on the Internet anonymously.

• For other resources about circumvention tools and tactics, and general digital security advice, please see Tactical Techology Collective's Security-in-a-box project.

• The Center for Internet and Society is an organisation based in Bangalore, India, which is actively working on policy for Internet governance, censorship and surveillance in India.

Current state of things

Following a new order issued on 31st December 2014, 4 of the 32 websites have subsequently been unblocked. The unblocked sites are github.com, vimeo.com, dailymotion.com and weebly.com

We will keep monitoring this censorship and publish any other relevant findings over the next few days. If you are a software developer or an IT professional who wants to help us collect more data from multiple ISPs in India, please contact us at censorship-in@chaoslab.in. Please use this PGP key if you would like to send us an encrypted email.

To read the larger dialogue on Twitter about this blocking of websites, please follow the hashtag #GOIblocks.