censorship

OONI report released: The State of Internet Censorship in Thailand

SEATTLE, WA, USA – Monday, March 20th, 2017 – The Tor Project announces the release of The State of Internet Censorship in Thailand, a report from a joint research study by Open Observatory of Network Interference (OONI), Sinar Project, and the Thai Netizen Network. The study aims to increase transparency of Internet controls in Thailand and to collect data that can potentially corroborate rumors and reports of Internet censorship events. The key finding of this report reveal that Internet Service Providers (ISPs) in Thailand appear to be blocking websites at their own discretion.

"We hope the findings of this report will enhance public debate around the necessity and proportionality of information controls," said Maria Xynou, Research and Partnerships Coordinator for OONI. Adding further that "A dozen websites, including The New York Post (nypost.com), were blocked in some networks, while accessible in others, indicates that Thai ISPs are likely blocking content at their own discretion."

Multiple censorship events in Thailand have been reported over the last decade. More than 10,000 URLs were reportedly blocked by the Government in 2010. Following Thailand’s most recent coup d’etat, Citizen Lab reported that 56 websites were blocked between May and June of 2014. One importance of undertaking this study, which collects and analyzes network measurements, is to examine whether Internet censorship events are persisting in the country.

Anyone can contribute to the research efforts by OONI by installing and running ooniprobe, thus increasing the transparency of Internet censorship in Southeast Asia and beyond.

About Open Observatory of Network Interference

The Open Observatory of Network Interference (OONI) is a free software project under The Tor Project that aims to empower decentralized efforts in increasing transparency of Internet censorship around the world. Since 2012, OONI has collected millions of network measurements from more than 190 countries, shedding light on multiple instances of network interference.

About Sinar Project

Sinar Project is an initiative using open technology and applications to systematically make important information public and more accessible to the Malaysian people. It aims to improve governance and encourage greater citizen involvement in the public affairs of the nation by making the Malaysian Government more open, transparent and accountable. We build open source civic tech applications, work to open government with open data and defend digital rights for citizens to apply their democratic rights.

About Thai Netizen Network

Thai Netizen Network (TNN), founded in 2008, is a leading nonprofit organization in Thailand that advocates for digital rights and civil liberties. The group was officially registered as มูลนิธิเพื่ออินเทอร์เน็ตและวัฒนธรรมพลเมือง (Foundation for Internet and Civic Culture) in May 2014.

About Tor Project, Inc

The Tor Project develops and distributes free software and has built an open and free network that helps people defend against online surveillance that threatens personal freedom and privacy. Tor is used by human rights defenders, diplomats, government officials, and millions of ordinary people who value freedom from surveillance.

The Tor Project's Mission Statement: "To advance human rights and freedoms by creating and deploying free and open anonymity and privacy technologies, supporting their unrestricted availability and use, and furthering their scientific and popular understanding."

Media Contacts

Joshua Gay
Communications Director
Tor Project
jgay@torproject.org

Maria Xynou (OONI)
maria@openobservatory.org

Arturo Filasto (OONI)
arturo@openobservatory.org

Tor at the Heart: OONI Highlights from 2016

During the month of December, we're highlighting other organizations and projects that rely on Tor, build on Tor, or are accomplishing their missions better because Tor exists. Check out our blog each day to learn about our fellow travelers. And please support the Tor Project! We're at the heart of Internet freedom.
Donate today!

In this post we provide some highlights from OONI, a project under The Tor Project.

The Open Observatory of Network Interference (OONI) is a free software project under The Tor Project that aims to uncover internet censorship around the world. Recently we published an overview of OONI which can be found here.

Today we are providing some OONI highlights from 2016. These include our research findings in collaboration with our partners, and the new features we have developed and released to meet our users’ needs.

Research findings

As part of the OONI Partnership Program we collaborate with various local and international non-profit organizations around the world on the study of internet censorship. Below we provide some highlights from our research findings this year.

Censorship during elections

  • Uganda: Facebook and Twitter blocked during 2016 general elections. In collaboration with DefendDefenders we examined the blocking of social media in Uganda during its 2016 general elections and when the country’s President was inaugurated. View our findings here.
  • Zambia: Internet censorship events during 2016 general elections. OONI monitored internet censorship events during Zambia’s 2016 general election period in collaboration with Strathmore University’s Centre for Intellectual Property and Information Technology Law (CIPIT). A full report of our study can be found here.
  • The Gambia: Internet shutdown during 2016 presidential election. We attempted to examine whether websites were blocked during the Gambia’s 2016 presidential election. Instead, we came across a country-wide internet blackout. View our findings here.
  • Venezuela: Blocking of sites during elections. IPYS conducted a study of internet censorship in Venezuela through the use of ooniprobe. Their full report can be found here.

Censorship during other political events

  • Ethiopia: Deep Packet Inspection (DPI) technology used to block media websites during major political protests. OONI joined forces with Amnesty International to examine internet censorship events during Ethiopia’s wave of protests. We not only detected DPI filtering technology, but we also found numerous sites - including news outlets, torproject.org, LGBTI and human rights sites - to be tampered with. Now Ethiopia is in a state of emergency. Our report can be found here.
  • Turkey: Internet access disruptions during attempted military coup. In collaboration with RIPE Atlas we examined the throttling of social media in Turkey during the attempted military coup in July. View the findings here.
  • Ethiopia: Internet shutdown amidst political protests. Ethiopia’s government pulled the plug on the internet in the middle of heavy protests in August. We examined the internet shutdown in collaboration with Strathmore University’s Centre for Intellectual Property and Information Technology Law (CIPIT) and published our findings here.

Tor blocking

  • Egypt: Tor interference. Our community informed us that certain services were inaccessible in Egypt. We investigated the issue and also found Tor to be tampered with. View our findings here.
  • Belarus: Tor block. An anonymous cypherpunk helped us collect evidence of Tor blocking in Belarus. View the data here.

WhatsApp blocking and DNS censorship

  • Brazil: Blocking of WhatsApp. Thanks to Coding Rights who ran our newly developed WhatsApp test, we were able to detect and collect evidence of the blocking of WhatsApp in Brazil earlier this year. View the data here.
  • Malaysia: DNS blocking of news outlets, medium.com, and sites expressing political criticism. Following the 1MDB scandal, various news outlets were reportedly blocked in Malaysia. OONI joined forces with Sinar Project to examine and collect evidence of internet censorship events in Malaysia. Our report can be found here.

New releases

If you’ve known OONI for a while, you might be more familiar with ooniprobe as a command line tool. To meet our users’ needs, we developed a variety of features this year, including the following:

  • OONI Explorer: A global map to explore and interact with all of the network measurements that OONI has collected from 2012 to date.
  • Measurement API: Explore and analyze OONI’s data via its new API.
  • OONI web UI: Run censorship tests from your web browser!
  • WhatsApp & Facebook Messenger tests: Examine the reachability of WhatsApp and Facebook Messenger with OONI’s new tests!
  • Web Connectivity test: Examine DNS, TCP/IP, HTTP blocking of sites all in one test!
  • Lepidopter: Run ooniprobe from a Raspberry Pi!
  • OONI mobile: We have developed the beta version of ooniprobe for Android and iOS. Look out for ooniprobe’s mobile app in early 2017!

Over the last year, many non-profit organizations around the world have started running ooniprobe daily. The graph below illustrates the expansion of ooniprobe’s global coverage thanks to our users.


By supporting Tor, you’re also supporting the OONI project. Help us continue to increase transparency around internet censorship by donating to The Tor Project.

Written by Maria Xynou, OONI’s Research and Partnerships Coordinator.

Tor at the Heart: Bridges and Pluggable Transports

During the month of December, we're highlighting other organizations and projects that rely on Tor, build on Tor, or are accomplishing their missions better because Tor exists. Check out our blog each day to learn about our fellow travelers. And please support the Tor Project! We're at the heart of Internet freedom.
Donate today

Technology against censorship: bridges and pluggable transports

You can use Tor to view websites that are censored or blocked. But what do you do when Tor itself is blocked? When it happens, you can use bridges and pluggable transports to get around the censors. Here is how to do it in Tor Browser:

Animated graphic showing 6 steps to configuring pluggable transports.

How does it work?

Censors block Tor in two ways: they can block connections to the IP addresses of known Tor relays, and they can analyze network traffic to find use of the Tor protocol. Bridges are secret Tor relays—they don't appear in any public list, so the censor doesn't know which addresses to block. Pluggable transports disguise the Tor protocol by making it look like something else—for example like HTTP or completely random.

There are several pluggable transports, and it can be hard to know which one to use. If it is your first time, try obfs4: it is a randomizing transport that works for most people. If obfs4 doesn't work, try fte. If that doesn't work, it may mean that the default bridges are blocked, and you should get a custom bridge from bridges.torproject.org. If the custom bridge doesn't work, try meek-azure or meek-amazon.

  • obfs4 is a randomizing transport: it adds an extra layer of specialized encryption between you and your bridge that makes Tor traffic look like random bytes. It also resists active-probing attacks, where the censor discovers bridges by trying to connect to them. obfs3 and scramblesuit are similar in nature to obfs4.
  • fte makes Tor traffic resemble plain HTTP. The name stands for "Format-Transforming Encryption."
  • meek makes Tor traffic look like a connection to an HTTPS website. Unlike the other transports, it doesn't connect directly to a bridge. meek first connects to a real HTTPS web server (in the Amazon cloud or the Microsoft Azure cloud) and from there connects to the actual bridge. Censors cannot easily block meek connections because the HTTPS servers also provide many other useful services.

There are a number of built-in, default bridges, which you can use just by choosing a pluggable transport name. For better secrecy, you should get custom bridges from bridges.torproject.org. meek doesn't need custom bridges; however it is slower and more expensive to operate than the other pluggable transports, so you should use obfs4 or fte if they work for you.

Tor is not the only project to use pluggable transports. We work often with researchers and developers to study Internet censorship, improve pluggable transports, and develop new ones. Psiphon and Lantern are two other projects that use pluggable transports. (Unlike Tor, they focus only on access and not on anonymity.)

If you are not censored yourself, you can help censored people by running a bridge with a pluggable transport. Running a bridge is the same as running a relay, just with a little extra configuration. See this guide: Become a PT bridge operator! Once your bridge is running, it will automatically become available to users at bridges.torproject.org.

The world of censorship is changing all the time. It's a good idea to learn how to use bridges and pluggable transports before you actually need them. Just last week, ISPs in Belarus began blocking public Tor relays—but bridges and pluggable transports are so far working to defeat the blocks. We are tracking other censorship events, such as those in Saudi Arabia, Kazakhstan, and elsewhere. If you know details of these or any other Tor blocks, please tell us. The best way to do that is to leave a comment on our bug tracker. (You can create an account first.)

Tor at the Heart: The OONI project

During the month of December, we're highlighting other organizations and projects that rely on Tor, build on Tor, or are accomplishing their missions better because Tor exists. Check out our blog each day to learn about our fellow travelers. And please support the Tor Project! We're at the heart of Internet freedom.
Donate today

In this post we provide an overview of OONI, a project under The Tor Project.

The Open Observatory of Network Interference (OONI) is a free software project under The Tor Project that aims to increase transparency about internet censorship around the world. To this end, OONI has developed multiple free software tests (called ooniprobe) that are designed to examine the following:

  • Blocking of websites;
  • Blocking of Instant Messaging software such as WhatsApp and Facebook Messenger;
  • Blocking of Tor, proxies, VPNs, and sensitive domains;
  • Detection of systems responsible for censorship, surveillance and traffic manipulation.

Anyone can run these tests to examine whether censorship is being implemented in their network. All data collected through ooniprobe is published and can serve as a resource for those who are interested in knowing how, when, and by whom internet censorship is being implemented. You can find OONI’s data in JSON format or via OONI Explorer: a global map for exploring and interacting with all the network measurement data that OONI has collected from 2012 to date.

Hundreds of volunteers have run ooniprobe across more than 100 countries around the world, shedding light on multiple instances of internet censorship. WhatsApp, for example, was found to be blocked in Brazil earlier this year, while Facebook and Twitter were censored during Uganda’s 2016 general elections. OONI data also shows that news websites were blocked in Iran and India, amongst many other countries, and that sites supporting LGBTI dating also appeared to be tampered with in Zambia.

OONI aims to equip the public around the world with data that can serve as evidence of internet censorship events. Such data not only shows whether a site or service was blocked, but more importantly, how it was blocked, when, where, and by whom. This type of information can be particularly useful to the following:

  • Lawyers: Examine the legality of the type of internet censorship implemented in your country, and use OONI’s data as evidence.
  • Journalists: Improve the credibility of your stories by referencing network measurement data as evidence of censorship events.
  • Researchers: Use OONI’s data to explore new questions. Researchers from the University of Cambridge and UC Berkeley, for example, were able to examine the differential treatment of anonymous users through the use of OONI data.
  • Activists, advocates, campaigners: Inform your work based on evidence of censorship events.
  • Circumvention tool projects: Inform the development of your tools and strategies based on OONI’s findings on censorship events around the world.

To empower participation in censorship research, OONI has established partnerships with local non-profit organizations around the world. Some of these organizations include:

These partnerships involve the daily collection of network measurements from local vantage points, determining which sites and services to test per country, and analyzing measurements within social, political, and legal context. Some partners, such as Sinar Project, even organize regional workshops to teach other groups and organizations how to measure internet censorship through the use of ooniprobe.

The Tor Project has supported the OONI project from day 1. Donate to The Tor Project today and help us continue to uncover internet censorship around the world.

Written by Maria Xynou, OONI’s Research and Partnerships Coordinator

Breaking through censorship barriers, even when Tor is blocked

Download video | view on YouTube


While Tor Browser provides many security and privacy properties and features, not everyone around the world has the luxury to connect to use it. By default, Tor Browser makes all of its users look alike by spoofing UserAgent (and other methods) to avoid fingerprinting attacks. However, it doesn't hide the fact you're connecting to Tor, an open network where anyone can get the list of relays. This network transparency has many benefits, but also has a downside: Many repressive governments and authorities benefit from blocking their users from having free and open access to the internet. They can simply get the list of Tor relays and block them. This bars millions of people from access to free information, often including those who need it most. We at Tor care about freedom of access to information and strongly oppose censorship. This is why we've developed methods to connect to the network and bypass censorship. These methods are called Pluggable Transports (PTs).

Pluggable Transports are a type of bridge to the Tor network. They take advantage of various transports and make encrypted traffic to Tor look like not-interesting or garbage traffic. Unlike normal relays, bridge information is kept secret and distributed between users via BridgeDB. If you're interested in helping censored users, you can become a bridge operator. And if you're a developer and have interesting ideas on how to make new PTs or want to contribute code, we've some good documents to get you up to speed.

And finally, if you're a censored user and want to take advantage of PTs, I've good news for you. They're already included in Tor Browser and this how-to graphic should help you configure it to bypass censorship.


How to use PTs: 1-download tor-send email to gettor@torproject.org; 2 select configure 3; check my isp blocks tor option; 4 select obfs4; 5 press connect
(download png)


And of course we didn't forget to make a gif version:


How to use PTs: 1-download tor-send email to gettor@torproject.org; 2 select configure 3; check my isp blocks tor option; 4 select obfs4; 5 press connect
(download gif)



In case you need more bridges, send an email to bridges@torproject.org or visit BridgeDB website.

At the end, I'd like to thank all anonymous contributors and Vivido Studio for making this work possible.

In solidarity,
Nima Fatemi

GetTor: New Ways to Download Tor Browser

We are pleased to announce the new features available in the GetTor, a service that provides alternative ways to download Tor Browser, aimed for people who live in places with high levels of censorship (e.g. when www.torproject.org is blocked) or people who just don't want to expose the fact that they are downloading Tor Browser. This work adds important new download options and capabilities and includes improvements to the current code, deployment of new channels and providers, and some brand new features such as the GetTor API. We would also like to give special thanks to Nima Fatemi, who was in charge of the non-coding parts of this project (from funding to technical management).


Update note: we now have the gettor@torproject.org account for the XMPP channel. However, we will have the get_tor@riseup.net account enabled for a couple of more weeks just in case you are still using it.


Landing page

A GetTor landing page has been created to offer information in one place (statistics, guides, etc.). If you are interested in what is going on with GetTor, following the landing page is highly recommended.


New Distribution Channels

In the past, GetTor has distributed packages by sending the bundles -- and then, later, just links -- via email. Now there are two more ways to interact with GetTor:


  1. Using Twitter: You can send a direct message to @get_tor account (you don't need to follow the @get_tor acount). Send the word help in a direct message to receive information on how to download the Tor Browser.

  2. Using XMPP: You can send a message to gettor@torproject.org using your favorite XMPP client. Simply enter help in an XMPP message to receive information on how to download the Tor Browser.


GitHub

GitHub is now a provider of Tor Browser (in addition to Dropbox and Google Drive), and the latest version of Tor Browser may be downloaded from our Github page and our Github repository.


Support for Android

Orbot is a free proxy (i.e. an intermediary) app that empowers other apps to use the Internet more securely. Orbot uses Tor to encrypt your Internet traffic and then hides it by sending it through a series of computers around the world. In addition to the download options provided by Guardian Project (Google Play, F-Droid, Direct download), GetTor provides yet another way to download Orbot to your mobile device. To do this, you have to reach one of our distribution channels and specify the android command (See Examples, at the bottom of this blog post). You will then receive instructions to download Orbot's Android Application Package (APK) file from Github, Google Drive or Dropbox. Once you have downloaded the APK file you can use it to install Orbot (similar to .exe files in Windows) and start using it.


Translated Versions of Tor Browser

GetTor provides a small set of translated packages focused on its end users. The available languages are Farsi, Chinese, Turkish, and English (which is the default). If you want to use this feature in the email autoresponder, for example, you send your request to:


    Farsi: gettor+fa@torproject.org
    Chinese: gettor+zh@torproject.org
    Turkish: gettor+tr@torproject.org
    English: gettor@torproject.org


For the Twitter and XMPP channels, you just need to add the language word to the
message (e.g. linux fa will get you links for Tor Browser in Farsi).


Mirrors

There are many volunteers who use their own servers to provide mirrors of Tor Project's website. One or more of these mirrors may be not blocked in places where torproject.org is censored and could help in downloading Tor Browser. With this new release, you can request a list of these mirrors from GetTor by sending an email (or message, in case of Twitter and XMPP) with the word mirrors in the body of the text.


Statistics

Some basic but effective improvements have been made to collect anonymous data and compile meaningful statistics about GetTor usage, including requests per channel, operating system, and language. Safeguards have been implemented so that all information collected is anonymous, and it is erased on a daily basis -- we just keep the number and types of requests. Reports about this data will soon be available on GetTor's website.


RESTful API

One of GetTor's major new features is its API. In simple terms, an API is a set of rules and specifications that allow applications to communicate with each other (following these rules). This is helpful to developers who want to create new services or applications based on the information provided by the API. In this case, the GetTor API provides the following information:

  1. Links to download Tor Browser by provider, with filters for operating system and language.

  2. Links to download Tor Browser from Tor Project's website, with filters for choosing the release (latest version , etc.), operating system, and language.

  3. List of mirrors of Tor Project's website.



You can find more information on the API documentation.


Invitation to Collaborate

If you are a Tor user, a developer, good at writing content for non-technical users or anything else, we are happy to hear from you! You can use the comments section below, the tor-talk and tor-dev mailing lists, or come talk to us on IRC (#tor-dev on OFTC; our nicknames are ilv, sukhe and mrphs).


How to Ask for Tor Browser--Some Examples

To help you get started, here are a few examples of GetTor requests with different locales (languages) and operating systems:


Example 1 (Email): To get links for downloading Tor Browser in Farsi for Windows, send an email to gettor+fa@torproject.org with the word windows in the body of the message.


Example 2 (Twitter): To get links for downloading Tor Browser in English for OS X, send a Direct Message to @get_tor with the words osx on it (you don't need to follow the account).


Example 3 (XMPP): To get links for downloading Tor Browser in Chinese for Linux, send a message to gettor@torproject.org account with the words linux zh on it.


Example 4 (Email): To get links for downloading Orbot for Android, send an email to gettor@torproject.org with the word android in the body of the message.

OONI Explorer: Censorship and other Network Anomalies Around the World

Today the Open Observatory of Network Interference (OONI) team is pleased to announce the public beta release of OONI Explorer: a global map of more than 8.5 million network measurements which have been collected across 91 countries around the world over the last 3 years.

OONI is based on 15 free software tests which are designed to measure the following:

  • Blocking of websites
  • Detection of systems responsible for censorship, surveillance and manipulation
  • Reachability of Tor, proxies, VPNs, and sensitive domains

These tests have been run across 398 different vantage points by volunteers around the world since 2012. The OONI Explorer announced today provides a location to interact and - dare we say - explore all of the collected measurements.

Key Findings

Some of the highlights in the data:

1. Confirmed cases of censorship in 9 countries

Multiple HTTP request tests were run around the world and based on our heuristics, we were able to detect block pages in 9 countries: Iran, Saudi Arabia, Turkey, Greece, China, Russia, India, Indonesia and Sudan.

Blocked websites include media, gambling and over-the-counter money exchanges. In Greece, for example, all of the tested ISPs employed DNS hijacking to block such websites, with the exception of Vodafone that also used Deep Packet Inspection. OONI tests in Turkey illustrate that 62 websites were blocked, including piratebay.com, livescore.com and 4shared.com, possibly under Law No. 5651 on the ‘Regulation of Publications on the Internet and Suppression of Crimes Committed by means of Such Publication’. Notably, 362 blocked websites were detected as blocked in Iran and 50 in Saudi Arabia, including arabtimes.com, mossad.gov.il and anonym.to, a URL shortening service with privacy properties.

Some of our tests for domains were focused on specific websites which were rumored or reported to be blocked. In January 2015, for example, the Government of India ordered the blocking of 32 websites under Section 69A of the Information Technology Act, 2000, and under the Information Technology (Procedures and Safeguards for Blocking of Access of Information by Public) Rules, 2009. Following these reports, OONI tests run on those websites were able to confirm that 23 of those websites were in fact blocked in the network that was tested, including websites such as pastebin.com, dailymotion.com and archive.org.

Leading up to the 2016 general elections in Uganda, OONI volunteers ran HTTP request tests in response to reports that Facebook and Twitter were being blocked. We did not detect block pages, but we did detect general network anomalies which indicate that it's likely the case that Ugandan ISPs were blocking some requests, but not others. It is also possible that Facebook and Twitter were only blocked in specific networks, and not countrywide.

2. Network anomalies in 71 countries

Out of the 91 countries with reported data, network anomalies were detected in 71 of them.

“Network anomalies” and “network interferences” are broad terms that we use to describe symptoms of censorship through the manipulation of internet traffic. These anomalies can take many forms, including connectivity failures, timeouts and unusual slowness, or unexpected error messages.

Not all HTTP request tests allow us to conclusively know that interference has occurred, because not all interference looks like a clear block page. Sometimes, censorship is hidden as connection failures instead. To gain confidence in detecting this type of interference, we can look at repeated failures to websites that are known to be operating normally. In Cuba, for example, it is interesting to see that while no block pages were detected, HTTP requests to cubafreepress.org failed multiple times.

Symptoms of traffic manipulation were detected in multiple countries around the world through HTTP invalid request line and HTTP header field manipulation tests, which look for middle boxes: network equipment that intercept and sometimes alter the traffic passing through them. Multiple HTTP invalid request line tests run in Vietnam from 2013 to 2015 triggered errors and indicate that middle boxes were regularly observing the traffic in the country. Similarly, many HTTP invalid request line tests in Pakistan and elsewhere indicate the presence of software which is capable of traffic manipulation.

3. Blue Coat, Squid and Privoxy detected in 11 countries

Transparent HTTP proxies can be used inside of small and large networks for various purposes: to intercept the web traffic of users, to implement caching or to speed up requests for commonly visited websites.

Through OONI tests we detected 3 different types of proxy technology: Blue Coat, Squid and Privoxy. Blue Coat Systems is a US security and networking solutions provider which has been called out for selling network appliances capable of filtering, censorship, and surveillance to governments with poor human rights records. Its presence, along with Squid and Privoxy, has been reported in the networks of 11 countries: USA, Canada, Portugal, Spain, Italy, the Netherlands, Switzerland, Moldova, Iraq, Myanmar and Uganda. It remains unclear though whether such middle boxes were actually used for online censorship, surveillance and traffic manipulation, or if they were merely used for caching purposes.

Furthermore, not all the detected instances of proxy technologies are necessarily deployed country-wide or even on an ISP level, but in some cases they might simply be running inside of the local network of the OONI user. It is interesting to note that the use of Blue Coat was first detected in Myanmar in 2012, but when another measurement was run from the same network in 2014 it was no longer detectable in the same way. This can either mean that it was removed or that it is no longer detectable.

Contribute to OONI Explorer

OONI Explorer was made possible by the growing community of volunteers around the world who have contributed to the project. You can contribute too by:

Happy OONI exploring!

Learning more about the GFW's active probing system

This blog post is also available in Chinese, translated by our friends from GreatFire.org.

Roya, David, Nick, nweaver, Vern, and I just finished a research project in which we revisited the Great Firewall of China's (GFW) active probing system. This system was brought to life several years ago to reactively probe and block circumvention proxies, including Tor. You might remember an earlier blog post that gave us some first insight into how the active probing system works. Several questions, however, remained. For example, we were left wondering what the system's physical infrastructure looked like. Is the GFW using dedicated machines behind their thousands of probing IP addresses? Does the GFW even "own" all these IP addresses? Rumour had it that the GFW was hijacking IP addresses for a short period of time, but there was no conclusive proof. As a result, we teamed up and set out to answer these, and other questions.

Because this was a network measurement project, we started by compiling datasets. We created three datasets, comprising hours (a Sybil-like experiment to attract many probes), months (an experiment to measure reachability for clients in China), and even years (log files of a long-established server) worth of active probing data. Together, these datasets allow us to look at the GFW's active probing system from different angles, illuminating aspects we wouldn't be able to observe with just a single dataset. We are able to share two of our datasets, so you are very welcome to reproduce our work, or do your own analysis.

We now want to give you an overview of our most interesting findings.

  • Generally, once a bridge is detected and blocked by the GFW, it remains blocked. But does this mean that the bridge is entirely unreachable? We measured the blocking effectiveness by continuously making a set of virtual private systems in China connect to a set of bridges under our control. We found that every 25 hours, for a short period of time, our Tor clients in China were able to connect to our bridges. This is illustrated in the diagram shown below. Every point represents one connection attempt, meaning that our client in China was trying to connect to our bridge outside of China. Note the curious periodic availability pattern for both Unicom and CERNET (the two ISPs in China we measured from). Sometimes, network security equipment goes into "fail open" mode while it updates its rule set, but it is not clear if this is happening here.

  • We were able to find patterns in the TCP headers of active probes that suggest that all these thousands of IP addresses are, in fact, controlled by a single source. Check out the initial sequence number (ISN) pattern in the diagram below. It shows the value of ISNs (y-axis) over time (x-axis). Every point in the graph represents the SYN segment of one active probing connection. If all probing connections would have come from independent computers, we would have expected a random distribution of points. That's because ISNs are typically chosen randomly to protect against off-path attackers. Instead, we see a clear linear pattern across IP addresses. We believe that active probes derive their ISN from the current time.

  • We discovered that Tor is not the only victim of active probing attacks; the GFW is targeting other circumvention systems, namely SoftEther and GoAgent. This highlights the modular nature of the active probing system. It appears to be easy for GFW engineers to add new probing modules to react to emerging, proxy-based circumvention tools.
  • The GFW is able to (partially) speak the vanilla Tor protocol, obfs2, and obfs3 to probe bridges. Interestingly, node-Tor—a JavaScript implementation of the Tor protocol—is immune to active probing because it implements the Tor protocol differently, which seems to confuse active probes. We were also able to resist active probes by modifying a bridge of ours to ignore old VERSIONS Tor cells. This is unlikely to be a sustainable circumvention technique, though.
  • Back in 2012, the system worked in 15-minute-queues. These days, it seems to be able to scan bridges in real-time. On average, it takes only half a second after a bridge connection for an active probe to show up.
  • Using a number of traceroute experiments, we could show that the GFW's sensor is stateful and seems unable to reassemble TCP streams.

Luckily, we now have several pluggable transports that can defend against active probing. ScrambleSuit and its successor, obfs4, defend against probing attacks by relying on a shared secret that is distributed out of band. Meek tunnels traffic over cloud infrastructure, which does not prevent active probing, but greatly increases collateral damage when blocked. While we keep developing and maintaining circumvention tools, we need to focus more on usability. A powerful and carefully-engineered circumvention tool is of little use if folks find it too hard to use. That's why projects like the UX sprint are so important.

Finally, you can find our research paper as well as our datasets and code on our project page. And don't hesitate to get in touch with us if you have any questions or feedback!

Say hi to the new GetTor

Hello people. It's been a while since Google Summer of Code 2014 ended, but I wanted to give you a brief review of the work done on GetTor.


What is GetTor?

GetTor is a program that serves Tor Browser over email. In the past, people would make requests by sending emails to GetTor, which would send back Tor Browser as email attachments. In highly censored countries (and places) where the Tor Project website is blocked, GetTor would be a convenient way for people to get access to Tor Browser.

There were lots of nice features incorporated in GetTor, such as specifying the operating system and language for the package wanted, or sending delay messages to let people know the package was on its way. But Tor Browser started to get larger in size (over 25 MB), to the point where it wasn't longer possible to send it via most email providers.


Revamp

It wasn't long until a solution for this problem came up. The idea consisted on uploading Tor Browser to the cloud (Dropbox) and when someone asked for it via GetTor, a reply with the links for download was sent. This worked quite well, but the fix was far from being complete and at that point the whole GetTor was in need of some love to get back to its shiny days.


Google Summer of Code

All of what I mentioned was listed on the Volunteer page of the Tor Project website, so when I got there looking for a project to work on for the Google Summer of Code, I immediatly considered it into my options, because of the social impact of GetTor as for the technical skills required. I was happy to learn that my proposal got accepted and I was one of the fourteen students selected to work on the Tor Project during the northern hemisphere summer (actually, it was winter here in Chile).

First, I started to work on the design, making sure that when I started to code, most of the ideas I would be implementing were carefully described and discussed. Of course, a lot of things did change over the coding period, some of them small stuff like how the links would be internally stored by GetTor, and some of them not so small, like changing one of the distribution modules.

Anyhow, I don't want to bore you with technical details here, but if you're interested, please read my biweekly reports and check the code repository.


Outcome

The coding period lasted a little more than three months, and I managed to pass both mid-term and final evaluations. But more importantly, the status of GetTor improved significantly during that time. I did a full rewrite of it, focusing on having clean and readable code, and on making it easy to add new distribution modules and cloud providers for storing Tor Browser. Two distribution modules were successfully finished: SMTP, for asking via email; and XMPP, for asking via Jabber (you know, chat style).

Even though the new GetTor is able to manage requests in multiple locales, for now the SMTP module has been deployed with support for English requests only; other locales and modules will eventually/gradually be supported. We will let you know when that happens (soon we hope!).

Almost all of the testing and other minor fixes were done after the Google Summer of Code ended, and this is because I explicitly mentioned to my mentors that I have the intention to keep working on it and to continue as the lead developer if needed. It's not just for the work I did, but more importantly for the possibility of helping other people, specially those that have the bad fortune to live under regimes and/or organizations which think they can impose control on the information you can access, spy on what you do and chase you for what you think. If I have the chance to help avoiding this dystopia, as little as I can, I would certainly do whatever is in my hands, and I invite you to do the same.


Great, but how do I use it?

You can reach GetTor by sending emails to gettor@torproject.org. To ask for Tor Browser, you just have to send an email with the word windows in the body to get it for Windows, osx to get it for Mac OSX, or linux to get it for Linux. The options are case insentitive, so it doesn't matter if you send Linux, or linux, or LiNuX, as long as it describes one of the options mentioned before; if you send anything different from that, you will receive a help message with detailed instructions on how to interact with it. Once you ask for Tor Browser, GetTor will reply to you with Dropbox links to download the required package for your architecture (32/64 bit) and operating system, along with some extra information to help you verify the integrity of the downloaded files. Please note that you can reach GetTor from any email address: gmail, yahoo, hotmail, riseup, etc. The only restriction is that you can do a maximum of three requests in a row, after that you'll have to wait 20 minutes to reach GetTor again. You can find out more about its purpose and how it works here.


Collaborate

The main way to collaborate is to use GetTor and provide feedback! Please tell us what you like, what you don't like, what works smoothly and what doesn't work or could work better; after all, GetTor is here for you, so you should tell us what we need to do :) For this, please open a ticket on the trac system under the GetTor component. You can file anything from usability suggestions/bugs to new development ideas.

On the other hand, I've read lots of people who are interested to collaborate with the Tor Project and they just don't know where to start or they are looking for something easy to collaborate with. The code and work on GetTor is quite straightforward, so if you know some Python and have some free time that you feel you want to give to an awesome open source organization, check the git repository and the tickets and you might find something easy to start with. There are various ideas and things left to do in GetTor, so please join us!


Other options

It's important to note that there are a couple more options to obtain Tor Browser when you cannot access Tor Project's website. The first and easiest is to access the official mirrors: EFF and torservers.net. If those sites are blocked too, you can try using Satori, an app for Google Chrome that distributes various circumvention tools in a difficult-to-block way, making it easy for users to check if the software has been tampered. If after all, you manage to get the Tor Browser but you are not able to reach the Tor network, you might want to use bridges or the pluggable transports. You can read more about that here, here and here.



Thanks

I want to end this blog post by thanking to the Tor Project organization in general for letting me be part of it during the summer and kindly answer any doubt that came up, and to Sukhbir and Nima in particular for their awesome job as mentors, I couldn't have done it without you, thanks a lot guys!

OONI Bridge reachability study and hackfest

Has a Tor bridge already been blocked in a given country? Being able to answer that question would allow Tor to provide more efficient circumvention methods to those who need them. OONI, the Open Observatory of Network Interference is now actively collecting data on bridge reachability. We are also interested in having a better understanding of how reactive censors are in blocking new bridges distributed via Tor Browser and how effective they are at inhibiting usage of particular pluggable transport.

The countries we are focusing on in this survey are China, Iran, Russia and Ukraine. We call these our test vantage points.

From every test vantage point we perform two types of measurements:

To establish a baseline to eliminate the cases in which the bridge is marked as blocked, while it is in fact just offline, we measure also from a vantage point located in the Netherlands.

So far we have collected about a month worth of data and it is as always publicly available for download by anybody interested in looking at it.

To advance this study at the end of October we did a OONI hackfest in Berlin. Helped by the ubiquitous sticky notes we were able to come up with a plan for those days of work and for continuing the project.

The first visualisation we produced is that of the reachability of bridges categorised by country and pluggable transport over time. This simple visualisation already conveys a lot of information and has proven itself a useful tool also in debugging issues with ooniprobe and the tools we use.



You can visit the actual page by clicking on the picture above.
Please note that because the tests are new and experimental you might find inaccuracies or bugs, so don't seriously rely on it for research just yet.

We also developed a data pipeline that places all of the collected OONI reports into a database. This makes it much easier to search/aggregate and visualise the data of the reports.

To read more about this project check out the ooni-dev mailing list thread on this topic.

This project is still in it's very early stages of development, but we would love to hear feedback on it or your cool visualization ideas, as well as any questions regarding Tor bridge reachability (or more in general on Internet censorship) that you would like us to answer!

Syndicate content Syndicate content