Over the last years, we learned a lot about how the Great Firewall of China is blocking Tor. Some questions remained unanswered, however. Roya, Mueen, Jed, and I just published a project which seeks to answer some of these open questions. Being curious as we are, we tried to find answers to the following questions:
- Is the filtering decentralised (i.e., happening in provinces) or centralised (i.e., happening in Internet exchange points (IXP))?
- Are there any temporal patterns in the filtering? Or in other words, are there certain times when people are more likely to be able to connect to Tor?
- Similarly, are there any spatial patterns? Are folks in some special regions of China able to connect to Tor while others cannot?
- When a computer in China tries to connect to a Tor relay, what part of the TCP handshake is blocked?
It turns out that some of these questions are quite tricky to answer. For example, to find spatial patterns, we need to be able to measure the connectivity between many Tor relays and many clients in China. However, we are not able to control even a single one of these machines. So how do we proceed from here? As so often, side channels come to the rescue! In particular, we made use of two neat network measurement side channels which are the hybrid idle scan and the SYN backlog scan. The backlog scan is a new side channel we discovered and discuss in our paper. Equipped with these two powerful techniques, we were able to infer if there is packet loss between relay A and client B even though we cannot control A and B.
You might notice that our measurement techniques are quite different from most other Internet censorship studies which rely on machines inside the censoring country. While our techniques give us a lot more geographical coverage, they come at a price which is flexibility; we are limited to measuring Internet filtering on the IP layer. More sophisticated filtering techniques such as deep packet inspection remain outside our scope.
Now what we did was to measure the connectivity between several dozen Tor relays and computers in China over four weeks which means that we collected plenty of data points, each of which telling us "was A able to talk to B at time T?". These data points reveal a number of interesting things:
- It appears that many IP addresses inside the China Education and Research Network (CERNET) are able to connect to at least our Tor relay.
- Apart from the CERNET netblock, the filtering seems to be quite effective despite occasional country-wide downtimes.
- It seems like the filtering is centralised at the IXP level instead of being decentralised at the provincial level. That makes sense from the censor's point of view because it is cheap, effective, and easy to control.
Now what does all of this mean for Tor users? Our results show that China still has a tight grip on its communication infrastructure, especially on the IP and TCP layer. That is why our circumvention efforts mostly focus on the application layer (with meek being an exception) and pluggable transport protocols such as ScrambleSuit (which is now part of the experimental version of TorBrowser) and obfs4 are specifically designed to thwart the firewall's active probing attacks.
The recently released 4.0-alpha-1 version of Tor Browser includes meek, a new pluggable transport for censorship circumvention. meek tunnels your Tor traffic through HTTPS, and uses a technique called “domain fronting” to hide the fact that you are communicating with a Tor bridge—to the censor it looks like you are talking to some other web site. For more details, see the overview and the Child’s Garden of Pluggable Transports.
You only need meek if your Internet connection is censored so that you can’t use ordinary Tor. Even then, you should try other pluggable transports first, because they have less overhead. My recommended order for trying transports is:
Use meek if other transports don’t work for you, or if you want to help development by testing it. I have been using meek for my day-to-day browsing for a few months now.
At this point, there are two different backends supported. meek-amazon makes it look like you are talking to an Amazon Web Services server (when you are actually talking to a Tor bridge), and meek-google makes it look like you are talking to the Google search page (when you are actually talking to a Tor bridge). It is likely that both will work for you. If one of them doesn’t work, try the other.
These instructions and screenshots are for the 4.0-alpha-1 release. If they change in future releases, they will be updated at https://trac.torproject.org/projects/tor/wiki/doc/meek#Quickstart.
How to use meek
First, download a meek-capable version of Tor Browser for your platform and language.
On the first screen, where it says Which of the following best describes your situation?, click the Configure button.
On the screen that says Does this computer need to use a proxy to access the Internet?, say No unless you know you need to use a proxy. meek supports using an upstream proxy, but most users don’t need it.
On the screen that says Does this computer's Internet connection go through a firewall that only allows connections to certain ports?, say No. As an HTTPS transport, meek only uses web ports, which are allowed by most firewalls.
On the screen that says Does your Internet Service Provider (ISP) block or otherwise censor connections to the Tor Network?, say Yes. Saying Yes will lead you to the screen for configuring pluggable transports.
On the pluggable transport screen, select Connect with provided bridges and choose either meek-amazon or meek-google from the list. Probably both of them will work for you, so choose whichever feels faster. If one of them doesn’t work, try the other. Then click the Connect button.
A few days ago, we published a blog post exposing the use of Deep Packet Inspection (DPI) to filter all Internet traffic in Ethiopia, including connections to the Tor network. We concluded that they are doing some sort of TLS fingerprinting, but had not been able to figure out exactly what they are fingerprinting on. Since then, we have managed to determine exactly how Ethiopia blocks Tor and we have developed a workaround. We will publish a full technical analysis very soon.
The long-term solution for Tor users in Ethiopia is to use the Obfsproxy Tor Browser Bundle. The bundles are, unfortunately, not up to date at the moment, but this is something we are working on (see #5937 for details). In the meantime, try using one of the following three bridges:
If the bridges are not working, or you have questions, send an email to firstname.lastname@example.org.
The Ethiopian Telecommunication Corporation, which happens to be the sole telecommunication service provider in Ethiopia, has deployed or begun testing Deep Packet Inspection (DPI) of all Internet traffic. We have previously analyzed the same kind of censorship in China, Iran, and Kazakhstan.
Reports show that Tor stopped working a week ago -- even with bridges configured. Websites such as https://gmail.com/, https://facebook.com/, https://twitter.com/, and even https://torproject.org/ continue to work. The graphs below show the effects of this deployment of censorship based on Deep Packet Inspection:
An analysis of data collected by a volunteer shows that they are doing some sort of TLS fingerprinting. The TLS server hello, which is sent by the Tor bridge after the TLS client hello, never reaches the client. We don't know exactly what they are fingerprinting on, but our guess is that it is either the client hello or the server hello. An illustration can be found in this network flow diagram.
Thanks to Philipp Winter and George Kadianakis for helping me analyze the data. If you have more information about the censorship in Ethiopia, please email email@example.com.
We're pleased to announce that the Tor Project and Tails are hosting students this year as part of Google Summer of Code. Out of the 26 applications to us we were able to take on six fantastic students:
Ravi Padmala - Introduction
Project: Stem Improvements and Arm port
Mentor: Damian / Sathyanarayanan
Feroze Naina - Introduction
Project: Implementing Hidden Service Configuration
Mentor: Tomás / Sebastian
Michele Orrù - Introduction
Project: Anonymous Python Application Framework
Mentor: Arturo / George
Julien Voisin - Introduction
Project: Tails Server
Mentor: intrigeri, anonym
Project: Pluggable Transports in Python
Mentor: George / Nick
Mentor: Zack Weinberg / Steven / Roger
Projects officially begin on May 21st. We're thrilled to have them with us, and have our fingers crossed that they'll stay afterward to become core developers.
Many thanks to Google for having the program again this year! -Damian
In December 2011 we were aware of Kazakhstan increasing Internet censorship in response to some unrest and protests in Zhanaozen in the west. The censorship was then deployed around the country, in many cases with the full support of the populace. The initial invesitgation showed simple IP address blocking coupled with basic dns censorship. Tor continued to work without incident until this week.
JSC KazTransCom, AS35104, has deployed or begun testing deep packet inspection (dpi) of all Internet traffic. They specifically target SSL-based protocols for blocking. This includes Tor, IPsec, and PPTP-based technologies, as well as some SSL-based VPNs. Business and private users of these technologies are equally affected.
An example of the censorship, as recorded by volunteers in country, can be found in this network flow diagram. Kazakhstan is identifying and blocking the SSL client key exchange during the setup of an SSL connection. This graph shows the effects of this deployment of censorship based on dpi.
Luckily, due to our recent experience with Iran we have an answer for people: use obfsproxy. Obfsproxy continues to work in Kazakhstan, as well as Iran. In fact, it works in any country where dpi is used to censor citizens' access to the Internet.
Thank you to the volunteers for spending their Valentine's Day collecting and analyzing data.
On Feb 9, Iran started to filter SSL connections on much of their network. Since the Tor protocol uses SSL, that means Tor stopped working too — even Tor with bridges, since bridges use SSL too.
We've been quietly developing Obfsproxy, a new tool to make it easier to change how Tor traffic looks on the network. In late 2011 Iran moved into the #2 position in global Tor user count, and several important political events are scheduled in Iran this month and next. This situation seemed like a good time to test our new tool while also helping improve Internet freedom around the world.
We started with a "Tor Obfsproxy Browser Bundle" with two test obfsproxy bridges in it, to verify that it worked in-country. Then we got over 300 volunteers running more obfsproxy bridges (even with our complex build instructions!), and picked fourteen fast stable trustworthy obfsproxy bridges for an updated bundle which we put out the morning of Feb 11. We spent the weekend fixing usability, stability, and scalability bugs, and put out another bundle on Feb 13 with new versions of Vidalia, Tor, and Obfsproxy.
Thousands of people in Iran successfully used the Obfsproxy Bundle over the weekend:
We did some spot-checking and it seems that the new addresses on Feb 14 are mostly different from the new addresses on Feb 13; but I would guess these are mostly returning users with dynamic IP addresses, rather than actually fresh users. More importantly, these people will be thinking about Obfsproxy next time the filter cracks down — and based on current events, that next time won't be far off. Finally, even though it looks like SSL and Tor are back, I expect Iran will keep throttling SSL traffic as they've been doing for months, so the Obfsproxy bundle will still be more fun to use in Iran than the normal Tor bundles.
How does it work?
Deep Packet Inspection (DPI) algorithms classify Internet traffic by protocol. That is, they look at a given traffic flow and decide whether it's http, ssl, bittorrent, vpn, etc. Governments like Iran, China, and Syria increasingly use these tools (and they often purchase them from Western corporations, but that's a different story) to implement their country-wide censorship, either by looking for a given protocol and outright blocking it, or by more subtle mechanisms like squeezing down the bandwidth available to a given protocol to discourage its use.
Obfsproxy's role is to make it easy for Tor's traffic flows to look like whatever we like. This way Tor can focus on security and anonymity, and Obfsproxy can focus on appearance. The first thing we decided to try looking like was nothing at all: the "obfs2" module adds an encryption wrapper around Tor's traffic, using a handshake that has no recognizable byte patterns.
It won't work perfectly. For example, the traffic flows will still have recognizable timing, volume, and packet size characteristics; a good entropy test would show that the handshake looks way more random than most handshakes; and the censors could always press the "only allow protocols my DPI box recognizes" panic button. Each step in this arms race aims to force the censor to a) put more development time and DPI resources into examining flows, and b) risk more false positives, that is, risk blocking innocent users that they didn't realize they'd be blocking.
This particular new obfuscating layer isn't the most important feature of Obfsproxy. The best part is that makes it easy to design, deploy, and test other obfuscating layers without messing with the rest of Tor. There's a lot of research into trying to make traffic flows look like other protocols, so for example we could rewrite the Tor flows as valid http that the DPI engine considers harmless. That problem is harder than it sounds — and it sounds hard. But by making a separate component that only has to worry about how the traffic looks, other researchers can try out different strategies without needing to learn so much about the rest of Tor. This approach will also let us easily plug in other transports like Telex, and it will also let other circumvention projects reuse Obfsproxy so they don't have to reinvent our wheels.
One of the choices we faced was how widely and loudly to mention the bundle. While we think it would be hard and/or risky for attackers to block the Obfsproxy protocol, the bundle included 14 preconfigured bridge addresses, and censors could just plug those addresses into their blacklists. We started the weekend telling people to only tell their close friends, but on Sunday we opted for a broader publicity push inside the activist community for two reasons. First, the new Vidalia release (0.2.17) lets users configure their own obfsproxy bridge addresses, so if the preconfigured addresses get blocked the user can just put in new ones. Second, it became clearer that the blocking would let up in a few days once the immediate political pressure was over, and we decided it was more important to get the word out about Obfsproxy in general so these users will know about it next time.
I should point out that I don't think they were targeting Tor here. They were targeting popular websites that use SSL, like Gmail and Facebook. Tor was collateral damage because we opted to make Tor traffic look like SSL. That said, we should not forget that we are on their radar: they targeted Tor by DPI in September 2011, and the Diginotar breakin obtained a fake SSL cert for torproject.org.
The next choice we face is: what other communities should we tell? The bundle works great in China too, where their aggressive censorship has been a real hassle for us the past year or so. Some other countries in Asia appear to be newly using DPI to recognize Tor traffic (more on that in an upcoming blog post). We have more development work to do before we can keep up with the China arms race, including teaching obfsproxy bridges to automatically report their addresses to us and teaching our bridgedb service to give them out, and we need to answer research questions around getting more bridges, giving them out in smarter ways, learning when they get blocked, and making it hard for censors to find them. We also need to spread the word carefully, since the arms race is as much about not drawing the attention of the censors as it is about the technology. But the Obfsproxy Bundle works out of the box right now in every censoring country we know of, so we shouldn't be too quiet about it.
And finally, thanks go to George Kadianakis for taking Obfsproxy on as his Google Summer of Code 2011 Project; to Nick Mathewson for mentoring him and getting the Obfsproxy architecture going in the right direction; to Sebastian Hahn for spending all weekend with me fixing bugs and making packages; and to Karsten Loesing, Erinn Clark, Robert Ransom, Runa Sandvik, Nick, George, and the broader Tor community for stepping up over the weekend to help us take it from "early prototype" to "deployed software in use by 5000+ people" in 72 hours.
Greetings! My name is Tim Wilde, Software Engineer at Team Cymru, Inc., and a big fan of the Tor Project and everything that they do. We've helped out the Tor Project with a few investigations into probing/blocking of Tor by oppressive regimes, and the guys asked me to write this one up for the Tor Blog, so here I am! Note: any opinions expressed here are mine, nor those of Team Cymru or the Tor Project.
In October 2011, ticket #4185 was filed in the Tor bug tracker by a user in China who found that their connections to US-based Tor bridge relays were being regularly cut off after a very short period of time. At the time we performed some basic experimentation and discovered that Chinese IPs (presumably at the behest of the Great Firewall of China, or GFW) would reach out to the US-based bridge and connect to it shortly after the Tor user in China connected, and, if successful, shortly thereafter the connection would be blocked by the GFW. There wasn't time for a detailed investigation and analysis at the time, but that kernel eventually grew into the investigation detailed below. We were, however, able to determine that limiting connections to the bridge relay to only the single IP expected to be its client would, in fact, block the probes and allow the connection to remain open for an extended period (>48 hours in our testing).
Between 05 DEC and 09 DEC 2011, we undertook a detailed and methodical investigation into probing and blocking of Tor connections originating within China. Unfortunately for our analysis, it appears that the GFW's active blocking of connections to Tor bridges had stopped, but we were still able to gather valuable data about the probing performed by the GFW, which previously led directly and verifiably to blocking.
To this end we discovered two types of probing. First, "garbage binary" probes, containing nothing more than arbitrary (but sometimes repeated in later probes) binary data, were experienced by the non-China side of any connection that originated from China to TCP port 443 (HTTPS) in which an SSL negotiation was performed. This probe was performed in near-real-time after the connection was established, implying near-line-rate deep packet inspection (DPI) capabilities. TCP/443 connections not performing an SSL handshake, such as using the obfsproxy obfs2 protocol or a plain-text protocol, did not provoke probing. The purpose of these probes is unknown, and further investigation is difficult to justify when it seems relatively clear that these probes are not aimed at Tor.
The second type of probe, on the other hand, is aimed quite directly at Tor. When a Tor client within China connected to a US-based bridge relay, we consistently found that at the next round 15 minute interval (HH:00, HH:15, HH:30, HH:45), the bridge relay would receive a probe from hosts within China that not only established a TCP connection, but performed an SSL negotiation, an SSL renegotiation, and then spoke the Tor protocol sufficiently to build a one-hop circuit and send a BEGIN_DIR cell. No matter what TCP port the bridge was listening on, once a Tor client from China connected, within 3 minutes of the next 15 minute interval we saw a series of probes including at least one connection speaking the Tor protocol.
The good news is, we were able to isolate which characteristic of the Tor handshake the GFW was using to decide whether or not to initiate these probes. By making a simple change to the list of supported SSL ciphers in the "hello" packet sent by the Tor client within China, we were able to prevent the probes from taking place. This has been documented and is being discussed in ticket #4744. This differs slightly from the method used by Iran to block Tor in September 2011, using the client-side of the SSL negotiation as its trigger rather than the server-side. It is likely, however, that technology capable of targeting either side of the connection to this degree could also target the other side, so it remains important to consider both the server and client sides of the Tor connection when attempting to blend in with normal, benign traffic.
The bad news, however, is that this is happening at all. This probe again implies sophisticated near-line-rate DPI technology, coupled with a system that is aimed directly at Tor, using code that actually speaks the Tor protocol. Clearly there is a target painted firmly on Tor, and it is quite likely that the Chinese will continue to adapt their censorship technology as the Tor Project adapts to them.
There is light at the end of the tunnel, though. A number of ideas have been put forward about new protocols, handshakes, and extensions to the Tor protocol that could be used to combat this type of censorship technology in a more long-term-sustainable way. Proposal 190 provides for password authorization for bridge relays. obfsproxy provides an extra layer on top of a Tor connection that makes it look like generic binary data. The Tor v3 link protocol/handshake, currently available and in testing in the 0.2.3.x-alpha series of Tor releases, eliminates SSL renegotiation in Tor session establishment, removing one of Tor's current "sore thumbs" that stick out from normal HTTPS SSL traffic.
You're welcome to check out my full report on this investigation for more detail than I could provide in the blog post here. Thanks very much to everyone who assisted with the investigation and the report! It was fun to investigate and report on this, and I hope to have the opportunity to help out with similar adventures in the future.
The July 2011 Progress Report is at the bottom of this post and at https://blog.torproject.org/files/2011-July-Monthly-Report.pdf.
Highlights include continued progress on protocol obfuscating proxy, a new bridge guard design, outreach, scalability improvements, orbot updates, and a number of translation updates.
Update 2011-08-15: based on feedback, created a plaintext version of the pdf. It doesn't contain the images obviously, but does contain all of the content. Generated the text file via pandoc. The text file is here, https://blog.torproject.org/files/2011-July-Monthly-Report.txt
The April 2011 Progress Report is attached to this post and available at https://blog.torproject.org/files/2011-April-Monthly-Report.pdf.
Highlights include releases for tor, vidalia, arm, and libevent. Updates to pluggable transports, obfsproxy, torbutton, many translation and core architecture updates.