bridges

Recently somebody asked me why our usage numbers in China are so low. More precisely, his question was "How do I read this graph in any way other than 'Tor is effectively blocked in China'?" After writing up an answer for him, I realized I should share it with the rest of the Tor community too.

The correct interpretation of the graph is "obfs3 bridges have not been deployed enough to keep up with the demand in China". So it isn't that Tor is blocked — it's that we haven't done much of a deployment for obfs3 bridges or ScrambleSuit bridges, which are the latest steps in the arms race.

The short explanation is that the old vanilla SSL Tor transport doesn't work in China anymore due to their active probing infrastructure. The obfs2 transport doesn't work anymore either, for the same reason. The obfs3 transport works great for now, and thousands of people are happily using it — and some of those people aren't reflected in the graphs you see (I'll explain that more below).

The medium-length explanation is that we've been leading and coordinating the international research effort at understanding how to design and analyze transports that resist both DPI and active probing, and approximately none of these approaches have been deployed at a large scale yet. So it doesn't make sense to say that Tor is blocked in China, because it mischaracterizes Tor as a static protocol. "Tor" when it comes to censorship circumvention is a toolbox of options — some of them work in China, some don't. The ones that work (i.e. that should resist both DPI and active probing) haven't been rolled out very widely, in large part because we have funders who care about the research side but we have nobody who funds the operations, deployment, or scale-up side.

The long explanation is that it comes down to three issues:

First, there are some technical steps we haven't finished deploying in terms of collecting statistics about users of bridges + pluggable transports. The reason is that the server side of the pluggable transport needs to inform the Tor bridge what country the user was from, so the Tor bridge can include that in its (aggregated, anonymized) stats that it publishes to the metrics portal. We've now built most of the pieces, but most of the deployed bridges aren't running the new code yet. So the older bridges that are reporting their user statistics aren't seeing very many users from China, while the bridges that *aren't* reporting their user statistics, which are the ones that offer the newer pluggable transports, aren't well-represented in the graph. We have some nice volunteers looking into what fraction of deployed obfs3 bridges don't have this new 'extended ORPort' feature. But (and you might notice the trend here) we don't have any funders currently who care about counting bridge users in China.

Second, we need to get more addresses. One approach is to get them from volunteers who sign up their computer as a bridge. That provides great sustainability in terms of community involvement (we did a similar push for obfs2 bridges back when Iran was messing with SSL, and got enough to make a real difference at the time), but one address per volunteer doesn't scale very well. The intuition is that the primary resource that relays volunteer is bandwidth, whereas the primary resource that bridges volunteer is their address — and while bandwidth is an ongoing contribution, once your IP address gets blocked then your contribution has ended, at least for the country that blocked it, or until you get another address via DHCP, etc. The more scalable approaches to getting bridge addresses involve coordinating with ISPs and network operators, and/or designs like Flashproxy to make it really easy for users to sign up their address. I describe these ideas more in "approach four" and "approach five" of the Strategies for getting more bridge addresses blog post. But broad deployment of those approaches is again an operational thing, and we don't have any funded projects currently for doing it.

Third, we need ways of letting people learn about bridges and use them without getting them noticed. We used to think the arms race here was "how do you give out addresses such that the good guys can learn a few while the bad guys can't learn all of them", a la the bridges.torproject.org question. But it's increasingly clear that scanning resistance will be the name of the game in China: your transport has to not only blend in with many other flows (to drive up the number of scans they have to launch), but also when they connect to that endpoint and speak your protocol, your service needs to look unobjectionable there as well. Some combination of ScrambleSuit and FTE are great starts here, but it sure is a good thing that the research world has been working on so many prototype designs lately.

So where does that leave us? It would be neat to think about a broad deployment and operations plan here. I would want to do it in conjunction with some other groups, like Team Cymru on the technical platform side and some on-the-ground partner groups for distributing bridge addresses more effectively among social networks. We've made some good progress on the underlying technologies that would increase the success chances of such a deployment — though we've mostly been doing it using volunteers in our spare time on the side, so it's slower going than it could be. And several other groups (e.g. torservers.net) have recently gotten funding for deploying Tor bridges, so maybe we could combine well with them.

In any case it won't be a quick and simple job, since all these pieces have to come together. It's increasingly clear that just getting addresses should be the easy part of this. It's how you give them out, and what you run on the server side to defeat China's scanning, that still look like the two tough challenges for somebody trying to scale up their circumvention tool design.

The Tor Cloud images have been updated to include the latest version of Ubuntu 12.04.2 LTS (Precise Pangolin). An instance created from any of the images will automatically be a normal bridge, an obfs2 bridge, and an obfs3 bridge.

When setting up an instance, please remember to edit the security group with the following rules: SSH (22), HTTPS (443), 40872, and 52176.

The Tor Cloud images for all the eight regions have been updated with a minor fix in the rc.local script. In addition, all private bridge images now include Obfsproxy. You will not need to start a new instance if you are already running a Tor Cloud instance with Ubuntu Precise.

The Tor Cloud images for all the seven regions have been updated to include the latest cloud image for stable Ubuntu release 12.04.1 LTS (Precise Pangolin). These new images are available on the Tor Cloud website. You will not need to start a new instance you are already running a Tor Cloud instance with Ubuntu Precise.

The Tor Cloud images for all the seven regions have been updated to fix a bug found in the unattended-upgrades configuration. The normal bridge images have also been updated to include obfsproxy, which attempts to help users circumvent censorship by transforming the Tor traffic between the client and the bridge.

If you are already running a Tor Cloud bridge, you will need to either manually update your image, or set up a new Tor Cloud bridge and terminate the old one. If you decide not to take action, your image will fail to upgrade Tor correctly and will not be running as a bridge.

If you just want to fix the bug in the unattended-upgrades configuration, do the following; log on with SSH and edit /etc/apt/apt.conf.d/50unattended-upgrades to say precise instead of lucid.

The Tor Cloud images for all the seven regions have been updated to include the latest cloud image for stable Ubuntu release 12.04.1 LTS (Precise Pangolin). These new images are available on the Tor Cloud website.

The new images include Tor's new GPG key, uses apt-get instead of aptitude, and also includes the deb.torproject.org-keyring package (#6776).

If you are already running a Tor Cloud bridge, you will need to either manually update your image, or set up a new Tor Cloud bridge and terminate the old one. If you decide not to take action, your image will fail to upgrade Tor correctly and will not be running as a bridge. To manually update your image; log on with SSH, and follow the instructions to add the new GPG key, upgrade Tor, and install the deb.torproject.org-keyring package.

The Tor Cloud images for all the seven regions have been updated to include the latest cloud image for stable Ubuntu release 10.04 LTS (Lucid Lynx). These new images are available on the Tor Cloud website.

The new images include a fix to allow Tor to upgrade automatically without requiring user intervention (#6511).

If you are already running a Tor Cloud bridge, you will need to either manually update your image, or set up a new Tor Cloud bridge and terminate the old one. If you decide not to take action, your image will fail to upgrade Tor correctly and will not be running as a bridge.

To manually update your image, do the following:

0. Log on with SSH
1. Open /etc/apt/apt.conf.d/50unattended-upgrades
2. Add the line: Dpkg::Options { --force-confold; }
3. Save and exit

Although Tor was originally designed to enhance privacy, the network is increasingly being used to provide access to websites from countries where Internet connectivity is filtered. To better support this goal, Tor introduced the feature of bridge nodes – Tor nodes which route traffic but are not published in the list of available Tor nodes. Consequently bridges are commonly not blocked in countries where access to the public Tor nodes are (with the exception of China and Iran).

For bridge nodes to work well, we need lots of them. This is both to make it hard to block them all, and also to provide enough bandwidth capacity for all their users. To help achieve this goal, the Tor Browser Bundle now makes it as easy as clicking a button to turn a normal Tor client into a bridge. The upcoming version of Tor will even set up port forwarding on home routers (through UPnP and NAT-PMP) to make it easier still.

There are, however, potential risks to Tor users if they turn their client into a bridge. These were summarized in a paper by Jon McLachlan and Nicholas Hopper: "On the risks of serving whenever you surf". I discussed these risks in a previous blog post, along with ways to mitigate them. In this blog post I'll cover some initiatives the Tor project is working on (or considering), which could further reduce the risks.

The attack than McLachlan and Hopper propose involves finding a list of bridges, and then probing them over time. If the bridge responds at all, then we know that the bridge operator could be surfing the web though Tor. So, if the attacker suspects that a blogger is posting through Tor and the blogger's Tor client is also a bridge, then only the bridges which are running at the time a blog post was written could belong to the blogger. There are likely to be quite a few such bridges, but if the attack is repeated, the set of potential bridge IP addresses will be narrowed down.

What is more, the paper also proposes measuring how quickly each bridge responds, which might tell the attacker something about how heavily the bridge operator is using their Tor client. If the attacker probes the rest of the Tor network, and sees a node with performance patterns similar to the bridge being probed, then it is likely that there is a connection going through that node and that bridge. In some circumstances this attack could be used to track connections all the way through the Tor network, and back to a client who is also acting as a bridge.

One way of reducing the risk of these attacks succeeding is to run the bridge on a device which is always on. In this case, just because a bridge is running doesn't mean the operator is using Tor. Therefore less information is leaked to a potential attacker. As laptops become more prevalent, keeping a bridge on 24/7 can be inconvenient so the Tor Project is working on two bits of hardware (known as Torouters) which can be left running a bridge even if the user's computer is off. If someone probes the bridge now, they can't learn much about whether the operator is using Tor as a client, and so it leaks less information about what the user is doing.

Having an always-on bridge helps resist profiling based on when a client is on. However, evaluating the risk of profiling based on bridge performance is more complex. The reason this attack works is that when a node is congested, increasing traffic on one connection (say, due to the bridge operator using Tor) decreases the traffic on another (say, the attacker probing the bridge). If you run a Tor client on your PC, and a Tor bridge on a separate device, this reduces the opportunity for congestion on one leading to congestion on the other. However, there is still the possibility for congestion on the network connection which the bridge and device share, and maybe you do want to use the bridge device as a client too. So the Torouter helps, but doesn't fix the problem.

To fully decouple bridges from clients, they need to run on different hardware and networks. Here the Tor Cloud project is ideal – bridges run on Amazon's (or another cloud provider's) infrastructure so congestion on the bridge can't affect the Tor client running on your PC, or it's network.

But maybe you do want to use your bridge as a client for whatever reason. Recall that the first step of the attack proposed by McLachlan and Hopper is to find the bridges' IP addresses. This is becoming increasingly difficult as more bridges are being set up. Also, there is work in progress to make it harder to scan networks for bridges. Originally this was intended to make it harder to find and block bridges, but it applies equally well to preventing probing. These schemes (e.g. BridgeSPA by Smits et. al.) mean that just because you have a bridge's IP address, it doesn't mean that you can route traffic through it (to measure performance) or even check if the bridge is running. To do either, you need to have a secret which is distributed only to authorized users of the bridge.

There is still more work to be done in protecting bridge operators, but the projects discussed above (and in the previous blog post) certainly improve the situation. Work continues both in academia and at the Tor Project to better understand the problem and develop further fixes.

The Tor network relies on volunteers to donate bandwidth. The more people who run Tor as a bridge or a relay, the faster and safer the network becomes. Tactical Tech created a video to encourage you to join the Tor Network. The video, and information about how you can set up a bridge or a relay, can be found on https://www.torproject.org/relays. If you want to help us translate the video into your language, let us know!

The full HD video can be found here: https://media.torproject.org/video/2012-03-04-BuildingBridges-HD.ogv

On Feb 9, Iran started to filter SSL connections on much of their network. Since the Tor protocol uses SSL, that means Tor stopped working too — even Tor with bridges, since bridges use SSL too.

We've been quietly developing Obfsproxy, a new tool to make it easier to change how Tor traffic looks on the network. In late 2011 Iran moved into the #2 position in global Tor user count, and several important political events are scheduled in Iran this month and next. This situation seemed like a good time to test our new tool while also helping improve Internet freedom around the world.

We started with a "Tor Obfsproxy Browser Bundle" with two test obfsproxy bridges in it, to verify that it worked in-country. Then we got over 300 volunteers running more obfsproxy bridges (even with our complex build instructions!), and picked fourteen fast stable trustworthy obfsproxy bridges for an updated bundle which we put out the morning of Feb 11. We spent the weekend fixing usability, stability, and scalability bugs, and put out another bundle on Feb 13 with new versions of Vidalia, Tor, and Obfsproxy.

Thousands of people in Iran successfully used the Obfsproxy Bundle over the weekend:

We did some spot-checking and it seems that the new addresses on Feb 14 are mostly different from the new addresses on Feb 13; but I would guess these are mostly returning users with dynamic IP addresses, rather than actually fresh users. More importantly, these people will be thinking about Obfsproxy next time the filter cracks down — and based on current events, that next time won't be far off. Finally, even though it looks like SSL and Tor are back, I expect Iran will keep throttling SSL traffic as they've been doing for months, so the Obfsproxy bundle will still be more fun to use in Iran than the normal Tor bundles.

How does it work?

Deep Packet Inspection (DPI) algorithms classify Internet traffic by protocol. That is, they look at a given traffic flow and decide whether it's http, ssl, bittorrent, vpn, etc. Governments like Iran, China, and Syria increasingly use these tools (and they often purchase them from Western corporations, but that's a different story) to implement their country-wide censorship, either by looking for a given protocol and outright blocking it, or by more subtle mechanisms like squeezing down the bandwidth available to a given protocol to discourage its use.

Obfsproxy's role is to make it easy for Tor's traffic flows to look like whatever we like. This way Tor can focus on security and anonymity, and Obfsproxy can focus on appearance. The first thing we decided to try looking like was nothing at all: the "obfs2" module adds an encryption wrapper around Tor's traffic, using a handshake that has no recognizable byte patterns.

It won't work perfectly. For example, the traffic flows will still have recognizable timing, volume, and packet size characteristics; a good entropy test would show that the handshake looks way more random than most handshakes; and the censors could always press the "only allow protocols my DPI box recognizes" panic button. Each step in this arms race aims to force the censor to a) put more development time and DPI resources into examining flows, and b) risk more false positives, that is, risk blocking innocent users that they didn't realize they'd be blocking.

This particular new obfuscating layer isn't the most important feature of Obfsproxy. The best part is that makes it easy to design, deploy, and test other obfuscating layers without messing with the rest of Tor. There's a lot of research into trying to make traffic flows look like other protocols, so for example we could rewrite the Tor flows as valid http that the DPI engine considers harmless. That problem is harder than it sounds — and it sounds hard. But by making a separate component that only has to worry about how the traffic looks, other researchers can try out different strategies without needing to learn so much about the rest of Tor. This approach will also let us easily plug in other transports like Telex, and it will also let other circumvention projects reuse Obfsproxy so they don't have to reinvent our wheels.

Moving forward

One of the choices we faced was how widely and loudly to mention the bundle. While we think it would be hard and/or risky for attackers to block the Obfsproxy protocol, the bundle included 14 preconfigured bridge addresses, and censors could just plug those addresses into their blacklists. We started the weekend telling people to only tell their close friends, but on Sunday we opted for a broader publicity push inside the activist community for two reasons. First, the new Vidalia release (0.2.17) lets users configure their own obfsproxy bridge addresses, so if the preconfigured addresses get blocked the user can just put in new ones. Second, it became clearer that the blocking would let up in a few days once the immediate political pressure was over, and we decided it was more important to get the word out about Obfsproxy in general so these users will know about it next time.

I should point out that I don't think they were targeting Tor here. They were targeting popular websites that use SSL, like Gmail and Facebook. Tor was collateral damage because we opted to make Tor traffic look like SSL. That said, we should not forget that we are on their radar: they targeted Tor by DPI in September 2011, and the Diginotar breakin obtained a fake SSL cert for torproject.org.

The next choice we face is: what other communities should we tell? The bundle works great in China too, where their aggressive censorship has been a real hassle for us the past year or so. Some other countries in Asia appear to be newly using DPI to recognize Tor traffic (more on that in an upcoming blog post). We have more development work to do before we can keep up with the China arms race, including teaching obfsproxy bridges to automatically report their addresses to us and teaching our bridgedb service to give them out, and we need to answer research questions around getting more bridges, giving them out in smarter ways, learning when they get blocked, and making it hard for censors to find them. We also need to spread the word carefully, since the arms race is as much about not drawing the attention of the censors as it is about the technology. But the Obfsproxy Bundle works out of the box right now in every censoring country we know of, so we shouldn't be too quiet about it.

And finally, thanks go to George Kadianakis for taking Obfsproxy on as his Google Summer of Code 2011 Project; to Nick Mathewson for mentoring him and getting the Obfsproxy architecture going in the right direction; to Sebastian Hahn for spending all weekend with me fixing bugs and making packages; and to Karsten Loesing, Erinn Clark, Robert Ransom, Runa Sandvik, Nick, George, and the broader Tor community for stepping up over the weekend to help us take it from "early prototype" to "deployed software in use by 5000+ people" in 72 hours.