pluggable transports

How to read our China usage graphs

Recently somebody asked me why our usage numbers in China are so low. More precisely, his question was "How do I read this graph in any way other than 'Tor is effectively blocked in China'?" After writing up an answer for him, I realized I should share it with the rest of the Tor community too.

The correct interpretation of the graph is "obfs3 bridges have not been deployed enough to keep up with the demand in China". So it isn't that Tor is blocked — it's that we haven't done much of a deployment for obfs3 bridges or ScrambleSuit bridges, which are the latest steps in the arms race.

The short explanation is that the old vanilla SSL Tor transport doesn't work in China anymore due to their active probing infrastructure. The obfs2 transport doesn't work anymore either, for the same reason. The obfs3 transport works great for now, and thousands of people are happily using it — and some of those people aren't reflected in the graphs you see (I'll explain that more below).

The medium-length explanation is that we've been leading and coordinating the international research effort at understanding how to design and analyze transports that resist both DPI and active probing, and approximately none of these approaches have been deployed at a large scale yet. So it doesn't make sense to say that Tor is blocked in China, because it mischaracterizes Tor as a static protocol. "Tor" when it comes to censorship circumvention is a toolbox of options — some of them work in China, some don't. The ones that work (i.e. that should resist both DPI and active probing) haven't been rolled out very widely, in large part because we have funders who care about the research side but we have nobody who funds the operations, deployment, or scale-up side.

The long explanation is that it comes down to three issues:

First, there are some technical steps we haven't finished deploying in terms of collecting statistics about users of bridges + pluggable transports. The reason is that the server side of the pluggable transport needs to inform the Tor bridge what country the user was from, so the Tor bridge can include that in its (aggregated, anonymized) stats that it publishes to the metrics portal. We've now built most of the pieces, but most of the deployed bridges aren't running the new code yet. So the older bridges that are reporting their user statistics aren't seeing very many users from China, while the bridges that *aren't* reporting their user statistics, which are the ones that offer the newer pluggable transports, aren't well-represented in the graph. We have some nice volunteers looking into what fraction of deployed obfs3 bridges don't have this new 'extended ORPort' feature. But (and you might notice the trend here) we don't have any funders currently who care about counting bridge users in China.

Second, we need to get more addresses. One approach is to get them from volunteers who sign up their computer as a bridge. That provides great sustainability in terms of community involvement (we did a similar push for obfs2 bridges back when Iran was messing with SSL, and got enough to make a real difference at the time), but one address per volunteer doesn't scale very well. The intuition is that the primary resource that relays volunteer is bandwidth, whereas the primary resource that bridges volunteer is their address — and while bandwidth is an ongoing contribution, once your IP address gets blocked then your contribution has ended, at least for the country that blocked it, or until you get another address via DHCP, etc. The more scalable approaches to getting bridge addresses involve coordinating with ISPs and network operators, and/or designs like Flashproxy to make it really easy for users to sign up their address. I describe these ideas more in "approach four" and "approach five" of the Strategies for getting more bridge addresses blog post. But broad deployment of those approaches is again an operational thing, and we don't have any funded projects currently for doing it.

Third, we need ways of letting people learn about bridges and use them without getting them noticed. We used to think the arms race here was "how do you give out addresses such that the good guys can learn a few while the bad guys can't learn all of them", a la the bridges.torproject.org question. But it's increasingly clear that scanning resistance will be the name of the game in China: your transport has to not only blend in with many other flows (to drive up the number of scans they have to launch), but also when they connect to that endpoint and speak your protocol, your service needs to look unobjectionable there as well. Some combination of ScrambleSuit and FTE are great starts here, but it sure is a good thing that the research world has been working on so many prototype designs lately.

So where does that leave us? It would be neat to think about a broad deployment and operations plan here. I would want to do it in conjunction with some other groups, like Team Cymru on the technical platform side and some on-the-ground partner groups for distributing bridge addresses more effectively among social networks. We've made some good progress on the underlying technologies that would increase the success chances of such a deployment — though we've mostly been doing it using volunteers in our spare time on the side, so it's slower going than it could be. And several other groups (e.g. torservers.net) have recently gotten funding for deploying Tor bridges, so maybe we could combine well with them.

In any case it won't be a quick and simple job, since all these pieces have to come together. It's increasingly clear that just getting addresses should be the easy part of this. It's how you give them out, and what you run on the server side to defeat China's scanning, that still look like the two tough challenges for somebody trying to scale up their circumvention tool design.

Pluggable transports bundles 2.4.18-rc-1-pt1 and 2.4.18-rc-2-pt1 with Firefox 17.0.11esr

There are new Pluggable Transports Tor Browser Bundles with Firefox 17.0.11esr. They are made from the Tor Browser Bundle 2.4.18-rc-1 release of November 19, except for the 64-bit GNU/Linux bundle, which is made from the 2.4.18-rc-2 release of November 20.

Pluggable Transports bundle download

Pluggable transports bundles 2.4.17-rc-1-pt2 with Firefox 17.0.10esr

There are new Pluggable Transports Tor Browser Bundles with Firefox 17.0.10esr. They are made from the Tor Browser Bundle release of November 1.

These are mostly the same as 2.4.17-rc-1-pt1 released a few days ago, the only change being a workaround that allows them to run on OS X Mavericks.

Pluggable transports bundles 2.4.17-rc-1-pt1 with Firefox 17.0.10esr

There are new Pluggable Transports Tor Browser Bundles with Firefox 17.0.10esr. They are made from the Tor Browser Bundle release of November 1.

The OS X bundle won't work in the new OS X Mavericks by default. It is caused by some changes in the new operating system release. We know about this problem and are working on fixing it. If you are an affected user, you can try this workaround of placing absolute paths in the torrc file.

The bundles contain flash proxy and obfsproxy configured to run by default. If you want to use flash proxy, you will have to take the extra steps listed in the flash proxy howto.

These bundles contain the same hardcoded obfs2 bridge addresses as the previous bundles which may work for some jurisdictions but you are strongly advised to get new bridge addresses from BridgeDB.

Pluggable transports bundles 2.4.17-beta-2-pt3 with Firefox 17.0.9esr

There are new Pluggable Transports Tor Browser Bundles with Firefox 17.0.9esr. They are made from the Tor Browser Bundle release of September 20 and contain important security fixes.

The bundles contain flash proxy and obfsproxy configured to run by default. If you want to use flash proxy, you will have to take the extra steps listed in the flash proxy howto.

These bundles contain the same hardcoded obfs2 bridge addresses as the previous bundles which may work for some jurisdictions but you are strongly advised to get new bridge addresses from BridgeDB.

Pluggable transports bundles 2.4.15-beta-2-pt1 with Firefox 17.0.8esr

We've updated the Pluggable Transports Tor Browser Bundles with Firefox 17.0.8esr and Tor 0.2.4.15-beta-2. These correspond to the Tor Browser Bundle release of August 9 and contain important security fixes.

These bundles contain flash proxy and obfsproxy configured to run by default. If you want to use flash proxy, you will have to take the extra steps listed in the flash proxy howto.

These bundles contain the same hardcoded obfs2 bridge addresses as the previous bundles which may work for some jurisdictions but you are strongly advised to get new bridge addresses from BridgeDB.

These bundles are signed by David Fifield (0x5CD388E5) with this fingerprint.

Pluggable transports bundles 2.4.12-alpha-2-pt1 with Firefox 17.0.6esr

We've updated the Pluggable Transports Tor Browser Bundles with Firefox 17.0.6esr and Tor 0.2.4.11-alpha. These correspond to the Tor Browser Bundle release of May 14.

These bundles contain contain flash proxy and obfsproxy configured to run by default. Flash proxy has a new faster registration method, flashproxy-reg-appspot. The existing flashproxy-reg-email and flashproxy-reg-http will be tried if flashproxy-reg-appspot doesn't work.

If you want to use flash proxy, you will have to take the extra steps listed in the flash proxy howto.

These bundles contain the same hardcoded obfs2 bridge addresses as the previous bundles which may work for some jurisdictions but you are strongly advised to get new bridge addresses from BridgeDB: https://bridges.torproject.org/?transport=obfs2 https://bridges.torproject.org/?transport=obfs3.

These bundles are signed by David Fifield (0x5CD388E5) with this fingerprint.

New Name for Obfsproxy Tor Browser Bundles

Some days ago we released new Pluggable Transport Bundles which aim to
replace the old Pyobfsproxy/Flashproxy bundles and the even older
Obfsproxy bundles.

Users are encouraged to upgrade to the new bundles since they support
all the currently deployed pluggable transports and the latest
versions of Firefox and Torbutton.

The new bundles also contain the latest release of the Python version
of Obfsproxy, which replaces the legacy version that was written in C.

Finally, from now on Obfuscated bundles will be called "Pluggable
Transport Bundles" and each new version will contain all the deployed
pluggable transports. Users need to use http://bridges.torproject.org
to get bridges with pluggable transports ("obfuscated/obfsproxy
bridges") for their bundles.

New Pluggable Transports bundles 0.2.4.11-alpha (flashproxy + obfsproxy)

We've updated the Pluggable Transports Tor Browser Bundles with Firefox 17.0.4esr and Tor 0.2.4.11-alpha. These releases have numerous bug fixes and a new Torbutton as well.

Like the previous bundles, these contain Flashproxy and the Python version of Obfsproxy.

Flash proxy is a transport that uses proxies running in web browsers as access points into Tor. Obfsproxy is a pluggable transport that makes network traffic look unlike normal Tor traffic. Both of these technologies make it harder to block access to Tor. If you previously used the obfsproxy bundle, please upgrade to this bundle, which in addition to flash proxy has new obfsproxy bridges.

Flash proxy works differently from other pluggable transports, and you need to take extra steps to make it work. In particular, you will probably need to configure port forwarding in order to receive connections from browser proxies. There are instructions and hints on how to do that at this page: flash proxy howto.

These bundles contain the same hardcoded obfs2 bridge addresses as the previous bundles which may work for some jurisdictions but you are strongly advised to get new bridge addresses from BridgeDB: https://bridges.torproject.org/?transport=obfs2.

Furthermore, we are looking for feedback on how the bundles work. Please leave comments on the flash proxy usability wiki page or ticket #7824 with your experience, good or bad.

There are other ways you can help beyond testing the bundles. One is to run a bridge with pyobfsproxy. Another is to put the flash proxy badge on your web site or blog, or add it to your Wikipedia profile. If you want your browser to continue to be a proxy after a switch to an opt-in model, click the “Yes” button on the options page.

Tor 0.2.3.9-alpha is out

Tor 0.2.3.9-alpha introduces initial IPv6 support for bridges, adds
a "DisableNetwork" security feature that bundles can use to avoid
touching the network until bridges are configured, moves forward on
the pluggable transport design, fixes a flaw in the hidden service
design that unnecessarily prevented clients with wrong clocks from
reaching hidden services, and fixes a wide variety of other issues.

https://www.torproject.org/download

Changes in version 0.2.3.9-alpha - 2011-12-08
Major features:

  • Clients can now connect to private bridges over IPv6. Bridges
    still need at least one IPv4 address in order to connect to
    other relays. Note that we don't yet handle the case where the
    user has two bridge lines for the same bridge (one IPv4, one
    IPv6). Implements parts of proposal 186.
  • New "DisableNetwork" config option to prevent Tor from launching any
    connections or accepting any connections except on a control port.
    Bundles and controllers can set this option before letting Tor talk
    to the rest of the network, for example to prevent any connections
    to a non-bridge address. Packages like Orbot can also use this
    option to instruct Tor to save power when the network is off.
  • Clients and bridges can now be configured to use a separate
    "transport" proxy. This approach makes the censorship arms race
    easier by allowing bridges to use protocol obfuscation plugins. It
    implements the "managed proxy" part of proposal 180 (ticket 3472).
  • When using OpenSSL 1.0.0 or later, use OpenSSL's counter mode
    implementation. It makes AES_CTR about 7% faster than our old one
    (which was about 10% faster than the one OpenSSL used to provide).
    Resolves ticket 4526.
  • Add a "tor2web mode" for clients that want to connect to hidden
    services non-anonymously (and possibly more quickly). As a safety
    measure to try to keep users from turning this on without knowing
    what they are doing, tor2web mode must be explicitly enabled at
    compile time, and a copy of Tor compiled to run in tor2web mode
    cannot be used as a normal Tor client. Implements feature 2553.
  • Add experimental support for running on Windows with IOCP and no
    kernel-space socket buffers. This feature is controlled by a new
    "UserspaceIOCPBuffers" config option (off by default), which has
    no effect unless Tor has been built with support for bufferevents,
    is running on Windows, and has enabled IOCP. This may, in the long
    run, help solve or mitigate bug 98.
  • Use a more secure consensus parameter voting algorithm. Now at
    least three directory authorities or a majority of them must
    vote on a given parameter before it will be included in the
    consensus. Implements proposal 178.

Major bugfixes:

  • Hidden services now ignore the timestamps on INTRODUCE2 cells.
    They used to check that the timestamp was within 30 minutes
    of their system clock, so they could cap the size of their
    replay-detection cache, but that approach unnecessarily refused
    service to clients with wrong clocks. Bugfix on 0.2.1.6-alpha, when
    the v3 intro-point protocol (the first one which sent a timestamp
    field in the INTRODUCE2 cell) was introduced; fixes bug 3460.
  • Only use the EVP interface when AES acceleration is enabled,
    to avoid a 5-7% performance regression. Resolves issue 4525;
    bugfix on 0.2.3.8-alpha.

Privacy/anonymity features (bridge detection):

  • Make bridge SSL certificates a bit more stealthy by using random
    serial numbers, in the same fashion as OpenSSL when generating
    self-signed certificates. Implements ticket 4584.
  • Introduce a new config option "DynamicDHGroups", enabled by
    default, which provides each bridge with a unique prime DH modulus
    to be used during SSL handshakes. This option attempts to help
    against censors who might use the Apache DH modulus as a static
    identifier for bridges. Addresses ticket 4548.

Minor features (new/different config options):

  • New configuration option "DisableDebuggerAttachment" (on by default)
    to prevent basic debugging attachment attempts by other processes.
    Supports Mac OS X and Gnu/Linux. Resolves ticket 3313.
  • Allow MapAddress directives to specify matches against super-domains,
    as in "MapAddress *.torproject.org *.torproject.org.torserver.exit".
    Implements issue 933.
  • Slightly change behavior of "list" options (that is, config
    options that can appear more than once) when they appear both in
    torrc and on the command line. Previously, the command-line options
    would be appended to the ones from torrc. Now, the command-line
    options override the torrc options entirely. This new behavior
    allows the user to override list options (like exit policies and
    ports to listen on) from the command line, rather than simply
    appending to the list.
  • You can get the old (appending) command-line behavior for "list"
    options by prefixing the option name with a "+".
  • You can remove all the values for a "list" option from the command
    line without adding any new ones by prefixing the option name
    with a "/".
  • Add experimental support for a "defaults" torrc file to be parsed
    before the regular torrc. Torrc options override the defaults file's
    options in the same way that the command line overrides the torrc.
    The SAVECONF controller command saves only those options which
    differ between the current configuration and the defaults file. HUP
    reloads both files. (Note: This is an experimental feature; its
    behavior will probably be refined in future 0.2.3.x-alpha versions
    to better meet packagers' needs.)

Minor features:

  • Try to make the introductory warning message that Tor prints on
    startup more useful for actually finding help and information.
    Resolves ticket 2474.
  • Running "make version" now displays the version of Tor that
    we're about to build. Idea from katmagic; resolves issue 4400.
  • Expire old or over-used hidden service introduction points.
    Required by fix for bug 3460.
  • Move the replay-detection cache for the RSA-encrypted parts of
    INTRODUCE2 cells to the introduction point data structures.
    Previously, we would use one replay-detection cache per hidden
    service. Required by fix for bug 3460.
  • Reduce the lifetime of elements of hidden services' Diffie-Hellman
    public key replay-detection cache from 60 minutes to 5 minutes. This
    replay-detection cache is now used only to detect multiple
    INTRODUCE2 cells specifying the same rendezvous point, so we can
    avoid launching multiple simultaneous attempts to connect to it.

Minor bugfixes (on Tor 0.2.2.x and earlier):

  • Resolve an integer overflow bug in smartlist_ensure_capacity().
    Fixes bug 4230; bugfix on Tor 0.1.0.1-rc. Based on a patch by
    Mansour Moufid.
  • Fix a minor formatting issue in one of tor-gencert's error messages.
    Fixes bug 4574.
  • Prevent a false positive from the check-spaces script, by disabling
    the "whitespace between function name and (" check for functions
    named 'op()'.
  • Fix a log message suggesting that people contact a non-existent
    email address. Fixes bug 3448.
  • Fix null-pointer access that could occur if TLS allocation failed.
    Fixes bug 4531; bugfix on 0.2.0.20-rc. Found by "troll_un".
  • Report a real bootstrap problem to the controller on router
    identity mismatch. Previously we just said "foo", which probably
    made a lot of sense at the time. Fixes bug 4169; bugfix on
    0.2.1.1-alpha.
  • If we had ever tried to call tor_addr_to_str() on an address of
    unknown type, we would have done a strdup() on an uninitialized
    buffer. Now we won't. Fixes bug 4529; bugfix on 0.2.1.3-alpha.
    Reported by "troll_un".
  • Correctly detect and handle transient lookup failures from
    tor_addr_lookup(). Fixes bug 4530; bugfix on 0.2.1.5-alpha.
    Reported by "troll_un".
  • Use tor_socket_t type for listener argument to accept(). Fixes bug
    4535; bugfix on 0.2.2.28-beta. Found by "troll_un".
  • Initialize conn->addr to a valid state in spawn_cpuworker(). Fixes
    bug 4532; found by "troll_un".

Minor bugfixes (on Tor 0.2.3.x):

  • Fix a compile warning in tor_inet_pton(). Bugfix on 0.2.3.8-alpha;
    fixes bug 4554.
  • Don't send two ESTABLISH_RENDEZVOUS cells when opening a new
    circuit for use as a hidden service client's rendezvous point.
    Fixes bugs 4641 and 4171; bugfix on 0.2.3.3-alpha. Diagnosed
    with help from wanoskarnet.
  • Restore behavior of overriding SocksPort, ORPort, and similar
    options from the command line. Bugfix on 0.2.3.3-alpha.

Build fixes:

  • Properly handle the case where the build-tree is not the same
    as the source tree when generating src/common/common_sha1.i,
    src/or/micro-revision.i, and src/or/or_sha1.i. Fixes bug 3953;
    bugfix on 0.2.0.1-alpha.

Code simplifications, cleanups, and refactorings:

  • Remove the pure attribute from all functions that used it
    previously. In many cases we assigned it incorrectly, because the
    functions might assert or call impure functions, and we don't have
    evidence that keeping the pure attribute is worthwhile. Implements
    changes suggested in ticket 4421.
  • Remove some dead code spotted by coverity. Fixes cid 432.
    Bugfix on 0.2.3.1-alpha, closes bug 4637.
Syndicate content Syndicate content