February 2011 Progress Report

New hires
We contracted Runa Sandvik to work on moving the torouter project forward,, translations, and integration of tor web server log analysis and publication.

New Releases
On February 23rd, we released an updated Tor -stable. Tor fixes a
variety of less critical bugs. The main other change is a slight tweak to Tor's TLS handshake that makes relays and bridges that run this new version reachable from Iran again. We don't expect this tweak will win the arms race long-term, but it
buys us time until we roll out a better solution. Full announcement at

Censorship resistance

  • Arm development has stayed relatively on track, with the revised
    connection panel very nearly achieving parity with its predecessor
    (and in most respects surpassing it). Most of what remains are
    refinements and tasty new features. Arm has also been added to Debian
    (Sid) and Ubuntu (Natty) with backports pending. Many thanks to Peter
    for his help.
  • Tom spent some time assisting Jacob with a satellite test. The test wound
    up breaking due to flaky hardware, however they were able to collect some usable
  • Created the trac ticket around hidden service improvements, We need to focus on
    improving hidden services and fixing some of the performance and reliability
    issues within.
  • Mike fixed a bunch of torbutton bugs. His summary iteration results are
  • Mike helped fix the bandwidth authority on salsa that exploded due to a

Architecture and Design Docs for better censorship resistance

  • Karsten and Sebastian tried to improve the database schema in metrics-db
    to speed up relay search performance. Unfortunately, the required updates
    from the old schema took forever, so we don't just need a better schema, but
    also a better migration strategy to go from one schema to the next.
  • Karsten started moving code from metrics-db to metrics-web to make the
    metrics website a self-contained unit that's independent of
    aggregating descriptors. The idea is that people can take the metrics-web
    code and improve it or replace it with a better metrics website written in
    the web language of their choice.
  • Karsten started working on better visualizations of Tor data using the
    Thematic Mapping API together with Rachel Binx.

Hide Tor's network signature.

  • Collaborated with George K on obfsproxy, a generic protocol
    obfuscator. It seems to work ok but needs more testing.
  • Nick worked on improving the pluggable-transport design.
  • Jacob did another revision on what is now prop 179,
  • Jacob looked at the EFF SSL data and have some improvements for how we can
    get better data for future research questions.

Outreach and Advocacy

Preconfigured privacy bundles

  • Jacob did some testing of Gibberbot's Tor and OTR integration. Gibberbot
    is an XMPP chat client for Android designed to work over Tor.
  • Jacob did a bunch of work on ttdnsd - some important (but not critical)
    bug fixes and he's planning on pushing out a release in the future. Jacob and
    Robert did some work on torsocks integration and in the process hammered out a
    reasonable torsocks API for people who want to have auto-magically Torified
    sockets without understanding Tor internals.
  • Jacob worked on OpenWRT packaging issues - as well as other work on the
    Torouter project.
  • Jacob worked on Tahoe ( and
    Tor related Hidden Service documentation; after moderate amount of Tor testing
    with Tahoe now and it seems to be partially functional.

Bridge work

  • Karsten prepared a patch for BridgeDB to export bridge pool assignments to
    a local file. This patch needs some cleanup before being deployed on
  • Karsten wrote a first draft of a BridgeDB specification that Nick
    commented on. The next step is to include Nick's comments and change the
    writing style, so that the specification describes what the current BridgeDB
    code does, not what a generic BridgeDB implemention should do.
  • Karsten extended the bridge descriptor sanitizing algorithm to include IP
    address hashes in the sanitized descriptors. Sanitized all existing
    bridge descriptors using this new algorithm. Instead of,
    bridges now have 10.x.y.z addresses with x.y.z being stable for a given
    bridge fingerprint in a given month. This allows analyses of how often
    bridges change their IP addresses in a given month.
  • Christian deployed a new version of BridgeDB, the one that's i18n enabled
    (#1613) and also can dump bridge pool assignments to files. We can now
    assign an amount of unassigned bridges to someone/something and dump
    them to file buckets. See #1612 for more infos. In theory, we can now
    have an amount of Twitter assigned bridges that we pump out over
  • Christian also started writing a python script that is able to dump
    stuff to Twitter.
  • After deployment of the new BridgeDB, some issues came up that were fixed
    (#2556 and others). It seems to run smoothly now. We'll be even more
    happy about it when we have important (read: Chinese and Farsi)
    translations ready and deployed.
  • Christian and Karsten discussed about whether his planned "dump bridge
    assignments to files" feature can use the bucket mechanism of #1612.
    Turns out it can't since both have a different set of goals and would
    be to painful to sync with every change.
  • Mike helped Karsten with improving the output of Torperf for future
    experiments involving circuit build timeouts.


  • Improved Torperf and finally deployed it to collect data about used
    paths and to measure performance with custom guard node selections.
    This is still work in progress together with Mike and Tom as part of
    our first Scrum iteration that ends on March 5.
  • Worked on Florian's and Björn's token bucket patch some more together
    with Sebastian. The current state of the patch is that it needs some
    more love before it can be merged into 0.2.3.x.
  • Nick collaborated a little with two volunteers on what we
    think at this point must be the 5th generation of a "launch a private
    network" tool. This one is called "chutney".
  • Nick reviewed a bunch of patches, reviewed a bunch of bugs, fixed a
    bunch of bugs, merged many people's code, got 0.2.2.x closer to done.
  • Sebastian wrote a proposal for a safer voting process for consensus
    parameters, and wrote an implementation for it.
  • Damian started thinking about our various projects in a more streamlined
    and easy-to-understand way. The results are at


  • Christian cleaned up the rather hackish installation of Weather on bahri.
    The stable installation now lives under `/home/weather/opt/current' and actually
    is update-able through `git pull'. There's also a testing installation to test
    stuff and play around at He's tried to
    update the documentation with all the stuff that is necessary to install and run
  • Christian tried looking into #2467. Some people complained that Weather
    didn't know their relay fingerprint. On Sebastian's and Mike's idea, Christian
    changed the torrc to include `FetchDirInfoEarly 1' and `FetchUselessDescriptors
    1'. Since that no one complained again about Weather not knowing a certain relay
    (except for one time, when the Weather process had silently crashed and
    therefore the database wasn't updated for a day).
  • After Tor was tagged and made it to the recommended versions',
    people running started complaining about getting "Node out of date!"
    emails from Weather. It turned out that Weather was actually doing the right
    thing, namely mailing them that they were not running the latest recommended
    stable version anymore. No one seemed to have read the text near the checkbox in
    the signup process. After discussing this intensely with Sebastian, we decided
    to go for a more simple solution: People now get email when they don't run one
    of the recommended versions or a more recent dev version of Tor.

More reliable downloads

  • Christian did a rather large GetTor overhaul. The way GetTor manages its
    packages is now much easier to understand and enhance. GetTor moved from a
    ini-style configuration file and parser to a more BridgeDB-like configuration
    management. Also, packages are now configured rather than hard coded. In
    addition, he cleaned up the i18n management of GetTor to something similar to
    what we use in BridgeDB. Not only are the translation strings cleaner now, but
    the translation and installation is smoother. Also, the logging was simplified
    because it had too many features that no one used and generally was polluting
    the log file with too much useless information. Furthermore, the
    script that
    creates GetTor's package statistics was simplified a lot.
  • Christian fixed #1586, users requesting non-existent split packages now
    are informed about that fact.
  • Nick worked on a thandy packaging spec with Erinn.


  • Sebastian Started figuring out a way how translations can be pulled from
    transifex and used in their respective products in a more automated
  • New or updated website translations in French, Russian, Italian, Japanese,
    Spanish, Mandarin Chinese, and Greek.

Reading links for 11 March 2011

Tor is out

Tor lets relays record their bandwidth history so when
they restart they don't lose their bandwidth capacity estimate. This
release also fixes a diverse set of user-facing bugs, ranging from
relays overrunning their rate limiting to clients falsely warning about
clock skew to bridge descriptor leaks by our bridge directory authority.

Major bugfixes:

  • Stop sending a CLOCK_SKEW controller status event whenever
    we fetch directory information from a relay that has a wrong clock.
    Instead, only inform the controller when it's a trusted authority
    that claims our clock is wrong. Bugfix on; fixes
    the rest of bug 1074.
  • Fix an assert in parsing router descriptors containing IPv6
    addresses. This one took down the directory authorities when
    somebody tried some experimental code. Bugfix on
  • Make the bridge directory authority refuse to answer directory
    requests for "all" descriptors. It used to include bridge
    descriptors in its answer, which was a major information leak.
    Found by "piebeer". Bugfix on
  • If relays set RelayBandwidthBurst but not RelayBandwidthRate,
    Tor would ignore their RelayBandwidthBurst setting,
    potentially using more bandwidth than expected. Bugfix on Reported by Paul Wouters. Fixes bug 2470.
  • Ignore and warn if the user mistakenly sets "PublishServerDescriptor
    hidserv" in her torrc. The 'hidserv' argument never controlled
    publication of hidden service descriptors. Bugfix on

Major features:

  • Relays now save observed peak bandwidth throughput rates to their
    state file (along with total usage, which was already saved)
    so that they can determine their correct estimated bandwidth on
    restart. Resolves bug 1863, where Tor relays would reset their
    estimated bandwidth to 0 after restarting.
  • Directory authorities now take changes in router IP address and
    ORPort into account when determining router stability. Previously,
    if a router changed its IP or ORPort, the authorities would not
    treat it as having any downtime for the purposes of stability
    calculation, whereas clients would experience downtime since the
    change could take a while to propagate to them. Resolves issue 1035.
  • Enable Address Space Layout Randomization (ASLR) and Data Execution
    Prevention (DEP) by default on Windows to make it harder for
    attackers to exploit vulnerabilities. Patch from John Brooks.

Minor bugfixes (on 0.2.1.x and earlier):

  • Fix a rare crash bug that could occur when a client was configured
    with a large number of bridges. Fixes bug 2629; bugfix on Bugfix by trac user "shitlei".
  • Avoid a double mark-for-free warning when failing to attach a
    transparent proxy connection. Bugfix on Fixes
    bug 2279.
  • Correctly detect failure to allocate an OpenSSL BIO. Fixes bug 2378;
    found by "cypherpunks". This bug was introduced before the first
    Tor release, in svn commit r110.
  • Country codes aren't supported in EntryNodes until 0.2.3.x, so
    don't mention them in the manpage. Fixes bug 2450; issue
    spotted by keb and G-Lo.
  • Fix a bug in bandwidth history state parsing that could have been
    triggered if a future version of Tor ever changed the timing
    granularity at which bandwidth history is measured. Bugfix on
  • When a relay decides that its DNS is too broken for it to serve
    as an exit server, it advertised itself as a non-exit, but
    continued to act as an exit. This could create accidental
    partitioning opportunities for users. Instead, if a relay is
    going to advertise reject *:* as its exit policy, it should
    really act with exit policy "reject *:*". Fixes bug 2366.
    Bugfix on Tor Bugfix by user "postman" on trac.
  • In the special case where you configure a public exit relay as your
    bridge, Tor would be willing to use that exit relay as the last
    hop in your circuit as well. Now we fail that circuit instead.
    Bugfix on Fixes bug 2403. Reported by "piebeer".
  • Fix a bug with our locking implementation on Windows that couldn't
    correctly detect when a file was already locked. Fixes bug 2504,
    bugfix on
  • Fix IPv6-related connect() failures on some platforms (BSD, OS X).
    Bugfix on; fixes first part of bug 2660. Patch by
  • Set target port in get_interface_address6() correctly. Bugfix
    on and; fixes second part of bug 2660.
  • Directory authorities are now more robust to hops back in time
    when calculating router stability. Previously, if a run of uptime
    or downtime appeared to be negative, the calculation could give
    incorrect results. Bugfix on; noticed when fixing
    bug 1035.
  • Fix an assert that got triggered when using the TestingTorNetwork
    configuration option and then issuing a GETINFO config-text control
    command. Fixes bug 2250; bugfix on

Minor bugfixes (on 0.2.2.x):

  • Clients should not weight BadExit nodes as Exits in their node
    selection. Similarly, directory authorities should not count BadExit
    bandwidth as Exit bandwidth when computing bandwidth-weights.
    Bugfix on; fixes bug 2203.
  • Correctly clear our dir_read/dir_write history when there is an
    error parsing any bw history value from the state file. Bugfix on
  • Resolve a bug in verifying signatures of directory objects
    with digests longer than SHA1. Bugfix on
    Fixes bug 2409. Found by "piebeer".
  • Bridge authorities no longer crash on SIGHUP when they try to
    publish their relay descriptor to themselves. Fixes bug 2572. Bugfix

Minor features:

  • Log less aggressively about circuit timeout changes, and improve
    some other circuit timeout messages. Resolves bug 2004.
  • Log a little more clearly about the times at which we're no longer
    accepting new connections. Resolves bug 2181.
  • Reject attempts at the client side to open connections to private
    IP addresses (like,, and so on) with
    a randomly chosen exit node. Attempts to do so are always
    ill-defined, generally prevented by exit policies, and usually
    in error. This will also help to detect loops in transparent
    proxy configurations. You can disable this feature by setting
    "ClientRejectInternalAddresses 0" in your torrc.
  • Always treat failure to allocate an RSA key as an unrecoverable
    allocation error.
  • Update to the March 1 2011 Maxmind GeoLite Country database.

Minor features (log subsystem):

  • Add documentation for configuring logging at different severities in
    different log domains. We've had this feature since,
    but for some reason it never made it into the manpage. Fixes
    bug 2215.
  • Make it simpler to specify "All log domains except for A and B".
    Previously you needed to say "[*,~A,~B]". Now you can just say
  • Add a "LogMessageDomains 1" option to include the domains of log
    messages along with the messages. Without this, there's no way
    to use log domains without reading the source or doing a lot
    of guessing.

Packaging changes:

  • Stop shipping the Tor specs files and development proposal documents
    in the tarball. They are now in a separate git repository at

New Tor Browser Bundles

All of the Tor Browser Bundles have been updated with Firefox 3.6.15 and the alpha bundles for Mac OS X and Linux have also been updated with Tor

Windows bundles
1.3.20: Released 2011-03-07

  • Update Firefox to 3.6.15

Linux bundles
1.1.5: Released 2011-03-09

  • Update Tor to
  • Update Firefox to 3.6.15
  • Update NoScript to
  • Update HTTPS-Everywhere to 0.9.9.development.3

OS X bundle
1.0.13: Released 2011-03-09

  • Update Tor to
  • Update Firefox to 3.6.15, and use the Mozilla version until I get it to build on OS X 10.6 (Snow Leopard)
  • Update NoScript to
  • Update HTTPS-Everywhere to 0.9.9.development.3

Tor is released

Tor fixes a variety of less critical bugs. The main other change is a slight tweak to Tor's TLS handshake that makes relays and bridges that run this new version reachable from Iran again. We don't expect this tweak will win the arms race long-term, but it buys us time until we roll out a better solution.

Major bugfixes:

  • Stop sending a CLOCK_SKEW controller status event whenever
    we fetch directory information from a relay that has a wrong clock.
    Instead, only inform the controller when it's a trusted authority
    that claims our clock is wrong. Bugfix on; fixes
    the rest of bug 1074.
  • Fix a bounds-checking error that could allow an attacker to
    remotely crash a directory authority. Bugfix on
    Found by "piebeer".
  • If relays set RelayBandwidthBurst but not RelayBandwidthRate,
    Tor would ignore their RelayBandwidthBurst setting,
    potentially using more bandwidth than expected. Bugfix on Reported by Paul Wouters. Fixes bug 2470.
  • Ignore and warn if the user mistakenly sets "PublishServerDescriptor
    hidserv" in her torrc. The 'hidserv' argument never controlled
    publication of hidden service descriptors. Bugfix on

Minor features:

  • Adjust our TLS Diffie-Hellman parameters to match those used by
    Apache's mod_ssl.
  • Update to the February 1 2011 Maxmind GeoLite Country database.

Minor bugfixes:

  • Check for and reject overly long directory certificates and
    directory tokens before they have a chance to hit any assertions.
    Bugfix on Found by "doorss".
  • Bring the logic that gathers routerinfos and assesses the
    acceptability of circuits into line. This prevents a Tor OP from
    getting locked in a cycle of choosing its local OR as an exit for a
    path (due to a .exit request) and then rejecting the circuit because
    its OR is not listed yet. It also prevents Tor clients from using an
    OR running in the same instance as an exit (due to a .exit request)
    if the OR does not meet the same requirements expected of an OR
    running elsewhere. Fixes bug 1859; bugfix on

Packaging changes:

  • Stop shipping the Tor specs files and development proposal documents
    in the tarball. They are now in a separate git repository at
  • Do not include Git version tags as though they are SVN tags when
    generating a tarball from inside a repository that has switched
    between branches. Bugfix on; fixes bug 2402.

London Internet Security & Privacy Workshop

We're helping to hold a mini "unconference" on Internet security and privacy in London. It is Monday night (March 7) from 6 to 9 PM at the Main College Building, Room 116, SOAS just off Russell Square. The goal is to create an environment where the attendees can share information and learn about security and privacy on the Internet. All attendees are encouraged to present or facilitate a session. If you're in the area and want to learn or share, please stop by!

Updated 11 PM GMT with details and location.

Hackfest Thanks

Thank you to all who showed up for the hackfest at MIT on Saturday the 19th. Roughly 50 people attended the event at some point throughout the day. People traveled from the local area, Maine, New York, Connecticut, and one person rearranged their flights from California to hack with us. The free pizza, drinks, and donuts were provided to all thanks to some generous attendees.

And a final thank you to the Center for Future Civic Media who once again offered the facilities and support for our hackfest.

Now that you've met us, are interested in helping the world, and want to learn more, here are some ideas on getting involved:


five minutes to speak

I was asked to give a five minute speech to open a panel in front of a number of policy makers and advisors in Washington, DC in the past few weeks. The discussion was under Chatham House Rule. A number of people have encouraged me to publish the speech notes as a blog post, it is as follows.

Here I am, a technologist in a room full of policy people. I'll stick
to what I know and try not to put anyone to sleep in the next five

Technology is agnostic, who uses and how they use it matters. Roads,
cars, phones, email, websites are all technologies used for good and bad.
In the 1930s, the feds and police warned of mass chaos if the interstate
highway system was built in the US. The ability for criminals to quickly
transit between cities was of grave concern. Crime would spread faster,
further, and this would hasten the breakdown of the very fabric of the
American society, community. Time has shown the benefits vastly outweighed
the costs. This same principle has shown to be true of the internet
and computer technology. Sure, we may have new kinds of crime with botnets,
zombies, phishing, but do we really? Lying, impersonation, and tricking
someone into doing your work are the same crimes they have been for the
past few millenia. It's just that the substrate that is used, has changed.
What are some of the largest companies in the world? GE, IBM, Apple,
Microsoft, Google. What one should or should not do is policy and law,
what one can actually do or not do is technology.

Circumvention, anonymity, and privacy tools used in a free world can be
a minor annoyance, i.e. wikileaks used wikis, ssl, email, and yes, tor,
but in the end, it's an annoyance. We don't have people in the streets
rioting trying to overthrow our govt. Wikipedia uses the same technology
in wikis, ssl, and email. Everyone loves Wikipedia and considers it a net positive.

The same circumvention, anonymity, and privacy tools are deadly to
repressive regimes. The free flow of information and education are of
great concern to a regime trying to control the horizontal and vertical
of every day life. The tactics a regime can use are legal, technical,
and physical. The regime can switch between tactics, generally
depending upon what's economical and most effective.

Roughly 1 billion people are online in some way. Berkman did a study
that found roughly 2% of that billion know what a proxy is, or even that
technology exists to circumvent internet censorship. 98% of the world
accepts that facebook, google, cnn, and the bbc, are blocked and doesn't
try to find ways around it. This doesn't even broach the topic of online
privacy relating to commercial entities nor law enforcement and
intelligence agencies trying to learn the who, what, where, and how of your Internet activities.

Arguing about which proxy technology should get all of the funding and
attention is silly. The budgets and adversaries vastly outweigh the
funding and research into proxies. It's not a zero-sum game, and
the different technologies take very different approaches to success;
freegate/ultrasurf, vpns, psiphon, and web proxies play a game of cat
and mouse with ip addresses and sometimes encryption; tor uses the
strategy of R&D and protecting ones anonymity and privacy first, the
secondary effects of this are well-suited to circumvention too. Tor,
freegate, psiphon, and vpns sum up to roughly $50m in funding from the US govt
of the past few years. Only a very small fraction of that total has made it to actual technology. Compare that to the billions spent on snakeoil
black box technology by the DoD and intelligence agencies preparing for
a cyberwar arms race, much like the nuclear arms race, to deter other
nations from attacking us.

I talked to a member of a terrorist organization in Vietnam. He's been
stalked, harassed, and had everything confiscated multiple times by his
government. You know his organization as Deutsche Welle. He's a
reporter. He had no idea how his plans, documents, and contacts were
being discovered and used against him. His ability to understand the
differences between Tor, JAP, and Freegate was like asking which tires
are best for gravel, snow, or tarmac. The question he didn't even know
to ask is, "What are safe and secure computing and online practices?"
to use my analogy, "what car do I want for those tires? the answer is
a rally car." I spent 4 hours going over how the internet works,
how to think about adversaries online, what is ssl, what it means, what
are phishing, viruses, botnets, and state-sponsored malware. By the
end of the 4th hour, he understood how tor is different than a simple
vpn or proxy server, and when to use tor and when it isn't needed. 3.5h
of that discussion was basic operational, computer, and online security
and safe practices.

So where does this leave us? It leaves us with a mix of education,
technology, and many, many unanswered questions. This is a young field
overall. As the censorship providers and technologies get better, so
will those circumvention technologies. Educating users about internet
safety, risks, and making the tools vastly easier and safer to use
should be a goal.

Tor published a "10 things to think about circumvention tools" paper to
try to distill what we've learned over the past 10 years of doing this.
In a few of these areas, tor is not the best choice, for now.

What about technology? Isn't it going to save us all? Currently,
freegate/ultrasurf, vpns, and web proxies are looking for money to fund
their growing infrastructure costs. The more users you have, the more
servers, more bandwidth, and more costs you incur. Its a quick way to
spend lots of money and get lots of users. However becoming the ISP for
the world gets very expensive, very quickly as you scale up to hundreds
of millions of users. Look at the infrastructures of google, facebook,
yahoo, and microsoft to see the challenges that lie ahead for these tools.

Tor and "distributed tools" look to improve the research and development
and rely on the scaling of users to both provide the circumvention and
grow to become a self-sustaining ISP to the world. We have only begun
to see the power and effects of these technologies with bittorrent, jap,
skype, freenet, i2p, and tor.

Syndicate content