Tor Weekly News — January 7th, 2015

Welcome to the first issue in 2015 of Tor Weekly News, the weekly newsletter that covers what’s happening in the Tor community.

Tor is out

Nick Mathewson announced the second alpha release in the Tor 0.2.6.x series. As well as including the cell scheduling changes and hidden service statistics collection reported in recent issues of TWN, this release makes it harder to portscan hidden services by closing circuits if a client tries to connect to a non-existent port. It also contains numerous bugfixes and new unit tests; please see Nick’s announcement for the full changelog. The source code is available as usual from the distribution directory.

Tor at 31c3

The 31st edition of the Chaos Communication Congress, an annual highlight in the free software and security calendar, took place in Hamburg, and as usual Tor featured in several key talks over the course of the long weekend.

Roger Dingledine and Jacob Appelbaum’s appropriately grand-sounding “State of the Onion” address, a round-up of the year’s events in the Tor community, took place once again, with guest contributions from journalist and filmmaker Laura Poitras and OONI developer Arturo Filastò. Topics included the relationship between censorship and surveillance, the misinterpretation of academic research by journalists, new pluggable transports, and much more.

Laura Poitras also joined Julia Angwin, Jack Gillum, and Nadia Heninger for “Crypto Tales from the Trenches”, in which the journalists described their experiences with security software when doing research and communicating with sources. “I don’t think any of us could do our work without Tor”, said Laura, while Julia described the Tails operating system as “her favorite success story” in this field.

Tor Browser developer Mike Perry joined Seth Schoen to discuss the concept of deterministic builds, the implementation of which has been one of the Tor Project’s major successes over the past year. Mike and Seth demonstrated some of the attacks that this system aims to defend against, as well as the work that Tor, F-Droid, and Debian have all been doing to make their processes compatible with the deterministic build process.

Finally, Dr. Gareth Owen of Portsmouth University presented the results of research into the content and usage of Tor hidden services. The research involved setting up a number of Tor relays, waiting until they gained the “HSDir” flag, then counting the number of times a particular service’s descriptor was requested, as well as manually categorizing the services whose descriptors were learned. Dr. Owen found that while the largest category of onion services by number could be characterized as “drugs”, the majority of the descriptor requests he saw were for services in his “abuse” category. The talk itself discusses some possible limitations of the data gathered, and Tor developers have responded on the Tor blog with further analysis.

Monthly status reports for December 2014

The wave of regular monthly reports from Tor project members for the month of December has begun. Philipp Winter released his report first, followed by reports from Damian Johnson, Pearl Crescent, Juha Nurmi, Nick Mathewson, Sherief Alaa, Sukhbir Singh, Leiah Jansen, David Goulet, Michael Schloh von Bennewitz, Colin C., Georg Koppen, Arlo Breault, and George Kadianakis.

Colin C. also sent out the help desk report, while Arturo Filastò reported on behalf of the OONI team and Mike Perry for the Tor Browser team.

Miscellaneous news

Nick Mathewson and Andrea Shepard drafted a proposal for including a hash chain in the consensus produced by Tor directory authorities, in order to prevent certain kinds of attack on the directory authorities and their keys.

Nick also clarified that a recently-discovered Libevent vulnerability has no effect on Tor.

In connection with the current push to collect statistics relating to Tor hidden services in a privacy-preserving manner, Aaron Johnson noted that there are two further desirable sets of statistics which might pose a risk to anonymity if gathered incorrectly, and discussed possible solutions to the problem.

David Fifield published a summary of costs incurred by the meek pluggable transport for the month of December 2014.

David also continued his experiments on historical Tor metrics data with visualizations of a recent Sybil attack, and wondered what might have been responsible for a sudden change in the way that users in Kazakhstan were choosing to connect to the Tor network in October 2014.

Sebastian Urbach noted a sudden drop in the number of Tor relays acting as hidden service directories, and wondered about the cause. As SiNA Rabbani clarified, the amount of time a relay needs to have been running before it earns the “HSDir” flag was increased by directory authorities, in response to a recent Sybil attack.

The developers of ChatSecure for iOS announced that their recent 3.0 release includes experimental support for connections to XMPP chat servers over Tor, and briefly described how they added the new feature.

This issue of Tor Weekly News has been assembled by Harmony, David Fifield, Catfish, and Karsten Loesing.

Want to continue reading TWN? Please help us create this newsletter. We still need more volunteers to watch the Tor community and report important news. Please see the project page, write down your name and subscribe to the team mailing list if you want to get involved!

Tor is released

Tor is the second alpha release in the 0.2.6.x series. It introduces a major new backend for deciding when to send cells on channels, which should lead down the road to big performance increases. It contains security and statistics features for better work on hidden services, and numerous bugfixes.

This release contains many new unit tests, along with major performance improvements for running testing networks using Chutney. Thanks to a series of patches contributed by "teor", testing networks should now bootstrap in seconds, rather than minutes.

You can download the source from the usual place on the website. Packages should be up in a few days.

NOTE: This is an alpha release. Please expect bugs.

Changes in version - 2014-12-31

  • Major features (relay, infrastructure):
    • Complete revision of the code that relays use to decide which cell to send next. Formerly, we selected the best circuit to write on each channel, but we didn't select among channels in any sophisticated way. Now, we choose the best circuits globally from among those whose channels are ready to deliver traffic.

      This patch implements a new inter-cmux comparison API, a global high/low watermark mechanism and a global scheduler loop for transmission prioritization across all channels as well as among circuits on one channel. This schedule is currently tuned to (tolerantly) avoid making changes in network performance, but it should form the basis for major circuit performance increases in the future. Code by Andrea; tuning by Rob Jansen; implements ticket 9262.

  • Major features (hidden services):
    • Make HS port scanning more difficult by immediately closing the circuit when a user attempts to connect to a nonexistent port. Closes ticket 13667.
    • Add a HiddenServiceStatistics option that allows Tor relays to gather and publish statistics about the overall size and volume of hidden service usage. Specifically, when this option is turned on, an HSDir will publish an approximate number of hidden services that have published descriptors to it the past 24 hours. Also, if a relay has acted as a hidden service rendezvous point, it will publish the approximate amount of rendezvous cells it has relayed the past 24 hours. The statistics themselves are obfuscated so that the exact values cannot be derived. For more details see proposal 238, "Better hidden service stats from Tor relays". This feature is currently disabled by default. Implements feature 13192.

  read more »

Tor Weekly News — December 31st, 2014

Welcome to the final issue in 2014 of Tor Weekly News, the weekly newsletter that covers what’s happening in the Tor community.

Attacks and rumors of attacks

Two weeks ago, the Tor Project relayed a warning from an unspecified source to the effect that someone may have been preparing to seize, attack, or otherwise disable one or more of Tor’s directory authorities in a bid to disrupt the entire Tor network. The lack of any specific information about the threat caused understandable concern in the Tor community, and several events that followed over the next fortnight did little to dispel this.

First, the operator of a large Tor exit relay cluster reported that his servers may have been physically interfered with by unknown parties a short while before his message. Later updates suggested that foul play was less likely than initially thought.

Several days later, a large number of small exit relays were created all at once, in what appeared to be a “Sybil attack”; this was detected and halted almost immediately, as was a second, more recent incident. As the Tor Project put it in a response, “we don’t expect any anonymity or performance effects based on what we've seen so far”, although a side-effect of the countermeasure is that relays hosted on some IP ranges are currently being rejected by dirauths.

As far as anyone can tell, these events are not related in any way to the initial warning. The Tor network has functioned normally throughout this period, and the appearance of a series of incidents is likely to be the result of coincidence (helped by the online rumor mill) rather than a coordinated campaign. It is never possible to say with certainty that attacks on the network will not occur, but the threat referred to in the original blog post has not yet materialized — and “no news is good news”.

Miscellaneous news

Lasse Øverlier discovered that ScrambleSuit’s protection against “replay attacks”, in which an adversary repeats a client authentication event to learn that the server is in fact a ScrambleSuit bridge, doesn’t work. Philipp Winter explained the issue, and suggested some simple fixes.

Tom van der Woerdt asked for review of a patch to remove the obsolete version 1 of Tor’s link protocol from the current software: “It’s a rather large patch, though not as large as the patch that will remove v2 of the protocol. However, before I write that one, can someone please check whether my patch is sane and I’m not violating any standards or policies?”

David Fifield trimmed the length of meek’s HTTP headers from 413 to 162 bytes, reducing the bandwidth it uses by “approximately” 3%.

Thanks to Kura for running a mirror of the Tor Project website and software archive!

This issue of Tor Weekly News has been assembled by Harmony, David Fifield, Chuck Peters, and Roger Dingledine.

Want to continue reading TWN? Please help us create this newsletter. We still need more volunteers to watch the Tor community and report important news. Please see the project page, write down your name and subscribe to the team mailing list if you want to get involved!

The Tor talk at 31c3

And on a more cheerful note, I heard that Jake and Roger gave a really excellent talk about Tor at this year's CCC. And the CCC people have thoughtfully put it online! If you didn't see it, you might want to check it out. The talking begins at around 15:16, and the introduction is well worth watching too.

I'm also going to spend a while looking at the other presentations too. Right now the one at the top of the page is an ECC talk by Daniel Bernstein and Tanja Lange; I'm looking forward to having a little time to watch it, and a bunch of others I see there.

Happy new year, everyone!

update (1/1/15): "State of the Onion" is available for download now. You can find torrent and direct links in various formats on C3TV website.

Video: MP4 HD (torrent) - WEBM (torrent) - MP4 (torrent)
Audio: MP3 (torrent) - OPUS (torrent)

Some thoughts on Hidden Services

Hi! Nick here.

I ought to post my own responses to that Andy Greenberg article, too. (Especially since most everybody else around here is at 31c3 right now, or sick with the flu, or both.)

When I saw the coverage of the hidden services study that was presented at CCC today, I was reminded of the media fallout from that old study from the 1990s that "proved" that a ridiculously high fraction of the internet was looking at Usenet*, and by counting newsgroups and bytes. (You might remember it; it was the basis of the delightful TIME Magazine "Cyberporn" cover.)

The 1990s researcher wasn't lying outright, but he and the press *were* conflating one question: "What fraction of Usenet groups are '' or 'alt.binaries' (file posting) groups" with two others: "What fraction of internet traffic is porn?" and "What fraction of internet-user hours are spent on porn?"

These are quite different things.

The presentation today focused on data about hidden service types and usage. Predictably, given the results from Biryukov, Pustogarov, Thill, and Weinmann, the researcher found that hidden services related to child abuse are only a small fraction of the total number of hidden service addresses on the network. And because of the way that hidden services work, traffic does not go through hidden service directories, but instead through rendezvous points (randomly chosen Tor nodes): so no relay that knows the hidden service's address will learn the actual amount of traffic transmitted. But, as previously documented, abusive services represent a disproportionate fraction of usage... if you're measuring usage with hidden service directory requests.

Why might that be?

First, some background. Basically, a Tor client makes a hidden service directory request the first time it visits a hidden service that it has not been to in a while. (If you spend hours at one hidden service, you make about 1 hidden service directory request. But if you spend 1 second each at 100 hidden services, you make about 100 requests.) Therefore, obsessive users who visit many sites in a session account for many more of the requests that this study measures than users who visit a smaller number of sites with equal frequency.

There are other confounding factors as well. Due to bugs in older Tor implementations, a hidden service that is unreliable (or completely unavailable) will get many, many more hidden service directory requests than a reliable one. So if any abuse sites are unusually unreliable, we'd expect their users to create a disproportionately large number of hidden service directory requests.

Also, a very large number of hidden service directory requests are probably not made by humans! See bug 13287: We don't know what's up with that. Could this be caused by some kind of anti-abuse organization running an automated scanning tool?

In any case, a methodology that looks primarily at hidden service directory requests will over-rate services that are frequently accessed from a Tor client that hasn't been there recently, and under-rate services that are used via tor2web, and so on. It also depends a lot on how hidden services are configured, how frequently Tor hidden service directories go up and down, and what times of day they change introduction points in comparison to what time of day their users tend to be awake.

The greater the number of distinct hidden services a person visits, and the less reliable those sites are, the more hidden service directory requests they will trigger.

Suppose 10 people use hidden services to look at conspiracy theories, 100 people use hidden services to buy Cuban cigars, and 1000 people use it for online chat.

But suppose that the average cigar purchaser visits only one or two sites to make purchases, and the average chat user joins one or two networks, whereas the average conspiracy theorist needs to visit several dozen forums and wikis.

Suppose also that the average Cuban cigar purchaser makes about two purchases a month, the average chat user logs in once a day, and the average conspiracy theorist spends 3 hours a day crawling the hidden web.

And suppose that conspiracy theory websites come and go frequently, whereas cigar sites and chat networks are more stable.

In this analysis, even though there are far more people buying cigars, users who use it for obsessive behavior that spans multiple unreliable hidden services will be far overrepresented in the count of hidden service directory requests than users who use it for activities done less frequently and across fewer services. So any comparison of hidden service directory request counts will say more about the behavioral differences of different types of users than about their relative numbers, or the amount of traffic they generated.

In conclusion, let's spend a minute talking about freedom and philosophy. Any system that provides security on the Internet will inevitably see some use by bad people that we'd rather not help at all. After all, cars are used for getaways, and window shades conceal all kinds of criminality. The only way to make a privacy tool that nobody abuses is to make it so weak that people aren't willing to touch it, or so unusable that nobody can figure it out.

Up till now, many of the early adopters for Tor hidden services have been folks for whom the risk/effort calculations have been quite extreme, since--as I'd certainly acknowledge--the system isn't terribly usable for the average person as it stands. Roger noted earlier that hidden services amount to less than 2% of our total traffic today. Given their privacy potential, I think that's not even close to enough. We've got to work over the next year or more to develop hidden services to the point where their positive impact is felt by the average netizen, whether they're publishing a personal blog for their friends, using a novel communications protocol more secure than email, or reading a news article based on information that a journalist received through an anonymous submission system. Otherwise, they'll remain a target for every kind of speculation, and every misunderstanding about them will lead people to conclude the worst about privacy online. Come lend a hand?

(Also, no offense to Andy on this: he is a fine tech reporter and apparently a fine person. And no offense to Dr. Owen, who explained his results a lot more carefully than they have been re-explained elsewhere. Now please forgive me, I'm off to write some more software and get some sleep. Please direct all media inquiries to the email of "press at torproject dot org".)

* Usenet was sort of like Twitter, only you could write paragraphs on it. ;)

Tor: 80 percent of ??? percent of 1-2 percent abusive.

Hi, Nick here.

Roger's at 31c3, so I'll post his statement about that article you might have seen:

Tor hidden service traffic, which Dr. Gareth Owen discussed in his talk this afternooon, is only 1.5% of all Tor traffic. Tor gets about 2 million users per day total.
The researcher ran a set of Tor relays for a six month period, and recorded how many times somebody attempted to look up a hidden service (this lookup is one of the steps in visiting a hidden service). Then at the end of that period, he scanned the hidden services he'd learned about, to find out what sort of content was on them.

Dr. Owen's data shows that there's a lot of churn in hidden services, so nearly all of the sites were gone by the time he did these scans. His graphs only show data about the sites that were still up many months later: so his data could either show a lot of people visiting abuse-related hidden services, or it could simply show that abuse-related hidden services are more long-lived than others. We can't tell from the data.

Without knowing how many sites disappeared before he got around to looking at them, it's impossible to know what percentage of fetches went to abuse sites.

There are important uses for hidden services, such as when human rights activists use them to access Facebook or to blog anonymously. These uses for hidden services are new and have great potential.

PS: Law enforcement agencies use Tor to stay anonymous while they catch bad guys. Law enforcement agencies use and run hidden services, too.

More info to follow.

Tor Weekly News — December 24th, 2014

Welcome to the fifty-first issue in 2014 of Tor Weekly News, the weekly newsletter that covers what’s happening in the Tor community.

Stem 1.3 is out

“After months down in the engine room”, Damian Johnson announced version 1.3 of Stem, the Tor controller library written in Python. Among the many improvements in this release, Damian singled out the new set of controller methods for working with hidden services, as well as the 40% increase in descriptor parsing speed.

Please see the changelog for full details of all the new features.

Miscellaneous news

The team of researchers working on the collection of hidden service statistics asked relay operators for help by enabling these statistics on their relays in the coming days and weeks. They included a step-by-step tutorial for enabling this feature, which has recently been merged into Tor’s main branch.

Building on Andrea Shepard’s recently-merged work on global cell scheduling, Nick Mathewson announced that the KIST socket management algorithm proposed earlier this year to reduce congestion in the Tor network is now “somewhat implemented” for Linux. You can follow the testing and reviews on the associated ticket.

Nick also asked for feedback on the proposal to increase the interval at which Tor relays report their bandwidth usage statistics from fifteen minutes to four hours : “Will this break anything you know about?”

Moritz Bartl invited Tor relay operators to a meet-up at the upcoming Chaos Communication Congress in Hamburg: “We will do quick presentations on recent and future activities around, talk about events relevant to the Tor relay community, and what lies ahead.”

Thanks to Thomas White for keeping the community updated following a brief period of suspicious activity around his exit relays and Onionoo application mirrors!

This week in Tor history

A year ago this week, Tor Browser hit version 3.5, bringing with it a pioneering deterministic build system that set a new standard in software distribution security, and has since drawn interest from many other projects, including the Debian operating system. It also laid the long-obsolete Vidalia graphical controller to rest, replacing it with the faster, sleeker Tor Launcher. The privacy-preserving browser is now approaching version 4.5, and users can look forward to a security slider offering finer-grained tuning of security preferences, as well as features that restore some of Vidalia’s circuit-visualization capabilities.

This issue of Tor Weekly News has been assembled by Harmony and Karsten Loesing.

Want to continue reading TWN? Please help us create this newsletter. We still need more volunteers to watch the Tor community and report important news. Please see the project page, write down your name and subscribe to the team mailing list if you want to get involved!

Stem Release 1.3


Greetings wonderful people of the world! After months down in the engine room I'm delighted to announce the 1.3.0 release of Stem.

For those who aren't familiar with it, Stem is a Python library for interacting with Tor. With it you can script against your relay, descriptor data, or even write applications similar to arm and Vidalia.

So what's new in this release?

Better Hidden Service Support

Now it's easier than ever to spin up hidden services!

Thanks to contributions from Federico Ceratto and Patrick O'Doherty we now have a set of methods specifically for working with hidden services. Check it out in our new tutorial...

Over the River and Through the Wood

Faster Descriptor Parsing

This release dramatically improves the speed at which Stem can parse decriptors. Thanks to optimizations from Nick Mathewson and Ossi Herrala we can now read descriptors 40% faster!

This is just the tip of the iceberg. For a full rundown on the myriad of improvements and fixes in this release see...

Cheers! -Damian

Syndicate content Syndicate content