hidden services

Cooking With Onions: Names for your onions





Hello again,

this blog post is the second issue of the Cooking with Onions series which aims to highlight interesting aspects of the onion space. Check-out our first issue as well!

Onion addresses are weird...

This post is about onion addresses being weird and the approaches that can be taken to improve onion service usability.

In particular, if you've cruised around the onionspace, you must have noticed that onion services typically have random-looking addresses that look like these:

  • 3g2upl4pq6kufc4m.onion
  • 33y6fjyhs3phzfjj.onion
  • propub3r6espa33w.onion

So for example, if you wanted to visit the Tor website onion service, you would have to use the address http://expyuzz4wqqyqhjn.onion/ instead of the usual https://www.torproject.org.

To better understand why onion addresses are so strange, it helps to remember that onion services don't use the insecure Domain Name System (DNS), which means there is no organization like ICANN to oversee a single root registry of onion addresses or to handle ownership dispute resolution of onion addresses. Instead, onion services get strong authentication from using self-authenticating addresses: the address itself is a cryptographic proof of the identity of the onion service. When a client visits an onion service, Tor verifies its identity by using the address as ground truth.

In other words, onion services have such absurd names because of all the cryptography that's used to protect them. Cryptographic material are basically huge numbers that look meaningless to most humans, and that's the reason onion addresses tend to look random as well.

To motivate this subject further, Tor developers have medium-term future plans for upgrading the cryptography of onion services, which has the side-effect of increasing onion address length to 54 characters! This means that in the future onion addresses will look like this:

  • llamanymityx4fi3l6x2gyzmtmgxjyqyorj9qsb5r543izcwymlead.onion
  • lfels7g3rbceenuuqmpsz45z3lswakqf56n5i3bvqhc22d5rrszzwd.onion
  • odmmeotgcfx65l5hn6ejkaruvai222vs7o7tmtllszqk5xbysolfdd.onion

Remembering onions

Over the years the Tor community has come up with various ways of handling these large and non-human-memorable onion addresses. Some people memorize them entirely or scribe them into secret notebooks, others use tattoos, third-party centralized directories or just google them everytime. We've heard of people using decks of cards to remember their favorite onion sites, and others who memorize them using the position of stars and the moon.

We believe that the UX problem of onion addresses is not actually solved with the above ad-hoc solutions and remains a critical usability barrier that prevents onion services from being used by a wider audience.

The onion world never had a system like DNS. Even though we are well aware that DNS is far from the perfect solution, it's clear that human memorable domain names play a fundamental role in the user experience of the Internet.

In this blog post we present you a few techniques that we have devised to improve the usability of onion addresses. All of these ideas are experimental and come with various fun open questions, so we are still in exploration mode. We appreciate any help in prototyping, analyzing and finding flaws in these ideas.






Idea 1) A modular name system API for Tor onion services


During the past years, many research groups have experimented and designed various secure name systems (e.g. GNS, Namecoin, Blockstack). Each of these systems has its own strengths and weaknesses, as well as different user models and total user experience. We are not sure which one works best for the onion space, so ideally we'd like to try them all and let the community and the sands of time decide for us. We believe that by integrating these experimental systems into Tor, we can greatly strengthen and improve the whole scientific field by exposing name systems to the real world and an active and demanding userbase.

For this reason and based on our experience with modular anti-censorship techniques, we designed a generic & modular scheme through which any name system can be integrated to Tor: Proposal 279 defines A Name System API for Tor Onion Services which can be used to integrate any complex name system (e.g. Namecoin) or even simple silly naming schemes (e.g. a local /etc/tor-hosts file).

Here is a graphical depiction of the Name System API with a Namecoin module enabled and resolving the domain sailing.tor for a user:


It's worth pointing out that proposal 279 is in draft status and we still need to incorporate feedback received in the mailing list. Furthermore, people have pointed out simple ways through which we can fast-track and prototype the proposal faster. Help in implementing this proposal is greatly appreciated (find us in IRC!).


Idea 2) Using browser extensions to improve usability


Other approaches for improving the usability of onion addresses use the Tor Browser as a framework: think of browser extensions that map human memorable names to onion addresses.

There are many variants here so let's walk through them:

Idea 2.1) Browser Extension + New pseudo-tld + Local onion registry


A browser extension like HTTPS-everywhere, uses an onion registry to map human-memorable addresses from a new pseudo-tld (e.g. ".tor") to onion addresses. For example, it maps "watchtower.tor" to "fixurqfuekpsiqaf.onion" and "globaleconomy.tor" to "froqh6bdgoda6yiz.onion". Such an onion registry could be local (like HTTPS-everywhere) or remote (e.g. a trusted append-only database).

Even an extension with a local onion registry would be a very effective improvement to the current situation since it would be pretty usable and its security model is easy to understand: an audited local database seems to work well for HTTPS-everywhere. However, there are social issues here: how would the onion registry be operated and how should name registrations be handled? I can see people fighting for who will get bitcoin.tor first. That said, this idea can be beneficial even with a small onion database (e.g. 50 popular domains).

Here is a graphical depiction of a browser extension with a local onion registry resolving the domain sailing.tor for a user:





Idea 2.2) Browser extension + New pseudo-tld + Remote onion registries


A more dynamic alternative here involves multiple trusted remote onion registries that the user can add to their torrc. Imagine a web-of-trust based system where you add your friend's Alice onion registry and then you can visit facebook using facebook.alice.onion.

A similar more decentralized alternative could be a browser addon that uses multiple remote onion registries/notaries to resolve a name, employing a majority or supermajority rule to decide the resolution results. Such a system could involve notary nodes similar to SSL schemes like Convergence.


Idea 2.3) Browser extension redirects existing DNS names


An easier but less effective approach would be for the browser extension to only map DNS domain names to onion names. So for example, it would map "duckduckgo.com" to "3g2upl4pq6kufc4m.onion". That makes the job of the name registrar easier, but it also heavily restricts users only to services with a registered DNS domain name. Some attempts have already been made in this area but unfortunately they never really took off.


Idea 2.4) Automatic Redirection using HTTP


The Alt-Svc HTTP header defines a way for a website to say "I'm facebook.com but you should talk to me using fbcdn.com." If we replace that fbcdn.com address with facebookcorewwi.onion - then when you typed in Facebook, the browser would, under the covers, use the .onion address. And this can be done without any browser extension whatsoever.

One problem is that the browser has to remember this mapping, and in Tor Browser that mapping could be used to track or correlate you. Preloading the mapping would solve this, but how to preload the mapping probably brings us back into the realm of a browser extension.


Idea 2.5) Smart browser bookmarks for onion addresses


Talking about random addresses, it's funny how people seem to be pretty happy handling phone numbers (big meaningless random numbers) using a phone book and contacts on their devices.

On the same note, an easier but less usable approach would be to enhance Tor Browser with some sort of smart bookmark/petname system which allows users to register custom names for onion sites, and allows them to trust them or share them with friends. Unfortunately, it' unclear whether the user experience of this feature would make it useful to anyone but power users.

Of course it's important to realize that any approach that relies on a browser extension will only work for the web, and you wouldn't be able to use it for arbitrary TCP services (e.g. visiting an IRC server)


Idea 3) Embed onion addresses in SSL certificates


So let's shift back to non-browser approaches!

Let's Encrypt is an innovative project which issues free SSL certificates in an automated fashion. It has greatly improved Internet security since now anyone can freely acquire an SSL certificate for their service and provide link security to their users.

Now let's imagine that Let's Encrypt embedded onion address information into the certificates it issues, for clients with both a normal service and an onion service. For example, the onion address could be embedded into a custom certificate extension or in the C/ST/L/O fields. Then Tor Browser, when visiting such an SSL-enabled website, would parse and validate the certificate and if an onion address is included, the browser would automagically redirect the user. Take a look at this paper for some more neat ideas on this area.


Idea 4) Embed onion addresses in DNS/DNSSEC records


A similar approach could use the DNS system instead of the SSL CA system. For example, site owners could add their onion address into their TXT or SRV DNS records and Tor could learn to redirect users to the onion address. Of course this approach only applies to operators that can afford a DNS domain. Oh yeah DNS also has zero security...

Conclusion

As you can see there are many approaches that we should explore to improve usability in this area. Each of them comes with its own tradeoffs and applies to different users, so it's important that we allow users to experiment with various systems and let each community decide which approach works best for them.

It's also worth pointing out that some of these approaches are not that hard to implement technically, but they still require lots of effort and community building to really take off and become effective. Involving and pairing with other friendly Internet privacy organizations is essential to achieve our goals.

Furthermore, we should think carefully of unintended usability and security consequences that come with using these systems. For example, people are not used to their browser automagically redirecting them from one domain to another: this can seriously freak people out. It's also not clear how Tor Browser should handle these special names to avoid SSL certificate verification issues and hostname leaks.

One thing is for sure: even though onion services are used daily by thousand of people, the random addresses confuse casual users and prevent the ecosystem from maturing and achieving widespread adoption. We hope that this blog post inspires researchers and developers to toy around with naming systems and take the initiative in building and experimenting with the various approaches. Please join the [tor-dev] mailing list and share your thoughts and projects with us!




And this brings us to the end of this post. Hope you enjoyed this issue of Cooking With Onions! We will be back soon, always with the finest produce and the greatest cooking tips! What would you like us to cook next?

[Thanks to Philipp Winter and Tom Ritter for the feedback on this blog post, as well as to everyone who has discussed and helped develop these ideas.]

What's new in Tor 0.2.9.8?

Today, we've released the first stable version of the 0.2.9.x series, bringing exciting new features to Tor. The series has seen 1406 commits from 32 different contributors. Please, see the ChangeLog for more details about what has been done.

This post will outline three features (among many other things) that we are quite proud of and want to describe in more detail.

Single Onion Service

Over the past several years, we've collaborated with many large scale service providers such as Facebook and Riseup, organizations that deployed Onion Services to improve their performance.

Onion services are great because they offer both anonymity on the service and the client side. However, there are cases where the onion service does not require anonymity. The main example of this is when the service provider does not need to hide the location of its servers.

As a reminder to the reader, an onion service connection between a client and a service goes through 6 hops, while a regular connection with Tor is 3 hops. Onion services are much slower than regular Tor connections because of this.

Today, we are introducing Single Onion Services! With this new feature, a service can now specify in its configuration file that it does not need anonymity, thus cutting the 3 hops between the service and its Rendezvous Point and speeding up the connection.

For security reasons, if this option is enabled, only single onion service can be configured. They can't coexist with a regular onion service. Because this removes the anonymity aspect of the service, we took extra precautions so that it's very difficult to enable a single onion by mistake. In your torrc file, here is how you do it:


HiddenServiceNonAnonymousMode 1
HiddenServiceSingleHopMode 1


Please read about these options before you enable them in the manual page.

Shared Randomness

We've talked about this before but now it is a reality. At midnight UTC every day, directory authorities collectively generate a global random value that cannot be predicted in advance. This daily fresh random value is the foundation of our next generation onion service work coming soon to a Tor near you.

In the consensus file, they will look like this; if all goes well, at 00:00 UTC, consensus will have a new one:


shared-rand-current-value Hq+hGlzwAVetJ2zkO70riH/SEMNri+c7Ps8xERZ3a0o=
shared-rand-previous-value CY5TncVAltDpkBKZUBYT1canvqmVoNuweiKVZIilHfs=


Thanks to atagar, the Stem Library version 1.5.0 supports parsing the shared random values from the consensus. See here for more information!

Voluntarily, we haven't exposed those values to the control port yet and will wait for a full stable release cycle in order to make sure it's stable enough for a third party application to use them (https://trac.torproject.org/19925).

Mandatory ntor handshake

This is another important security feature introduced in the new release. Authorities, relays and clients now require ntor keys in all descriptors, for all hops, for all circuits, and for all other roles.

In other words, except for onion services (and this will be addressed with the next generation), only ntor is used--now finally dropping the TAP handshake.

This results in better security for the overall network and users.


Enjoy this new release!

Cooking with Onions: Finding the OnionBalance

Hello,

This blog post is the first part of the Cooking with Onions series which aims to highlight various interesting developments on the .onion space. This particular post presents a technique for efficiently scaling busy onion services.

The need for scaling

Onion services have been around for a while. During the past few years, they have been deployed by many serious websites like major media organizations (like the Washington Post), search engines (such as DuckDuckGo) and critical Internet infrastructure (e.g. PGP keyservers). This has been a great opportunity for us, the development team, since our code has been hardened and tested by the sheer volume of clients that use it every day.

This recent widespread usage also gave us greater insights on the various scalability issues that onion service operators face when they try to take their service to the next level. More users means more load to the onion service, and there is only so much that a single machine can handle. The scalability of the onion service protocol has been a topic of interest to us for a while, and recently we've made advancements in this area by releasing a tool called OnionBalance.

So what is OnionBalance?

OnionBalance is software designed and written by Donncha O'Cearbhaill as part of Tor's Summer of Privacy 2015. It allows onion service operators to achieve the property of high availability by allowing multiple machines to handle requests for a single onion service. You can think of it as the onion service equivalent of load balancing using round-robin DNS.

OnionBalance has recently started seeing more and more usage by onion service operators! For example, the Debian project recently started providing onion services for its entire infrastructure, and the whole project is kept in line by OnionBalance.





How OnionBalance works

Consider Alice, an onion operator, who wants to load balance her overloaded onion service using OnionBalance.

She starts by setting up multiple identical instances of that onion service in multiple machines, makes a list of their onion addresses, and passes the list to OnionBalance. OnionBalance then fetches their descriptors, extracts their introduction points, and publishes a "super-descriptor" containing all their introduction points. Alice now passes to her users the onion address that corresponds to the "super-descriptor". Multiple OnionBalance instances can be run with the same configuration to provide redundancy when publishing the super descriptor.

When Bob, a client, wants to visit Alice's onion service, his Tor client will pick a random introduction point out of the super-descriptor and use it to connect to the onion service. That introduction point can correspond to any of the onion service instances, and this way the client load gets spread out.

With OnionBalance, the "super-descriptor" can be published from a different machine to the one serving the onion service content. Your onion service private key can be kept in a more isolated location, reducing the risk of key compromise.

For information on how to set up OnionBalance, please see the following article:
http://onionbalance.readthedocs.io/en/latest/

Conclusion

OnionBalance is a handy tool that allows operators to spread the load of their onion service to multiple machines. It's easy to set up and configure and more people should give it a try.

In the meanwhile, we'll keep ourselves busy coming up with other ways to scale onion services in this brave new world of onions that is coming!

Take care until the next episode :)

Mission: Montreal! (Building the Next Generation of Onion Services)







A few weeks ago, a small group of Tor developers got together in Montreal and worked on onion services for a full week. The event was very rewarding and we wrote this blog post to share with you how we spent our week! For the record, it was our second onion service hackfest, following the legendary Arlington Accords of July 2015.

Our main goal with this meeting was to accelerate the development of the Next Generation Onion Services project (aka proposal 224). We have been working on this project for the past several months and have made great progress. However, it's a huge project! Because of its volume and complexity, it has been extremely helpful to meet and work together in the same physical space as we hammer out open issues, review each other's code, and quickly make development decisions that would take days to coordinate through mailing lists.

During the hackfest, we did tons of work. Here is an incomplete list of the things we did:

  • In our previous hidden service hackfest, we started designing a system for distributed random number generation on the Tor network. A "distributed random number generator" is a system where multiple computers collaborate and generate a single random number in a way that nobody could have predicted in advance (not even themselves). Such a system will be used by next generation onion services to inject unpredictability into the system and enhance their security.

    Tor developers finished implementing the protocol several months ago, and since then we've been reviewing, auditing, and testing the code.

    As far as we know, a distributed random generation system like this has never been deployed before on the Internet. It's a complex system with multiple protocol phases that involves many computers working together in perfect synergy. To give you an idea of the complexity, here are the hackfest notes of a developer suggesting a design improvement to the system:







    Complicated protocols require lots of testing! So far, onion service developers have been testing this system by creating fake small virtual Tor networks on their laptops and doing basic tests to ensure that it works as intended. However, that's not enough to properly test such an advanced feature. To really test something like this, we need to make a Tor network that works exactly like the real Tor network. It should be a real distributed network over the Internet, and not a virtual thing that lives on a single laptop!

    And that's exactly what we did during the Montreal hackfest! Each Tor developer set up their own Tor node and enabled the "distributed random number generation" feature. We had Tor nodes in countries all around the world, just like the real Tor network, but this was a network just for ourselves! This resulted in a "testing Tor network" with 11 nodes, all performing the random number generation protocol for a whole week.

    This allowed us to test scenarios that could make the protocol burp and fail in unpredictable ways. For example, we instructed our testing Tor nodes to abort at crucial protocol moments, and come back in the worst time possible ways, just to stress test the system. We had our nodes run ancient Tor versions, perform random chaotic behaviors, disappear and never come back, etc.

    This helped us detect various bugs and edge cases. We also confirmed that our system can survive network failures that can happen on the real Internet. All in all, it was a great educational experience! We plan to keep our testing network live, and potentially recruit more people to join it, to test even more features and edge cases!

    For what it's worth, here is a picture of the two first historic random values that our Tor test network generated. The number "5" means that 5 Tor nodes contributed randomness in generating the final random value:





  • We also worked to improve the design of next generation onion services in other ways. We improved the clarity of the specification of proposal 224 and fixed inconsistencies and errors in the text (see latest prop224 commits).

    We designed various improvements to the onion service descriptor download logic of proposal 224 as well as ways to improve the handling of clients with skewed clocks. We also brainstormed ways we can improve the shared randomness protocol in the future.





    We discussed ways to improve the user experience of the 55-character-long onion addresses for next generation onion services (compared to the 16-character-long onion addresses used currently). While no concrete design has been specified yet, we identified the need for a checksum and version field on them. We also discussed modifications to the Tor Browser Bundle that could improve the user experience of long onion addresses.

  • We don't plan to throw away the current onion service system just yet! When proposal 224 first gets deployed, the Tor network will be able to handle both types of onion services: the current version and the next generation version.

    For this reason, while writing the code for proposal 224, we've been facing the choice of whether to refactor a particular piece of code or just rewrite it completely from scratch. The Montreal hackfest allowed us to make quick decisions about these, saving tons of time we would have spent trying to decide over mailing lists and bug trackers.







  • We also worked on breaking down further the implementation plan for proposal 224. We split each task into smaller subtasks and decided how to approach them. Take a look at our notes.




All in all, we got crazy amounts of work done and we all have our hands full for months to come.

Finally, if you find these sort of hackfests exciting and you would like to host or sponsor one, don't hesitate to get in touch with us! Contact us at press@torproject.org and we will forward your message to the appropriate people.

Be seeing you!

Nine Questions about Hidden Services

This is an interview with a Tor developer who works on hidden services. Please note that Tor Browser and hidden services are two different things. Tor Browser (downloadable at TorProject.org) allows you to browse, or surf, the web, anonymously. A hidden service is a site you visit or a service you use that uses Tor technology to stay secure and, if the owner wishes, anonymous. The secure messaging app ricochet is an example of a hidden service. Tor developers use the terms "hidden services" and "onion services" interchangeably.
 
1. What are your priorities for onion services development?
 
Personally I think it’s very important to work on the security of hidden services; that’s a big priority.
 
The plan for the next generation of onion services includes enhanced security as well as improved performance. We’ve broken the development down into smaller modules and we’re already starting to build the foundation. The whole thing is a pretty insane engineering job.

2. What don't people know about onion Services?
 
Until earlier this year, hidden services were a labor of love that Tor developers did in their spare time. Now we have a very small group of developers, but in 2016 we want to move the engineering capacity a bit farther out. There is a lot of enthusiasm within Tor for hidden services but we need funding and more high level developers to build the next generation.
 
3. What are some of Tor's plans for mitigating attacks?
 
The CMU attack was fundamentally a "guard node" attack; guard nodes are the first hop of a Tor circuit and hence the only part of the network that can see the real IP address of a hidden service. Last July we fixed the attack vector that CMU was using (it was called the RELAY_EARLY confirmation attack) and since then we've been divising improved designs for guard node security.
 
For example, in the past, each onion service would have three guard nodes assigned to it. Since last September, each onion service only uses one guard node—-it exposes itself to fewer relays. This change alone makes an attack against an onion service much less likely.
 
Several of our developers are thinking about how to do better guard node selection. One of us is writing code on this right now.
 
We are modeling how onion services pick guard nodes currently, and we're simulating other ways to do it to see which one exposes itself to fewer relays—the fewer relays you are exposed to, the safer you are.
 
We’ve also been working on other security things as well. For instance, a series of papers and talks have abused the directory system of hidden services to try to estimate the activity of particular hidden services, or to launch denial-of-service attacks against hidden services.

We’re going to fix this by making it much harder for the attacker's nodes to become the responsible relay of a hidden service (say, catfacts) and be able to track uptime and usage information. We will use a "distributed random number generator"--many computers teaming up to generate a single, fresh unpredictable random number. 

Another important thing we're doing is to make it impossible for a directory service to harvest addresses in the new design. If you don't know a hidden service address, then under the new system, you won't find it out just by hosting its HSDir entry.

There are also interesting performance things: We want to make .onion services scalable in large infrastructures like Facebook--we want high availability and better load balancing; we want to make it serious.

[Load balancing distributes the traffic load of a website to multiple servers so that no one server gets overloaded with all the users. Overloaded servers stop responding and create other problems. An attack that purposely overloads a website to cause it to stop responding is called a Denial of Service (DoS) attack.  - Kate]
 
There are also onion services that don’t care to stay hidden, like Blockchain or Facebook; we can make those much faster, which is quite exciting.
 
Meanwhile Nick is working on a new encryption design--magic circuit crypto that will make it harder to do active confirmation attacks. [Nick Mathewson is the co-founder of the Tor Project and the chief architect of our software.] Active confirmation attacks are much more powerful than passive attacks, and we can do a better job at defending against them.
 
A particular type of confirmation attack that Nick's new crypto is going to solve is a "tagging attack"—Roger wrote a blog post about them years ago called, "One Cell Is Enough"—it was about how they work and how they are powerful.
 
4. Do you run an onion service yourself?  
 
Yes, I do run onion services; I run an onion services on every box I have.  I connect to the PC in my house from anywhere in the world through SSH—I connect to my onion service instead of my house IP. People can see my laptop accessing Tor but don’t know who I am or where I go. 
 
Also, onion services have a property called NAT-punching; (NAT=Network Address Translation). NAT blocks incoming connections;it builds walls around you. Onion services have NAT punching and can penetrate a firewall. In my university campus, the firewall does not allow incoming connections to my SSH server, but with an onion service the firewall is irrelevant.
 
5. What is your favorite onion service that a nontechnical person might use? 
 
I use ricochet for my peer to peer chatting--It has a very nice UI and works well.
 
6. Do you think it’s safe to run an onion service? 
 
It depends on your adversary. I think onion services provide adequate security against most real life adversaries.

However, if a serious and highly motivated adversary were after me, I would not rely solely on the security of onion services. If your adversary can wiretap the whole Western Internet, or has a million dollar budget, and you only depend on hidden services for your anonymity then you should probably up your game. You can add more layers of anonymity by buying the servers you host your hidden service on anonymously (e.g. with bitcoin) so that even if they deanonymize you, they can't get your identity from the server. Also studying and actually understanding the Tor protocol and its threat model is essential practice if you are defending against motivated adversaries.

7. What onion services don’t exist yet that you would like to see? 
 
Onion services right now are super-volatile; they may appear for three months and then they disappear. For example, there was a Twitter clone, Tor statusnet; it was quite fun--small but cozy. The guy or girl who was running it couldn’t do it any longer. So, goodbye! It would be very nice to have a Twitter clone in onion services. Everyone would be anonymous. Short messages by anonymous people would be an interesting thing.
 
I would like to see apps for mobile phones using onion services more—SnapChat over Tor, Tinder over Tor—using Orbot or whatever. 
 
A good search engine for onion services. This volatility comes down to not having a search engine—you could have a great service, but only 500 sketchoids on the Internet might know about it.
 
Right now, hidden services are misty and hard to see, with the fog of war all around. A sophisticated search engine could highlight the nice things and the nice communities; those would get far more traffic and users and would stay up longer.
 

The second question is how you make things. For many people, it’s not easy to set up an onion service. You have to open Tor, hack some configuration files, and there's more.

We need a system where you double click, and bam, you have an onion service serving your blog. Griffin Boyce is developing a tool for this named Stormy. If we have a good search engine and a way for people to start up onion services easily, we will have a much nicer and more normal Internet in the onion space. 
 
8. What is the biggest misconception about onion services?
  
People don't realize how many use cases there are for onion services or the inventive ways that people are using them already. Only a few onion services ever become well known and usually for the wrong reasons.
 
I think it ties back to the previous discussion--—the onion services we all enjoy have no way of getting to us. Right now, they are marooned on their island of hiddenness. 
 
9. What is the biggest misconception about onion services development?
 
It’s a big and complex project—it’s building a network inside a network; building a thing inside a thing. But we are a tiny team. We need the resources and person power to do it.
 
 

(Interview conducted by Kate Krauss)

Did the FBI Pay a University to Attack Tor Users?

The Tor Project has learned more about last year's attack by Carnegie Mellon researchers on the hidden service subsystem. Apparently these researchers were paid by the FBI to attack hidden services users in a broad sweep, and then sift through their data to find people whom they could accuse of crimes. We publicized the attack last year, along with the steps we took to slow down or stop such an attack in the future:
https://blog.torproject.org/blog/tor-security-advisory-relay-early-traffic-confirmation-attack/

Here is the link to their (since withdrawn) submission to the Black Hat conference:
https://web.archive.org/web/20140705114447/http://blackhat.com/us-14/briefings.html#you-dont-have-to-be-the-nsa-to-break-tor-deanonymizing-users-on-a-budget
along with Ed Felten's analysis at the time:
https://freedom-to-tinker.com/blog/felten/why-were-cert-researchers-attacking-tor/

We have been told that the payment to CMU was at least $1 million.

There is no indication yet that they had a warrant or any institutional oversight by Carnegie Mellon's Institutional Review Board. We think it's unlikely they could have gotten a valid warrant for CMU's attack as conducted, since it was not narrowly tailored to target criminals or criminal activity, but instead appears to have indiscriminately targeted many users at once.

Such action is a violation of our trust and basic guidelines for ethical research. We strongly support independent research on our software and network, but this attack crosses the crucial line between research and endangering innocent users.

This attack also sets a troubling precedent: Civil liberties are under attack if law enforcement believes it can circumvent the rules of evidence by outsourcing police work to universities. If academia uses "research" as a stalking horse for privacy invasion, the entire enterprise of security research will fall into disrepute. Legitimate privacy researchers study many online systems, including social networks — If this kind of FBI attack by university proxy is accepted, no one will have meaningful 4th Amendment protections online and everyone is at risk.

When we learned of this vulnerability last year, we patched it and published the information we had on our blog:
https://blog.torproject.org/blog/tor-security-advisory-relay-early-traffic-confirmation-attack/

We teach law enforcement agents that they can use Tor to do their investigations ethically, and we support such use of Tor — but the mere veneer of a law enforcement investigation cannot justify wholesale invasion of people's privacy, and certainly cannot give it the color of "legitimate research".

Whatever academic security research should be in the 21st century, it certainly does not include "experiments" for pay that indiscriminately endanger strangers without their knowledge or consent.

A Hidden Service Hackfest: The Arlington Accords



At the beginning of July, a few of us gathered in Washington DC for the first hidden service hackfest. Our crew was comprised of core Tor developers and researchers who were in the area; mostly attendees of PETS. The aim was to push hidden service development forward and swiftly arrive at decisions that were too tiresome and complex to make over e-mail.

Since we were mostly technical folks, we composed technical proposals and prioritized development, and spent less time with organizational or funding tasks. Here is a snapshot of the work that we did during those 5 days:

  • The first day, we discussed current open topics on hidden services and tasks we should be doing in the short-to-medium-term future.

    Our list of tasks included marketing and fundraising ones like "Re-branding hidden services" and "Launch crowdfunding campaign", but we spent most of the first day discussing Proposal 224 aka the "Next Generation Hidden Services" project.

  • Proposal 224 is our master plan for improving hidden services in fundamental ways: The new system will be faster, use better cryptography, have more secure onion addresses, and offer advanced security properties like improved DoS resistance and keeping identity keys offline. It's heavy engineering work, and we are still fine-tuning the design, so implementation has not started yet.

    While discussing how we would implement the system, we decided that we would need to write most of the code for this new protocol from scratch, instead of hooking into the old and rusty hidden service code. To move this forward, we spent part of the following days splitting the proposal into individual modules and figuring out how to refactor the current data structures so that the new protocol can coexist with the old protocol.

  • One open design discussion on proposal 224 has been an earlier suggestion of merging the roles of "hidden service directory" and "introduction point" on the hidden service protocol. This change would improve the security and performance as well as simplify the relevant code, and reduce load on the network. Because it changes the protocol a bit, it would be good to have it specified precisely. For this reason, we spent the second and third days writing a proposal that defines how this change works.
  • Another core part of proposal 224 is the protocol for global randomness calculation. That's a system where the Tor network itself generates a fresh, unpredictable random value everyday; basically like the NIST Randomness Beacon but decentralized.

    Proposal 225 specifies a way that this can be achieved, but there are still various engineering details that need to be ironed out. We spent some time discussing the various ways we can implement the system and the engineering decisions we should take, and produced a draft Tor proposal that specifies the system.

  • We also discussed guard discovery attacks, and the various defenses that we could deploy. The fact that many core Tor people were present helped us decide rapidly which various parameters and trade-offs that we should pick. We sketched a proposal and posted it to the [tor-dev] mailing list and it has already received very helpful feedback.
  • We also took our old design for "Direct Onion Services" and revised it into a faster and far more elegant protocol. These types of services trade service-side location privacy for improved performance, reliability, and scalability. They will allow sites like reddit to offer their services faster on hidden services while respecting their clients anonymity. During the last days of the hackfest, we wrote a draft proposal for this new design.
  • We did more development on OnioNS, the Onion Name System, which allows a hidden service operator to register a human memorable name (e.g. example.tor) that can be used instead of the regular onion address. In the last days of the hackfest we prepared a proof-of-concept demo wherein a domain name was registered and then the Tor Browser successfully loaded a hidden service under that name. That was a significant step for the project.
  • We talked about rebranding the "Hidden Services" project to "Onion Services" to reduce "hidden"/"dark"/"evil" name connotations, and improve terminology. In fact, we've been on this for a while, but we are still not sure what the right name is. What do you think?




And that's only part of what we did. We also wrote code for various tickets, reviewed even more code and really learned how to use Ricochet.

All in all, we managed to fit more things than we hoped into those few days and we hope to do even more focused hackfests in the near future. Email us if you are interested in hosting a hackfest!

If you'd like to get involved with hidden service development, you can contact the hackfest team. Our nicks on IRC OFTC are armadev, asn, dgoulet, kernelcorn, mrphs, ohmygodel, robgjansen, saint, special, sysrqb, and syverson.

Until next time!

How We Work

The Tor Project is driven by ideas. We believe in the right to privacy for every person on the planet. Our community—paid and volunteer—brainstorms projects that embody those ideas, like decentralized hidden messaging systems or ingenious new ways to get uncensored Internet access to people in China.

On our public wikis, we make lists of what we need to build these projects—and then we approach potential sponsors with these lists. If we’re lucky, a sponsor will pay to do the project. If not, we may make it for free.

This is true whether the potential sponsor is a government agency or anyone else.

Because of this system, some projects, like hidden services, need more funding, and we are seeking individual contributions to make this technology stronger. One day we hope to build it into many more programs—for instance, phone apps--to make them private and secure by default.

Our diverse, international community includes thousands of men and women inspired by the ideals we share. They work to support Tor and create important tools based on Tor, like Tails and Orbot (there are at least a dozen of these). Our group includes visionaries who think and talk publicly about the Internet and the future of privacy; among them: @nickm_tor, @ioerror and @RogerDingledine. @aaronsw was one of us.

We will accept no back doors to our software, ever. You can watch @ioerror talk about this at last year’s 31c3 talk in Hamburg. We believe in and build free, open source software—free as in freedom. Tor’s source code is online for everyone to see.

We are proud of our people, our work, and our ideals. We are a human rights organization. We are inventors. Our community is a workshop for the future of privacy tools; maybe even for the future of privacy.

The Tor community is open to newcomers; we hope you will join us.

Crowdfunding the Future (of Hidden Services)

Hidden Services have received a lot of attention in 2015, and Tor is at the center of this conversation. Hidden Services are a Tor technology that allows users to connect to services (blogs, chats, and many other things) with neither the user nor the site giving up identifying information.

In fact, anything you can build on the internet, you can build on hidden services. But they're better--they give users things that normal networking doesn't authentication and confidentiality are built in; anonymity is built in. An internet based on hidden services would be an internet with Tor built in--a feature that users could take for granted. Think of what this might mean to millions of users in countries like China, Iran, or the UK. Yet currently, only about 4% of Tor's traffic comes from hidden services.

So we at Tor have been considering how we might meet the challenge of making them more widely available. In this post, we will briefly discuss the role of hidden services before we explore the idea of using crowdfunding to pay for bold, long-term tech initiatives that will begin to fulfill the promise of this technology.

Hidden Services are a critical part of Tor's ecosystem

Hidden Services provide a means for Tor users to create sites and services that are accessible exclusively within the Tor network, with privacy and security features that make them useful and appealing for a wide variety of applications.

For example, hidden services are currently used by activists and journalists to publish blogs--in anonymity and free from retaliation. They are used by NGOs to securely receive information on government corruption and injustice from concerned citizens. Newspapers such as the Washington Post, and human rights groups such as Amnesty International use them to receive leaked information. They are used by people looking for the latest cat facts, companies that want to secure the path of their clients or by people chatting securely and anonymously -- including at-risk journalists talking to sources.

In addition, developers use hidden services as a building block to incorporate Tor's security and anonymity features into totally separate products. The potential of hidden services is huge, and much of it is yet to be explored.

Next Steps for Hidden Services

We want to make this technology available to the wider public as these services will play a key role in the future of secure communications. This means that we must increase the uses for hidden services, bring them to mobile platforms for anonymous mobile apps, and vastly increase the number of people who use them.

Since our goal is wider use, it is imperative that we build them to be more secure, easier to set up, better performing, and more usable. Clearly, the questions that we answer in early deployment efforts will inform how we answer the deeper questions pertaining to massive worldwide deployment.

We must engage a large number of people to bring hidden services to the next level. Until now, hidden services development largely relied on the volunteer work of developers in their spare time. This will not be sufficient if we are to make the leap to transformative hidden services.

We are currently evaluating funding strategies that will support our Hidden Service initiatives in the short-, intermediate- and long-term. In order to fit the requirements more conservative large funders have, so we can fully sponsor the Next Generation Hidden Services, we must put preliminary pieces in place. And for that we will reach out to crowdfunding. To do this right, we need your feedback.

Why Crowdfunding?

Crowdfunding allows us to engage the broader community in grasping the opportunity that this new technology promises. We are confident that we can deliver significant advancements in the hidden services field in the short-term, and that many small donors who understand their context will be eager to contribute. We intend to begin by prioritizing the improvement of the security, usability, and performance of the current hidden services system.

Further, we want to make sure we support the efforts of community projects and that the community is participating in shaping the evolution of hidden services. For example, it would be important to assist and improve the Tor integration of projects such as SecureDrop, Pond, Ahmia and Ricochet. We are in the unique position to be able to shape the Tor protocol to make these projects easier to use and better performing, and we would like to identify ways to promote broader deployment of these projects.

Identifying, prioritizing and meeting future challenges will require engagement throughout the greater community. For instance, as changes and enhancements are introduced, we hope to speak with the best bug hunters, cryptographers and privacy experts and ask them to audit our code and designs. Non-technical users could help us evaluate the usability of our improvements.

For this crowdfunding campaign we have identified a few possible ideas-- but the point of this post is to ask you for yours. Here are three projects that we have come up with so far:

  • Information Panel for Hidden Service Operators

  • An application that Hidden Service operators could use to learn more about the activity of their Hidden Service. The operator would have access to information on user activity, security information, etc., and will receive important system-generated updates, including log messages


  • Fast-but-not-hidden services

  • A way to set up public hidden services with improved performance but reduced server-side anonymity. Basically, hidden services that don't care about anonymity but still want to protect their clients with Tor's cryptography and anonymity, will be able to run faster since they don't need to protect their own anonymity. This is an optional feature that suits the needs of large sites like Facebook and reddit, and will make their hidden services faster while also reducing the traffic they cause to the network. Also by optimizing for performance in this specialized feature, we can optimize for security even more in the default hidden services configuration.


  • Next Generation Hidden Services

  • Tor has been at the center of hidden services from the beginning. We have big lists of changes we need to do to the Tor protocol to increase the security of hidden services against cryptanalysis, DoS and deanonymization attacks. We also want to improve guard security, allow operators to store their cryptographic keys offline and enable scaling of hidden services to new levels. This is a big project but we hope to start crunching through it as part of this crowdfunding campaign.


    Your Idea for Hidden Services?

    Long story short, we are looking for feedback!

  • What hidden services projects would you like to see us crowdfund?

  • How do you use hidden services; what makes them important to you? How you want to see them evolve?

  • We'd love to hear your ideas on picking crowdfunding rewards and stretch goals.
    Also, we are curious about which crowdfunding platforms you prefer and why.

  • Feel free to use the comments of this blog, or contact us directly at tor-assistants@torproject.org. Also see our wiki page with more information!

    In the following weeks, we will update you on our progress, incorporating feedback we receive from the community. We hope to make this process as transparent and public as possible!

    Thanks!

    EDIT: The "Unhidden Services" paragraph was expanded and changed to "Fast-but-not-hidden Services". The previous name was too scary and the description not sufficient to show the potential of the project. Please send us better names for this feature!

    Some statistics about onions

    Non-technical abstract:


    We are starting a project to study and quantify hidden services traffic. As part of this project, we are collecting data from just a few volunteer relays which only allow us to see a small portion of hidden service activity (between 2% and 5%). Extrapolating from such a small sample is difficult, and our data are preliminary.

    We've been working on methods to improve our calculations, but with our current methodology, we estimate that about 30,000 hidden services announce themselves to the Tor network every day, using about 5 terabytes of data daily. We also found that hidden service traffic is about 3.4% of total Tor traffic, which means that, at least according to our early calculations, 96.6% of Tor traffic is *not* hidden services. We invite people to join us in working to research methodologies and develop systems for better understanding Tor hidden services.



    Hello,

    Over the past months we've been working on hidden service statistics. Our goal has been to answer the following questions:

    • "Approximately how many hidden services are there?"
    • "Approximately how much traffic of the Tor network is going to hidden services?"


    We chose the above two questions because even though we want to understand hidden services, we really don't want to harm the privacy of Tor users. From a privacy perspective, the above two questions are relatively easy questions to answer because we don't need data from clients or the hidden services themselves; we just need data from hidden service directories and rendezvous points. Furthermore, the measurements reported by each relay cannot be linked back to specific hidden services or their clients.

    Our first move was to research various ways we could collect these statistics in a privacy-preserving manner. After days of discussions on obfuscating statistics, we began writing a Tor proposal with our design, as well as code that implements the proposal. The code has since been reviewed and merged to Tor! The statistics are currently disabled by default so we asked volunteer relay operators to explicitly turn them on. Currently there are about 70 relays publishing measurements to us every 24 hours:


    Number of relays reporting stats


    So as of now we've been receiving these measurements for over a month, and we have thought a lot about how to best use the reported measurements to derive interesting results. We finally have some preliminary results we would like to share with you:

    How many hidden services are there?

    All in all, it seems that every day about 30000 hidden services announce themselves to the hidden service directories. Graphically:


    Number of hidden services


    By counting the number of unique hidden service addresses seen by HSDirs, we can get the approximate number of hidden services. Keep in mind that we can only see between 2% and 5% of the total HSDir space, so the extrapolation is, naturally, messy.

    How much traffic do hidden services cause?

    Our preliminary results show that hidden services cause somewhere between 400 to 600 Mbit of traffic per second, or equivalently about 4.9 terabytes a day. Here is a graph:


    Hidden services traffic volume


    We learned this by getting rendezvous points to publish the total number of cells transferred over rendezvous circuits, which allows us to learn the approximate volume of hidden service traffic. Notice that our coverage here is not very good either, with a probability of about 5% that a hidden service circuit will use a relay that reports these statistics as a rendezvous point.

    A related statistic here is "How much of the Tor network is actually hidden service usage?". There are two different ways to answer this question, depending on whether we want to understand what clients are doing or what the network is doing. The fraction of hidden-service traffic at Tor clients differs from the fraction at Tor relays because connections to hidden services use 6-hop circuits while connections to the regular Internet use 3-hop circuits. As a result, the fraction of hidden-service traffic entering or leaving Tor is about half of the fraction of hidden-service traffic inside of Tor. Our conclusion is that about 3.4% of client traffic is hidden-service traffic, and 6.1% of traffic seen at a relay is hidden-service traffic.

    Conclusion and future work

    In this blog post we presented some preliminary results that could be extracted from these new hidden service statistics. We hope that this data can help us better gauge the future development and maturity of the onion space as well as detect potential incidents and bugs on the network. To better present our results and methods, we wrote a short technical report that outlines the exact process we followed. We invite you to read it if you are curious about the methodology or the results.

    Finally, this project is only a few months old, and there are various plans for the future. For example:

    • There are more interesting questions that we could examine in this area. For example: "How many people are using hidden services every day?" and "How many times does someone try to visit a hidden service that does not exist anymore?."

      Unfortunately, some of these questions are not easy to answer with the current statistics reporting infrastructure, mainly because collecting them in this way could reveal information about specific hidden services but also because the results of the current system contain too much obfuscating data (each reporting relay randomizes its numbers a little bit before publishing them, so we can learn about totals but not about specific events).

      For this reason, we've been analyzing various statistics aggregation protocols that could be used in place of the current system, allowing us to safely collect other kinds of statistics.


    • We need to incorporate these statistics in our Metrics portal so that they are updated regularly and so that everyone can follow them.

    • Currently, these hidden service statistics are not collected in relays by default. Unfortunately, that gives us very small coverage of the network, which in turn makes our extrapolations very noisy. The main reason that these statistics are disabled by default is that similar statistics are also disabled (e.g. CellStatistics). Also, this allows us more time to consider privacy consequences. As we analyze more of these statistics and think more about statistics privacy, we should decide whether to turn these statistics on by default.

      It's worth repeating that the current results are preliminary and should be digested with a grain of salt. We invite statistically-inclined people to review our code, methods, and results. If you are a researcher interested in digging into the measurements themselves, you can find them in the extra-info descriptors of Tor relays.

      Over the next months, we will also be thinking more about these problems to figure out proper ways to analyze and safely measure private ecosystems like the onion space.


    Till then, take care, and enjoy Tor!

    Syndicate content Syndicate content