A Hidden Service Hackfest: The Arlington Accords

by asn | August 4, 2015

At the beginning of July, a few of us gathered in Washington DC for the first hidden service hackfest. Our crew was comprised of core Tor developers and researchers who were in the area; mostly attendees of PETS. The aim was to push hidden service development forward and swiftly arrive at decisions that were too tiresome and complex to make over e-mail.

Since we were mostly technical folks, we composed technical proposals and prioritized development, and spent less time with organizational or funding tasks. Here is a snapshot of the work that we did during those 5 days:

  • The first day, we discussed current open topics on hidden services and tasks we should be doing in the short-to-medium-term future.

    Our list of tasks included marketing and fundraising ones like "Re-branding hidden services" and "Launch crowdfunding campaign", but we spent most of the first day discussing Proposal 224 aka the "Next Generation Hidden Services" project.

  • Proposal 224 is our master plan for improving hidden services in fundamental ways: The new system will be faster, use better cryptography, have more secure onion addresses, and offer advanced security properties like improved DoS resistance and keeping identity keys offline. It's heavy engineering work, and we are still fine-tuning the design, so implementation has not started yet.

    While discussing how we would implement the system, we decided that we would need to write most of the code for this new protocol from scratch, instead of hooking into the old and rusty hidden service code. To move this forward, we spent part of the following days splitting the proposal into individual modules and figuring out how to refactor the current data structures so that the new protocol can coexist with the old protocol.

  • One open design discussion on proposal 224 has been an earlier suggestion of merging the roles of "hidden service directory" and "introduction point" on the hidden service protocol. This change would improve the security and performance as well as simplify the relevant code, and reduce load on the network. Because it changes the protocol a bit, it would be good to have it specified precisely. For this reason, we spent the second and third days writing a proposal that defines how this change works.
  • Another core part of proposal 224 is the protocol for global randomness calculation. That's a system where the Tor network itself generates a fresh, unpredictable random value everyday; basically like the NIST Randomness Beacon but decentralized.

    Proposal 225 specifies a way that this can be achieved, but there are still various engineering details that need to be ironed out. We spent some time discussing the various ways we can implement the system and the engineering decisions we should take, and produced a draft Tor proposal that specifies the system.

  • We also discussed guard discovery attacks, and the various defenses that we could deploy. The fact that many core Tor people were present helped us decide rapidly which various parameters and trade-offs that we should pick. We sketched a proposal and posted it to the [tor-dev] mailing list and it has already received very helpful feedback.
  • We also took our old design for "Direct Onion Services" and revised it into a faster and far more elegant protocol. These types of services trade service-side location privacy for improved performance, reliability, and scalability. They will allow sites like reddit to offer their services faster on hidden services while respecting their clients anonymity. During the last days of the hackfest, we wrote a draft proposal for this new design.
  • We did more development on OnioNS, the Onion Name System, which allows a hidden service operator to register a human memorable name (e.g. example.tor) that can be used instead of the regular onion address. In the last days of the hackfest we prepared a proof-of-concept demo wherein a domain name was registered and then the Tor Browser successfully loaded a hidden service under that name. That was a significant step for the project.
  • We talked about rebranding the "Hidden Services" project to "Onion Services" to reduce "hidden"/"dark"/"evil" name connotations, and improve terminology. In fact, we've been on this for a while, but we are still not sure what the right name is. What do you think?

And that's only part of what we did. We also wrote code for various tickets, reviewed even more code and really learned how to use Ricochet.

All in all, we managed to fit more things than we hoped into those few days and we hope to do even more focused hackfests in the near future. Email us if you are interested in hosting a hackfest!

If you'd like to get involved with hidden service development, you can contact the hackfest team. Our nicks on IRC OFTC are armadev, asn, dgoulet, kernelcorn, mrphs, ohmygodel, robgjansen, saint, special, sysrqb, and syverson.

Until next time!

Comments

Please note that the comment area below has been archived.

August 04, 2015

Permalink

So what’s up with “direct onion services”, provided that server-side anonymity is pretty much the only feature of onion services to begin with? In other words, if you lose server-side anonymity, what do you have left that can’t be done with regular, non-Tor severs? You can get encryption and integrity protection with TLS, and you can even force clients to use Tor by only permitting connections from Tor exit nodes.

Hello,

the past months we've been seeing various organizations who don't care about server-side location privacy, setting up their own hidden services. For example, sites like Facebook and blockchain.info use hidden services to better load-balance Tor users, and protect them from phishing and exit node attacks. Furthermore, the fact that Tor onion URLs are self-authenticating, allows users to reach those websites safely without relying on the security of SSL certificate authorities or the DNS system.

With "direct onion services" (or however we name this), the above websites could do this and also get greatly improved performance, since the server-side does not need to be hidden. We think that clients and the network will appreciate this.

Also check out the old proposal for this system (from 4 years ago) and its Motivation section for more use cases: https://gitweb.torproject.org/torspec.git/tree/proposals/ideas/xxx-encr…

And if you really think that hidden services are all about location-privacy, check out the following short paper by Syverson and Boyce:
https://raw.githubusercontent.com/saint/w2sp-2015/master/w2sp-simple-on…
that shows various ways onion services can be used for authentication and the new use cases they enable.

In general we think that in the future, we should decouple all these security properties of hidden services (as we learn them ourselves), and allow operators to pick and choose which properties they are interested in.

Cheers!

So, "direct onion services", only have a 3-hop circuit instead of a 6-hop circuit?
This means, that a "direct onion services" is only more private, and not more anonymous than visiting a regular clearnet website.

If it really isn't as anonymous than visiting a regular clearnet website, I'd suggest the TLD to be changed, or make sure that it is not being confused with Hidden Services that have a 6-hop circuit.

Hm, I'm not sure what you mean here.

The security of the client is the same regardless of whether the hidden service is a direct onion service or not.

Here is how a hidden service circuit looks like now:
client -> c_hop1 - c_hop2 - c_hop3 (RP) - s_hop3 - s_hop2 - s_hop1 - HS
(where c_hop1 is the first hop of the client, and s_hop2 is the second hop of the HS)

And here is how it's going to look like with direct onion services:
client -> c_hop1 - c_hop2 - c_hop3 - HS

As you can see, in both cases the client is using a 3 hop Tor circuit, hence the client's anonymity is protected. In this sense it's as anonymous as "visiting a regular clearnet website".

Hope that cleared it up.

"safely without relying on the security of SSL certificate authorities or the DNS system."
I do not understand this sentence ; could you explain better pls (hidden service does not need certificate authentification/exchange key ???)

no "calomel" add-on are implemented with tor ; of course, it seems that it should be a hole if it was but or something similar is yet done or certificate has no value and calomel is useless on tor.

It's all because onion URLs (like duskgytldkxiuqc6.onion) are self-authenticating.

That's because this weird duskgytldkxiuqc6 name is not just a simple name, it's a cryptographic identity. It corresponds to a specific private key, in the same way that an SSL certificate corresponds to a specific SSL private key.

Hence, when you visit `duskgytldkxiuqc6.onion` over Tor, the Tor client ensures that you will only speak with the hidden service that knows the correct private key.

That saves you from the man-in-the-middle attacks of SSL like sslstrip (see alleged attacks on blockchain.info), or from DNS spoofing attacks that someone can do over WiFi.

Hope that helps.

This has nothing to do with hidden services, and "clickbait garbage" by a "journalist""senior editor" who didn't read the original paper isn't a valid attack on Tor Browser.

From the actual research paper:
> To the best of our knowledge, the only browser that has a strong
> defense against fingerprinting by the Battery Status API is Tor
> Browser. Tor Browser completely disables the API [20] to thwart
> possible fingerprinting attempts."

Check for yourself: https://robnyman.github.io/battery/

Edit: Corrected "journalist" to "senior editor" to better reflect reality.

August 05, 2015

Permalink

Anonymous: not named or identified.

Cryptographically: any of various mathematical techniques for encrypting and decrypting data in order to keep it private when transmitted or stored electronically.

Elaborate: worked out with great care and nicety of detail.

A.C.E (Anonymous, cryptographically, elaborate) services for those that want location anonymity.

C.E (Cryptographically, elaborate) services for those that just want the strong encryption Tor provides.

The IT community shouldn't have a problem with abbreviations. I believe the above is easy and smooth to say and provides the interpreter everything they need to understand the purpose of the protocols.

My new ACE service is now accepting registrations.
Announcing the new Facebook CE service.

Sounds good to me!

If the abbreviation is pronounceable like Tor (The Onion Router), then okay. But mostly I dislike the use of abbreviations. We have the time to think about a good name, right?

ACE can be said as one word. Like ace pilot. CE is easy to say. As in windows CE.

The more time that passes the more exposure the term "hidden" gets. The sudden change seen by this growing crowed might do little to reverse the pre-associated negative connotations. Unfortunately abbreviations mean a tiny amount of research on part of the curious. We should foster curiosity. A balanced approach is necessary. If we make things too easy. This currently imperfect implementation of security and anonymity we call the Tor ecosystem will open informational holes and facilitate op sec errors by means of user ignorance. Only a fool proof implementation will make the former sentence irrelevant.

August 07, 2015

Permalink

>We talked about rebranding the "Hidden Services" project to "Onion Services" to reduce "hidden"/"dark"/"evil" name connotations, and improve terminology. In fact, we've been on this for a while, but we are still not sure what the right name is. What do you think?

I think that
1) the name "Hidden Service" has no bad connotations. Hidden is good. I will explain why, but please read carefully whole my comment.

If the one disobeys state's will, he is a criminal (it is a definition, not the legal one, but the real one, all the following mentions of criminal and crime are according this definition).
I) There will be some examples:
If the one has something to hide something from the government, he is a criminal, because
a) if he wasn't there would be no reason to hide (state is a huge corporation and it doesn't care about someones wardrobe skeletons unless these skeletons violates its interests);
b) more important is that it violates states' will to know everything about everyone.

II) If the one protects himself from attacks of the state, he is criminal, because he violates state's will to attack him.

III) If the one takes part in anonymous network (including Tor), he is not only a criminal, he is a member of criminal extremist group which aim is make the state powerless to enforce its will in electronic communications.

IV) If the one is a member of another state, he is criminal, because he helps the rival, and helping the rival violates state's will that all the world, all the resources and all your bases must belong to them.

So the purpose of Tor is to hide criminals (every Tor user) from the legal system of today (the legal systems of tomorrow will prosecute the criminals in definition I have made in the beginning of the comment) and to hide their doings from the state, the hiding is also a crime.

You must understand, that criminal is a normal status of free man. Say for yourself 100 times "I'm a criminal and I'm OK with it" and welcome to the cruel world.

The intention to remove signs of criminal activity Tor Project and Tor users are involved into from the names means that the ones who propose this either believe that they live in the world with rainbows and unicorns or wanted the users and the devs beleive such. Such beliefs can only lead us to failure.

2) I propose distinguish zero-circuit hidden services from real hidden services and give zero-circuit services top-level domain .leek (onion is hidden in the dark under ground, leek grows above the ground).

Using the term "criminal" you mix " privacy " and walking on the wrong or right side.
It is for everyone , criminal or not , and the minimum that every user can hope/expect.
i do not care of the name connotations evil:dark.
evil doe not mean perversity /pornography/pedophilia (without gay movement the half of the power and the fund should disappear) and dark does not mean criminal activity (without hidden plan the half of the economy and creation should disappear).
it is not criminal to be different and to wish a free world governed in another way by another person, it is even not a revolution ; it is only a modern evolution of this period that you are leaving now..

August 08, 2015

Permalink

pls let's talk about ricochet , it is a very nice app and need to be improved/involved in an official tor project if and only if it is a safe tool _ experimental means even not yet alpha version _
does it need to be re wrote or to be taken in another manner ?
when tor team will agree to audit it and will bring their help ?
We doi need a stable and very secure version.

It would be nice if ricochet were expanded into a file sharing program. Damn grubby media organizations are never satisfied with just billions they neeeeed mooooorrrreeee!!! I understand intellectual property and the desire to profit, but there has to be a better way. Art while an expression, is also a means of inspiration.

Limiting its impact due to greed is morally reprehensible and a sad reflection of humanities current values. At least half of this planet is impoverished. People are struggling just to get connected to the rest of this world. To further burden their curiosity and their potential for great contribution we then place price tags on information and art!!! A great number of influential people owe their passion to art and information others have produced. Let's not forget all the other legitimate use cases for a Tor p2p file-sharing network.

Ricochet or a fork is the perfect candidate for it. From my understanding it creates ephemeral anonymous (a.k.a hidden) services for each user that then connects to other ephemeral anonymous services. If each user were then turned into a ricochet only relay, then boom! You have the beginnings of a Tor powered p2p file-sharing network with chat built in. Take it a step further and allow users to create simple web pages on their newly created hidden service only accessible using ricochet. Of course things need to be thoroughly evaluated and fleshed out, but hey it not as hard as the rest of Tor.
.
.
Here's a light breakdown:

1. Client creates the hidden service (as per standard ricochet practice).
2. Client proceeds to contact the ricochet hidden server for list of peers (bootstrap).
3. Client receives the list then connects to the other peers/hidden services.
4. Over time and based on certain parameters, peers become bootstrap nodes, anonymous DHT's that must come to a consensus about the peer population and available network files, etc.
5. Requests are forwarded to several index nodes that then all communicate with each other to propagate the request and detect faulty or malicious behavior among them (i.e one of the index nodes modifies or denies the request etc).
6. Requester receives reply then begins retrieval of file/s.
7. File owners can be chat with if desired.
8. Using the upcoming OnioNS protocol, users can publish (in ricochet boards or associated with the file) user memorable names for their services.
9. If the service is popular, the network automatically caches it on other nodes effectively mitigating DDoS and compensating for popularity.
.
.
The hidden service protocol is in line to receive a huge upgrade so speed wont be a problem. A Tor backed p2p system would be hugely popular and therefore not need to worry about peer diversity. This could be a separate network or a hybrid network that only minimally uses the current Tor network (it must at first piggy back of Tor though). This new network could configured to help the current Tor network deal with network killing bandwidth surges.

The beauty is that Tor would become massively popular while still being able to steer clear of bad press as ricochet or its fork is not directly affiliated.

Please don't JUST criticize, make the idea better!

I do not know if a p2p will not spoil the beauty of Ricochet ...
A fork especially written for this particular purpose could be a better idea.
( why not a seedbox sharing our torrent as hidden service through this fork ? )

i suggest the possibility of using gpg message (copy & paste is not possible) , so every message even saw (you write in clear text) could not be read.
Unfortunately , i do not think it be possible except maybe with a special plugin embedding my key (one or two ?) and public key (how many maximum ? ) ...

All these improvement/idea/suggestion could break maybe the quality of the app so, let's stay prudent..

August 08, 2015

Permalink

This is all too much really. In order to remain anonymous one must have adequate security. Anonymity relies on security. Security however, does not rely on anonymity.

If a site wants security without anonymity, then naturally they want a "secure service".

If a site wants anonymity, then they obviously want an "anonymous service" (which relies on security).

Wanna brand it with the name Tor? Just add it to the beginning of each of the above (i.e Tor secure service, Tor anonymous service).

Calling these services anything else will have a greater chance of confusing the uninitiated. The term anonymous has from my understanding the connotation of something secure and or revolutionary. The desire to have ones identity unknown for fear of retribution by the corrupt and those with great power.

Onions, leeks, potatoes or any other niche name is probably far too cryptic for the average person.

News broadcasts all over commonly promote the use of anonymous call-ins in regards to reporting crime.

Time is ticking people. This is being over-thought. As the popularity of Tor grows by means of all of this outreach we subsequently expose this term "Hidden" to a greater populous.

They then by word of mouth tell others about Tor and its "hidden services" without all of the necessary contextual data. Now confused and unwilling to ask for more information for fear of a bruised ego or just laziness begin to formulate misconceptions that then devolve into disinterest or hatred.

Those same people the go on to spread informational garbage. This can have a profound effect. Merely speaking about Tor can irrationally become social taboo. Hearsay and negative rumors spread exponentially faster than the truth.

Outreach is unfortunately the catalyst.

* This is all too much really.

I disagree : it is not about tor or hidden service ; it is irrelevant.
let's take two examples ; gaming on line & private chat.
1° take a gaming craphic card on private Lan and you have both (anonymity & security) without to be connected at the net but the protocol most of time is proprietary and you can be hacked/recorded.
2° go to a paying service private chat and you have both (anonymity & security) but you are /protected/censored under survey and you can be infiltrated/banned.
Anonymity and security are not a "service" and both are independent ... or linked together according on the will of the dev_admin : it is not always a matter of conception or idea or coding but also an compromising option for a paying service.
Usage of such service is at your own, reporting crime or using as a fun tool.

Tor & hidden service does not (afaik) need "the necessary contextual data" ... every body can use it like they wish without to be concerned by the war/crime/news/underground or your private life ... but it is true that the user must be educated and learn by himself how to use it correctly for not be exposed at a real danger.

I'm unsure of what is meant by all of this as most of what you said was not written clearly.

As for 1, LAN is a poor example. The anonymity is achieved by the locality, but all those involved are clearly not anonymous. Security can also be more easily undermined by local peers with malicious intent. First example == doesn't seem to work.

For number 2, I can almost see a clear argument forming based on what follows it.

When the two terms are applied to current Tor hidden services they are mutual and not independent. Tor by it's very nature seeks to achieve both anonymity and security. The words anonymous and secure are qualifiers that denote the nature of the protocol not a representation of the services provided. Upcoming protocols aim to provide security independently of anonymity. Having clear labels that are easily interpreted without negative connotations is IMHO the goal.

Tor is proudly open source and not proprietary. Trust isn't needed and therefore not a requirement for the aforementioned qualities to be achieved (only it's proper use).

Contextualizing is indeed necessary. Tor doesn't just pop into peoples mind with all of it's relevant information. It is instead discovered by various means and like all tools, prospective users must learn of it's capabilities and use cases. Without that, mass adoption would likely not rapidly occur, op sec errors would be far more prevalent, and a larger number of people would assume only the worst of Tor and it's hidden services capabilities. Contextualizing also helps the casual thinker and quells misinformation.

well, i was not speaking about tor and i replied about the precedent post which the terms were confusing : anonymity & security ...
I said that another solutions well known existed ; e.g. Lan & Private chat.
you are in security because no one else can interfere in your Lan & in the Chat ; it is protected (strict rules) .
Your choice is respected ; you choose your correspondent.
You are in anonymity because no one wish to meet you in the real life.
And you can say all that you want whom you wish and share that you want.
The mode is clearly "without none responsibility, fun, personalized" .

But tor open a door where every body must be involved , concerned (... without rules ... ? almost together walking in the same way ... ?).
Is it this responsibility the worst of tor , the hidden evidence where a lot of users must be present for the benefit of happy few ?
Will hidden service not be the future of tor ?
Do i need as user a special morality fingerprint approved from who ?
Do i need as user a special immorality fingerprint approved from who ?

I still not understand the reason why so many people are so afraid about hidden service , bad rumors, reputation , misinformation or negative connotations, etc.

I am happy you wrote this " they are mutual and not independent. Tor by it's very nature seeks to achieve both anonymity and security " and that " The words anonymous and secure are qualifiers that denote the nature of the protocol not a representation of the services provided." even if most of people had yet understood that since a long time.

August 10, 2015

Permalink

It is written clearly , anonymity & security are not a tor concept and has been applied on gaming lan & private chat _as secure service & anonymous service_since a long time (support is great) .
Tor is not an opening toward a new frontier.

August 15, 2015

Permalink

OT:

Thanks for sticking with pencils and paper for the larger aspects of this great gathering of nerds with a common goal.

August 16, 2015

Permalink

I nominate TorNet as a possible replacement name for hidden services. It's simple, to the point, and non-technical. People know what Tor is, but they don't necessarily know how Tor works and what Onion routing is. They probably don't even know what the acronym 'Tor' stands for, The Onion Router.

Keep it simple for all the non-techies new to Tor.