Tor at the Heart: apt-transport-tor and Debian onions

During the month of December, we're highlighting other organizations and projects that rely on Tor, build on Tor, or are accomplishing their missions better because Tor exists. Check out our blog each day to learn about our fellow travelers. And please support the Tor Project! We're at the heart of Internet freedom.
Donate today!

apt-transport-tor and Debian onions

Did you know that when you're using Debian, you can configure your operating system package installs and updates to route through Tor?

Doing updates via Tor provides some really compelling security properties. One of the big benefits is that an attacker can't target you for a malicious update: even if they manage to steal some package signing keys and break into the update server and replace packages, they still can't tell that it's you asking for the update. So if they want to be sure you get the malicious update, they're forced to attack everybody who updates, which means a really noisy attack that is much more likely to get noticed. This anonymity goal is one of the main reasons that Tor Browser does its updates over Tor as well.

Another big feature of updating via Tor is that the package repository, or somebody watching it, can't track what programs you've installed. Similarly, somebody spying on your Internet connection will have a tougher time learning which packages you fetch (though this part of the protection is not as strong, since maybe they can count bytes or something and guess).

As Debian's blog puts it:

"The freedom to use open source software may be compromised when access to that software is monitored, logged, limited, prevented, or prohibited. As a community, we acknowledge that users should not feel that their every action is trackable or observable by others. Consequently, we are pleased to announce that we have started making several of the various web services provided by both Debian and Tor available via onion services."

Not showing the world what packages you fetch is good common-sense data hygiene, but it can also provide safety when you're updating a package due to a security vulnerability, and you don't want people to learn that you're running a vulnerable version right now.

How does it work from a technical perspective? The apt-transport-tor deb package introduces a new "tor+http" transport that you can use in your /etc/apt/sources.list file -- so while before you would typically list a Debian package repository as being an "http" address, now you can list it as being a "tor+http" address. Debian has its own official onion addresses for its package repositories, along with onion addresses for many of its other sites and services — and they even use Donncha's OnionBalance tool to provide redundancy and scaling. (Also, since the nice person who helps run Debian's infrastructure also helps to run our infrastructure, that means we now have onion addresses for many of Tor's sites and services too!)

You can configure your Debian system to update via Tor by following the directions at the bottom of the Debian blog post. A growing number of privacy-oriented Debian derivatives, including Tails, use apt-transport-tor as their default way of doing updates, and we think that's a great and important trend.

kata

December 04, 2016

Permalink

Thanks! This is one of the things I'm really excited about.
Please be vigilant while upgrading though. There were times (although few and far between) when apt warned me that I'm attempting to upgrade with a package that has a bad signature (can't remember the exact wording of the message).
Never happened to me with upgrading via clearnet, so I figure this is probably someone on the Tor network that's up to no-good.

Anyway, please keep 'em coming. I'm really starting to enjoy these posts.

Hm! Yes, apt uses plain http to fetch its debs (and then checks the signatures afterwards), so it is possible that somebody somewhere on the Internet (the exit relay, an attacker at the repository, or somebody in between) could mess with those.

That is indeed one of the advantages of using the onion address for reaching the repository -- you get end-to-end authentication and encryption, which pretty much stops all of those types of possible attacks.

(Another advantage of using the onion address is that it shifts load away from exit relays in the Tor network -- which once thousands of people are using this for their package installs might add up to be a big deal.)

It doesn't surprise me that someone would try to deliver a malicious package, but except in rare cases where the user disables signature verification, these can be caught pretty reliably. A bigger concern is version downgrade attacks, like I discussed in a comment below, because the signature is actually valid, but it forces an older (usually vulnerable) version of the package on the client. Using a .onion can help with mitigating version downgrade attacks, too. I'm also not sure if Apt already has a defense against this, e.g. signed/timestamped repository files.

@ Roger, Peter, and all: many thanks for providing the onion mirrors, which I regard as one of the most promising developments involving Tor!

Do we have any usage statistics? I'd worry if only a tiny handful of people were using the onion mirrors.

@ the OP:

> Please be vigilant while upgrading though. There were times (although few and far between) when apt warned me that I'm attempting to upgrade with a package that has a bad signature (can't remember the exact wording of the message). Never happened to me with upgrading via clearnet, so I figure this is probably someone on the Tor network that's up to no-good.

I have not experienced anything like that (yet!), but one problem I *have* experienced while using the onion Debian repository mirrors: at times the gpg file containing all the signed cryptographic hashes fails to download, which results in a warning that "you are about to install unverified software" (words, effects). Of course, the thing to do when this happens is to abandon the upgrade and try again a few hours later. In so circumstances should anyone ever install unverified software.

@ Peter and Debian Project:

For those who need to use scientific software, which is too often unsigned, this injunction can be almost impossible to follow, requiring us to maintain additional "untrusted machines", which is in itself a liability.

I hope Peter can use his influence to move Debian towards to trying to persuade more scientific software authors and institutions to use strong cryptographic signing and database integrity--- that this is a serious issue has been confirmed by extensive hacking of scientists working on global warming for example. Major international physics collaborations and institutions such as CERN have led the way in awareness of the need to ensure the integrity of shared scientific data in large projects, and we need to bring such standards to even small projects.

About the signatures of Debian packages: I cannot find documentation at debian.org of how this happens, but my understanding is that there is a frequently updated signed gpg file which should be automatically fetched whenever you try to update your system and which contains all the package hashes. Fine, except that I fear these hashes might still use MD-5, a long broken system. NSA has a peculiar expertise in faking cryptographic hashes (and also designed the SHA hash protocols), so I hope Debian will explain how their package signing works and will consider adopting GPG signed individual packages, or at least using a strong hash such as SHA-256, assuming my understanding of how the current system works is not mistaken.

Peter?

> my understanding is that there is a frequently updated signed gpg file which should be automatically fetched whenever you try to update your system and which contains all the package hashes. Fine, except that I fear these hashes might still use MD-5, a long broken system...

Can someone who knows the answer comment on this question? Or a link to a debian.org explainer which clearly answers the question?

The recently reported Yahoo gigabreach has once again highlighted the dangers of using MD5:

http://www.theregister.co.uk/2016/12/15/yahoos_password_hash/
Security! experts! slam! Yahoo! management! for! using! old! crypto!
Suits should have done more to protect users, rather than user numbers
Darren Pauli
15 Dec 2016

> ...
> But that the Purple Palace was even using the algorithm has drawn steep criticism from established security boffins. "The MD5 hashing algorithm has been considered not just insecure, but broken, for two decades," says Ty Miller, director of Sydney-based security firm Threat Intelligence, noting that MD5 collision vulnerabilities were found in 1996 with practical attacks developed in 2005.

Oh no, for "in so circumstances" please read "in no circumstances".

See

http://thehill.com/policy/cybersecurity/310234-typo-may-have-caused-pod…
Typo led to Podesta email hack: report
Joe Uchill
13 Dec 2016

> The hack and eventual release of a decade’s worth of Hillary Clinton campaign chairman John Podesta’s emails may have been caused by a typo, The New York Times reported Tuesday in an in-depth piece on Russian cyberattacks.

kata

December 04, 2016

Permalink

Install apt-transport-tor, edit your sources.list to include only tor://

If I leave http or https (non.onion-services) sources on my sources.list are they still routed through the Tor network?

If not is it possible to route them through tor too? (I know that the e2e-encryption just exists as long the traffic stayes in the onion network and thereby some of the advantages are lost if i choose to use clearnet server)

Lines with tor://, tor+http://, and tor+https:// are routed through Tor. The others are not.

I'm not sure where you're quoting from, but based on the reference to tor://, it looks like it might be older documentation. "tor+http" (or "tor+https") is probably the more helpful way to say it, since it makes it clear what's going on: you're using Tor for transport, and http or https as the protocol. From
https://github.com/diocles/apt-transport-tor/commit/2178b5a7f8f4d581dc4…
it looks like the old notation was tor:// and the new notation is tor+http://.

As for how to route the other lines in your apt sources file through Tor -- the ones that you aren't using the tor+http transport for -- see this post from the Guardian project for some hints:
https://guardianproject.info/2016/07/31/howto-get-all-your-debian-packa…

I meant to write the same thing when I saw this post as it took me a while to figure out why the instructions would not work. A coincidental use of tails made me realize of the correct format, as apt/synaptics in tails is already preconfigured with tor+https://...

Great link, and thanks!

This is still a very new development, and my experience suggests that it is better to carefully *remove* the "main" section from the parts of the mirrors which now are available via onions, and then to *append* new lines containing the torified fetch specifications.

As always, if anything unexpected happens, abort the upgrade and try again later.

Obviously, it would be highly desirable to extend the onion mirrors to cover the full (and enormous!) Debian software repository. We all know that mixing clearnet and Tor is never a good thing, and currently it seems that most users (who are not sure of their iptables fu status) are forced to mix and match in non-ideal manner.

kata

December 04, 2016

Permalink

Very nice! Never heard of this before!
Will there be in the future something like this for Ubuntu, Fedora, ... repositories?

I think it should work right now for Ubuntu. If somebody could track down details like which versions you need, and post them here, that would be grand.

I don't know about Fedora. Maybe there are Fedora users here who want to find the answer and tell us? :)

I believe the answer is "yes", but Ubuntu would have to set up its own onions and I think to make available their own version of apt Tor transport (it is possible that the Debian version works well with Ubuntu but it would be an unnecessary obstacle if Ubuntu did not simply provide their own version at their own repos)/

However, some reliable sources have warned that Ubuntu appears to engage in an unacceptable level of surveillance of their own users (e.g. the desktop search fiasco). Debian is probably much more privacy friendly, and also has a much larger repository, and is not really any less user-friendly than Ubuntu.

That said, Debian is not really intended as a "secure OS" and probably cannot be made reasonably secure even by sophisticated users. Currently, it seems that if you use much software and also use Tor Browser on Debian, information leakages through non-browser softwares are all too possible. Still, if you use Tor Browser, it is probably best to use it on a Debian system, not Windows or Mac.

@ Debian Project: what's with that shared memory thingie for PulseAudio? Is that safe? And what's with that long unfixed critical glibc error in Synaptic? That can't be good, can it? Is RPC safe? And do we have good reason to trust systemd?

Currently it appears that one of the major hazards for at-risk people using Debian or Ubuntu is that these systems are hard to even install unless you include some risky and potentially dangerous but almost ubiquitous softwares (e.g. activity-tracking loggers, webcam "fun", unsafe chat softwares, poorly verified screensavers, insecure automatic "transparent" [invisible] bug reporting softwares, etc). And your hardware may have many unwanted and potentially dangerous capabilities (such as a radio transceiver or Firewire direct memory access) which most people cannot hope to disable.

Once alternatives such as Qubes becomes easier to install and use, a good compromise may be to

o install Debian on a dedicated "offline except for software updates via Tor" machine, for use with all your fancy (but unsafe) software for doing data journalism,

o install Qubes on a dedicated "exposed machine" for accessing information on the Internet with Tor Browser and Tor Messenger,

o use Tails (possibly on a third dedicated machine) for writing your most sensitive anonymous documents, accessing webmail, posting on activist websites, contacting other journalists/dissidents, etc.

Once consumer devices such as Faraday cage bags and spectrum analyzers become easily available to consumers, everyone should use these to further improve their computer security by guarding against extraneous EM emanations (e.g. in a laser printer or flat screen) and checking for covert RF channels or unwanted bluetooth connections (for example).

A recent news story highlights both the dangers of using Ubuntu and the danger posed by (well-intentioned but insecure) automatic bug reporting tools.

As I understand it, Debian uses different automatic bug reporting tools, but IMO Debian Project (and Tails Project?) should take a hard look at their own automatic bug-reporting tools.

Pleae see:

http://www.theregister.co.uk/2016/12/15/researcher_details_justpatched_…
Dear hackers, Ubuntu's app crash reporter will happily execute your evil code on a victim's box
To everyone else, get patching
15 Dec 2016
Shaun Nichols

> Users and administrators of Ubuntu Linux desktops are being advised to patch their systems following the disclosure of serious security flaws. Researcher Donncha O'Cearbhaill, who discovered and privately reported the vulnerabilities to the Ubuntu team, said that a successful exploit of the bugs could allow an attacker to remotely execute code by tricking a victim into downloading a maliciously booby-trapped file. The exploitable flaws are present in Ubuntu 12.10 and greater. He notes that while the security oversights are dangerous in their own right, the process of discovering and reporting the bugs opened his eyes to a wider problem affecting researchers working with open platforms. In this case, O'Cearbhaill says, his exploit code takes advantage of the Apport crash reporting tool on Ubuntu.
> ...

kata

December 04, 2016

Permalink

Also I think it's important to note that, like targeted delivery of a malicious package, Tor forces the adversary to deliver malicious data to all users during a downgrade attack. Usually a malicious package will be caught when it's signature is verified, but during a downgrade attack, the package is (was) a legitimate package so it's signature is good, except the server delivers an older version of the package (or hides the new one from the victim). Thus, the signature is good but the package is intentionally outdated; very useful for adversaries wanting to exploit vulnerabilities that were present in old packages but have since been patched. Tor means that everyone would receive the old version, making the attack much more detectable. Some package managers already protect against these kind of replay attacks using signed repository (as opposed to just package) files, but many do not.

> like targeted delivery of a malicious package, Tor forces the adversary to deliver malicious data to all users during a downgrade attack.

You meant "unlike", agreed?

I hear FBI hates this particular characteristic very much; they know they will eventually get caught red-handed if they wind up attacking everyone in order to try to catch a few. From the point of view of activists, of course, it is a very good thing that FBI cannot easily attack us without risking exposure of their crimes.

Tim Weiner has written two excellent histories of FBI and CIA, respectively, and recently authored an article at Esquire on Dir. Comey's troubles. Suddenly everyone is repeating what we said in this blog a few months ago, that Comey's chickens were about to come home to roost. That prediction seems to be coming true.

kata

December 04, 2016

Permalink

I'm definitely not trying to rain on anyone's parade here, just asking a question, but how is this different from 'torsocks '. Aside from .onion addresses, but it sounds like those are manually configured anyway.

I see how the .onion addresses do, but that's a matter of putting them in sources.list. They could then be used by 'torsocks apt-get ...' just the same. My question is what does apt-transport-tor do beyond what torsocks does exactly? Does it automatically enable the .onion addresses without the need to edit sources.list manually? Other than that, I can't think of anything besides configuring the proxy settings to use Tor's SocksPort.

Even so, anything to make those two things easier for the user is a win. I'm just wondering if Apt has some special superpowers over using other package managers with torsocks, or if it's more about convenience.

I think it's mainly convenience. The apt-transport-tor package is not much code, and that's a good thing.

That said, having your program be Tor-aware, rather than just trying to capture all the system calls with torsocks and hope for the best, can be useful. For example, in the newer versions of apt, it seems that they're doing DNS SRV lookups on the repository address before trying to visit it. I don't know that torsocks would necessarily know how to handle that -- if it intercepts the request and drops it, does apt get confused? If it allows it out anyway, that could undermine the privacy goals.

Now, to be fair, I think apt does not handle the DNS SRV requests well currently when faced with tor+http repositories. But if so that is a bug in apt-transport-tor, and it should be straightforward to fix. Whereas with torsocks, you're always wondering if the new version of your application added some feature that torsocks is going to have a tough time handling. apt isn't a huge program, so it's not a big deal here, but in the general case having Tor awareness in your program is way more fun than just blindly trying to torify it.

I think it's that it will do so in the future -- once you move to apt 1.1 or later. (Also, it's apt that does the leaking -- apt-transport-tor just registers some new transports, and does not yet, to my knowledge, fix this problem. Being tor-aware can be tough.)

I sent mail to the maintainers to ask them what they think, e.g. if there's already a bug like this filed.

Thanks, that answers my question. I know DNS is tricky business with Tor. I'm pretty sure torsocks is able to intercept calls like getaddrinfo(2) and gethostbyname(2), and tunnel them through Tor much like tor-resolve does. I think it's also able to do the same with .onion addresses by creating a process-local mapping of the .onion address to unused inet/inet6 address space. When it comes to more complicated tasks like getting a specific record (other than A/AAAA, like dig(1)), I'm not aware of any tools that are able to do that over Tor without a third-party service, though with enough effort I'm sure it's possible. I'll have to take a look at the apt-transport-tor code to see how they're doing it in this case.

kata

December 04, 2016

Permalink

When I used apt-transport-tor on Linux Mint, I had so much hash sum mismatches. What a bummer. Manually configuring apt's proxy in 95proxies in /etc/apt/apt.conf.d/ settings work much better for me without having to fix the problem as much.

kata

December 05, 2016

In reply to by Anonymous (not verified)

Permalink

I don't see why apt-transport-tor would cause a greater number of corrupt downloads as compared to configuring Apt's proxy settings to use Tor. If anything, using .onion addresses should cause fewer corrupt downloads, since the download is end-to-end encrypted at the transport layer. Can you give us more details, or can anyone comment on this?

kata

December 05, 2016

In reply to by Anonymous (not verified)

Permalink

OP here, maybe the problem is that those Linux Mint and Ubuntu packages location are in the clearnet and not in onion websites. Oh well, too bad for me.

kata

December 04, 2016

Permalink

About Debian -

Why haven't they updated Tor & Torbirdy for j-bp/stretch/sid yet? It's been over three days now.

kata

December 05, 2016

Permalink

i prefer that debian team erase their genuine source-list and replace it by onion address than doing by myself : it sounds like an experimental test from usa-tor foundation and not a real & sincere initiative/innovation coming from debian team/users _ was it a request from expert in security or another way to spread the world about tor ?

> it sounds like an experimental test from usa-tor foundation

It's a new feature offered by Debian Project in cooperation with Tor Project, and while it could be regarded as experimental its security features are so outstanding in comparison to http apt or even https apt that I think everyone should use it.

> and not a real & sincere initiative/innovation coming from debian team/users

Here is the official announcement (Aug 2016) from Debian of their new onion mirrors of the "main" repositories:

https://bits.debian.org/2016/08/debian-and-tor-services-available-as-on…

>> We, the Debian project and the Tor project are enabling Tor onion services for several of our sites. These sites can now be reached without leaving the Tor network, providing a new option for securely connecting to resources provided by Debian and Tor.
>> ...

This announcement was also posted in the Tor Project blog:

https://blog.torproject.org/blog/debian-and-tor-services-available-onio…

>>> We, the Debian project and the Tor project are enabling Tor onion services for several of our sites. These sites can now be reached without leaving the Tor network, providing a new option for securely connecting to resources provided by Debian and Tor.
>>> ...

So I think it's sincere.

> was it a request from expert in security or another way to spread the world about tor ?

Do you mean, was it a publicity stunt? I don't think so. DNS lookups and Certificate Authorities are horridly unsafe in the real world; using the onions is likely to fix the most serious problems.

All "sensitive" sites (e.g. banks, government sites) should sponsor Tor nodes and offer their services exclusively via onions. The cybersecurity situation is so awful that it seems there is a real chance that this might happen in a few years. Let's hope so. It will make Tor Project pretty invulnerable if Chase and the U.S. Social Security Administration and Dpt of Defense start saying they to run onion sites to protect their users.

Heck, even the FBI should give up attacking Tor and start promoting Tor in the interests of public safety. It might take a few years and a new Director, but I think they may get there in the end.

> DNS lookups and Certificate Authorities are horridly unsafe in the real world; using the onions is likely to fix the most serious problems.

A fabulous well-informed article reviewing the DigiNotar scandal of 2011 and the continuing problems of generally untrustworhty Certificate Authorities:

http://www.slate.com/articles/technology/future_tense/2016/12/how_the_2…
FROM SLATE, NEW AMERICA, AND ASU
How a 2011 Hack You’ve Never Heard of Changed the Internet’s Infrastructure
It all started with an internet user in Iran who couldn’t get into his Gmail account.
Josephine Wolff
21 Dec 2016

>> On Saturday, Aug. 27, 2011, an Iranian man who went by the online alias alibo tried to check his email—only to find he couldn’t connect to Gmail. Yet the problem disappeared when he connected to a virtual private network that disguised his location. Whatever was going on, it seemed to only affect computer users in Iran. His first hunch was that the problem might be somehow tied to the Iranian government—which was known for interfering with online activity—or a problem with his local internet service provider. So alibo posted a question about the issue on the Gmail Help Forum. Two days later, Google responded to this apparently small problem in a big way: It issued a public statement about the incident, attributing the problem to security issues at a Dutch company called DigiNotar. Within a month, DigiNotar had been taken over by the Dutch government. Not long after that, it declared bankruptcy and dissolved.

> I think apt should download packages over HTTPS in the first place.

I complained for years and was told it was a cost-saving measure.

To be fair, Debian has offered an https transport for APT for several years, but I could never make it work. The Tor transport does work for me, with some quibbles.

> It was actually surprising/shocking when I learned that it doesn't.

It would be even more shocking if APT relies on MD5 hashes to authenticate packages/

As far as I can tell (I am no coder so independent reading of the source code by a professional would be much appreciated), apt fetches a frequently updated file (signed with Debian's GPG archive signing key) which contains hashes for all the packages. Unfortunately, as far as I can tell, the hashes are MD5, which has been broken (by a practical collision attack allowing sophisticated actors to replace the valid package with a trojaned version having the same MD5 hash) for more than a decade (and probably broken by NSA even earlier). If so, SHA-256 (also from NSA, alas) would be a much better choice.

It would be even better to have individual packages signed using GPG, but the Debian repos are so enormous I have been told that would be impractical.

If I misunderstand anything, I would welcome correction from Debian Project.

It may be significant that the two types of failures I have often observed while using Tor transport are:

o failure to fetch the "Release" files (implicated in a recent dangerous attack on APT)

o failure to fetch the GPG signed file containing the package hashes (which leads me to abandon the update, in order to try again later which so far has fixed this issue).

@ Peter Palfrader: any comment?

kata

December 05, 2016

Permalink

gpg has an onion experimental address.
hkps can be set by a ca.pm , i tried and it does not work.
i conclude that even gpg is checked in http from the pool.hkp so the same.job.is done by onion.
the download package of debian are coming from a source-list - in http or ftp or onion - i do not know if it is possible in https -the security update has its own server.
<>So, it's strange...it is not at all strange because it is verified with a gpg key so it is very secure , far more than using https..

kata

December 12, 2016

Permalink

I hope all Debian users will adopt the new onions for their upgrades. (It would be helpful if the onions covered the entire repository, which is vast so I acknowledge this could be a challenge to set up.)

I also hope everyone using the Debian onions will watch for signs of possible attempts to block or interfere with the secure software fetches via the onions. If anything odd happens, stop the fetch and try again later.

Concerns about state sponsored hacking absolutely include the strong possibility of USG attacks; see the many news articles in Ars Technica and other tech news sites about the recent changes to Rule 41 which allow the FBI to freely attack any number of computers anywhere for any reason, without any need to prove "reasonable suspicion" of specific activities which would be criminal under increasing vague and broadly stated US cyberlaw. Many journalists and political dissidents around the world suspect, with good reason, that the real motivation for the changes to Rule 41 was not to attack botnets or child pron purveyors, but whistleblowers and dissidents living in authoritarian countries around the world (apparently the USA will be numbered among these, starting in the next prime numbered year in the Gregorian calendar).

kata

December 27, 2016

Permalink

>Doing updates via Tor provides some really compelling security properties. One of the big benefits is that an attacker can't target you for a malicious update: even if they manage to steal some package signing keys and break into the update server and replace packages, they still can't tell that it's you asking for the update. So if they want to be sure you get the malicious update, they're forced to attack everybody who updates, which means a really noisy attack that is much more likely to get noticed.

The dark side of it is the attacker's targets may be only the users using Tor. And an attacker can not to care about the noise because if anyone publish an unintended update it is already a noise.

kata

December 27, 2016

Permalink

Another question is if apt-transport-tor leaks metadata about the system? I'm sure it is since apt leaks its version..