KIST and Tell: Tor’s New Traffic Scheduling Feature

by tommy | October 3, 2017

 

The latest Tor alpha release includes a new feature to address traffic congestion in the Tor network. The new algorithm —Kernel Informed Socket Transport (KIST)— prevents connections between Tor relays from becoming overwhelmed by changing how traffic is distributed throughout the Tor network.

Our Relays Got KIST

The previous design often meant too much data was being written to each Tor relay connection, which would overwhelm relays and lead to traffic delays. KIST, on the other hand, intelligently considers how to write data across all connections to other relays in a way that allows traffic to pass through the network more quickly. Clients can run KIST, but the benefits accrue when it’s widely used by relays. Currently, KIST is only available on Linux-like systems because of how they handle TCP information, but a variant, KISTlite, runs on all systems.

In a study which measured the amount of congestion in Tor (both at individual relays and along the entire circuit path), researchers led by Dr. Rob Jansen found that, on average, KIST “reduces circuit congestion by over 30 percent, reduces network latency by 18 percent, and increases network throughput by nearly 10 percent.” You can read more about the technical details in the two technical papers.

Latency matters for a bunch of reasons. We’re often asked why Tor Browser is slower than other browsers, and congestion is a big culprit. The improvements from implementing KIST will benefit those who use Tor for regular web browsing -- websites will load faster, and the Tor network will be able to scale more easily. Diversity in the types of people using Tor makes it harder to do traffic analysis, and so these improvements strengthen Tor as privacy-enhancing software and make users safer.

The KIST algorithm also enables Tor to better prioritize low-volume traffic over high-volume traffic, effectively giving precedence to Tor web and chat traffic over people who use Tor for massive downloads. (We discourage this use of Tor: running BitTorrent or similar applications over Tor strains the network for those who rely on it, and doesn’t give the anonymity people expect of it.)

We Could Use Your Help 

Tor is configured to use KIST as the default connection scheduler when available, but it’s still in early stages, so we could use your help. We’re interested in seeing how KIST runs in the wild, so if you’re technically savvy and run a relay, you can help out by running the latest alpha version of Tor and filing a bug report (or hopping on IRC) if you notice any unusual behavior. As ever, Tor relies on dedicated volunteers, so if you want to protect internet freedom, please consider running a relay or making a tax-deductible donation to Tor.

Comments

Please note that the comment area below has been archived.

October 03, 2017

Permalink

it enhances tor traffic but do not give us a better protection against hacking or survey.
it runs on the 'server'side (relays) but what could be the settings _on the user side_ to be more 'near' of KIST ?
$ running a relay or making a donation : yes but another solution could be provide us some advice about configuring or hardening better in compliance with the kist-model no ?
no, Tor is not slow & latency means it retains information locked until it is unblocked (if i well understood the principle) ...
run a relay or make a donation : yes do it.

> it runs on the 'server'side (relays) but what could be the settings _on the user side_ to be more 'near' of KIST ?

I'm not sure I understand what you're asking. But maybe this answers your question: KIST runs on clients too. It just makes the most improvement when run on relays.

KIST is in the Tor source code. All you need to do if you want to run a Tor process that has KIST in it is update to the latest alpha version of Tor.

> another solution could be provide us some advice about configuring or hardening better in compliance with the kist-model no ?

You don't need to change or configure anything in order to have a Tor client/relay running KIST. Right now you need to be using an alpha version of Tor. Someday soon it'll be in regular stable Tor.

October 03, 2017

In reply to pastly

Permalink

1 mistake : it runs on the server side until you provide a kist tor version (alpha ? no ! i will wait a stable version !).
2 good news :
"You don't need to change or configure anything in order to have a Tor client/relay running KIST. "
"Someday soon it'll be in regular stable Tor."
have a god day.

October 03, 2017

Permalink

I have read in the paper that KIST makes traffic correlation a little bit easier but that adding some kind of padding remedies that, will this padding be implemented?

Does netflow padding mitigate the issues caused by KIST at all? I thought it was very specifically about exploiting the default timeout parameters in netflow and most netflow-like applications, not size or traffic padding.

Unlike something like an ISP where they can just throw money at the problem to make it go away but prefer to throttle traffic, having a quality scheduling algorithm in Tor improves things for everybody, and is one of the only ways to do so. In the end, this even benefits people who are downloading, because the Tor network as a whole will be able to better handle traffic.

This is about changing behavior on a network-scale to improve performance. Little changes in each relay improves things a lot. This is not about simply detecting a big download and throttling it to oblivion.

October 05, 2017

Permalink

> so if you want to protect internet freedom, please consider ...
"support the work , the recruitment of our dedicated volunteers by running a relay or making a tax-deductible donation to Tor" could be a better argument.

'protect internet freedom' is prohibited here (where i live) and means something like a terrorist action , a conspiracy. About kist , i wonder what will be the riposte of the isp.
The reason why pc, laptop, mini, rapspberry are cheaper & cheaper is simple ; the product is beginning to be obsolete & useless running in a no neutral:free network.
kist is maybe opening the next independent road but for how long time ?

October 06, 2017

Permalink

Tor & kist can't do nothing against that :
https://theintercept.com/2017/09/23/police-schedule-7-uk-rabbani-gchq-p…
It sounds that traveling in the e.u. [uk has its brexit=no, fr has its referendum=no but still apply their own *illegal* rules which are e.u & u.s laws giving at few elected person (rajoy_ spain included) a personal illegitimate power] carrying an usb key or laptop is dangerous_encrypted or not !

How could one spread the world about internet freedom or foss, gpg, net neutrality, tor if one cannot travel, work, play with a p.c. ?

Kist/Tor will become soon a tool for in a reserved small network which users will have to stay at home.
Tor is beginning to be a professional service for registered users , a government labelled , designed from an university mind for supporting their future background & standing.

Support Tor , Make a donation , Run a relay ?
They seize things & body and target unknown live like protected product at home like at the frontiers.
Kist is a nice improvement & Tor a nice tool but with a p.c never in the hands of its owner (which his mental health , his the physical integrity , his money ) ; it becomes a project outside of the scoop of a normal user : it is for police force exclusively.

Even using an "invisible" road, running by night, without light, if they know where you start where you arrive whom you speak to and what you say:write/send:receive ; Tor can help you only if you are working on the oppression side ... but it is not anymore for so many people/client _in danger or not_ it is a hidden circuit for public/military force (court, jail, airport, mental hospital included) even if most of it are not public enterprises.

It is not new & Tor is still open for everybody yet but i am afraid that 'anybody' & 'anonymous' user will not be welcome in a near future , traveler or not : it is time for innovation .

Why do you say that? Tor browser is based on Firefox, and there will always be exploitable bugs in large browsers like that. If you simply meant to say that Tor browser does not provide a comfortable level of exploit mitigation, I would agree with you, but saying "any more" implies that it used to be safe and now is no longer. The truth is that the browser is only improving in security. Could it be better? Yes. Is it getting less safe? No, quite the opposite.

Please don't spread FUD without a solid understanding of application exploitation.

October 08, 2017

Permalink

Oct 08 22:46:54.338 [notice] Tor 0.3.2.2-alpha (git-e2a2704f17415d8a) running on FreeBSD with Libevent 2.1.8-stable, OpenSSL 1.0.2l, Zlib 1.2.11, Liblzma 5.2.3, and Libzstd 1.3.1.

If, when i try new style onion addres, e,g,. http://ozmh2zkwx5cjuzopui64csb5ertcooi5vya6c2gm4e3vcvf2c2qvjiyd.onion/

I get this:
Oct 08 23:23:02.000 [warn] Could not pick a hidden service directory for the requested hidden service: they are all either down or excluded, and StrictNodes is set.
Oct 08 23:23:02.000 [warn] conn_read_callback(): Bug: Unhandled error on read for Socks connection (fd 11); removing (on Tor 0.3.2.2-alpha e2a2704f17415d8a)
Oct 08 23:23:02.000 [warn] tor_bug_occurred_(): Bug: src/or/main.c:744: conn_read_callback: This line should not have been reached. (Future instances of this warning will be silenced.) (on Tor 0.3.2.2-alpha e2a2704f17415d8a)
Oct 08 23:23:02.000 [warn] Bug: Line unexpectedly reached at conn_read_callback at src/or/main.c:744. Stack trace: (on Tor 0.3.2.2-alpha e2a2704f17415d8a)
Oct 08 23:23:02.000 [warn] Bug: 0x119ffa5 at /usr/local/bin/tor (on Tor 0.3.2.2-alpha e2a2704f17415d8a)
Oct 08 23:23:02.000 [warn] Bug: 0x11ba6bd at /usr/local/bin/tor (on Tor 0.3.2.2-alpha e2a2704f17415d8a)
Oct 08 23:23:02.000 [warn] Bug: 0x106e7ea at /usr/local/bin/tor (on Tor 0.3.2.2-alpha e2a2704f17415d8a)
Oct 08 23:23:02.000 [warn] connection_ap_about_to_close(): Bug: Closing stream (marked at src/or/main.c:748) without sending back a socks reply. (on Tor 0.3.2.2-alpha e2a2704f17415d8a)

I suppose it may be because of this ExcludeNodes {GB},{FR},{??},{be},F21DE9C7DE31601D9716781E17E24380887883D1,C0192FF43E777250084175F4E59AC1BA2290CE38,{US},{CA}?
It actually still works just throws a lot of errors out.
Whats the deal with bridges almost always being a service dedicated to traffic analyisis?

October 09, 2017

Permalink

ANY WAY FOR TOR TO IDENTIFY SPOOFED TOR EXIT NODES?

IS TEHER ANY WAY TO ENSURE TOR TO DOWNLOAD LARGE FILES PDF? OR SIMILAR?

IS THERE ANY WAY FOR TOR TO TERMINATE MALICIOUS PROCESSES ON YOUR CPU TO TO RUN IN A SPECIAL VIRTUAL DOMAIN LIKE QUBES OS?

There's no need for the caps. I'll try to answer your three questions as best as I understand them.

1) By spoofed exit nodes I assume you mean malicious exits? There is no way the Tor software can directly determine that, but HTTPS Everywhere in the browser makes it harder to take advantage of being able to modify traffic in an exit node. The Tor Project does attempt to identify malicious exit relays when they are reported, or detected with certain (questionably efficient) detection tools, and they quickly send an update to the consensus file which causes Tor clients to stop using the suspected bad relays. If you suspect a certain relay is malicious, please read https://blog.torproject.org/how-report-bad-relays and follow the instructions.

2) I don't know what you mean by this. Are you asking how to allow the browser to download PDF files rather than open them in the browser? You simply right-click the link and select "save as". That will give you the option to save it to your computer, rather than viewing it directly.

3) Tor is not an antivirus or host intrusion detection system, and it was never supposed to be. Tor does, at least on Linux, have a sandbox which runs it in a "special domain". Unlike Qubes, this does not rely on virtualization, but on a security tool called mode 2 seccomp. Mode 2 seccomp filters syscalls, the vocabulary that software uses to communicate with the core of the operating system, so only trusted ones are permitted. By filtering syscalls, it makes it harder for someone who is attacking the Tor process to break out of the sandbox by exploiting the lower levels of your operating system. If someone has a 0day in the syscall called vm86(), for example, they will not be able to use it because Tor blocks that syscall. This is a general class of techniques termed "attack surface area reduction".

Qubes is designed primarily to protect from certain hardware attacks like compromised network cards. It is not particularly good at protecting from threats within, such as a compromised browser, because the isolation technology it uses (Xen) has rather questionable security in the way it has been configured for Qubes. If you want to isolate Tor, use something like Whonix (ideally with real hardware, not a virtual machine), so you get both network isolation and the sandboxing provided by seccomp. Tails also provides an easy-to-use environment with Tor, sandboxed with an additional technology called AppArmor.

October 09, 2017

Permalink

internt proviersders can regular tor downloads for some reason.

what this mean si sthat isp can block downloads from tor use like downloading a file or something related.

> what this mean si sthat isp can block downloads from tor use like downloading a file or something related.

Such non-specific information is not useful. However, I suspect that you may simply have misunderstood what you saw. Experienced Tor users know it is not uncommon to experience problems downloading large files using Tor Browser. Often such issues can be "fixed" by using Tails and running

torify wget -c url

in a directory where Tails allows downloads. Such a procedure can perhaps be risky but some users may decide to assume the estimated risk in some cases.

October 18, 2017

Permalink

New update in mozilla software is coming - Mozilla Quantum does that mean that Tor will transfer on new Quantum software or is it still remaining to be seen?

Ty