tor

Installing and using Tor

phobos — Thu, 19 Nov 2009 18:31:30 -0800

Thanks to Rob at Freedom House for putting together some videos about how to get, install, and use Tor, Tor Browser Bundle, and Bridges.

Installing and Using Tor, http://tinyvid.tv/show/3lejztnthk2tm
Installing and Using the Tor Browser Bundle, http://tinyvid.tv/show/b0e2hzylie8r
Installing and Using Bridges with Tor, http://tinyvid.tv/show/3uiwckrlqynqv

Freedom House has put together other videos on various tools to use to stay secure online at, http://www.youtube.com/freedom4internet.

Check them out and leave constructive feedback. I'm sure Rob will appreciate help with translating these videos as well.

Tor 0.2.1.17-rc released

phobos — Sun, 12 Jul 2009 20:47:06 -0700

Tor 0.2.1.17-rc marks the fourth -- and hopefully last -- release
candidate for the 0.2.1.x series. It lays the groundwork for further
client performance improvements, and also fixes a big bug with directory
authorities that were causing them to assign Guard and Stable flags
poorly.

The Windows bundles also finally include the geoip database that we
thought we'd been shipping since 0.2.0.x (oops), and the OS X bundles
should actually install Torbutton rather than giving you a cryptic
failure message (oops).

This is a release candidate! That means that we don't know of any
remaining show-stopping bugs, and 0.2.1.18 will be the new stable if
there are no problems. Please test it, and tell us about any problems
that you find.

https://www.torproject.org/download

Changes in version 0.2.1.17-rc - 2009-07-02
Major features:

Clients now use the bandwidth values in the consensus, rather than
the bandwidth values in each relay descriptor. This approach opens
the door to more accurate bandwidth estimates once the directory
authorities start doing active measurements. Implements more of
proposal 141.

Major bugfixes:

When Tor clients restart after 1-5 days, they discard all their
cached descriptors as too old, but they still use the cached
consensus document. This approach is good for robustness, but
bad for performance: since they don't know any bandwidths, they
end up choosing at random rather than weighting their choice by
speed. Fixed by the above feature of putting bandwidths in the
consensus. Bugfix on 0.2.0.x.
Directory authorities were neglecting to mark relays down in their
internal histories if the relays fall off the routerlist without
ever being found unreachable. So there were relays in the histories
that haven't been seen for eight months, and are listed as being
up for eight months. This wreaked havoc on the "median wfu"
and "median mtbf" calculations, in turn making Guard and Stable
flags very wrong, hurting network performance. Fixes bugs 696 and
969. Bugfix on 0.2.0.6-alpha.

Minor bugfixes:

Serve the DirPortFrontPage page even when we have been approaching
our quotas recently. Fixes bug 1013; bugfix on 0.2.1.8-alpha.
The control port would close the connection before flushing long
replies, such as the network consensus, if a QUIT command was issued
before the reply had completed. Now, the control port flushes all
pending replies before closing the connection. Also fixed a spurious
warning when a QUIT command is issued after a malformed or rejected
AUTHENTICATE command, but before the connection was closed. Patch
by Marcus Griep. Bugfix on 0.2.0.x; fixes bugs 1015 and 1016.
When we can't find an intro key for a v2 hidden service descriptor,
fall back to the v0 hidden service descriptor and log a bug message.
Workaround for bug 1024.

Minor features:

If we're a relay and we change our IP address, be more verbose
about the reason that made us change. Should help track down
further bugs for relays on dynamic IP addresses.

Measuring Tor and Iran

phobos — Fri, 19 Jun 2009 01:35:08 -0700

I've been fielding some calls from the press about Tor and Iran. Someone quoted me as saying "double the clients from Iran over the past few days". We wondered, what are the real numbers? What does our network see from Iran? Is port 443 or https:// really blocked? Here's what we've discovered in the past day of working with the new metrics we've developed to be safe to collect without compromising anyone's anonymity.

This first dataset is from one of the Directory Authorities. We have six authorities, so a plausible scaling factor is 6, assuming all authorities are seeing equal requests (it could be more or less than 6, too). We don't know if the authorities are seeing equal requests, as they listen on different TCP ports, are located in different parts of the world, and clients will chose one randomly. This graph roughly shows the number of requests from new Tor clients coming from IP addresses that our geoip database reports as Iran.

UPDATE 2009-06-24: Updated the graph with numbers through yesterday

The second dataset is from one Directory Mirror, of which there are hundreds. This mirror is only accessible on port 443, which is rumored to be blocked in parts of Iran over the past few days.

The data points in the second graph are very rough, since they're an estimate of the total number of Tor users in Iran based on numbers from only one relay. In addition, we looked at some other relays running on port 443, and they also didn't show anywhere near the spike that we see in the directory authority graph above. The authority isn't listening on 443 -- perhaps that means there's some truth to the rumors that port 443 has been blocked recently in Iran. We look forward to having more precise data later on.

Tor 0.2.0.30 is released as stable

phobos — Mon, 25 Aug 2008 19:48:37 -0700

Tor 0.2.0.30 is released. A better formatted version of this report can be found at gmane.org

Tor 0.2.0.30 switches to a more efficient directory distribution design,
adds features to make connections to the Tor network harder to block,
allows Tor to act as a DNS proxy, adds separate rate limiting for relayed
traffic to make it easier for clients to become relays, fixes a variety
of potential anonymity problems, and includes the usual huge pile of
other features and bug fixes.

https://www.torproject.org/download.html

Changes in version 0.2.0.30 - 2008-07-15
o New v3 directory design:
- Tor now uses a new way to learn about and distribute information
about the network: the directory authorities vote on a common
network status document rather than each publishing their own
opinion. Now clients and caches download only one networkstatus
document to bootstrap, rather than downloading one for each
authority. Clients only download router descriptors listed in
the consensus. Implements proposal 101; see doc/spec/dir-spec.txt
for details.
- Set up moria1, tor26, and dizum as v3 directory authorities
in addition to being v2 authorities. Also add three new ones:
ides (run by Mike Perry), gabelmoo (run by Karsten Loesing), and
dannenberg (run by CCC).
- Switch to multi-level keys for directory authorities: now their
long-term identity key can be kept offline, and they periodically
generate a new signing key. Clients fetch the "key certificates"
to keep up to date on the right keys. Add a standalone tool
"tor-gencert" to generate key certificates. Implements proposal 103.
- Add a new V3AuthUseLegacyKey config option to make it easier for
v3 authorities to change their identity keys if another bug like
Debian's OpenSSL RNG flaw appears.
- Authorities and caches fetch the v2 networkstatus documents
less often, now that v3 is recommended.

o Make Tor connections stand out less on the wire:
- Use an improved TLS handshake designed by Steven Murdoch in proposal
124, as revised in proposal 130. The new handshake is meant to
be harder for censors to fingerprint, and it adds the ability
to detect certain kinds of man-in-the-middle traffic analysis
attacks. The new handshake format includes version negotiation for
OR connections as described in proposal 105, which will allow us
to improve Tor's link protocol more safely in the future.
- Enable encrypted directory connections by default for non-relays,
so censor tools that block Tor directory connections based on their
plaintext patterns will no longer work. This means Tor works in
certain censored countries by default again.
- Stop including recognizeable strings in the commonname part of
Tor's x509 certificates.

o Implement bridge relays:
- Bridge relays (or "bridges" for short) are Tor relays that aren't
listed in the main Tor directory. Since there is no complete public
list of them, even an ISP that is filtering connections to all the
known Tor relays probably won't be able to block all the bridges.
See doc/design-paper/blocking.pdf and proposal 125 for details.
- New config option BridgeRelay that specifies you want to be a
bridge relay rather than a normal relay. When BridgeRelay is set
to 1, then a) you cache dir info even if your DirPort ins't on,
and b) the default for PublishServerDescriptor is now "bridge"
rather than "v2,v3".
- New config option "UseBridges 1" for clients that want to use bridge
relays instead of ordinary entry guards. Clients then specify
bridge relays by adding "Bridge" lines to their config file. Users
can learn about a bridge relay either manually through word of
mouth, or by one of our rate-limited mechanisms for giving out
bridge addresses without letting an attacker easily enumerate them
all. See https://www.torproject.org/bridges for details.
- Bridge relays behave like clients with respect to time intervals
for downloading new v3 consensus documents -- otherwise they
stand out. Bridge users now wait until the end of the interval,
so their bridge relay will be sure to have a new consensus document.

o Implement bridge directory authorities:
- Bridge authorities are like normal directory authorities, except
they don't serve a list of known bridges. Therefore users that know
a bridge's fingerprint can fetch a relay descriptor for that bridge,
including fetching updates e.g. if the bridge changes IP address,
yet an attacker can't just fetch a list of all the bridges.
- Set up Tonga as the default bridge directory authority.
- Bridge authorities refuse to serve bridge descriptors or other
bridge information over unencrypted connections (that is, when
responding to direct DirPort requests rather than begin_dir cells.)
- Bridge directory authorities do reachability testing on the
bridges they know. They provide router status summaries to the
controller via "getinfo ns/purpose/bridge", and also dump summaries
to a file periodically, so we can keep internal stats about which
bridges are functioning.
- If bridge users set the UpdateBridgesFromAuthority config option,
but the digest they ask for is a 404 on the bridge authority,
they fall back to contacting the bridge directly.
- Bridges always use begin_dir to publish their server descriptor to
the bridge authority using an anonymous encrypted tunnel.
- Early work on a "bridge community" design: if bridge authorities set
the BridgePassword config option, they will serve a snapshot of
known bridge routerstatuses from their DirPort to anybody who
knows that password. Unset by default.
- Tor now includes an IP-to-country GeoIP file, so bridge relays can
report sanitized aggregated summaries in their extra-info documents
privately to the bridge authority, listing which countries are
able to reach them. We hope this mechanism will let us learn when
certain countries start trying to block bridges.
- Bridge authorities write bridge descriptors to disk, so they can
reload them after a reboot. They can also export the descriptors
to other programs, so we can distribute them to blocked users via
the BridgeDB interface, e.g. via https://bridges.torproject.org/
and bridges torproject.org.

o Tor can be a DNS proxy:
- The new client-side DNS proxy feature replaces the need for
dns-proxy-tor: Just set "DNSPort 9999", and Tor will now listen
for DNS requests on port 9999, use the Tor network to resolve them
anonymously, and send the reply back like a regular DNS server.
The code still only implements a subset of DNS.
- Add a new AutomapHostsOnResolve option: when it is enabled, any
resolve request for hosts matching a given pattern causes Tor to
generate an internal virtual address mapping for that host. This
allows DNSPort to work sensibly with hidden service users. By
default, .exit and .onion addresses are remapped; the list of
patterns can be reconfigured with AutomapHostsSuffixes.
- Add an "-F" option to tor-resolve to force a resolve for a .onion
address. Thanks to the AutomapHostsOnResolve option, this is no
longer a completely silly thing to do.

o Major features (relay usability):
- New config options RelayBandwidthRate and RelayBandwidthBurst:
a separate set of token buckets for relayed traffic. Right now
relayed traffic is defined as answers to directory requests, and
OR connections that don't have any local circuits on them. See
proposal 111 for details.
- Create listener connections before we setuid to the configured
User and Group. Now non-Windows users can choose port values
under 1024, start Tor as root, and have Tor bind those ports
before it changes to another UID. (Windows users could already
pick these ports.)
- Added a new ConstrainedSockets config option to set SO_SNDBUF and
SO_RCVBUF on TCP sockets. Hopefully useful for Tor servers running
on "vserver" accounts. Patch from coderman.

o Major features (directory authorities):
- Directory authorities track weighted fractional uptime and weighted
mean-time-between failures for relays. WFU is suitable for deciding
whether a node is "usually up", while MTBF is suitable for deciding
whether a node is "likely to stay up." We need both, because
"usually up" is a good requirement for guards, while "likely to
stay up" is a good requirement for long-lived connections.
- Directory authorities use a new formula for selecting which relays
to advertise as Guards: they must be in the top 7/8 in terms of
how long we have known about them, and above the median of those
nodes in terms of weighted fractional uptime.
- Directory authorities use a new formula for selecting which relays
to advertise as Stable: when we have 4 or more days of data, use
median measured MTBF rather than median declared uptime. Implements
proposal 108.
- Directory authorities accept and serve "extra info" documents for
routers. Routers now publish their bandwidth-history lines in the
extra-info docs rather than the main descriptor. This step saves
60% (!) on compressed router descriptor downloads. Servers upload
extra-info docs to any authority that accepts them; directory
authorities now allow multiple router descriptors and/or extra
info documents to be uploaded in a single go. Authorities, and
caches that have been configured to download extra-info documents,
download them as needed. Implements proposal 104.
- Authorities now list relays who have the same nickname as
a different named relay, but list them with a new flag:
"Unnamed". Now we can make use of relays that happen to pick the
same nickname as a server that registered two years ago and then
disappeared. Implements proposal 122.
- Store routers in a file called cached-descriptors instead of in
cached-routers. Initialize cached-descriptors from cached-routers
if the old format is around. The new format allows us to store
annotations along with descriptors, to record the time we received
each descriptor, its source, and its purpose: currently one of
general, controller, or bridge.

o Major features (other):
- New config options WarnPlaintextPorts and RejectPlaintextPorts so
Tor can warn and/or refuse connections to ports commonly used with
vulnerable-plaintext protocols. Currently we warn on ports 23,
109, 110, and 143, but we don't reject any. Based on proposal 129
by Kevin Bauer and Damon McCoy.
- Integrate Karsten Loesing's Google Summer of Code project to publish
hidden service descriptors on a set of redundant relays that are a
function of the hidden service address. Now we don't have to rely
on three central hidden service authorities for publishing and
fetching every hidden service descriptor. Implements proposal 114.
- Allow tunnelled directory connections to ask for an encrypted
"begin_dir" connection or an anonymized "uses a full Tor circuit"
connection independently. Now we can make anonymized begin_dir
connections for (e.g.) more secure hidden service posting and
fetching.

o Major bugfixes (crashes and assert failures):
- Stop imposing an arbitrary maximum on the number of file descriptors
used for busy servers. Bug reported by Olaf Selke; patch from
Sebastian Hahn.
- Avoid possible failures when generating a directory with routers
with over-long versions strings, or too many flags set.
- Fix a rare assert error when we're closing one of our threads:
use a mutex to protect the list of logs, so we never write to the
list as it's being freed. Fixes the very rare bug 575, which is
kind of the revenge of bug 222.
- Avoid segfault in the case where a badly behaved v2 versioning
directory sends a signed networkstatus with missing client-versions.
- When we hit an EOF on a log (probably because we're shutting down),
don't try to remove the log from the list: just mark it as
unusable. (Bulletproofs against bug 222.)

o Major bugfixes (code security fixes):
- Detect size overflow in zlib code. Reported by Justin Ferguson and
Dan Kaminsky.
- Rewrite directory tokenization code to never run off the end of
a string. Fixes bug 455. Patch from croup.
- Be more paranoid about overwriting sensitive memory on free(),
as a defensive programming tactic to ensure forward secrecy.

o Major bugfixes (anonymity fixes):
- Reject requests for reverse-dns lookup of names that are in
a private address space. Patch from lodger.
- Never report that we've used more bandwidth than we're willing to
relay: it leaks how much non-relay traffic we're using. Resolves
bug 516.
- As a client, do not believe any server that tells us that an
address maps to an internal address space.
- Warn about unsafe ControlPort configurations.
- Directory authorities now call routers Fast if their bandwidth is
at least 100KB/s, and consider their bandwidth adequate to be a
Guard if it is at least 250KB/s, no matter the medians. This fix
complements proposal 107.
- Directory authorities now never mark more than 2 servers per IP as
Valid and Running (or 5 on addresses shared by authorities).
Implements proposal 109, by Kevin Bauer and Damon McCoy.
- If we're a relay, avoid picking ourselves as an introduction point,
a rendezvous point, or as the final hop for internal circuits. Bug
reported by taranis and lodger.
- Exit relays that are used as a client can now reach themselves
using the .exit notation, rather than just launching an infinite
pile of circuits. Fixes bug 641. Reported by Sebastian Hahn.
- Fix a bug where, when we were choosing the 'end stream reason' to
put in our relay end cell that we send to the exit relay, Tor
clients on Windows were sometimes sending the wrong 'reason'. The
anonymity problem is that exit relays may be able to guess whether
the client is running Windows, thus helping partition the anonymity
set. Down the road we should stop sending reasons to exit relays,
or otherwise prevent future versions of this bug.
- Only update guard status (usable / not usable) once we have
enough directory information. This was causing us to discard all our
guards on startup if we hadn't been running for a few weeks. Fixes
bug 448.
- When our directory information has been expired for a while, stop
being willing to build circuits using it. Fixes bug 401.

o Major bugfixes (peace of mind for relay operators)
- Non-exit relays no longer answer "resolve" relay cells, so they
can't be induced to do arbitrary DNS requests. (Tor clients already
avoid using non-exit relays for resolve cells, but now servers
enforce this too.) Fixes bug 619. Patch from lodger.
- When we setconf ClientOnly to 1, close any current OR and Dir
listeners. Reported by mwenge.

o Major bugfixes (other):
- If we only ever used Tor for hidden service lookups or posts, we
would stop building circuits and start refusing connections after
24 hours, since we falsely believed that Tor was dormant. Reported
by nwf.
- Add a new __HashedControlSessionPassword option for controllers
to use for one-off session password hashes that shouldn't get
saved to disk by SAVECONF --- Vidalia users were accumulating a
pile of HashedControlPassword lines in their torrc files, one for
each time they had restarted Tor and then clicked Save. Make Tor
automatically convert "HashedControlPassword" to this new option but
only when it's given on the command line. Partial fix for bug 586.
- Patch from "Andrew S. Lists" to catch when we contact a directory
mirror at IP address X and he says we look like we're coming from
IP address X. Otherwise this would screw up our address detection.
- Reject uploaded descriptors and extrainfo documents if they're
huge. Otherwise we'll cache them all over the network and it'll
clog everything up. Suggested by Aljosha Judmayer.
- When a hidden service was trying to establish an introduction point,
and Tor *did* manage to reuse one of the preemptively built
circuits, it didn't correctly remember which one it used,
so it asked for another one soon after, until there were no
more preemptive circuits, at which point it launched one from
scratch. Bugfix on 0.0.9.x.

o Rate limiting and load balancing improvements:
- When we add data to a write buffer in response to the data on that
write buffer getting low because of a flush, do not consider the
newly added data as a candidate for immediate flushing, but rather
make it wait until the next round of writing. Otherwise, we flush
and refill recursively, and a single greedy TLS connection can
eat all of our bandwidth.
- When counting the number of bytes written on a TLS connection,
look at the BIO actually used for writing to the network, not
at the BIO used (sometimes) to buffer data for the network.
Looking at different BIOs could result in write counts on the
order of ULONG_MAX. Fixes bug 614.
- If we change our MaxAdvertisedBandwidth and then reload torrc,
Tor won't realize it should publish a new relay descriptor. Fixes
bug 688, reported by mfr.
- Avoid using too little bandwidth when our clock skips a few seconds.
- Choose which bridge to use proportional to its advertised bandwidth,
rather than uniformly at random. This should speed up Tor for
bridge users. Also do this for people who set StrictEntryNodes.

o Bootstrapping faster and building circuits more intelligently:
- Fix bug 660 that was preventing us from knowing that we should
preemptively build circuits to handle expected directory requests.
- When we're checking if we have enough dir info for each relay
to begin establishing circuits, make sure that we actually have
the descriptor listed in the consensus, not just any descriptor.
- Correctly notify one-hop connections when a circuit build has
failed. Possible fix for bug 669. Found by lodger.
- Clients now hold circuitless TLS connections open for 1.5 times
MaxCircuitDirtiness (15 minutes), since it is likely that they'll
rebuild a new circuit over them within that timeframe. Previously,
they held them open only for KeepalivePeriod (5 minutes).

o Performance improvements (memory):
- Add OpenBSD malloc code from "phk" as an optional malloc
replacement on Linux: some glibc libraries do very poorly with
Tor's memory allocation patterns. Pass --enable-openbsd-malloc to
./configure to get the replacement malloc code.
- Switch our old ring buffer implementation for one more like that
used by free Unix kernels. The wasted space in a buffer with 1mb
of data will now be more like 8k than 1mb. The new implementation
also avoids realloc();realloc(); patterns that can contribute to
memory fragmentation.
- Change the way that Tor buffers data that it is waiting to write.
Instead of queueing data cells in an enormous ring buffer for each
client->OR or OR->OR connection, we now queue cells on a separate
queue for each circuit. This lets us use less slack memory, and
will eventually let us be smarter about prioritizing different kinds
of traffic.
- Reference-count and share copies of address policy entries; only 5%
of them were actually distinct.
- Tune parameters for cell pool allocation to minimize amount of
RAM overhead used.
- Keep unused 4k and 16k buffers on free lists, rather than wasting 8k
for every single inactive connection_t. Free items from the
4k/16k-buffer free lists when they haven't been used for a while.
- Make memory debugging information describe more about history
of cell allocation, so we can help reduce our memory use.
- Be even more aggressive about releasing RAM from small
empty buffers. Thanks to our free-list code, this shouldn't be too
performance-intensive.
- Log malloc statistics from mallinfo() on platforms where it exists.
- Use memory pools to allocate cells with better speed and memory
efficiency, especially on platforms where malloc() is inefficient.
- Add a --with-tcmalloc option to the configure script to link
against tcmalloc (if present). Does not yet search for non-system
include paths.

o Performance improvements (socket management):
- Count the number of open sockets separately from the number of
active connection_t objects. This will let us avoid underusing
our allocated connection limit.
- We no longer use socket pairs to link an edge connection to an
anonymous directory connection or a DirPort test connection.
Instead, we track the link internally and transfer the data
in-process. This saves two sockets per "linked" connection (at the
client and at the server), and avoids the nasty Windows socketpair()
workaround.
- We were leaking a file descriptor if Tor started with a zero-length
cached-descriptors file. Patch by "freddy77".

o Performance improvements (CPU use):
- Never walk through the list of logs if we know that no log target
is interested in a given message.
- Call routerlist_remove_old_routers() much less often. This should
speed startup, especially on directory caches.
- Base64 decoding was actually showing up on our profile when parsing
the initial descriptor file; switch to an in-process all-at-once
implementation that's about 3.5x times faster than calling out to
OpenSSL.
- Use a slightly simpler string hashing algorithm (copying Python's
instead of Java's) and optimize our digest hashing algorithm to take
advantage of 64-bit platforms and to remove some possibly-costly
voodoo.
- When implementing AES counter mode, update only the portions of the
counter buffer that need to change, and don't keep separate
network-order and host-order counters on big-endian hosts (where
they are the same).
- Add an in-place version of aes_crypt() so that we can avoid doing a
needless memcpy() call on each cell payload.
- Use Critical Sections rather than Mutexes for synchronizing threads
on win32; Mutexes are heavier-weight, and designed for synchronizing
between processes.

o Performance improvements (bandwidth use):
- Don't try to launch new descriptor downloads quite so often when we
already have enough directory information to build circuits.
- Version 1 directories are no longer generated in full. Instead,
authorities generate and serve "stub" v1 directories that list
no servers. This will stop Tor versions 0.1.0.x and earlier from
working, but (for security reasons) nobody should be running those
versions anyway.
- Avoid going directly to the directory authorities even if you're a
relay, if you haven't found yourself reachable yet or if you've
decided not to advertise your dirport yet. Addresses bug 556.
- If we've gone 12 hours since our last bandwidth check, and we
estimate we have less than 50KB bandwidth capacity but we could
handle more, do another bandwidth test.
- Support "If-Modified-Since" when answering HTTP requests for
directories, running-routers documents, and v2 and v3 networkstatus
documents. (There's no need to support it for router descriptors,
since those are downloaded by descriptor digest.)
- Stop fetching directory info so aggressively if your DirPort is
on but your ORPort is off; stop fetching v2 dir info entirely.
You can override these choices with the new FetchDirInfoEarly
config option.

o Changed config option behavior (features):
- Configuration files now accept C-style strings as values. This
helps encode characters not allowed in the current configuration
file format, such as newline or #. Addresses bug 557.
- Add hidden services and DNSPorts to the list of things that make
Tor accept that it has running ports. Change starting Tor with no
ports from a fatal error to a warning; we might change it back if
this turns out to confuse anybody. Fixes bug 579.
- Make PublishServerDescriptor default to 1, so the default doesn't
have to change as we invent new directory protocol versions.
- Allow people to say PreferTunnelledDirConns rather than
PreferTunneledDirConns, for those alternate-spellers out there.
- Raise the default BandwidthRate/BandwidthBurst to 5MB/10MB, to
accommodate the growing number of servers that use the default
and are reaching it.
- Make it possible to enable HashedControlPassword and
CookieAuthentication at the same time.
- When a TrackHostExits-chosen exit fails too many times in a row,
stop using it. Fixes bug 437.

o Changed config option behavior (bugfixes):
- Do not read the configuration file when we've only been told to
generate a password hash. Fixes bug 643. Bugfix on 0.0.9pre5. Fix
based on patch from Sebastian Hahn.
- Actually validate the options passed to AuthDirReject,
AuthDirInvalid, AuthDirBadDir, and AuthDirBadExit.
- Make "ClientOnly 1" config option disable directory ports too.
- Don't stop fetching descriptors when FetchUselessDescriptors is
set, even if we stop asking for circuits. Bug reported by tup
and ioerror.
- Servers used to decline to publish their DirPort if their
BandwidthRate or MaxAdvertisedBandwidth were below a threshold. Now
they look only at BandwidthRate and RelayBandwidthRate.
- Treat "2gb" when given in torrc for a bandwidth as meaning 2gb,
minus 1 byte: the actual maximum declared bandwidth.
- Make "TrackHostExits ." actually work. Bugfix on 0.1.0.x.
- Make the NodeFamilies config option work. (Reported by
lodger -- it has never actually worked, even though we added it
in Oct 2004.)
- If Tor is invoked from something that isn't a shell (e.g. Vidalia),
now we expand "-f ~/.tor/torrc" correctly. Suggested by Matt Edman.

o New config options:
- New configuration options AuthDirMaxServersPerAddr and
AuthDirMaxServersperAuthAddr to override default maximum number
of servers allowed on a single IP address. This is important for
running a test network on a single host.
- Three new config options (AlternateDirAuthority,
AlternateBridgeAuthority, and AlternateHSAuthority) that let the
user selectively replace the default directory authorities by type,
rather than the all-or-nothing replacement that DirServer offers.
- New config options AuthDirBadDir and AuthDirListBadDirs for
authorities to mark certain relays as "bad directories" in the
networkstatus documents. Also supports the "!baddir" directive in
the approved-routers file.
- New config option V2AuthoritativeDirectory that all v2 directory
authorities must set. This lets v3 authorities choose not to serve
v2 directory information.

o Minor features (other):
- When we're not serving v2 directory information, there is no reason
to actually keep any around. Remove the obsolete files and directory
on startup if they are very old and we aren't going to serve them.
- When we negotiate a v2 link-layer connection (not yet implemented),
accept RELAY_EARLY cells and turn them into RELAY cells if we've
negotiated a v1 connection for their next step. Initial steps for
proposal 110.
- When we have no consensus, check FallbackNetworkstatusFile (defaults
to $PREFIX/share/tor/fallback-consensus) for a consensus. This way
we can start out knowing some directory caches. We don't ship with
a fallback consensus by default though, because it was making
bootstrapping take too long while we tried many down relays.
- Authorities send back an X-Descriptor-Not-New header in response to
an accepted-but-discarded descriptor upload. Partially implements
fix for bug 535.
- If we find a cached-routers file that's been sitting around for more
than 28 days unmodified, then most likely it's a leftover from
when we upgraded to 0.2.0.8-alpha. Remove it. It has no good
routers anyway.
- When we (as a cache) download a descriptor because it was listed
in a consensus, remember when the consensus was supposed to expire,
and don't expire the descriptor until then.
- Optionally (if built with -DEXPORTMALLINFO) export the output
of mallinfo via http, as tor/mallinfo.txt. Only accessible
from localhost.
- Tag every guard node in our state file with the version that
we believe added it, or with our own version if we add it. This way,
if a user temporarily runs an old version of Tor and then switches
back to a new one, she doesn't automatically lose her guards.
- When somebody requests a list of statuses or servers, and we have
none of those, return a 404 rather than an empty 200.
- Merge in some (as-yet-unused) IPv6 address manipulation code. (Patch
from croup.)
- Add an HSAuthorityRecordStats option that hidden service authorities
can use to track statistics of overall hidden service usage without
logging information that would be as useful to an attacker.
- Allow multiple HiddenServicePort directives with the same virtual
port; when they occur, the user is sent round-robin to one
of the target ports chosen at random. Partially fixes bug 393 by
adding limited ad-hoc round-robining.
- Revamp file-writing logic so we don't need to have the entire
contents of a file in memory at once before we write to disk. Tor,
meet stdio.

o Minor bugfixes (other):
- Alter the code that tries to recover from unhandled write
errors, to not try to flush onto a socket that's given us
unhandled errors.
- Directory mirrors no longer include a guess at the client's IP
address if the connection appears to be coming from the same /24
network; it was producing too many wrong guesses.
- If we're trying to flush the last bytes on a connection (for
example, when answering a directory request), reset the
time-to-give-up timeout every time we manage to write something
on the socket.
- Reject router descriptors with out-of-range bandwidthcapacity or
bandwidthburst values.
- If we can't expand our list of entry guards (e.g. because we're
using bridges or we have StrictEntryNodes set), don't mark relays
down when they fail a directory request. Otherwise we're too quick
to mark all our entry points down.
- Authorities no longer send back "400 you're unreachable please fix
it" errors to Tor servers that aren't online all the time. We're
supposed to tolerate these servers now.
- Let directory authorities startup even when they can't generate
a descriptor immediately, e.g. because they don't know their
address.
- Correctly enforce that elements of directory objects do not appear
more often than they are allowed to appear.
- Stop allowing hibernating servers to be "stable" or "fast".
- On Windows, we were preventing other processes from reading
cached-routers while Tor was running. (Reported by janbar)
- Check return values from pthread_mutex functions.
- When opening /dev/null in finish_daemonize(), do not pass the
O_CREAT flag. Fortify was complaining, and correctly so. Fixes
bug 742; fix from Michael Scherer. Bugfix on 0.0.2pre19.

o Controller features:
- The GETCONF command now escapes and quotes configuration values
that don't otherwise fit into the torrc file.
- The SETCONF command now handles quoted values correctly.
- Add "GETINFO/desc-annotations/id/≤OR digest>" so controllers can
ask about source, timestamp of arrival, purpose, etc. We need
something like this to help Vidalia not do GeoIP lookups on bridge
addresses.
- Allow multiple HashedControlPassword config lines, to support
multiple controller passwords.
- Accept LF instead of CRLF on controller, since some software has a
hard time generating real Internet newlines.
- Add GETINFO values for the server status events
"REACHABILITY_SUCCEEDED" and "GOOD_SERVER_DESCRIPTOR". Patch from
Robert Hogan.
- There is now an ugly, temporary "desc/all-recent-extrainfo-hack"
GETINFO for Torstat to use until it can switch to using extrainfos.
- New config option CookieAuthFile to choose a new location for the
cookie authentication file, and config option
CookieAuthFileGroupReadable to make it group-readable.
- Add a SOURCE_ADDR field to STREAM NEW events so that controllers can
match requests to applications. Patch from Robert Hogan.
- Add a RESOLVE command to launch hostname lookups. Original patch
from Robert Hogan.
- Add GETINFO status/enough-dir-info to let controllers tell whether
Tor has downloaded sufficient directory information. Patch from Tup.
- You can now use the ControlSocket option to tell Tor to listen for
controller connections on Unix domain sockets on systems that
support them. Patch from Peter Palfrader.
- New "GETINFO address-mappings/*" command to get address mappings
with expiry information. "addr-mappings/*" is now deprecated.
Patch from Tup.
- Add a new config option __DisablePredictedCircuits designed for
use by the controller, when we don't want Tor to build any circuits
preemptively.
- Let the controller specify HOP=%d as an argument to ATTACHSTREAM,
so we can exit from the middle of the circuit.
- Implement "getinfo status/circuit-established".
- Implement "getinfo status/version/..." so a controller can tell
whether the current version is recommended, and whether any versions
are good, and how many authorities agree. Patch from "shibz".
- Controllers should now specify cache=no or cache=yes when using
the +POSTDESCRIPTOR command.
- Add a "PURPOSE=" argument to "STREAM NEW" events, as suggested by
Robert Hogan. Fixes the first part of bug 681.
- When reporting clock skew, and we know that the clock is _at least
as skewed_ as some value, but we don't know the actual value,
report the value as a "minimum skew."

o Controller bugfixes:
- Generate "STATUS_SERVER" events rather than misspelled
"STATUS_SEVER" events. Caught by mwenge.
- Reject controller commands over 1MB in length, so rogue
processes can't run us out of memory.
- Change the behavior of "getinfo status/good-server-descriptor"
so it doesn't return failure when any authority disappears.
- Send NAMESERVER_STATUS messages for a single failed nameserver
correctly.
- When the DANGEROUS_VERSION controller status event told us we're
running an obsolete version, it used the string "OLD" to describe
it. Yet the "getinfo" interface used the string "OBSOLETE". Now use
"OBSOLETE" in both cases.
- Respond to INT and TERM SIGNAL commands before we execute the
signal, in case the signal shuts us down. We had a patch in
0.1.2.1-alpha that tried to do this by queueing the response on
the connection's buffer before shutting down, but that really
isn't the same thing at all. Bug located by Matt Edman.
- Provide DNS expiry times in GMT, not in local time. For backward
compatibility, ADDRMAP events only provide GMT expiry in an extended
field. "GETINFO address-mappings" always does the right thing.
- Use CRLF line endings properly in NS events.
- Make 'getinfo fingerprint' return a 551 error if we're not a
server, so we match what the control spec claims we do. Reported
by daejees.
- Fix a typo in an error message when extendcircuit fails that
caused us to not follow the \r\n-based delimiter protocol. Reported
by daejees.
- When tunneling an encrypted directory connection, and its first
circuit fails, do not leave it unattached and ask the controller
to deal. Fixes the second part of bug 681.
- Treat some 403 responses from directory servers as INFO rather than
WARN-severity events.

o Portability / building / compiling:
- When building with --enable-gcc-warnings, check for whether Apple's
warning "-Wshorten-64-to-32" is available.
- Support compilation to target iPhone; patch from cjacker huang.
To build for iPhone, pass the --enable-iphone option to configure.
- Detect non-ASCII platforms (if any still exist) and refuse to
build there: some of our code assumes that 'A' is 65 and so on.
- Clear up some MIPSPro compiler warnings.
- Make autoconf search for libevent, openssl, and zlib consistently.
- Update deprecated macros in configure.in.
- When warning about missing headers, tell the user to let us
know if the compile succeeds anyway, so we can downgrade the
warning.
- Include the current subversion revision as part of the version
string: either fetch it directly if we're in an SVN checkout, do
some magic to guess it if we're in an SVK checkout, or use
the last-detected version if we're building from a .tar.gz.
Use this version consistently in log messages.
- Correctly report platform name on Windows 95 OSR2 and Windows 98 SE.
- Read resolv.conf files correctly on platforms where read() returns
partial results on small file reads.
- Build without verbose warnings even on gcc 4.2 and 4.3.
- On Windows, correctly detect errors when listing the contents of
a directory. Fix from lodger.
- Run 'make test' as part of 'make dist', so we stop releasing so
many development snapshots that fail their unit tests.
- Add support to detect Libevent versions in the 1.4.x series
on mingw.
- Add command-line arguments to unit-test executable so that we can
invoke any chosen test from the command line rather than having
to run the whole test suite at once; and so that we can turn on
logging for the unit tests.
- Do not automatically run configure from autogen.sh. This
non-standard behavior tended to annoy people who have built other
programs.
- Fix a macro/CPP interaction that was confusing some compilers:
some GCCs don't like #if/#endif pairs inside macro arguments.
Fixes bug 707.
- Fix macro collision between OpenSSL 0.9.8h and Windows headers.
Fixes bug 704; fix from Steven Murdoch.
- Correctly detect transparent proxy support on Linux hosts that
require in.h to be included before netfilter_ipv4.h. Patch
from coderman.

o Logging improvements:
- When we haven't had any application requests lately, don't bother
logging that we have expired a bunch of descriptors.
- When attempting to open a logfile fails, tell us why.
- Only log guard node status when guard node status has changed.
- Downgrade the 3 most common "INFO" messages to "DEBUG". This will
make "INFO" 75% less verbose.
- When SafeLogging is disabled, log addresses along with all TLS
errors.
- Report TLS "zero return" case as a "clean close" and "IO error"
as a "close". Stop calling closes "unexpected closes": existing
Tors don't use SSL_close(), so having a connection close without
the TLS shutdown handshake is hardly unexpected.
- When we receive a consensus from the future, warn about skew.
- Make "not enough dir info yet" warnings describe *why* Tor feels
it doesn't have enough directory info yet.
- On the USR1 signal, when dmalloc is in use, log the top 10 memory
consumers. (We already do this on HUP.)
- Give more descriptive well-formedness errors for out-of-range
hidden service descriptor/protocol versions.
- Stop recommending that every server operator send mail to tor-ops.
Resolves bug 597. Bugfix on 0.1.2.x.
- Improve skew reporting: try to give the user a better log message
about how skewed they are, and how much this matters.
- New --quiet command-line option to suppress the default console log.
Good in combination with --hash-password.
- Don't complain that "your server has not managed to confirm that its
ports are reachable" if we haven't been able to build any circuits
yet.
- Detect the reason for failing to mmap a descriptor file we just
wrote, and give a more useful log message. Fixes bug 533.
- Always prepend "Bug: " to any log message about a bug.
- When dumping memory usage, list bytes used in buffer memory
free-lists.
- When running with dmalloc, dump more stats on hup and on exit.
- Put a platform string (e.g. "Linux i686") in the startup log
message, so when people paste just their logs, we know if it's
OpenBSD or Windows or what.
- When logging memory usage, break down memory used in buffers by
buffer type.
- When we are reporting the DirServer line we just parsed, we were
logging the second stanza of the key fingerprint, not the first.
- Even though Windows is equally happy with / and \ as path separators,
try to use \ consistently on Windows and / consistently on Unix: it
makes the log messages nicer.
- On OSX, stop warning the user that kqueue support in libevent is
"experimental", since it seems to have worked fine for ages.

o Contributed scripts and tools:
- Update linux-tor-prio.sh script to allow QoS based on the uid of
the Tor process. Patch from Marco Bonetti with tweaks from Mike
Perry.
- Include the "tor-ctrl.sh" bash script by Stefan Behte to provide
Unix users an easy way to script their Tor process (e.g. by
adjusting bandwidth based on the time of the day).
- In the exitlist script, only consider the most recently published
server descriptor for each server. Also, when the user requests
a list of servers that _reject_ connections to a given address,
explicitly exclude the IPs that also have servers that accept
connections to that address. Resolves bug 405.
- Include a new contrib/tor-exit-notice.html file that exit relay
operators can put on their website to help reduce abuse queries.

o Newly deprecated features:
- The status/version/num-versioning and status/version/num-concurring
GETINFO controller options are no longer useful in the v3 directory
protocol: treat them as deprecated, and warn when they're used.
- The RedirectExits config option is now deprecated.

o Removed features:
- Drop the old code to choke directory connections when the
corresponding OR connections got full: thanks to the cell queue
feature, OR conns don't get full any more.
- Remove the old "dns worker" server DNS code: it hasn't been default
since 0.1.2.2-alpha, and all the servers are using the new
eventdns code.
- Remove the code to generate the oldest (v1) directory format.
- Remove support for the old bw_accounting file: we've been storing
bandwidth accounting information in the state file since
0.1.2.5-alpha. This may result in bandwidth accounting errors
if you try to upgrade from 0.1.1.x or earlier, or if you try to
downgrade to 0.1.1.x or earlier.
- Drop support for OpenSSL version 0.9.6. Just about nobody was using
it, it had no AES, and it hasn't seen any security patches since
2004.
- Stop overloading the circuit_t.onionskin field for both "onionskin
from a CREATE cell that we are waiting for a cpuworker to be
assigned" and "onionskin from an EXTEND cell that we are going to
send to an OR as soon as we are connected". Might help with bug 600.
- Remove the tor_strpartition() function: its logic was confused,
and it was only used for one thing that could be implemented far
more easily.
- Remove the contrib scripts ExerciseServer.py, PathDemo.py,
and TorControl.py, as they use the old v0 controller protocol,
and are obsoleted by TorFlow anyway.
- Drop support for v1 rendezvous descriptors, since we never used
them anyway, and the code has probably rotted by now. Based on
patch from Karsten Loesing.
- Stop allowing address masks that do not correspond to bit prefixes.
We have warned about these for a really long time; now it's time
to reject them. (Patch from croup.)
- Remove an optimization in the AES counter-mode code that assumed
that the counter never exceeded 2^68. When the counter can be set
arbitrarily as an IV (as it is by Karsten's new hidden services
code), this assumption no longer holds.
- Disable the SETROUTERPURPOSE controller command: it is now
obsolete.

July 2008 Progress Report

phobos — Sun, 17 Aug 2008 20:11:31 -0700

Releases:

Torbutton 1.2.0rc5 (released July 6) provides improved addon compatibility, better preservation of Firefox preferences that we touch, fixing issues with Tor toggle breaking for some option combos, and an improved 'Restore Defaults' button. This version also features Firefox 3 cookie jar support, and support for storing cookie jars in memory.
http://archives.seul.org/or/talk/Jul-2008/msg00026.html

Vidalia 0.1.6 (released July 8) fixes a bug introduced in 0.1.3 that could cause excessive CPU usage or crashing on some platforms; continues to prepare Vidalia's strings for easier translation; adds a Romanian GUI and installer translation; and updated the Farsi, Finnish, French, German, and Swedish translations.
http://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.1.6/CHANG...

Tor 0.2.0.29-rc (released July 8) fixes two big bugs with using bridges, fixes more hidden-service performance bugs, and fixes a bunch of smaller bugs.
http://archives.seul.org/or/talk/Jul-2008/msg00038.html

Torbutton 1.2.0rc6 (released July 12) features fixes for a nasty history loss bug, an exception during Tor toggle, javascript being disabled in some tabs, better pref handling, and more.
http://archives.seul.org/or/talk/Jul-2008/msg00049.html

Tor 0.2.0.30 (released July 15) is the first stable release of the 0.2.0.x branch. The previous stable branch (0.1.2.x) went stable in April of 2007. We are still waiting for Torbutton and Vidalia to stabilize before announcing the Windows and OS X packages on the or-announce announcements
list. We expect to do that in August.

Tor Browser Bundle 1.1.1 (released July 20) updates Vidalia to release 0.1.6, updates Pidgin Portable to 2.4.3, updates Pidgin OTR plugin to 3.2, updates Tor to 0.2.1.2-alpha, updates Torbutton to 1.2.0rc6, and sets TZ=UTC environment variable in RelativeLink (needed by Torbutton).
https://svn.torproject.org/svn/torbrowser/trunk/README

Torbutton 1.2.0 (released July 30) is finally a stable release for the new Torbutton tree that includes application-level privacy protections.
https://svn.torproject.org/svn/torbutton/trunk/src/CHANGELOG

From the Tor 0.2.0.29-rc ChangeLog:
"When a hidden service was trying to establish an introduction point, and Tor had built circuits preemptively for such purposes, we were ignoring all the preemptive circuits and launching a new one instead. Bugfix on 0.2.0.14-alpha."
"When a hidden service was trying to establish an introduction point, and Tor *did* manage to reuse one of the preemptively built circuits, it didn't correctly remember which one it used, so it asked for another one soon after, until there were no more preemptive circuits, at which point it launched one from scratch. Bugfix on 0.0.9.x."

The upcoming Tor 0.2.1.3-alpha and 0.2.1.4-alpha releases include more fixes for hidden service performance and robustness, have slightly improved bootstrap status event behavior, and start hunting down a horrible bug that looks like it could leak private information:
https://bugs.torproject.org/flyspray/index.php?do=details&id=779

Proposals:

Proposal 145 (Separate "suitable as a guard" from "suitable as a new guard") suggests one approach for separating the role of "is still useful as an entry guard" from "should be an option when choosing a new entry guard". This step will help us load balance over the network better.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/145-newguard...

Proposal 146 (Add new flag to reflect long-term stability) discusses how to ship the Tor client with a set of alternate sources for initial bootstrap directory information. We already have this feature in Tor 0.2.0.x, called the "fallback consensus", but we never enabled it because the Tor client would spend too long trying directory mirrors that were long since gone from the network. This proposal moves us closer to being able to distinguish the more long-term reliable mirrors.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/146-long-ter...

Proposal 147 (Eliminate the need for v2 directories in generating v3 directories) helps wean us off of needing the old deprecated v2 directory design. Currently we only use it to give advance warning to the v3 authorities about relays that haven't heard about yet, so they can fetch information about those relays before the time arrives to make an official vote about their state.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/147-prevotin...

Proposal 148 (Stream end reasons from the client side should be uniform) describes a simple fix for a potential anonymity flaw in Tor's core protocol for passing explanations from one end of a Tor circuit to the other when an application stream ends.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/148-uniform-...

Proposal 149 (Using data from NETINFO cells) starts talking about how to make use of the timestamp and IP address listed in Tor's new NETINFO cells. In theory we can use them to decide if our clock is skewed, and to decide if a traffic analysis man-in-the-middle attack is happening against us. In practice it appears more complex than we expected.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/149-using-ne...

Proposal 150 (Exclude Exit Nodes from a circuit) allows users to specify which relays should never be used as the last (exit) hop in a circuit. We took the proposal one step further and allowed users to also specify IP addresses and netmasks for which relays to avoid in the exit position.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/150-exclude-...

Proposal 151 (Improving Tor Path Selection) is a draft proposal to implement the results of Fallon Chen's Google Summer of Code project. Her plan is to measure the expected time it takes to establish a circuit, and then abandon circuits that take significantly longer than that to form. The assumption is that circuits that take a long time to set up will generally have unacceptably high latency as well.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/151-path-sel...

Proposal 154 (Automatic Software Update Protocol) starts the discussion of how to let Vidalia automatically manage updates for Tor, Polipo, Vidalia, etc. This is very important for keeping users up to date with respect to security and stability fixes. We will especially aim to do the updates over Tor, a) for privacy, and b) so users who are blocked from the Tor website will still be able to upgrade seamlessly.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/154-automati...

Research:

Karsten Loesing's report on 7 ways to improve the performance and robustness of Tor hidden services:
http://freehaven.net/~karsten/hidserv/discussion-2008-07-15.pdf

Four new research papers on Tor came out in July:
http://freehaven.net/anonbib/#loesing2008performance
http://freehaven.net/anonbib/#improved-clockskew
http://freehaven.net/anonbib/#mccoy-pet2008
http://freehaven.net/anonbib/#danezis-pet2008

We continued evaluating the TBB footprints here:
https://svn.torproject.org/svn/torbrowser/trunk/docs/traces.txt

In particular, we added a new "Registry modifications" section to that file, describing some new traces that appear to be left behind after operating Tor Browser Bundle, even from the USB key. One of the most worrying is the "user assist" registry key that gets set, and (incredible as it sounds) is obfuscated by rot-13 before being set.

Ease of Use:

The first Incognito (Gentoo-based Tor LiveCD) release of 2008 is also nearing completion, and we expect to see it released in August.

Finally, we contracted to start work on the Tor VM project. The idea is to run a Linux kernel and a Tor client inside a thin VM (like QEMU) on Windows, and then transparently intercept outgoing connections and redirect them into Tor. This approach will a) make proxy-avoiding side-channel and sidejacking attacks less devastating, and b) isolate the Tor client from the rest of the OS to provide a more robust security approach. Current design document is under development at
https://svn.torproject.org/svn/torvm/trunk/doc/design.html

Getting Tor:

We have established our "gettor" email auto-responder script that lets people mail gettor@torproject.org and retrieve a copy of Tor from their mailbox. We still need to ponder more usability issues, such as translation.
https://www.torproject.org/finding-tor

We have also automated the process of checking Tor website mirrors: there's a new update-mirrors.pl script in the website directory that generates a list of mirrors ordered by when they last synced with the main website.
https://www.torproject.org/mirrors

Translations:

We have our translation server up and online:
https://translation.torproject.org/

We revised our translation tutorial here:
https://www.torproject.org/translation-portal

Users continued to submit updated translations for many different languages.

We continued enhancements to the Chinese and Russian Tor website
translations. We added Vidalia, Torbutton, and website translations
into Farsi.

We also added the strings for Vidalia's installer; this required writing several scripts to convert from the "nsh" (nullscript installer language) format to the "po" (preferred by Pootle) format and back.

June 2008 Progress Report

phobos — Tue, 22 Jul 2008 20:25:34 -0700

Torbutton 1.2.0rc1 (released June 1), the first release candidate for the next stable series of the security-enhanced Torbutton Firefox extension, features functional support for Firefox 3. However, this support has not been extensively tested. In particular, timezone masking does not work at all. The workaround is to manually set the environment variable 'TZ' to 'UTC' before starting Firefox. This works on both Linux and Windows:
http://archives.seul.org/or/talk/Jun-2008/msg00044.html

Tor 0.2.0.27-rc (released June 3) adds a few features we left out of the earlier release candidates. In particular, we now include an IP-to-country GeoIP database, so controllers can easily look up what country a given relay is in, and so bridge relays can give us some sanitized summaries about which countries are making use of bridges. (See proposal 126-geoip-fetching.txt for details.)
http://archives.seul.org/or/talk/Jun-2008/msg00055.html

Torbutton 1.2.0rc2 (released June 8) features a fix for an annoying bug on MacOS, and adds much clamored for options to start Firefox in a specific Tor state:
http://archives.seul.org/or/talk/Jun-2008/msg00103.html

Tor 0.2.0.28-rc (released June 13) fixes an anonymity-related bug, fixes a hidden-service performance bug, and fixes a bunch of smaller bugs.
http://archives.seul.org/or/talk/Jun-2008/msg00165.html

Tor 0.2.1.1-alpha (released June 13) fixes a lot of memory fragmentation problems that were making the Tor process bloat especially on Linux; makes our TLS handshake blend in better; sends "bootstrap phase" status events to the controller, so it can keep the user informed of progress (and problems) fetching directory information and establishing circuits; and adds a variety of smaller features. http://archives.seul.org/or/talk/Jun-2008/msg00185.html

Vidalia 0.1.4 (released June 13) adds a bootstrap progress bar, UPnP support, a new set of freely licensed GUI icons, and fixes a few bugs.
http://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.1.4/CHANG...

The Tor Browser Bundle 1.1.0 (released June 13) replaces startup batch script with application (RelativeLink) so there is a helpful icon, optionally installs Pidgin (for Tor IM Browser Bundle), optionally uses WinRAR to produce a self-extracting split bundle, and includes upgraded versions of Tor, Vidalia, and Torbutton.
https://svn.torproject.org/svn/torbrowser/trunk/README

Tor 0.2.1.2-alpha (released June 20) includes a new "TestingTorNetwork" config option to make it easier to set up your own private Tor network; fixes several big bugs with using more than one bridge relay; fixes a big bug with offering hidden services quickly after Tor starts; and uses a better API for reporting potential bootstrapping problems to the controller.
http://archives.seul.org/or/talk/Jun-2008/msg00247.html

Vidalia 0.1.5 (released June 21) switches Vidalia's internal string representation so it can use the new Pootle-based translation system.
http://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.1.5/CHANG...

Torbutton 1.2.0rc3 and 1.2.0rc4 (both released June 27) provide improved addon compatibility, better preservation of Firefox preferences that we touch, fixing issues with Tor toggle breaking for some option combos, and an improved 'Restore Defaults' button.
https://torbutton.torproject.org/dev/CHANGELOG

We finally got around to writing down the details of many of our architecture and technical design changes:

Proposal 137 ("Keep controllers informed as Tor bootstraps") modifies Tor so it keeps Vidalia informed of each "bootstrap phase" -- that is, progress Tor makes at learning directory information, making connections to the network, etc. Now Vidalia has a progress bar on Tor startup that explains what's going on. Further, Tor reports "bootstrap problems" when it believes it's having troubles starting up correctly, and Vidalia can now tell the user. All of this is in as of the Tor 0.2.1.2-alpha release (June 20).
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/137-bootstra...

Proposal 138 ("Remove routers that are not Running from consensus documents") modifies the directory "networkstatus consensus" documents so they no longer list relays that are believed to be unusable. They used to list these relays so clients could decide for themselves, but in practice clients just ignored them. This change saves 30% to 40% in download bandwidth for consensus documents. It is included in the 0.2.1.2-alpha release (June 20).
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/138-remove-d...

Proposal 139 ("Download consensus documents only when it will be trusted") tries to make Tor clients better handle the case when new directory authorities have been added to the system, or when directory authorities have changed (for example, this could happen if we have another bug like the one in May that caused us to change keys for half the directory authorities). Now clients specify which directory authorities they trust, so the directory mirrors can give them a consensus document they'll be willing to use. This change is included in Tor 0.2.1.1-alpha, and a bugfix on it was included in Tor 0.2.1.2-alpha.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/139-conditio...

Proposal 140 ("Provide diffs between consensuses") is still under development, but is scheduled to be included in the Tor 0.2.1.x tree. The idea is that most parts of the consensus document don't change from one hour to the next, so we can give clients a diff on the previous one rather than a whole new document, changing the size of the document every client must download every few hours from 92KB on average to 13KB on average.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/140-consensu...

Proposal 141 ("Download server descriptors on demand") is still under discussion, and may not be ready until for inclusion until Tor 0.2.2.x. This is the more detailed version of our "grand scaling plan" first mentioned in April. The idea is to have clients download networkstatus consensus documents as they do now, but rather than preemptively fetching every relay descriptor just in case, they fetch descriptors "just in time" only when they need them. The trick is to keep the bandwidth overhead low while not introducing too many new anonymity attacks e.g. due to leaking which relays you're picking.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/141-jit-sd-d...

We've instrumented a Tor client to collect stats on how much bandwidth we use now for directory overhead and how much we'd save with this new approach:
http://archives.seul.org/or/dev/Jun-2008/msg00024.html

Proposals 142 ("Combine Introduction and Rendezvous Points") and 143 ("Improvements of Distributed Storage for Tor Hidden Service Descriptors") are still in the discussion phase. Their goal is to improve the experience for clients accessing Tor hidden services, both by making the handshake faster and by making hidden service reachability more reliable and more robust.
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/142-combine-...
https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/143-distribu...

The "spoofing Firefox cipher suites and extensions" features are now in the Tor 0.2.1.1-alpha release, meaning they're in the Tor Browser Bundle 1.1.0 release also. From the 0.2.1.1-alpha ChangeLog:
"More work on making our TLS handshake blend in: modify the list of ciphers advertised by OpenSSL in client mode to even more closely resemble a common web browser. We cheat a little so that we can advertise ciphers that the locally installed OpenSSL doesn't know about."

We've done some initial security auditing (though there's always room for more, and we plan to do some more concrete auditing in July).

Nick also wrote some early thoughts on doing pass-through to an Apache server to improve scanning resistance:
http://archives.seul.org/or/dev/Jun-2008/msg00014.html

We also looked into running two Firefoxes in parallel:
https://svn.torproject.org/svn/torbrowser/trunk/docs/two-firefox.txt
and we even hacked in some Torbutton fixes that will come out in version 1.2.0rc3 that should get us closer:
http://archives.seul.org/or/cvs/Jun-2008/msg00213.html

Speaking of which, we also hacked in another feature in Torbutton 0.1.2rc2, to add a "locked" mode so Tor Browser Bundle can start Torbutton and not fear that the user will click and disable Tor. I believe TBB 1.1.0 doesn't use this feature yet though.
http://archives.seul.org/or/cvs/Jun-2008/msg00186.html

From the Tor 0.2.1.2-alpha ChangeLog:
"If you have more than one bridge but don't know their digests, you would only learn a request for the descriptor of the first one on your list. (Tor considered launching requests for the others, but found that it already had a connection on the way for $0000...0000 so it didn't open another.) Bugfix on 0.2.0.x."
"If you have more than one bridge but don't know their digests, and the connection to one of the bridges failed, you would cancel all pending bridge connections. (After all, they all have the same digest.) Bugfix on 0.2.0.x."
"If you're using bridges, generate "bootstrap problem" warnings as soon as you run out of working bridges, rather than waiting for ten failures -- which will never happen if you have less than ten bridges."

We put up a new webpage to describe bridges, how to fetch bridge relay addresses, etc:
https://www.torproject.org/bridges

We also modified the BridgeDB database (that is, the server that runs https://bridges.torproject.org/ and answers mail to bridges@torproject.org) to autodetect if the address hitting https://bridges.torproject.org/ is currently a Tor exit relay, and if so to treat it specially -- that is, we reserve a set of bridge addresses and give those out only to folks coming in over Tor.

The updated BridgeDB version now makes sure to give out at least one bridge that's listed as Stable in the bridge authority's networkstatus document, and at least one bridge that listens on port 443. The goal here is to increase the odds that at least one of the bridges we give the user will be usable even if he's in a tightly firewalled situation.

From the Tor 0.2.0.27-rc ChangeLog:
"Include an IP-to-country GeoIP file in the tarball, so bridge relays can report sanitized summaries of the usage they're seeing."

We finished work on a patch for OpenSSL that will make it keep less buffer space around. Currently fast Tor relays use (waste) as much as 100M of memory in OpenSSL's buffers. This patch was accepted and included in the main OpenSSL tree in June:
http://marc.info/?l=openssl-cvs&m=121246471627426&w=2

The Vidalia 0.1.4 release has folded the UPnP library and GUI changes into the main Vidalia tree, along with a "test" button to try speaking UPnP at the local router and tell the user whether it worked; these features will be available by default in the 0.2.0.x stable release.

We've put a lot of effort into reducing Tor's memory footprint again. The main issue was a "memory fragmentation" problem in Linux's memory allocator, which was causing Tor servers on Linux to slowly grow without bound. As of Tor 0.2.1.2-alpha, the issue appears to be substantially better. Many more details are here:
http://archives.seul.org/or/dev/Jun-2008/msg00001.html

From the Tor 0.2.1.2-alpha ChangeLog:
"New TestingTorNetwork config option to allow adjustment of previously constant values that, while reasonable, could slow bootstrapping. Implements proposal 135. Patch from Karsten Loesing."
"When building a consensus, do not include routers that are down. This will cut down 30% to 40% on consensus size. Implements proposal 138."

We've added clear user-oriented instructions for the Tor Browser Bundle split-download page:
https://www.torproject.org/torbrowser/split.html.en

We're starting work on a "gettor" email auto-responder script that will let people mail gettor@torproject.org and retrieve a copy of Tor from their mailbox. More info forthcoming in July.

More generally, we have a new https://www.torproject.org/finding-tor page that describes various mechanisms such as mirrors.

In July we plan to deploy a more automated mechanism for tracking which Tor mirrors are up-to-date.

We have our translation server up and online:
https://translation.torproject.org/

We have imported the strings from Vidalia, Torbutton, and Torcheck, and we currently have active translations for Spanish, French, German, Italian, Polish, Romanian, Swedish, Turkish, Finnish, Russian, Chinese, and Arabic.

We have a more useful overall translation tutorial here:
https://www.torproject.org/translation-portal

And we have internal documentation here for how to deal with the translation stuff behind the scenes:
https://svn.torproject.org/svn/tor/trunk/doc/translations.txt

In July we plan to add the strings for Vidalia's installer; the challenge is that we need to write a script to convert from the "nsh" (nullscript installer language) format to the "po" (preferred by Pootle) format and back.

In July we also expect to see the first version of our "wml to po and back" conversion tool, that will allow us to start putting our website pages into the translation server.

Incognito and The Tor Project sign a licensing agreement

phobos — Fri, 27 Jun 2008 14:42:44 -0700

Incognito is an open source LiveDistro assisting you to securely and anonymously use the Internet almost anywhere you go. Incognito can be used from either a CD or a USB drive and has several Internet applications (Web browser, IRC client, Mail client, Instant messenger, etc.) pre-configured with security in mind, and all Internet traffic will be anonymized.

At the core of this anonymity is the Tor™ software and network. In recognition of the transparency, open source base, continued development, and improvement of the Incognito software, The Tor Project is proud to list Incognito as a licensee of the Tor brands.

Incognito has the right to use the Tor name and the Tor onion logo™ as needed. The high quality graphics will improve the user experience. The usage of the Tor brand will only further reinforce that Incognito is a legitimate solution using the Tor software.

We welcome the further cooperation and collaboration between Incognito and The
Tor Project.

May 2008 Progress Report

phobos — Tue, 24 Jun 2008 20:39:17 -0700

Tor 0.2.0.26-rc (released May 13) fixes a major security vulnerability caused by a bug in Debian's OpenSSL packages. All users running any 0.2.0.x version should upgrade, whether they're running Debian or not.
http://archives.seul.org/or/talk/May-2008/msg00048.html

Vidalia 0.1.3 (released May 25) adds a hidden service configuration UI designed and implemented by Domenik Bork, as well as a few other bugfixes.
http://trac.vidalia-project.net/browser/vidalia/tags/vidalia-0.1.3/CHANG...

The Tor Browser Bundle 1.0.2 (released May 3) and 1.0.3 (released May 16) include upgraded versions of Tor, Vidalia, Torbutton, and Firefox.

We added three new part-time developers in May. We hired Matt Edman as a part-time employee at the beginning of May, to work on Vidalia maintenance, bugfixes, and new features. We also are funding Karsten Loesing to work on making hidden service rendezvous and interaction faster, and Peter Palfrader to work on lowering the overhead of directory requests, especially during bootstrap, which should directly improve the experience for Tor users on modems or cell phones.

Google has agreed to give us some funding to work on auto-update for Windows. Our plan is for Vidalia to look at the majority-signed network status consensus to decide when to update and to what version (Tor already lists what versions are considered safe, in each network status document). We should actually do the update via Tor if possible, for additional privacy, and we need to make sure to check package signatures to ensure package validity. Last, we need to give the user an interface for these updates, including letting her opt to migrate from one major Tor version to the next.

We continued enhancements to the Chinese and Russian Tor website translations. Vidalia also added a Turkish translation.

From the Vidalia 0.1.3 ChangeLog:
"If we're running Tor >= 0.2.0.13-alpha, then check the descriptor annotations for each descriptor before deciding to do a geoip lookup on its IP address. If the annotations indicate it is a special purpose descriptor (e.g., bridges), then don't do the lookup at all."

"Remove the 'Run Tor as a Service' checkbox. Lots of people seem to be clicking it even though they don't really need to, and we end up leaving them in a broken state after a reboot."

"Only display the running relays in the big list of relays to the left of the network map. Listing a big pile of unavailable relays is not particularly useful, and just clutters up the list."

We worked toward a Torbutton 1.2.0rc1 release candidate, which will include support for Firefox 3 along with a huge pile of privacy-related bugfixes.

We spent much of the first half of May dealing with a surprise massive security vulnerability in a crypto library that comes with Debian:
http://archives.seul.org/or/announce/May-2008/msg00000.html

You can read a more detailed explanation of the effects of the flaw here:
https://blog.torproject.org/blog/debian-openssl-flaw%3A-what-does-it-mea...

Part of dealing with the flaw meant doing some quick design work so we could let new Tor users be safe without making it so old Tor users were cut off from the network:
https://www.torproject.org/svn/trunk/doc/spec/proposals/136-legacy-keys....

Sometime in late June or early July we will disable this workaround, meaning all the 0.2.0.x users who haven't upgraded yet will be cut off.

We are preparing for the Tor gathering at the Privacy Enhancing Technologies Symposium in Leuven in July. This is looking like it will be the largest physical gathering of Tor developers ever -- main developers attending include Roger Dingledine, Nick Mathewson, Jacob Appelbaum, Mike Perry, Matt Edman, Steven Murdoch, and Karsten Loesing; Tor researchers include Paul Syverson and Ian Goldberg; and we'll have 5 of our 7 Google Summer of Code students there as well.
https://blog.torproject.org/events/roger%2C-nick%2C-steven%2C-matt%2C-ka...
http://petsymposium.org/2008/program.php

The upcoming TBB release in June will include optional instant messaging support via Pidgin + Off-The-Record Messaging; replace the startup batch script with an actual application (named RelativeLink), so TBB now has a helpful Tor icon rather than an ugly batch file icon; and optionally support using WinRAR to produce a self-extracting split bundle.

We now have a more thorough set of TBB build instructions:
https://svn.torproject.org/svn/torbrowser/trunk/build-scripts/INSTALL

We also documented the build and deploy process for a new TBB version:
https://svn.torproject.org/svn/torbrowser/trunk/build-scripts/DEPLOYMENT

We finished integrating a UPnP library into Vidalia. This feature allows users who want to set up a Tor relay but don't want to muck with manual port forwarding on their router/firewall to just click a button and have Vidalia interact with their router/firewall automatically. This approach won't work in all cases, but it should work in at least some. The upcoming Vidalia 0.1.4 (scheduled for June) release has folded the UPnP library and GUI changes into the main Vidalia tree, along with a "test" button to try speaking UPnP at the local router and tell the user whether it worked; these features will be available by default in the 0.2.0.x stable release.

We spent May hunting for a better online translation option, since Launchpad (intended to be used for Vidalia translation) has an ugly interface and can't handle our file formats well, and Babelzilla (intended to be used for Torbutton translation) artificially limited the number of concurrent translators we could have.

In early June we hit upon Pootle, which is a translation server that we host, as opposed to a shared web service that other organizations host. We've set up a test server at http://translation.torproject.org/ and imported strings for Vidalia, Torbutton, and Torcheck. We hope to have a lot more to show here in June.

Google funds an auto-update for Vidalia

phobos — Fri, 06 Jun 2008 19:13:30 -0700

Google is funding a project to create an auto-update feature in Vidalia. This auto-update feature will provide a better user experience for Tor users. The goal is to create a system where Vidalia can detect when a new release is available, fetch the package, verify authenticity, and assist the user in upgrading the Vidalia/Tor package. The auto-update feature preserves the user's privacy and anonymity. Over the next six months we'll develop the auto-update system for general release around November 15, 2008.

We're excited to work with Google on this project and look forward to the collaboration.

Jacob and Matt join the Tor Project

phobos — Thu, 29 May 2008 13:36:47 -0700

Jacob Appelbaum joins us to help out with:

developing a translation portal. This should help us find translators
and make their updates easier.
coordinating the Tor translation team and getting parts that need
translating, translated.
helping to better document Tor for non-technical users.
writing an auto-responder to use Google's gmail to deliver Tor to
users who request it
helping to get auto-updating for Tor and Vidalia working seamlessly
maintaining the code that runs the tor exitlist
generally advocating Tor

Matt Edman joins the Tor Project. Matt joins to help us enhance Tor's
interactions with Vidalia. Specifically, he's working on:

integrating upnp libraries into vidalia to make it easier to setup servers
displaying Tor's startup status more visually in Vidalia to help users
understand what's going on as Tor starts
assist with making translating Vidalia's interface and help files
easier for translators
helping to flesh out proposals in queue on or-dev
helping to get auto-updating or Tor and Vidalia working seamlessly
tackling the "matt" section of the TODO file.

Welcome Jacob and Matt!