controller

Exploring Tor with carml

carml is a command-line, pipe-friendly tool for exploring and controlling a running Tor daemon. Most of the sub-commands will be interesting to developers and tinkerers; a few of these will be interesting to end users. This post concentrates on the developers and tinkerers.

carml is a Python program written using Twisted and my library txtorcon. If you're familiar with Python, create a new virtualenv and pip install carml. There are more verbose install instructions available. Once this works, you should be able to type carml and see the help output.

Connecting to Tor

carml works somewhat like git, in that a normal invocation is carml followed by some global options and then a sub-command with its own options. The most-useful global option is --connect <endpoint> which tells carml how to connect to the control-port. Technically this can be any Twisted client endpoint-string but for Tor will be one of tcp:<port> (or simply a port) or unix:/var/run/tor/control for a unix-socket.

For Tor Browser Bundle, use carml --connect 9151. Typically a "system" Tor is reachable at carml --connect 9051 or carml --connect unix:/var/run/tor/control. You may need to enable the control-port in the configuration and re-load (or re-start) Tor. More details are in the documentation.

Start Exploring

The most interesting general purpose command is probably carml monitor -- try running it for a while and you can see what your Tor client is doing. This gives some good insight into Tor behavior.

A (very basic) usage graph is available via carml graph to see what bandwidth you're using (this needs work on the scaling -- PRs welcome!)

Explicit Circuits

Sometimes, you want to use a particular circuit. For example, you're trying to confirm some possibly-nefarious activity of an Exit. We can combine the carml circ and carml stream commands:

carml circ --build "*,*,4D08D29FDE23E75493E4942BAFDFFB90430A81D2"

This means make a 3-hop circuit through any entry-guard, any middle and then one particular exit (identified by ID). You can*= identify via name (only if it's unique!) but hashes are highly recommended. Of course, you could explicitly choose the other hops as well. Note that the stars still leave the selection up to carml / txtorcon which cannot (and does not) use Tor's exact selection algorithm.

Next, you'll want to actually attach circuits to that stream. It will have printed out something like "Circuit ID 1234". Now we can use carml stream:

carml stream --attach 1234

This will cause all new streams to be attached to circuit 1234 (until we exit the carml stream command). In another terminal, try torsocks curl https://www.torproject.org to visit Tor Project's web site via your new circuit. Once you kill the above carml stream command, Tor will select circuits via its normal algorithm once again.

Note that it's not currently possible to attach streams destined for onion services (this is a Tor limitation, see connection_edge.c).

Debugging Tor

The control protocol reveals all Tor events, which includes INFO and DEBUG logging events. This allows you to easily turn on DEBUG and INFO logging via the carml events command:

carml events INFO DEBUG

This can of course be piped through grep or anything else. You can give a --count to carml events, which is useful for some of the other events.

For example, if you want to "do something" every time a new consensus document is published, you could do this:

carml events --once NEWCONSENSUS

This will wait until exactly one NEWCONSENSUS event is produced, dump the contents of it to stdout (which will be the new consensus) and exit. Using a bash script that runs the above (maybe piped to /dev/null) you can ensure a new consensus is available before continuing.

Events that Tor emits are documented in torspec section 4.1. You can use carml to list them, with carml events --list.

Another example might be that you want to ensure your relay is still listed in the consensus every hour. One way would be to schedule a cron-job shortly before the top of each hour which does something like:

carml events --once NEWCONSENSUS | grep 
# log something useful if grep didn't find anything

Raw Commands

You can issue a raw control-port command to Tor via the carml cmd sub-command. This takes care of authentication, etc. and exits when the command succeeds (or errors). This can be useful to test out new commands under development etc (as the inputs / outputs are not in any way validated).

Every argument after cmd is joined back together with spaces before being sent to Tor so you don't have to quote things.

carml cmd getinfo info/names
carml cmd ADD_ONION NEW:BEST Port=1234

End-User Commands

Briefly, the commands intended to be "end-user useful" are:

carml pastebin: create a new hidden service and serve a directory, single file, or stdin at it. You can combine with carml copybin or simply torsocks curl ... on the other side. Still an "exercise to the reader" to securely distribute the address.

carml tbb: download, verify and run a new Tor Browser Bundle. This pins the public-key of torproject.org and bundles the keys of likely suspects that sign the bundles. It is less useful now that TBB auto-updates.

carml newid: sends the NEWNYM signal, which clears the DNS cache and causes Tor to not re-use any existing circuits for new requests.

carml monitor shows you what Tor is doing currently. Similarly, carml graph shows you just the current in/out bandwidth.

Pure Entertainment

Commands that can provide hours of entertainment include:

  • carml xplanet
  • carml tmux

I hope you find carml useful. Suggestions, bugs, and fixes all welcome on carml's GitHub page.

See Also

There is also a curses-based Tor tool called ARM (blog post). This is being re-written as "Nyx" currently.

TorCtl Deprecation and Stem Plans

Hi all. Just a friendly heads up concerning a couple things going on in our python controller space.

The first is that Mike and I have decided to deprecate TorCtl and make it a part of TorFlow (the framework used to support the Bandwidth Authorities and SoaT). The TorCtl codebase has largely been frozen for years out of concern for the stability of the Bandwidth Authorities (which lack any tests).

If you're writing scripts or controller applications for Tor then you're encouraged to move to either...

  • Stem

    Library with a similar design to TorCtl, but friendlier APIs and documentation. This has reached feature parity with TorCtl and is still being actively developed, so if there's something it can do to better suit your needs then please let me know!

  • Txtorcon

    Twisted controller library written by Meejah, and used in projects like Ooni Probe.

Both of these libraries have extensive test suites and are being very actively maintained.


The second part are my plans regarding Stem. As of early December we've reached feature completion, covering just about everything in the control-spec and dir-spec.

Next up is migrating our controllers. So far we've moved arm (the largest python controller we have) and the consensus-tracker. Other controllers we have queued up to move are TorBEL, Tor Weather, and the control interpretor.

I've avoided making an initial release announcement for stem because until we have actual users of the library we won't be sure that we nailed a nice, intuitive API (and hence, can't promise that it'll be frozen).

On reflection this is letting the perfect be the enemy of the good. Stem's API is unlikely to change substantially, and holding off on an initial release poses a chicken-and-egg situation. Users want a frozen API before using stem, but we need users before feeling confident enough to lock down the API.

So here's what I propose. For the next couple months stem will have an open beta. If you'd like to have input on the future of our python controller space then please give Stem a try and tell me the following...

  • What pain points did you encounter? Is there anything that you'd like to see changed or that we're missing?

  • If your project is public then please tell me where I can find your code. I'll review it, both to suggest improvements and see how we can tweak stem to better suit your needs.

In the unlikely event we make a backward incompatible change I'll check with the beta participants to be sure we don't break anyone (and submit fixes if we do).

Cheers! -Damian (atagar on irc)

PS. Many thanks to Ravi, Sean, Eoin, Beck, Erik, Megan, Sathyanarayanan, and everyone else who has helped stem get to this point. Happy New Year!

Syndicate content Syndicate content