The DigiNotar Debacle, and what you should do about it

Recently it has come to the attention of, well, nearly the entire world that the Dutch Certificate Authority DigiNotar incorrectly issued certificates to a malicious party or parties. Even more recently, it's come to light that they were apparently compromised months ago or perhaps even in May of 2009 if not earlier.

This is pretty unfortunate, since correctly issuing certificates is exactly the function that a certificate authority (CA) is supposed to perform. By comparison, ComodoGate looks fairly minor.

This incident doesn't affect the functionality of Tor clients or the Tor Network itself, since Tor doesn't use the flawed CA system. The Tor network uses a much simpler and flatter trust design that protects us from many of these CA issues. Further, Tor's distributed-trust design limits the damage from compromise of any given network component.

But the incident does affect users that are attempting to reach The Tor Project's infrastructure: with one of these bogus certificates, an attacker could convince your browser that you were talking to The Tor Project website, when really you were talking to the attacker.

We have taken direct action in an attempt to stop this kind of attack in the future with two major browser vendors and we hope to integrate a fix with all other willing browsers. Please contact us if you ship a browser and you'd like to help your users to be proactively secure when visiting our sites. TorBrowser users should upgrade to our latest release and verify signatures for all downloaded files. All Tor Browser Bundles have been updated to Firefox 6 with a patch to stop trusting the offending CA, and users are encouraged to upgrade. Below, we describe what we found out, what we're doing about it, and what you should do to keep yourself safe.

The attack

In the last seventy-two hours we were working to find positive confirmation that The Tor Project was one of the targeted groups. It was originally disclosed that at least one certificate was issued for '*.google.com' and that it was being used to actively Man-In-The-Middle SSL and TLS connections. Quite quickly we found a similar pattern to the ComodoGate fiasco. It appears likely that the Mozilla Addons site, Yahoo, Facebook, Twitter, and a few other major players were targeted. We do not have an authoritative list but I personally believe those targets to be accurate; time will tell. Additionally, we heard rumors that we had graduated to the big leagues and we had also heard that DigiNotar had reached out to the major browser vendors. We did not receive any proactive contact from DigiNotar as a browser vendor and this worried us greatly when compounded with the rumor of being one of the targets as well.

We ship a rather specific and special browser and it appeared that all of our sites are specifically in the attacker's target list. Having received no contact from DigiNotar, we reached out to DigiNotar by email and by telephone.

I spoke on the telephone with a rather nice but obviously overworked DigiNotar point of contact who will remain anonymous. He was guarded and careful in what he said but was clearly sympathetic to the severity of the matter at hand. It seemed quite clear that he repeated similar information to other impacted callers:

"What I can say is the following " ... "Any fraudulent torproject.org certificate that has been requested has been revoked. Any serial numbers that we know about have been revoked. All serial numbers have been communicated to the major browsers vendors." ... "Any certificate that we know of is revoked by OCSP server."

We emailed quite a bit back and forth after the phone call. A few hours later that same point of contact from DigiNotar sent a list of all of the certificates in a spreadsheet. It appears that the attackers requested twelve certificates, and each certificate was for '*.torproject.org'. The first batch of six certificates was issued on July 18th and the second batch of six certificates was issued on July 20th. According to the spreadsheet, the first six of the certificates expired on August 17th, 2011 and second batch of six certificates expired on August 19th, 2011. According to the information disclosed by DigiNotar the certificates in question should all have expired. The contact at DigiNotar stressed that there was no confirmation about the attacker(s) receiving the certificates. I have no reason to believe that these certificates would have any more trouble reaching the requesting party than the Google certificate used in the wild.

This is the current list of serial numbers for all twelve Tor Project certificates as disclosed to us by DigiNotar:

899AE120CD44FCEC0FFCD62F6FC4BB81
7DD16C03DF0438B2BE5FC1D3E19F138B
5432FC98141883F780897BC829EB9080
73024E7C998B3DDD244CFD313D5E43B6
B01D8C6F2D5373EABF0C00319E92AE95
FF789632B8D4AECD94A0AAB33074A058
86633B957280BC65A5ADFD1D153BDE52
E7F58683066112DC5EB244FCF208E850
1A07D8D6DDC7E623E71205074A05CEA2
79C8E8B7DE36539FFC4B2B5825305324
06CBB1CC51156C6D465F14829453DD68
ED1A1008190A5D1654D138EB8FD1154A

DigiNotar has not provided us with a copy of any of the certificates that they issued. We are not sure that they have copies nor if they are willing to disclose any copies they may or may not have. This point is extremely disconcerting as the CRL/OCSP revocation process is essentially worthless. Mere serial numbers are simply not enough in some cases — especially when a full list of all likely compromised serial numbers has not been disclosed as happens to currently be the case.

To the best of our knowledge and by analyzing the CRLs for DigiNotar, we do not believe that any of the fraudulently issued '*.torproject.org' certificates have been revoked at the time of this writing. It may be the case that they are simply not in the business of revoking certificates after they have expired. There is no evidence to support revocation during the time that these certificates were perfectly valid.

I believe that you can clearly see the MITM attack in action around the tenth hop of this traceroute thanks to an anonymous person in Iran:

1 3 ms 14 ms 2 ms 192.168.1.1
2 67 ms 67 ms 65 ms 91.99.***.***.parsonline.net [91.99.***.***]
3 65 ms 67 ms 93 ms 10.220.1.2
4 67 ms 72 ms 66 ms 2.180.2.1
5 66 ms 64 ms 64 ms 217.219.64.115
############### [ MORE Nodes ] #################
6 451 ms 195 ms 154 ms 78.38.245.6
7 626 ms 231 ms 88 ms 78.38.245.5
8 93 ms 91 ms 96 ms 78.38.244.242
9 88 ms 94 ms 120 ms 78.38.244.241
################### [ MORE ] ###################
10 88 ms 88 ms 88 ms 10.10.53.33 ####DIfferent IP (0.0.0.33)

#### [ OUT OF IRAN ] ####
11 340 ms * * pos3-1.palermo5.pal.seabone.net [195.22.198.77]

To quote someone I respect greatly: "That's not dodgy at all!"

Early statements By DigiNotar translated by someone and mentioned by a friendly Dutch man lead us to believe that DigiNotar and their parent company are in damage-control mode. It would be unsurprising to hear that the Dutch Government is similarly in the dark about the scope of the compromise, as it appears DigiNotar does not control a canonical list for all certificates issued. While some Browser vendors have received a list, I do not have confidence that this list actually contains all malicious certificates that have been issued: rather it appears to be a subset that did not even include the Google certificate that was being used in the wild. We hope that DigiNotar will fully disclose whatever information they have and explain what information they honestly lack.

The defense

Modern versions of Chrome (13) were able to prevent MITM attacks against most, if not all of the Google sites where they had certificate pinning and where HSTS was enabled. Google has also announced the attacks and updated information about it. Additionally, they have distrusted DigiNotar in Chrome.

We've sent a request to Google that they enable HSTS and pin certificates for some of the critical Tor sites and that patch is pending. Google has been very good about all of this and I can't thank them enough for their help.

As it stands, Chrome appears to have shipped a fix that distrusts DigiNotar and it appears to treat hundreds of certificates as if they are specifically known to be malicious or hostile. Mozilla and others have shipped a fix as well. Sadly, it appears that the Dutch Government asked various browser vendors to create an exemption for certain trust chains as some kind of compromise. However, we were not party to any of the discussions, and we don't understand the core concerns for such a compromise. We're not willing to take a leap of faith for a Certificate Authority that did not contact us when they first noticed this problem. Right now, if we found a DigiNotar-issued certificate certifying that water was wet, we wouldn't believe it without checking for ourselves. Twice.

We have proactively given DigiNotar an "Internet Death Sentence" in the Tor Browser. The direct impact of removing DigiNotar should be on the order of around seven hundred certificates according to some cursory queries run against the EFF's SSL Observatory. I believe that the number of certificates revoked is nearing parity with the number of possibly legitimate still-valid certificates issued by DigiNotar. That's a sad state of affairs.

We do not currently have evidence of any tampering with Tor downloads, but we're looking. If an attacker can successfully perform a MITM attack, there is nothing to prevent them from giving you a bogus package instead of the software you were actually looking for — if you're not checking package signatures, there's no easy way to tell good software from bad.

What you should do

First of all, upgrade your browser(s). See this blog post announcing the new Tor Browser Bundles with Firefox 6.

Note that verifying the signatures on Tor packages prevents attackers like this from causing you to install a possibly backdoored version of Tor. You should always verify the signature of any software you download. We encourage you to learn more about secure signature verification.

If you have downloaded copies of any Tor software in the past few months, and you did it over any network that you don't trust, please help us check to see whether there was any attempt to alter them. We don't expect you'll find anything, but if you do, we really want to know about it. In any case, it will be good practice for checking signatures.

If you have any information about certificates that you believe to be false, please do send us the certificates and we'll take a look.

The Certificate Authority system as it stands today is a house of cards and we're witnessing in public what many have known for years in private. The entire system is soaked in petrol and waiting for a light. There are some new directions for trust in the works such as Convergence and various ways to do DNSSEC authenticated HTTPS as well as other hacks. Still, nothing is set in stone or standardized and this is why we need to remain vigilant. We're hoping to detect these kinds of attacks in the future with our distributed SSL Observatory and we hope that you'll join us.

I'd like to end on a positive note and quote a personal hero and friend, Matt Blaze: "A decade ago, I observed that commercial certificate authorities protect you from anyone from whom they are unwilling to take money. That turns out to be wrong; they don't even do that much."

AttachmentSize
traceroutes-inside-iran.txt4.28 KB
translation-of-press-release.txt3.4 KB
google.com_.crt_.txt1.81 KB
gmail_certificate_error.jpg141.84 KB
DigiNotar-Tor-serial-list.xls8.94 KB

About the Dutch government certificates: The PKI overheids Diginotar is an intermediate CA which should have a seperate infrastructure and process. (Requirements for PKI overheids intermediate CA are also harder to satisfy, it seems that Diginotar has convinced the Dutch government and Firefox that this CA was not breached)

The Firefox fix is in line with the other browsers now (removing the root CA) plus something extra: distrust every certificate issued by CN=Diginotar*, except the intermediate PKI overheids diginotar CA.

In reality this means:
Diginotar root CA (banned by IE,Chrome and Firefox by root certificate removal)
Entrust root CA->Diginotar intermediate CA (banned by Firefox by code fix, allowed by IE Chrome?)
Dutch PKI overheid root -> Diginotar intermediate CA (allowed by all browsers)

Firefox distrusts more certificates (including future certificates from a new CA) than the other browsers. It would take a code fix to trust a new Diginotar root CA.

In the Tor Browser Bundle, we blocked the Dutch government root, on the basis that DigiNotar didn't contact us (or others) once they realized they'd been compromised which puts their integrity and trustworthiness in an even more questionable state. On a technical level, supposing they may have been completely compromised, an attacker could still exploit Chrome and Firefox users by creating an intermediate cert with an issuer name matching "CN=Staat der Nederlanden Root CA,O=Staat der Nederlanden,C=NL". So we distrust them completely.

See:
https://gitweb.torproject.org/torbrowser.git/commit/0be3b043afa0e54d207f...

And specifically:
https://gitweb.torproject.org/torbrowser.git/blob/0be3b043afa0e54d207f60...

Ok, I see, that would add at least 2 certificates to the chain
If pathlen is in the CA certificates this should not be possible, haven't checked this, but I saw a tweet claiming it was not.
http://twitter.com/#!/okoeroo/status/108962505098928129

The "Staat der Nederlanden Root CA" will, in it's self be save. This key was used to sign a handfull (7 at the moment) intermediate certificates for commercial parties selling certificates to Dutch governmental organizations. Diginotar is just one of them, the others are totally unrelated to Diginotar and won't be affected. As such there is no reason to distrust the "Staat der Nederlanden Root CA" apart from the certs which have Diginotar in their chain.

Additionally, the requirements for joining the "PKI Overheid" program granting you the right to sign on behalf of "Staat der Nederlanden" dus come with quite a pack of requirements, documented (in Dutch) on http://www.logius.nl/producten/toegang/pkioverheid/aansluiten/toetreden-tot-pkioverheid/
Amongst others this demands ETSI TS 101456 and ETSI TS 102042 certification and yearly audits. A quick scan of those requirements makes me feel it is quite likely the infrastructure for the "PKI Overheid" certificates at Diginotar is indeed separate from their standard infrastructure, but obviously I can't actually confirm this.

The intent of the Firefox patch was to allow only if the Staat der Nederlanden certs were the root of the chain. Should not be accepted as an intermediate (but it was rushed--do you think the code is wrong?)

Entrust root CA->Diginotar intermediate CA (banned by Firefox by code fix, allowed by IE Chrome?)

Chrome blacklisted a hash of the public key in that DigiNotar intermediate CA cert, so Chrome will distrust any certificate chain that includes it, even if the chain leads to a still-trusted CA certificate.

It does not appear that diginotar includes either CRL or OCSP pointers in most of their SSL certificates

Do you have some example certificates that we are able to examine?

If for various reasons you are using an older browser is it OK to leave the DigiNotar certificates in but edit them (uncheck the boxes) so they aren't used to identify sites?

I've got a tor-browser-2.2.32-2_en-US.exe that I downloaded over tor that fails validation with a bad signature. I downloaded a fresh copy over the plain internet and it passes. Are you interested in my bad copy?

In all fairness, this is probably just a false positive -- the bad version is 17,824KB and the good version is 18,300KB, so I bet the copy I downloaded of tor was just incomplete.

Let me know here how you'd like me to get the bad version to you if you're interested in analyzing it.

https://www.torproject.org/dist/torbrowser/ doesnt list binaries or sigs before august 28. please post signatures for old binaries so people can tell if they have a bad older version.

I've downloaded the version: tor-browser-gnu-linux-i686-1.1.14-dev-en-US.tar.gz
And checking the signature I obtain this:

gpg: Signature made Sun 28 Aug 2011 06:53:25 PM CEST using RSA key ID 63FEE659
gpg: Good signature from "Erinn Clark "
gpg: aka "Erinn Clark "
gpg: aka "Erinn Clark "
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 8738 A680 B84B 3031 A630 F2DB 416F 0610 63FE E659

So that package was malicious?

Mark

So, Mark, did anyone ever answer your question? Appears not. Odd

No, not malicious -- it says "good signature from".

The big WARNING you're seeing means that you actually have no idea who Erinn Clark is, or what her key really is. See the last piece of https://www.torproject.org/docs/faq#KeyManagement for more explanation.

I apologize on behalf of the GPG team for their crummy usability. :)

Fox-IT has conducted a limited investigation. The report has been made public, and can be read on scribd: http://www.scribd.com/doc/64011372/Operation-Black-Tulip-v1-0

The appendix lists serial numbers of illegal certificates. The report also mentions that the OCSP revocation process has been modified for unknown serial numbers that are apparently issued by DigiNotar. They will now be marked as revoked instead of good (as per RFC). See the report for full details.

I've been looking at the traceroutes provided, and alibo was actually saying that he thought the extra nodes (6 to 9 above) were the evidence that a man-in-the-middle attack was occurring. With respect to your comment about 10.10.53.33, though, I've been googling the surrounding IP addresses, looking at other traceroute output from Iran, and I'm fairly sure it's just 78.38.255.33 renamed.

For instance this guy

http://forum.fdcservers.net/archive/index.php/t-3955.html

has a few routes that go through 78.38.255.61 and 78.38.255.69 on the way out and alibo's other routes go through 10.10.53.61 and 10.10.53.69.

Other people's google routes also go through 10.10.53.33, but this only seems to have begun recently.

http://www.sat4u.org/showthread.php?p=6054703

It wasn't happening in September 2010: http://pastie.org/pastes/1178693

and as you noticed in your tweet it tends to happen on the way out:

http://www.webhostingtalk.ir/archive/index.php/t-29996.html

Similarly at http://security.nl/artikel/38299/1/Iraanse_overheid%3F.html someone comments that it's not proof that an MITM attack is occurring, and it seems more likely it's accomplished via routing rather than DNS. An MITM attack wouldn't necessarily require those extra hops.

R

It's perfectly normal. It happens when some part of the route uses private addresses to save on public addresses.

M4

They snoop on peoples identities for 1 year? EFF.ORG? ALISINA.ORG?

FYI, MITM attacks occurring today October 9, 2011 11:00 AM Tehran time. Traffic is going through Shatel (ISP). Not sure where else I can report this:
Tracing route to www.l.google.com [74.125.65.106]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 55 ms 54 ms 54 ms 85-15-0-xxx.rasana.net [85.15.0.xxx]
3 54 ms 54 ms 54 ms 85-15-0-xxx.rasana.net [85.15.0.xxx]
4 53 ms 78 ms 51 ms 85-15-0-1.rasana.net [85.15.0.1]
5 52 ms 54 ms 54 ms 85-15-0-66.rasana.net [85.15.0.66]
6 62 ms 57 ms 57 ms 78.38.255.89
7 56 ms 54 ms 85 ms 195.146.63.217
8 56 ms 61 ms 58 ms 10.10.53.61
9 345 ms 342 ms 345 ms pos5-3.cr01.nyc02.pccwbtn.net [63.218.109.245]
10 382 ms * 335 ms TenGE11-2.br01.ash01.pccwbtn.net [63.216.0.38]
11 * * * Request timed out.
12 326 ms 332 ms 328 ms 209.85.252.46
13 * 386 ms 518 ms 72.14.238.105
14 363 ms 362 ms * 72.14.239.127
15 * 364 ms * 209.85.253.209
16 360 ms 289 ms 291 ms gx-in-f106.1e100.net [74.125.65.106]

===============================

Tracing route to any-fp3-real.wa1.b.yahoo.com [67.195.160.76]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 53 ms 78 ms 54 ms 85-15-0-xxx.rasana.net [85.15.0.xxx]
3 52 ms 53 ms 56 ms 85-15-0-xxx.rasana.net [85.15.0.xxx]
4 55 ms 54 ms 63 ms 85-15-0-1.rasana.net [85.15.0.1]
5 70 ms 97 ms 63 ms 85-15-0-98.rasana.net [85.15.0.98]
6 * 183 ms 169 ms 78.38.255.89
7 80 ms 89 ms 60 ms 78.38.245.137
8 60 ms * 57 ms 10.10.53.61
9 267 ms 270 ms 270 ms pos5-3.cr01.nyc02.pccwbtn.net [63.218.109.245]
10 296 ms 276 ms 285 ms TenGE11-2.br01.ash01.pccwbtn.net [63.216.0.38]
11 286 ms 288 ms 285 ms TenGigabitEthernet6-3.1101.ar5.DCA3.gblx.net [2
6.57.3.233]
12 282 ms 281 ms 276 ms 208.178.62.126
13 285 ms 275 ms 342 ms xe-8-0-0.msr2.ac2.yahoo.com [216.115.108.137]
14 278 ms 273 ms 272 ms xe-2-2-0.clr4.ac4.yahoo.com [72.30.96.5]
15 280 ms 275 ms 274 ms UNKNOWN-76-13-0-X.yahoo.com [76.13.0.23]
16 283 ms 285 ms 283 ms ir1.fp.vip.ac4.yahoo.com [67.195.160.76]

Trace complete.

And here's another thing I've not seen before. A trace to yahoo now (Oct 9 2011, 11:30 AM Tehran) goes through Russian IP space. If you do a whois for the ip (80.81.193.115) hop after rostelecom.ru, it is DE-CIX Frankfurt IXP who already say don't bug us about spams/hacks in their whois. I did a DNS query for ge-1-3-0.pat2.dee.yahoo.com and got no results.

Tracing route to eu-fp3.wa1.b.yahoo.com [87.248.122.122]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.1.1
2 52 ms 54 ms 53 ms 85-15-0-xxx.rasana.net [85.15.0.xxx]
3 53 ms 54 ms 53 ms 85-15-0-xxx.rasana.net [85.15.0.xxx]
4 54 ms 56 ms 52 ms 85-15-0-1.rasana.net [85.15.0.1]
5 52 ms 53 ms 56 ms 85-15-0-98.rasana.net [85.15.0.98]
6 64 ms 60 ms 72 ms 78.38.255.89
7 53 ms 58 ms 80 ms 78.38.245.13
8 248 ms 250 ms 249 ms 92.50.194.237
9 280 ms 278 ms 275 ms xe-9-1-0.frkt-ar2.intl.ip.rostelecom.ru [87.226.
133.42]
10 299 ms 286 ms 353 ms ge-1-3-0.pat2.dee.yahoo.com [80.81.193.115]
11 284 ms 289 ms 295 ms ge-1-2-0.pat1.dee.yahoo.com [66.196.65.132]
12 273 ms 275 ms 321 ms xe-0-1-0.msr2.ch1.yahoo.com [66.196.65.139]
13 275 ms 274 ms 315 ms gi-1-5.bas-a2.ch1.yahoo.com [87.248.127.51]
14 274 ms 300 ms 326 ms ir1.fp.vip.ch1.yahoo.com [87.248.122.122]

Trace complete.

It's really strange that Oct. 9th's post regarding a MItM attack has not even been acknowledged, let alone answered. I simply do not trust TOR or any of its developers any more, you are ALL shady as can be. Thanks for taking an awesome, helpful idea and turning it into yet another honeypot. /end sarcasm

Wuh?

This was somebody reporting that Iran was mitming traffic in Iran.

That sucks, but the developers are busy working on improvements to Tor.

What did you want us to 'answer'?

(Also, more generally, the blog is a terrible interface for trying to do Q&A. We're working on that, but then, see above about busy.)

I do some work that involves a government I do not turst. They are ever on the look out for people like me. I'm not a spy, illegal alien etc.
I am a U. S. Navey Veteran with several honars. I can't discuss them as they would help identify me. That reminds me. I allowed my real name to be posted in my profile.
Hummmm. Better start thinking about what I'm doing here.

I also have to wonder just how much pressure the U S Government is putting on Tor to have a back door. Would you resist such requests/demands? I sure do hope so.
I know I've got my system setup so if they are slapping me around I'll just give them a bad password to my encrypted large Raid 10. RAID for obvious reasons, plus data stripped across multiple drives.
Purpose is to have everything encryped so forensics will only read encrypted data when they go to it. The bad password causes my LUKS system to indicate a wrong password. It is suffidiently long and complicated to make it look like a valid typo.
While you are attempting to get the password right, it actually started a wipe of the entire array with a random number generator. This is also encrypted with LUKS and to top that the LUKS guts have been ripped out. You cannot even open a LUKS file system with the proper password after about 300ms.
So I guess the beating will continue because I can't get them in now. Even the boot member is written over. The Swap files too and yes I have a partition on each of the drives to raid array for swap and it is also encrypted.
The only thing I actually worry about it how secure Tor really is.
Nothing is 100% secure but Tor is so far my best friend.
Please tell me there are no actual way for even the U S government to just drop in and read everything.
Please do not get angry with me. I'm an old man and I'm tired tonight.
Maybe I should cut and paste this to a text file and sleep on it.
Read it later and decide if this is a ligitamate question. Besides anyone who the Gov has gotten to isn't going to tell, now are they?

Yes the IP this is coming from is mine. No I'm not at that site and this is being proxied several times some through private IP addresses.
Example the last and worst is a wireless connection from NAT private address.
Oh well.

The way three men keep a secret is that two are dead!

Syndicate content Syndicate content