Last week, a user in Jordan reported seeing a fake certificate for torproject.org. The user did not report any errors when browsing to sites such as Gmail, Facebook, and Twitter, which suggests that this was a targeted attack. The certificate was issued by a company called Cyberoam. We first believed that this incident was similar to that of Comodo and DigiNotar, and that Cyberoam had been tricked to issue a fake certificate for our website.
After a bit of research, we learned that Cyberoam make a range of devices used for Deep Packet Inspection (DPI). The user was not just seeing a fake certificate for torproject.org, his connection was actually being intercepted by one of their devices. While investigating this further, Ben Laurie and I found a security vulnerability affecting all Cyberoam DPI devices.
Examination of a certificate chain generated by a Cyberoam DPI device shows that all such devices share the same CA certificate and hence the same private key. It is therefore possible to intercept traffic from any victim of a Cyberoam device with any other Cyberoam device - or to extract the key from the device and import it into other DPI devices, and use those for interception.
Ben and I wrote a security advisory and notified Cyberoam of this vulnerability at 17:00 UTC on Saturday, June 30. We made it clear that we intended to publish this blog post and the security advisory on Tuesday, July 3, and encouraged them to respond promptly if they had any comments. At the same time, we notified browser vendors and asked that they blacklist the Cyberoam CA certificate in their browsers.
Cyberoam have not yet commented on this issue, apart from acknowledging our first email and saying that they are looking into it. The Cyberoam CA certificate is not trusted, and so browsers will show users a warning (unless someone has already installed the certificate). Users with the Tor Browser Bundle are not affected.
To check if this CA is installed in your browser, see the following instructions for Internet Explorer, Firefox, Chrome, and Safari. The instructions mention DigiNotar, but they are still valid. If you have more information about this issue, please email firstname.lastname@example.org.
About an hour ago I was contacted by the Dutch Government with more details about the DigiNotar Debacle. It seems that they're doing a great job keeping on top of things and doing the job that DigiNotar should've done in July. They sent a spreadsheet with a list of 531 entries on the currently known bad DigiNotar related certificates.
The list isn't pretty and I've decided that in the interest of defenders everywhere without special connections, I'm going to disclose it. The people that I have spoken with in the Dutch Government agree with this course of action.
This disclosure will absolutely not help any attacker as it does not contain the raw certificates; it is merely metadata about the certificates that were issued. It includes who we should not trust in the future going forward and it shows what is missing at the moment. This is an incomplete list because DigiNotar's audit trail is incomplete.
This is the list of CA roots that should probably never be trusted again:
DigiNotar Cyber CA
DigiNotar Extended Validation CA
DigiNotar Public CA 2025
DigiNotar Public CA - G2
Koninklijke Notariele Beroepsorganisatie CA
Stichting TTP Infos CA
The most egregious certs issued were for *.*.com and *.*.org while certificates for Windows Update and certificates for other hosts are of limited harm by comparison. The attackers also issued certificates in the names of other certificate authorities such as "VeriSign Root CA" and "Thawte Root CA" as we witnessed with ComodoGate, although we cannot determine whether they succeeded in creating any intermediate CA certs. That's really saying something about the amount of damage a single compromised CA might inflict with poor security practices and regular internet luck.
Of particular note is this certificate:
CN=*.RamzShekaneBozorg.com,SN=PK000229200006593,OU=Sare Toro Ham Mishkanam,L=Tehran,O=Hameye Ramzaro Mishkanam,C=IR
The text here appears to be be an entry like any other but it is infact a calling card from a Farsi speaker. RamzShekaneBozorg.com is not a valid domain as of this writing.
Thanks to an anonymous Farsi speaker, I now understand that the above certificate is actually a comment to anyone who bothers to read between the lines:
"RamzShekaneBozorg" is "great cracker"
"Hameyeh Ramzaro Mishkanam" translates to "I will crack all encryption"
"Sare Toro Ham Mishkanam" translates to "i hate/break your head"
Without any further delay, I've uploaded the original spreadsheet and a CSV text file for people who don't trust spreadsheets. The information contained in both files should be the same. Hopefully this information will help people to mitigate certain harm from the DigiNotar Debacle.
Recently it has come to the attention of, well, nearly the entire world that the Dutch Certificate Authority DigiNotar incorrectly issued certificates to a malicious party or parties. Even more recently, it's come to light that they were apparently compromised months ago or perhaps even in May of 2009 if not earlier.
This is pretty unfortunate, since correctly issuing certificates is exactly the function that a certificate authority (CA) is supposed to perform. By comparison, ComodoGate looks fairly minor.
This incident doesn't affect the functionality of Tor clients or the Tor Network itself, since Tor doesn't use the flawed CA system. The Tor network uses a much simpler and flatter trust design that protects us from many of these CA issues. Further, Tor's distributed-trust design limits the damage from compromise of any given network component.
But the incident does affect users that are attempting to reach The Tor Project's infrastructure: with one of these bogus certificates, an attacker could convince your browser that you were talking to The Tor Project website, when really you were talking to the attacker.
We have taken direct action in an attempt to stop this kind of attack in the future with two major browser vendors and we hope to integrate a fix with all other willing browsers. Please contact us if you ship a browser and you'd like to help your users to be proactively secure when visiting our sites. TorBrowser users should upgrade to our latest release and verify signatures for all downloaded files. All Tor Browser Bundles have been updated to Firefox 6 with a patch to stop trusting the offending CA, and users are encouraged to upgrade. Below, we describe what we found out, what we're doing about it, and what you should do to keep yourself safe.
In the last seventy-two hours we were working to find positive confirmation that The Tor Project was one of the targeted groups. It was originally disclosed that at least one certificate was issued for '*.google.com' and that it was being used to actively Man-In-The-Middle SSL and TLS connections. Quite quickly we found a similar pattern to the ComodoGate fiasco. It appears likely that the Mozilla Addons site, Yahoo, Facebook, Twitter, and a few other major players were targeted. We do not have an authoritative list but I personally believe those targets to be accurate; time will tell. Additionally, we heard rumors that we had graduated to the big leagues and we had also heard that DigiNotar had reached out to the major browser vendors. We did not receive any proactive contact from DigiNotar as a browser vendor and this worried us greatly when compounded with the rumor of being one of the targets as well.
We ship a rather specific and special browser and it appeared that all of our sites are specifically in the attacker's target list. Having received no contact from DigiNotar, we reached out to DigiNotar by email and by telephone.
I spoke on the telephone with a rather nice but obviously overworked DigiNotar point of contact who will remain anonymous. He was guarded and careful in what he said but was clearly sympathetic to the severity of the matter at hand. It seemed quite clear that he repeated similar information to other impacted callers:
"What I can say is the following " ... "Any fraudulent torproject.org certificate that has been requested has been revoked. Any serial numbers that we know about have been revoked. All serial numbers have been communicated to the major browsers vendors." ... "Any certificate that we know of is revoked by OCSP server."
We emailed quite a bit back and forth after the phone call. A few hours later that same point of contact from DigiNotar sent a list of all of the certificates in a spreadsheet. It appears that the attackers requested twelve certificates, and each certificate was for '*.torproject.org'. The first batch of six certificates was issued on July 18th and the second batch of six certificates was issued on July 20th. According to the spreadsheet, the first six of the certificates expired on August 17th, 2011 and second batch of six certificates expired on August 19th, 2011. According to the information disclosed by DigiNotar the certificates in question should all have expired. The contact at DigiNotar stressed that there was no confirmation about the attacker(s) receiving the certificates. I have no reason to believe that these certificates would have any more trouble reaching the requesting party than the Google certificate used in the wild.
This is the current list of serial numbers for all twelve Tor Project certificates as disclosed to us by DigiNotar:
DigiNotar has not provided us with a copy of any of the certificates that they issued. We are not sure that they have copies nor if they are willing to disclose any copies they may or may not have. This point is extremely disconcerting as the CRL/OCSP revocation process is essentially worthless. Mere serial numbers are simply not enough in some cases — especially when a full list of all likely compromised serial numbers has not been disclosed as happens to currently be the case.
To the best of our knowledge and by analyzing the CRLs for DigiNotar, we do not believe that any of the fraudulently issued '*.torproject.org' certificates have been revoked at the time of this writing. It may be the case that they are simply not in the business of revoking certificates after they have expired. There is no evidence to support revocation during the time that these certificates were perfectly valid.
1 3 ms 14 ms 2 ms 192.168.1.1
2 67 ms 67 ms 65 ms 91.99.***.***.parsonline.net [91.99.***.***]
3 65 ms 67 ms 93 ms 10.220.1.2
4 67 ms 72 ms 66 ms 126.96.36.199
5 66 ms 64 ms 64 ms 188.8.131.52
############### [ MORE Nodes ] #################
6 451 ms 195 ms 154 ms 184.108.40.206
7 626 ms 231 ms 88 ms 220.127.116.11
8 93 ms 91 ms 96 ms 18.104.22.168
9 88 ms 94 ms 120 ms 22.214.171.124
################### [ MORE ] ###################
10 88 ms 88 ms 88 ms 10.10.53.33 ####DIfferent IP (0.0.0.33)
#### [ OUT OF IRAN ] ####
11 340 ms * * pos3-1.palermo5.pal.seabone.net [126.96.36.199]
To quote someone I respect greatly: "That's not dodgy at all!"
Early statements By DigiNotar translated by someone and mentioned by a friendly Dutch man lead us to believe that DigiNotar and their parent company are in damage-control mode. It would be unsurprising to hear that the Dutch Government is similarly in the dark about the scope of the compromise, as it appears DigiNotar does not control a canonical list for all certificates issued. While some Browser vendors have received a list, I do not have confidence that this list actually contains all malicious certificates that have been issued: rather it appears to be a subset that did not even include the Google certificate that was being used in the wild. We hope that DigiNotar will fully disclose whatever information they have and explain what information they honestly lack.
Modern versions of Chrome (13) were able to prevent MITM attacks against most, if not all of the Google sites where they had certificate pinning and where HSTS was enabled. Google has also announced the attacks and updated information about it. Additionally, they have distrusted DigiNotar in Chrome.
We've sent a request to Google that they enable HSTS and pin certificates for some of the critical Tor sites and that patch is pending. Google has been very good about all of this and I can't thank them enough for their help.
As it stands, Chrome appears to have shipped a fix that distrusts DigiNotar and it appears to treat hundreds of certificates as if they are specifically known to be malicious or hostile. Mozilla and others have shipped a fix as well. Sadly, it appears that the Dutch Government asked various browser vendors to create an exemption for certain trust chains as some kind of compromise. However, we were not party to any of the discussions, and we don't understand the core concerns for such a compromise. We're not willing to take a leap of faith for a Certificate Authority that did not contact us when they first noticed this problem. Right now, if we found a DigiNotar-issued certificate certifying that water was wet, we wouldn't believe it without checking for ourselves. Twice.
We have proactively given DigiNotar an "Internet Death Sentence" in the Tor Browser. The direct impact of removing DigiNotar should be on the order of around seven hundred certificates according to some cursory queries run against the EFF's SSL Observatory. I believe that the number of certificates revoked is nearing parity with the number of possibly legitimate still-valid certificates issued by DigiNotar. That's a sad state of affairs.
We do not currently have evidence of any tampering with Tor downloads, but we're looking. If an attacker can successfully perform a MITM attack, there is nothing to prevent them from giving you a bogus package instead of the software you were actually looking for — if you're not checking package signatures, there's no easy way to tell good software from bad.
What you should do
First of all, upgrade your browser(s). See this blog post announcing the new Tor Browser Bundles with Firefox 6.
Note that verifying the signatures on Tor packages prevents attackers like this from causing you to install a possibly backdoored version of Tor. You should always verify the signature of any software you download. We encourage you to learn more about secure signature verification.
If you have downloaded copies of any Tor software in the past few months, and you did it over any network that you don't trust, please help us check to see whether there was any attempt to alter them. We don't expect you'll find anything, but if you do, we really want to know about it. In any case, it will be good practice for checking signatures.
If you have any information about certificates that you believe to be false, please do send us the certificates and we'll take a look.
The Certificate Authority system as it stands today is a house of cards and we're witnessing in public what many have known for years in private. The entire system is soaked in petrol and waiting for a light. There are some new directions for trust in the works such as Convergence and various ways to do DNSSEC authenticated HTTPS as well as other hacks. Still, nothing is set in stone or standardized and this is why we need to remain vigilant. We're hoping to detect these kinds of attacks in the future with our distributed SSL Observatory and we hope that you'll join us.
I'd like to end on a positive note and quote a personal hero and friend, Matt Blaze: "A decade ago, I observed that commercial certificate authorities protect you from anyone from whom they are unwilling to take money. That turns out to be wrong; they don't even do that much."