Detecting Certificate Authority compromises and web browser collusion

Thanks to Ian Gallagher, Seth Schoen, Jesse Burns, Chris Palmer, and other anonymous birds for their invaluable feedback on this writeup.

The Tor Project has long understood that the certification authority (CA) model of trust on the internet is susceptible to various methods of compromise. Without strong anonymity, the ability to perform targeted attacks with the blessing of a CA key is serious. In the past, I’ve worked on attacks relating to SSL/TLS trust models and for quite some time, I’ve hunted for evidence of non-academic CA compromise in the wild.

I’ve also looked for special kinds of cooperation between CAs and browsers. Proof of collusion will give us facts. It will also give us a real understanding of the faith placed in the strength of the underlying systems.

Does certificate revocation really work? No, it does not. How much faith does a vendor actually put into revocation, when verifiable evidence of malice is detected or known? Not much, and that’s the subject of this writing.

Last week, a smoking gun came into sight: A Certification Authority appeared to be compromised in some capacity, and the attacker issued themselves valid HTTPS certificates for high-value web sites. With these certificates, the attacker could impersonate the identities of the victim web sites or other related systems, probably undetectably for the majority of users on the internet.

I watch the Chromium and Mozilla Firefox projects carefully, because they are so important to the internet infrastructure. On the evening of 16 March, I noticed a very interesting code change to Chromium: revision 78478, Thu Mar 17 00:48:21 2011 UTC.

In this revision, the developers added X509Certificate::IsBlacklisted, which returns true if a HTTPS certificate has one of these particular serial numbers:

  • 047ecbe9fca55f7bd09eae36e10cae1e
  • d8f35f4eb7872b2dab0692e315382fb0
  • b0b7133ed096f9b56fae91c874bd3ac0
  • 9239d5348f40d1695a745470e1f23f43
  • d7558fdaf5f1105bb213282b707729a3
  • f5c86af36162f13a64f54f6dc9587c06

A comment marks the first as "Not a real certificate. For testing only." but we don’t know if this means the other certificates are or are not also for testing.

With just these serial numbers, we are not able to learn much about the certificates that Chromium now blocks. To get more information, I started the crlwatch project. Nearly every certificate contains a reference to a Certificate Revocation List (CRL). A CRL is a list of certificates that the CA has revoked for whatever reason. In theory, this means that an attacker is unable to tamper with the certificate to prevent revocation as the browser will check the CRL it finds in a certificate. In practice the attacker simply needs to tamper with the network - this is something they’re already able to do if they are performing a SSL/TLS Machine-In-The-Middle attack. Even if an attacker has a certificate, they generally are unable to modify the certificate without breaking the digital signature issued by the CA. That CA signature is what gives the certificate value to an attacker and tampering takes the attacker back to square zero. So while investigating these serials, we clearly lack the CRL distribution point in the Chrome source. However, the project that I announced on March 17th, crlwatch, was specifically written to assist in finding who issued, and potentially revoked the serial numbers in question. By matching the serial numbers found in the source for Chrome with the serial numbers of revoked certificates, we’re able to link specific serials to specific CA issuers. The more serial numbers we match in revocation lists, the higher our probability of having found the CA that issued the certificates.

About twelve hours (Thursday, March 17, 2011 | 13:00) after the above patch was pushed into source control - Google announced an important Chrome Update that involved HTTPS certificate issues.

This also is mostly uninteresting until we notice that this is not isolated to Google. Mozilla pushed out two patches of interest:

The complete changeset is semi-informative. Mozilla references a private bug in that fix that Mozilla will hopefully disclose. Similar to Chromium, the Mozilla patches create a list of certificate serial numbers that will be treated as invalid. However, the serial numbers from the Mozilla patches are different:

  • 009239d5348f40d1695a745470e1f23f43
  • 00d8f35f4eb7872b2dab0692e315382fb0
  • 72032105c50c08573d8ea5304efee8b0
  • 00b0b7133ed096f9b56fae91c874bd3ac0
  • 00e9028b9578e415dc1a710a2b88154447
  • 00d7558fdaf5f1105bb213282b707729a3
  • 047ecbe9fca55f7bd09eae36e10cae1e
  • 00f5c86af36162f13a64f54f6dc9587c06
  • 392a434f0e07df1f8aa305de34e0c229
  • 3e75ced46b693021218830ae86a82a71

Thus, both Mozilla and Google shipped similar patches to their code at roughly the same time. The two browsers now have partially overlapping certificate blocklists. Here is the union of the two lists:

  • 009239d5348f40d1695a745470e1f23f43
  • 00b0b7133ed096f9b56fae91c874bd3ac0
  • 00d7558fdaf5f1105bb213282b707729a3
  • 00d8f35f4eb7872b2dab0692e315382fb0
  • 00e9028b9578e415dc1a710a2b88154447
  • 00f5c86af36162f13a64f54f6dc9587c06
  • 047ecbe9fca55f7bd09eae36e10cae1e
  • 392a434f0e07df1f8aa305de34e0c229
  • 3e75ced46b693021218830ae86a82a71
  • 72032105c50c08573d8ea5304efee8b0
  • 9239d5348f40d1695a745470e1f23f43
  • b0b7133ed096f9b56fae91c874bd3ac0
  • d7558fdaf5f1105bb213282b707729a3
  • d8f35f4eb7872b2dab0692e315382fb0
  • f5c86af36162f13a64f54f6dc9587c06

Why do the browsers have these blocklists, and why don’t they have the same blocklists?

This returns me to the reason for creating crlwatch last week - I wanted to find the someones who knowingly revoked the above listed special certificates. Anyone looking from the same starting point as I did, obviously lacks the leaf certificates in question and as a result, I had to look in a rather round about manner. Thanks to the EFF’s SSL Observatory, I was able to populate the base list for crlwatch. Armed with a nearly canonical list of all CRLs, I fetched them over Tor and parsed the CRL data into human readable text. The goal was to search for the above serial numbers and to find something linkable.

This is the result from searching the crlwatch data:

  1. <br />
  2. Looking for 009239d5348f40d1695a745470e1f23f43 in parsed CRLs...<br />
  3. Looking for 00b0b7133ed096f9b56fae91c874bd3ac0 in parsed CRLs...<br />
  4. Looking for 00d7558fdaf5f1105bb213282b707729a3 in parsed CRLs...<br />
  5. Looking for 00d8f35f4eb7872b2dab0692e315382fb0 in parsed CRLs...<br />
  6. Looking for 00e9028b9578e415dc1a710a2b88154447 in parsed CRLs...<br />
  7. Looking for 00f5c86af36162f13a64f54f6dc9587c06 in parsed CRLs...<br />
  8. Looking for 047ecbe9fca55f7bd09eae36e10cae1e in parsed CRLs...<br />
  9. Match! Serial Number: 047ECBE9FCA55F7BD09EAE36E10CAE1E<br />
  10. Match! Serial Number: 047ECBE9FCA55F7BD09EAE36E10CAE1E<br />
  11. Match! Serial Number: 047ECBE9FCA55F7BD09EAE36E10CAE1E<br />
  12. Looking for 392a434f0e07df1f8aa305de34e0c229 in parsed CRLs...<br />
  13. Match! Serial Number: 392A434F0E07DF1F8AA305DE34E0C229<br />
  14. Match! Serial Number: 392A434F0E07DF1F8AA305DE34E0C229<br />
  15. Match! Serial Number: 392A434F0E07DF1F8AA305DE34E0C229<br />
  16. Looking for 3e75ced46b693021218830ae86a82a71 in parsed CRLs...<br />
  17. Match! Serial Number: 3E75CED46B693021218830AE86A82A71<br />
  18. Match! Serial Number: 3E75CED46B693021218830AE86A82A71<br />
  19. Match! Serial Number: 3E75CED46B693021218830AE86A82A71<br />
  20. Looking for 72032105c50c08573d8ea5304efee8b0 in parsed CRLs...<br />
  21. Looking for 9239d5348f40d1695a745470e1f23f43 in parsed CRLs...<br />
  22. Match! Serial Number: 9239D5348F40D1695A745470E1F23F43<br />
  23. Match! Serial Number: 9239D5348F40D1695A745470E1F23F43<br />
  24. Match! Serial Number: 9239D5348F40D1695A745470E1F23F43<br />
  25. Looking for b0b7133ed096f9b56fae91c874bd3ac0 in parsed CRLs...<br />
  26. Match! Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0<br />
  27. Match! Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0<br />
  28. Match! Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0<br />
  29. Looking for d7558fdaf5f1105bb213282b707729a3 in parsed CRLs...<br />
  30. Match! Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  31. Match! Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  32. Match! Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  33. Looking for d8f35f4eb7872b2dab0692e315382fb0 in parsed CRLs...<br />
  34. Match! Serial Number: D8F35F4EB7872B2DAB0692E315382FB0<br />
  35. Match! Serial Number: D8F35F4EB7872B2DAB0692E315382FB0<br />
  36. Match! Serial Number: D8F35F4EB7872B2DAB0692E315382FB0<br />
  37. Looking for f5c86af36162f13a64f54f6dc9587c06 in parsed CRLs...<br />
  38. Match! Serial Number: F5C86AF36162F13A64F54F6DC9587C06<br />
  39. Match! Serial Number: F5C86AF36162F13A64F54F6DC9587C06<br />
  40. Match! Serial Number: F5C86AF36162F13A64F54F6DC9587C06<br />

Huzzah! It appears that we've found a few matches!

Here are the three matching files in human readable format:

Matching entries in the list look like this:

  1. <br />
  2. Serial Number: 392A434F0E07DF1F8AA305DE34E0C229<br />
  3. Revocation Date: Mar 15 20:15:38 2011 GMT<br />

An interesting note is that this date is a bit earlier than the above patches. The CA knew to revoke it on March 15th and the above patches were worked into software a few days later. If the attacker was targeting specific users, the damage to those users may have already been inflicted.

All three of the CRLs in question belong to the same CA:

  1. <br />
  2. issuer=/C=US/ST=UT/L=Salt Lake City/O=The USERTRUST<br />
  3. Network/OU=<a href="http://www.usertrust.com/CN=UTN-USERFirst-Hardware
  4. [/geshifilter-code" rel="nofollow">http://www.usertrust.com/CN=UTN-USERFirst-Hardware<br />
  5. [/geshifilter-code</a>]</p>
  6. <p>This appears to be a reseller or something similar for the <a href="http://www.comodo.com/" rel="nofollow">Comodo CA company</a> when the trust chain for USERTRUST is inspected:</p>
  7. <p>[geshifilter-code]<br />
  8. CN = COMODO High Assurance Secure Server CA<br />

We appear to have no initial matches for the following Mozilla specific serials from the data that I gathered during the initial crlwatch data population run:

  • 009239d5348f40d1695a745470e1f23f43
  • 00b0b7133ed096f9b56fae91c874bd3ac0
  • 00d7558fdaf5f1105bb213282b707729a3
  • 00d8f35f4eb7872b2dab0692e315382fb0
  • 00e9028b9578e415dc1a710a2b88154447
  • 00f5c86af36162f13a64f54f6dc9587c06

Those serial numbers appear to not match, right? Nope. Mozilla appears
to deal with certificate serial numbers in a slightly different manner - Chrome does the same internally but Mozilla exposes a weird quirk of certificate encoding directly in the source. The human readable data does not contain this quirk. Thus if you remove the prefix of ‘00’ from those serial numbers and search for the sixteen byte values, we find what we'd expect:

  1. <br />
  2. Looking for 9239d5348f40d1695a745470e1f23f43 in parsed CRLs...<br />
  3. Match! Serial Number: 9239D5348F40D1695A745470E1F23F43<br />
  4. Match! Serial Number: 9239D5348F40D1695A745470E1F23F43<br />
  5. Match! Serial Number: 9239D5348F40D1695A745470E1F23F43<br />
  6. Looking for b0b7133ed096f9b56fae91c874bd3ac0 in parsed CRLs...<br />
  7. Match! Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0<br />
  8. Match! Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0<br />
  9. Match! Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0<br />
  10. Looking for d7558fdaf5f1105bb213282b707729a3 in parsed CRLs...<br />
  11. Match! Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  12. Match! Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  13. Match! Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  14. Looking for d8f35f4eb7872b2dab0692e315382fb0 in parsed CRLs...<br />
  15. Match! Serial Number: D8F35F4EB7872B2DAB0692E315382FB0<br />
  16. Match! Serial Number: D8F35F4EB7872B2DAB0692E315382FB0<br />
  17. Match! Serial Number: D8F35F4EB7872B2DAB0692E315382FB0<br />
  18. Looking for e9028b9578e415dc1a710a2b88154447 in parsed CRLs...<br />
  19. Match! Serial Number: E9028B9578E415DC1A710A2B88154447<br />
  20. Match! Serial Number: E9028B9578E415DC1A710A2B88154447<br />
  21. Match! Serial Number: E9028B9578E415DC1A710A2B88154447<br />
  22. Looking for f5c86af36162f13a64f54f6dc9587c06 in parsed CRLs...<br />
  23. Match! Serial Number: F5C86AF36162F13A64F54F6DC9587C06<br />
  24. Match! Serial Number: F5C86AF36162F13A64F54F6DC9587C06<br />
  25. Match! Serial Number: F5C86AF36162F13A64F54F6DC9587C06<br />

Here's sample from one of those CRLs:

  1. <br />
  2. Serial Number: D7558FDAF5F1105BB213282B707729A3<br />
  3. Revocation Date: Mar 15 20:15:26 2011 GMT<br />

Ironically, after all of this work, the Mozilla patch also leaks the CA name and confirmed my suspicions without question.

In the end, when the lists are merged, we find eleven certificates with two certificates probably acting as testing certificates for the two vendors involved:

  • 077a59bcd53459601ca6907267a6dd1c
  • 047ecbe9fca55f7bd09eae36e10cae1e
  • 392a434f0e07df1f8aa305de34e0c229
  • 3e75ced46b693021218830ae86a82a71
  • 72032105c50c08573d8ea5304efee8b0
  • 9239d5348f40d1695a745470e1f23f43
  • b0b7133ed096f9b56fae91c874bd3ac0
  • d7558fdaf5f1105bb213282b707729a3
  • d8f35f4eb7872b2dab0692e315382fb0
  • e9028b9578e415dc1a710a2b88154447
  • f5c86af36162f13a64f54f6dc9587c06

This is evidence of a rather serious event and one that cannot be ignored. If I had to make a bet, I'd wager that an attacker was able to issue high value certificates, probably by compromising USERTRUST in some manner, this was discovered sometime before the revocation date, each certificate was revoked, the vendors notified, the patches were written, and binary builds kicked off - end users are probably still updating and thus many people are vulnerable to the failure that is the CRL and OCSP method for revocation. Even after users update, I'm guessing they may be unequally protected. Mozilla and other browsers should force OCSP verification by default as part of their next release and remove CAs that are unable to handle this requirement. Users of Mozilla Firefox that are concerned about this issue should enable security.OCSP.require in the about:config dialog. The surveillance concerns of enabling OCSP are serious - a CA learns what sites you’re visiting. However, they are nullified by the fact that OCSP checking is enabled by default on Firefox at least; it simply doesn’t provide any security gains for the end user because when it fails, it fails open!

I contacted both Google and Mozilla (bug #643056) for comment after discovering the above data. Mozilla expressed some concern about disclosure and I offered to embargo this document until Tuesday March 22nd, the launch day for Firefox 4. They agreed and I kept this under my hat. After discussions between Comodo and Microsoft, passed on to me by Mozilla, the embargo was to be extended until Wednesday, March 23rd. This extension was ostensibly to ensure that Microsoft would be able to ship their Internet Explorer mitigation pack. After further discussion, Mozilla pushed their blog post about the issue and I now consider the embargo lifted. Google has already shipped a fix to users. Install the latest Firefox to get a patch, if you haven't already. A Tor Browser update is in the works and will be available soon.

Mozilla offered some additional information and disclosed that addons.mozilla.org was one of the certificates acquired by the attacker. In total, nine certificates were acquired. Seven were uniquely named. Two of the certificates were re-issued for a previously issued host name. One certificate was issued for "global trustee" rather than a valid host name. With testing certificates in the list, we have a good accounting of the certificates found in the source code of each browser. Google clarified their discrepancy with the list, acknowledged the duplicate serial mistake and issued subsequent patches. Saving for test hosts, the lists are now identical.

If I had to guess at sites, I'd speculate that Facebook, Skype, Google, Microsoft, Mozilla, and others are worthy of targeting. Comodo should disclose this information and clear up this speculation with very clear information about who was targeted.

Both vendors expressed that the CA in question had done something quite remarkable by disclosing this compromise. The incentives may not be in the favor of the CA for disclosure. Many CAs may fall to similar attacks and simply refuse to disclose. Hopefully crlwatch will provide us with meaningful data regarding revocation events. The EFF and The Tor Project are working on solutions for detecting anomalies in certificates found in the wild. Still, some CAs may simply be unaware of compromises or unwilling to revoke for fear of detection.

Are all other browsers deploying similar countermeasures? Thanks to the free software nature of Firefox and Chrome, we have an answer for at least two projects. I wish that we could say the same for the rest of the browser world. One may assume that the CA in question did their best to contact all impacted vendors and targets.

Comodo has not yet revealed the extent of the compromise to the public - were their signing keys in a hardware security module? How many certificates in total were ever issued by this specific signing key? Wouldn't it be best to remove the specific signing keys from all trust roots to be extra careful given the stakes? Who exactly did they deem important enough for disclosure? The Tor Project, which ships the Tor Browser Bundle was not notified. Clearly some groups are being left out of the loop and this is where even a single attack really causes the entire CA trust model to fall apart.

Comodo should release the full certificates to the internet as well as all of the details relating to the attack. Mozilla and other browsers should open their bug reports, explain their process and lay out a path forward where we won't have to repeat this entire process again.

There is some suspicion that this action was taken by a state level adversary and there are some specific states that have been named. I’ll leave further speculation about which nation states may be involved to others. The mere fact that the web’s system of trust relies on an all or nothing property should be enough of a cause for alarm. It’s obvious that this has been and will continue to be exploited.

The impact on other cryptographic systems, such as S/MIME signatures and other cryptographic systems secured by CAs is entirely un-discussed. Quite seriously, when a CA is compromised, it will impact a great deal more than the web; users of email systems (SMTP, IMAP, POP,etc), Jabber servers, and any other SSL/TLS enabled systems are all at risk. Blocking specific serial numbers or relying on flawed, provably broken methods of revocation will simply not cut it anymore. When the actual protection mechanisms are not enforced, there is little hope of end users being protected.

This should serve as a wake up call to the internet. We need to research, build, and share new methods for ensuring trust, identity, authenticity, and confidentiality on the internet. Proposals such as DANE, CAA, HASTLS, and Monkeysphere are steps in the right direction but they face an uphill battle from entrenched economic interests.

Certification Authorities may continue to provide a piece of the puzzle but it’s high time we ensure that they’re not the alpha and the omega, anymore.

Update:
Comodo has issued a statement confirming everything that I've said and more. They believe that this was a targeted attack by a state level actor and they have named Iran as the country they suspect. Mozilla has now opened the bug reports about the issue to the public. Microsoft has now disclosed their report as well.

In the details of their statement we have a confirmation that they have the ability to monitor and thus surveille people who wish to know if certificates are valid.

Comodo also clearly demonstrate a mis-understanding - they believe that checks for revocation are proof positive that certificates are being used. They need to read and understand why this is not true.

The browsers have dropped the ball and they have chosen to fail open in nearly every single case; an attacker who is able to MITM SSL/TLS will also MITM the OCSP/CRL requests. Moxie's sslstrip demonstrated that an attacker would do this automatically and his software has done this for OCSP in public since 2009. Mozilla did not fix this issue at the time and they have once again punted on the issue. An even lower tech attack is possible and it's why revocation does not work: By returning a HTTP 500 error, the browser will the continue on as if revocation checks showed the certificate to be perfectly fine.

The browsers chose a user privacy invasive stance without the user protecting security properties. They did this because they claim that CAs are unable to provide working OCSP/CRL systems for request handling. This is a fair claim if true but it must not stand any longer. If the CA cannot provide even a basic level of revocation, it's clearly irresponsible to ship that CA root in a browser. Browsers should give insecure CA keys an Internet Death Sentence rather than expose the users of the browsers to known problems.

It's probably the case that Mozilla and other browsers should write a secure, caching OCSP server for use when a CA has a failure. It should probably be run by a neutral third party such as the EFF with a strong user privacy stance. This would only serve as a temporary fix and until Browsers get their act together, users are doomed anyway.

OCSP stapling does not fix this issue. The browsers treat revocation errors as soft errors and a MITM is deadly for revocation. The browsers believe they have to treat them as soft errors because the CAs are failing to do their job properly and are almost entirely unaccountable. The browsers are failing users by refusing to hold CAs to account. If OCSP and CRL failures mean the internet doesn't work, we need to create alternatives and not simply sweep these issues under the rug for later analysis. Browsers should hard fail on certificate revocation errors.

Comodo has further failed by:

  • Failing to produce further information about those certificates
  • Selective disclosure to "principal browsers and domain owners"
  • Failing to disclose what sub-CA/intermediate root actually did the signing
  • Believing that the attacker must control DNS for these attacks to succeed
  • Waiting eight days to disclose evidence of a specific targeted attack

I believe that the browsers, such as Mozilla, are doing the best that they can in some ways but the lack of immediate full disclosure is a major failure.

I share your concern about Comodo's practices but at this point, I don't think they have much to gain by fudging or hiding info. Their reputation is seriously at stake and if they mess up, they will be history very soon.

John

March 23, 2011

Permalink

Does the SSL Observatory have any legit certs with global trustee in the cn or san fields?

John

March 23, 2011

Permalink

I may be completely wrong here, but couldn't an attacker beat revocation by serving a man-in-the-middle victim an old version of the CRL, downloaded before the cert was revoked.
The CRL files are all stored on http servers, not https.

John

March 23, 2011

Permalink

Oh, that wacky Comodo / Usertrust / UTN CA.

For debian and ubuntu users, here is a quick way to quit trusting this CA. I posted it back in 2008, when they issued a certificate they shouldn't have (see slashdot link above).

sudo sed -ri '/comodo|utn|addtrust/Is/^!*/!/' /etc/ca-certificates.conf; sudo update-ca-certificates

For firefox users on linux, run this with firefox closed:

  1. locate -er '/cert8\.db$' |sed 's;[^/]*$;;' |while read db; do echo $db; certutil -d "$db" -L |egrep -i 'comodo|utn|usertrust|addtrust' |sed -re 's; +[a-zA-Z]*,.*;;' |while read cn; do echo "- $cn"; certutil -d "$db" -M -t cw,cw,cw -n "$cn"; certutil -d "$db" -D -n "$cn"; done; done<br />

For firefox users, using a gui:
http://benjamin.smedbergs.us/blog/2008-12-24/how-to-disable-the-comodo-…

Somewhere there is a UserTrust cert that signs itself:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589023

John

March 23, 2011

Permalink

I've marked all my CAs as invalid and use the perspectives firefox extension:

http://www.networknotary.org/firefox.html

Not perfect, but if I am seeing a cert that doesn't correspond to the notaries, it will inform me. (it doesn't override firefiox xpi signatures, so installs will simply fail).

John

March 23, 2011

Permalink

Wonderful, wonderful. Unearthing this is clearly a sackload of work, for which kudos. Also notice how it is necessary and not something a random user would do. Since it IS necessary, it means safely using the thing is just about impossible except for experts.

This, by the way, is a glaring error, omission, and oversight in security thinking. It keeps on cropping up any time crypto is introduced, and is closely related to the problems detailed in _Why Johnny Can't Encrypt_. Even Appelbaum thinks about this technically. The biggest concern is not that, it's that without deep technical understanding of the system it can be used neither safely nor securely. How's that for userfriendlyness? --bounce

John

March 23, 2011

Permalink

Went to upgrade FF, upgrade failed. Downloaded 4 and installed. Immediately got OCSP errors going to Facebook. Turned on WS and started sniffing packets. Fatal error, bad certificates! Checked against list. Nope different numbers. Did about:config and made sure OCSP was turned on.

It fails if it does not get a response. Good.

Checked the IP where I was getting errors. Skype. Other errors, Facebook.

Read the Freedom To Tinker post, aha. Read this post, aha. Nice.

Noticed that both CA use c=US in their name but do not support the listing of their X.509v3 certs in a c=US X.500 directory when it is just simpler to hard code them into browsers and make the process opaque to users.

The error becomes obvious, if one could easily lookup the right certificates one would not trust the rogue ones. Following the path in browsers has led me down strange alleyways smothered with snake oil, non-working addresses, and some developer who rolled up a demo and it ends up being on a financial site that I am supposed to be able to use to check a balance. Why, because no one is really paying attention, they just want to turn on the crypto, and who cares if it is valid.

But tell that to a browser. Any software with LDAP could fetch that certificate and make a comparison if the CAs supported the name space they have been exploiting. RFC4398...please give me a break. This is a systemic problem that is killing privacy for convenience.

John

March 23, 2011

Permalink

"Comodo also clearly demonstrate a mis-understanding - they believe that checks for revocation are proof positive that certificates are being used"

I might be being dense - but isn't this statement backwards. Shouldn't this be: they believe that NO checks for revocation are proof positive that the certificates are NOT being used.

John

March 24, 2011

Permalink

Report of incident on 15-MAR-2011
An RA suffered an attack that resulted in a breach of one user account of that specific RA.
This RA account was then used fraudulently to issue 9 certificates (across 7 different domains).

http://www.comodo.com/Comodo-Fraud-Incident-2011-03-23.html

This only means that someone had got hold of a RA username and password. Comodo uses 2Tiers for their reselling. This has nothing to do with any CA certificate or breach of the PKI infrastrukture. The Certificate was revoked, as normal procedure in such circumstances. However, customes are protected by the Comodo insurance if they suffered any loss because of this, based on the type of certificate that was issued.

If it was an attack by Iranian authorities, like they suggest, the loss will be that they are hanged slowly in the Evin prison. That's not covered by the insurance, I'm pretty sure.

John

March 24, 2011

Permalink

Infidels! One day great nation of Iran will have the funds to purchase their own CA like your NSA.

Hey, you still getting molested at the airport Jacob?

John

March 24, 2011

Permalink

In a perfect world
people who invest their time in detecting and reporting security holes which affect the entire world,
people who helped reporters from Egypt and Libya to bypass censorship
should deserve a medal.

In this brave new world they only get harassed in US airports.

John

March 24, 2011

Permalink

Jake, in your update you have harsh words for Comodo. This is surely justified.

Yet your own company, Tor, is hardly a sterling example of protecting its users.

The Torbutton extension for Firefox is one of the most frequently downloaded extensions on that site, 4,687,639 downloads to date and 182,056 daily users on March 23rd. Torbutton bundles Tor, Polipo, and Vidalia.

It advertises itself as "currently the only addon that will safely manage your Tor browsing to prevent IP address leakage, cookie leakage, and general privacy attacks", thus giving the user a reasonable expectation of privacy.

Yet it was last updated April 19, 2010 -- nearly a year ago!

Since then there have been many bugfixes to Tor, including fixes to serious security-relevant issues. When will these bugfixes find their way into Torbutton?

By not updating Torbutton, you are putting millions of users of the extension at risk. Some of them may be dissidents in brutal dictatorships and the consequences to them of being found out are dire.

You seem to misunderstand what Torbutton does. It only handles the Firefox side of things and has nothing to do with Tor core.

Aside from Firefox 4 compatibility and a couple addon conflicts, not much new in the way of security updates for Torbutton has been needed. If you want Firefox 4 support and fixes for some minor issues, use the alpha series at:
https://www.torproject.org/torbutton/

John

March 24, 2011

Permalink

The 'global trustee' cert sounds to me like it could be a signing certificate rather than simply a normal normal site certificate. If so then the attacker could sign further certs of their own. Even worse is that OCSP is generally only applied to the leaf of trust, meaning that even if OCSP worked properly it wouldn't help should this be true.

John

March 24, 2011

Permalink

Lots of talk about Mozilla, IE, and Chrome but what about Opera and Safari? Are they posting patches? Do they automatically check CRLs?

John

March 24, 2011

Permalink

Great post.

1. I have a question here, you mention how browsers soft fail on a OCSP/CRL attempts. Now, assuming they are coded to hard fail. Even then an adversary can MITM the OCSP request and defeat this process right ? In this case, the browsers will be helpless sitting birds, unless the OCSP contact made by browser to CA has some measures in place to avoid a MITM.

2. Opera seems to be maintaining a separate server of their own with certificates etc. Need to dig this deeper, but it seems to evade the whole CA down thing (assuming Opera will try to ensure that those servers are up)

John

March 24, 2011

Permalink

I wonder why if the lead byte of cert serial is > 0x7F they pre-pend that 0x00? Anyone gone through source to see why that is? Not an issue, just curious.

John

March 24, 2011

Permalink

What makes you guys think that solving for the CA's private key from the public key is all that hard. Have you looked into how much (or how little) computation power is need these days to do this? It certainly is within the capability of large corporations and even small countries.

The whole https security thing is bogus anyway unless the domain name server system can be secured. Most companies are better off using their own (self created) certificates and changing them frequently (like weekly or daily). The browser preapproved CA list just encourages forgery.

There is no proof that finding the private key from the public key is a 'hard' or 'large' problem in the mathematical algorithmic sense - only that most people don't know how to do it efficiently. In any event it is certainly no 'bigger' than the factoring problem (factoring the product of two large primes) in which there has been alot of progress algoritmically. Practically, in light of modern processors, say clusters of GPUs, backsolving private keys is almost personally affordable.

John

March 25, 2011

Permalink

Comodo has not been hacked, but the user credentials of a sovereign account at UserTrust were leaked. The scandal is not the hack, but that these accounts even exist and are allowed to generate certificates at will.

An unknown entity just tried to gain the same level of arbitrary access to the CA system, that western states already have.

John

March 25, 2011

Permalink

Amazing research, fascinating article. Thank you for your work on this issue and on Tor in general.

John

March 25, 2011

Permalink

I have some contacts with human rights activists working in Iran. What kind of precautions do they need to take in light of the possibility that the attack originated from the state?

John

March 25, 2011

Permalink

I notice more and more certificates are getting added and updated all the time. Is there a better and more secure and safe way than certificates or can more safeguards be applied to help keep certificates safe. If Iran did indeed have dealings in this then we must in carefully analyze the information and discover how to better safeguard and protect certificates to prevent man in the middle attacks from happening. Furthermore, what other safeguards besides obvious ones like passwords, biometrics, secure limited access, need to know data, etc. can be used to safeguard information. I know certain things could just be done the old fashioned way by pen and paper and safeguarded in a vault and/or an older computer that has access restricted to only a few individuals with supplies carefully protected and fully scanned to entrance to vault. We still need more and better ideas and soon. Thank you.

John

March 25, 2011

Permalink

I notice more and more certificates are getting added and updated all the time. Is there a better and more secure and safe way than certificates or can more safeguards be applied to help keep certificates safe. If Iran did indeed have dealings in this then we must in carefully analyze the information and discover how to better safeguard and protect certificates to prevent man in the middle attacks from happening. Furthermore, what other safeguards besides obvious ones like passwords, biometrics, secure limited access, need to know data, etc. can be used to safeguard information. I know certain things could just be done the old fashioned way by pen and paper and safeguarded in a vault and/or an older computer that has access restricted to only a few individuals with supplies carefully protected and fully scanned to entrance to vault. We still need more and better ideas and soon. Thank you.

John

March 25, 2011

Permalink

So how come security.OCSP.require is set to false in the latest Tor Browser Bundle if you yourself have repeatedly and strongly recommended setting it to true.

I don't produce the TBB. I've argued internally that we should enable this flag and this build fixes the immediate compromise problems at hand.

Anyone who hasn't upgraded should enable that flag by hand, of course.

Personally, I think that we should run an anonymous OCSP or CRL caching proxy on a hidden service. I don't have time to write one but it does seem like a reasonable compromise for hard failure modes and privacy concerns.

John

March 25, 2011

Permalink

Don't trust CAs that you do not actually know. Who is Comodo ? I don't know anyone there, hence I don't trust them. Same goes for all the others, I just live without any CAs on my "trusted browser".

I trust my friends CAs, I trust my bank and other services certificates because I manually verify them. For the non important stuff, I take the risk of reading an altered blog post (or use my "non trusted browser").

John

March 27, 2011

Permalink

It appears that the Microsoft update doesn’t just contain the certificate serial numbers, but the whole certificates. They include in the subject:

OU = Hosted by GTI Group Corporation

I’d guess that makes the registration authority GlobalTrust Inc (a division of GTI Group Corporation). They’re in Italy, which matches some news reports. Their websites have also done a disappearing act.

I wonder if this goes some way to explaining the strange "global trustee" certificate. I apologise if this is already common knowledge, but I’ve not seen it mentioned anywhere yet.

John

March 27, 2011

Permalink

The extent of the problem is surely a matter of whether the signing key was obtained or not. If the attackers obtained that, then revoking the certs issued on the CA's computer are almost immaterial as the attacker can continue issuing certs on their own computer. Gah, this is such a mess.

John

March 29, 2011

Permalink

Google's ones :
047ecbe9fca55f7bd09eae36e10cae1e
d8f35f4eb7872b2dab0692e315382fb0
b0b7133ed096f9b56fae91c874bd3ac0
9239d5348f40d1695a745470e1f23f43
d7558fdaf5f1105bb213282b707729a3
f5c86af36162f13a64f54f6dc9587c06

Mozilla's ones :
009239d5348f40d1695a745470e1f23f43
00d8f35f4eb7872b2dab0692e315382fb0
72032105c50c08573d8ea5304efee8b0
00b0b7133ed096f9b56fae91c874bd3ac0
00e9028b9578e415dc1a710a2b88154447
00d7558fdaf5f1105bb213282b707729a3
047ecbe9fca55f7bd09eae36e10cae1e
00f5c86af36162f13a64f54f6dc9587c06
392a434f0e07df1f8aa305de34e0c229
3e75ced46b693021218830ae86a82a71

G9239... == M009239 ...
Gf5c8 == M00f5c8
Gd755 == M00d755
Gb0b7 == M00b0b7
Gd8f3 = M00d8f3
G047ecb == M047ecb

Every hash in the Google list are found in the Mozilla list : just a matter of visual representation.
db