SecurityEngineering/Certificate Verification

Certificate Verification in Firefox Today

Firefox currently relies on NSS to implement various cryptographic functions. NSS consists of a collection of loosely-coupled libraries. libssl, for example, is the TLS implementation. NSS is a Mozilla project, but its development differs significantly from the rest of the tree. In fact, it has its own tree that is periodically imported wholesale into mozilla-central. The component that uses the NSS libraries in Firefox is a layer called PSM ("Personal Security Manager" or "Privacy and Security Module").

To enable secure TLS connections to the best of our ability, PSM implements a certificate verification callback. It performs a number of checks, but ultimately it must determine if it trusts a certificate presented by a peer. It currently does this by calling one of two certificate verification libraries in NSS: "classic" or libpkix. The classic library handles DV ("domain validation") certificates while libpkix handles EV ("extended validation") certificates.

"classic" verification

The classic certificate verification algorithm performs issuer-independent checks on the given certificate, finds a potential issuer, verifies that the signature matches, and recurses. If multiple issuers are found, it attempts to use the "best" one. However, this is a heuristic, and as it does not perform backtracking, it can fail to verify a valid certificate. This is spectacularly apparent in the case of key-pinning if the algorithm chooses to not traverse a certificate path that contains a necessary key.

Because this library is written in C and because NSS makes strong guarantees about API backwards-compatibility, it would require significant work to fix. This would be on par with writing a new verification library.

The code is here: https://mxr.mozilla.org/mozilla-central/source/security/nss/lib/certhigh/

libpkix

libpkix was auto-translated from Java to C. It attempts to implement Java's exception semantics in C. It makes liberal use of unclear macros (e.g. https://mxr.mozilla.org/mozilla-central/source/security/nss/lib/libpkix/pkix/util/pkix_tools.h#67 ). A source-line-counting tool clocks it in at 45,000 lines of code (the code is here: https://mxr.mozilla.org/mozilla-central/source/security/nss/lib/libpkix ). There are known bugs in the implementation. No one who works on it wants to continue working on it.

These libraries do not serve our needs and are impeding progress.

Certificate Verification in Firefox Tomorrow

We have a number of options:

1. We can fix the classic verification implementation. As already stated, this would require considerable work.
2. We can fix and maintain libpkix ourselves. This is undesirable for the aforementioned reasons. Furthermore, as Google moves away from NSS, we will have less and less help working on this library.
3. We can use whatever verification implementation Google develops. We may have to wait a year or more for this, and we would be depending on Google to share their work.
4. We can use OpenSSL's certificate verification routine. Apparently it is buggy as well.
5. We can start over from scratch and write another entirely new verification library. This would set us back a year.
6. Finally, we can use the new verification library known as "mozilla::pkix" (formerly known as "insanity::pkix").

mozilla::pkix Design

As a library, mozilla::pkix uses the notion of a "trust domain" provided by the application to build a trusted chain from an end-entity certificate to a root. The trust domain is responsible for saying what trust level a certificate has, finding potential issuers of a certificate, and checking the revocation for a certificate. A certificate can be a trust anchor, it can inherit its trust, or it can be actively distrusted. Given an end-entity certificate and a trust domain, the library will perform issuer-independent checks on that certificate (e.g. expiration, appropriate key usages), get a list of potential issuers, and perform a depth-first traversal. If it encounters a distrusted certificate, it abandons searching that path. If it finds a trust anchor, it returns that successful path (here is where we will add a callback to the trust domain interface that checks the otherwise good chain for application-specific features, such as the presence of a specific key).

Unlike the NSS libraries, mozilla::pkix is written in C++. As a result, we can use scoped data types that automatically clean up after themselves rather than having to manually manage memory. This reduces memory-safety bugs as well as error-handling bugs.

Progress

We have been working on this project for a long time. Last year, progress was slower than everyone would have liked. However, starting in late January, development picked up considerably to the point where we had landed a working implementation (albeit with no OCSP checking) within a month. Since then, we have landed OCSP checking and test improvements, and we are about to land the OCSP cache. The library code is here: https://mxr.mozilla.org/mozilla-central/source/security/pkix/ and the trust domain is here: https://mxr.mozilla.org/mozilla-central/source/security/certverifier/

Tests

Due to the sensitive nature of this code, we want to ensure proper testing. To that end, we first made sure the new implementation passed the same tests as the current implementation. We then added more tests, finding some bugs in both implementations in the process. At this point, while we will add still more tests, we believe it would be beneficial for the community at large to inspect the design and implementation of the code. Note that this stems not from a lack of confidence in code quality but rather the understanding that the privacy of our users depends on the correctness of this code.

Matt Wobensmith just completed compatibility-testing of 200k HTTPS sites and found 16 with issues. These are being investigated.

Implementation status

Released in FF 31.

Remaining bugs:

1. Enforce consistent handling of isCA bit and certSign/crlSign key usages: bug 970196 (:briansmith)
2. Add low-level DER decoder tests: bug 968490 (:st3fan) [code written - needs review]
3. Enable all PSM xpcshell tests on Android/B2G: bug 676972 (:briansmith) [code mostly written - needs review]
4. Add SHA-2 support to the OCSP implementation: bug 966856 (:keeler) [code written - needs review]
5. Test decoding OCSP responses with multiple certificates: bug 972753 (:keeler) [this may take a week or two]
6. Adjust OCSP stapling telemetry: bug 969048 (:keeler) [code written and reviewed - can land after the OCSP cache lands]
7. Improve error handling in VerifyEncodedOCSPResponse: bug 977870 (:keeler) [code written - needs review]
8. Document functions exported from the library: bug 968451 (:briansmith)

For more details, see the dependency trees for bug 915930 and bug 976961, respectively.

Action Items

• Keeler/Camilo continue landing prefing on in Nightly and Aurora only, not in release/Beta
• Richard/Doug/Ekr work on collaboration with Google's openssl efforts to see if there's anything to do in the immediate term
• Doug to talk to Bob Relyea about reviewing mozilla::pkix
• Richard or Ekr evaluate openssl certificate verification path
• Ekr to reach out to rsleevi to start collaboration
• Richard to reach out to openssl

Choosing your Verification Library

mozilla::pkix has been enabled by default in Nightly. The boolean pref "security.use_mozillapkix_verification" controls this. There is an additional hidden boolean pref "security.use_libpkix_verification" that can be used to enable libpkix instead. If both of these prefs are false, classic verification is used. mozilla::pkix takes precedence over libpkix. See the following table:

		security.use_libpkix_verification
		false (default)	true
security.use_mozillapkix_verification	false	Classic	libpkix
	true (default)	mozilla::pkix	mozilla::pkix