SecurityEngineering/Certificate Verification: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (added link)
 
(33 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== Certificate Verification in Firefox Today ==
== Background ==


Firefox currently relies on NSS to implement various cryptographic functions. NSS consists of a collection of loosely-coupled libraries. libssl, for example, is the TLS implementation. NSS is a Mozilla project, but its development differs significantly from the rest of the tree. In fact, it has its own tree that is periodically imported wholesale into mozilla-central. The component that uses the NSS libraries in Firefox is a layer called PSM ("Personal Security Manager" or "Privacy and Security Module").
[https://www.guru99.com/gecko-marionette-driver-selenium.html Gecko] (and therefore Firefox) relies on [https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS NSS] to implement various cryptographic functions. NSS consists of a collection of [https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/NSS_API_Guidelines loosely-coupled libraries]. libssl, for example, is the TLS implementation. NSS is a Mozilla project, but its development differs significantly from the rest of the tree. In fact, it has its own tree that is periodically imported wholesale into [https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/mozilla-central mozilla-central]. The component that uses the NSS libraries in Firefox is a layer called [https://developer.mozilla.org/en-US/docs/Mozilla/Projects/PSM PSM] ("Personal Security Manager" or "Privacy and Security Module").


To enable secure TLS connections to the best of our ability, PSM implements a certificate verification callback. It performs a number of checks, but ultimately it must determine if it trusts a certificate presented by a peer. It currently does this by calling one of two certificate verification libraries in NSS: "classic" or libpkix. The classic library handles DV ("domain validation") certificates while libpkix handles EV ("extended validation") certificates.
To enable secure TLS connections to the best of our ability, PSM implements a certificate verification callback. It performs a number of checks, but ultimately it must determine if it trusts a certificate presented by a peer. The approach PSM takes is to repeatedly build a potential path to a [https://en.wikipedia.org/wiki/Trust_anchor trust anchor], validate the potential path, and either return that path if it validates correctly or find another potential path. [https://blog.mozilla.org/security/2014/04/24/exciting-updates-to-certificate-verification-in-gecko/ mozilla::pkix] is a C++ library that provides a framework to implement this approach.


=== "classic" verification ===
== mozilla::pkix ==


The classic certificate verification algorithm performs issuer-independent checks on the given certificate, finds a potential issuer, verifies that the signature matches, and recurses. If multiple issuers are found, it attempts to use the "best" one. However, this is a heuristic, and as it does not perform backtracking, it can fail to verify a valid certificate. This is spectacularly apparent in the case of key-pinning if the algorithm chooses to not traverse a certificate path that contains a necessary key.
mozilla::pkix was originally implemented as part of mozilla-central (i.e. gecko) but has since been moved into NSS. However, it is not part of NSS' stable C API. As a library, mozilla::pkix uses the notion of a "trust domain" provided by the application to build a trusted chain from an end-entity certificate to a root. The trust domain is responsible for saying what trust level a certificate has, finding potential issuers of a certificate, and [[CA/Revocation_Checking_in_Firefox|checking the revocation for a certificate]]. A certificate can be a trust anchor, it can inherit its trust, or it can be [[CA/Maintenance_and_Enforcement#Actively_Distrusting_a_Certificate|actively distrusted]]. Given an end-entity certificate and a trust domain, the library will perform issuer-independent checks on that certificate (e.g. expiration, appropriate key usages), get a list of potential issuers, and perform a [https://en.wikipedia.org/wiki/Depth-first_search depth-first traversal]. If it encounters a distrusted certificate, it abandons searching that path. If it finds a trust anchor, it queries the trust domain again to see if that path is acceptable (this is where gecko implements checks that are specific to the platform and not the abstract problem of building a trusted certificate chain). If so, the end-entity certificate has successfully been verified.


Because this library is written in C and because NSS makes strong guarantees about API backwards-compatibility, it would require significant work to fix. This would be on par with writing a new verification library.
Because mozilla::pkix uses a depth-first strategy rather than a [https://en.wikipedia.org/wiki/Breadth-first_search breadth-first] one, it is not guaranteed to find the shortest path from an end-entity certificate to a trust anchor. However, as a heuristic, the trust domain implemented by the platform prefers trust anchors at each step. That is, when mozilla::pkix asks for an issuer certificate, the trust domain will first try any available trust anchors before trying non-trust anchors.


The code is here: https://mxr.mozilla.org/mozilla-central/source/security/nss/lib/certhigh/
For example, suppose an end-entity certificate E were signed by intermediate I that was signed by root R. Further suppose that R were cross-signed by another root T, producing intermediate certificate R'. Two valid paths are E -> I -> R and E -> I -> R' -> T. At the step where mozilla::pkix is trying to find an issuer for I, the trust domain finds both R and R' as potential issuers. However, only R is a trust anchor - R' is not. So, the trust domain has mozilla::pkix try that one first. It successfully verifies, so the library returns the result E -> I -> R. If at some later point R were distrusted, that chain would not verify successfully. mozilla::pkix would then ask the trust domain for another potential issuer of I, and the trust domain would provide R'. R' is not a trust anchor either, so mozilla::pkix asks for potential issuers of that certificate. The trust domain provides T, and mozilla::pkix verifies the chain, resulting in E -> I -> R' -> T.


=== libpkix ===
Extended Validation changes the situation somewhat in that when mozilla::pkix is verifying an EV certificate, it must do so for a particular policy OID, and only certain roots are trusted for particular policy OIDs. This policy OID will be the first policy OID that is recognized by Firefox as a supported EV policy OID from the end-entity's certificatePolicies extension. Other than that, path building proceeds as before. So, using the example from before, suppose R were a trust anchor but not for a particular EV OID. Further suppose that T is also a trust anchor but is trusted for a particular EV OID. When evaluating the potential chain E -> I -> R, the trust domain determines that R is not a trust anchor for the policy being verified for, so mozilla::pkix keeps searching. It finds potential chain E -> I -> R' -> T, where T is trusted for that EV OID, and thus it verifies and returns that chain.
libpkix was auto-translated from Java to C. It attempts to implement Java's exception semantics in C. It makes liberal use of unclear macros (e.g. https://mxr.mozilla.org/mozilla-central/source/security/nss/lib/libpkix/pkix/util/pkix_tools.h#67 ). A source-line-counting tool clocks it in at 45,000 lines of code (the code is here: https://mxr.mozilla.org/mozilla-central/source/security/nss/lib/libpkix ). There are known bugs in the implementation. No one who works on it wants to continue working on it.


These libraries do not serve our needs and are impeding progress.
Unlike the other NSS libraries, mozilla::pkix is written in C++ and can take advantage of more modern language features.


== Certificate Verification in Firefox Tomorrow ==
=== Trust Anchors ===
We have a number of options:
# We can fix the classic verification implementation. As already stated, this would require considerable work.
# We can fix and maintain libpkix ourselves. This is undesirable for the aforementioned reasons. Furthermore, as Google moves away from NSS, we will have less and less help working on this library.
# We can use whatever verification implementation Google develops. We may have to wait a year or more for this, and we would be depending on Google to share their work.
# We can use OpenSSL's certificate verification routine. Apparently it is buggy as well.
# We can start over from scratch and write another entirely new verification library. This would set us back a year.
# Finally, we can use the new verification library known as "insanity::pkix".


=== insanity::pkix Design ===
The platform looks for trust anchors in a few locations. First, Mozilla ships a list of trust anchors with the platform corresponding to the [[CA/Included_Certificates|root Certificate Authorities (CAs) in the Mozilla Root CA Program]]. Additionally, the user may [[PSM:Changing_Trust_Settings|import their own trust anchors]]. These are stored in the profile's [[NSS_Shared_DB|cert9.db file]]. The user may also import third-party PKCS#11 modules that provide trust anchors. The [https://support.mozilla.org/en-US/kb/setting-certificate-authorities-firefox enterprise roots feature], if enabled, may collect trust anchors provided by the operating system.
As a library, insanity::pkix uses the notion of a "trust domain" provided by the application to build a trusted chain from an end-entity certificate to a root. The trust domain is responsible for saying what trust level a certificate has, finding potential issuers of a certificate, and checking the revocation for a certificate. A certificate can be a trust anchor, it can inherit its trust, or it can be actively distrusted. Given an end-entity certificate and a trust domain, the library will perform issuer-independent checks on that certificate (e.g. expiration, appropriate key usages), get a list of potential issuers, and perform a depth-first traversal. If it encounters a distrusted certificate, it abandons searching that path. If it finds a trust anchor, it returns that successful path (here is where we will add a callback to the trust domain interface that checks the otherwise good chain for application-specific features, such as the presence of a specific key).


Unlike the NSS libraries, insanity::pkix is written in C++. As a result, we can use scoped data types that automatically clean up after themselves rather than having to manually manage memory. This reduces memory-safety bugs as well as error-handling bugs.
=== Intermediate Certificates ===


=== Progress ===
Similarly, the platform gathers [[CA/Intermediate_Certificates|intermediate certificates]] from a few locations. The TLS specification mandates that the peer include in the handshake any necessary intermediate certificates to verify the [https://en.wikipedia.org/wiki/Public_key_certificate#End-entity_or_leaf_certificate end-entity]. In practice this may not be the case, but the platform uses these if they are available. The user may manually add intermediate certificates. Again these are stored in the profile's cert9.db file. The enterprise roots feature also looks for intermediates provided by the operating system.
We have been working on this project for a long time. Last year, progress was slower than everyone would have liked. However, starting in late January, development picked up considerably to the point where we had landed a working implementation (albeit with no OCSP checking) within a month. Since then, we have landed OCSP checking and test improvements, and we are about to land the OCSP cache.
The library code is here: https://mxr.mozilla.org/mozilla-central/source/security/insanity/ and the trust domain is here: https://mxr.mozilla.org/mozilla-central/source/security/certverifier/


These are bugs that have been fixed since January:
When the platform successfully verifies an end-entity certificate, it caches the intermediates from that verified chain in the profile (cert9.db) in case they will be useful in the future (for example, when connecting to a different peer that uses a certificate issued by the same CA but neglects to include intermediate certificates in the handshake).


https://bugzilla.mozilla.org/buglist.cgi?j_top=OR&emailtype3=exact&f1=assigned_to&o3=equals&email3=brian%40briansmith.org&list_id=9671843&v3=brian%40briansmith.org&o1=equals&resolution=FIXED&emailtype1=exact&o2=equals&chfieldto=Now&query_format=advanced&chfield=resolution&f3=assigned_to&chfieldfrom=2014-01-01&f2=assigned_to&chfieldvalue=FIXED&bug_status=RESOLVED&bug_status=VERIFIED&email1=brian%40briansmith.org&v1=dkeeler%40mozilla.com&component=Security%3A%20PSM&v2=cviecco%40mozilla.com&product=Core
Note that Firefox does not do AIA chasing. That is, it does not use information embedded in certificates to remotely fetch potential issuer certificates. So, if a server does not include the appropriate intermediate certificates in the TLS handshake, Firefox may not be able to verify its certificate.
(not all of them are insanity::pkix-related - search for "insanity::pkix" or anything certificate- or test-related)


=== Tests ===
=== Extended Validation ===
Due to the sensitive nature of this code, we want to ensure proper testing. To that end, we first made sure the new implementation passed the same tests as the current implementation. We then added more tests, finding some bugs in both implementations in the process. At this point, while we will add still more tests, we believe it would be beneficial for the community at large to inspect the design and implementation of the code. Note that this stems not from a lack of confidence in code quality but rather the understanding that the privacy of our users depends on the correctness of this code.


Matt Wobensmith just completed compatibility-testing of 200k HTTPS sites and found 16 with issues. These are being investigated.
As part of [[CA|Mozilla's Root CA Program]], there is a list of root certificates that are trusted to issue [[CA/EV_Processing_for_CAs|Extended Validation (EV) certificates]]. This list is available in code form at https://hg.mozilla.org/mozilla-central/annotate/tip/security/certverifier/ExtendedValidation.cpp.


=== The Plan ===
== Other Verification Routines in NSS ==
Some work remains on insanity::pkix. We have broken this work into two parts: prerequisites for it to be enabled by default on Nightly, and prerequisites for it to be enabled by default on Beta and then Release. There is not enough bake time left on Nightly 30, so Nightly 31 will be the first version with this on by default.


To turn insanity::pkix on in Nightly, we need to:
NSS exposes other certificate verification functions that are not yet implemented using mozilla::pkix. The following is informational and has little bearing on the behavior of Firefox.


# Add low-level OCSP unit tests: {{Bug|916629}} (:briansmith, :st3fan)
=== "classic" verification ===
# Test that results from the certificate database are interpreted correctly: {{Bug|966820}} (:cviecco)
# Test that x509 v1 and v2 certificates are handled correctly {{Bug|969188}} (:cviecco)
# Expand EKU (extended key usage) tests: {{Bug|970470}} (:cviecco)


These items should be done by the end of next week.
The so-called "classic" certificate verification algorithm performs issuer-independent checks on the given certificate, finds a potential issuer, verifies that the signature matches, and recurses. If multiple issuers are found, it attempts to use the "best" one. However, this is a heuristic, and as it does not perform backtracking, it can fail to verify a valid certificate. This is spectacularly apparent in the case of key-pinning if the algorithm chooses to not traverse a certificate path that contains a necessary key.


To turn insanity::pkix on in Beta/Release, we need to:
The code is here: https://hg.mozilla.org/projects/nss/file/tip/lib/certhigh


# Add backoff for OCSP requests when the responder fails: {{bug|977865}} (:keeler) [this may take a week or two]
=== libpkix ===
# Enforce consistent handling of isCA bit and certSign/crlSign key usages: {{bug|970196}} (:briansmith)
libpkix was auto-translated from Java to C. It attempts to implement Java's exception semantics in C. It makes liberal use of unclear macros (e.g. https://hg.mozilla.org/projects/nss/annotate/5d9f8b809e6f7020529ba1345b64e36f61994c8d/lib/libpkix/pkix/util/pkix_tools.h#l67 ). A source-line-counting tool clocks it in at 45,000 lines of code (the code is here: https://hg.mozilla.org/projects/nss/file/tip/lib/libpkix ). There are known bugs in the implementation.
# Add low-level DER decoder tests: {{bug|968490}} (:st3fan) [code written - needs review]
# Enable all PSM xpcshell tests on Android/B2G: {{bug|676972}} (:briansmith) [code mostly written - needs review]
# Add SHA-2 support to the OCSP implementation: {{bug|966856}} (:keeler) [code written - needs review]
# Test decoding OCSP responses with multiple certificates: {{bug|972753}} (:keeler) [this may take a week or two]
# Adjust OCSP stapling telemetry: {{bug|969048}} (:keeler) [code written and reviewed - can land after the OCSP cache lands]
# Improve error handling in VerifyEncodedOCSPResponse: {{bug|977870}} (:keeler) [code written - needs review]
# Document functions exported from the library: {{bug|968451}} (:briansmith)
 
These items should be done by April 28.
 
For more details, see the dependency trees for {{bug|915930}} and {{bug|976961}}, respectively.
 
=== Action Items ===
* Keeler/Camilo continue landing prefing on in Nightly and Aurora only, not in release/Beta
* Richard/Doug/Ekr work on collaboration with Google's openssl efforts to see if there's anything to do in the immediate term
* Doug to talk to Bob Relyea about reviewing insanity
* Richard or Ekr evaluate openssl certificate verification path
* Ekr to reach out to rsleevi to start collaboration
* Richard to reach out to openssl
* Keeler to schedule an actual code review

Latest revision as of 23:25, 13 April 2022

Background

Gecko (and therefore Firefox) relies on NSS to implement various cryptographic functions. NSS consists of a collection of loosely-coupled libraries. libssl, for example, is the TLS implementation. NSS is a Mozilla project, but its development differs significantly from the rest of the tree. In fact, it has its own tree that is periodically imported wholesale into mozilla-central. The component that uses the NSS libraries in Firefox is a layer called PSM ("Personal Security Manager" or "Privacy and Security Module").

To enable secure TLS connections to the best of our ability, PSM implements a certificate verification callback. It performs a number of checks, but ultimately it must determine if it trusts a certificate presented by a peer. The approach PSM takes is to repeatedly build a potential path to a trust anchor, validate the potential path, and either return that path if it validates correctly or find another potential path. mozilla::pkix is a C++ library that provides a framework to implement this approach.

mozilla::pkix

mozilla::pkix was originally implemented as part of mozilla-central (i.e. gecko) but has since been moved into NSS. However, it is not part of NSS' stable C API. As a library, mozilla::pkix uses the notion of a "trust domain" provided by the application to build a trusted chain from an end-entity certificate to a root. The trust domain is responsible for saying what trust level a certificate has, finding potential issuers of a certificate, and checking the revocation for a certificate. A certificate can be a trust anchor, it can inherit its trust, or it can be actively distrusted. Given an end-entity certificate and a trust domain, the library will perform issuer-independent checks on that certificate (e.g. expiration, appropriate key usages), get a list of potential issuers, and perform a depth-first traversal. If it encounters a distrusted certificate, it abandons searching that path. If it finds a trust anchor, it queries the trust domain again to see if that path is acceptable (this is where gecko implements checks that are specific to the platform and not the abstract problem of building a trusted certificate chain). If so, the end-entity certificate has successfully been verified.

Because mozilla::pkix uses a depth-first strategy rather than a breadth-first one, it is not guaranteed to find the shortest path from an end-entity certificate to a trust anchor. However, as a heuristic, the trust domain implemented by the platform prefers trust anchors at each step. That is, when mozilla::pkix asks for an issuer certificate, the trust domain will first try any available trust anchors before trying non-trust anchors.

For example, suppose an end-entity certificate E were signed by intermediate I that was signed by root R. Further suppose that R were cross-signed by another root T, producing intermediate certificate R'. Two valid paths are E -> I -> R and E -> I -> R' -> T. At the step where mozilla::pkix is trying to find an issuer for I, the trust domain finds both R and R' as potential issuers. However, only R is a trust anchor - R' is not. So, the trust domain has mozilla::pkix try that one first. It successfully verifies, so the library returns the result E -> I -> R. If at some later point R were distrusted, that chain would not verify successfully. mozilla::pkix would then ask the trust domain for another potential issuer of I, and the trust domain would provide R'. R' is not a trust anchor either, so mozilla::pkix asks for potential issuers of that certificate. The trust domain provides T, and mozilla::pkix verifies the chain, resulting in E -> I -> R' -> T.

Extended Validation changes the situation somewhat in that when mozilla::pkix is verifying an EV certificate, it must do so for a particular policy OID, and only certain roots are trusted for particular policy OIDs. This policy OID will be the first policy OID that is recognized by Firefox as a supported EV policy OID from the end-entity's certificatePolicies extension. Other than that, path building proceeds as before. So, using the example from before, suppose R were a trust anchor but not for a particular EV OID. Further suppose that T is also a trust anchor but is trusted for a particular EV OID. When evaluating the potential chain E -> I -> R, the trust domain determines that R is not a trust anchor for the policy being verified for, so mozilla::pkix keeps searching. It finds potential chain E -> I -> R' -> T, where T is trusted for that EV OID, and thus it verifies and returns that chain.

Unlike the other NSS libraries, mozilla::pkix is written in C++ and can take advantage of more modern language features.

Trust Anchors

The platform looks for trust anchors in a few locations. First, Mozilla ships a list of trust anchors with the platform corresponding to the root Certificate Authorities (CAs) in the Mozilla Root CA Program. Additionally, the user may import their own trust anchors. These are stored in the profile's cert9.db file. The user may also import third-party PKCS#11 modules that provide trust anchors. The enterprise roots feature, if enabled, may collect trust anchors provided by the operating system.

Intermediate Certificates

Similarly, the platform gathers intermediate certificates from a few locations. The TLS specification mandates that the peer include in the handshake any necessary intermediate certificates to verify the end-entity. In practice this may not be the case, but the platform uses these if they are available. The user may manually add intermediate certificates. Again these are stored in the profile's cert9.db file. The enterprise roots feature also looks for intermediates provided by the operating system.

When the platform successfully verifies an end-entity certificate, it caches the intermediates from that verified chain in the profile (cert9.db) in case they will be useful in the future (for example, when connecting to a different peer that uses a certificate issued by the same CA but neglects to include intermediate certificates in the handshake).

Note that Firefox does not do AIA chasing. That is, it does not use information embedded in certificates to remotely fetch potential issuer certificates. So, if a server does not include the appropriate intermediate certificates in the TLS handshake, Firefox may not be able to verify its certificate.

Extended Validation

As part of Mozilla's Root CA Program, there is a list of root certificates that are trusted to issue Extended Validation (EV) certificates. This list is available in code form at https://hg.mozilla.org/mozilla-central/annotate/tip/security/certverifier/ExtendedValidation.cpp.

Other Verification Routines in NSS

NSS exposes other certificate verification functions that are not yet implemented using mozilla::pkix. The following is informational and has little bearing on the behavior of Firefox.

"classic" verification

The so-called "classic" certificate verification algorithm performs issuer-independent checks on the given certificate, finds a potential issuer, verifies that the signature matches, and recurses. If multiple issuers are found, it attempts to use the "best" one. However, this is a heuristic, and as it does not perform backtracking, it can fail to verify a valid certificate. This is spectacularly apparent in the case of key-pinning if the algorithm chooses to not traverse a certificate path that contains a necessary key.

The code is here: https://hg.mozilla.org/projects/nss/file/tip/lib/certhigh

libpkix

libpkix was auto-translated from Java to C. It attempts to implement Java's exception semantics in C. It makes liberal use of unclear macros (e.g. https://hg.mozilla.org/projects/nss/annotate/5d9f8b809e6f7020529ba1345b64e36f61994c8d/lib/libpkix/pkix/util/pkix_tools.h#l67 ). A source-line-counting tool clocks it in at 45,000 lines of code (the code is here: https://hg.mozilla.org/projects/nss/file/tip/lib/libpkix ). There are known bugs in the implementation.

Retrieved from "https://wiki.allizom.org/index.php?title=SecurityEngineering/Certificate_Verification&oldid=1241878"