View source for Security/CryptoEngineering/Platform Use of NSS

=== The Current Setup ===
The collection of libraries known as NSS provides various cryptographic and security-related functionality to the platform (Gecko). In most cases, NSS must be initialized before the platform makes use of any function it exposes. Currently this requires initializing PSM, which takes care of finding the profile directory, initializing NSS using the certificate, key, and PKCS#11 module databases in that directory, and other configuration tasks such as enabling approved cipher suites. At that point, the platform is free to use NSS functionality. However, because NSS in theory could be shut down at any point (by the process of shutting down the browser as a whole), code must first acquire a 'nsNSSShutDownPreventionLock' and check if NSS has indeed already been shut down. Similarly, code that holds NSS resources must be aware of the possibility of NSS shutting down and be ready to release those resources by implementing the 'nsNSSShutDownObject' interface. Unfortunately, this scheme is difficult to work with from a development perspective and currently doesn't even cover all eventualities (for instance, if NSS has shut down, for objects created after that point, the "has NSS shut down" check will return the wrong answer).

=== The Desired Setup ===
The obvious solution would be to make NSS always be available (for some value of "always"). Given that the code using NSS is bound by the lifetime of XPCOM, guaranteeing that NSS be available for any XPCOM service/component seems like a good fit. The first hurdle here is that the profile directory is only available after XPCOM has been initialized (or sometimes not at all, in the case of xpcshell tests that never use a profile directory). Thus, the first significant difference in the new scheme is to initialize NSS early in XPCOM initialization in a memory-only mode where it doesn't open any persistent databases. It can be shut down again late in XPCOM teardown, when any XPCOM code that would use it has been terminated.

At this point, platform code that relies on NSS can safely run. Moreover, as long as that code doesn't run after XPCOM shutdown (and as long as it properly releases its NSS resources when it's done running), the nsNSSShutDownObject/nsNSSShutDownPreventionLock framework can be removed. This constitutes a major simplification in existing code and any future platform development using NSS.

The catch, of course, is that the user's persistent certificate, key, and PKCS#11 module databases won't be available when NSS is in this memory-only mode. To address this, PSM can call SECMOD_OpenUserDB with the user's profile directory. This will make these data and modules usable by the platform.

=== Remaining Issues ===

==== Use of PK11_GetInternalKeySlot() ====
PK11_GetInternalKeySlot is used to get a handle on the software token that backs the user's persistent certificate and key database. When NSS is initialized in memory-only mode, operations that expect to modify persistent data on this token will fail. Consequently, any platform code that calls PK11_GetInternalKeySlot must instead get a handle on the software slot/token as loaded by the call to SECMOD_OpenUserDB (see above). Similarly, any NSS code that calls PK11_GetInternalKeySlot must be avoided or worked around, as it will not work correctly. For example, CERT_AddTempCertToPerm (which is used to take an existing temporary certificate and save it in the persistent database) is hard-coded to use PK11_GetInternalKeySlot. However, PK11_ImportCert can be used in combination with CERT_ChangeCertTrust in place of CERT_AddTempCertToPerm to save the certificate on a specific slot (in this case, the real persistent database as loaded by SECMOD_OpenUserDB).

PK11SDR_Encrypt and PK11SDR_Decrypt have no equivalent functions that take a caller-provided slot. Either new functions will have to be added to NSS (e.g. as PK11SDR_EncryptOnSlot and PK11SDR_DecryptOnSlot) or they will have to be reimplemented in PSM. Luckily, these two functions do not call internal NSS APIs and can be easily duplicated in terms of functionality. Doing so might be desirable as it would enable a step towards towards using more modern cryptography to protect the information in the user's key database.

==== Loading New PKCS#11 Modules ====
Currently when a new PKCS#11 module is loaded, its presence is persisted in the user's module database, meaning that it will automatically be loaded the next time the platform runs. When NSS is initialized in memory-only mode, this will not work. Consequently, we must come up with some way of persisting the PKCS#11 modules the user wants the platform to load when it starts. Upon PSM initialization, the known modules can be loaded for the duration of the session.

==== PKCS#11 XPCOM APIs ====
Currently the platform exposes some PKCS#11 module APIs that include such functionality as getting a handle on the internal module, listing known modules, and searching for modules, tokens, and/or slots. Because the user's software token is now not the same as that returned by PK11_GetInternalKeySlot, these APIs must be special-cased to continue to behave as expected. For example, if the user searches for a module with a blank name, the internal module (as loaded by SECMOD_OpenUserDB) must be returned.

==== FIPS ====
Continuing to support FIPS with these changes appears to be problematic in two ways. First, since from NSS' perspective the "internal" database is loaded in memory-only mode, that the user put the database in FIPS mode isn't persisted across restarts. This can be addressed by a mechanism similar to or along side that which persists new PKCS#11 modules (see above). Second, turning on FIPS mode appears to unload the real persistent certificate and key database. It may be that this can simply be reloaded as part of the process of enabling FIPS. Further investigation in required here.

=== Resolved Issues ===

==== The NSS Certificate Nickname API ====
NSS exposes APIs whereby certificates can be referred to by nickname. Certificates on tokens other than that returned by PK11_GetInternalKeySlot prefix their nickname with the name of the token. Because platform code now must operate on a token that isn't the internal one, this behavior must be worked around by special-casing unprefixed nicknames when using these and related APIs.

Alternatively, we could remove and/or rework XPCOM interfaces that expose the NSS nickname API and replace them with an equivalent mechanism that doesn't have this and other drawbacks (see the discussion in [https://bugzilla.mozilla.org/show_bug.cgi?id=857627 bug 857627]).

Update: As of the landing of bug 857627, this shouldn't be an issue any longer.