SecurityEngineering/NSS Startup and Shutdown in Gecko

---

Draft

---

This is an informational document outlining the modernization and simplification of NSS startup and shutdown in Gecko. It is organized in three parts: the current setup, the desired setup, and a roadmap for achieving the desired setup. If the current date is later than 1 November 2016, this document is likely out of date.

The Current Setup

Classes in PSM (and other parts of Gecko) implement a number of interfaces that require NSS functionality. Instantiating any of these classes causes the PSM component to be initialized. This initialization starts a few services needed by certificate verification. It also performs the initialization of NSS. This involves calling the overall NSS_Initialize function as well as some configuration options like only enabling specific cipher suites and loading the trust anchors for certificate verification.

The PSM component observes a number of events, including notification of profile changes, preference changes, and XPCOM shutdown. Upon receiving the "profile-before-change" notification, the PSM component releases all NSS resources held by PSM objects and calls NSS_Shutdown. Consequently, whenever a PSM object attempts to use an NSS resource or call an NSS function, it first must check if NSS has been shut down. This is coordinated by having such classes inherit from nsNSSShutDownObject. At each point NSS resources are used, these classes first acquire an nsNSSShutDownPreventionLock and check isAlreadyShutDown() (see nsNSSShutDown.h).

On the topic of profile changes, the current implementation implies that profile switching is supported. That is, the implementation expects that Gecko may initialize NSS in one profile, shut it down, change profiles, and re-initialize NSS (all in the same process). This functionality has not been supported for a long time.

As a consequence of this design, when implementing new functionality that uses NSS, it is easy to do the wrong thing. For instance, code that merely calls NSS functions but do not hold NSS resources may forget to check and prevent NSS from shutting down during the use of that function. Another common mistake is to acquire the nsNSSShutDownPreventionLock but not actually check if NSS has been shut down. These have often led to shutdown crashes (see for example bug 1114741, bug 1046221, bug 1029173, and bug 911336).

The Desired Setup

NSS should be initialized exactly once and shut down exactly once. Code that uses it should only be able to run after NSS is guaranteed to be initialized. While such code is running, it should prevent NSS from being shut down out from under it. Once NSS has been shut down (upon notification that the entire process is shutting down), all methods that would use NSS should check for this and return an error.

Writing new code that correctly deals with these restrictions should be easy.

How to Get There

• Remove unused and/or unnecessary cruft from PSM/NSS initialization (bug 1215267)
  • nsPSMUITracker has already been removed: bug 1215690)
  • Profile change event handling: bug 1222179
  • (etc.)
• Fix all NSS shutdown leaks: bug 1230312
• Make NSS shutdown leaks fatal
• Separate NSS-only initialization from PSM component initialization
• Ensure NSS is initialized before execution reaches any code that requires it