MozillaWiki

NSS Shared DB: Difference between revisions

Page Discussion

NSS Shared DB (view source)

Revision as of 21:16, 23 March 2018

13,159 bytes removed ,  23 March 2018
→‎Shared Database Proposal: Fix NSS version number for the SQLite change.
(→‎Shared Database Proposal: Fix NSS version number for the SQLite change.)
 
(22 intermediate revisions by 6 users not shown)
Line 1: Line 1:
== Shared Database Proposal ==
== Shared Database Proposal ==


NSS has been using an old version of the Berkeley DataBase as its database engine since Netscape Navigator 2.0 in 1994. This database engine is commonly described in NSS documents as "DBM" and has a  
Prior to version 3.35, NSS had been using an old version of the Berkeley DataBase as its database engine since Netscape Navigator 2.0 in 1994. This database engine is commonly described in NSS documents as "DBM" and has a  
number of limitations.  One of the most severe limitations concerns the  
number of limitations.  One of the most severe limitations concerns the  
number of processes that may share a database file.  While any process has  
number of processes that may share a database file.  While any process has  
Line 7: Line 7:
Multiple processes may share a DBM database ONLY if they ALL access it  
Multiple processes may share a DBM database ONLY if they ALL access it  
READ-ONLY.  Processes cannot share a DBM database file if ANY of them wants
READ-ONLY.  Processes cannot share a DBM database file if ANY of them wants
to update it.
to update it.
 
NOTE: Since 3.35, NSS has moved to an [[NSS SQLite-based DB]].


This limitation has been cumbersome for applications that wish to use NSS.   
This limitation has been cumbersome for applications that wish to use NSS.   
Line 13: Line 15:


*Synchronized updates, with application down time: The applications share the database read-only.  If any update is desired, all the applications are shut down, and a database update program performs the update, then all the applications are restarted in read-only mode.  Some server products, for example, have an administration program that stops the servers, updates the database that they share, and then restarts the servers.  This results in undesirable downtime and desired database changes are delayed until the next interval in which such downtime is acceptable.
*Synchronized updates, with application down time: The applications share the database read-only.  If any update is desired, all the applications are shut down, and a database update program performs the update, then all the applications are restarted in read-only mode.  Some server products, for example, have an administration program that stops the servers, updates the database that they share, and then restarts the servers.  This results in undesirable downtime and desired database changes are delayed until the next interval in which such downtime is acceptable.
*Multiple copies with duplicated updates.  Each application keeps its own copy of its databases, and applications communicate their changes to each other, so that each application may apply received changes to its own DB. FireFox and Thunderbird are examples of this.  When one of those applications gets a new certificate and private key, the user may "export" that pair to a PKCS#12 file, and then import that file into the other application.  Most users never master these steps, and so have databases entirely out of sync.
*Multiple copies with duplicated updates.  Each application keeps its own copy of its databases, and applications communicate their changes to each other, so that each application may apply received changes to its own database. FireFox and Thunderbird are examples of this.  When one of those applications gets a new certificate and private key, the user may "export" that pair to a PKCS#12 file, and then import that file into the other application.  Most users never master these steps, and so have databases entirely out of sync.


These workarounds for the DBM engine's limitations are sufficiently onerous  
These workarounds for the DBM engine's limitations are sufficiently onerous  
Line 20: Line 22:


In 2001 NSS was modified to enable applications to supply their own database engines.  Applications could share a common database if they supplied their  
In 2001 NSS was modified to enable applications to supply their own database engines.  Applications could share a common database if they supplied their  
own shared database implementation, and configured NSS to use it.   
own shareable database implementation, and configured NSS to use it.   


Today, there exists a process level, ACID, open source, and widely available database engine with which multiple processes may simultaneous have read and write access to a shared database.  It is named SQLite.  The NSS team proposes to use this database to give all NSS-based applications Shared Database access.
Today, there exists a process level, ACID, open source, and widely available database engine with which multiple processes may simultaneous have read and write access to a shared database.  It is named SQLite.  The NSS team proposes to use this database to give all NSS-based applications Shared Database access.
Line 31: Line 33:
=== Where we are today ===
=== Where we are today ===


At initialization time, the application gives NSS a string that it uses as the pathname of a directory to store NSS's security and configuration data. NSS typically stores 3 dbm files in that directory:
At initialization time, the application gives NSS a string that it uses as the pathname of a directory to store NSS's security and configuration data. NSS typically stores 3 DBM files in that directory:


* cert8.db  - stores publicly accessible objects (certs, CRLs, S/MIME records).
* cert8.db  - stores publicly accessible objects (certs, CRLs, S/MIME records).
Line 38: Line 40:


Also in that directory:<br />
Also in that directory:<br />
* If it has very large security objects (such as large CRLs), NSS will store them in files in a subdirectory named cert8.dir.
* If it has very large security objects (such as large CRLs), NSS will store them in files in a subdirectory named cert8.dir. (Yes, really!)
* If the cert8.db and/or key3.db files are missing, NSS will read data from older versions of those databases (e.g., cert7.db, cert5.db, if they exist) and may build new cert8.db and/or key3.db files with that data (upgrade).
* If the cert8.db and/or key3.db files are missing, NSS will read data from older versions of those databases (e.g., cert7.db, cert5.db, if they exist) and may build new cert8.db and/or key3.db files with that data (upgrade).


Line 57: Line 59:


In the presence of a multiaccess initialization string, during initialization
In the presence of a multiaccess initialization string, during initialization
NSS will try to find a shared database named librdb.so (rdb.dll on Windows) in its path and load it. This shared library is expected to implement a superset of the old dbm interface.  The main entry point is rdbopen, which will be passed the appName, database name, and open flags. The rdb shared library will pick a location or method to store the database (it may not necessarily be a file), then handle the raw db records from NSS.  The records passed to and from this library use exactly the same schema and record formats as the records in the DBM library.   
NSS will try to find a shared library named librdb.so (rdb.dll on Windows) in its path and load it. This shared library is expected to implement a superset of the old DBM interface.  The main entry point is rdbopen, which will be passed the appName, database name, and open flags. The rdb shared library will pick a location or method to store the database (it may not necessarily be a file), then handle the raw database records from NSS.  The records passed to and from this library use exactly the same schema and record formats as the records in the DBM library.   


=== The proposal ===
=== The proposal ===
Line 312: Line 314:


MACS are PBMAC1 data structures defined in pkcs5 2.0. Since pkcs5 v1
MACS are PBMAC1 data structures defined in pkcs5 2.0. Since pkcs5 v1
does not have integrity checks and pkcs12 has not definition for storing  
does not have integrity checks and pkcs12 has no definition for storing  
purely mac data,the shared DB integrity checks uses pkcs5 v2 to store that  
purely MAC data,the shared database integrity checks use pkcs5 v2 to store that  
pbe and mac data.
PBE and MAC data.


===== Database coherency issues =====
===== Database coherency issues =====


In our previous database, we had issues with database corruption resulting in hard to diagnose issues. In order to mitigate that, This section analyses how various forms of corruption can affect the new database design, and possible ways of repairing that corruption in the field.
In our previous database, we had issues with database corruption resulting in hard to diagnose issues. In order to mitigate that, this section analyzes how various forms of corruption can affect the new database design, and possible ways of repairing that corruption in the field.


'''Hidden Meta-data records'''
'''Hidden Meta-data records'''
Line 341: Line 343:
If the Private key is deleted, but the corresponding public key is not, then NSS may be confused and think that the private key exists for the certificate. Also, if the public key is deleted, but the private key is not, NSS may be confused and think the private key does not exist for the certificate. Both of these cases will act correctly if the token is logged in. This kind of corruption can be repaired by reimporting the pkcs12 file, or by a PKCS #11 level tool to delete or restore the public key.
If the Private key is deleted, but the corresponding public key is not, then NSS may be confused and think that the private key exists for the certificate. Also, if the public key is deleted, but the private key is not, NSS may be confused and think the private key does not exist for the certificate. Both of these cases will act correctly if the token is logged in. This kind of corruption can be repaired by reimporting the pkcs12 file, or by a PKCS #11 level tool to delete or restore the public key.


CKO_NSS_SMIME object holds the email address, subject of the S/MIME certificate, and the s/mime profile. In the legacy database, multiple email addresses could hold the same profile data and certificate subject, but only one s/mime profile data and subject would be allowed for each email address. In the new database, multiple independent email records can exist for the same email address. While S/MIME will function without CKO_NSS_SMIME objects, Certificates that verify multiple email address can not be found by the email addresses (other then the 'primary' address) without CKO_NSS_SMIME objects. If S/MIME records are corrupted, the Cerificates will not be findable for other email addresses. Unlike the legacy database, these records are destroyed by S/MIME records for other certificates with the same email address. Now if you have multiple certificates with the same email address, all those certificates can be found. This corruption can be repaired with a PKCS #11 level tool.
CKO_NSS_SMIME object holds the email address, subject of the S/MIME certificate, and the s/mime profile. In the legacy database, multiple email addresses could hold the same profile data and certificate subject, but only one s/mime profile data and subject would be allowed for each email address. In the new database, multiple independent email records can exist for the same email address. While S/MIME will function without CKO_NSS_SMIME objects, Certificates that verify multiple email address can not be found by the email addresses (other then the 'primary' address) without CKO_NSS_SMIME objects. If S/MIME records are corrupted, the Certificates will not be findable for other email addresses. Unlike the legacy database, these records are destroyed by S/MIME records for other certificates with the same email address. Now if you have multiple certificates with the same email address, all those certificates can be found. This corruption can be repaired with a PKCS #11 level tool.


==== Accessing the shared Database ====
==== Accessing the shareable Database ====


In order to maintain binary compatibility, the following keywords will be understood and used by softoken.
In order to maintain binary compatibility, the following keywords will be understood and used by softoken.


'''multiaccess:'''''appName''[''':'''''directory''] works as it does today, including using the cert8/key3 record version.<br/>
'''multiaccess:'''''appName''[''':'''''directory''] works as it does today, including using the cert8/key3 record version.<br/>
'''dbm:'''''directory'' opens an existing non-shared libdbm version 8 database.<br/>
'''dbm:'''''directory'' opens an existing non-shareable libdbm version 8 database.<br/>
'''sql:'''''directory1''[''':'''''directory2''] opens a shared database,
'''sql:'''''directory1''[''':'''''directory2''] opens a shareable database,
cert9.db (& key4.db) in ''directory1'' if cert9.db does exist. If the database does not exist, then ''directory2'' is searched for a libdbm cert8.db and key3.db. If ''directory2'' is not supplied, ''directory1'' is searched. <br/>
cert9.db (& key4.db) in ''directory1'' if cert9.db does exist. If the database does not exist, then ''directory2'' is searched for a libdbm cert8.db and key3.db. If ''directory2'' is not supplied, ''directory1'' is searched. <br/>
'''extern:'''''directory'' open a sql-like database by loading an external module, a. la. rdb and ''multiaccess:''. This option would not be implemented in the initial release, but the ''extern:'' keyword would be reserved for future use.
'''extern:'''''directory'' open a sql-like database by loading an external module, a. la. rdb and ''multiaccess:''. This option would not be implemented in the initial release, but the ''extern:'' keyword would be reserved for future use.


Plain directory spec. For binary compatibility, the plain directory spec as the same as '''dbm:'''''directory'' unless overridden with the NSS_DEFAULT_DATABASE environment variable. Applications will not need to change for this release of NSS. (particularly unfriendly applications that want to tweak with the actual database file). Users can force older applications to share the database with the environment variable. The environment variable only affects non-tagged directories.  
Plain directory spec. For binary compatibility, the plain directory spec as the same as '''dbm:'''''directory'' unless overridden with the NSS_DEFAULT_DB_TYPE environment variable. Applications will not need to change for this release of NSS. (particularly unfriendly applications that want to tweak with the actual database file). Users can force older applications to share the database with the environment variable. The environment variable only affects non-tagged directories.  


When accessing the '''dbm:''' and '''multiaccess:''' directories, external shared library will be loaded which knows how to handle these legacy databases. This allows us to move much of the current mapping code into this shared library.
When accessing the '''dbm:''' and '''multiaccess:''' directories, external shared library will be loaded which knows how to handle these legacy databases. This allows us to move much of the current mapping code into this shared library.
Line 367: Line 369:
One of the goals of making a shareable database version of NSS is to create a 'system crypto library' in which applications will automatically share database and configuration settings. In order for this to work, applications need to be able to open NSS databases from standard locations.
One of the goals of making a shareable database version of NSS is to create a 'system crypto library' in which applications will automatically share database and configuration settings. In order for this to work, applications need to be able to open NSS databases from standard locations.


This design assumes that new NSS init functions will be defined for applications wanting to do 'standard user initialization', rather than building special knowledge into softoken or the database model. Note: This is different from the 2001 design, or and earlier prototype shared database, where the database code knew the location of the shared database.
This design assumes that new NSS init functions will be defined for applications wanting to do 'standard user initialization', rather than building special knowledge into softoken or the database model. Note: This is different from the 2001 design, or and earlier prototype shareable database, where the database code knew the location of the shareable database.


==== Database Upgrade ====
==== Database Upgrade ====
Line 373: Line 375:
NSS has traditionally performed automatic updates when moving to new database formats. If NSS cannot find a database that matches it's current database type, it looks for older versions of it's database and automatically updates those to the new database version. In these cases database upgrade is automatic and mandatory for all applications.
NSS has traditionally performed automatic updates when moving to new database formats. If NSS cannot find a database that matches it's current database type, it looks for older versions of it's database and automatically updates those to the new database version. In these cases database upgrade is automatic and mandatory for all applications.


In the shared database design, upgrade is no longer mandatory. Applications  
In the shareable database design, upgrade is no longer mandatory. Applications  
may choose to continue to use the old DBM database, update to use the new shared database from old DBM databases, or update and merge old DBM database into a new location shared by multiple apps. There is still a desire for this update to be automatic, at least as far as the application user is concerned. The following describe how NSS deals with update in different applications, and what the different applications must do to get the correct update behavior.
may choose to continue to use the old DBM database, update to use the new shareable database from old DBM databases, or update and merge old DBM database into a new location shareable by multiple apps. There is still a desire for this update to be automatic, at least as far as the application user is concerned. The following describe how NSS deals with update in different applications, and what the different applications must do to get the correct update behavior.


To understand the issues of migration to the Shareable Database version of NSS from the traditional (legacy) versions, we group applications that use the new version of NSS into three 'modes' of operation, and into two types for a total of five valid combinations (Mode 1 B is not valid)..  
To understand the issues of migration to the Shareable Database version of NSS from the traditional (legacy) versions, we group applications that use the new version of NSS into three 'modes' of operation, and into two types for a total of five valid combinations (Mode 1 B is not valid)..  
Line 383: Line 385:
These applications will continue to use the legacy database support and the
These applications will continue to use the legacy database support and the
old DBM database format. The applications cannot take advantage of new features
old DBM database format. The applications cannot take advantage of new features
in the shared database. In this Mode, the nssdbm3 shared library must be  
in the shareable database. In this Mode, the nssdbm3 shared library must be  
present. No update from legacy DBM to sharable is needed in this mode.
present. No update from legacy DBM to sharable is needed in this mode.


Line 402: Line 404:
     minor changes to Init,  
     minor changes to Init,  
     Changes to code that updates trust values or imports certificates with  
     Changes to code that updates trust values or imports certificates with  
         trust.
         trust. ###kaie please clarify what existing code must get changed


====== Mode 3 ======
====== Mode 3 ======
Line 416: Line 418:
     minor changes to Init,  
     minor changes to Init,  
     Changes to code that updates trust values or imports certificates with  
     Changes to code that updates trust values or imports certificates with  
         trust.
         trust. ###kaie please clarify what existing code must get changed
     Changes to aid in update
     Changes to aid in update


Line 439: Line 441:


====== Mode 2A ======
====== Mode 2A ======
Mode 2A Applications can also continue to call traditional NSS_Initialize() functions. The should, however, prepend the string "sql:" to the directory path passed to NSS in the configdir parameter. If the sql databases do not exist, NSS will automatically update any old DBM databases in the config directory to shared databases. Like the upgrade from cert7 to cert8, if the update does not work, the app will open and use the old DBM database. Upgrade will not happen if
Mode 2A Applications can also continue to call traditional NSS_Initialize() functions. The should, however, prepend the string "sql:" to the directory path passed to NSS in the configdir parameter. If the sql databases do not exist, NSS will automatically update any old DBM databases in the config directory to shareable databases. Like the upgrade from cert7 to cert8, if the update does not work, the app will open and use the old DBM database. Upgrade will not happen if
# NSS is opened readOnly.
# NSS is opened readOnly.
# NSS_Initialization fails.
# NSS_Initialization fails.
#The database is password protected and the user never logs into the token during the lifetime of the application.
#The database is password protected and the user never logs into the token during the lifetime of the application.


Applications can avoid final case of failure by forcing the user to authenticate to softoken using PK11_Authenticate().
Applications can avoid that third failure case by forcing the user to authenticate to softoken using PK11_Authenticate().


Sample code fragment.
<pre>
  configdir = PR_smprintf("sql:%s",oldconfigdir);
  rv = NSS_InitReadWrite(configdir);
  if (rv != SECSuccess) {
      goto fail;
  }
 
  slot = PK11_GetInternalKeySlot();
 
  /* optional */
  rv = PK11_Authenticate(slot, PR_FALSE, pwArg);
  if (rv != SECSuccess) {
      goto fail;
  }
  /* database has been updated... */
</pre>


====== Mode 3A ======
====== Mode 3A ======
Mode 3A Applications are the most complicated. NSS provides some services to help applications get through and update and merge with the least interaction with the user of the application. Below is the steps a Mode 3A application should use whenever initializing NSS.
Mode 3A Applications are the most complicated. NSS provides some services to help applications get through an update and merge with the least interaction with the user of the application. The steps a Mode 3A application should use whenever initializing NSS are listed below.


Step 0: Preparation: collect the directory and prefix names of both the source and target DBs.  Prepare two strings for the operation:
Step 0: Preparation: collect the directory and prefix names of both the source and target databases.  Prepare two strings for the operation:
# <nowiki>A string to uniquely identify the source DB, for the purpose of
# <nowiki>A string to uniquely identify the source database, for the purpose of
avoiding a repeat of this merge (making the merge idempotent).  This
avoiding a repeat of this merge (making the merge idempotent).  This
string could be derived from the name of the application that used the
string could be derived from the name of the application that used the
source DB, from any application "instance" names (such as profile
source database, from any application "instance" names (such as profile
names), from the absolute path name of the source DB directory and the
names), from the absolute path name of the source database directory and the
DB prefixes, and from the last modification time of the source DBs.</nowiki><br><br></nowiki>
database prefixes, and from the last modification time of the source databases.</nowiki><br><br><nowiki>
The algorithm for deriving this string should always produce the same
The algorithm for deriving this string should always produce the same
result for the same set of source files, so that the code can detect a
result for the same set of source files, so that the code can detect a
second or subsequent attempt to merge the same source file into the
second or subsequent attempt to merge the same source file into the
destination file.</nowiki><br><br><nowiki>
destination file.</nowiki><br><br><nowiki>
Note: The purpose of this string is to prevent multiple updates from the same old Database. This merge sequence is meant to be light enough weight that applications can safely call it each time they initialize.</nowiki><br>
Note: The purpose of this string is to prevent multiple updates from the same old database. This merge sequence is meant to be sufficiently light weight that applications can safely call it each time they initialize.</nowiki><br>
# <nowiki>A string that will be the name of the removable PKCS#11 token that
# <nowiki>A string that will be the name of the removable PKCS#11 token that
will represent the source DB.  This string must follow the rules for a
will represent the source database.  This string must follow the rules for a
valid token name and must not contain any colon (:) characters.</nowiki>
valid token name and must not contain any colon (:) characters.</nowiki>


Step 1: Call NSS_InitWithMerge, passing as arguments the destination
Step 1: Call NSS_InitWithMerge, passing as arguments the destination
directory name, destination file name prefix, source directory name,
directory name, destination file name prefix, source directory name,
source file name prefix, unique source DB identifier string, and source
source file name prefix, unique source database identifier string, and source
token name string.
token name string.


Line 475: Line 492:
* Otherwise proceed to step 2.
* Otherwise proceed to step 2.


Step 2: Determine if a merge is even necessary. If a merge is necessary,
Step 2: Determine if a merge is necessary. If a merge is necessary,
NSS will set the slot to a 'removable slot'. You can use PK11_IsPerm to
NSS will set the slot to a 'removable slot'. You can use PK11_IsRemovable to
test for this.
test for this.


* If the DB slot token is not removable, then no update/merge is necessary, goto step 7.
* If the database slot token is not removable, then no update/merge is necessary, goto step 7.
* (optional) If PK11_NeedLogin() is not true then NSS has already completed the merge for you (no passwords were needed), skip to step 7.
* (optional) If PK11_NeedLogin() is not true then NSS has already completed the merge for you (no passwords were needed), skip to step 7.
* Otherwise it is necessary to authenticate to the source token, at step 3 below.
* Otherwise it is necessary to authenticate to the source token, at step 3 below.
Line 487: Line 504:


# <nowiki>(optional) Call PK11_GetTokenName to get the name of the token.  With
# <nowiki>(optional) Call PK11_GetTokenName to get the name of the token.  With
that name, you can be sure that you are authenticating to the source token. Skipping this step is not harmful, it is only necessary if the application absolutely needs to know which token the following PK11_Authenticate() will be called on (for instance pwArg contains the actual password for the token). For most NSS applications the underlying password prompt system will properly disambiguate the appropriate password to the user (or it's password cache).</nowiki>
that name, you can be sure that you are authenticating to the source token. Skipping this step is not harmful, it is only necessary if the application or user absolutely needs to know which token the following PK11_Authenticate() will be called on (for instance pwArg contains the actual password for the token). For some NSS applications the underlying password prompt system will properly disambiguate the appropriate password to the user (or it's password cache).</nowiki>
#* If the token name does not match the token name skip to step 5.
#* If the token name does not match the token name skip to step 5.
#* Otherwise proced to step 3.2.
#* Otherwise proced to the next substep.
# <nowiki>Call PK11_Authenticate() to authenticate to the source token.  This
# <nowiki>Call PK11_Authenticate() to authenticate to the source token.  This
step is likely to call the application-supplied PKCS11 password callback
step is likely to call the application-supplied PKCS11 password callback
Line 496: Line 513:
#*Otherwise, continue with step 4.
#*Otherwise, continue with step 4.


Step 4.  Determine if it is necessary to authenticate to the target DB.
Step 4.  Determine if it is necessary to authenticate to the target database.
This is done by calling PK11_IsLoggedIn for the DB slot.
This is done by calling PK11_IsLoggedIn for the database slot.
*If the function indicates that the DB token is NOT logged in, then it is necessary to authenticate to the target DB, with the next step below.
*If the function indicates that the database token is NOT logged in, then it is necessary to authenticate to the target database, with the step 5 below.
*If the function indicates that the token is logged in, skip down to step 7.
*(optional) Otherwise skip down to step 7.


Step 5. Call PK11_IsPresent().  You may think of this step as telling
Step 5. Call PK11_IsPresent().  You may think of this step as telling
you if the removable source token has been removed and the target token
you if the removable source token has been removed and the target token
has been inserted into the DB slot.  In reality, this call makes those
has been inserted into the database slot.  In reality, this call makes those
things happen.  After this call succeeds the token name should be that
things happen.  After this call succeeds the token name should be that
of the target token (see next step).
of the target token (see next step).
Line 513: Line 530:
callback function to retrieve the password.
callback function to retrieve the password.


* If this step fails: stop.  A Failure at this point is described below as "Exception B".
* If this step fails: stop.  A failure at this point is described below as "Exception B".
* Otherwise, continue with step 7.
* Otherwise, continue with step 7.


Line 520: Line 537:
====== Failures and recovery ======
====== Failures and recovery ======


Exception A. Application needs to decide what happens if the legacy password
Exception A. Failure to authenticate to the source database
 
Application needs to decide what happens if the legacy password
is not supplied. Application can choose to:  
is not supplied. Application can choose to:  
# continue to use the legacy DB and try to update later. (Probably a future restart of the application).
# continue to use the legacy database and try to update later. (Probably a future restart of the application).
# reset the legacy database, throwing away any private or secret keys in the old database.
# reset the legacy database password, discarding any private or secret keys in the old database.
# shutdown NSS and initialize it only with the new shared database.
# shutdown NSS and initialize it only with the new shareable database.
: The exact strategy for recovering is application dependent and depends on factors like
 
:# the sensitivity of the application to loosing key data.
: The exact strategy for recovering is application dependent and depends on factors such as:
:# possible input from the user.
:* the sensitivity of the application to losing key data.
:# the likelihood that the password will every be recovered.
:* possible input from the user.
:* the likelihood that the password will every be recovered.
 
Exception B. Failure to authenticate to the target database


Exception B. Applications needs to decide what happens if the new shared DB
Applications needs to decide what happens if the new shareable database
password is not supplied. Application can choose to:
password is not supplied. Application can choose to:
# continue to use the legacy DB and try to update later.
# continue to use the legacy database and try to update later.
# force NSS to update those objects it can from the legacy DB,throwing away private keys and saved passwords, and trust information from the legacy DB.
# force NSS to update those objects it can from the legacy database, throwing away private keys and saved passwords, and trust information from the legacy database.
# force NSS to reset the shared database password, throwing away private keys and saved passwords, and trust information from the shared DB.
# force NSS to reset the shareable database password, throwing away private keys and saved passwords, and trust information from the shareable database.


Notes:
Notes:
Line 541: Line 563:
step 7; that is, during the call to NSS_InitWithMerge or during either
step 7; that is, during the call to NSS_InitWithMerge or during either
of the calls to PK11_Authenticate.  This will depend on the ability of
of the calls to PK11_Authenticate.  This will depend on the ability of
the code to open the necessary DBs, the presence or absence of passwords
the code to open the necessary databases, the presence or absence of passwords
on the DBs, and if both have passwords, it will depend on whether they
on the databases, and if both have passwords, it will depend on whether they
have the same password or different passwords, and when the
have the same password or different passwords, and when the
authentication attempts, if any, succeed. The system tries to complete the
authentication attempts, if any, succeed. The system tries to complete the
merge as soon as it is able, to increase reliability of the merge update actually completing. Therefore API does not make it
merge as soon as it is able, to increase reliability of the merge update actually completing. The API does not make it
possible to predict, accurately, which step will actually perform the
possible to predict, accurately, which step will actually perform the
merge.  The application must try it and see if it cares. Since multiple calls to PK11_Authenticate() do not hurt, the application can simply follow each step in order, and failing only on bad returns from PK11_Authenticate. (PK11_Authenticate will automatically return without prompting, so applications that just need to update (without necessarily caring which step the update completes)).
merge.  The application can only follow the steps. Since multiple calls to PK11_Authenticate() do not hurt, the application can simply follow each step in order, and fail only on bad returns from PK11_Authenticate. (PK11_Authenticate will automatically return without prompting, so applications that just need to merge, without caring which step does the merge, may do so.)


2. If the attempt to open the
2. If the attempt to open the
source DB fails for any reason, the operation will behave as if the
source database fails for any reason, the operation will behave as if the
source DB was empty.  It will record the unique source DB identifier
source database was empty.  It will record the unique source database identifier
string in the target DB and act as if the merger is complete. This is similiar to what happens in all previous versions of NSS during database update. See "Database Merge" below for how to recover from this.
string in the target database and act as if the merger is complete. This is similar to what happens in all previous versions of NSS during database update. See "Database Merge" below for how to recover from this.
<pre>
<pre>
     /*
     /*
Line 569: Line 591:
       * Step 2: Determine if update/merge is needed.
       * Step 2: Determine if update/merge is needed.
       */  
       */  
     if (!PK11_IsPerm(slot) && !PK11_NeedLogin(slot)) {
     if (PK11_IsRemovable(slot) && PK11_NeedLogin(slot)) {
/* need to update/Merge the database */
        /* need to update/Merge the database */
/*
        /*
         * Step 3.1: Do we need to authenticate to the update Token?
         * Step 3.1: Do we need to authenticate to the update Token?
         */
         */
         if (PORT_Strcmp(PK11_GetTokenName(slot), updateTokenName) == 0)) {
         if (PORT_Strcmp(PK11_GetTokenName(slot), updateTokenName) == 0)) {
             rv = PK11_Authenticate(slot, PR_FALSE, pwArg);
             rv = PK11_Authenticate(slot, PR_FALSE, pwArg);
    if (rv != SECSuccess) {
            if (rv != SECSuccess) {
handle_failure_to_get_old_DB_Password();
                handle_failure_to_get_old_DB_Password();
                 goto fail; /* or done or fall through depending on recovery scheme */
                 goto fail; /* or done or fall through depending *
    }
                            * on recovery scheme */
}
            }
        }
          
          
         /* Step 4: */
         /* Step 4: */
Line 599: Line 622:
         if (rv != SECSuccess) {
         if (rv != SECSuccess) {
             /* handle exception B */
             /* handle exception B */
    handle_failure_to_get_new_DB_Password();
            handle_failure_to_get_new_DB_Password();
             goto fail; /* or done depending on recovery scheme */
             goto fail; /* or done depending on recovery scheme */
         }
         }
Line 625: Line 648:
       * Step 2: Determine if update/merge is needed.
       * Step 2: Determine if update/merge is needed.
       */  
       */  
     if (!PK11_IsPerm(slot)) {
     if (PK11_IsRemovable(slot)) {
/* need to update/Merge the database */
/* need to update/Merge the database */
/*
/*
Line 631: Line 654:
         */
         */
         rv = PK11_Authenticate(slot, PR_FALSE, pwArg);
         rv = PK11_Authenticate(slot, PR_FALSE, pwArg);
if (rv != SECSuccess) {
        if (rv != SECSuccess) {
    goto fail;
            goto fail;
}
        }
          
          
         /* just update the state machine */
         /* just update the state machine */
Line 659: Line 682:
In Mode 1, NSS never needs to do an update or a merge.  
In Mode 1, NSS never needs to do an update or a merge.  


<pre>
  State machine of NSS update actions for Mode 1:
  State machine of NSS update actions for Mode 1:


Line 668: Line 692:
                   V
                   V
                 done
                 done
</pre>


In Mode 2, the new database is uninitialized, so NSS only needs the
In Mode 2, the new database is uninitialized, so NSS only needs the
password for the legacy database so it can read the secret keys
password for the legacy database so it can read the secret keys
in that legacy database, and so the new shared database password
in that legacy database, and so the new shareable database password
matches the old one. NSS can find the legacy database because
matches the old one. NSS can find the legacy database because
it's in the same directory that the shared database lives in. NSS opens both
it's in the same directory that the shareable database lives in. NSS opens both
databases at initialization time and uses the legacy database until the user
databases at initialization time and uses the legacy database until the user
authenticates (providing the legacy database password). NSS then uses that
authenticates (providing the legacy database password). NSS then uses that
password to update the new shared database with the records from the old.  
password to update the new shareable database with the records from the old.  
The new database takes on the password from the legacy database, and the
The new database takes on the password from the legacy database, and the
legacy database is closed.  Future NSS initializations only open the new
legacy database is closed.  Future NSS initializations only open the new
shared database. If the user never supplies a password, NSS will continue to
shareable database. If the user never supplies a password, NSS will continue to
treat the new shared database as uninitialized and will attempt to update from
treat the new shareable database as uninitialized and will attempt to update from
the old database on future opens until the update succeeds.
the old database on future opens until the update succeeds.


<pre>
   State machine of NSS update actions for Mode 2:
   State machine of NSS update actions for Mode 2:
          
          
Line 688: Line 714:
                   |
                   |
                   V  
                   V  
             open shared DB
             open shareable DB
                   |
                   |
                   V
                   V
           < is open shared DB >  yes
           < is open shareable >  yes
           <  initialized?   >-------> done
           <  DB initialized? >-------> done
                   | no
                   | no
                   V
                   V
Line 716: Line 742:
   +--------------+
   +--------------+
                   V
                   V
             update (and use) shared DB
             update (and use) shareable DB
                   |
                   |
                   V
                   V
Line 723: Line 749:
                   V
                   V
                 done
                 done
------------------------------------------
</pre>
 
<pre>
             PK11_Authenticate
             PK11_Authenticate
                   |
                   |
Line 735: Line 763:
                   |
                   |
                   V
                   V
             update (and use) shared DB
             update (and use) shareable DB
                   |
                   |
                   V
                   V
Line 742: Line 770:
                   V
                   V
                 done
                 done
 
</pre>         
               
 


In Mode 3, the new database may or may not be initialized. For the first mode 3
In Mode 3, the new database may or may not be initialized. For the first mode 3
application, the new database will be uninitialized. NSS can proceed the with
application, the new database will be uninitialized. NSS can proceed the with
the same procedure as Mode 2. When the second and subsequent applications  
the same procedure as Mode 2. When the second and subsequent applications  
start, the new shared database will already be initialized with it's own
start, the new shareable database will already be initialized with it's own
password. We potentially need both passwords, the first to read the keys
password. We potentially need both passwords, the first to read the keys
out of the legacy database, and the second to write those keys, as well as
out of the legacy database, and the second to write those keys, as well as
Line 759: Line 785:
updated, so the applications needs to tell us some unique identifier for its
updated, so the applications needs to tell us some unique identifier for its
database. The application must be able to tell us where the old database lives,
database. The application must be able to tell us where the old database lives,
since it's a application private directory compared the the multiple  
since it's an application private directory compared the the multiple  
application shared directory that the shared DB lives in.
application shared directory that the shared database lives in.


<pre>
   Flow chart of NSS update actions for Mode 3:
   Flow chart of NSS update actions for Mode 3:


Line 811: Line 838:
                   V
                   V
                 done
                 done
</pre>


---------------------------------------------------------------
<pre>
 
               PK11_Authenticate
               PK11_Authenticate
                       |
                       |
Line 848: Line 875:
                   V
                   V
                 done
                 done
</pre>


===== Merge Conflicts (Mode 3A only) =====
===== Merge Conflicts (Mode 3A only) =====


When merging databases in, it's possible (even likely), that the shared  
When merging databases in, it's possible (even likely), that the shared  
database and legacy DB's have the same objects. In the case of certs and keys,
database and legacy databases have the same objects. In the case of certs and keys,
the merge is a simple matter of identifying duplicates and not updating them.
the merge is a simple matter of identifying duplicates and not updating them.
In the case of trust attributes, however, there are a number of choices:
# don't update duplicate trust (shared database copy wins).
# overwrite trust from the legacy DB (legacy db copy wins).
# calculate the least common denominator trust between them (take the least trusted values). (turning off trust wins).
# calculate the most common demonimnator trust between the two (turning on trust wins).
From the user perspective, each of these choices means:
# after the update the application that just updated may trust certs that it had previously marked untrusted, or certs that it has marked trusted are no longer trusted.
# after the update other applications that share the database may trust certs they had previously marked untrusted, or certs that they had marked as trusted are no longer trusted.
# after the update all apps may find the certs that they marked trust are no longer trusted.
# after the update all apps may find that they trust certs that have  previously been marked untrusted.
Option 3 is the most secure, Option 4 will break have less breakage. Trust
merge conflicts that are real conflicts (application 'A' turned off trust and
application 'B' turned on trust) are expected to be rare. The common case would
be application 'A' turned on SSL trust and application 'B' turned on email
trust. In this case Option 4 is clearly the correct choice.


From a programming point of view, NSS should pick a default and implement it.
Trust records are made up of several entries, such as one for SSL Server Auth, SSL Client Auth, S/MIME, etc. Each entry could have several values, including CKT_NSS_MUST_VERIFY, CKT_NSS_TRUSTED_DELEGATOR, CKT_NSS_TRUSTED, CKT_NSS_VALID, etc).
Ideally no user interaction will occur.


Finally password entries are merge issues. If the two databases have different
Merge updates the trust records by the entries in that trust record separately:
passwords, the merged database will have to have a
# if the trust record entries are identical, no update is done.
# if either trust record entry has an explicit unknown (CKT_NSS_TRUST_UNKNOWN) or invalid trust record entry (entry does not exist), then the one that is valid and known is used.
# if one of the trust record entries has hard trust attributes (Trust flags with NSS_TRUSTED or NSS_UNTRUSTED in the name) and the other has soft attributes (NSS_VALID or NSS_MUST_VERIFY) the entry with hard attributes is used. Hard trust attributes are attributes that will terminate a certificate validation.
# if non of these cases apply, then the value in the target database is preserved.


===== Mozilla Applications =====
===== Mozilla Applications =====
Line 901: Line 913:
# The Mozilla app is starting as a fresh instance.
# The Mozilla app is starting as a fresh instance.
# The Mozilla app has already been updated.
# The Mozilla app has already been updated.
# The shared database does not have a master password set and The legacy database for Mozilla app does a master password set.
# The shared database does not have a master password set and the legacy database for Mozilla app does a master password set.


These are the most common cases.
These are the most common cases.
Line 910: Line 922:


UI question. At this point should we notify the user that we are updating
UI question. At this point should we notify the user that we are updating
the database to a shared database? In order to complete this we will need  
the database to a shareable database? In order to complete this we will need  
to do user interaction below.
to do user interaction below.


If the legacy db for the mozilla app has a master password set, we prompt for  
If the legacy database for the mozilla app has a master password set, we prompt for  
it. This prompt must be clear we are asking for the master password for  
it. This prompt must be clear we are asking for the master password for  
the running Mozilla app (Thunderbird, Firefox, Seamonkey, etc).
the running Mozilla app (Thunderbird, Firefox, Seamonkey, etc).




  Exception case A
:Exception case A
 
  If we fail to get this password, we need to handle the exception A case.
  If the user has a master password set, but does not know what the master
  password is, then the following data is lost for sure:


    The user's private keys.
:If we fail to get this password, we need to handle the exception A case. If the user has a master password set, but does not know what the master password is, then the following data is lost for sure:
    The user's secret keys.
    Any data encrypted to the private keys.
    Any data encrpted with the secret keys.


  I believe we can identify if the private keys are associated with a
:* The user's private keys.
  certificate. If so, then we can tell the user what certificate would no
:* The user's secret keys.
  longer work. Data encrypted with the private keys in Mozilla products are
:* Any data encrypted to the private keys.
  currently only email messages. Secret keys encrypt saved passwords. The
:* Any data encrpted with the secret keys.
  Mozilla app knows which saved passwords are encrypted with that key.


If we hit Exception case A we can do one of the following:
:I believe we can identify if the private keys are associated with a certificate. If so, then we can tell the user what certificate would no longer work. Data encrypted with the private keys in Mozilla products are currently only email messages. Secret keys encrypt saved passwords. The Mozilla app knows which saved passwords are encrypted with that key.


1) attempt to just update the certs, trust, crl and s/mime records, skipping
:If we hit Exception case A we can do one of the following:
the all the keys. We would loose all the data described above.
2) decide not to update. In this case we would loose all the data in the
paragraph above as well as all the certs, trust crl and s/mime records.
3) run with the legacy database and allow the user to update later.
4) run with the new shared database and allow the user to update later.


I would suggest we only offer the user the choice of 1 or 4. Note: if the user
:# attempt to just update the certs, trust, crl and s/mime records, skipping the all the keys. We would loose all the data described above.
selects 1, the update could fail again in exception case B. From a UI
:# decide not to update. In this case we would loose all the data in the paragraph above as well as all the certs, trust crl and s/mime records.
perspective, we may want to handle exception case B as we handle case A so
:# run with the legacy database and allow the user to update later.
the user is only asked once about forcing an update while losing data.
:# run with the new shared database and allow the user to update later.


:I would suggest we only offer the user the choice of 1 or 4. Note: if the user selects 1, the update  could fail again in exception case B. From a UI perspective, we may want to handle exception case B as we handle case A so the user is only asked once about forcing an update while losing data.


Once we have a legacy db password, or if we determine we don't need the legacy
Once we have a legacy database password, or if we determine we don't need the legacy
db password (either because there isn't one, or because we are willing to loose
database password (either because there isn't one, or because we are willing to loose
the data that was protected by it). We need to acquire the shared db's  
the data that was protected by it). We need to acquire the shareable database's  
password so we can encrypt and mac the data properly. If the shared db doesn't
password so we can encrypt and MAC the data properly. If the shareable database doesn't
have a password we can proceed with the update without further prompting the
have a password we can proceed with the update without further prompting the
user. If the shared db has the same password as the legacy db, then we can  
user. If the shareable database has the same password as the legacy database, then we can  
detect that and again proceed with the update without further prompting.
detect that and again proceed with the update without further prompting.


If both of these fail, we prompt for the password for the shared database. This
If both of these fail, we prompt for the password for the shareable database. This
prompt is trickier, because we need to ask the user for the password that
prompt is trickier, because we need to ask the user for the password that
he percieves to be the Master password for a different mozilla app. Note: at  
he percieves to be the Master password for a different mozilla app. Note: at  
Line 969: Line 969:




Exception case B  
:Exception case B  


If we fail to get this password, we need to handle the exception B case.
:If we fail to get this password, we need to handle the exception B case. If the user has a master password set on his shareable database, but does not know what that master password is, we now have the following choices:
If the user has a master password set on his shared database, but does not  
know what that master password is, we now have the following choices:


1) eshew any private keys, secret keys and trust updates from the  
:# eshew any private keys, secret keys and trust updates from the legacy database.
legacy database.
:# reset the password on the shareable database (losing all private and secret keys, possibly losing some trust).
2) reset the password on the shared database (loosing all private and secret
:# run with the legacy database and allow the user to update later.
keys, possibly loosing some trust).
:# run with the new shareable database and allow the user to update later.
3) run with the legacy database and allow the user to update later.
4) run with the new shared database and allow the user to update later.


It seems pretty unlikely that the user truly does not know the shared database
:It seems pretty unlikely that the user truly does not know the shareable database password, since he had to create or set it recently. However as the deployment time increases, this becomes more likely.
password, since he had to create or set it recently. However as the deployment
time increases, this becomes more likely.


Again, I think giving the user a choice between options 1) and 4) are the
:Again, I think giving the user a choice between options 1) and 4) are the best alternatives. If the user had already tripped over Exception case A, we can presume the user intends to make a similiar choice here. Case 2 can be handled later under the same way the user handles a forgotten master password today (only now resetting the master password affects all mozilla apps).
best alternatives. If the user had already tripped over Exception case A, we
can presume the user intends to make a similiar choice here. Case 2 can be
handled later under the same way the user handles a forgotten master password
today (only now resetting the master password affects all mozilla apps).




Line 996: Line 986:


Mozilla apps can create more than one profile. Developers use this capability  
Mozilla apps can create more than one profile. Developers use this capability  
to test bugs that new users are likely to run into without loosing their own  
to test bugs that new users are likely to run into without losing their own  
production environment.
production environment.


Shared databases, in general, means that some of the current semantics of user  
Shared databases, in general, mean that some of the current semantics of user  
profiles will break. Creating a new profile will not create a new  
profiles will break. Creating a new profile will not create a new  
key/cert/master password profile. For developers (the primary users of profiles)
key/cert/master password profile. For developers (the primary users of profiles)
Line 1,005: Line 995:
couple of options.
couple of options.


# Allow profiles to be marked with 'private key/cert DB's. This will change The Mozilla app from a Mode 3A app to a Mode 2A app. This will return developers to their previous semantic if they want, while allowing them to also test the interaction of different profiles and the same database. It would require UI changes to the profile manager, and it will require action on the part of the developer to get back to the old semantic.
# Allow profiles to be marked with 'private key/cert databases. This will change The Mozilla app from a Mode 3A app to a Mode 2A app. This will return developers to their previous semantic if they want, while allowing them to also test the interaction of different profiles and the same database. It would require UI changes to the profile manager, and it will require action on the part of the developer to get back to the old semantic.
# Treat only the default profile as Mode 3A and all other profiles as Mode 2A. This will allow profile separation to operate as is today with no changes. It does mean, however, that only default profiles will share keys with application.
# Treat only the default profile as Mode 3A and all other profiles as Mode 2A. This will allow profile separation to operate as is today with no changes. It does mean, however, that only default profiles will share keys with application.
# Provide the checkbox in option 1, but make it default as in option 2.
# Provide the checkbox in option 1, but make it default as in option 2.
Line 1,013: Line 1,003:
===== Database Merge =====
===== Database Merge =====


While not necessarily a feature of shared database, it is an important tool for successful shared database deployments.
While not necessarily a feature of shareable databases, it is an important tool for successful shareable database deployments.


Database merge is different from database update, and in particular, database update with merge, in the following ways.
Database merge is different from database update, and in particular, database update with merge, in the following ways.
Line 1,021: Line 1,011:
# Because merge does not require the complicated state machine to manage password acquisition, it can (and is) implemented outside the softoken itself.
# Because merge does not require the complicated state machine to manage password acquisition, it can (and is) implemented outside the softoken itself.


Characteristic 3 allows database merge to work on arbitrary database types. You can merge a shared db into a shared db as well as an old datase into a shared db (in fact, to a point, on arbitrary tokens - you can merge a hardware token into a shared db as long as the keys are extractable).
Characteristic 3 allows database merge to work on arbitrary database types. You can merge a shareable database into a shareable database as well as an old database into a shareable database (in fact, to a point, on arbitrary tokens - you can merge a hardware token into a shareable database as long as the keys are extractable).


To merge 2 databases, the application simply opens the both databases and calls the new PK11_MergeTokens() call. PK11_MergeTokens() has the following signature:
To merge 2 databases, the application simply opens both databases (using SECMOD_OpenUserDB) and then calls the new PK11_MergeTokens() function. PK11_MergeTokens() has the following signature:


  #include <pk11pub.h>
<pre>
  #include "pk11pub.h"


  SECStatus PK11_MergeTokens(
  SECStatus PK11_MergeTokens(
Line 1,032: Line 1,023:
     PK11MergeLog *log,
     PK11MergeLog *log,
     void *pwdata);
     void *pwdata);
</pre>


Parameters:
Parameters:
Line 1,043: Line 1,035:
''pwdata'' password arg
''pwdata'' password arg


The ''targetSlot'' and ''sourceSlot'' parameters could be slots that are simply looked up, or additional databases opened with SECMOD_OpenUserDB(). In order for the merge to be successful, ''targetSlot'' must support all the intersection of the following object list and the token objects in the ''sourceSlot'' (CKO_CERTIFICATE, CKO_PUBLIC_KEY, CKO_PRIVATE_KEY, CKO_SECRET_KEY, CKO_NSS_TRUST, CKO_NSS_CRL, CKO_NSS_SMIME). The source Slot must also have extractable keys or the merge will fail (sensitive keys are OK, as long as the source slot supports PBE's if it contains private keys). All softoken slots (including those opened with SECMOD_OpenUserDB()) support these charateristics.
The ''targetSlot'' and ''sourceSlot'' parameters could be slots that are simply looked up, or additional databases opened with SECMOD_OpenUserDB(). In order for the merge to be successful, ''targetSlot'' must support all the object types in the following list for which token objects exist in the ''sourceSlot'':
* CKO_CERTIFICATE,  
* CKO_PUBLIC_KEY,  
* CKO_PRIVATE_KEY,  
* CKO_SECRET_KEY,  
* CKO_NSS_TRUST,  
* CKO_NSS_CRL,  
* CKO_NSS_SMIME.  
The source Slot must also have extractable keys or the merge will fail (sensitive keys are OK, as long as the source slot supports PBE's if it contains private keys). All softoken slots (including those opened with SECMOD_OpenUserDB()) support these charateristics.


Multiple calls to merge will only attempt to merge those objects which were created since the last merge, or failed to merge in the last call to merge.
Multiple calls to merge will only attempt to merge those objects which were created since the last merge, or failed to merge in the last call to merge.
Line 1,102: Line 1,102:


* directory full path to where the database lives.
* directory full path to where the database lives.
* certPrefix a prefix string to add in front of the key and cert db (if keyPrefix is null), null means add no prefix.
* certPrefix a prefix string to add in front of the key and cert database names (if keyPrefix is null), null means add no prefix.
* keyPrefix a prefix string to add in front of the key db. Null means use the same prefix as the cert db.
* keyPrefix a prefix string to add in front of the key database name. Null means use the same prefix as the cert database name uses.
* cert_version current version is the current database version
* cert_version current version is the current database version
* key_version is the current key database version
* key_version is the current key database version
Line 1,116: Line 1,116:
The returned SDB structure has the following format:
The returned SDB structure has the following format:


<pre>
  typedef struct SDBStr SDB;
  typedef struct SDBStr SDB;


Line 1,144: Line 1,145:
     CK_RV (*sdb_Close)(SDB *sdb);
     CK_RV (*sdb_Close)(SDB *sdb);
  };
  };
</pre>


where:
where:


* private is a pointer to opaque private data specific to the Shared DB implementation.
* private is a pointer to opaque private data specific to the Shareable database implementation.
* sdb_type is the type of database (key [aka private] or cert [aka public]).
* sdb_type is the type of database (key [aka private] or cert [aka public]).
* sdb_flags specifies how the database was opened (ReadOnly, Create, etc).
* sdb_flags specifies how the database was opened (ReadOnly, Create, etc).
Line 1,260: Line 1,262:
==== legacy DB support ====
==== legacy DB support ====


The old dbm code can be supported with the above SDB structure with the following exceptions:
The old DBM code can be supported with the above SDB structure with the following exceptions:
 
# The old db code cannot be extensible (can't dynamically handle new types).
# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format.
 
This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:
* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.
* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.
* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).
* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys.
 
NSS will automaticall load the legacy database support under the following conditions:
 
# The application requests that the old databases be loaded (either implicitly or explicitly).
# The application request that new databases are loaded, but the new databases do not exist and the old databases do.
 
[[Category:NSS]]
 
 
 
==== legacy DB support ====
 
The old dbm code can be supported with the above SDB structure with the following exceptions:
 
# The old db code cannot be extensible (can't dynamically handle new types).
# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format.
 
This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:
* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.
* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.
* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).
* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys.
 
NSS will automaticall load the legacy database support under the following conditions:
 
# The application requests that the old databases be loaded (either implicitly or explicitly).
# The application request that new databases are loaded, but the new databases do not exist and the old databases do.
 
[[Category:NSS]]
 
==== Layering ====
 
In order to keep clean separation between the data and database operations, we will continue to maintain an layer between the actual data handling and interpretation and the database itself. The database code will not need to understand:
 
#What objects are actually stored in it.
#The types of the attributes.
#The meaning of the stored data.
 
Softoken (not the database adapter layer) will manage canonicalizing any CK_ULONGs, encrypting or decrypting private data blobs, checking integrity and deciding what attributes an object should have and setting the appropriate defaults if necessary.
 
Since softoken deals with PKCS #11 templates internally, its interface to the database will be in terms of those templates.
 
The database layer must be multi-thread safe. If the underlying database is not thread safe, sdb_ layer must implement the appropriate locking.
 
===== s_open =====
 
The database API consists of an initialization call, which returns an SDB data structure (defined below).
 
CK_RV
s_open(const char *directory, const char *certPrefix, const char *keyPrefix,
        int cert_version, int key_version, int flags,
        SDB **certdb, SDB **keydb, int *newInit)
 
The sdb_init function takes:
 
* directory full path to where the database lives.
* certPrefix a prefix string to add in front of the key and cert db (if keyPrefix is null), null means add no prefix.
* keyPrefix a prefix string to add in front of the key db. Null means use the same prefix as the cert db.
* cert_version current version is the current database version
* key_version is the current key database version
* flags are:
** FORCE
** READONLY
** READ/WRITE/CREATE
* certdb is the returned cert SDB structure
* keydb is the returned key SDB structure
* newInit returns 1 of s_open created new instances of cert and key (used for update).
 
The returned SDB structure has the following format:
 
typedef struct SDBStr SDB;
 
struct SDBStr {
    void *private;
    void *sdb_app_private;
    int  sdb_type;
    int  sdb_flags;
    int  sdb_version;
    CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template,
                                int count, SDBFind **find);
    CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids,
                                int arraySize, int *count);
    CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find);
    CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object,
                                CK_ATTRIBUTE *template, int count);
    CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object,
                                const CK_ATTRIBUTE *template, int count);
    CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object,
                                const CK_ATTRIBUTE *template, int count);
    CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object);
    CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry);
    CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry);
    CK_RV (*sdb_Begin)(SDB *sdb);
    CK_RV (*sdb_Commit)(SDB *sdb);
    CK_RV (*sdb_Abort)(SDB *sdb);
    CK_RV (*sdb_Reset)(SDB *sdb);
    CK_RV (*sdb_Close)(SDB *sdb);
};
 
where:
 
* private is a pointer to opaque private data specific to the Shared DB implementation.
* sdb_type is the type of database (key [aka private] or cert [aka public]).
* sdb_flags specifies how the database was opened (ReadOnly, Create, etc).
* sdb_version specifies the version of the underlying sdb structure. This allows us to handle future expansion of the sdb data structure safely.
* The rest are function pointers to database primitives described next.
 
===== sdb_FindObjectsInit =====
 
CK_RV (*sdb_FindObjectsInit)(SDB *sdb, const CK_ATTRIBUTE *template,
                                int count, SDBFind **find);
 
This function is the equivalent of PKCS #11 C_FindObjectsInit(). It returns a SDBFind context with is opaque to the caller. The caller must call sdb_FindObjectsFinal with this context if sdb_FindobjectsInit succeeds.
 
===== sdb_FindObjects =====
 
CK_RV (*sdb_FindObjects)(SDB *sdb, SDBFind *find, CK_OBJECT_HANDLE *ids,
                                int arraySize, int *count);
 
This function is the equivalent of PKCS #11 C_FindObjects(). It takes a SDBFind
context returned by sdb_FindObjectsInit. This function has the same semantics as C_FindObjects with respect to handling how many objects are returned in a single call.
 
===== sdb_FindObjectsFinal =====
 
CK_RV (*sdb_FindObjectsFinal)(SDB *sdb, SDBFind *find);
 
This function is the equivalent of PKCS #11 C_FindObjectsFinal(). It frees any resources associated with SDBFIND.
 
===== sdb_GetAttributeValue =====
 
CK_RV (*sdb_GetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object,
                                CK_ATTRIBUTE *template, int count);
 
This function is the equivalent of PKCS #11 C_GetAttributeValue(). It has the
same memory allocation and error code semantics of the PKCS #11 call.
The attributes passed to sdb_GetAttributeValues are already transformed from
their native representations in the following ways:
 
# CKU_LONG values are stored as 32-bit values in network byte order.
# Private attributes will be encrypted.
 
===== sdb_SetAttributeValue =====
 
CK_RV (*sdb_SetAttributeValue)(SDB *sdb, CK_OBJECT_HANDLE object,
                                const CK_ATTRIBUTE *template, int count);
 
This function is the equivalent of PKCS #11 C_SetAttributeValue().
The attributes returned to sdb_SetAttributeValues are transformed from
their native representations in the following ways:
 
# CKU_LONG values returned 32-bit values in network byte order.
# Private attributes returned encrypted.
 
===== sdb_CreateObject =====
 
CK_RV (*sdb_CreateObject)(SDB *sdb, CK_OBJECT_HANDLE *object,
                                const CK_ATTRIBUTE *template, int count);
 
This function is the equivalent of PKCS #11 C_CreateObject(). The value of 'object' is chosen by the implementer of sdb_CreateObject. This value must be unique for this sdb instance. It should be no more than 30 bits long.
 
===== sdb_DestroyObject =====
 
CK_RV (*sdb_DestroyObject)(SDB *sdb, CK_OBJECT_HANDLE object);
 
This function is the equivalent of PKCS #11 C_Destroy object(). It removed the object from the database.
 
===== sdb_GetPWEntry =====
 
CK_RV (*sdb_GetPWEntry)(SDB *sdb, SDBPasswordEntry *entry);
 
Get the password entry. This only applies to the private database.
 
===== sdb_PutPWEntry =====
 
CK_RV (*sdb_PutPWEntry)(SDB *sdb, SDBPasswordEntry *entry);
 
Write the password entry. This only applies to the private database.
Writing a password entry will overwrite the old entry.
 
===== sdb_Begin =====
 
CK_RV (*sdb_Begin)(SDB *sdb);
 
Begin a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holding
a transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction.
 
===== sdb_Commit =====
 
CK_RV (*sdb_Commit)(SDB *sdb);
 
Commit a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holding
a transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction.
 
===== sdb_Abort =====
 
CK_RV (*sdb_Abort)(SDB *sdb);
 
Abort a transaction. Any write to the database (sdb_CreateObject, sdb_DestroyObject, sdb_SetAttributeValue) must be accomplished while holding
a transaction. Transactions are completed by calling sdb_Commit to commit the change, or sdb_Abort to discard the change. More than one write operation may be made while holding a transaction. Aborting the transaction will discard all writes made while in the transaction.
 
===== sdb_Close =====
 
CK_RV (*sdb_Close)(SDB *sdb);
 
Close the SDB and free up any resources associated with it.
 
===== sdb_Reset =====
 
CK_RV (*sdb_Reset)(SDB *sdb);
 
Reset zeros out the key database and resets the password.
 
==== legacy DB support ====
 
The old dbm code can be supported with the above SDB structure with the following exceptions:
 
# The old db code cannot be extensible (can't dynamically handle new types).
# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format.
 
This code would live in its own shared library, called lgdbm (with the appropriate platform semantics, lgdbm.dll on windows, liblgdbm.so on unix, etc). Most of the low level cert, CRL, key handling, and translation to PKCS #11 objects and attributes that was part of softoken will moved to this legacy shared library. When access to old databased are needed, the lgdbm shared library will be loaded, and the following symbols will be dynamically found:
* legacy_Open - This has the same signature as s_open and returns SDB handles for the legacy database.
* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.
* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).
* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys.
 
NSS will automaticall load the legacy database support under the following conditions:
 
# The application requests that the old databases be loaded (either implicitly or explicitly).
# The application request that new databases are loaded, but the new databases do not exist and the old databases do.
 
[[Category:NSS]]
 
 
 
==== legacy DB support ====
 
The old dbm code can be supported with the above SDB structure with the following exceptions:


# The old db code cannot be extensible (can't dynamically handle new types).
# The old database code cannot be extensible (can't dynamically handle new types).
# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format.
# A private interface may be needed to unwrap the private keys, or provide a handle to the password so the keys can be presented in the attribute format.


Line 1,515: Line 1,271:
* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.
* legacy_ReadSecmodDB, legacy_ReleaseSecmodDBData, legacy_DeleteSecmodDB, legacy_AddSecmodDB - These functions provide access to the old secmod databases.
* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).
* legacy_Shutdown - This is called when NSS is through with all database support (that is when softoken shuts down).
* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy db can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys.
* legacy_SetCryptFunctions - This is used to set some callbacks that the legacy database can call to decrypt and encrypt password protected records (pkcs8 formatted keys, etc.). This allows the legacy database to translate it's database records to the new format without getting direct access to the keys.


NSS will automaticall load the legacy database support under the following conditions:
NSS will automatically load the legacy database support under the following conditions:


# The application requests that the old databases be loaded (either implicitly or explicitly).
# The application requests that the old databases be loaded (either implicitly or explicitly).
Jcjones
122

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/82367...1191032"