View source for Services/Sync/SyncKey/J-PAKE/NewOrder

'''This wiki page has been integrated in the Mozilla Services doc portal''': http://docs.services.mozilla.com/keyexchange/index.html.


= Overview  =

Reworking Setup Flow, using JPAKE. 

Tracking bug for setup changes is {{bug|675826}}. 

Feature page is https://wiki.mozilla.org/Firefox/Features/Sync_Setup_Improvements

= Engineers Involved  =

*Ally (FxSync) 
*Philipp (FxSync) 
*server engineers?

= User Requirements  =

*Setting up a new mobile device should only involve entering a short code on the desktop device. 
*If the user has no account, enter the code then create account.

= Desired User Flow  =

#User chooses "quick setup" on new device 
#Device displays a setup key that contains both the initial secret and a channel ID (3 codes)
#User goes to the desktop computer.
#User clicks 'Add Device' link on bottom of home tab on Desktop.
*If the user has a sync account, just display 'Pair A Device', possibly some welcome/identifying info.
*If the user does not have a sync account, display 'Setup Sync' & 'Pair a Device'
#User enters modal wizard on Desktop.
#User enters 3 codes.
#Desktop & Device exchange messages to build secure tunnel.
#Device displays a screen informing the user that it is waiting on the Desktop.
#If User already has a Sync Account:
*The already-authenticated device (Desktop) passes all credentials (username/password/sync key) to Device.
*Desktop tells Device to start downloading.
#If User does not have a Sync Account:
*Prompt for account details.
*Create account.
*Start first upload to server.  Display uploading bar on Desktop.
*Desktop signals to Device that upload has finished.
#Device displays downloading bar & downloads sync data to phone.
#Device informs user that sync has finished and they can now start using it.

= Implementation =

== Terminology  ==

*Desktop: Client that has Fx Sync already set up 
*Device:&nbsp;Client that needs to be set up, usually phone (of course this could be another desktop computer, too) 
*PIN:&nbsp;code that is displayed on Device and entered on Desktop made up of the weak secret & the channel id.
*Secret:&nbsp;weak secret that is used to start the J-PAKE algorithm
*Key:&nbsp;strong secret that both clients derive through J-PAKE

== Overview  ==

*Device and Desktop complete the two roundtrips of J-PAKE messages to agree upon a strong secret K 
*A 256 bit key is derived from K using HMAC-SHA256 using a fixed extraction key. 
*The encryption and HMAC keys are derived from that 256 bit key using HMAC-SHA256. 
*In third round trip: 
**Device encrypts the known message "0123456789ABCDEF" with the AES key and uploads it. 
**Between 3 and 4 is when the user may have to create an account. There may be significant delay here.
*In the 4th round trip:
**Desktop verifies that against the known message encrypted with its own key, encrypts the credentials with the encryption key and uploads the encrypted credentials in turn, adding a HMAC-SHA256 hash of the ciphertext (using the HMAC key). 
**Device verifies whether Desktop had the right key by checking the ciphertext against the HMAC-SHA256 hash. 
**If that verification is successful, Device decrypts ciphertext and applies credentials, and triggers the first download to Device.


<br> 
<pre>Device                        Server                        Desktop
===================================================================
                                 |
retrieve channel &lt;---------------|
generate random secret           |
show PIN = secret + channel      |                 ask user for PIN
upload Device's message 1 ------&gt;|
                                 |----&gt; retrieve Device's message 1
                                 |&lt;----- upload Desktop's message 1
retrieve Desktop's message 1 &lt;---|
upload Device's message 2 ------&gt;|
                                 |----&gt; retrieve Device's message 2
                                 |                      compute key
                                 |&lt;----- upload Desktop's message 2
retrieve Desktop's message 2 &lt;---|
compute key                      |
encrypt known value ------------&gt;|
                                 | verify against local known value
                                 |
                                 |      Optionally create account
                                 |-------&gt; retrieve encrypted value
                                 |              encrypt credentials
                                 |&lt;------------- upload credentials
retrieve credentials &lt;-----------|
verify HMAC                      |
decrypt credentials              
Device begins download           |
</pre>

== Key derivation ==

The AES encryption key T(1) and the HMAC key T(2) will be derived from J-PAKE's strong secret K as follows:

 extraction_key = "\x00" * 32
 key_string = HMAC-SHA256(K, extraction_key)

 T(1) = HMAC-SHA256(key_string, "" + "Sync-AES_256_CBC-HMAC256" + 0x01)
 T(2) = HMAC-SHA256(key_string, T(1) + Sync-AES_256_CBC-HMAC256" + 0x02)

(See http://tools.ietf.org/html/rfc5869)

To verify the key on both ends, the value

  0123456789ABCDEF

is encrypted with the AES key.

== Data format ==

All request and response bodies are JSON objects as produced by [http://github.com/warner/python-jpake python-jpake] (messages 1 and 2) and specified below. An <code>application/json</code> HTTP Content-Type header is optional. Within the JSON objects,

<ul>
<li>the big numbers are encoded as hex strings (messages 1 and 2),</li>
<li>the ciphertext, IV and hmac are encoded in Base64 (messages 3)</li>
<li>the credentials are a JSON object with the following properties (UTF-8 encoded strings):
  <ul><li><code>account</code>,</li>
      <li><code>password</code>,</li>
      <li><code>synckey</code>,</li>
      <li><code>serverURL</code></li></ul></li>
</ul>

== Server API ==

The only valid HTTP response codes are 200 and 304 since those are part of the protocol and expected to happen. Anything else, like 400, 403, 404 or 503 must result in a complete termination of the password exchange. The client can retry the exchange then at a later time, starting all over with clean state.

Every call must be done with a ''X-KeyExchange-Id'' header, containing a half-session identifier for the channel. This client ID must be a string of 256 chars. The server will keep track of the two first ids used for a given channel, from its creation to its deletion and will close the channel and issue a 400 if any request is made with an unknown id or with no id at all.

Last, if a given IP attempts to flood the server with a lot of calls in a short time, it will be blacklisted for 10 minutes return 403s in the interim for any requests made from the same IP. When receiving this error code, legitimate clients can fall back to a manual transaction. A client that generates a lot of bad requests will also be blacklisted, but for an hour.


The server API are:

'''GET''' '''https://''server''/new_channel'''

  Returns in the response body a JSON-encoded random channel id of N chars 
  from [a-z0-9].
    
  When the API is called, The id returned is guaranteed to be unique. 
  The channel created will have a limited TTL (currently configured to 
  5 minutes, should be changed to 10).
      
  Return codes:
     - 200: channel created successfully  
     - 503: the server was unable to create a new channel.
     - 400: Bad or no client ID. The channel is deleted.
     - 403: the IP is blacklisted. 


'''GET''' '''https://''server''/''channel_id'' '''

  Returns in the response body the content of the channel of id ''channel_id''. 
  Returns an ''ETag'' response header containing a unique hash.
  
  The request can contain a ''If-None-Match'' header containing a hash,
  If the hash is similar to the current hash of the channel, the server 
  will return a 304 and an empty body.
  
  The number of GET calls for a given channel are limited to 6. The channel will
  be deleted by the server after 6 successful GETs.
    
  Return codes:
     - 200: data retrieved successfully  
     - 404: the channel does not exists. It was not created by a call 
            to ''new_channel'' or timed out.
     - 304: the data was not changed.
     - 400: Bad or no client ID. The channel is deleted.
     - 403: the IP is blacklisted. 

   
'''PUT''' '''https://''server''/''channel_id'' '''

  Put in the channel of id ''channel_id'' the content of the request body. 
  Returns an ''ETag'' response header containing a unique hash.
     
  Return codes:
     - 200: data set successfully  
     - 404: the channel does not exists. It was not created by a call 
            to ''new_channel'' or timed out.
     - 400: Bad or no client ID. The channel is deleted.
     - 403: the IP is blacklisted. 


'''POST''' '''https://''server''/''report'' '''

  Reports a log to the server, and optionally ask for a channel deletion. 
  
  The log is the body of the request. If the 
  request contains a ''X-KeyExchange-Log'' header, its value is prepended
  to the log provided in the body. In other words, the header can be used
  for small logs, and the body for more info. The body size is limited to 
  2000 chars.  If both body and headers are empty, a 400 is raised.
    
  Optionally, if the request contains the ''X-KeyExchange-Id'' header and a
  ''X-KeyExchange-Cid'' header containing the channel id, the channel will
  be deleted by the server.
  
  Return codes:
     - 200: logged successfully  
     - 403: the IP is blacklisted.
     - 400: bad request (missing log or bad ids)

The messages reported are described at {{J-PAKE Report API}}.

== Detailed Flow ==

<ol>
<li>Device asks server for new channel ID (3 characters a-z0-9)
<pre>C: GET /new_channel HTTP/1.1
S: "a7id"</pre></li>
<li>Device generates PIN from random weak secret (4 characters a-z0-9) and the channel ID, computes and uploads J-PAKE msg 1.To prevent double uploads in case of retries, the If-None-Match: * header is specified. This makes sure that the message is only uploaded if the channel is empty. If it is not then the request will fail with a 412 Precondition Failed which should be considered the same as 200 OK. The 412 will also contain the Etag of the data was the client just uploaded.
<b>New in version 3:</b> Timeouts will need to be increased to accommodate the case where users have to create an account on the Desktop before proceeding. 
<pre>
C: PUT /a7id HTTP/1.1
C: If-None-Match: *
C: 
C: {
C:    'type': 'receiver1',
C:    'payload': {
C:       'gx1': '45...9b',
C:       'zkp_x1': {
C:          'b': '09e22607ead737150b1a6e528d0c589cb6faa54a',
C:          'gr': '58...7a'
C:          'id': 'receiver',
C:       }
C:       'gx2': 'be...93',
C:       'zkp_x2': {
C:          'b': '222069aabbc777dc988abcc56547cd944f056b4c',
C:          'gr': '5c...23'
C:          'id': 'receiver',
C:       }
C:    }
C: }
</pre>

Success response:

<pre>
S: HTTP/1.1 200 OK
S: ETag: "etag-of-receiver1-message"
</pre>

Response that will be returned on retries if the Desktop already replaced the message.

<pre>
S: HTTP/1.1 412 Precondition Failed
S: ETag: "etag-of-receiver1-message"
</pre></li>
<li>Desktop asks user for the PIN, extracts channel ID and weak secret, fetches Devce's msg 1
<pre>
C: GET /a7id HTTP/1.1
</pre>

Success response:

<pre>
S: HTTP/1.1 200 OK
S: ETag: "etag-of-receiver1-message"
</pre></li>

<li>Desktop computes and uploads msg 1. The If-Match header is set so that we only upload this message if the other side's previous message is still in the channel. This is to prevent double PUTs during retries. If a 412 is received then it means that our first PUT was actually correctly received by the server and that the other side has already uploaded it's next message. So just consider the 412 to be a 200.
<pre>
C: PUT /a7id HTTP/1.1
C: If-Match: "etag-of-receiver1-message"
C: 
C: {
C:    'type': 'sender1',
C:    'payload': {
C:       'gx1': '45...9b',
C:       'zkp_x1': {
C:          'b': '09e22607ead737150b1a6e528d0c589cb6faa54a',
C:          'gr': '58...7a'
C:          'id': 'sender',
C:       }
C:       'gx2': 'be...93',
C:       'zkp_x2': {
C:          'b': '222069aabbc777dc988abcc56547cd944f056b4c',
C:          'gr': '5c...23'
C:          'id': 'sender',
C:       }
C:    }
C: }
</pre>

Success response:

<pre>
S: HTTP/1.1 200 OK
S: Etag: "etag-of-sender1-message"
</pre>

<pre>
S: HTTP/1.1 412 Precondition Failed
S: Etag: "etag-of-sender1-message"
</pre></li>
<li>Device polls for Desktop's msg 1
<pre>
C: GET /a7id HTTP/1.1
C: If-None-Match: "etag-of-receiver1-message"

S: HTTP/1.1 304 Not Modified
</pre>
<p>Device tries again after 1s</p>
<pre>
C: GET /a7id HTTP/1.1

S: HTTP/1.1 200 OK
S: Etag: "etag-of-sender1-message"
...
</pre>
 <p>Device computes and uploads msg 2. The If-Match header is set so that we only upload this message if the other side's previous message is still in the channel. This is to prevent double PUTs during retries. If a 412 is received then it means that our first PUT was actually correctly received by the server and that the other side has already uploaded it's next message. So just consider the 412 to be a 200.
</p>
 <pre>C: PUT /a7id HTTP/1.1
C: If-Match: "etag-of-sender1-message"
C: 
C: {
C:    'type': 'receiver2',
C:    'payload': {
C:       'A': '87...82',
C:       'zkp_A': {
C:          'b': '6f...08',
C:          'id': 'receiver',
C:          'gr': 'f8...49'
C:       }
C:    }
C: }

S: HTTP/1.1 200 OK
S: ETag: "etag-of-receiver2-message"

S: HTTP/1.1 412 Precondition Failed
S: ETag: "etag-of-receiver2-message"
</pre></li>
<li>Desktop polls for and eventually retrieves Device's msg 2
<pre>
C: GET /a7id HTTP/1.1
C: If-None-Match: "etag-of-sender1-message"

S: HTTP/1.1 200 OK
S: Etag: "etag-of-receiver2-message"
...

S: HTTP/1.1 412 Precondition Failed
S: Etag: "etag-of-receiver2-message"
...
</pre>
<p>Desktop computes key, computes and uploads msg 2. The If-Match header is set so that we only upload this message if the other side's previous message is still in the channel. This is to prevent double PUTs during retries. If a 412 is received then it means that our first PUT was actually correctly received by the server and that the other side has already uploaded it's next message. So just consider the 412 to be a 200.
</p>
<pre>
C: PUT /a7id HTTP/1.1
C: If-Match: "etag-of-receiver2-message"
C: 
C: {
C:    'type': 'sender2',
C:    'payload': {
C:       'A': '87...82',
C:       'zkp_A': {
C:          'b': '6f...08',
C:          'id': 'sender',
C:          'gr': 'f8...49'
C:       }
C:    }
C: }

S: HTTP/1.1 200 OK
S: ETag: "etag-of-sender2-message"

S: HTTP/1.1 412 Precondition Failed
S: ETag: "etag-of-sender2-message"
</pre></li>

<li>Device retrieves Desktop's msg 2
<pre>
C: GET /a7id HTTP/1.1
C: If-No-Match: "etag-of-receiver2-message"

S: HTTP/1.1 200 OK
S: Etag: "etag-of-sender2-message"
{ 'type': 'sender2', ... }

S: HTTP/1.1 412 Precondition failed
S: Etag: "etag-of-sender2-message"
</pre>
<p>Device uploads encrypted known message "0123456789ABCDEF" to prove its knowledge (msg 3).
<b>New in version 3:</b>No longer includes the key, just the known message.
 The If-Match header is set so that we only upload this message if the other side's previous message is still in the channel. This is to prevent double PUTs during retries. If a 412 is received then it means that our first PUT was actually correctly received by the server and that the other side has already uploaded it's next message. So just consider the 412 to be a 200.
</p>
<pre>
C: PUT /a7id HTTP/1.1
C: If-Match: "etag-of-sender2-message"
C: 
C: {
C:    'type': 'receiver3',
C:    'payload': {
C:       'IV': "base64encoded=",
C:    }
C: }

S: HTTP/1.1 200 OK
S: Etag: "etag-of-receiver3-message"

S: HTTP/1.1 412 Precondition failed
S: Etag: "etag-of-receiver3-message"
</pre></li>
</li>

<li>Desktop retrieves Device's msg 3 to confirm connection. <b>new in version 3</b> No longer includes encrypted key. That will come in the 4th exchange once account creation has happened if needed and first sync upload has occurred. 
<pre>
C: GET /a7id HTTP/1.1
C: If-No-Match: ""

S: HTTP/1.1 200 OK
C: ETag: "etag-of-receiver3-message"
...
</pre>
Desktop verifies it against its own version. If the hash matches, it acquires (either the existing account or by creating a new one), then encrypts and uploads Sync credentials.

The If-Match header is set so that we only upload this message if the other side's previous message is still in the channel. This is to prevent double PUTs during retries. If a 412 is received then it means that our first PUT was actually correctly received by the server and that the other side has already uploaded it's next message. So just consider the 412 to be a 200.

<pre>
C: PUT /a7id HTTP/1.1
C: If-Match: "etag-of-receiver3-message"
C: 
C: {
C:    'type': 'sender3',
C:    'payload': {
C:       'IV': "base64encoded=",
C:       'hmac': "base64encoded=",
C:    }
C: }

S: HTTP/1.1 200 OK
S: Etag: "etag-of-sender3-message"

S: HTTP/1.1 412 Precondition failed
S: Etag: "etag-of-sender3-message"
</pre>
If the hash does not match, the Desktop deletes the session.
<pre>
C: DELETE /a7id HTTP/1.1

S: HTTP/1.1 200 OK
... 
</pre>
This means that Device will receive a 404 when it tries to retrieve the encrypted credentials.
</li></li>

</li>
<li><b>New in version 3:</b> Desktop signals that uploading sync has finished by encrypting and sending over the key. This is the 4th and last exchange.
<pre>
C: PUT /a7id HTTP/1.1
C: If-Match: "etag-of-receiver4-message"
C: 
C: {
C:    'type': 'sender4',
C:    'payload': {
C:       'ciphertext': "base64encoded=",
C:    }
C: }
Success response:

S: HTTP/1.1 200 OK
S: ETag: "etag-of-receiver4-message"
</pre>
</li>
<li><b>New in version 3:</b> Device retrieves encrypted key, decrypts it, starts downloading. Device kills session if HMAC does not match.
<pre>
C: GET /a7id HTTP/1.1
C: If-None-Match: "etag-of-receiver4-message"
</pre>
<li>Device retrieves encrypted credentials
<pre>
C: GET /a7id HTTP/1.1
C: If-None-Match: "etag-of-receiver4-message"

S: HTTP/1.1 200 OK
... 
</pre>
<p>decrypts Sync credentials and verifies HMAC.</p></li>
<pre>
C: DELETE /a7id HTTP/1.1

S: HTTP/1.1 200 OK
... 
</pre>
</li>


</ol>

== Security Logging & Defense ==
===DOS Defense===
* Least Recently Used (LRU) queue approach for monitoring IP addresses issuing frequent requests
** Configurable threshold for adding IP address to Blacklist/Penalty Box
** Configurable time-out for IP addresses added to Blacklist/Penalty Box
* A single shared blacklist will exist within memcache
* LRU queues will be unique to each server and will penalize an IP to the shared blacklist on memcache
* All thresholds will be controlled via the configuration page

=== TearDown DOS Defense ===
* Tear down requires valid channel and valid x-keyexchange-id value
* Statistically unlikely. Channel is 4 characters and keyexchange-id is 255 characters
* Brute force attempts will generate lots of noise and will be limited per DOS defense 

===Logging Points===
====CEF Logging====
* Bad action taken against a valid channel id (denoted by 400 error code)
** Examples: non-existent x-keyexchange-id, bad x-keyexchange-id
* Action taken against an invalid channel id
** Examples: request for properly formed, but not existing, channel id
* IP address sent to black list due to DOS prevention controls
** Examples: Flood of requests from a single IP
* Client fallback to original sync method
** Examples: Client unable to complete J-PAKE sync for any number of reasons and falls back to original sync approach
** Reported by client to server via reporting API

====Application Logging====
* Full application logging will be created to enable incident response review
* Logged to application server and not via CEF
* Logs will include:
** Timestamp
** IP address
** Full URL
** x-keyexchange-id
** Event
** Other non-essential headers will be discarded
=== Admin Web Page===
* A small web administrator page will be created which will allow an admin to view all IP addresses that are currently blacklisted. 
* The admin will be able to un-block any of the IP addresses through this page
* Otherwise the IP address will be removed from the black list after the time has elapsed that is defined within the configuration file
* Access to the web page will be password-protected with a simple .htaccess file and IP filtering access (10.*.*.*)