Identity/AttachedServices/Key Stretching Performance Tests: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (oops, s not ms)
No edit summary
Line 13: Line 13:
! [3] 20kPBKDF + local(scrypt(64k,8,1)) + 20kPBKDF
! [3] 20kPBKDF + local(scrypt(64k,8,1)) + 20kPBKDF
| -- || -- || -- || --  || -- || 7.3s (0.1369 ops/sec, 10 runs sampled, extremes discarded -- 7362-7505msec)
| -- || -- || -- || --  || -- || 7.3s (0.1369 ops/sec, 10 runs sampled, extremes discarded -- 7362-7505msec)
|-
|-
! [4] 20kPBKDF + emscripten(scrypt(64k,8,1)) + 20kPBKDF
| -- || -- || -- || --  || -- || --
|-
|-
|}
|}
Line 21: Line 25:
* [2] '''20k*PBKDF+scrypt(64k,8,1)+20k*PBKDF''' keyStretch.derive(email, password) - Full Key Stretch with a remote scrypt helper at "http://scrypt.dev.lcip.org/" (EC2 m1.small, est scrypt(64k,8,1) time = 2.2s).  [https://wiki.mozilla.org/Identity/AttachedServices/KeyServerProtocol#stretch-KDF #stretch-KDF + #main-KDF]
* [2] '''20k*PBKDF+scrypt(64k,8,1)+20k*PBKDF''' keyStretch.derive(email, password) - Full Key Stretch with a remote scrypt helper at "http://scrypt.dev.lcip.org/" (EC2 m1.small, est scrypt(64k,8,1) time = 2.2s).  [https://wiki.mozilla.org/Identity/AttachedServices/KeyServerProtocol#stretch-KDF #stretch-KDF + #main-KDF]
* [3]  '''20kPBKDF + local(scrypt(64k,8,1)) + 20kPBKDF'''
* [3]  '''20kPBKDF + local(scrypt(64k,8,1)) + 20kPBKDF'''
* [4]  '''20kPBKDF + emscripten(scrypt(64k,8,1)) + 20kPBKDF'''  Using the [https://github.com/tonyg/js-scrypt Pure-Javascript Emscripten-compiled scrypt routine]


Note that the m1.small instance runs scrypt in 2.2s, whereas the more cost-effective (given serious load) c1.medium or c1.xlarge instances will run it in 1.0s .
Note that the m1.small instance runs scrypt in 2.2s, whereas the more cost-effective (given serious load) c1.medium or c1.xlarge instances will run it in 1.0s .

Revision as of 22:14, 21 August 2013

Test Results (in seconds per operation)
Browser / Test Firefox 23 Desktop Google Chrome 28 Firefox Android Beta 24 Firefox OS Android WebView Android + OpenSSL
[1] 20k PBKDF2-SHA256 186ms (5.39 ops/sec ±4.20%, 30 runs sampled) 123ms (8.12 ops/sec ±1.15%, 43 runs sampled) 2.13s (0.47 ops/sec ±7.73%, 7 runs sampled) 4.17ms (0.24 ops/sec ±7.06%, 6 runs sampled) 1.52s (0.66 ops/sec ±4.35%, 8 runs sampled) 351ms (2.85 ops/sec, **1 run sampled**)
[2] 20k*PBKDF + remote(scrypt(64k,8,1)) + 20k*PBKDF 1.52s (0.66 ops/sec ±10.70%, 8 runs sampled) 1.27s (0.79 ops/sec ±1.98%, 8 runs sampled) 6.25s (0.16 ops/sec ±9.02%, 5 runs sampled) 10s (0.10 ops/sec ±3.37%, 5 runs sampled) 5.56s (0.18 ops/sec ±21.96%, 6 runs sampled) --
[3] 20kPBKDF + local(scrypt(64k,8,1)) + 20kPBKDF -- -- -- -- -- 7.3s (0.1369 ops/sec, 10 runs sampled, extremes discarded -- 7362-7505msec)
[4] 20kPBKDF + emscripten(scrypt(64k,8,1)) + 20kPBKDF -- -- -- -- -- --


Tests

Note that the m1.small instance runs scrypt in 2.2s, whereas the more cost-effective (given serious load) c1.medium or c1.xlarge instances will run it in 1.0s .

Test Framework

These performance tests use Benchmark.js

  • "ops/sec" stands for operations per second. That is how many times a test is projected to execute in a second.
  • A test is repeatedly executed until it reaches the minimum time needed to get a percentage uncertainty for the measurement of less than or equal to 1%. The number of iterations will vary depending on the resolution of the environment’s timer and how many times a test can execute in the minimum run time. We collect completed test runs for 5 seconds (configurable), or at least 5 runs (also configurable), and then perform statistical analysis on the sample. So, a test may be repeated 100,000 times in 50 ms (the minimum run time for most environments), and then repeated 100 times more (5 seconds). A larger sample size (in this example, 100), leads to a smaller margin of error.

Test Suite Page: http://v14d.com/picl/benchmark.html

Retrieved from "https://wiki.allizom.org/index.php?title=Identity/AttachedServices/Key_Stretching_Performance_Tests&oldid=697975"