|
|
| Line 9: |
Line 9: |
| == Current Tests == | | == Current Tests == |
|
| |
|
| === Cold Load Time and FPS === | | === Cold Load Time === |
| * '''Cold Load Time''': [https://github.com/mozilla/b2gperf b2gperf] measures time taken from launching an app to the apploadtime event firing. This generates app startup times which are useful for identifying some kinds of performance regressions, but which are much shorter than camera-based startup times measured by Eideticker. This is the exact same test as startup time, but it runs more iterations.
| | The [https://github.com/mozilla/b2gperf b2gperf] tool measures time taken from launching an app to the apploadtime event firing. This generates app startup times which are useful for identifying some kinds of performance regressions, but which are much shorter than camera-based startup times measured by [[Project_Eideticker|Eideticker]]. |
| * '''FPS''': [https://github.com/mozilla/b2gperf b2gperf] performs a smooth scroll/swipe against the app and measures the frames per second using Gecko
| |
|
| |
|
| The apps used for '''Cold Load Time''' and '''FPS''' are:
| | Any app should work with b2gperf for gathering cold load times, however the following will also populate reference workloads: |
| * contacts
| | {| class="wikitable" |
| ** startup test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L408 200 pre populated contacts]]
| | |- |
| ** fps test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L549 200 pre popultaed contacts]]
| | ! Application !! Workload |
| * messages
| | |- |
| ** startup test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L420 200 pre populated messages]]
| | | [https://github.com/mozilla/b2gperf/blob/232d2d0d18b01fd16f2e9bbf30f48e7c91907a58/b2gperf/b2gperf.py#L420 Contacts] || 200 contacts |
| ** fps test, [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L679 200 pre populated messages]]
| | |- |
| * settings
| | | [https://github.com/mozilla/b2gperf/blob/232d2d0d18b01fd16f2e9bbf30f48e7c91907a58/b2gperf/b2gperf.py#L426 Gallery] || 700 pictures |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L387 startup test]]
| | |- |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L736 fps test]]
| | | [https://github.com/mozilla/b2gperf/blob/232d2d0d18b01fd16f2e9bbf30f48e7c91907a58/b2gperf/b2gperf.py#L432 Messages] || 200 messages |
| * gallery
| | |- |
| ** startup test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L414 700 pre populated pictures]]
| | | [https://github.com/mozilla/b2gperf/blob/232d2d0d18b01fd16f2e9bbf30f48e7c91907a58/b2gperf/b2gperf.py#L438 Music] || 500 tracks |
| ** fps test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L602 50 pre populated pictures]]
| | |- |
| * video
| | | [https://github.com/mozilla/b2gperf/blob/232d2d0d18b01fd16f2e9bbf30f48e7c91907a58/b2gperf/b2gperf.py#L444 Video] || 100 videos |
| ** startup test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L432 100 pre populated videos]]
| | |} |
| ** fps test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L750 50 pre populated videos]]
| |
| * music
| |
| ** startup test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L426 500 pre populated songs]]
| |
| ** fps test, with [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L703 200 pre populated songs]]
| |
| * email
| |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L387 startup test]]
| |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L579 fps test]]
| |
| * browser
| |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L387 startup test]]
| |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L522 fps test]]
| |
| * homescreen
| |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L387 startup test]]
| |
| ** [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L637 fps test]]
| |
|
| |
|
| * NOTE: the populated data is from [[https://github.com/mozilla/b2gpopulate/blob/master/b2gpopulate/b2gpopulate.py b2gpopulate]]
| | The full list of applications currently tested is below, however this may become out of date and the best place to check the latest list is in [http://jenkins1.qa.scl3.mozilla.com:8080/ Jenkins] or [https://datazilla.mozilla.org/b2g/?branch=master&test=cold_load_time Datazilla]: |
| | | * Phone |
| [[https://github.com/mozilla/b2gperf/blob/master/b2gperf/b2gperf.py#L387 Startup only]] tests (no fps, and no pre populated data):
| | * Contacts |
| * camera | | * Messages |
| * phone | | * Settings |
| * calendar | | * Gallery |
| * clock | | * Video |
| * fm_radio | | * Music |
| * usage | | * Camera |
| * template | | * Email |
| | * Calendar |
| | * Clock |
| | * FM Radio |
| | * Usage |
| | * Template |
| | * Browser |
|
| |
|
| === Endurance === | | === Endurance === |