Firefox OS/Performance/Automation/Tools: Difference between revisions

Latest revision as of 13:59, 1 February 2015

Progress

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 400 [1] => Bad Request ) ) 1

Performance

There are two automated performance testing tools for Gaia; b2gperf and make test-perf. Both use marionette.

b2gperf
- An external Python tool maintained by the A-team.
- Source code and documentation are hosted on github at https://github.com/mozilla/b2gperf

Running b2gperf

Install reqs with 'pip install b2gperf'
Update reqs with 'pip install --upgrade b2gperf'
run 'adb forward tcp:2828 tcp:2828' - This lets marionette connect to the device
Run b2g perf with 'b2gperf --delay=10 --reset --iterations=30 AppNameHere'. For example, to run settings, run 'b2gperf --delay=10 --reset --iterations=30 Settings'. CASE SENSITIVE

Make test-perf

The in-tree Gaia testing framework maintained by the FirefoxOS Performance team and documented on MDN.
Code
Tests
- See <APP_NAME>/test/performance folders within the Gaia repository.
- Examples
  - Dialer
  - Settings

Endurance

Endurance Tests

@@ Line 18: / Line 18: @@
 ** Source code and documentation are hosted on github at https://github.com/mozilla/b2gperf
-== Running B2g Perf ==
+== Running b2gperf ==
 # Install reqs with 'pip install b2gperf'
 # Update reqs with 'pip  install --upgrade b2gperf'
+# run 'adb forward tcp:2828 tcp:2828' - This lets marionette connect to the device
 # Run b2g perf with 'b2gperf --delay=10 --reset --iterations=30 AppNameHere'. For example, to run settings, run 'b2gperf --delay=10 --reset --iterations=30 Settings'. CASE SENSITIVE
-* Gaia Test Framework aka '''make test-perf'''
+== Make test-perf ==
-** In-tree testing framework for gaia maintained by the [[FirefoxOS/Performance|FirefoxOS Performance]] team.
+* The in-tree Gaia testing framework maintained by the [[FirefoxOS/Performance|FirefoxOS Performance]] team and [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests documented on MDN].
-** Documented on MDN at https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests
+* '''Code'''
-** Contains app specific tests
+** [https://github.com/mozilla-b2g/gaia/blob/master/bin/gaia-perf-marionette          Runner]
-*** [https://github.com/mozilla-b2g/gaia/blob/master/apps/communications/dialer/test/performance Dialer Perf Tests]
+** [https://github.com/mozilla-b2g/gaia/blob/master/tests/reporters/basemozperf.js    Reporter]
-*** See ''<APP_NAME>/test/performance'' folders within the [https://github.com/mozilla-b2g/gaia Gaia Repo] for more performance tests.
+** [https://github.com/mozilla-b2g/gaia/blob/master/tests/reporters/consolemozperf.js Reporter : console]
+** [https://github.com/mozilla-b2g/gaia/blob/master/tests/reporters/jsonmozperf.js    Reporter : json]
-* [[/FPS|FPS via Eideticker]]
+** [https://github.com/mozilla-b2g/gaia/tree/master/tests/performance                 Source]
+* '''Tests'''
+** See ''<APP_NAME>/test/performance'' folders within the [https://github.com/mozilla-b2g/gaia/tree/master/apps Gaia] repository.
+** Examples
+*** [https://github.com/mozilla-b2g/gaia/blob/master/apps/communications/dialer/test/performance Dialer]
+*** [https://github.com/mozilla-b2g/gaia/tree/master/apps/settings/test/performance Settings]
 ==Endurance==
@@ Line 40: / Line 46: @@
 ==References==
+* [[../FPS|FPS via Eideticker]]
 * [https://fxos-perf.etherpad.mozilla.org/2013-07-17-kickoff Performance Automation Kickoff]
+* [[/Regression_Detection_Requirements|Regression Detection Requirements]]