Firefox OS/Performance/Automation/Tools: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
m (Lakrits moved page FirefoxOS/Performance/Automation/Tools to Firefox OS/Performance/Automation/Tools: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the spelling of its name...)
 
(20 intermediate revisions by 4 users not shown)
Line 1: Line 1:
There are two automated performance testing tools for Gaia.
==Progress==
<bugzilla type="count" display="bar">
    {
        "product": "Firefox OS",
        "keywords":"perf",
        "keywords_type":"contains",
        "whiteboard":"c=automation",
        "whiteboard_type":"contains",
        "x_axis_field":"status"
    }
</bugzilla>


b2gperf and make test-perf. They both share their use of [https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette marionette].
==Performance==
There are two automated performance testing tools for Gaia; '''''b2gperf''''' and '''''make test-perf'''''. Both use [https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette marionette].


==b2gperf==
* '''b2gperf'''
** An external Python tool maintained by the [[Auto-tools/The_Ateam|A-team]].
** Source code and documentation are hosted on github at https://github.com/mozilla/b2gperf


It is maintained by the [[Auto-tools/The_Ateam|A-team]]. It is an external tool in Python.
== Running b2gperf ==
# Install reqs with 'pip install b2gperf'
# Update reqs with 'pip  install --upgrade b2gperf'
# run 'adb forward tcp:2828 tcp:2828' - This lets marionette connect to the device
# Run b2g perf with 'b2gperf --delay=10 --reset --iterations=30 AppNameHere'. For example, to run settings, run 'b2gperf --delay=10 --reset --iterations=30 Settings'. CASE SENSITIVE


Its source code and documentation is hosted on github.
== Make test-perf ==
* The in-tree Gaia testing framework maintained by the [[FirefoxOS/Performance|FirefoxOS Performance]] team and [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests documented on MDN].
* '''Code'''
** [https://github.com/mozilla-b2g/gaia/blob/master/bin/gaia-perf-marionette          Runner]
** [https://github.com/mozilla-b2g/gaia/blob/master/tests/reporters/basemozperf.js    Reporter]
** [https://github.com/mozilla-b2g/gaia/blob/master/tests/reporters/consolemozperf.js Reporter : console]
** [https://github.com/mozilla-b2g/gaia/blob/master/tests/reporters/jsonmozperf.js    Reporter : json]
** [https://github.com/mozilla-b2g/gaia/tree/master/tests/performance                Source]
* '''Tests'''
** See ''<APP_NAME>/test/performance'' folders within the [https://github.com/mozilla-b2g/gaia/tree/master/apps Gaia] repository.
** Examples
*** [https://github.com/mozilla-b2g/gaia/blob/master/apps/communications/dialer/test/performance Dialer]
*** [https://github.com/mozilla-b2g/gaia/tree/master/apps/settings/test/performance Settings]


https://github.com/mozilla/b2gperf
==Endurance==
* [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance Endurance Tests]


==make test-perf==
==Integration==
* [https://developer.mozilla.org/en-US/docs/Marionette/Marionette_JavaScript_Tools Marionette JS Tools]
* [https://github.com/mozilla-b2g/marionette-js-runner/blob/master/README.md Marionette JS Tools ReadMe]


It is the in-tree testing framework for gaia. It is maintained my the fxos-perf team.
==References==
 
* [[../FPS|FPS via Eideticker]]
It is documented on MDN:
* [https://fxos-perf.etherpad.mozilla.org/2013-07-17-kickoff Performance Automation Kickoff]
 
* [[/Regression_Detection_Requirements|Regression Detection Requirements]]
https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests

Latest revision as of 13:59, 1 February 2015

Progress

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 400 [1] => Bad Request ) ) 1


Performance

There are two automated performance testing tools for Gaia; b2gperf and make test-perf. Both use marionette.

Running b2gperf

  1. Install reqs with 'pip install b2gperf'
  2. Update reqs with 'pip install --upgrade b2gperf'
  3. run 'adb forward tcp:2828 tcp:2828' - This lets marionette connect to the device
  4. Run b2g perf with 'b2gperf --delay=10 --reset --iterations=30 AppNameHere'. For example, to run settings, run 'b2gperf --delay=10 --reset --iterations=30 Settings'. CASE SENSITIVE

Make test-perf

Endurance

Integration

References

Retrieved from "https://wiki.allizom.org/index.php?title=Firefox_OS/Performance/Automation/Tools&oldid=1052507"