B2G/QA/Automation/UI/MarionetteJS On Device Status: Difference between revisions
(Created page with "== Introduction == Currently, the MarionetteJS tests can only be run on b2g desktop. The plan is to get them running on device too, just like the Gaia UI tests are running on...") |
(added references and limitations) |
||
| Line 29: | Line 29: | ||
4) Connect device and run: | 4) Connect device and run: | ||
$ gaia-integration --buildapp device | $ gaia-integration --buildapp device | ||
== Limitations of MarionetteJS == | |||
* Not possible to run from a manifest | |||
** So things like filter by abilities (like "is a SIM card required?") is not possible | |||
* Not possible to push/pull/delete files on the device | |||
* Not possible to run on a 319MB device | |||
** Currently, MarionetteJS requires more memory than gaia-ui test | |||
== Status == | == Status == | ||
| Line 39: | Line 47: | ||
failed: 0 | failed: 0 | ||
todo: 0 | todo: 0 | ||
== References == | |||
* https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_integration_tests | |||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1131866 (MarionetteJS can't be executed on device: runner-service err) | |||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1141793 (meta: Bug 1141793 - [fxos-automation] Enable Marionette tests to be run on device ) | |||
* https://bugzilla.mozilla.org/show_bug.cgi?id=1138542 ( Draft report on MarionetteJS on-device status ) | |||
* https://bugzilla.mozilla.org/show_bug.cgi?id=994764#c6 | |||
* https://docs.google.com/a/mozilla.com/spreadsheets/d/1L1QAOuKx5pWNizMmotmyauCZ0zl3JrgsGDx0PnWxGd8/edit#gid=0 | |||
Revision as of 15:52, 16 March 2015
Introduction
Currently, the MarionetteJS tests can only be run on b2g desktop. The plan is to get them running on device too, just like the Gaia UI tests are running on device. The work being done to get MarionetteJS running on device is being done by Ghislain Aus Lacroix [:aus], James Lal [:lightsofapollo] and Gareth Aye [:gaye]
Setup
Instructions for running them on device are here: https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_integration_tests
Basically, what you need to do is: - Connect a device (Flame) that has an engineering build on it - On the command line, type: BUILDAPP=device make test-integration
In addition, the following steps probably need to be carried out (although that might change): https://bugzilla.mozilla.org/show_bug.cgi?id=994764#c6 1) Install libzmq, one of: $ sudo apt-get install libzmq3-dev $ sudo yum install zeromq3-devel $ sudo brew install zmq
2) Install runner-service: $ cd gaia/tests/python/runner-service $ python setup.py install
3) Install npm packages $ npm install marionette-socket-host $ npm install mocha-socket-reporter
4) Connect device and run: $ gaia-integration --buildapp device
Limitations of MarionetteJS
- Not possible to run from a manifest
- So things like filter by abilities (like "is a SIM card required?") is not possible
- Not possible to push/pull/delete files on the device
- Not possible to run on a 319MB device
- Currently, MarionetteJS requires more memory than gaia-ui test
Status
Currently, trying to run this fails for me with this message: [TypeError: Cannot read property '1' of null] Couldn't find todo count in marionette-mocha output:
- ~*~* Results *~*~*
passed: 0 failed: 0 todo: 0
References
- https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_integration_tests
- https://bugzilla.mozilla.org/show_bug.cgi?id=1131866 (MarionetteJS can't be executed on device: runner-service err)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1141793 (meta: Bug 1141793 - [fxos-automation] Enable Marionette tests to be run on device )
- https://bugzilla.mozilla.org/show_bug.cgi?id=1138542 ( Draft report on MarionetteJS on-device status )
- https://bugzilla.mozilla.org/show_bug.cgi?id=994764#c6
- https://docs.google.com/a/mozilla.com/spreadsheets/d/1L1QAOuKx5pWNizMmotmyauCZ0zl3JrgsGDx0PnWxGd8/edit#gid=0