Mobile/Fennec/Android/Testing: Difference between revisions

no edit summary
No edit summary
Line 1: Line 1:
This page has instructions for running Firefox tests locally on a device (Android phone, tablet, or emulator) of your choice.
This page has instructions for running Firefox tests locally on a device (Android phone, tablet, or emulator) of your choice.


Having trouble? Ping :gbrown on #mobile, or ask for help on #ateam.
Having trouble? Ping :gbrown on #mobile.


== For the impatient... ==
== For the impatient... ==
Line 14: Line 14:
   mach cppunittest
   mach cppunittest
   mach geckoview-junit
   mach geckoview-junit
  mach web-platform-test
  mach marionette-test
  mach test


They all run against a connected Android device using your Firefox for Android build. Don't have a device? These commands will offer to start an emulator.
They all run against a connected Android device using your Firefox for Android build. Don't have a device? These commands will offer to start an emulator.
Line 142: Line 145:


   mach robocop <test-name>
   mach robocop <test-name>
If you make changes to the tests and want to see them run on a device, you need to build the tests again and reinstall.
  mach build mobile/android/tests/browser/robocop
  mach robocop
This builds the tests in <tt>mobile/android/tests/browser/robocop</tt> and then installs the debug-signed Robocop APK onto the device. (Building <tt>mobile/android</tt> also builds the tests within <tt>mobile/android/tests</tt>.)


Notes:
Notes:
* Mach is self documenting!  For help, try <tt>mach help robocop</tt>.
* To run one test at a time, find the test name (like "testLoad") in <tt>mobile/android/tests/browser/robocop/robocop.ini</tt> and pass it as an argument, like: <tt>mach robocop testLoad</tt>.
* To run one test at a time, find the test name (like "testLoad") in <tt>mobile/android/tests/browser/robocop/robocop.ini</tt> and pass it as an argument, like: <tt>mach robocop testLoad</tt>.
* A rooted device is required. Test harnesses may need to kill processes, copy and delete files, or perform other operations which may require special permissions on some devices.
* A rooted device is required. Test harnesses may need to kill processes, copy and delete files, or perform other operations which may require special permissions on some devices.
Line 228: Line 223:


Notes:
Notes:
* A rooted device is recommended, but may not be required. Test harnesses may need to kill processes, copy and delete files, or perform other operations which may require special permissions on some devices.
* A rooted device is required. Test harnesses may need to kill processes, copy and delete files, or perform other operations which may require special permissions on some devices.
* Setup can take several minutes! Setup is faster if unzip is available on the remote device; if your device does not have unzip, try installing busybox.
* Setup can take several minutes! Setup is faster if unzip is available on the remote device; if your device does not have unzip, try installing busybox.


Line 385: Line 380:


   --remoteTestRoot=<remote-directory>
   --remoteTestRoot=<remote-directory>
== S1/S2 Automation ==
These tests start Firefox for Android with a URL and measure the time to throbber start, time to throbber stop, and drawing end times.
S1/S2 graphs can be viewed at:  http://phonedash.mozilla.org/
Manual run instructions can be found at: https://etherpad.mozilla.org/fennec-perf-ts-take2
See also: https://wiki.mozilla.org/Mobile/Performance/S1S2-Tests


== Trouble-shooting testing problems ==
== Trouble-shooting testing problems ==
Line 410: Line 396:
** Make sure _SERVER_ADDR in the test output is the same as your machine's IP address.
** Make sure _SERVER_ADDR in the test output is the same as your machine's IP address.
* If using MOZ_HOST_BIN, ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel.
* If using MOZ_HOST_BIN, ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel.
* Is your device rooted? Many test suites require root permissions. Don't want to root your device? Consider using an emulator with 'mach android-emulator'.
Confirmed users
1,759

edits