MozillaWiki

Mobile/Fennec/Android/Testing: Difference between revisions

Page Discussion

Mobile/Fennec/Android/Testing (view source)

Revision as of 16:50, 25 November 2015

434 bytes added ,  25 November 2015
m
no edit summary
No edit summary
mNo edit summary
Line 1: Line 1:
This page has instructions for running tests locally on a device (Android phone, tablet, or emulator) of your choice.
This page has instructions for running Firefox tests locally on a device (Android phone, tablet, or emulator) of your choice.


Having trouble? Ping :gbrown on #mobile, or ask for help on #ateam.
Having trouble? Ping :gbrown on #mobile, or ask for help on #ateam.
Line 17: Line 17:


== Test Environment ==
== Test Environment ==
When testing Firefox for Android with make or mach on your local computer, tests run on an Android device, but are controlled by a test harness running on your computer. The test environment usually consists of:
When testing Firefox for Android with mach on your local computer, tests run on an Android device, but are controlled by a test harness running on your computer. The test environment usually consists of:
* a "host" computer, running Linux or OSX
* a "host" computer, running Linux or OSX
* a usb-connected Android device, such as a phone or tablet, or an Android emulator running on your computer
* a usb-connected Android device, such as a phone or tablet, or an Android emulator running on your computer
Line 35: Line 35:
[[Auto-tools/Projects/Robocop|General Robocop information]] and http://mxr.mozilla.org/mozilla-central/source/mobile/android/tests/browser/robocop/README.rst.
[[Auto-tools/Projects/Robocop|General Robocop information]] and http://mxr.mozilla.org/mozilla-central/source/mobile/android/tests/browser/robocop/README.rst.


The Robocop test suite verifies UI behavior in Firefox for Android by pointing-and-clicking through the UI on running device or emulator.  It is built on the Robotium testing framework. To run tests locally, a separate Robocop test APK also needs to be installed.
The Robocop test suite verifies UI behavior in Firefox for Android by pointing-and-clicking through the UI on a running device or emulator.  It is built on the Robotium testing framework. To run tests locally, a separate Robocop test APK also needs to be installed.


To get tests running locally, first build and install Firefox for Android,  
To run robocop tests, first build and install Firefox for Android,  


   mach build
   mach build
Line 62: Line 62:
* To run one test at a time, find the test name (like "testLoad") in <tt>mobile/android/tests/browser/robocop/robocop.ini</tt> and pass it as an argument, like: <tt>mach robocop testLoad</tt>.
* To run one test at a time, find the test name (like "testLoad") in <tt>mobile/android/tests/browser/robocop/robocop.ini</tt> and pass it as an argument, like: <tt>mach robocop testLoad</tt>.
* A rooted device is NOT required.
* A rooted device is NOT required.
Troubleshooting:
* Ensure the device's screen is on
* Ensure that the device and host machine are on the same network.
** Are the phone and the desktop both using wifi? (wifi vs ethernet??)
** Are the phone and the desktop both using the same wifi network? (Mozilla vs Mozilla Guest??)
** Is the desktop environment running in a VM? If so, you likely want a "Bridged" connection -- not NAT or Host-only.
* Ensure Robocop has the correct IP address for your machine
** Make sure _SERVER_ADDR in the Robocop output is the same as your machine's IP address.
* If using MOZ_HOST_BIN, ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel.
*Additional tips at [[Auto-tools/Projects/Robocop#Frequently_found_errors]]
*Additional tips at [[Auto-tools/Projects/Robocop#Frequently_found_errors]]


Line 82: Line 72:
* Ensure your device is connected and visible with "adb devices".
* Ensure your device is connected and visible with "adb devices".
* '''Ensure that the device and host machine are on the same network''' <-- This is important. The device and the host need to communicate over the network to run the tests. Having the device connected to the host via USB is '''not''' sufficient.
* '''Ensure that the device and host machine are on the same network''' <-- This is important. The device and the host need to communicate over the network to run the tests. Having the device connected to the host via USB is '''not''' sufficient.
* Ensure that MOZ_HOST_BIN has been set according to the [[Mobile/Fennec/Android#Host_Builds_.28MOZ_HOST_BIN.29|directions above]].


Running tests:
Running tests:
Line 99: Line 88:
[https://developer.mozilla.org/en-US/docs/Mozilla/QA/Writing_xpcshell-based_unit_tests General xpcshell test information.]
[https://developer.mozilla.org/en-US/docs/Mozilla/QA/Writing_xpcshell-based_unit_tests General xpcshell test information.]


Be sure you have successfully built Firefox for Android and generated an APK, as described above.
Be sure you have successfully built Firefox for Android and generated an APK, as described above. xpcshell tests do not require Firefox for Android to be installed, but the APK must exist on the host.


To run all tests referenced by the master xpcshell manifest:
To run all tests referenced by the master xpcshell manifest:


   mach xpcshell-test
   mach xpcshell-test
(Currently the master manifest is restricted: not all tests are run.)


To run a subset of tests in the specified directory:
To run a subset of tests in the specified directory:
Line 123: Line 110:
== cppunittests ==
== cppunittests ==
[https://developer.mozilla.org/en/docs/Compiled-code_automated_tests General cppunit test information]
[https://developer.mozilla.org/en/docs/Compiled-code_automated_tests General cppunit test information]
Be sure you have successfully built Firefox for Android, as described above; this should build the unit test executables.


To run a single compiled code test:
To run a single compiled code test:
Line 143: Line 132:
== reftests (and crashtests and js-reftests) ==
== reftests (and crashtests and js-reftests) ==
[https://developer.mozilla.org/en/docs/Creating_reftest-based_unit_tests General reftest information.]
[https://developer.mozilla.org/en/docs/Creating_reftest-based_unit_tests General reftest information.]
Pre-requisites:
* Ensure that Firefox for Android has been built and installed: mach build && mach package && mach install
* Ensure your device is connected and visible with "adb devices".
* '''Ensure that the device and host machine are on the same network''' <-- This is important. The device and the host need to communicate over the network to run the tests. Having the device connected to the host via USB is '''not''' sufficient.


Running reftests:
Running reftests:
Line 390: Line 384:
* Is your device connected? Does it appear in the output from "adb devices"?
* Is your device connected? Does it appear in the output from "adb devices"?
* Can you run adb shell?
* Can you run adb shell?
* Ensure the device's screen is on.
* Ensure that the device and host machine are on the same network.
** Are the phone and the desktop both using wifi? (wifi vs ethernet??)
** Are the phone and the desktop both using the same wifi network? (Mozilla vs Mozilla Guest??)
** Is the desktop environment running in a VM? If so, you likely want a "Bridged" connection -- not NAT or Host-only.
* Ensure Robocop has the correct IP address for your machine
** Make sure _SERVER_ADDR in the test output is the same as your machine's IP address.
* If using MOZ_HOST_BIN, ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel.
Gbrown
Confirmed users
1,759

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1107199"