MozillaWiki

Auto-tools/Projects/Robocop/WritingTests: Difference between revisions

Page Discussion

Auto-tools/Projects/Robocop/WritingTests (view source)

Revision as of 14:20, 9 March 2016

1,143 bytes added ,  9 March 2016
→‎Logging
 
(19 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= What's this? =
Robotium is a test framework created to make it easy to write powerful and robust automatic black-box test cases for Android applications. Robocop provides a wrapper around Robotium making it even easier to write and execute UI tests for native Fennec.


Robotium is a test framework created to make it easy to write powerful and robust automatic black-box test cases for Android applications. Robocop provides a wrapper around Robotium making it even easier to write and execute UI tests for native Fennec.
New tests should probably use the UITest base class - see the documentation at [[Mobile/Fennec/Android/UITest|UITest]]. Most of the information below will also be relevant to users of UITest (though the information should probably be curated).
 
== Creating a new test ==
These tests will extend BaseTest.


= Creating a new test =
The best way to create a new Robocop test is to copy and modify an existing one -- see mobile/android/base/tests.


The best way to create a new Robocop test is to copy and modify an existing one -- see mobile/android/base/tests.  
(We are currently upgrading our tests from Robotium 3.6 to 4.2 [{{bug|912590}}], so the APIs used in the tests may be out of date. It might be helpful to take a look at the [http://code.google.com/p/robotium/wiki/Changelog Robotium changelog] to see some of the new API methods.)


Create a new file in the test directory named test[YourFeature].java.in.
Create a new file in the test directory named test[YourFeature].java.in.
Line 20: Line 23:
</pre>
</pre>


For convenience, you can also extend one of the provided abstract classes, like BaseTest. This derives from ActivityInstrumentationTestCase2 but also provides a bunch of boilerplate and utility functions.
For convenience, you can also extend one of the provided abstract classes, like BaseTest, PixelTest, or AboutHomeTest. BaseTest derives from ActivityInstrumentationTestCase2 but also provides a bunch of boilerplate and utility functions.
<pre>
<pre>
public class testMyFeature extends BaseTest
public class testMyFeature extends BaseTest
Line 26: Line 29:


Each test class must have three methods:  
Each test class must have three methods:  
<pre>
protected void setUp() // Starts Fennec and sets up commonly used member variables. This is usually very similar for every test class.
protected void setUp() // Starts Fennec and sets up commonly used member variables. This is usually very similar for every test class.
public void test[YourFeature]() // Your test code goes here. Use the Robocop API to access Fennec elements, click, send keys, and assert conditions.
public void test[YourFeature]() // Your test code goes here. Use the Robocop API to access Fennec elements, click, send keys, and assert conditions.
Line 31: Line 35:
</pre>
</pre>


If you extend BaseTest, the setUp() and tearDown() are provided for you, but you can still override them and do additional setup/teardown if needed; just make sure you call super.setUp() and super.tearDown() in your overiddes as necessary.
If you extend one of the provided abstract classes, the setUp() and tearDown() are provided for you, but you can still override them and do additional setup/teardown if needed; just make sure you call super.setUp() and super.tearDown() in your overiddes as necessary.


Finally, add your new test file to the test manifest: mobile/android/base/tests/robocop.ini.
Finally, add your new test file to the test manifest: mobile/android/base/tests/robocop.ini:
<pre>
[testAllPagesTab]
[testHistoryTab]
...
[testMasterPassword]
[testDeviceSearchEngine]
[testMyFeature] # NEW!
</pre>


== Creating a new Content Provider test ==
=== Creating a new Content Provider test ===


We have infrastructure for testing content providers in an isolated environment. This means that we ensure that any updates to the databases behind content providers will not affect or be affected by the Fennec app.
We have infrastructure for testing content providers in an isolated environment. This means that we ensure that any updates to the databases behind content providers will not affect or be affected by Firefox.


To write a content provider test, you should follow the same file naming and Java packaging guidelines described for UI tests. However, you should create a class that inherits from ''ContentProviderTest'' instead of ''BaseTest'' or ''ActivityInstrumentationTestCase2''. Each Content Provider test class must have those the same three methods than UI tests (setUp(), test[YourFeature]() and tearDown) with one important difference: the ''setUp()'' method should call the following parent method:
To write a content provider test, follow the same file naming and Java packaging guidelines described for UI tests, but create a class that inherits from ''ContentProviderTest'' instead of ''BaseTest''. Each Content Provider test class must have the same three methods as UI tests (setUp(), test[YourFeature]() and tearDown) with one important difference: the ''setUp()'' method should call the following parent method:


<pre>
<pre>
Line 54: Line 66:
</pre>
</pre>


After this is called, ''mProvider'' will point to your an isolated instance of your content provider and you can make insert/query/update/delete calls on it as expected. For more details and sample code on how to implement Content Provider tests, have a look at the testBrowserProvider.java.in.
After this is called, ''mProvider'' will point to an isolated instance of your content provider and you can make insert/query/update/delete calls on it as expected. For more details and sample code on how to implement Content Provider tests, see testBrowserProvider.java.in.


= APIs  =
== APIs  ==
Robotium itself provides a rich API through the Solo class - javadocs for Solo are available at [1].
Robotium itself provides a rich API through the Solo class - [http://robotium.googlecode.com/svn/doc/index.html javadocs] [1] for Solo are available.


"Robocop" provides an additional API to make common tasks easier. The main interfaces are Actions, Elements, and Driver.  
"Robocop" provides an additional API to make common tasks easier. The main interfaces are Actions, Elements, and Driver.  
Line 104: Line 116:
   int getGeckoWidth();
   int getGeckoWidth();
</pre>
</pre>
See <objdir>/mobile/android/base/R.java to find ids that can be used with Driver.findElement()


Finally, an evolving set of test base classes - BaseTest, PixelTest, etc - can be leveraged for some types of tests.
Finally, an evolving set of test base classes - BaseTest, PixelTest, etc - can be leveraged for some types of tests.


= Usable IDs =
== Tips ==
See <objdir>/mobile/android/base/R.java to find ids that can be used with Driver.findElement().
 
= Tips =
=== Event timing ===
=== Event timing ===
A recurring issue when writing UI tests is the timing of events. To enter a URL, you need to click on the awesome bar and then send the key events for the text. If you click() and then immediately sendKeys(), the text probably won't get to the awesome bar. If you sleep() briefly before and after clicking, the task will probably succeed...but how long are those sleep() calls? Will the test still work on other devices, reliably? Avoid the temptation to scatter sleep() throughout your test. Whenever possible, wait for events or other feedback to verify the UI is in the required state before proceeding. For example, many tests will want to wait for startup before starting the test: driver.waitForGeckoEvent("Gecko:Ready").
A recurring issue when writing UI tests is the timing of events. To enter a URL, you need to click on the awesome bar and then send the key events for the text. If you click() and then immediately sendKeys(), the text probably won't get to the awesome bar. If you sleep() briefly before and after clicking, the task will probably succeed...but how long are those sleep() calls? Will the test still work on other devices, reliably? Avoid the temptation to scatter sleep() throughout your test. Whenever possible, wait for events or other feedback to verify the UI is in the required state before proceeding. For example, many tests will want to wait for startup before starting the test: driver.waitForGeckoEvent("Gecko:Ready").
=== Activities ===
=== Use waitForCondition instead of sleep() ===
Most tests will access UI views using FennecNativeDriver.findElement, which takes an activity as an argument. Some tests will only interact with one activity, but others may need to use multiple activities. For instance, when there are multiple tabs, there may be multiple awesome bar activities; use the wrong activity and you'll be accessing an unexpected element / view! See testNewTab for an example with multiple activities.
If you cannot find an event to wait for and you are still tempted to sleep(), consider using waitForCondition() instead. waitForConditon() gives you a simple way to poll for a condition for a specified maximum amount of time. This provides the advantage that the test can proceed as soon as the condition is met, reducing test time.


=== Logging ===
=== Logging ===
Robocop logs to both logcat and a file log via Assert.dumpLog (or FennecNativeDriver.log). Test code can use these, or log as a side-effect of Assert.is, Assert.ok, etc. Do not call android.util.Log.i/w/e/etc directly.
Robocop logs to both logcat and a file log via Assert.dumpLog (or FennecNativeDriver.log). Test code can use these, or log as a side-effect of Assert.is, Assert.ok, etc. Do not call android.util.Log.i/w/e/etc directly.
'''Robocat''' is a command line tool to transform the JSON robocop dumps into logcat into a human-readable format: https://github.com/pocmo/Robocat


=== Indirect Database Operations ===
=== Indirect Database Operations ===
Line 156: Line 169:
=== Be careful when relying on scrolling in waitForText ===
=== Be careful when relying on scrolling in waitForText ===
Solo.waitForText (and some other Robotium functions) will conveniently scroll down in a list to find text that is not displayed. But it will not scroll up again! Consider a test like this:
Solo.waitForText (and some other Robotium functions) will conveniently scroll down in a list to find text that is not displayed. But it will not scroll up again! Consider a test like this:
<open a menu or display a list>
mAsserter.ok(waitForText("Item 1"), ...
mAsserter.ok(waitForText("Item 2"), ...
If both Item 1 and Item 2 are visible, this test passes.
If Item 1 occurs first in the list, with Item 2 later, and Item 2 is off the bottom of the screen, then the first waitForText succeeds immediately, the second waitForText scrolls down, then succeeds, and the test passes.
If Item 2 occurs first in the list, with Item 1 later, and Item 1 is off the bottom of the screen, then the first waitForText scrolls down to find Item 1. If that operation scrolls Item 2 off the top of the screen, then the second waitForText will fail!


= More Information =
  <open a menu or display a list>
  mAsserter.ok(waitForText("Item 1"), ...
  mAsserter.ok(waitForText("Item 2"), ...
 
* If both Item 1 and Item 2 are visible, this test passes.
* If Item 1 occurs first in the list, with Item 2 later, and Item 2 is off the bottom of the screen, then the first waitForText succeeds immediately, the second waitForText scrolls down, then succeeds, and the test passes.
* If Item 2 occurs first in the list, with Item 1 later, and Item 1 is off the bottom of the screen, then the first waitForText scrolls down to find Item 1. If that operation scrolls Item 2 off the top of the screen, then the second waitForText will fail!


[1] http://code.google.com/p/robotium/
Note that the test is therefore dependent on screen size and orientation, as well as Robotium scrolling behavior.


[2] https://github.com/jayway/robotium/tree/master/robotium-solo/src/main/java/com/jayway/android/robotium/solo
== Useful links ==
#[[Mobile/Fennec/Android/UITest|Our UITest base class documentation]]
#Latest Robotium javadocs: http://robotium.googlecode.com/svn/doc/index.html
#Robotium project page: http://code.google.com/p/robotium/
#Robotium changelog for versions: http://code.google.com/p/robotium/wiki/Changelog
#Robotium GitHub repo: https://github.com/jayway/robotium/tree/master/robotium-solo/src/main/java/com/jayway/android/robotium/solo
Skaspari
118

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/651978...1120946"