B2G/QA/Automation/Style Guide/Python Script Style: Difference between revisions

From MozillaWiki
< B2G‎ | QA‎ | Automation‎ | Style Guide
Jump to navigation Jump to search
(Moved away the external consumer parts and fixed nits)
 
(25 intermediate revisions by 2 users not shown)
Line 70: Line 70:
# Bad
# Bad
class test_this_site:
class test_this_site:
</source>
== Locators ==
* Locator variables should be prefixed with <code>_</code> to show that it is [http://docs.python.org/tutorial/classes.html#private-variables private].
* Variables should be descriptive of the area and not clash with any properties.
* Should have a suffix of <code>_locator</code>.
* Accessing locators should be done through a property or method as this keeps the locator as read-only.
<source lang="python">
@property
def search_term(self):
    return self.marionette.find_element(*self._search_box_locator).value
</source>
* We should use locators in the following order of preference (there will be exceptions):
** ID
** Name
** Class name
** CSS selector
** XPath
* CSS locators should use whitespace for readability when using direct descendants.
<source lang="python">
# Good
_my_locator = "css=#content > p > a"
   
# Bad
_my_locator = "css=#content>p>a"
</source>
* Use Python tuples to define locators:
<source lang="python">
# Good
_my_locator = (By.ID, "content")
</source>
</source>


Line 85: Line 119:
* Actions should wait for the appropriate action to complete. This could be an implicit or explicit wait. For example, clicking a login button might explicitly wait for a username field to be visible.
* Actions should wait for the appropriate action to complete. This could be an implicit or explicit wait. For example, clicking a login button might explicitly wait for a username field to be visible.


== Use testvars.json <njpark> ==
== Using High-Level Methods ==
Using test variable file when running gaiatest can avoid defining variables inside the script.  The testvars template is located [http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/testvars_template.json here].
* A test script should remain a short entry point which leads to more details (like the main() function of a program). Don't detail every single tap and wait, in the test. Use methods that reflect a user's intent rather than a sequence of small steps the user has to do. The details of how to fulfill that intent will be in the page classes.
<br>
Make sure to fill in the appropriate section if you're planning to use it, and supply the name and location of the .json file as the parameter to the '''gaiatest''' command with '''--testvars=''' option.
<br><br>
If you want to access the varable value defined in the .json file, you can do as the following example:
<br>
<source lang="python">
<source lang="python">
test_phone_number = self.testvars['remote_phone_number']
# Good
messages.send_an_sms_to_yourself()
 
# Bad
messages.open_sms_app()
messages.create_new_sms(receiver)
messages.type_content(content)
messages.send()
</source>
</source>
<br>
 
If you need to access the sub-variable, consider below example as well:
== Variable Naming ==
* Name your variables with units.
<source lang="python">
<source lang="python">
self.testvars['plivo']['auth_id'],
# Good
self.testvars['plivo']['auth_token'],
timeout_in_seconds, width_in_pixels
self.testvars['plivo']['phone_number']
 
# Bad
timeout, width
</source>
</source>
* Don't shorten variable names.
<source lang="python">
# Good
self.time_out_error_time = 1000


== Using high-level methods ==
# Bad
TBD - jlorenzo
self.t_out_err_tim = 1000
* In a test, make sure to only have steps that are high level (for instance: messages.send_an_sms_to_yourself() instead of detailing every single tap and click)
== Avoid sleep() calls <njpark> ==
Sleep methods can be used as follows:
<source python>
import time
time.sleep(seconds)
</source>
</source>
<br>
However, sleep() calls should be used very sparingly, only when there is no other way to delay the action of Marionette.  Using sleep() when Wait() can accomplish the same thing would be bad for following reasons:
* Sleep() does not care about the UI changes in app.  If you're using sleep() to just 'wait enough', you'll run into problems when the app behavior changes and requires more/less time to wait.
* Sleep() does not care about the phone performance.  If the speed of the execution changes because of the changes in memory allocation or running on a newer/older devices, it will still wait for specified time.
<br>
When you have to use the sleep() call, make sure to put in the comment explaining why other methods won't work.


== Variable naming ==
== Making Tests Locale Independent ==  
TBD - jlorenzo
It is recommended to make the test scripts (and especially the helper methods) locale independent, because this enables us to test devices in RTL locale. There are still some sections in gaiatest where it checks for the displayed English text, but unless one cannot avoid it, data-l10n-id attribute should be checked in place of any raw text comparison.
* Name your variables with units. For example: timeout_in_seconds, width_in_pixels
 
* Don't shorten variable names. A bad example: self.t_out_err_tim = 1000 instead of self.time_out_error_time
<source lang="python">
== app.py and regions/helper.py ==
       
TBD - njpark
        # below method will not work in a non-English locale,
== How to create filename strings ==
        # should only be used when there is no appropriate tag for the Open App button
TBD - njpark
        button = self.root_element.find_element(*self._install_button_locator)
http://mxr.mozilla.org/gaia/source/tests/python/gaia-ui-tests/gaiatest/apps/settings/app.py#296
        Wait(self.marionette).until(lambda m: button.text == 'Open app')
== Meaningful custom Assert() messages ==
        button.tap()
TBD - njpark
 
https://bugzilla.mozilla.org/show_bug.cgi?id=1198449#c2
        # below code will tap 'Send Mozilla Feedback' button in any locale
== Clean up afterwards ==
        _send_feedback_locator = (By.CSS_SELECTOR, '[data-l10n-id="sendMozillaFeedback"]')
TBD - njpark
        element = Wait(self.marionette).until(
== Avoid code duplication ==
            expected.element_present(*self._send_feedback_locator))
TBD - jlorenzo
        Wait(self.marionette).until(expected.element_displayed(element))
* Use parameterized()
        element.tap()
* Centralize workarounds
</source> 
== Simulate end-user check ==
TBD - jlorenzo
* Check what an end-user would check.
* Make sure manifest.ini is updated with right flags


== Making test multi-locale ==
The rules of thumb are:
TBD - njpark
* We can check for the raw text in order to verify the user input (i.e., a phone number entered in the dialer app)
* we check the raw text in order to verify the input from the user (for instance a phone number put in the dialer, we verify it appears in the call log)
* We check the data-l10n-id for any string that comes from Gaia only (i.e., an error message)
* we test the l10n-id for any string that comes from Gaia only (like an error message)
== Be aware of outside consumers of ui-test ==
TBD - mwargers   
* Be aware if you change the Gaia UI test API, that outside consumers (mtbf, etc) might get broken

Latest revision as of 16:37, 10 November 2015

Python Script Style Guide

Naming

  • Module names should be called test_ and then behavioral areas.
   test_search.py
  • Test method names should always show the intent of the test case.
# Good
def test_that_advanced_search_does_not_find_item(self):

# Bad
def test_advanced_search(self):

File Headers

  • Each file should have a completed copy of the MPL2 license block, immediately followed by an empty line.
  • Each file should pass PEP8 except for line length, see below.
# Good
def method(self, parameter) 

# Bad
def method(self,parameter)
  • Lines should try not to have more than 100 characters.
  • Docstrings should conform to PEP0257 and should be on a single line wherever possible.
# Good
def click_login():
"""Clicks the login link."""

# Bad
def click_login():
"""
Clicks the login link.
"""

Where not possible, the first line should be a summary.

# Good
def login():
"""Logs in.

Clicks the login link and then waits for the home page to load.

"""

# Bad
def login():
"""Logs in.
Clicks the login link and then waits for the home page to load."""
  • Indenting should be a soft tab (4 spaces) as common with in Python. Do not mix tabs and spaces!
  • There should be no whitespace at the end of the file (as per PEP8).
  • Comments should be on the line above. Remember to update comments when changing code so that code matches the comments.
  • Comments should be used carefully. Try not to put comments that don't provide any additional value
  • Class names should be in Pascal style as this is Python idiomatic.
# Good
class TestThisSite:
    
# Bad
class test_this_site:

Locators

  • Locator variables should be prefixed with _ to show that it is private.
  • Variables should be descriptive of the area and not clash with any properties.
  • Should have a suffix of _locator.
  • Accessing locators should be done through a property or method as this keeps the locator as read-only.
@property
def search_term(self):
    return self.marionette.find_element(*self._search_box_locator).value
  • We should use locators in the following order of preference (there will be exceptions):
    • ID
    • Name
    • Class name
    • CSS selector
    • XPath
  • CSS locators should use whitespace for readability when using direct descendants.
# Good
_my_locator = "css=#content > p > a"
    
# Bad
_my_locator = "css=#content>p>a"
  • Use Python tuples to define locators:
# Good
_my_locator = (By.ID, "content")

Actions

  • Methods that perform actions on the page should indicate the action in the method name.
# Good
def click_report_with_length(length)

# Bad
def report_length(length)
  • Actions should wait for the appropriate action to complete. This could be an implicit or explicit wait. For example, clicking a login button might explicitly wait for a username field to be visible.

Using High-Level Methods

  • A test script should remain a short entry point which leads to more details (like the main() function of a program). Don't detail every single tap and wait, in the test. Use methods that reflect a user's intent rather than a sequence of small steps the user has to do. The details of how to fulfill that intent will be in the page classes.
# Good
messages.send_an_sms_to_yourself()

# Bad
messages.open_sms_app()
messages.create_new_sms(receiver)
messages.type_content(content)
messages.send()

Variable Naming

  • Name your variables with units.
# Good 
timeout_in_seconds, width_in_pixels

# Bad
timeout, width
  • Don't shorten variable names.
# Good
self.time_out_error_time = 1000

# Bad
self.t_out_err_tim = 1000

Making Tests Locale Independent

It is recommended to make the test scripts (and especially the helper methods) locale independent, because this enables us to test devices in RTL locale. There are still some sections in gaiatest where it checks for the displayed English text, but unless one cannot avoid it, data-l10n-id attribute should be checked in place of any raw text comparison.

        
        # below method will not work in a non-English locale, 
        # should only be used when there is no appropriate tag for the Open App button
        button = self.root_element.find_element(*self._install_button_locator)
        Wait(self.marionette).until(lambda m: button.text == 'Open app')
        button.tap()

        # below code will tap 'Send Mozilla Feedback' button in any locale
        _send_feedback_locator = (By.CSS_SELECTOR, '[data-l10n-id="sendMozillaFeedback"]')
        element = Wait(self.marionette).until(
            expected.element_present(*self._send_feedback_locator))
        Wait(self.marionette).until(expected.element_displayed(element))
        element.tap()

The rules of thumb are:

  • We can check for the raw text in order to verify the user input (i.e., a phone number entered in the dialer app)
  • We check the data-l10n-id for any string that comes from Gaia only (i.e., an error message)