QA/Execution/Web Testing/Automation/Jenkins: Difference between revisions
< QA | Execution | Web Testing | Automation
Jump to navigation
Jump to search
| Line 33: | Line 33: | ||
Several environment variables are configured: | Several environment variables are configured: | ||
* ''' | * '''APPIUM_BROWSER_NAME''' - Browser name to use with Appium. The default is '''Browser'''. | ||
* ''' | * '''APPIUM_DEVICE_NAME''' - Device name to use with Appium. The default is '''Android Emulator'''. | ||
* '''APPIUM_PLATFORM_NAME''' - Platform name to use with Appium. The default is '''Android'''. | |||
* '''APPIUM_PLATFORM_VERSION''' - Platform version to use with Appium. The default is '''5.1'''. | |||
* '''APPIUM_VERSION''' - Version of Appium to use. The default is '''1.5.3'''. | |||
* '''PIP_INDEX_URL''' - Sets the PyPI index URL. The default is empty, to use the main PyPI index. | * '''PIP_INDEX_URL''' - Sets the PyPI index URL. The default is empty, to use the main PyPI index. | ||
* '''PIP_USE_MIRRORS''' - Controls use of the PyPI mirrors. By default this is '''false''' to speed up PIP installations. | * '''PIP_USE_MIRRORS''' - Controls use of the PyPI mirrors. By default this is '''false''' to speed up PIP installations. | ||
* '''PIP_VERBOSE''' - Controls the verbosity of PIP output. By default this is '''false''' and useful to enable for debugging. | * '''PIP_VERBOSE''' - Controls the verbosity of PIP output. By default this is '''false''' and useful to enable for debugging. | ||
* '''PYTEST_ADDOPTS''' - Add py.test options to all jobs. The default is '''-r=fsxXR --verbose --tb=short'''. | * '''PYTEST_ADDOPTS''' - Add py.test options to all jobs. The default is '''-r=fsxXR --verbose --tb=short'''. | ||
* '''PYTEST_PROCESSES_SAUCE''' - Number of parallel py.test processes to use on [http://saucelabs.com/ SauceLabs] jobs. | * '''PYTEST_PROCESSES_SAUCE''' - Number of parallel py.test processes to use on [http://saucelabs.com/ SauceLabs] jobs. | ||
* '''PYTEST_RERUNS''' - Number of times to rerun a failed test (used by [https://pypi.python.org/pypi/pytest-rerunfailures pytest-rerunfailures]). The default is '''5'''. | * '''PYTEST_RERUNS''' - Number of times to rerun a failed test (used by [https://pypi.python.org/pypi/pytest-rerunfailures pytest-rerunfailures]). The default is '''5'''. | ||
* '''SAUCE_FIREFOX_VERSION_MAC''' - The default Firefox version to use for OS X [http://saucelabs.com/ SauceLabs] jobs. | * '''SAUCE_FIREFOX_VERSION_MAC''' - The default Firefox version to use for OS X [http://saucelabs.com/ SauceLabs] jobs. | ||
* '''SAUCE_FIREFOX_VERSION_WINDOWS''' - The default Firefox version to use for Windows [http://saucelabs.com/ SauceLabs] jobs. | * '''SAUCE_FIREFOX_VERSION_WINDOWS''' - The default Firefox version to use for Windows [http://saucelabs.com/ SauceLabs] jobs. | ||
* '''SAUCE_MAC_VERSION''' - The default OS X version for [http://saucelabs.com/ SauceLabs] jobs. | * '''SAUCE_MAC_VERSION''' - The default OS X version for [http://saucelabs.com/ SauceLabs] jobs. | ||
* '''SAUCE_WINDOWS_VERSION''' - The default Windows version for [http://saucelabs.com/ SauceLabs] jobs. | * '''SAUCE_WINDOWS_VERSION''' - The default Windows version for [http://saucelabs.com/ SauceLabs] jobs. | ||
* '''SAUCELABS_SELENIUM_VERSION''' - The default Selenium version for [http://saucelabs.com/ SauceLabs] jobs. | * '''SAUCELABS_SELENIUM_VERSION''' - The default Selenium version for [http://saucelabs.com/ SauceLabs] jobs. | ||
* '''SAUCELABS_USERNAME''' - The default username to use for [http://saucelabs.com/ SauceLabs] jobs. | * '''SAUCELABS_USERNAME''' - The default username to use for [http://saucelabs.com/ SauceLabs] jobs. | ||
* ''' | * '''SELENIUM_EXCLUDE_DEBUG''' - Exclude capturing these debug types from Selenium tests. The default is '''logs'''. | ||
* '''TOX_TESTENV_PASSENV''' - Environment variables to pass down into Tox environments. The default is '''http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY | * '''TOX_TESTENV_PASSENV''' - Environment variables to pass down into Tox environments. The default is '''http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY SAUCELABS_USERNAME SAUCELABS_API_KEY'''. | ||
= Known Issues = | = Known Issues = | ||
Revision as of 12:59, 6 October 2016
Summary
Jenkins CI is used by Web QA to run automated tests. The production instance is publicly accessible and can be found here. Details of the setup and administration are available on our internal Mana page. We also have a staging instance for adhoc jobs and experimentation. This is secured behind Mozilla VPN, and can be found here. Documentation for the staging instance is also available on Mana.
Plugins
Below is a list of plugins in use (dependencies have been omitted):
- Build Failure Analyzer - Helps collate build failures
- Build Monitor - Highly visible view of the status of selected jobs
- Build Timeout - Allows users to take responsibility for failed builds
- Claim - Ability to assign build failures
- Config File Provider - Centrally maintained configuration files for jobs
- Dashboard View - Dashboard view for jobs
- EnvInject - Set environment variables for jobs
- GitHub - Integrates GitHub
- Green Balls - Green is better than blue!
- HTML Publisher - Publishes HTML reports
- IRC - IRC bot for triggering builds and notifications
- Job Configuration History - Provides history of configuration changes
- LDAP - User authentication
- Managed Script - Centrally maintained scripts for jobs
- Monitoring - Monitoring of Jenkins itself
- Ownership - Show job owners and co-owners
- Parameterized Trigger - Trigger builds with parameters
- Rebuild - Rebuild a parametrized build without re-entering the parameters
- Role Strategy - Role-based strategy for user management
- Testdroid Marionette - Provisions a Firefox OS device via Testdroid
- Test Stability - Historical information about the stability of tests
- Timestamper - Adds timestamps to the console output
- URLTrigger - Trigger builds based on the response of a URL
- View Job Filters - Advanced filters for views
- Workspace Cleanup - Deletes the workspace before or after the build
Global Environment Variables
Several environment variables are configured:
- APPIUM_BROWSER_NAME - Browser name to use with Appium. The default is Browser.
- APPIUM_DEVICE_NAME - Device name to use with Appium. The default is Android Emulator.
- APPIUM_PLATFORM_NAME - Platform name to use with Appium. The default is Android.
- APPIUM_PLATFORM_VERSION - Platform version to use with Appium. The default is 5.1.
- APPIUM_VERSION - Version of Appium to use. The default is 1.5.3.
- PIP_INDEX_URL - Sets the PyPI index URL. The default is empty, to use the main PyPI index.
- PIP_USE_MIRRORS - Controls use of the PyPI mirrors. By default this is false to speed up PIP installations.
- PIP_VERBOSE - Controls the verbosity of PIP output. By default this is false and useful to enable for debugging.
- PYTEST_ADDOPTS - Add py.test options to all jobs. The default is -r=fsxXR --verbose --tb=short.
- PYTEST_PROCESSES_SAUCE - Number of parallel py.test processes to use on SauceLabs jobs.
- PYTEST_RERUNS - Number of times to rerun a failed test (used by pytest-rerunfailures). The default is 5.
- SAUCE_FIREFOX_VERSION_MAC - The default Firefox version to use for OS X SauceLabs jobs.
- SAUCE_FIREFOX_VERSION_WINDOWS - The default Firefox version to use for Windows SauceLabs jobs.
- SAUCE_MAC_VERSION - The default OS X version for SauceLabs jobs.
- SAUCE_WINDOWS_VERSION - The default Windows version for SauceLabs jobs.
- SAUCELABS_SELENIUM_VERSION - The default Selenium version for SauceLabs jobs.
- SAUCELABS_USERNAME - The default username to use for SauceLabs jobs.
- SELENIUM_EXCLUDE_DEBUG - Exclude capturing these debug types from Selenium tests. The default is logs.
- TOX_TESTENV_PASSENV - Environment variables to pass down into Tox environments. The default is http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY SAUCELABS_USERNAME SAUCELABS_API_KEY.
Known Issues
- None
Troubleshooting
Try enabling verbose PIP output to determine if installation of Python packages is causing issues.
We use the main PyPI (Python Package Index) for installing Python packages. If this experiences issues then we can enable use of PIP mirrors.
PIP installation is slow
If Python packages are slow to install you should check that PIP mirrors are not enabled. If not, it may be that the main PyPI mirror is slow to respond.