76
edits
QA/Automation/Projects/Mozmill Automation/On Demand Test Framework/CI Documentation: Difference between revisions
Jump to navigation
Jump to search
QA/Automation/Projects/Mozmill Automation/On Demand Test Framework/CI Documentation (view source)
Revision as of 19:06, 17 January 2012
, 17 January 2012FIXED SPELLING ERRORS
No edit summary |
(FIXED SPELLING ERRORS) |
||
| Line 9: | Line 9: | ||
* Create core jobs on itself via Jenkins API | * Create core jobs on itself via Jenkins API | ||
** Grabs from davehunt's | ** Grabs from davehunt's Github repo | ||
** Uses python-jenkins to update the jobs on itself | ** Uses python-jenkins to update the jobs on itself | ||
** Will leave old jobs behind if a platform is removed from requirements | ** Will leave old jobs behind if a platform is removed from requirements | ||
| Line 21: | Line 21: | ||
** Copy Artifact plugin | ** Copy Artifact plugin | ||
* Run the test jobs ( | * Run the test jobs (Mozmill scripts) | ||
* Mozmill scripts push to brasstacks (slaves need network visibility to results repo) | * Mozmill scripts push to brasstacks (slaves need network visibility to results repo) | ||
* Currently one build executor per node because no simultaneous | * Currently one build executor per node because no simultaneous Mozmill. New focus monitor changes gives us a shot at parallelism | ||
=== Flow === | === Flow === | ||
| Line 37: | Line 37: | ||
# We kick off on-demand job on master w/ configuration file | # We kick off on-demand job on master w/ configuration file | ||
# Configuration job used to create a new specific job w/ UUID in name | # Configuration job used to create a new specific job w/ UUID in name | ||
# New specific job has build triggers in it for each configuration to trigger | # New specific job has build triggers in it for each configuration to trigger parametrized core jobs. Immediately finishes after kicking all these off. | ||
# Core jobs added to build queue and processed as nodes come available | # Core jobs added to build queue and processed as nodes come available | ||
# Core jobs have a dependencies on environments. Use artifacts as built above. | # Core jobs have a dependencies on environments. Use artifacts as built above. | ||
# Core jobs each translate to one run of the | # Core jobs each translate to one run of the Mozmill scripts. | ||
=== Environment Targets === | === Environment Targets === | ||
| Line 68: | Line 68: | ||
#* Green Balls | #* Green Balls | ||
#* Xvnc (for Linux slaves) | #* Xvnc (for Linux slaves) | ||
#* | #* Parametrized Trigger | ||
#* GitHub | #* GitHub | ||
#* Git | #* Git | ||
| Line 124: | Line 124: | ||
* Create a new (free-style) project named trigger-ondemand with the following configuration: | * Create a new (free-style) project named trigger-ondemand with the following configuration: | ||
** GitHub project: https://github.com/davehunt/mozmill-jenkins-config/ | ** GitHub project: https://github.com/davehunt/mozmill-jenkins-config/ | ||
** Check 'This build is | ** Check 'This build is parametrized' | ||
** Add a file parameter | ** Add a file parameter | ||
** File location: config.ini | ** File location: config.ini | ||
| Line 149: | Line 149: | ||
**# Clear the workspace. | **# Clear the workspace. | ||
**# Copy the latest artifact from the appropriate mozmill-environment job. | **# Copy the latest artifact from the appropriate mozmill-environment job. | ||
**# Unzip and configure the | **# Unzip and configure the Mozmill environment for use. | ||
**# Clone mozmill-automation repository. | **# Clone mozmill-automation repository. | ||
**# Run the download.py script to get the appropriate Firefox binary. | **# Run the download.py script to get the appropriate Firefox binary. | ||
| Line 165: | Line 165: | ||
** JUnit compatible report (patch in review) is generated and presented by Bamboo. | ** JUnit compatible report (patch in review) is generated and presented by Bamboo. | ||
** Test reports are sent to http://mozmill-archive.brasstacks.mozilla.com/#/addons. | ** Test reports are sent to http://mozmill-archive.brasstacks.mozilla.com/#/addons. | ||
** Log is generated and archived as a build | ** Log is generated and archived as a build artifact. | ||
* Prerequisites: The job is run on a Bamboo agent that has Python, Mercurial, and Mozmill preinstalled. Any updates to these would have to be taken care of manually. It would be possible to install Mozmill as part of the job, however Mercurial is an absolute requirement as it's needed to check out mozmill-tests and mozmill-automation from hg.mozilla.org | * Prerequisites: The job is run on a Bamboo agent that has Python, Mercurial, and Mozmill preinstalled. Any updates to these would have to be taken care of manually. It would be possible to install Mozmill as part of the job, however Mercurial is an absolute requirement as it's needed to check out mozmill-tests and mozmill-automation from hg.mozilla.org | ||
* Customizations: As we are running against a snapshot version of Selenium IDE, it is necessary to overwrite the addons.ini file with the location of the snapshot XPI file. This requires first checking out mozmill-tests from hg.mozilla.org, creating the replacement file, | * Customizations: As we are running against a snapshot version of Selenium IDE, it is necessary to overwrite the addons.ini file with the location of the snapshot XPI file. This requires first checking out mozmill-tests from hg.mozilla.org, creating the replacement file, committing the change to the local repository, and then specifying the location of the local repository when running the mozmill-automation script. | ||
* Notifications: | * Notifications: | ||
** An e-mail report is sent to Dave Hunt. | ** An e-mail report is sent to Dave Hunt. | ||
** selbot2 in #selenium on irc.freenode.net reports build results via RSS feed. | ** selbot2 in #selenium on irc.freenode.net reports build results via RSS feed. | ||
=== Mozmill === | === Mozmill === | ||
* CI server: Jenkins (running in a local VM at this time) | * CI server: Jenkins (running in a local VM at this time) | ||
| Line 191: | Line 192: | ||
** JUnit compatible report (patch in review) is generated and presented by Jenkins. | ** JUnit compatible report (patch in review) is generated and presented by Jenkins. | ||
** Test reports are sent to http://mozmill-archive.brasstacks.mozilla.com/#/endurance. | ** Test reports are sent to http://mozmill-archive.brasstacks.mozilla.com/#/endurance. | ||
** Log is generated and archived as a build | ** Log is generated and archived as a build artifact. | ||
* Notifications: An e-mail report is sent to Dave Hunt if this job fails. | * Notifications: An e-mail report is sent to Dave Hunt if this job fails. | ||
| Line 213: | Line 214: | ||
== Questions/Proposals == | == Questions/Proposals == | ||
# A plugin like the following could help us to prepare the workspace for testruns across slaves of the same platform. Downloads will not have to happen in parallel by the slaves but will all be done by the "master" and handed over to the slaves | # A plugin like the following could help us to prepare the workspace for testruns across slaves of the same platform. Downloads will not have to happen in parallel by the slaves but will all be done by the "master" and handed over to the slaves | ||
# Do we want to place the master's Jenkins folder under the home directory of the user? Therefore we should work with the .war file instead of installing a native | # Do we want to place the master's Jenkins folder under the home directory of the user? Therefore we should work with the .war file instead of installing a native Jenkins build onto the system? | ||
# Do we have a fallback strategy if something is broken? Could we still trigger/execute tests manually for a given platform. | # Do we have a fallback strategy if something is broken? Could we still trigger/execute tests manually for a given platform. | ||
# For environment targets: Unzipping vs downloading without packing. Which is more complicated? | # For environment targets: Unzipping vs downloading without packing. Which is more complicated? | ||