TestEngineering/Web/Automation: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
m (Remove link to Selenium Grid documentation.)
 
(40 intermediate revisions by 6 users not shown)
Line 1: Line 1:
= Automation Homepage  =
= Automation Homepage  =


The WebQA Automation team builds automated tests to run against Mozilla websites to make sure they are working as expected.
The Web QA Automation team builds automated tests to run against Mozilla websites to make sure they are working as expected. Mozilla's team is excited to see your interest in our browser-based, automated checks.  Thanks for stopping by! :)


== Running to-do list for Grid/Hudson ==
This page describes our Web QA projects which use browser-based automation as well as everything you need to set up your own environment for running and contributing to our checks. This will get you started running Web QA automated tests. (If you would like to help out with Mozmill automation, be sure to check out the [http://quality.mozilla.org/teams/automation/ Mozmill wiki page].)
* https://wiki.mozilla.org/QA/Execution/Web_Testing/Automation/ToDo


== Selenium ==
== Getting started with Automated Testing ==
* [[QA/Execution/Web_Testing/Automation/Selenium_Grid|Selenium Grid]] (Hub and RCs)
 
* [https://code.google.com/p/selenium/wiki/PageObjects Page Object Pattern] (from the Selenium page on Google)
Our automated tests use [http://seleniumhq.org/ Selenium] and [http://www.python.org/ Python]. If you are familiar with either of them we can use your help, and helping us is a great way to learn more. The steps to get started with automation include:
* [[QA/Execution/Web_Testing/Docs/Automation/StyleGuide|Python Style Guide (+Page Object Model)]]
 
* [[QA/Execution/Web_Testing/Automation/Credentials|Credentials]]
=== Choose a Project ===
* [[Web_Testing/Automation/CodeReviewProcess|Code Review Process]]
All of our automation efforts are organized based on the sites that are being tested, with each site having its own project. Your first step, therefore, is to choose a project to work on. You can [https://wiki.mozilla.org/QA/Execution/Web_Testing/Automation#Web_QA_Projects explore our projects] to find one that interests you, but if you're just looking for a simple place to start we suggest [https://github.com/mozilla/mcom-tests/ mcom-tests].
* [[QA/Execution/Web_Testing/Automation/Screen_Tutorial|Screen Tutorial]]
 
* [[QA/Execution/Web_Testing/Automation/github|Github Usage]]
=== Run Some Tests ===
* [https://gist.github.com/926431 Marlena's GitHub cheat sheet]
Your next step should be to grab a copy of the tests for your chosen project and try to run them. You can find all the details for doing this in our [https://developer.mozilla.org/en-US/docs/Mozilla/QA/Running_Web_QA_automated_tests Running Web QA Tests guide].
* [[Web_Testing/Automation/UnittestZero|UnittestZero]]
 
=== Write Some Tests ===
Each project tracks its automation needs using Git issues. Here is a dashboard of [http://mozilla.github.io/mozwebqa-dashboard/#/issues all current Web QA Issues] listed that are open and ready to take.  Assign a Git issue to yourself by leaving a comment on that issue that you want to work on it.
 
== Security Testing ==
 
If you are interested in helping with security testing, check out the [https://wiki.mozilla.org/WebAppSec/Secure_Coding_QA_Checklist security testing page] on our wiki.
 
== More Information ==
* If you would like to write some automated checks for us, you will need to be familiar with version control through [https://github.com/ GitHub], programming in [http://www.python.org/ Python] and writing automated checks in [http://seleniumhq.org/ Selenium]. We recognize that volunteering is always a learning opportunity, so here are a few of our favorite resources if you need to refresh what you know or learn something new before contributing:
 
** [http://www.diveintopython.net/ Python: Dive into Python]
** [http://progit.org/book/ GitHub: ProGit]
** [http://code.google.com/p/selenium/wiki/PageObjects A brief introduction to POM from the Selenium site.]
** [http://seleniumhq.org/docs/ SeleniumHQ Documentation]
** [http://www.developsense.com/blog/2009/08/testing-vs-checking/ The difference between tests and checks]


=== Architecture ===
* As you begin writing checks, please adhere as closely as possible to the [https://wiki.mozilla.org/QA/Execution/Web_Testing/Docs/Automation/StyleGuide Style Guide], so we’re all on the same page, and reviews go more smoothly. Once you’ve written your tests and submitted a GitHub pull request, they will go through our [https://wiki.mozilla.org/Web_Testing/Automation/CodeReviewProcess Code Review process].
* File bugs to help us improve the product (Learn how to file bugs [https://developer.mozilla.org/en/bug_writing_guidelines here]).
* Each project's GitHub repository has a README.md that will you help you set up an environment and get tests running for that particular project.
Our tests use the page object model:
* If you are interested in the pre-POM Python checks we have pages that discuss [https://wiki.mozilla.org/QA/Execution/Web_Testing/Docs/Automation/Testcases/SharedLibraries the libraries] and [https://wiki.mozilla.org/QA/Execution/Web_Testing/Docs/Automation/Testcases/ConfigFiles configuration files] shared by the checks.
   


*'''qa-selenium'''.mv.mozilla.com (10.250.1.143)
== Web QA Projects ==
**Mac Mini; 2.66 Ghz Intel Core 2 Duo; 8 GB
**Mac OS X 10.6.8
**Jenkins (http://qa-selenium.mv.mozilla.com:8080/)
**Selenium Grid Hub (http://qa-selenium.mv.mozilla.com:4444/grid/console)
**VM: Windows 7 Professional, Internet Explorer 8
*'''qa-selenium2'''.mv.mozilla.com (10.250.1.167)
**Mac Mini; 1.66 Ghz Intel Core Duo; 2 GB
**Mac OS X 10.5.8
**VM: Windows 7 Professional, Internet Explorer 8
*'''qa-selenium3'''.mv.mozilla.com (10.250.1.196)
**Mac Mini; 2.4 Ghz Intel Core 2 Duo; 4 GB
**Mac OS X 10.6.8
*'''qa-selenium4'''.mv.mozilla.com (10.250.1.197)
**Mac Mini; 2.4 Ghz Intel Core 2 Duo; 4 GB
**Mac OS X 10.6.8
**VM: Windows 7 Professional, Internet Explorer 8
*'''qa-selenium5'''.mv.mozilla.com (10.250.1.222)
**Mac Mini; 2.66 Ghz Intel Core 2 Duo; 8 GB
**Mac OS X 10.6.8
*'''qa-selenium6'''.mv.mozilla.com (10.250.1.221)
**Mac Mini; 2.53 Ghz Intel Core 2 Duo; 4 GB
**Mac OS X 10.6.8


[[Image:Selenium-grid-setup.png|thumb|left|200px|Selenium-grid-setup.png]]
We are seeking volunteers to help write checks for the following projects. Each group below contains all the relevant information for that particular project.


We have several Mac Minis running various Selenium RC environments, integrated via Jenkins Continuous Build Integration and Selenium Grid. All machines can be VNC'ed into from within the MV-Office Intranet (VPN if offsite). In order to start the entire system (Jenkins, Selenium Grid, and All VMs with Selenium RCs), ensure that all the Mac Minis are powered on and logged in. <strike>Run startAll.sh on the desktop of qa-selenium.mv.mozilla.com (VPN&nbsp;required), and everything should start up. Everything can be shut down by running killAll.sh on the desktop of qa-selenium.mv.mozilla.com. This safely tears down Jenkins and Selenium grid, terminates all Terminal sessions on OS X-based computers, and sends the shutdown command to all VMs.</strike> This needs to be updated.
* Add-ons
** Homepage: [https://addons.mozilla.org addons.mozilla.org]
** Staging: [https://addons.allizom.org https://addons.allizom.org]
** GitHub Repository: [https://github.com/mozilla/Addon-Tests https://github.com/mozilla/Addon-Tests]


=== SUMO  ===
* Bouncer Tests
** Homepage: [https://download.mozilla.org download.mozilla.org]
** Staging: [https://download.allizom.org download.allizom.org]
** GitHub Repository: [https://github.com/mozilla-services/go-bouncer https://github.com/mozilla-services/go-bouncer]


=== Mozilla.com ===
* Mozillians
** Homepage: [https://mozillians.org mozillians.org]
** Staging: [https://mozillians.allizom.org/ mozillians.allizom.org/]
** GitHub Repository: https://github.com/mozilla/mozillians-tests


== Current Coverage / Plans ==
* MozTrap Tests
''Would a table/grid with checkmarks be better to represent coverage?''
** Homepage: [https://moztrap.mozilla.org moztrap.mozilla.org]
=== AMO | [http://qa-selenium.mv.mozilla.com:8080/view/AMO%20Tests/ CI]===
** Staging: [https://moztrap-dev.allizom.org moztrap-dev.allizom.org]
*Current
** GitHub Repository: [https://github.com/mozilla/moztrap-tests https://github.com/mozilla/moztrap-tests]
**Smoketest
**BFT
**FFT


=== SUMO | [http://qa-selenium.mv.mozilla.com:8080/view/SUMO%20Tests/ CI]===
* Mozwebqa Test Templates
*BFT
** Homepage: [http://www.mozilla.org http://www.mozilla.org]
** GitHub Repository: [https://github.com/mozilla/mozwebqa-test-templates https://github.com/mozilla/mozwebqa-test-templates]


=== Socorro | [http://qa-selenium.mv.mozilla.com:8080/job/socorro/ CI]===
* Snippets Tests
* Current
** Homepage: [https://snippets.stage.mozilla.com/ snippets.stage.mozilla.com]
** Smoke Tests
** GitHub Repository: https://github.com/mozilla/snippets-tests
** BFT


=== Mozilla.com / Firefox.com ===
* Soccorro
*Tests for redirects/download button (see Tanay)
** Homepage: [https://crash-stats.mozilla.com/home/products/Firefox crash-stats.mozilla.org]
*Broken mobile tests (see Jason Grlicky/Alex Buchanan)
** GitHub Repository: [https://github.com/mozilla/socorro-tests https://github.com/mozilla/socorro-tests]


=== Input | [http://qa-selenium.mv.mozilla.com:8080/job/input%20stage/ CI]===
* SUMO
* Current
** Homepage: [https://support.mozilla.org/ support.mozilla.org]
** Smoke Tests
** Staging: [https://support.allizom.org https://support.allizom.org]
** BFT
** GitHub Repository: [https://github.com/mozilla/sumo-tests https://github.com/mozilla/sumo-tests]


== Meeting Notes ==
== Documentation ==
* Meetings are held every other Friday at 19am PST/ 17:00 GMT
* [[QA/Execution/Web_Testing/Automation/Add_Project_to_Jenkins|Add Project to Jenkins]]
** +1 650 903 0800 x92 Conf# 303 (US/International)
* [[QA/Execution/Web_Testing/Automation/Jenkins|Jenkins]]
** +1 416 848 3114 x92 Conf# 303 (Canada)
* [[QA/Execution/Web_Testing/Automation/Build_Notifications|Build Notifications]]
** +1 800 707 2533 (pin 369) Conf# 303 (US Toll Free)
* [[QA/Execution/Web_Testing/Automation/Repositories|Repositories]]
** join irc.mozilla.org [irc://irc.mozilla.org:6697/#mozwebqa #mozwebqa] for backchannel discussion
* [[QA/Execution/Web_Testing/Automation/Git_Mirrors|Git Mirrors]]
* [[QA/Execution/Web_Testing/Automation/Virtual_Environments|Virtual Environments]]
* [[QA/Execution/Web_Testing/Automation/Credentials|Credentials]]
* [[QA/Execution/Web_Testing/Docs/Automation/StyleGuide|Style Guide]]
* [[QA/Execution/Web_Testing/Automation/Flake8_Pre_Commit_Hook|Flake8 Pre-Commit Hook]]
* [[Web_Testing/Automation/CodeReviewProcess|Code Review Process]]
* [[QA/Execution/Web_Testing/Automation/PIP_Requirements|PIP Requirements]]
* [[QA/Execution/Web_Testing/Automation/Running_Mobile_Tests|Running Mobile Tests]]
* [[QA/Execution/Web_Testing/Automation/Gaia_UI_Testing|Gaia UI Testing]]
* [[QA/Execution/Web_Testing/Automation/Web_QA_Procedures|Web QA Testing Procedures]]


*September
== Useful Resources ==
**[https://wiki.mozilla.org/Web_Testing/Automation/automation-30-09-2011  Meeting Notes 30-09-2011]
* [http://rogerdudler.github.com/git-guide/ git - the simple guide]
**Meeting Notes 16-09-2011 - no meeting due to All-Hands
* [[QA/Execution/Web_Testing/Automation/github|github Usage]]
**[https://wiki.mozilla.org/Web_Testing/Automation/automation-02-09-2011  Meeting Notes 02-09-2011]
* [https://gist.github.com/926431 Marlena's github cheat sheet]
*August
**[https://wiki.mozilla.org/Web_Testing/Automation/automation-19-08-2011  Meeting Notes 19-08-2011]
**[https://wiki.mozilla.org/Web_Testing/Automation/automation-05-08-2011  Meeting Notes 05-08-2011]
<br>
*July
**[[Web_Testing/Automation/automation-22-07-2011 | Meeting Notes 22-07-2011]]
**[[Web_Testing/Automation/automation-08-07-2011 | Meeting Notes 08-07-2011]]
**[[Web_Testing/Automation/automation-01-07-2011 | Meeting Notes 01-07-2011]]
<br>
*May
**[[Web_Testing/Automation/automation-13-05-2011| Meeting Notes 13-05-2011]]
<br>
*March
**[[Web_Testing/Automation/automation-31-03-2011| Meeting Notes 31-03-2011]]
**[[Web_Testing/Automation/automation-17-03-2011| Meeting Notes 17-03-2011]]
**[[Web_Testing/Automation/automation-03-03-2011| Meeting Notes 03-03-2011]]
<br>
*February, 2011
**[[Web Testing/Automation/automation-10-02-2011| Meeting Noties 10-02-2011]]
<br>
*January, 2011
**[[Web Testing/Automation/automation-27-01-2011| Meeting Noties 27-01-2011]]
**[[Web Testing/Automation/automation-13-01-2011| Meeting Noties 13-01-2011]]
<br>
*December, 2010
**[[Web Testing/Automation/automation-02-12-2010|Meeting Notes 02-12-2010]]
<br>
*November,2010
**[[Web Testing/Automation/automation-18-11-2010|Meeting Notes-18-11-2010]]
**[[Web Testing/Automation/automation-01-11-2010|Meeting Notes-01-11-2010]]
<br>
*October,2010
**[[Web Testing/Automation/automation-10-11-2010|Meeting Notes-10-11-2010]]
<br>
*September,2010
**[[Web Testing/Automation/automation-09-20-2010|Meeting Notes-09-20-2010]]
<br>
*August,2010
**[[Web Testing/Automation/automation-08-23-2010|Meeting Notes-08-23-2010]]
<br>
*July,2010
**[[Web Testing/Automation/automation-07-24-2010|Meeting Notes-07-24-2010]]
**[[Web Testing/Automation/automation-07-15-2010|Meeting Notes-07-15-2010]]
**[[Web Testing/Automation/automation-07-01-2010|Meeting Notes-07-01-2010]]


== Test Days ==
== Meeting Notes  ==
* [[Web Testing/Automation/testday-29-03-2011|Test Day - 29-03-2011]]
The Web QA no longer have a regular meeting to discuss automation. You can either attend the [[QA/Execution/Web_Testing#Meetings|Web QA weekly meeting]], or the [[Auto-tools/Automation_Development/Meetings|Automation Development meeting]]. The meeting notes from past meetings are available [[QA/Execution/Web_Testing/Automation/Archived_Meeting_Notes|here]].
* [[Web Testing/Automation/testday-24-05-2011|Test Day - 24-05-2001]]


== AMO - ('''outdated section''') ==
== AMO - ('''outdated section''') ==

Latest revision as of 15:11, 1 November 2016

Automation Homepage

The Web QA Automation team builds automated tests to run against Mozilla websites to make sure they are working as expected. Mozilla's team is excited to see your interest in our browser-based, automated checks. Thanks for stopping by! :)

This page describes our Web QA projects which use browser-based automation as well as everything you need to set up your own environment for running and contributing to our checks. This will get you started running Web QA automated tests. (If you would like to help out with Mozmill automation, be sure to check out the Mozmill wiki page.)

Getting started with Automated Testing

Our automated tests use Selenium and Python. If you are familiar with either of them we can use your help, and helping us is a great way to learn more. The steps to get started with automation include:

Choose a Project

All of our automation efforts are organized based on the sites that are being tested, with each site having its own project. Your first step, therefore, is to choose a project to work on. You can explore our projects to find one that interests you, but if you're just looking for a simple place to start we suggest mcom-tests.

Run Some Tests

Your next step should be to grab a copy of the tests for your chosen project and try to run them. You can find all the details for doing this in our Running Web QA Tests guide.

Write Some Tests

Each project tracks its automation needs using Git issues. Here is a dashboard of all current Web QA Issues listed that are open and ready to take. Assign a Git issue to yourself by leaving a comment on that issue that you want to work on it.

Security Testing

If you are interested in helping with security testing, check out the security testing page on our wiki.

More Information

  • If you would like to write some automated checks for us, you will need to be familiar with version control through GitHub, programming in Python and writing automated checks in Selenium. We recognize that volunteering is always a learning opportunity, so here are a few of our favorite resources if you need to refresh what you know or learn something new before contributing:
  • As you begin writing checks, please adhere as closely as possible to the Style Guide, so we’re all on the same page, and reviews go more smoothly. Once you’ve written your tests and submitted a GitHub pull request, they will go through our Code Review process.
  • File bugs to help us improve the product (Learn how to file bugs here).
  • Each project's GitHub repository has a README.md that will you help you set up an environment and get tests running for that particular project.

Our tests use the page object model:


Web QA Projects

We are seeking volunteers to help write checks for the following projects. Each group below contains all the relevant information for that particular project.

Documentation

Useful Resources

Meeting Notes

The Web QA no longer have a regular meeting to discuss automation. You can either attend the Web QA weekly meeting, or the Automation Development meeting. The meeting notes from past meetings are available here.

AMO - (outdated section)

All of the above are legacy, and should be used solely as reference - stephend

Retrieved from "https://wiki.allizom.org/index.php?title=TestEngineering/Web/Automation&oldid=1153215"