Web Testing/Automation/automation-10-02-2011: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with " << Last meeting | [https://wiki.mozilla.org/QA/Execution/Web_Testing/Automation WebQA Automation] [[Web_Testing/Automation/auto...")
 
No edit summary
 
(26 intermediate revisions by 5 users not shown)
Line 1: Line 1:
[[Web_Testing/Automation/automation-27-01-2011| << Last meeting]] | [https://wiki.mozilla.org/QA/Execution/Web_Testing/Automation WebQA Automation]  [[Web_Testing/Automation/automation-24-02-2011| Next meeting >>]]
[[Web_Testing/Automation/automation-27-01-2011| << Last meeting]] | [https://wiki.mozilla.org/QA/Execution/Web_Testing/Automation WebQA Automation]  [[Web_Testing/Automation/automation-03-03-2011| Next meeting >>]]
== Discussion topics ==
== Discussion topics ==
* Grid Updating
 
** Current plan: Have multiple environments with the same name running for the popular test environments (i.e. Firefox-default;en-us;MacOSX6) and let Grid distribute the jobs accordingly. There will be ~4 environments of Firefox-default;en-us;MacOSX6, and so 4 concurrent executors active on Hudson. As for the other environments, we can have as many copies of selenium running of them as we see fit.
* Someone (or someones!) needs to own Grid, which means:
* Tanay sent email about using [http://www.gnu.org/software/screen/ screen] - any objections?  I confess it'd probably take me a bit to ramp up on, but if the folks who are maintaining Grid think it'd be more useful, then I'm all for it - stephend
** Keeping browsers and the OS up-to-date (within versions)
* Someone needs to own Grid, which means:
** Keeping browsers up-to-date (within versions)
** Upgrading VMWare Fusion
** Upgrading VMWare Fusion
** Standardising VNC user names
** Upgrading RCs on all nodes
** Upgrading libs/bindings
** etc.
** etc.
* Input Tests require the latest beta, can Raymond/Tanay tell us if we have one set up so we can add a new test to be run?
* Py.Test
* Adding Notes to CI Task so others can see
* Naming different environments 'Firefox-default;en-us;MacOSX6' is possibly causing '[http://qa-selenium.mv.mozilla.com:8080/job/amo.next.smoketests/921/console could not retrieve new session]' issues when builds collide
* I think we should remove our crashing Firefox 3.5 env on Mac OS X 10.6, due to [https://bugzilla.mozilla.org/show_bug.cgi?id=513747 bug 513747]
* In general, what's our plan for vetting/resurrecting failed (and now disabled,) tests (once our CI is more stable, of course)?
* Envs: Try Chrome again?  What about IE 9?  One or two more instances of Firefox 4 beta?


==Status Update==
==Status Update==




===[http://addons.mozille.org AMO] | [http://qa-selenium.mv.mozilla.com:8080/view/AMO%20Tests/ CI]===
===[http://addons.mozilla.org AMO] | [http://qa-selenium.mv.mozilla.com:8080/view/AMO%20Tests/ CI]===
* AMO rewrite has started
* AMO Rewrite tests on Hudson
** Tests are in my Github at the moment. Unfortunately not had time to expand because of playing "whack the mole" with current AMO tests
* Fixed AMO Smoke/Sort/Search/Bft suites


===[https://wiki.mozilla.org/Sumo SUMO] [http://qa-selenium.mv.mozilla.com:8080/view/SUMO%20Tests/ CI]===
===[https://wiki.mozilla.org/Sumo SUMO] [http://qa-selenium.mv.mozilla.com:8080/view/SUMO%20Tests/ CI]===
* We should flag in-testsuite? for bugs that either aren't covered by unit tests, or could use more coverage
* Smoke/BFT running fine
** Absolutely- there are some waiting, and more will be added [RB]
** BFT always passes, now (intermittent Mac failure was a profile issue wrt the cert). See http://qa-selenium.mv.mozilla.com:8080/job/sumo.bft/280/console
* Can't get Sel grid to launch custom profile on Win 7 (or even Win XP). Is this code correct [http://pastebin.mozilla.org/1041414]
<pre>@echo off
title Firefox36
set profile=c:\SumoFFProfile
cd C:\Users\mozilla\Desktop\selenium-grid\
ant -Dport=5580 -Denvironment="Firefox-default;en-us;Win7" -Dhost=10.250.3.161 -DhubURL=http://qa-selenium.mv.mozilla.com:4444 -DseleniumArgs="-firefoxProfileTemplate C:\SumoFFProfile" launch-remote-control
sleep 20</pre>
* Has its FFT evar passed? http://qa-selenium.mv.mozilla.com:8080/job/sumo.fft/
* Removed the commented out "#"Firefox-default-b;en-us;MacOSX6"," (we don't have a browser for that env) in http://viewvc.svn.mozilla.org/vc/projects/sumo/tests/frontend/python_tests/vars.py?r1=82334&r2=82366


===[https://wiki.mozilla.org/Firefox/Input Input] [http://qa-selenium.mv.mozilla.com:8080/job/input%20stage/ CI]===
===[https://wiki.mozilla.org/Firefox/Input Input] [http://qa-selenium.mv.mozilla.com:8080/job/input%20stage/ CI]===
* Have Tobbi, a contributor and soon to be intern, started looking at these
* Tobbi has his first test pulled into the main github repo
* We still get lots (too many) service unavailable messages
** Takes time to be sure it's the cause of failures - screenshots would help here
* We currently need to update when new Firefox desktop/mobile versions are available, perhaps come up with better solution if this becomes a maintenance issue


===[https://wiki.mozilla.org/Breakpad Socorro] [http://qa-selenium.mv.mozilla.com:8080/job/socorro/ CI]===
===[https://wiki.mozilla.org/Breakpad Socorro] [http://qa-selenium.mv.mozilla.com:8080/job/socorro/ CI]===
* Have been running fine since move to PHX
*  


===Mozilla.com [http://qa-selenium.mv.mozilla.com:8080/job/mozilla.com/ CI]===
===Mozilla.com [http://qa-selenium.mv.mozilla.com:8080/job/mozilla.com/ CI]===
* [https://github.com/retornam/mcom-tests Tests] for review from David Burns
*https://github.com/retornam/mcom-tests working on review suggestions from dburns


==Action Items==
==Action Items==
* Raymond and Tanay to have a look at capacity needs for the Grid and mail the team. Do we need more than 1 Mac Mini?
* Remove Fx3.5 for Mac OS 10.6 - DONE (tgavankar)
* DavidB is to have a look at Screen
* DB to have a look at Google Chrome Self-Signed issue
* DavidB to look at screen shots on Hudson
* DB To have a look at Win7 on QA-Selenium4

Latest revision as of 22:07, 2 March 2011

<< Last meeting | WebQA Automation Next meeting >>

Discussion topics

  • Someone (or someones!) needs to own Grid, which means:
    • Keeping browsers and the OS up-to-date (within versions)
    • Upgrading VMWare Fusion
    • Standardising VNC user names
    • Upgrading RCs on all nodes
    • Upgrading libs/bindings
    • etc.
  • Py.Test
  • Naming different environments 'Firefox-default;en-us;MacOSX6' is possibly causing 'could not retrieve new session' issues when builds collide
  • I think we should remove our crashing Firefox 3.5 env on Mac OS X 10.6, due to bug 513747
  • In general, what's our plan for vetting/resurrecting failed (and now disabled,) tests (once our CI is more stable, of course)?
  • Envs: Try Chrome again? What about IE 9? One or two more instances of Firefox 4 beta?

Status Update

AMO | CI

  • AMO Rewrite tests on Hudson
  • Fixed AMO Smoke/Sort/Search/Bft suites

SUMO CI

@echo off
title Firefox36
set profile=c:\SumoFFProfile
cd C:\Users\mozilla\Desktop\selenium-grid\
ant -Dport=5580 -Denvironment="Firefox-default;en-us;Win7" -Dhost=10.250.3.161 -DhubURL=http://qa-selenium.mv.mozilla.com:4444 -DseleniumArgs="-firefoxProfileTemplate C:\SumoFFProfile" launch-remote-control
sleep 20

Input CI

  • Tobbi has his first test pulled into the main github repo

Socorro CI

Mozilla.com CI

Action Items

  • Remove Fx3.5 for Mac OS 10.6 - DONE (tgavankar)
  • DB to have a look at Google Chrome Self-Signed issue
  • DB To have a look at Win7 on QA-Selenium4
Retrieved from "https://wiki.allizom.org/index.php?title=Web_Testing/Automation/automation-10-02-2011&oldid=288260"