Confirmed users
2,479
edits
Releases/Firefox 47/Test Plan: Difference between revisions
Releases/Firefox 47/Test Plan (view source)
Revision as of 16:39, 11 November 2016
, 11 November 2016→47.0.2 Build 2
Florin.Mezei (talk | contribs) m (→47.0.1 Build 1) |
Florin.Mezei (talk | contribs) |
||
| (18 intermediate revisions by 3 users not shown) | |||
| Line 360: | Line 360: | ||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/47.0.1-candidates/build1/ 47.0.1-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/7f5abf95991b 7f5abf95991b] | (ID: 20160623154057) | * Build 1: [https://archive.mozilla.org/pub/firefox/candidates/47.0.1-candidates/build1/ 47.0.1-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/7f5abf95991b 7f5abf95991b] | (ID: 20160623154057) | ||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_47/Test_Plan/Beta/47.0.1 Firefox 47.0.1] | * Manual - [https://wiki.mozilla.org/Releases/Firefox_47/Test_Plan/Beta/47.0.1 Firefox 47.0.1] | ||
** {{ | ** {{mdone|}} [https://docs.google.com/spreadsheets/d/1h8PABdVzuYsIXQQz5Vj8ymkCVOBh0jhbcGShcxU9cWs/edit#gid=843686709 Regression Testing] with focus on Web Compatibility. | ||
** {{ | ** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=312280130 Bug verification] with focus on the latest fixes to land in this build. | ||
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx47.0.1|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | * Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx47.0.1|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | ||
** {{ | ** {{mdone|}} Updates on ''release-localtest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=7f5abf95991b&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest( report]) | ||
** {{ | ** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=7f5abf95991b&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report]) | ||
** {{ | ** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=7f5abf95991b&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report]) | ||
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details) | ** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details) | ||
*** Used commands: | *** Used commands: | ||
| Line 377: | Line 377: | ||
*** After disabling updates (expected to be @ 0%), 2016-06-29: '''Win32: % - Win64: % - Mac: % - Linux32: % - Linux64: %''' | *** After disabling updates (expected to be @ 0%), 2016-06-29: '''Win32: % - Win64: % - Mac: % - Linux32: % - Linux64: %''' | ||
*** After fully enabling updates (expected to be @ 100%): 2016-07-01: '''Win32: % - Win64: % - Mac: % - Linux32: % - Linux64: %''' | *** After fully enabling updates (expected to be @ 100%): 2016-07-01: '''Win32: % - Win64: % - Mac: % - Linux32: % - Linux64: %''' | ||
*** '''NOTE: throttle tests do not apply to this release as we released at 100% for versions < 47.0, and 0% for 47.0 (except for forced updates) - this was confirmed by ondemand update tests on the [https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=7f5abf95991b&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( release channel] (for forced updates), and throttling tests which were at 0% (using the throttling commands above)''' | |||
=== 47.0.2 Build 1 === | |||
==== Build Information ==== | |||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/47.0.2-candidates/build1/ 47.0.2-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/3dc66a6a33a0 3dc66a6a33a0] | (ID: 20161027172622) | |||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_47/Test_Plan/Beta/47.0.2 Firefox 47.0.2] | |||
** {{mdone|}} Sanity Testing with focus on: general Web Compatibility. | |||
*** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/reports/view/104 in TestRail] (requires access rights) or [https://drive.google.com/open?id=xxxxxxx as a pdf]. | |||
** {{mdone|}} [https://public.etherpad-mozilla.org/p/1305787 Verification] of driver [https://bugzilla.mozilla.org/show_bug.cgi?id=1305787 bug 1305787] | |||
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx47.0.2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | |||
** {{mdone|}} Updates on ''release-localtest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=3dc66a6a33a0&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest( report]) | |||
*** Also tested manually that: | |||
**** Fx < 47.0.2 with the hotfix installed and (websense) get updated to 47.0.2 | |||
**** Fx < 47.0.2 with the hotfix installed and (nowebsense) get updated to 48.0.2 and then 49.0.2 | |||
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=3dc66a6a33a0&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report]) | |||
** {{mnotstarted|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=3dc66a6a33a0&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report]) | |||
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details) | |||
*** '''N/A, as Build 2 was generated''' | |||
=== 47.0.2 Build 2 === | |||
==== Build Information ==== | |||
* Build 2: [https://archive.mozilla.org/pub/firefox/candidates/47.0.2-candidates/build2/ 47.0.2-build2] | [https://hg.mozilla.org/releases/mozilla-release/rev/d7f98b92fdec d7f98b92fdec] | (ID: 20161031133903) | |||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_47/Test_Plan/Beta/47.0.2 Firefox 47.0.2] | |||
** {{mdone|}} Sanity Testing with focus on: general Web Compatibility. | |||
*** Detailed test results: [https://testrail.stage.mozaws.net/index.php?/plans/view/1136 in TestRail] (requires access rights) or [https://drive.google.com/open?id=0BzOV3ybvBASXUmVDVTRPUm9Zanc as a pdf]. | |||
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx47.0.2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | |||
** {{mdone|}} Updates on ''release-localtest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=d7f98b92fdec&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest( report]) | |||
*** Also tested manually that: | |||
**** Fx < 47.0.2 with the hotfix installed and (websense) get updated to 47.0.2 | |||
**** Fx < 47.0.2 with the hotfix installed and (nowebsense) get updated to 48.0.2 and then 49.0.2 | |||
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=d7f98b92fdec&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report]) | |||
** {{mdone|}} Updates on ''release'' channel | |||
*** Automated tests failures - ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=d7f98b92fdec&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report]) | |||
*** Manual tests [https://public.etherpad-mozilla.org/p/Fx47.0.2-updatetests results] | |||
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details) | |||
*** Used commands: | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/47.0.1/20160623154057/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/47.0.1/20160623154057/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/47.0.1/20160623154057/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/47.0.1/20160623154057/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/47.0.1/20160623154057/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml | |||
*** Results | |||
*** After release push (expected to be @ 25%), 2016-10-31: '''N/A - Releasing directly to 100%''' | |||
*** After disabling updates (expected to be @ 0%), 2016-10-31: '''N/A - Releasing directly to 100%''' | |||
*** After fully enabling updates (expected to be @ 100%): 2016-11-04: '''Win32: 100% - Win64: 100% - Mac: 100% - Linux32: 100% - Linux64: 100%''' | |||
[[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]] | [[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]] | ||