Confirmed users
2,479
edits
Releases/Firefox 46/Test Plan: Difference between revisions
m
→46.0.1 Build 1
No edit summary |
Florin.Mezei (talk | contribs) m (→46.0.1 Build 1) |
||
| (27 intermediate revisions by 3 users not shown) | |||
| Line 319: | Line 319: | ||
== Release Candidates == | == Release Candidates == | ||
=== 46.0 RC1 | === 46.0 RC1 (build 3) === | ||
==== Build Information ==== | ==== Build Information ==== | ||
* Build | * Build 3: [https://archive.mozilla.org/pub/firefox/candidates/46.0-candidates/build3/ 46.0RC-build3] | [https://hg.mozilla.org/releases/mozilla-release/rev/521d7102e93d 521d7102e93d] | (ID: 20160418114253) | ||
==== Testing ==== | ==== Testing ==== | ||
| Line 330: | Line 330: | ||
** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest. | ** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest. | ||
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=521d7102e93d&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report]) | ** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=521d7102e93d&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report]) | ||
** {{ | ** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=521d7102e93d&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report]) | ||
=== 46.0 RC2 (build 4) === | |||
==== Build Information ==== | |||
* Build 4: [https://archive.mozilla.org/pub/firefox/candidates/46.0-candidates/build4/ 46.0RC-build4] | [https://hg.mozilla.org/releases/mozilla-release/rev/53f1fadd9d33 53f1fadd9d33] | (ID: 20160420024437) | |||
==== Testing ==== | |||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/RC2 Firefox 46.0 RC2] | |||
** {{mdone|}} [https://goo.gl/WLc9vT Smoke Testing] with focus on Web Compatibility and the Password Manager. | |||
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build. | |||
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46RC2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | |||
** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest. | |||
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=53f1fadd9d33&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report]) | |||
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=53f1fadd9d33&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report]) | |||
=== 46.0 RC3 (build 5) === | |||
==== Build Information ==== | |||
* Build 5: [https://archive.mozilla.org/pub/firefox/candidates/46.0-candidates/build5/ 46.0RC-build5] | [https://hg.mozilla.org/releases/mozilla-release/rev/078baf501b55 078baf501b55] | (ID: 20160421124000) | |||
==== Testing ==== | |||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/RC3 Firefox 46.0 RC3] | |||
** {{mdone|}} [https://goo.gl/n2ftBZ Smoke Testing] with focus on Search. | |||
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build. | |||
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46RC2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | |||
** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest. | |||
** {{mdone|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report]) | |||
** {{mdone|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report]) | |||
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report]) | |||
** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report]) | |||
=== Release 46.0 === | |||
==== Build Information ==== | |||
* See Release Candidate Build 5 | |||
==== Testing ==== | |||
* Update Testing - see Release Candidate Build 5 | |||
* Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details) | |||
** Used commands: | |||
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml | |||
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml | |||
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml | |||
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml | |||
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml | |||
** Results | |||
*** After release push (expected to be @ 25%), 2016-04-26: Win32: 23% - Win64: 24% - Mac: 29% - Linux32: 21% - Linux64: 30% | |||
*** After disabling updates (expected to be @ 0%), 2016-04-27: Win32: 0% - Win64: 0% - Mac: 0% - Linux32: 0% - Linux64: 0% | |||
*** After fully enabling updates (expected to be @ 100%): N/A - updates not re-enabled (dot release planned) | |||
=== 46.0.1 Build 1 === | |||
==== Build Information ==== | |||
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/46.0.1-candidates/build1/ 46.0.1-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/0b8492c110be 0b8492c110be] | (ID: 20160502172042) | |||
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/46.0.1 Firefox 46.0.1] | |||
** {{mdone|}} [https://goo.gl/jzWxrj Regression Testing] with focus on Web Compatibility, and Add-on Compatibility. | |||
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build. | |||
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46.0.1|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand]) | |||
** {{mdone|}} Updates on ''release-localtest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=0b8492c110be&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest( report]) | |||
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=0b8492c110be&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report]) | |||
** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=0b8492c110be&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report]) | |||
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details) | |||
*** Used commands: | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml | |||
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml | |||
*** Results | |||
**** After release push (expected to be @ 25%), 2016-05-03: Win32: 23% - Win64: 26% - Mac: 24% - Linux32: 24% - Linux64: 24% | |||
**** After disabling updates (expected to be @ 0%): | |||
**** After fully enabling updates (expected to be @ 100%), 2016-05-05: Win32: 100% - Win64: 100% - Mac: 100% - Linux32: 100% - Linux64: 100% | |||
[[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]] | [[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]] | ||