MozillaWiki

Releases/Firefox 46/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 46/Test Plan (view source)

Revision as of 11:25, 16 May 2016

13,139 bytes added ,  16 May 2016
m
→‎46.0.1 Build 1
No edit summary
 
(44 intermediate revisions by 3 users not shown)
Line 275: Line 275:
==== Testing ====
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/8 Firefox 46 Beta 8]
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/8 Firefox 46 Beta 8]
** {{mok|}} [https://goo.gl/EeMGXN 46.0b8-build1 Regression Testing] targeting Plug-in Compatibility, Downloads, Safe Browsing and Session Restore.
** {{mdone|}} [https://goo.gl/EeMGXN 46.0b8-build1 Regression Testing] targeting Plug-in Compatibility, Downloads, Safe Browsing and Session Restore.
*** The build has been <font style="background: yellow; font-size:80%; font-weight:bold;">CONDITIONALLY SIGNED OFF</font> because of [https://bugzilla.mozilla.org/show_bug.cgi?id=1262149 1262149].
*** The build has been <font style="background: yellow; font-size:80%; font-weight:bold;">CONDITIONALLY SIGNED OFF</font> because [https://bugzilla.mozilla.org/show_bug.cgi?id=1259482 Bug 1259482] still reproduces intermittently, and 3 new issues were found for Hello (not blockers).
*** Please note that this task is still in progress as we're currently waiting on test results for Safe Browsing.
** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1950697052 Bug verification] with focus on the latest fixes landed in beta.
** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1950697052 Bug verification] with focus on the latest fixes landed in beta.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46b8|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])  
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46b8|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])  
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=0334bcac4033&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])  
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=0334bcac4033&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])  
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=0334bcac4033&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=0334bcac4033&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
=== Beta 9 ===
==== Build Information ====
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/46.0b9-candidates/build1/ 46.0b9-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/b007110e9005 b007110e9005] | (ID: 20160407053945)
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/9 Firefox 46 Beta 9]
** {{mok|}} [https://docs.google.com/spreadsheets/d/1XIeNS_gl-PitOy-lzlPGoIo1dUBQjsfOAOWeCdqUb8c/edit?pref=2&pli=1#gid=432237209 46.0b9-build1 Regression Testing] targeting OS X Signing, OS X 32-bit mode, Hello add-on (v1.2.4).
** {{mok|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1950697052 Bug verification] with focus on the latest fixes landed in beta.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46b9|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=b007110e9005&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=b007110e9005&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
=== Beta 10 ===
==== Build Information ====
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/46.0b10-candidates/build1/ 46.0b10-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/9ea83990839b 9ea83990839b] | (ID: 20160411042519)
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/10 Firefox 46 Beta 10]
** {{mdone|}} [https://goo.gl/fSUC4i 46.0b10-build1 Regression Testing] targeting Sync, Default browser settings and Hello (Loop).
** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1950697052 Bug verification] with focus on the latest fixes landed in beta.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46b10|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=9ea83990839b&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=9ea83990839b&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
=== Beta 11 ===
==== Build Information ====
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/46.0b11-candidates/build1/ 46.0b11-build1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/076bf6a0ac85 076bf6a0ac85] | (ID: 20160414152344)
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/11 Firefox 46 Beta 11]
** {{mdone|}} [https://docs.google.com/spreadsheets/d/1XIeNS_gl-PitOy-lzlPGoIo1dUBQjsfOAOWeCdqUb8c/edit#gid=276552732 46.0b11-build1 Regression Testing] targeting Video Playback.
** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1950697052 Bug verification] with focus on the latest fixes landed in beta.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46b11|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{mdone|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=076bf6a0ac85&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{mdone|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=076bf6a0ac85&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
== Release Candidates ==
=== 46.0 RC1 (build 3) ===
==== Build Information ====
* Build 3: [https://archive.mozilla.org/pub/firefox/candidates/46.0-candidates/build3/ 46.0RC-build3] | [https://hg.mozilla.org/releases/mozilla-release/rev/521d7102e93d 521d7102e93d] | (ID: 20160418114253)
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/RC1 Firefox 46.0 RC1]
** {{mdone|}} [https://docs.google.com/spreadsheets/d/1XIeNS_gl-PitOy-lzlPGoIo1dUBQjsfOAOWeCdqUb8c/edit#gid=1074355926 Smoke Testing] with focus on Web Compatibility.
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46RC1|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=521d7102e93d&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=521d7102e93d&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
=== 46.0 RC2 (build 4) ===
==== Build Information ====
* Build 4: [https://archive.mozilla.org/pub/firefox/candidates/46.0-candidates/build4/ 46.0RC-build4] | [https://hg.mozilla.org/releases/mozilla-release/rev/53f1fadd9d33 53f1fadd9d33] | (ID: 20160420024437)
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/RC2 Firefox 46.0 RC2]
** {{mdone|}} [https://goo.gl/WLc9vT Smoke Testing] with focus on Web Compatibility and the Password Manager.
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46RC2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{done|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=53f1fadd9d33&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{done|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=53f1fadd9d33&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
=== 46.0 RC3 (build 5) ===
==== Build Information ====
* Build 5: [https://archive.mozilla.org/pub/firefox/candidates/46.0-candidates/build5/ 46.0RC-build5] | [https://hg.mozilla.org/releases/mozilla-release/rev/078baf501b55 078baf501b55] | (ID: 20160421124000)
==== Testing ====
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/RC3 Firefox 46.0 RC3]
** {{mdone|}} [https://goo.gl/n2ftBZ Smoke Testing] with focus on Search.
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46RC2|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** This is a condidate for the 46.0 release build. We should test updates on beta-cdntest for a preliminary sign-off (Release Management must give the "go" for pushing to beta), then on beta after the push to beta, and then on the day before release (Monday), after a full sign-off and push to that channel, run tests on release-cdntest.
** {{mdone|}} Updates on ''beta-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest( report])
** {{mdone|}} Updates on ''beta'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta( report])
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report])
** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=078baf501b55&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report])
=== Release 46.0 ===
==== Build Information ====
* See Release Candidate Build 5
==== Testing ====
* Update Testing - see Release Candidate Build 5
* Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details)
** Used commands:
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml
./throttle.sh https://aus3.mozilla.org/update/3/Firefox/45.0.2/20160407164938/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml
** Results
*** After release push (expected to be @ 25%), 2016-04-26: Win32: 23% - Win64: 24% - Mac: 29% - Linux32: 21% - Linux64: 30%
*** After disabling updates (expected to be @ 0%), 2016-04-27: Win32: 0% - Win64: 0% - Mac: 0% - Linux32: 0% - Linux64: 0%
*** After fully enabling updates (expected to be @ 100%): N/A - updates not re-enabled (dot release planned)
=== 46.0.1 Build 1 ===
==== Build Information ====
* Build 1: [https://archive.mozilla.org/pub/firefox/candidates/46.0.1-candidates/build1/ 46.0.1-build1] | [https://hg.mozilla.org/releases/mozilla-release/rev/0b8492c110be 0b8492c110be] | (ID: 20160502172042)
* Manual - [https://wiki.mozilla.org/Releases/Firefox_46/Test_Plan/Beta/46.0.1 Firefox 46.0.1]
** {{mdone|}} [https://goo.gl/jzWxrj Regression Testing] with focus on Web Compatibility, and Add-on Compatibility.
** {{mdone|}} [https://goo.gl/MVvF4a Bug verification] with focus on the latest fixes to land in this build.
* Automated ([[QA/Desktop_Firefox/Releases/Configs/Fx46.0.1|configs]] for [http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/build?delay=0sec ondemand])
** {{mdone|}} Updates on ''release-localtest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=0b8492c110be&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-localtest( report])
** {{mdone|}} Updates on ''release-cdntest'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=0b8492c110be&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release-cdntest( report])
** {{mdone|}} Updates on ''release'' channel ([https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=0b8492c110be&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-release( report])
** Throttle Testing (see [[QA/Desktop_Firefox/Walkthroughs/Release_Coordination#Update_Throttling|Walkthrough]] for details)
*** Used commands:
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/WINNT_x86-msvc/en-US/release/Windows_NT%206.3/default/default/update.xml
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/WINNT_x86_64-msvc/en-US/release/Windows_NT%2010.0/default/default/update.xml
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/Darwin_x86_64-gcc3-u-i386-x86_64/en-US/release/Darwin%2013.4.0/default/default/update.xml
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/Linux_x86-gcc3/en-US/release/Linux%203.13.0-45-generic%20%28GTK%202.24.23%29/default/default/update.xml
**** ./throttle.sh https://aus3.mozilla.org/update/3/Firefox/46.0/20160421124000/Linux_x86_64-gcc3/en-US/release/Linux%203.16.0-30-generic%20%28GTK%202.24.25%29/default/default/update.xml
*** Results
**** After release push (expected to be @ 25%), 2016-05-03: Win32: 23% - Win64: 26% - Mac: 24% - Linux32: 24% - Linux64: 24%
**** After disabling updates (expected to be @ 0%):
**** After fully enabling updates (expected to be @ 100%), 2016-05-05: Win32: 100% - Win64: 100% - Mac: 100% - Linux32: 100% - Linux64: 100%


[[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]]
[[category:Test_Plans|Firefox]][[category:QA|Firefox Test Plans]]
Florin.Mezei
Confirmed users
2,479

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/1126154...1132783"