Releases/Firefox 68/Test Plan: Difference between revisions

← Older edit

Releases/Firefox 68/Test Plan (view source)

Revision as of 13:51, 14 August 2019

12,270 bytes added , 14 August 2019

m

→‎Release 68 milestones

Camelia.badau

Confirmed users

644

edits

@@ Line 138: / Line 138: @@
 === [prior to merge] Nightly regression testing ===
 <div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
-<div style="font-weight:bold;line-height:1.6;">Nightly 68.0a1 (date1)</div>
+<div style="font-weight:bold;line-height:1.6;">Nightly 68.0a1 (2019-05-16)</div>
 <div class="mw-collapsible-content">
-* '''Build''': 68.0a1 (commit, ID) - Sign off_Status
+* '''Build''': 68.0a1 (commit, ID) - <font style="color:green; font-weight:bold;">GREEN</font>
-** {{mnotstarted|}} Regression testing (manual) → report
+** {{mdone|}} Regression testing (manual) → [https://testrail.stage.mozaws.net/index.php?/plans/view/17266 report]
 *** New bugs
-**** Regressions: ID1, ID2, IDN
+**** Regressions: 1550523, 1551554, 1551215, 1551452
-**** Non-regressions: ID1, ID2, IDN
+**** Non-regressions: 1544509, 1549360, 1550459, 1550533, 1551534
 *** Known bugs
-**** High-impact: ID1, ID2, IDN
+**** High-impact: none
-**** Low/medium-impact: <bug list>
+**** Low/medium-impact: [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1354992%2C%201370743%2C%201511869%2C%201481496%2C%201445644%2C%201360545%2C%201517548%2C%201511999%2C%201378105%2C%201470409%2C%201430114%2C%201358102%2C%201487467%2C%201328984%2C%201288773%2C%201406899%2C%201194214%2C%201441881%2C%201308007%2C%201515289%2C%201396732%2C%201485135%2C%201443511%2C%201470152%2C%201491251%2C%201527607%2C1393460%2C%201532730%2C%201512611%2C%201282075%2C%201528958%2C%201525607%2C%201525585%2C%20558178%2C%201413233%2C%201527257%2C%201509317%2C%201423615%2C%201198363%2C%201533409%2C%201496429%2C%201547806%2C%20169132%2C%201443145%2C%201428269%2C%201496472%2C%201428354&list_id=14766802 bug list]
 </div></div>
@@ Line 189: / Line 189: @@
 *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=271991687 Bug fix verification]
 ** '''Update tests (manual)''':
-*** {{N/A|}} aurora-cdntest → [N/A report]
+*** {{mdone|}} aurora-cdntest (as part of Beta 68.0b5 update tests)
-*** {{N/A|}} aurora → [N/A report]
+*** {{mdone|}} aurora (as part of Beta 68.0b5 update tests)
 </div></div>
@@ Line 198: / Line 198: @@
 * '''Build''': 68.0b9-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/927b23f07645dd3 927b23f07645dd3], 20190610153228) - <font style="color:green; font-weight:bold;">GREEN</font>
 ** '''Functional tests (manual)''':
-*** {{mprog|}} Developer Edition Recurring Sign off → [https://testrail.stage.mozaws.net/index.php?/plans/view/18100 TestRail]
+*** {{mdone|}} Developer Edition Recurring Sign off → [https://testrail.stage.mozaws.net/index.php?/plans/view/18100 TestRail]
 *** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=271991687 Bug fix verification]
 ** '''Update tests (manual)''':
-*** {{N/A|}} aurora-cdntest → [N/A report]
+*** {{mdone|}} aurora-cdntest  (as part of Beta 68.0b9 update tests)
-*** {{N/A|}} aurora → [N/A report]
+*** {{mdone|}} aurora (as part of Beta 68.0b9 update tests)
 </div></div>
@@ Line 273: / Line 273: @@
 ** '''Update tests (manual)''':
 *** {{mdone|}} beta-cdntest, aurora-cdntest → [https://docs.google.com/document/d/1LDOlEpLR2zvDCsJw2B_EG1frEZwOV69GTGXGReHpNhg/edit report]
-*** {{mnotstarted|}} beta, aurora → [https://docs.google.com/document/d/1LDOlEpLR2zvDCsJw2B_EG1frEZwOV69GTGXGReHpNhg/edit report]
+*** {{mdone|}} beta, aurora → [https://docs.google.com/document/d/1LDOlEpLR2zvDCsJw2B_EG1frEZwOV69GTGXGReHpNhg/edit report]
 </div></div>
@@ Line 308: / Line 308: @@
 <div class="mw-collapsible-content">
-* Build: 68.0b10-build1 (https://hg.mozilla.org/releases/mozilla-beta/rev/TBD TBD], TBD) - <font style="color:grey; font-weight:bold;">Not Started</font>
+* Build: 68.0b10-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/4c434d19d03d 4c434d19d03d], 20190613141208) - <font style="color:green; font-weight:bold;">GREEN</font>
 ** '''Functional tests (manual)''':
-*** {{mnotstarted|}} Smoke, Regression and Exploratory testing on '''TBD ''', '''TBD''' and '''TBD''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18165 TestRail]
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''WebExtensions compatibility & support''', '''Search''' and '''Profile support & compatibility''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18165 TestRail]
-*** {{mnotstarted|}} Smoke testing on Ubuntu snap package → [TBD report]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18187 report]
-*** {{mnotstarted|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pref=2&pli=1#gid=271991687 Bug List]
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pref=2&pli=1#gid=271991687 Bug List]
 ** '''Update tests (manual)''':
-*** {{mnotstarted|}} beta-cdntest, aurora-cdntest → [TBD report]
+*** {{mdone|}} beta-cdntest, aurora-cdntest → [https://docs.google.com/document/d/1nHr4G9FtrcqCOXhpdaL3E_kWcf7Seyko2fmOJBQcEGg/edit report]
-*** {{mnotstarted|}} beta, aurora → [TBD report]
+*** {{mdone|}} beta, aurora → [https://docs.google.com/document/d/1nHr4G9FtrcqCOXhpdaL3E_kWcf7Seyko2fmOJBQcEGg/edit report]
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">Beta 68.0b11</div>
+<div class="mw-collapsible-content">
+* Build: 68.0b11-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/60f0c79f0f53fcbe 60f0c79f0f53fcbe], 20190617172838) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Data Migration''' and '''Toolbars & window controls''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18218 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18234 report]
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pref=2&pli=1#gid=271991687 Bug List]
+** '''Update tests (manual)''':
+*** {{mdone|}} beta-cdntest, aurora-cdntest → [https://docs.google.com/document/d/1PrJvP60wvSvyyA04Bf7C0DtM-g_rojkYdHb7cLFUSuQ/edit report]
+*** {{mnotstarted|}} beta, aurora → [https://docs.google.com/document/d/1PrJvP60wvSvyyA04Bf7C0DtM-g_rojkYdHb7cLFUSuQ/edit report]
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">Beta 68.0b12</div>
+<div class="mw-collapsible-content">
+* Build: 68.0b12-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/046390a25abd 046390a25abd], 20190619234730) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Crash reporting''', '''Plugin compatibility & support'''and '''Geolocation''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18292 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18293 report]
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pref=2&pli=1#gid=271991687 Bug List]
+** '''Update tests (manual)''':
+*** {{mdone|}} beta-cdntest, aurora-cdntest → [https://docs.google.com/document/d/1dkGW5enod_IXRJxhqrIm06IAkVpwPMrhtjTvNakY24Y/edit report]
+*** {{mdone|}} beta, aurora → [https://docs.google.com/document/d/1dkGW5enod_IXRJxhqrIm06IAkVpwPMrhtjTvNakY24Y/edit report]
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">Beta 68.0b13</div>
+<div class="mw-collapsible-content">
+* Build: 68.0b13-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/da9f01df3bef da9f01df3bef], 20190624133534) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Session restore'''and '''Content scroll, zoom & drag support''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18452 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18474 report]
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pref=2&pli=1#gid=271991687 Bug List]
+** '''Update tests (manual)''':
+*** {{mdone|}} beta-cdntest, aurora-cdntest → [https://docs.google.com/document/d/1EJ3GgYVhTGIBdVTt5XI58OfsUDktO05Y0cUrxm87RCk/edit report]
+*** {{mdone|}} beta, aurora → [https://docs.google.com/document/d/1EJ3GgYVhTGIBdVTt5XI58OfsUDktO05Y0cUrxm87RCk/edit report]
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">Beta 68.0b14</div>
+<div class="mw-collapsible-content">
+* Build: 68.0b14-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/a38e711fb744 a38e711fb744], 20190627143605) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Bookmarks''' and '''Media playback & support''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18522 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18537 report]
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit?pref=2&pli=1#gid=271991687 Bug List]
+** '''Update tests (manual)''':
+*** {{mdone|}} beta-cdntest, aurora-cdntest → [https://docs.google.com/document/d/1gsRcOLCZ6sSYf0TJPOmC85S_nFoDCMxHjcP3pLIibFU/edit# report]
+*** {{mnotstarted|}} beta, aurora → [https://docs.google.com/document/d/1gsRcOLCZ6sSYf0TJPOmC85S_nFoDCMxHjcP3pLIibFU/edit# report]
 </div></div>
@@ Line 322: / Line 378: @@
 <div style="font-weight:bold;line-height:1.6;">RC1 68.0</div>
 <div class="mw-collapsible-content">
-* '''Build''': 68.0-build1 (commit, ID) - Sign off_Status
+* '''Build''': 68.0-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/3fc582b1f4b2 3fc582b1f4b2], 20190701181138) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Web compatibility''' and '''WebExtensions compatibility & support''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18586 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18608 report]
+*** {{mdone|}} Smoke testing on Windows Acer builds → [https://docs.google.com/document/d/1n4Kg6mvy9GBr0KxUVvYnd-LP4S6bHcYHE8KajXuS4UM/edit# report]
+*** {{mprog|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=271991687 Bug verification]
+** '''Update tests (manual)''':
+*** {{mdone|}} beta-cdntest → [https://docs.google.com/document/d/1nbK0ScU8PekxQLrJ99Fz2R3OJNuUT-K4dSuz26AYKi4/edit report]
+*** {{mnotstarted|}} beta → skipped
+*** {{mnotstarted|}} release-localtest → skipped
+*** {{mnotstarted|}} release-cdntest → skipped
+*** {{mnotstarted|}} release → skipped
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">RC2 68.0</div>
+<div class="mw-collapsible-content">
+* '''Build''': 68.0-build2 ([https://hg.mozilla.org/releases/mozilla-beta/rev/3a52b315fac0 3a52b315fac0], 20190704163141) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Web compatibility''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18753 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/plans/view/18774 report]
+*** {{mdone|}} Smoke testing on Windows Acer builds → [https://docs.google.com/document/d/17D7JnBOj5tEFdfavEafWaliSLQxybCDbj4NAbgC_qGs/edit#heading=h.ra59jc33u21r report]
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=271991687 Bug verification]
+** '''Update tests (manual)''':
+*** {{mnotstarted|}} beta-cdntest → [https://drive.google.com/open?id=1RotoD7oZcSYC9hOcdnTP_-S8RTLoDgiz3qI7OnKjOZY report]
+*** {{mnotstarted|}} beta → skipped
+*** {{mnotstarted|}} release-localtest → skipped
+*** {{mnotstarted|}} release-cdntest → skipped
+*** {{mnotstarted|}} release → skipped
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">RC3 68.0</div>
+<div class="mw-collapsible-content">
+* '''Build''': 68.0-build3 ([https://hg.mozilla.org/releases/mozilla-beta/rev/353628fec415 353628fec415], 20190705220548) - <font style="color:green; font-weight:bold;">GREEN</font>
 ** '''Functional tests (manual)''':
-*** {{mnotstarted|}} Smoke, Regression and Exploratory testing on _ → report
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Web compatibility''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/18789 TestRail]
-*** {{mnotstarted|}} Smoke testing on Ubuntu snap package → report
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/reports/view/1832 report]
-*** {{mnotstarted|}} Smoke testing on Windows Acer builds → report
+*** {{mdone|}} Smoke testing on Windows Acer builds → [https://docs.google.com/document/d/17D7JnBOj5tEFdfavEafWaliSLQxybCDbj4NAbgC_qGs/edit#heading=h.ra59jc33u21r report]
-*** {{mnotstarted|}} Bug fix verification – <bug list>
+*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=271991687 Bug verification]
 ** '''Update tests (manual)''':
-*** {{mnotstarted|}} beta-cdntest → report
+*** {{mdone|}} beta-cdntest → [https://docs.google.com/document/d/1PtwR_oXQzg_HWhkFNLbFfTPL_-crCa__IPgHtUQoge8/edit# report]
-*** {{mnotstarted|}} beta → report
+*** {{mdone|}} beta → [https://docs.google.com/document/d/1PtwR_oXQzg_HWhkFNLbFfTPL_-crCa__IPgHtUQoge8/edit# report]
-*** {{mnotstarted|}} release-localtest → report
+*** {{mnotstarted|}} release-localtest → skipped
-*** {{mnotstarted|}} release-cdntest → report
+*** {{mdone|}} release-cdntest → [https://docs.google.com/document/d/1AuH79AthcY2DiitN4IQWYqK-VJxlZP_kN5yTWVEX0v0/edit report]
 *** {{mnotstarted|}} release → report
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">RC 68.0.1</div>
+<div class="mw-collapsible-content">
+* '''Build''': 68.0.1-build1 ([https://hg.mozilla.org/releases/mozilla-release/rev/837bbcb850cd 837bbcb850cd], 20190717172542) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Web compatibility''' → [https://testrail.stage.mozaws.net/index.php?/reports/view/1848 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/reports/view/1853 report]
+*** {{mdone|}} Smoke testing on Windows Acer builds → [https://docs.google.com/document/d/1nfUcNS0WtDoVRVb5SCA_a_q3ZgOpwiivJ_AeMnbGcIE/edit# report]
+*** {{mdone|}} Bug fix verification – [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1553586%2C1556846%2C1558503%2C1565779%2C1566206&list_id=14810958 Bug verification]
+** '''Update tests (manual)''':
+*** {{mdone|}} release-cdntest → [https://docs.google.com/document/d/1x62rHtXpS9q6XS6Ft8iWgCU-wMIHthloWt-20Kn4Wz8/edit report]
+*** {{mnotstarted|}} release → [https://docs.google.com/document/d/1x62rHtXpS9q6XS6Ft8iWgCU-wMIHthloWt-20Kn4Wz8/edit report]
+</div></div>
+<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
+<div style="font-weight:bold;line-height:1.6;">RC 68.0.2</div>
+<div class="mw-collapsible-content">
+* '''Build''': 68.0.2-build1 ([https://hg.mozilla.org/releases/mozilla-release/rev/7ece03f69719 7ece03f69719], 20190813150448) - <font style="color:green; font-weight:bold;">GREEN</font>
+** '''Functional tests (manual)''':
+*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Web compatibility''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/19576 TestRail]
+*** {{mdone|}} Smoke testing on Ubuntu snap package → [https://testrail.stage.mozaws.net/index.php?/reports/view/1922 report]
+*** {{mdone|}} Smoke testing on Windows Acer builds → [https://docs.google.com/document/d/1C-_C19mjcl351jygT5Dyo64AT2zFWlu1oHsQoEOrBU4/edit report]
+*** {{mdone|}} Bug fix verification – [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=1560228%2C%201565542%2C%201567614%2C%201569244&list_id=14850527 Bug verification]
+** '''Update tests (manual)''':
+*** {{mdone|}} release-cdntest → [https://docs.google.com/spreadsheets/d/1aLXxd7yMJaisaWPaW_lBTeMyOyGXiie-pRjoJrphu4E/edit#gid=850719639 report]
+*** {{mnotstarted|}} release → [https://docs.google.com/spreadsheets/d/1aLXxd7yMJaisaWPaW_lBTeMyOyGXiie-pRjoJrphu4E/edit#gid=850719639 report]
 </div></div>