Releases/Firefox 73/Test Plan: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(updated status)
 
(47 intermediate revisions by 8 users not shown)
Line 111: Line 111:


<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">Nightly 72.0a1 (2019-05-30)</div>
<div style="font-weight:bold;line-height:1.6;">Nightly 73.0a1(2019-05-30)</div>
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
* '''Build''': 73.0a1 (commit, ID) - <font style="color:grey; font-weight:bold;">N.A.</font>
* '''Build''': 73.0a1 (commit, ID) - <font style="color:grey; font-weight:bold;">N.A.</font>
Line 124: Line 124:


<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">Nightly 72.0a1 (2019-06-13)</div>
<div style="font-weight:bold;line-height:1.6;">Nightly 73.0a1 (2019-06-13)</div>
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
* '''Build''': 73.0a1 (commit, ID) - <font style="color:green; font-weight:bold;">N.A.</font>
* '''Build''': 73.0a1 (commit, ID) - <font style="color:green; font-weight:bold;">N.A.</font>
Line 158: Line 158:
=== Developer Edition build readiness ===
=== Developer Edition build readiness ===
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">DevEd 72.0b4</div>
<div style="font-weight:bold;line-height:1.6;">DevEd 73.0b1</div>
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
* '''Build''': 73.0b1-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6 ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6], 20200106175743) - <font style="color:green; font-weight:bold;">N.A.</font>
* '''Build''': 73.0b1-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6 ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6], 20200106175743) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Tabbed Browser''' , '''OS Integration''' , '''User Profile''' , '''Form Autofill''' , '''Bookmarks'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24041 TestRail]
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Tabbed Browser''' , '''OS Integration''' , '''User Profile''' , '''Form Autofill''' , '''Bookmarks'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24041 TestRail]
</div></div>
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">DevEd 73.0b5</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b5-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/087b0cab93cc2c55a84e3fd491160cc183e60bfd 087b0cab93cc2c55a84e3fd491160cc183e60bfd], 20200115020958) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Startup and Profile''' , '''Build Branding''' , '''WebExtensions''' , '''Theme and Toolbar Customization''' , '''Web Compatibility''', '''Sync & Firefox Account''', '''OS Integration'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24225 TestRail]
</div></div>
</div></div>


=== Beta build readiness ===
=== Beta build readiness ===
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">DevEd 72.0b4</div>
<div style="font-weight:bold;line-height:1.6;">73.0b1</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b1-build2 ([https://hg.mozilla.org/releases/mozilla-beta/rev/ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6 ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6], 20200106175743) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Tabbed Browser''', '''OS Integration''', '''User Profile''', '''Form Autofill''', '''Bookmarks'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24041 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1360074389 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1af4R89RIjABPnU1bGXyKxoO7KMbp3uNZEmARxBPWH_0/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b2</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b2-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/4ec2c9a971e1a9a49b5f24a59343fba245470a91 4ec2c9a971e1a9a49b5f24a59343fba245470a91], 20200107212705) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Security and Privacy ''', '''Theme and Toolbar Customization''', '''Browser Window''', '''Bookmarks and History''', '''Form Autofill'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24132 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1FqUHR7nggpa0S4r1693Hhnl9x3fX-8wV0k9jeSHT6F0/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b3</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b3-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/e592eb8fc8c2b5a7fd971c1430c70de8f058a071 e592eb8fc8c2b5a7fd971c1430c70de8f058a071], 20200110003145) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Geolocation''', '''Form Autofill''', '''Password Manager 2.0''', '''Preferences ''', '''Notifications, Push Notifications and Alerts''', '''Menus'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24268 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/u/1/d/1tl2K8XD9gwrMASHwIJ6dcmkV4bEZQD1mVhHm2U0Yn9Y/edit?usp=sharing report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b4</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b4-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/599d3e91827c8c124e9345fb88ba42b7a238b75d 599d3e91827c8c124e9345fb88ba42b7a238b75d], 20200112220634) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Startup and Profile''', '''Downloads''', '''Security and Privacy''' and  '''Preferences'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24315 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1qJA1xAWNH6qsEZevnQ3QkUxmSD-No3ojNoQu8APyIac/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b5</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b5-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/087b0cab93cc2c55a84e3fd491160cc183e60bfd 087b0cab93cc2c55a84e3fd491160cc183e60bfd], 20200115020958) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Installer''', '''Address Bar and Search''', '''Security and Privacy''' and  '''Scrolling, Panning and Zooming'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24547 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1cEJN-xbbanskopWUOx9mjU8-lJO_gJ0gcdiVIHk_-Xk/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b6</div>
<div class="mw-collapsible-content">
<div class="mw-collapsible-content">
* '''Build''': 73.0b1-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6 ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6], 20200106175743) - <font style="color:green; font-weight:bold;">In progress</font>
* '''Build''': 73.0b6-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/21980c8b11cca172f8d69f0e31ddfde00c98e02f 21980c8b11cca172f8d69f0e31ddfde00c98e02f], 20200116232741) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Tabbed Browser''' , '''OS Integration''' , '''User Profile''' , '''Form Autofill''' , '''Bookmarks'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24041 TestRail]
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Audio/Video''', '''Notifications, Push Notifications and Alerts''', '''Session Restore''' and  '''Password Manager'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24547 TestRail]
*** {{mprog|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1360074389 Bug fix verification]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/u/1/d/1772hSxW2E3IyGuR-KBC_zcpr8ksyoMlPwSKd3itBcTI/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b7</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b7-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/a89261364728dbfe37351e0de812e906c436a5d4 a89261364728dbfe37351e0de812e906c436a5d4], 20190325125126) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Web Compatibility''', '''Graphics''', '''Printing''', '''Drag and Drop'''  and  '''Security and Privacy'''  → [https://testrail.stage.mozaws.net/index.php?/plans/view/24665 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1O-s28rVL-9u72E9pvYoEqIhDZUqXfEkJaMxFUUT0NRI/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b8</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b8-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/6a9e651e0831 6a9e651e0831], 20200122012721) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Plug-ins''', '''Uninstaller''' and '''Pocket New Tab''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/24793 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1q7Dh9rG4nMkeOLQnsCRIQP5DUf0J4TsjhPRhXojXmlA/edit?pli=1#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b9</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b9-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/3f19dfb367d6 3f19dfb367d6], 20200124022546) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1hvwohnE53jvCXdboPqpdk69kcBGcKk12PLoy3mcHw1g/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b10</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b10-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/38fc7c5ed55f 38fc7c5ed55f], 20200125104833) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Crash Reporting''', '''Security and Privacy''', '''Images''', '''PDF Viewer''', '''Password Manager''' and '''Preferences''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/24870 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/15_dTJXC85an6EeWv1Z4Gw96OUrtSRJg0LSgGx8O3R50/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b11</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b11-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/c24dfbda267e c24dfbda267e], 20200128001646) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Password Manager''', '''Data Migration''', '''Sync & Firefox Account''' and '''Menus''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/25024 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mdone|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1GjEWX8pYwt5lRZDQGHzGwGr2_sjYCO2yVk77QApAFtU/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">73.0b12</div>
<div class="mw-collapsible-content">
* '''Build''': 73.0b12-build1 ([https://hg.mozilla.org/releases/mozilla-beta/rev/14ac73797dd8 14ac73797dd8], 20200130174018) - <font style="color:green; font-weight:bold;">GREEN</font>
** '''Functional tests (manual)''':
*** {{mdone|}} Smoke, Regression and Exploratory testing on '''Preferences''', '''Menus''', '''Reader View''', '''Find Toolbar''' and '''Firefox Screenshots''' → [https://testrail.stage.mozaws.net/index.php?/plans/view/25099 TestRail]
*** {{mdone|}} [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug fix verification]
** '''Update tests (manual)''':
*** {{mprog|}} beta, aurora → [https://docs.google.com/spreadsheets/d/19eRl3jOlR2i3Zn4EcNRUkrNLamdLT2aVPCbD_YbmjDY/edit#gid=850719639 report]
</div></div>
 
=== Release 73 build readiness ===
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">RC1 73.0</div>
<div class="mw-collapsible-content">
* Build: 73.0-build1 ([https://hg.mozilla.org/releases/mozilla-release/rev/2b0af2646bc152c852bbe8effda16188318d6aff  2b0af2646bc152c852bbe8effda16188318d6aff], 20200203203546) - <font style="color:green; font-weight:bold;">GREEN</font>
*** {{mdone|}} Smoke, regression, exploratory → [https://testrail.stage.mozaws.net/index.php?/plans/view/25195 TestRail]
*** {{mdone|}} Smoke testing on Ubuntu snap package → Installation
*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug list]
** '''Update tests (manual)''':
*** {{mdone|}} beta → [https://docs.google.com/spreadsheets/d/1usdmHVC_zMHB_uMmGNtTY-ozR44-md2n3OBn-7SWAFU/edit#gid=850719639 report]
*** {{mdone|}} release-localtest → [https://docs.google.com/spreadsheets/d/1Cl2Cd05y4thFMyXoY_1LrRVXk3h3T3dcl6YuP9Gy5kA/edit report]
*** {{mnotstarted|}} release-cdntest → [https://docs.google.com/spreadsheets/d/1Cl2Cd05y4thFMyXoY_1LrRVXk3h3T3dcl6YuP9Gy5kA/edit report]
*** {{mnotstarted|}} release → [https://docs.google.com/spreadsheets/d/1Cl2Cd05y4thFMyXoY_1LrRVXk3h3T3dcl6YuP9Gy5kA/edit report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">RC2 73.0</div>
<div class="mw-collapsible-content">
* Build: 73.0-build2 ([https://hg.mozilla.org/releases/mozilla-release/rev/870866f45468 870866f45468], 20200207001703) - <font style="color:green; font-weight:bold;">GREEN</font>
*** {{mdone|}} Smoke, regression, exploratory → [https://testrail.stage.mozaws.net/index.php?/plans/view/25351 TestRail]
*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug list]
** '''Update tests (manual)''':
*** {{mdone|}} beta → [https://docs.google.com/spreadsheets/d/1OyseSFBf3sUWvXyxMo5gzkIgrgddaHRpoCSeVhtNskw/edit#gid=850719639 report]
*** {{mdone|}} release-localtest → [https://docs.google.com/spreadsheets/d/1OKpQz0D48JrI0s32XAURrcra5d0R8pwbR2UfNa9tODE/edit#gid=850719639 report]
*** {{mnotstarted|}} release-cdntest → [https://docs.google.com/spreadsheets/d/1OKpQz0D48JrI0s32XAURrcra5d0R8pwbR2UfNa9tODE/edit#gid=850719639 report]
*** {{mnotstarted|}} release → [https://docs.google.com/spreadsheets/d/1OKpQz0D48JrI0s32XAURrcra5d0R8pwbR2UfNa9tODE/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">RC3 73.0</div>
<div class="mw-collapsible-content">
* Build: 73.0-build3 ([https://hg.mozilla.org/releases/mozilla-release/rev/816e7f294781 816e7f294781], 20200207195153) - <font style="color:green; font-weight:bold;">GREEN</font>
*** {{mdone|}} Smoke, regression, exploratory → [https://testrail.stage.mozaws.net/index.php?/reports/view/2440 TestRail]
*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug list]
** '''Update tests (manual)''':
*** {{mdone|}} beta → [https://docs.google.com/spreadsheets/d/1OyseSFBf3sUWvXyxMo5gzkIgrgddaHRpoCSeVhtNskw/edit#gid=850719639 report]
*** {{mnotstarted|}} release-cdntest → [https://docs.google.com/spreadsheets/d/1JAo62JKPvzu6kpeeIiyq7clOnAabCdIlNdzCIVIkkT0/edit#gid=850719639 report]
*** {{mnotstarted|}} release → [https://docs.google.com/spreadsheets/d/1OKpQz0D48JrI0s32XAURrcra5d0R8pwbR2UfNa9tODE/edit#gid=850719639 report]
</div></div>
 
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto; overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">RC1 73.0.1</div>
<div class="mw-collapsible-content">
* Build: 73.0.1-build1 ([https://hg.mozilla.org/releases/mozilla-release/rev/5b0905233e5d29d9fbad98c1380030387cda5dd9 5b0905233e5d], 20200217142647) - <font style="color:green; font-weight:bold;">GREEN</font>
*** {{mdone|}} Smoke, regression, exploratory → [https://testrail.stage.mozaws.net/index.php?/plans/view/25880 TestRail]
*** {{mdone|}} Bug fix verification – [https://docs.google.com/spreadsheets/d/12Dc8SHzJK5E4PoLzuCdsQ5DGx_oKLsajBCR3pm91RXA/edit#gid=1293702786 Bug list]
** '''Update tests (manual)''':
** '''Update tests (manual)''':
*** {{mprog|}} beta, aurora → [https://docs.google.com/spreadsheets/d/1af4R89RIjABPnU1bGXyKxoO7KMbp3uNZEmARxBPWH_0/edit#gid=850719639 report]
*** {{mdone|}} release-cdntest → [https://docs.google.com/spreadsheets/d/18riwBhRp74gjeP_s_2l648xPJMSJ4TbbjZiknCKz2vg/edit#gid=850719639 report]
*** {{mdone|}} release → [https://docs.google.com/spreadsheets/d/18riwBhRp74gjeP_s_2l648xPJMSJ4TbbjZiknCKz2vg/edit#gid=850719639 report]
</div></div>
</div></div>

Latest revision as of 16:05, 19 February 2020

Summary

The purpose of this page is to track the testing of Firefox 73 from Nightly to Release, and post Release if needed.

Release schedule

Status meetings

The Channel Status meeting is used to raise quality concerns with the Release Management team.

QA points of contact

Communication channels

Firefox for Desktop

Engineering QA
  • Reporting to: Tom Grabowski, Tania Maity
  • Softvision Manager: Rares Bologa
  • Team Leads: Brindusa Tot
  • Offices: Romania (EEST), Argentina (GMT-3)
  • Responsible for: new feature testing, Nightly build readiness, Nightly build regression testing, bug triage, bug verification, bug investigation
Release QA
  • Reporting to: Tom Grabowski, Tania Maity
  • Softvision Manager: Andrei Vaida
  • Team Leads: Bogdan Maris, Cornel Ionce
  • Office: Romania (EEST)
  • Responsible for: new feature testing, Beta build readiness, Developer Edition build readiness, ESR build readiness, RC build readiness, Dot release build readiness, Chemspill build readiness, bug verification, bug investigation

Firefox for Mobile

Release QA
Firefox for iOS QA

AMO, WebExtensions

Add-ons QA
  • Reporting to: Krupa Raj
  • Softvision Manager: Rares Bologa
  • Team Leads: Victor Carciu, Madalin Cotetiu
  • Office: Romania (EEST)
  • Responsible for: AMO feature testing, AMO weekly release, AMO Sanity, AMO bug verification, Webextensions features, Webextensions bug triage and verification, Telemetry

Experiments, Normandy, Subscription services, and other Firefox features

Experiments QA
  • Reporting to: Krupa Raj
  • Softvision Manager: Rares Bologa
  • Team Leads: Paul Oiegas
  • Office: Romania (EEST)
  • Responsible for: testing Experiments, Normandy, Experimenter website, Activity Stream, Screenshots, Send (Web & Android) and Lockbox Desktop.

QA activities

New feature readiness

All new features tested by QA and shipping with Firefox 73 are available here.

The feature testing process followed by QA is available here.

Bug fix verification

  • Daily verification of uplifted bugs - these are uplifts, triaged daily and verified under ~48hrs
  • Ongoing verification of Nightly bugs - this is the general bug verification done throughout the Nightly cycle; it sometimes overlaps with uplifts verification; also involves active monitoring of qe-verify? bugs
  • Ongoing verification of Beta bugs - this is the general bug verification done throughout the Beta cycle; it sometimes overlaps with uplifts verification; also involves active monitoring of qe-verify? bugs
  • On-demand verification of bugs - these are special requests (ideally coming in through PI, but also through Bugzilla needinfo?, chat or emails)
  • Periodical verification of security bugs - these are post-critsmash bugs, triaged and verified ~every 2wks

Unconfirmed bug triage

This is the triage conducted by Engineering QA.

Security bug triage

This is the post-CritSmash triage conducted by Engineering QA (on Nightly) and Release QA (on Beta, Release, ESR).

Bug investigation

This is the daily check and investigation of new regressionwindow-wanted bugs, conducted by Engineering QA and Release QA.

Nightly 73 milestones

[early/mid cycle] Nightly build readiness

Scope: assess build readiness for shipping to Beta by testing Nightly periodically and providing a list of issues (if any) that need to be addressed before the merge (more on this).

Nightly 73.0a1(2019-05-30)
  • Build: 73.0a1 (commit, ID) - N.A.
    • [DONE] Functional testing (manual) → [- report]
      • New bugs
        • Regressions: --
        • Non-regressions: --
      • Known bugs
        • High-impact: --
        • Low/medium-impact: [link bug list]
Nightly 73.0a1 (2019-06-13)
  • Build: 73.0a1 (commit, ID) - N.A.
    • [DONE] Functional testing (manual) → report
      • New bugs
        • Regressions: --
        • Non-regressions: --
      • Known bugs
        • High-impact: --
        • Low/medium-impact: [link bug list]

[prior to merge] Nightly regression testing

Nightly 73.0a1 (date)
  • Build: 73.0a1 (commit, ID) - Sign off_Status
    • [NOT STARTED] Regression testing (manual) → [report]
      • New bugs
        • Regressions: ID1, ID2, IDn
        • Non-regressions: ID1, ID2, IDn
      • Known bugs
        • High-impact: ID1, ID2, IDn
        • Low/medium-impact: [bug list]

[optional] [prior to merge] Nightly new feature smoke testing

Note: this is optional, based on available QA bandwidth.

Beta 73 milestones

Beta/DevEdition build readiness

Developer Edition build readiness

DevEd 73.0b1
  • Build: 73.0b1-build1 (ea93a9ceeea1a0b8cacb5c9038e1fa1d00e50bc6, 20200106175743) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Tabbed Browser , OS Integration , User Profile , Form Autofill , Bookmarks → TestRail
DevEd 73.0b5
  • Build: 73.0b5-build1 (087b0cab93cc2c55a84e3fd491160cc183e60bfd, 20200115020958) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Startup and Profile , Build Branding , WebExtensions , Theme and Toolbar Customization , Web Compatibility, Sync & Firefox Account, OS Integration → TestRail

Beta build readiness

73.0b1
73.0b2
  • Build: 73.0b2-build1 (4ec2c9a971e1a9a49b5f24a59343fba245470a91, 20200107212705) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Security and Privacy , Theme and Toolbar Customization, Browser Window, Bookmarks and History, Form Autofill → TestRail
      • [DONE] Bug fix verification
    • Update tests (manual):
      • [DONE] beta, aurora → report
73.0b3
  • Build: 73.0b3-build1 (e592eb8fc8c2b5a7fd971c1430c70de8f058a071, 20200110003145) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Geolocation, Form Autofill, Password Manager 2.0, Preferences , Notifications, Push Notifications and Alerts, Menus → TestRail
      • [DONE] Bug fix verification
    • Update tests (manual):
      • [DONE] beta, aurora → report
73.0b4
73.0b5
73.0b6
73.0b7
73.0b8
  • Build: 73.0b8-build1 (6a9e651e0831, 20200122012721) - GREEN
    • Functional tests (manual):
    • Update tests (manual):
      • [DONE] beta, aurora → report
73.0b9
  • Build: 73.0b9-build1 (3f19dfb367d6, 20200124022546) - GREEN
    • Update tests (manual):
      • [DONE] beta, aurora → report
73.0b10
  • Build: 73.0b10-build1 (38fc7c5ed55f, 20200125104833) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Crash Reporting, Security and Privacy, Images, PDF Viewer, Password Manager and Preferences → TestRail
      • [DONE] Bug fix verification
    • Update tests (manual):
      • [DONE] beta, aurora → report
73.0b11
  • Build: 73.0b11-build1 (c24dfbda267e, 20200128001646) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Password Manager, Data Migration, Sync & Firefox Account and Menus → TestRail
      • [DONE] Bug fix verification
    • Update tests (manual):
      • [DONE] beta, aurora → report
73.0b12
  • Build: 73.0b12-build1 (14ac73797dd8, 20200130174018) - GREEN
    • Functional tests (manual):
      • [DONE] Smoke, Regression and Exploratory testing on Preferences, Menus, Reader View, Find Toolbar and Firefox Screenshots → TestRail
      • [DONE] Bug fix verification
    • Update tests (manual):
      • [IN PROGRESS] beta, aurora → report

Release 73 build readiness

RC1 73.0
RC2 73.0
  • Build: 73.0-build2 (870866f45468, 20200207001703) - GREEN
      • [DONE] Smoke, regression, exploratory → TestRail
      • [DONE] Bug fix verification – Bug list
    • Update tests (manual):
      • [DONE] beta → report
      • [DONE] release-localtest → report
      • [NOT STARTED] release-cdntest → report
      • [NOT STARTED] release → report
RC3 73.0
  • Build: 73.0-build3 (816e7f294781, 20200207195153) - GREEN
      • [DONE] Smoke, regression, exploratory → TestRail
      • [DONE] Bug fix verification – Bug list
    • Update tests (manual):
      • [DONE] beta → report
      • [NOT STARTED] release-cdntest → report
      • [NOT STARTED] release → report
RC1 73.0.1
  • Build: 73.0.1-build1 (5b0905233e5d, 20200217142647) - GREEN
      • [DONE] Smoke, regression, exploratory → TestRail
      • [DONE] Bug fix verification – Bug list
    • Update tests (manual):
Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_73/Test_Plan&oldid=1223975"