Confirmed users
1,477
edits
Add-ons/QA/Testplan/OOP-WebExtensions: Difference between revisions
→Channel dependent settings (configs) and environment setups
No edit summary |
|||
| (11 intermediate revisions by 2 users not shown) | |||
| Line 8: | Line 8: | ||
|- | |- | ||
| 04/10/2017 || 1.1 || Krupa Raj || Updated test strategy | | 04/10/2017 || 1.1 || Krupa Raj || Updated test strategy | ||
|- | |||
| 05/19/2017 || 1.2 || Vasilica Mihasca || Updates | |||
|} | |} | ||
| Line 42: | Line 44: | ||
* Webextensions running in their own process | * Webextensions running in their own process | ||
* Regression testing for the top 20 webextensions | * Regression testing for the top 20 webextensions | ||
* Regression testing for webextensions examples | |||
* Regression testing for old XUL add-ons | |||
* Webextension crashing | |||
* Add-ons behavior after a Firefox crash | |||
===what's out of scope?=== | ===what's out of scope?=== | ||
| Line 57: | Line 63: | ||
== Environments == | == Environments == | ||
OSed covered: Windows, Mac OS X, Linux | OSed covered: Windows, Mac OS X, Linux | ||
== Channel dependent settings (configs) and environment setups == | |||
<div class="toccolours mw-collapsible mw-collapsed" style="width:auto"> | |||
=== Nightly === | |||
<div class="mw-collapsible-content"> | |||
* Disable check for signing if testing on AMO dev or stage | |||
* Webextension process crashing: | |||
** Set "extensions.webextensions.remote” to true | |||
** Insert the following line in Browser Console input filed: Cu.import("resource://gre/modules/Extension.jsm").GlobalManager.extensionMap.values().next().value.parentMessageManager.loadProcessScript("data:,Components.utils.import('resource://gre/modules/ctypes.jsm').ctypes.cast(ctypes.uintptr_t(1), ctypes.int.ptr).contents", false) | |||
</div> | |||
=== Beta === | |||
<div class="mw-collapsible-content"> | |||
* [Unbranded build] Disable check for signing if testing on AMO dev or stage | |||
</div> | |||
=== Post Beta / Release === | |||
<div class="mw-collapsible-content"> | |||
The feature is enabled by default. | |||
</div> | |||
</div> | |||
== Test Objectives == | == Test Objectives == | ||
| Line 64: | Line 93: | ||
{| class="wikitable" | {| class="wikitable" | ||
|- | |- | ||
! Ref !! Function !! Test Objective | ! Ref !! Function !! Test Objective !! Test Type !! Owners | ||
|- | |||
| TO-1 || Installing with e10s enabled || To verify that installation process runs without any issue || Manual || Add-ons QA Team | |||
|- | |||
| TO-2 || Webextensions running in their own process || To verify that webextensions run in separate process || Manual || Add-ons QA Team | |||
|- | |||
| TO-3 || Regression testing for the top 20 webextensions || To verify that the most popular webextensions works without issues || Manual || Add-ons QA Team | |||
|- | |||
| TO-4 || Regression testing for webextensions example master || To verify that the webextensions works without issues || Manual || Add-ons QA Team | |||
|- | |||
| TO-5 || Webextensions testing in Private Window|| To verify the webextension behavior in private winodow || Manual || Add-ons QA Team | |||
|- | |||
| TO-6 || Webextension crashing || To verify that crashing the webextension process does not cause a Firefox crash || Manual || Add-ons QA Team | |||
|- | |||
| TO-7 || Firefox crashing || To verify that Firefox will load all the add-ons properly without any loss of saved add-on data || Manual || Add-ons QA Team | |||
|- | |- | ||
| TO-8 || Regression testing on XUL add-ons || To verify that the old XUL add-ons works without any issue || Manual || Add-ons QA Team | |||
|} | |} | ||
== Test Strategy == | |||
Here are the main ways in which we will ascertain that OOP functions as expected | |||
* Test all the webextensions listed at https://github.com/mdn/webextensions-examples and ensure there are no regressions | |||
* Check webextensions which are using APIs which are most affected by OOP. These APIs include - | |||
** context Menus | |||
** devtools | |||
** omnibox | |||
** tabs | |||
** identity | |||
** permissions | |||
* Check that a firefox doesn't crash if a webextension crashes. See [https://bugzilla.mozilla.org/show_bug.cgi?id=1353959 Bug 1353959] for more details | |||
* Check that extensions work as expected after a firefox crash | |||
webRequest | |||
== Builds == | == Builds == | ||
This section should contain links for builds with the feature - | This section should contain links for builds with the feature - | ||
* [ | * [http://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/ Link] for Nightly builds | ||
* [https://archive.mozilla.org/pub/firefox/candidates/ Link] for Beta builds | * [https://archive.mozilla.org/pub/firefox/candidates/ Link] for Beta builds | ||
* Beta Unbranded builds | |||
** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-win64-add-on-devel/ Windows Link] | |||
** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-linux64-add-on-devel/ Linux Link] | |||
** [https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-macosx64-add-on-devel/ Mac Link] | |||
== Test Execution Schedule == | == Test Execution Schedule == | ||
| Line 93: | Line 154: | ||
|- | |- | ||
| QA - Nightly Testing | | QA - Nightly Testing | ||
|style="text-align:center;" | | |style="text-align:center;" | 03-22-2017|| | ||
|- | |- | ||
| QA - Beta Testing | | QA - Beta Testing | ||
| Line 113: | Line 171: | ||
| Test plan creation || Mozilla wiki | | Test plan creation || Mozilla wiki | ||
|- | |- | ||
| Test case creation || [ TestRail] / [Google docs] / [etherpad] | | Test case creation || [ TestRail] / [Google docs] / [https://public.etherpad-mozilla.org/p/OOP-WebExtension-Testing etherpad] | ||
|- | |- | ||
| Test case execution || [TestRail] | | Test case execution || [TestRail] | ||