Revision as of 07:12, 4 June 2013

«Fennec 22 Beta 4 | Fennec 22 Beta 5 | Fennec 22 Beta 6 »

Fennec (Firefox for Android) 22.0 Beta 5 Test Plan

Release Week: June 10th, 2012
QA Contact: Aaron Train; Kevin Brosnan
IRC: irc://irc.mozilla.org#qa ping AaronMT; kbrosnan; ioanachiorean;
Build 1: Fennec (Firefox for Android) 22 Beta 45 build 1 | changeset | pushlog

Features

See the Feature Release Tracking wiki for a general overview of this release. Found below are a subset of all high-priority items

Feature	Owner	Status	Tests
Added locale: Slovak		[ON TRACK]	QA Feature Tracking & MozTrap References & Product Features
ANR Reporter - Not user facing		[ON TRACK]	QA Feature Tracking & MozTrap References & Product Features
Refining Tab UI		[ON TRACK]	QA Feature Tracking & MozTrap References & Product Features
Performance improvements - Including WebGL, texture streaming, async canvas updates		[ON TRACK]	QA Feature Tracking & MozTrap References & Product Features

Smoketests

QA Task	Person/Group	How Often?
Smoketests	Softvision	Every Beta

Light verification tests (or smoketests) will be first run when the builds are ready. Tests are available on Mozilla Moztrap under the 18.0 smoketest suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Assigned	Device	Completed	Results
Adrian Tamas	LG Optimus ARMv6 (Android 2.3.4)	June 13th, 2013	PASS
Adrian Tamas	LG Nexus 4 (Android 4.2.2)	June 13th, 2013	PASS
Paul Feher	Asus Transformer TF101 (Android 4.0.4)	June 13th, 2013	PASS
Aaron Train	Motorola XT890 (Android 4.1)	June 14th, 2013	PASS

Detailed Smoketest results

Exploratory & BFTs Testing

QA Task	Person/Group	How Often?
Exploratory Testing	Softvision	Alternating - done in Beta 1

Assigned	Device	Completed	Results
Paul Feher	Asus Eee Transformers (Android 4.0.4)	June 13th, 2013	PASS
Ioana Chiorean	Acer Iconia A500 (Android 3.2.1)	June 13th, 2013	PASS
Mihai Pop	Samsung Galaxy R (Android 2.3.4)	June 14th, 2013	PASS
Catalin Suciu	Samsung Galaxy SII (Android 4.0.3)	June 14th, 2013	PASS
Teodora Vermesan	Samsung Galaxy Tab (Android 4.0.4)	June 13th, 2013	PASS
Adrian Tamas	LG Nexus 4 (Android 4.2)	June 14th, 2013	PASS

Detailed Exploratory & BFTs results

Android Permissions Testing

QA Task	Person/Group	How Often?
Verification	Softvision	Every Beta

Assigned	Device	Completed	Results
Ioana Chiorean	Acer Iconia A500 (Android 3.2.1)	June 13th , 2013	Pass
Adrian Tamas	LG Optimus Slider ARMv6 (Android 2.3.4)	June 13th, 2013	PASS
Adrian Tamas	LG Nexus 4 (Android 4.2.2)	June 13th, 2013	PASS

Detailed Android Permissions Testing Results
Verify permissions between what is currently on the store and with this beta build candidate.
Details: Android Permissions

Branch Bug Fix Verifications

QA Task	Person/Group	How Often?
Verification	Softvision	Every Beta

L10N (Localization) Spot Checks

QA Task	Person/Group	How Often?
Strings Verification	SoftVision	Every Beta

Light testing on featured locales to ensure strings fit the space allotted and there are no entity errors from missing or corrupt localizations. QA/Fennec/22/Beta/5/L10N-strings matrix Detailed L10N matrix

QA Task	Person/Group	How Often?
Builds Verification	SoftVision	Every Beta

Light testing on featured locales to ensure strings fit the space allotted and there are no entity errors from missing or corrupt localizations.

Assigned	Device	Completed	Results
Teodora Vermesan	LG nexus 4 (Android 4.2.2)	June 13th, 2013	PASS

Detailed L10N matrix

AppThwack Automation

QA Task	Person/Group	How often?
Runs against ARMv6 and ARMv7	Ioana	Alternating

Automated installation, uninstallation, and Android Monkeyrunner tests against uploaded ARM6 and ARMv7 builds; primarily useful for detecting startup crashes, incompatibilities, install-times and UI oddities

Assigned	Build	Completed	Results
Ioana Chiorean	fennec-22.0b5.multi.android-arm-armv6.apk	June 13th, 2013	not run
Catalin Suciu	fennec-22.0b5.multi.android-arm.apk	June 13th, 2013	PASS

Detailed AppThwack Results

@@ Line 1: / Line 1: @@
-== QA Assignment ==
+__NOTOC__
-<onlyinclude>
+<small> «[[QA/Fennec/22/Beta/4|Fennec 22 Beta 4]] | [[QA/Fennec/22/Beta/5|Fennec 22 Beta 5]] | [[QA/Fennec/22/Beta/6|Fennec 22 Beta 6]] »</small>
-{| class="fullwidth-table"
+= Fennec (Firefox for Android) 22.0 Beta 5 Test Plan =
-  |- style="background-color: #efefef;"
+* Release Week: June 10th, 2012
-  | Assigned
+* QA Contact: [mailto:atrain@mozilla.com Aaron Train]; [mailto:kbrosnan@mozilla.com Kevin Brosnan]
-  | Device
+* IRC: irc://irc.mozilla.org#qa ping AaronMT; kbrosnan; ioanachiorean;
-  | Completed
+* Build 1: [ftp://ftp.mozilla.org/pub/mozilla.org/mobile/candidates/22.0b4-candidates/build1/ Fennec (Firefox for Android) 22 Beta 45 build 1] | [https://hg.mozilla.org/releases/mozilla-beta/rev/   changeset] | [https://hg.mozilla.org/releases/mozilla-beta/pushloghtml?fromchange=FENNEC_22_0b4_RELEASE&tochange=FENNEC_22_0b5_RELEASE pushlog]
- | Results
+=== Features ===
+''See the [https://wiki.mozilla.org/Mobile/Fx22 Feature Release Tracking] wiki for a general overview of this release. Found below are a subset of all high-priority items''
+{| class="querytable"
+  | class="header" | Feature
+  | class="header" | Owner
+  | class="header" | Status
+  | class="header" | Tests
   |-
+ | [[Mobile/Projects/Localization | Added locale: Slovak]]
   |
-  | LG Optimus ARMv6 (Android 2.3.4)
+  | {{mok}}
-  | th, 2013
+  | [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdGVKQlRHUnFWVEZXNFFJbHo0RVpQMmc#gid=0 QA Feature Tracking & MozTrap References] & [https://wiki.mozilla.org/Mobile/Roadmap#Firefox_22:_Simple_Pleasures_.28Beta.29 Product Features]
- | '''  '''
- |-
- |
- | Galaxy Nexus (Android 4.2.2)
- |  , 2013
- | '''  '''
- |-
- |
- | Acer Iconia A500 (Android 3.2.1)
- |  th , 2013
- | ''' '''
   |-
+ | [[Mobile/Projects/ANR | ANR Reporter]] - Not user facing
   |
-  | Samsung Galaxy R (Android 2.3.4)
+  | {{mok}}
-  |  th , 2013
+  | [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdGVKQlRHUnFWVEZXNFFJbHo0RVpQMmc#gid=0 QA Feature Tracking & MozTrap References] & [https://wiki.mozilla.org/Mobile/Roadmap#Firefox_22:_Simple_Pleasures_.28Beta.29 Product Features]
- | ''' '''
   |-
+ | [[Mobile/Projects/New tab UI | Refining Tab UI]]
   |
-  | Asus Transformer TF101 (Android 4.0.4)
+  | {{mok}}
- |  th, 2013
+ | [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdGVKQlRHUnFWVEZXNFFJbHo0RVpQMmc#gid=0 QA Feature Tracking & MozTrap References] & [https://wiki.mozilla.org/Mobile/Roadmap#Firefox_22:_Simple_Pleasures_.28Beta.29 Product Features]
- | '''  '''
   |-
+ | [[Mobile/Roadmap/Tracking_Fx22#Performance_improvements | Performance improvements]] - Including WebGL, texture streaming, async canvas updates
   |
-  | Samsung Galaxy Tab 2 (Android 4.1.1)
+  | {{mok}}
-  |  th, 2013
+  | [https://docs.google.com/spreadsheet/ccc?key=0AhE7m4JB2j6tdGVKQlRHUnFWVEZXNFFJbHo0RVpQMmc#gid=0 QA Feature Tracking & MozTrap References] & [https://wiki.mozilla.org/Mobile/Roadmap#Firefox_22:_Simple_Pleasures_.28Beta.29 Product Features]
- | '''  '''
   |}
-</onlyinclude>
-== Results ==
+=== Smoketests ===
-=== Phone ===
+{|width=100%
-*[https://moztrap.mozilla.org/results/cases/?filter-run=1441 MozTrap smoketest results for ARMv6]
+|-
-*[https://moztrap.mozilla.org/results/cases/?filter-run=1442 MozTrap smoketest results for ARMv7]
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|'''How Often?'''
+|-
+| Smoketests
+| Softvision
+| Every Beta
+|}
+Light verification tests (or smoketests) will be first run when the builds are ready. Tests are available on [https://moztrap.mozilla.org/manage/cases/ Mozilla Moztrap] under the [https://moztrap.mozilla.org/results/cases/?filter-run= 18.0 smoketest suite]. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.
+{{:QA/Fennec/22/Beta/5/smoketests-results}}
+[[QA/Fennec/22/Beta/5/smoketests-results|Detailed Smoketest results]]
+=== Exploratory & BFTs Testing ===
+{|width=100%
+|-
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|'''How Often?'''
+|-
+| Exploratory Testing
+| Softvision
+| Alternating - done in Beta 1
+|}
+{{:QA/Fennec/22/Beta/5/exploratory-bfts-results}}
+[[QA/Fennec/22/Beta/5/exploratory-bfts-results|Detailed Exploratory & BFTs results]]
+=== Android Permissions Testing ===
+{|width=100%
+|-
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|'''How Often?'''
+|-
+| Verification
+| Softvision
+| Every Beta
+|}
+{{:QA/Fennec/22/Beta/5/permissions-results}}
+[[QA/Fennec/22/Beta/5/permissions-results|Detailed Android Permissions Testing Results]]
+<br/>''' Verify permissions between what is currently on the store and with this beta build candidate'''.
+<br/>Details: [[QA/Fennec/Permissions|Android Permissions]]
-*'''New'''
+=== Branch Bug Fix Verifications ===
-*'''New to this Beta'''
+{|width=100%
-*'''Known Issues'''
+|-
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|'''How Often?'''
+|-
+| Verification
+| Softvision
+| Every Beta
+|}
+* '''[http://bit.ly/Fixed22 Firefox for Android (21) Bugs]'''
+* '''[http://bit.ly/fx22-verify Firefox 21- Verification Query URL]'''
-=== Tablet ===
+=== L10N (Localization) Spot Checks ===
-* [https://moztrap.mozilla.org/results/cases/?filter-run=1443 MozTrap smoketest results for tablet]
+{|width=100%
+|-
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|'''How Often?'''
+|-
+| Strings Verification
+| SoftVision
+| Every Beta
+|}
+Light testing on featured locales to ensure strings fit the space allotted and there are no entity errors from missing or corrupt localizations.
+{{:QA/Fennec/22/Beta/5/L10N-strings matrix}}
+[[QA/Fennec/22/Beta/5/L10N-strings matrix|Detailed L10N matrix]]
-*'''New'''
+{|width=100%
+|-
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|'''How Often?'''
+|-
+| Builds Verification
+| SoftVision
+| Every Beta
+|}
+Light testing on featured locales to ensure strings fit the space allotted and there are no entity errors from missing or corrupt localizations.
+{{:QA/Fennec/22/Beta/5/L10N-builds matrix}}
+[[QA/Fennec/22/Beta/5/L10N-builds matrix|Detailed L10N matrix]]
-*'''New to this Beta'''
+=== AppThwack Automation ===
-*'''Known Issues'''
+{|width=100%
+|-
+| style="background-color: #efefef;"|'''QA Task'''
+| style="background-color: #efefef;"|'''Person/Group'''
+| style="background-color: #efefef;"|''How often?''
+|-
+| Runs against ARMv6 and ARMv7
+| Ioana
+| Alternating
+|}
+Automated installation, uninstallation, and Android Monkeyrunner tests against uploaded ARM6 and ARMv7 builds; primarily useful for detecting startup crashes, incompatibilities, install-times and UI oddities
+{{:QA/Fennec/22/Beta/5/AppThwack Results}}
+[[QA/Fennec/22/Beta/5/AppThwack Results|Detailed AppThwack Results]]

QA/Fennec/22/Beta/5: Difference between revisions

Revision as of 07:12, 4 June 2013

Fennec (Firefox for Android) 22.0 Beta 5 Test Plan

Features

Smoketests

Exploratory & BFTs Testing

Android Permissions Testing

Branch Bug Fix Verifications

L10N (Localization) Spot Checks

AppThwack Automation

Navigation menu

QA/Fennec/22/Beta/5: Difference between revisions

Revision as of 07:12, 4 June 2013

Fennec (Firefox for Android) 22.0 Beta 5 Test Plan

Features

Smoketests

Exploratory & BFTs Testing

Android Permissions Testing

Branch Bug Fix Verifications

L10N (Localization) Spot Checks

AppThwack Automation

Navigation menu

Search