QA/Fennec/Milestones/5.0: Difference between revisions

From MozillaWiki
< QA‎ | Fennec‎ | Milestones
Jump to navigation Jump to search
 
(43 intermediate revisions by 3 users not shown)
Line 3: Line 3:
== Overview ==
== Overview ==


[[QA/Fennec/TestStrategy| Fennec Test Strategy]]
The goal of this Overall Test Plan is to lay out testing strategy, results, and tracking for each milestone as it crosses through each channel of the Firefox development process.
 
The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.


All of our releases are tested off of the "Fennec 5.0 Catch-All Test Run" on the [http://litmus.mozilla.org/ Litmus TCM].
All of our releases are tested off of the "Fennec 5.0 Catch-All Test Run" on the [http://litmus.mozilla.org/ Litmus TCM].


For more information about Fennec 5.0, take a look at the mobile team's [[Mobile/Planning/6.0|Feature Planning page]]
For more information about Fennec 5.0, take a look at the mobile team's [[Firefox/Flight_Tracking#Firefox_5|Feature Planning page]]


=== Current Blocking Bugs List ===
=== Bugs in Release ===


* [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking-fennec%3A%2B fennec blockers]
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=247247&field0-0-0=cf_blocking_fennec&resolution=FIXED&query_format=advanced&bug_status=RESOLVED&bug_status=VERIFIED&type0-0-0=substring&value0-0-0=%2B&target_milestone=Firefox%205&target_milestone=mozilla5 Bugs FIXED]
* [http://blocker-reports.brasstacks.mozilla.com/mobile.html blocker trends]


=== Build Location ===
== General Responsibilities ==
=== Nightly ===


Fennec 5.0 is currently built from trunk (mozilla-central) as well as the front end mobile-browser repositories.  Nightly builds for all platforms are located at: http://ftp.mozilla.org/pub/mozilla.org/mobile/nightly/latest-mobile-trunk/
{|width=100%
|-
|style="background-color: #efefef;"|'''Test Plan and Results'''
|style="background-color: #efefef;"|'''Start Date'''
|-
| style="background-color: #FEF1B5;"|[[QA/Fennec/Milestones/5.0a1|Nightly]]
| style="background-color: #FEF1B5;"|---
|}


== Releases Tested ==
{|width=100%
|-
| style="background-color: #efefef;"|'''QA Task'''
| style="background-color: #efefef;"|'''Person/Group'''
| style="background-color: #efefef;"|'''How Often?'''
|-
| Smoketests
| Waverley
| Daily
|-
| BFTs
| Waverley/Feature QA
| Weekly
|-
| OS Integration
| Waverley
| Weekly
|-
| Bug Verification
| nhirata/Andreea
| Daily
|-
| Crasher Triage
| mw22
| Daily
|-
| Bug Triage
| MoCo team
| Weekly
|-
| Testday
| kbrosnan
| Once
|}


* Aurora 1 - ??? - [[QA/Fennec/Milestones/5.0/ReleaseTest|Release Test Results]]
=== Aurora ===
{|width=100%
|-
|style="background-color: #efefef;"|'''Test Plan and Results'''
|style="background-color: #efefef;"|'''Start Date'''
|-
| style="background-color: #FEF1B5;"|[[QA/Fennec/Milestones/5.0a2|Aurora]]
| style="background-color: #FEF1B5;"|4/11
|}


== Test Areas ==
{|width=100%
|-
| style="background-color: #efefef;"|'''QA Task'''
| style="background-color: #efefef;"|'''Person/Group'''
| style="background-color: #efefef;"|'''How Often?'''
|-
| Smoketests
| Waverley/Feature QA
| Weekly
|-
| Bug Verification
| nhirata/Andreea
| As-needed
|}


=== General Test Areas ===
=== Beta ===
{| class="standard-table"
{|width=100%
|-
|-
|class="header"|Status
|style="background-color: #efefef;"|'''Test Plan and Results'''
|class="header"|Action to Take
|style="background-color: #efefef;"|'''Start Date'''
 
|-
|-
|Testcases added
| style="background-color: #FEF1B5;"|[[QA/Fennec/Milestones/5.0b1|Beta]]
|
| style="background-color: #FEF1B5;"|5/17
* Test and re-test on a daily basis
|- style="background:#CD5555;"
|Landed
|
* Add testcases to Litmus
* Begin filing/verifying bugs
|- style="background:#FFCC99"
|Development started
|
* Contact the developer
* Build the test plan
|- style="background:#DFFFA5;"
|Feature is being planned
|
* Stay in a holding pattern
|}
|}


==== Features  ====
{|width=100%
 
|-
{| class="standard-table sortable"
| style="background-color: #efefef;"|'''QA Task'''
| style="background-color: #efefef;"|'''Person/Group'''
| style="background-color: #efefef;"|'''How Often?'''
|-
| Smoketests
| Waverley
| Once
|-
|-
| class="header" | Feature Area and Spec
| Crasher Triage
| class="header" | Litmus Testcases Needed/Added
| class="header" | QA Contact
| class="header" | Dev Contact
| class="header" | Links
|- style="background:#CD5555;"
| Do Not Track
| Yes/No
| AaronMT
| mbrubeck
| [https://bugzilla.mozilla.org/show_bug.cgi?id=648654 Bug 648654]
|- style="background:#CD5555;"
| Improve position:fixed support
| No/No
| mw22
| mw22
| romaxa
| Daily
| [https://bugzilla.mozilla.org/show_bug.cgi?id=607417 Bug 607417]
|-
|- style="background:#CD5555;"
| L10n Tier 1 Locale Check
| Tell Android to scan downloaded media
| No/No
| nhirata
| nhirata
| wesj
| Once
| [https://bugzilla.mozilla.org/show_bug.cgi?id=646550 Bug 646550]
|-
|- style="background:#CD5555;"
| Device Compatibility
| Handle option overflow in Android menu
| AaronMT
| No/No
| Once
| ashah
|-
| wesj
| Website Compatibility
| [https://bugzilla.mozilla.org/show_bug.cgi?id=610784 Bug 610784]
| AaronMT
|- style="background:#CD5555;"
| Once
| Fix broken color conversion
|-
| No/No
| Testday
| Andreea Pod
| derf
| [https://bugzilla.mozilla.org/show_bug.cgi?id=641014 Bug 641014]
|- style="background:#CD5555;"
| Add remoteXUL strings for Fennec
| No/No
| Anna Moldovan
| volkmar
| [https://bugzilla.mozilla.org/show_bug.cgi?id=648268 Bug 648268]
|- style="background:#CD5555;"
| Slightly improve page load performance
| No/No
| nhirata
| ehsan
| [https://bugzilla.mozilla.org/show_bug.cgi?id=561694 Bug 561694]
|- style="background:#CD5555;"
| Dialogs/Prompts look like Android dialogs
| No/No
| Anna Moldova
| wesj
| [https://bugzilla.mozilla.org/show_bug.cgi?id=618989 Bug 618989]
|- style="background:#CD5555;"
| Consecutive pans should not be jumpy
| No/No
| kbrosnan
| kbrosnan
| stechz
| Once
| [https://bugzilla.mozilla.org/show_bug.cgi?id=639179 Bug 639179]
|- style="background:#CD5555;"
| No more fullscreen landscape keyboard
| Yes/Yes
| nhirata
| blassey
| [https://bugzilla.mozilla.org/show_bug.cgi?id=614355 Bug 614355]
|}
|}


=== Automation ===
== Features Tracked ==
QA is undertaking the task of porting the unit tests from Firefox to Fennec for each platform. This is a lengthy task, but our goal is to have 100% unit tests passing by the 5.0 ship.


We have created a [https://wiki.mozilla.org/Mobile/Fennec_Automation tracking system] for the unit tests which we update on a regular basis.
{| class="fullwidth-table sortable" style="width:100%;"
|-
| style="font-weight: bold; background: #DDD;" | Feature
| style="font-weight: bold; background: #DDD;" | Status
| style="font-weight: bold; background: #DDD;" | Owner
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=607417 Optimized layout and interaction with some mobile web pages (improved position:fixed support)]
| landed
SDR: N &#124;&#124; SIR: N
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=646550 Add images downloaded in Firefox to the Android Gallery immediately (before: images were visible only after reboot]
| landed
SDR: N &#124;&#124; SIR: <span style="font-weight: bold;">C</span> - 04/27
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=610784 Add a "More" button to the Android Menu when needed)]
| landed
SDR: N &#124;&#124; SIR: N
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=641014 Fix issue with color conversion for better display of images and video (Fix broken NEON Y'CbCr to RGB conversion)]
| landed
SDR: N &#124;&#124; '''SIR: P&nbsp;'''
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=648268 Resolves recent breakage that would show an "undefined" error when trying to load web pages while outside of wifi or network coverave (Add remoteXUL strings for Firefox Mobile)]
| landed
SDR: N &#124;&#124; SIR: N
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=561694 Slight improvement of page load speed, especially on 3G network (e10s HTTP: Implement nsIProgressEventSink:part2)]
| landed
SDR: N &#124;&#124; SIR: N
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=648654 Add user-visible pref for Do Not Track]  
SDR: N &#124;&#124; SIR: N
QA: Verified/Tests Created
| landed
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=618989 Finish new theme styles for dialogs/prompts (prompts and confirms look like android dialogs)]
| landed
SDR: N &#124;&#124; '''SIR: C''' - 04-27
QA: Verified
| Thomas
|-
| [https://bugzilla.mozilla.org/show_bug.cgi?id=639179 Smooth panning (resolves an issue where consecutive pans were a bit jumpy before)]
| landed
SDR: N &#124;&#124; SIR: N
QA: Verified
| Thomas
|}

Latest revision as of 22:59, 17 May 2011

« Mobile

Overview

The goal of this Overall Test Plan is to lay out testing strategy, results, and tracking for each milestone as it crosses through each channel of the Firefox development process.

All of our releases are tested off of the "Fennec 5.0 Catch-All Test Run" on the Litmus TCM.

For more information about Fennec 5.0, take a look at the mobile team's Feature Planning page

Bugs in Release

General Responsibilities

Nightly

Test Plan and Results Start Date
Nightly ---
QA Task Person/Group How Often?
Smoketests Waverley Daily
BFTs Waverley/Feature QA Weekly
OS Integration Waverley Weekly
Bug Verification nhirata/Andreea Daily
Crasher Triage mw22 Daily
Bug Triage MoCo team Weekly
Testday kbrosnan Once

Aurora

Test Plan and Results Start Date
Aurora 4/11
QA Task Person/Group How Often?
Smoketests Waverley/Feature QA Weekly
Bug Verification nhirata/Andreea As-needed

Beta

Test Plan and Results Start Date
Beta 5/17
QA Task Person/Group How Often?
Smoketests Waverley Once
Crasher Triage mw22 Daily
L10n Tier 1 Locale Check nhirata Once
Device Compatibility AaronMT Once
Website Compatibility AaronMT Once
Testday kbrosnan Once

Features Tracked

Feature Status Owner
Optimized layout and interaction with some mobile web pages (improved position:fixed support) landed

SDR: N || SIR: N QA: Verified

Thomas
Add images downloaded in Firefox to the Android Gallery immediately (before: images were visible only after reboot landed

SDR: N || SIR: C - 04/27 QA: Verified

Thomas
Add a "More" button to the Android Menu when needed) landed

SDR: N || SIR: N QA: Verified

Thomas
Fix issue with color conversion for better display of images and video (Fix broken NEON Y'CbCr to RGB conversion) landed

SDR: N || SIR: P  QA: Verified

Thomas
Resolves recent breakage that would show an "undefined" error when trying to load web pages while outside of wifi or network coverave (Add remoteXUL strings for Firefox Mobile) landed

SDR: N || SIR: N QA: Verified

Thomas
Slight improvement of page load speed, especially on 3G network (e10s HTTP: Implement nsIProgressEventSink:part2) landed

SDR: N || SIR: N QA: Verified

Thomas
Add user-visible pref for Do Not Track

SDR: N || SIR: N QA: Verified/Tests Created

landed Thomas
Finish new theme styles for dialogs/prompts (prompts and confirms look like android dialogs) landed

SDR: N || SIR: C - 04-27 QA: Verified

Thomas
Smooth panning (resolves an issue where consecutive pans were a bit jumpy before) landed

SDR: N || SIR: N QA: Verified

Thomas
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Fennec/Milestones/5.0&oldid=309814"