QA/Fennec/Milestones/5.0b1: Difference between revisions
| (24 intermediate revisions by 5 users not shown) | |||
| Line 111: | Line 111: | ||
*Device: Samsung Google Nexus S (Android 2.3) | *Device: Samsung Google Nexus S (Android 2.3) | ||
*Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0 | *Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0 | ||
*Results Fennec 5.0b2 build 1 : | *Results Fennec 5.0b2 build 1 : '''PASS''' | ||
Known bugs: | Known bugs: | ||
*{{bug|628269}} - Pages are zoomed in at reload | *{{bug|628269}} - Pages are zoomed in at reload | ||
*{{bug|639504}} - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made | *{{bug|639504}} - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made | ||
*{{bug|625981}} - places.sqlite-wal takes too much space for Fennec | *{{bug|625981}} - places.sqlite-wal takes too much space for Fennec | ||
=== Virtual Machine (DeviceAnywhere)=== | |||
{|width=100% | |||
|- | |||
| style="background-color: #efefef;"|'''QA Task''' | |||
| style="background-color: #efefef;"|'''Person/Group''' | |||
| style="background-color: #efefef;"|'''How Often?''' | |||
|- | |||
| DeviceAnywhere Studio 5.1 | |||
| IoanaChiorean | |||
| Once | |||
|} | |||
Tests are available on litmus under the [https://litmus.mozilla.org/run_tests.cgi?test_run_id=195 Fennec 2.0 Test Suite]. | |||
Results details can be found in [https://wiki.mozilla.org/QA/Fennec/Waverley#Results Waverley's results wiki]. | |||
Results: | |||
*Device: Mototrola Droid X (MB810)( (Android 2.3) | |||
*Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0 | |||
*Results Fennec 5.0b2 build 1 : '''PASS''' | |||
'''New issues''': | |||
*'''strange line in the left part (as if the right panel is overlaying the page) - [https://picasaweb.google.com/116017172139610272348/WORK#5608455352085456978 screenshot ] - should be posted bugs also for DeviceAnywhere?''' | |||
*'''{{Bug|658276}} - Notification for Geolocation is not dismissed after choosing option Share''' | |||
Known bugs: | |||
* all bugs found on Smoketest or BFTs | |||
=== BFTs for 5.0 Features === | === BFTs for 5.0 Features === | ||
| Line 135: | Line 161: | ||
*Device: HTC Desire Z (Android 2.2) and LG Optimus 2X (Android 2.2) | *Device: HTC Desire Z (Android 2.2) and LG Optimus 2X (Android 2.2) | ||
*Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0 | *Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0 | ||
*Results: | *Results: '''PASS''' | ||
* | *[[QA/Fennec/Milestones/5.0b1/BFT_notes | Notes: Fennec 5.0b2 build 1]] | ||
=== Crasher Triage === | === Crasher Triage === | ||
| Line 184: | Line 173: | ||
|- | |- | ||
| Crasher Triage | | Crasher Triage | ||
| | | nhirata | ||
| Daily | | Daily | ||
|} | |} | ||
[http://crash-stats.mozilla.com/products/Fennec Socorro] is Mozilla's primary web interface for triaging crash reports sent in by our user base. Due to the relatively large amount of changes coming in on a nightly basis, there is a need to check for crash reports coming in for every build. The owner of this task will triage through top crashers on a daily basis, file bugs if necessary and find steps to reproduce for those crashers if possible. | [http://crash-stats.mozilla.com/products/Fennec Socorro] is Mozilla's primary web interface for triaging crash reports sent in by our user base. Due to the relatively large amount of changes coming in on a nightly basis, there is a need to check for crash reports coming in for every build. The owner of this task will triage through top crashers on a daily basis, file bugs if necessary and find steps to reproduce for those crashers if possible. | ||
Results: | |||
https://wiki.mozilla.org/index.php?title=QA/Fennec/crash_rep_25may | |||
=== Device Compatibility === | === Device Compatibility === | ||
| Line 219: | Line 211: | ||
As part of our testing cycle we want to cover and include a general testing process that focuses on website compatbility with sites viewed in mobile Firefox. We want you to use mobile Firefox to assist us in identifying any noticeably major issues found in websites. Issues can range from pages that do not look right, layout and positioning problems, interactivity issues, font issues and pages that grealy differ from browser to browser and what you would expect. You can help us by completing a website exploration form [http://bit.ly/mobile-firefox-website-compatibility here], and following the companion instructional [https://wiki.mozilla.org/QA/Fennec/WebsiteCompatibility test-plan]. | As part of our testing cycle we want to cover and include a general testing process that focuses on website compatbility with sites viewed in mobile Firefox. We want you to use mobile Firefox to assist us in identifying any noticeably major issues found in websites. Issues can range from pages that do not look right, layout and positioning problems, interactivity issues, font issues and pages that grealy differ from browser to browser and what you would expect. You can help us by completing a website exploration form [http://bit.ly/mobile-firefox-website-compatibility here], and following the companion instructional [https://wiki.mozilla.org/QA/Fennec/WebsiteCompatibility test-plan]. | ||
=== Web Properties Review === | |||
{| width="100%" | |||
|- | |||
| style="background-color: #efefef;" | '''QA Task''' | |||
| style="background-color: #efefef;" | '''Person/Group''' | |||
| style="background-color: #efefef;" | '''How Often?''' | |||
|- | |||
| Web Properties Review | |||
| kbrosnan | |||
| Once | |||
|} | |||
Examine the copy for the staged websites that are part of the release promotion. We review the pages for technical errors in the copy. Additionally we help flag bugs and issues for notice in the release notes. | |||
=== Testday === | === Testday === | ||
Latest revision as of 21:55, 24 May 2011
Fennec 5.0 Beta Test Plan
Overview
The goal of this Overall Test Plan is to lay out our testing strategy, determine who will own certain test areas and when they will do it and, finally, track the status. All of our releases are tested off of the "Fennec 5.0 Beta Catch-All Test Run" on the Litmus TCM. For more information about Fennec 5.0, take a look at the mobile team's Feature Planning page.
Project contacts
PM: Thomas Arend (tarend)
Dev Lead: Stuart Parmenter (stuart)
Build: John Oduinn (oduinn)
QA: Aakash Desai (aakashd)
Schedule
- Start Date: 5/17/2011
- End Date: 5/21/2011
Bugs in Release
Build Location and IDs
- Go to the nightly builds folder in our ftp directory
- Go to the "latest-mozilla-beta" folder
- Download and install the candidate onto your Android phone/device
Features Landed
Test Areas
Smoketests
| QA Task | Person/Group | How Often? |
| Smoketests | IoanaChiorean | Once |
Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.
Tests are available on litmus under the Fennec 2.0 Test Suite.
Results details can be found in Waverley's results wiki.
Results:
- Device: Samsung Google Nexus S (Android 2.3)
- Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0
- Results Fennec 5.0b2 build 1 : PASS
Known bugs:
- bug 628269 - Pages are zoomed in at reload
- bug 639504 - Unable to zoom in/out a webpage on header/footer when a double tap gesture is made
- bug 625981 - places.sqlite-wal takes too much space for Fennec
Virtual Machine (DeviceAnywhere)
| QA Task | Person/Group | How Often? |
| DeviceAnywhere Studio 5.1 | IoanaChiorean | Once |
Tests are available on litmus under the Fennec 2.0 Test Suite.
Results details can be found in Waverley's results wiki.
Results:
- Device: Mototrola Droid X (MB810)( (Android 2.3)
- Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0
- Results Fennec 5.0b2 build 1 : PASS
New issues:
- strange line in the left part (as if the right panel is overlaying the page) - screenshot - should be posted bugs also for DeviceAnywhere?
- bug 658276 - Notification for Geolocation is not dismissed after choosing option Share
Known bugs:
- all bugs found on Smoketest or BFTs
BFTs for 5.0 Features
| QA Task | Person/Group | How Often? |
| BFTs | Anna and Andreea | Once |
BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.
Results details can be found in Waverley's results wiki.
- Device: HTC Desire Z (Android 2.2) and LG Optimus 2X (Android 2.2)
- Build: Mozilla /5.0 (Android;Linux armv7l;rv:5.0) Gecko/20110517 Firefox/5.0 Fennec/5.0
- Results: PASS
Crasher Triage
| QA Task | Person/Group | How Often? |
| Crasher Triage | nhirata | Daily |
Socorro is Mozilla's primary web interface for triaging crash reports sent in by our user base. Due to the relatively large amount of changes coming in on a nightly basis, there is a need to check for crash reports coming in for every build. The owner of this task will triage through top crashers on a daily basis, file bugs if necessary and find steps to reproduce for those crashers if possible.
Results: https://wiki.mozilla.org/index.php?title=QA/Fennec/crash_rep_25may
Device Compatibility
| QA Task | Person/Group | How Often? |
| Device Compatibility | AaronMT | Once |
As part of our testing strategy, we will want to verify device compatibiltiy checking for regressions or major issues in Firefox by running through our compatbility criteria spreadsheet . Mobile users can also contribute towards device qualifying by submitting feedback via our untested device survey.
Website Compatibility
| QA Task | Person/Group | How Often? |
| Website Compatibility | Aaronmt | Once |
As part of our testing cycle we want to cover and include a general testing process that focuses on website compatbility with sites viewed in mobile Firefox. We want you to use mobile Firefox to assist us in identifying any noticeably major issues found in websites. Issues can range from pages that do not look right, layout and positioning problems, interactivity issues, font issues and pages that grealy differ from browser to browser and what you would expect. You can help us by completing a website exploration form here, and following the companion instructional test-plan.
Web Properties Review
| QA Task | Person/Group | How Often? |
| Web Properties Review | kbrosnan | Once |
Examine the copy for the staged websites that are part of the release promotion. We review the pages for technical errors in the copy. Additionally we help flag bugs and issues for notice in the release notes.
Testday
| QA Task | Person/Group | How Often? |
| Testday | kbrosnan | Once |
To continue determining better quality metrics, we'll want to work with our community to find new bugs and performance/stability metrics through the use of a Testday once per cycle per channel (i.e. nightly and beta). The QA owner for this task will send announcements, create a testplan, moderate the channel and post results for the Testday. To see when the next testday is schedule, please take a look at the Test_Days wiki.