QA/Firefox3.5/TestPlan: Difference between revisions

From MozillaWiki
< QA‎ | Firefox3.5
Jump to navigation Jump to search
 
(30 intermediate revisions by 8 users not shown)
Line 1: Line 1:
<small>[[Firefox3.1|&laquo; Firefox3.1]]</small>
<small>[[Firefox3.5|&laquo; Firefox3.5]]</small>


== <FONT COLOR=RED>'''Top Critical issues for QA'''</FONT> ==
== <FONT COLOR=RED>'''Top Critical issues for QA'''</FONT> ==
Line 8: Line 8:
* Major bugs blocking your feature that should be addressed?
* Major bugs blocking your feature that should be addressed?
* Major bugs blocking overall regression of the trunk that should be fixed before shipping?
* Major bugs blocking overall regression of the trunk that should be fixed before shipping?
=== Blocking Beta 2 Issues ===
* (marcia) {{Bug|463301}} -  Crash in [@ fbFetchPixel_x8r8g8b8] when loading sandisk.com using Vista Ultimate
** Blocking b2 granted
* (bc) {{Bug|463956}} - TM: JIT: ecma/GlobalObject/15.1.2.4.js FAIL 
** Flagged for blocking b2, no triage yet
* (bc) {{Bug|458857}} - TM: "Assertion failure: ngslots == tm->globalTypeMap->length()" with gc and generator as getters
** blocking flag, needs a patch
* (bc) {{Bug|451843}} -  Running Dijit Tree Test with Content JIT enabled causes the TreeView not to appear.
** TM is b2, but no one assigned to it yet.  Might need to be punted to next milestone
* (bc) {{Bug|457660}} -  js1_8/genexps/regress-380237-01.js: InternalError: too much recursion (64bit)
** TM is b2, but no one assigned to it yet.  Might need to be punted to next milestone


=== Blocking Next Milestone and Beyond ===
=== Blocking Next Milestone and Beyond ===
* (tomcat) {{Bug|463377}} - Fatal assertion on topsites (baidu.com, ebay.com, www.megarotic.com, etc) in String_p_match
** Flagged for beta 2, no patch in sight yet
* (tomcat) {{Bug|460518}} -  CRASHED signal 10 SIGBUS using regress-271716-n.js Test 
** Regression crash bug, probably affects debug builds
* (bc) {{Bug|445178}} -  (decimal) Implement ES 3.1 Decimal Support
** Blocking flag set
* (tchung) {{Bug|407875}} -  Unprivileged users are not notified of security updates
** Flag has been minus'd for beta 2
* (ctalbert) {{Bug|459148}} -  SVG curves are drawn differently than curves drawn by the CSS system
** Testcase development blocked by pixel rounding issues.  No blocking flag requested yet
* (ctalbert) {{Bug|458169}} -  implement downloadable font support on Linux
** Blocking flag set, Assigned, but no work yet
* (juanb) {{Bug|459878}} - Tracking bug: do major update from FF3.0.x to FF3.1 to make sure it all works
** Would like to get a MU testrun for beta 2.  When can we have this?  '''Target the last week of november'''
*  See [https://wiki.mozilla.org/QA/Topcrashes#Current_topcrashes_being_tracked_3.1b1 Topcrashes from Beta 1].  Particularly needing blocking beta 2 attention:
** #1, #2, #4, #8, (#6 & #7 need more external investigation)
* (hskupin) {{Bug|451267}} - Master password dialog is shown on each website with at least one text field due to sqlite import
** Has blocking1.9.1 flag.
* <del> (Marcia) {{Bug|347930}} -  Tab strip should be a toolbar instead </del>
** Pushed out of 3.1.  flag has been minused.
* <del> (Stephend) {{Bug|450314}} - use a special tag to block results from the awesomebar </del> '''resolved wontfixed'''


== Overview ==
== Overview ==
Line 55: Line 22:
* Milestone testing
* Milestone testing


[[Firefox3.1|Firefox 3.1 Central]]
[[Firefox3.5|Firefox 3.5 Central]]


== Requirements and Scheduling ==  
== Requirements and Scheduling ==  
* [[Firefox3.1/Features|Fx3.1 Firefox Product Requirements]]
* [[Firefox3.5/Features|Fx3.5 Firefox Product Requirements]]
** [[Firefox3.1/StatusMeetings/2008-07-15#Firefox_3.1| '''A more update list of features''']]
** [[Firefox3.1/StatusMeetings/2008-07-15#Firefox_3.1| '''A more update list of features''']]
* [[Firefox3.1/Schedule|Fx3.1 Schedule]]
* [[Firefox3.5/Schedule|Fx3.5 Schedule]]
* [[Testplan/TestPlanReview|Test Plan Review Schedule]]
* [[Testplan/TestPlanReview|Test Plan Review Schedule]]


== Test Results ==  
== Test Results ==  


* [[QA/Firefox3.5/TestPlan/Live| Live Results]]
* [[QA/Firefox3.5/TestPlan/RC3|RC3 Results]]
* [[QA/Firefox3.5/TestPlan/RC2|RC2 Results]]
* [[QA/Firefox3.5/TestPlan/RC1|RC1 Results]]
* [[QA/Firefox3.5/TestPlan/BetaPreview|Beta Preview Results]]
* [[QA/Firefox3.5/TestPlan/Beta4|Beta 4 Results]]
* [[QA/Firefox3.1/TestPlan/Beta3|Beta 3 Results]]
* [[QA/Firefox3.1/TestPlan/Beta2|Beta 2 Results]]
* [[QA/Firefox3.1/TestPlan/Beta2|Beta 2 Results]]
* [[QA/Firefox3.1/TestPlan/Beta1|Beta 1 Results]]
* [[QA/Firefox3.1/TestPlan/Beta1|Beta 1 Results]]
Line 301: Line 275:
* [http://ed.agadak.net/2008/07/firefox-31-restricts-matches-keywords Mardak's blog post]
* [http://ed.agadak.net/2008/07/firefox-31-restricts-matches-keywords Mardak's blog post]
|-
|-
|Ctrl-Tab
|Clear Private Data Enhancements
|[[QA/Firefox3.1/Ctrl-Tab_Test_Plan|Ctrl Tab Test Plan]]
|[[QA/Firefox3.1/CPD_TestPlan|Test Plan]]
|marcia
|Marcia (related to PB)
|Johnathan Nightingale
|S
|Ff
|
* [http://people.mozilla.com/~faaborg/files/shiretoko/privacy_i2.png Flowchart]
* {{Bug|453440}} - Extend "Clear Private Data" dialog with time period option
* {{Bug|463607}} - Interaction of Clear Recent History dialog and the private browsing mode
|-
|<del>Ctrl-Tab</del>
|<del>[[QA/Firefox3.1/Ctrl-Tab_Test_Plan|Ctrl Tab Test Plan]]</del>
|<del>marcia</del>
|Dao
|Dao
|S
|S
|Ff
|Ff
|
|
* UPDATE - feature cut for 1.9.1
* [http://jboriss.wordpress.com/2008/07/16/control-tab-a-new-feature-for-firefox/ Boriss's blog post]
* [http://jboriss.wordpress.com/2008/07/16/control-tab-a-new-feature-for-firefox/ Boriss's blog post]
* {{Bug|395980}} - Ctrl-Tab bug and dependencies
* {{Bug|395980}} - Ctrl-Tab bug and dependencies
Line 411: Line 397:
|-
|-
|Memory Leaks
|Memory Leaks
|[http://wiki.mozilla.org/QA:Home_Page:Firefox_3.0_TestPlan:Leaks Memory Leaks Test Plan]
|[http://wiki.mozilla.org/QA:Home_Page:Firefox_3.0_TestPlan:Leaks:3.1manualtest Memory Leaks Test Plan]
|Tomcat
|Tomcat
|Tomcat
|Tomcat
Line 457: Line 443:
* add on & related bugs: {{bug|448674}}, {{bug|448603}}
* add on & related bugs: {{bug|448674}}, {{bug|448603}}
* {{Bug|448909}} -  Need more controls WHATWG Video tag (video front end work)
* {{Bug|448909}} -  Need more controls WHATWG Video tag (video front end work)
* [http://jboriss.wordpress.com/2008/09/19/html-5-video-tag-pirate-edition/ Video Scrubber Blog]
** {{Bug|462113}} -  Implement progress bar / scrubber for video controls
* {{Bug|470596}} -  css border breaks video controls
|-
|-
|JavaScript 1.8.1
|JavaScript 1.8.1
Line 494: Line 483:
|class="header"|FF or Platform?
|class="header"|FF or Platform?
|class="header"|Description
|class="header"|Description
|-
|<strike>Windows TSF integration</strike>
|<strike>?</strike>
|<strike>??, Gary Kwong (only IME QA)</strike>
|<strike>roc, Masayuki, jim chen</strike>
|<strike>S</strike>
|<strike>Pf</strike>
|
* '''Update - Feature is pushed to post 1.9.1'''
<strike>
* {{Bug|88831}}
* [http://msdn.microsoft.com/en-us/library/ms629032%28VS.85%29.aspx TSF Spec]
* [https://wiki.mozilla.org/Firefox3.1/TSF_Security_Review Security Review]</strike>
|-
|-
|Linkified View Source  
|Linkified View Source  
|?
|[https://wiki.mozilla.org/QA/Firefox3.1/ViewSource_Testplan here]
|?
|Curtis Bartley
|Curtis Bartley
|Curtis Bartley
|S
|S
Line 529: Line 531:
|
|
* {{Bug|418454}} - Add support for Aero Glass effects to Widget layer
* {{Bug|418454}} - Add support for Aero Glass effects to Widget layer
* {{Bug|367993}} -  Make Firefox have the Translucent theme like IE7 
* Need unit tests.  Landed
* Need unit tests.  Landed
|}
|}
=== Operating System test plans ===
*[https://wiki.mozilla.org/QA/Firefox3.5/Snow_Leopard_Test_Plan Snow Leopard test plan]


=== Regression Areas ===
=== Regression Areas ===
Line 544: Line 550:
* L10N Testing (may coincide with Feature area)
* L10N Testing (may coincide with Feature area)
* Distribution Testing (may coincide with Feature area)
* Distribution Testing (may coincide with Feature area)
== Tiger Team Testing ==
Daily smoketests are ran against nightly trunk builds.  Please see marcia for more information.


Link to [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Fx3_QA_Daily_Smoketest Schedule and Assignments]
Link to [http://wiki.mozilla.org/MozillaQualityAssurance:Home_Page:Firefox_3.0_TestPlan:Fx3_QA_Daily_Smoketest Schedule and Assignments]
Line 559: Line 562:


== Banking and Financial Sites ==
== Banking and Financial Sites ==
For these types of sites, we’re asking community members and MoCo QA to [[QA/Firefox3/TestPlan/Banking|state which banks and financial institutions they're willing to test]]. Please add your name to the list.
For these types of sites, we’re asking community members and MoCo QA to [[QA#Website_Testing|state which banks and financial institutions they're willing to test]]. Please add your name to the list.


== L10n Testing with Third Parties ==
== L10n Testing with Third Parties ==
Line 585: Line 588:
== Additional References ==  
== Additional References ==  
This section is left to add any additional references relevant to the Firefox 3.1 project
This section is left to add any additional references relevant to the Firefox 3.1 project
[[Category:Shiretoko|TestPlan]]

Latest revision as of 14:25, 30 June 2009

« Firefox3.5

Top Critical issues for QA

This is a working list that QA is using to request be fixed or completed for the Upcoming Fx3 Milestones.

If you have untracked issues in these areas, please enter them here:

  • Design still incomplete?
  • Major bugs blocking your feature that should be addressed?
  • Major bugs blocking overall regression of the trunk that should be fixed before shipping?

Blocking Next Milestone and Beyond

Overview

Firefox 3 QA Test Strategy

The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.

Major QA Activities:

  • New feature testing - see below
  • L10n testing
  • Milestone testing

Firefox 3.5 Central

Requirements and Scheduling

Test Results

Test Areas

Feature Focused Areas

Key: L = Large (3-6 weeks) M = Medium (2-4 weeks) S = Small (1-2 weeks)

Test Plan Template

Feature Area and Spec Test Plan Location QA Contact Dev Contact Size FF or Platform? Description
Downloadable Fonts (@font-face rule) here Akira, ctalbert jdaggett/zack L Pf
moz-transform (CSS Transform) here martijn, ctalbert Keith Schwarz L Pf
NPAPI Cocoa Event Model ? marcia Josh Aas L Pf
Places here Tracy dietrich L Ff
Private Browsing Mode Test Plan Marcia Ehsan Akhgari [:ehsan] L Ff
Query Selectors ? martijn, ctalbert bzbarsky@mit.edu L Pf
SVG CSS here Martijn roc L Pf
WHATWG <canvas> text API here Tchung ebutler L Pf
Border Image ? martijn, ctalbert robarnold, dbaron M Pf
Cross Site XHR here bc, ctalbert sicking@mozilla.com M Pf
Drag and Drop of Tabs (to new Windows) here juanb Bz M Ff
Color Management here Ezadkiel Villarico Marbella (seneca community) bholley M Pf
HTML 5 Drag and Drop API here Joelm Ndeakin M Pf
Media Queries here Joelm, (formerly Hasham) dbaron M Pf
  • Allows for html to query for a media type based on specific attributes in order to determine which style sheet or style rule to apply
  • Primary Bug: bug 156716
  • spec draft
Native JSON here Mikeal rsayre@mozilla.com M Pf
Worker Threads here Adam or bc bent@mozilla.com M Pf
  • currently, everything is on one UI thread. affects performance of browser if its a slow thread. Want Multi-threading
  • look for security sided, extra threads are well scoped.
  • look for browsers immune to threads that are slow in some places
  • Detect areas we can break the platform, regard to threading (eg. mxr causes slow threading, loading of all the text and linkifying and color coding -- all on the main UI thread)
  • bug 437152
  • Status: potential risk, still looking hard
  • WhatWG Specification
AwesomeBar here stephend dietrich/mardak S Ff
Clear Private Data Enhancements Test Plan Marcia (related to PB) Johnathan Nightingale S Ff
  • Flowchart
  • bug 453440 - Extend "Clear Private Data" dialog with time period option
  • bug 463607 - Interaction of Clear Recent History dialog and the private browsing mode
Ctrl-Tab Ctrl Tab Test Plan marcia Dao S Ff
Session Restore Session Restore Test Plan marcia Dietrich S Ff
Exiting the Browser Experience ? Tomcat ? S Ff ?
  • Need owner
Search Text Widget Search Widget Testplan whimboo Dao S Ff
SMIL SMIL testplan Martijn dholbert S Pf
Element Traversal here Joelm Jonas Sicking S Pf
Script Defer here Joelm Jonas Sicking S Pf
  • bug 28293 - (defer) scripts with defer attribute not deferred
  • Comment 24 & 25, 39 has simplified testcases (landed)
ACID 3 Testplan Tomcat Roc or dbaron M Pf
Add-on Update Addons Testplan Tchung/Tomcat ? M Ff
  • Beta2: bug 324121 – Extensions with available updates for the next version of the app should not be listed in app update extension will be disabled warning
    • Resolved Fixed, needs testing
  • Beta 2: bug 455906 - Support severities for blocklist entries
    • Resolved Fixed, Need to finish hard block testing
  • bug 421470 – Software update does not download/update even it is set to automatic
    • No fix in site
  • Beta 2: bug 407875 - Unprivileged users are not notified of security updates
    • Flagged for beta 2, but no patch in sight yet.
  • bug 434155 – Help -> Check for Updates does not verify add-on compatibility
    • Resolved Fixed, needs testing
  • bug 434465 – Software update's background add-on update check ignores updated compatibility
    • Fixed by bug 324121. Test that bug, and this bug is good.
  • CUT bug 392180 – Provide force variable to use in the updateURL
  • Beta2: Plugin blocklist - bug 391714, bug 449027, bug 391728
Memory Leaks Memory Leaks Test Plan Tomcat Tomcat L Pf/Ff
  • Checking for memory leaks and asserts on debug builds
Offline/Local Storage here juanb dcamp Pf
Password Manager to mozStorage here harthur zpao ? Pf
Video Decoders here Tchung, ctalbert, mikeal roc/cdouble? Pf
  • bug 422538 – Ogg Theora backend for HTML5 video element (tracking bug)
  • Core element patch has landed
  • --enable-media on trunk?
  • Ogg Vorbis/Theora backend could get review, but legal issues may block
  • Need to push GStreamer bug 422540 and Quicktime bug 435298 backends to patch submission
  • DirectShow backend playing some movies, needs more work bug 435339
  • Primary bug: bug 382267
  • add on & related bugs: bug 448674, bug 448603
  • bug 448909 - Need more controls WHATWG Video tag (video front end work)
  • Video Scrubber Blog
    • bug 462113 - Implement progress bar / scrubber for video controls
  • bug 470596 - css border breaks video controls
JavaScript 1.8.1 bc brendan Pf
CSS Transitions ? mw22 and/or ctalbert Zwol, Jonathon Jongsma Pf

Unowned Features

Feature Area and Spec Test Plan Location QA Contact Dev Contact Size FF or Platform? Description
Windows TSF integration ? ??, Gary Kwong (only IME QA) roc, Masayuki, jim chen S Pf
  • Update - Feature is pushed to post 1.9.1

Linkified View Source here Curtis Bartley Curtis Bartley S Ff
  • Blog post
  • Beta 2: bug 17612 - view-source link-browsing
  • bug 455888 - Make View Source not suck (view source meta bug)
  • bug 455948 - The view source window should have a toolbar with toggle-line-wrap and goto-line buttons
  • bug 455945 - View Source should "linkify" links in the source -- i.e. show them as clickable hyperlinks
  • bug 246620 - Add line numbers to View Source for Firefox
  • bug 464314 - view-source link-browsing doesn't decode entities
  • bug 79518 - tracking view source UI
Improved CMS performance ? ? Bobby Holly S Pf
  • bug 444661 - Decrease Performance Penalty of color management linear interpolations
  • Unclear how to test (landed)
Aero Glass support in chrome on Vista ? ? Bobby Holly S Pf
  • bug 418454 - Add support for Aero Glass effects to Widget layer
  • bug 367993 - Make Firefox have the Translucent theme like IE7
  • Need unit tests. Landed

Operating System test plans

Regression Areas

  • Top Web accessibility Tests
    • Plugins (eg. Flash, java, AJAX)
    • Extensions (eg. Firebug, NoScript, Fotofox)
    • Themes (eg. Vista, Joga, Simpsons)
    • Top Site Tests (eg. Alexa top 50)
    • Financial Sites (eg. banks, online trading)
  • Accessibility Tests
  • User Performance Testing
  • Major / Minor Update Testing
  • L10N Testing (may coincide with Feature area)
  • Distribution Testing (may coincide with Feature area)

Link to Schedule and Assignments

Extensions

The following extensions in the list below will be ran through regression tests during selected milestone releases.

See examples here.

Top Crash reports

The main report is the easiest to view.

Banking and Financial Sites

For these types of sites, we’re asking community members and MoCo QA to state which banks and financial institutions they're willing to test. Please add your name to the list.

L10n Testing with Third Parties

For additional string testing, Mozilla will work with the l10n Team for localization and string testing.

Results link to be posted later.

Review Feedback

In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:

QA / Support Team Issues

Add any common issues you are finding during your nightly /release testing that you feel would be good for Mozilla support to be aware of.

  • See Topcrashers in beta 1
  • Private Browsing has landed, but bugs are being found as we speak
  • Ctrl-Tab preview pane has landed, but bugs are being found as we speak


Additional References

This section is left to add any additional references relevant to the Firefox 3.1 project

Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3.5/TestPlan&oldid=152731"