Releases/Firefox 4b10/Test Plan:Feature Area Testing: Difference between revisions
< Releases
Jump to navigation
Jump to search
(Created page with "<small> Fx 4b10 Test Plan</small> = Firefox 4 Beta 10 BFTs = In addition to running all the Mozmill BFT automated tests, we'll run BFTs corr...") |
Andreea.pod (talk | contribs) |
||
| (139 intermediate revisions by 13 users not shown) | |||
| Line 1: | Line 1: | ||
<small>[[Releases/ | <small>[[Releases/Firefox 4b10/Test Plan|Fx 4b10 Test Plan]]</small> | ||
= Firefox 4 Beta 10 BFTs = | = Firefox 4 Beta 10 BFTs = | ||
In addition to running all the Mozmill BFT automated tests, we will test around areas with fixes. The end goal is to make sure the [http://hg.mozilla.org/mozilla-central/pushloghtml?startdate=11+days+ago&enddate=now fixes in beta 10] are actually fixed and haven't introduced new problems. | |||
'''Instructions:''' | |||
#Download builds: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b10-candidates/ | |||
#See the list of fixes for each area | |||
#Run the [https://litmus.mozilla.org/run_tests.cgi?test_run_id=257 Litmus BFTs] that more closely exercise those fixes | |||
Where there are no related Litmus testcases it is up to the feature owners to test their areas. | Where there are no related Litmus testcases it is up to the feature owners to test their areas. | ||
There are a number of JavaScript fixes due to a JS engine merge. We will not focus on those. Instead we will rely on nightly testers feedback. | |||
{| class="fullwidth-table" | {| class="fullwidth-table" | ||
! Focus Area | |- | ||
! Tester | ! Focus Area | ||
! Tester | |||
! Status (detailed results at the bottom) | ! Status (detailed results at the bottom) | ||
|- | |- | ||
| Sync | | Sync <br>[http://bit.ly/gKuL3k Fixes] | ||
| | | Andreea | ||
| WinXP: '' | | WinXP: '''Complete''' <br> Win7: '''Complete''' <br> Mac OS 10.6: '''Complete''' <br> Linux: ... | ||
|- | |- | ||
| Panorama ( | | Panorama <br> <a href="http://bit.ly/gyEqz1">Fixes</a> | ||
| George (Waverley), Tracy/Tony Marcia, Vlad | |||
| WinXP: '' | | WinXP: '''''Complete''' (spot check/did not run BFT)'' <br> Win7: '''Complete''' <br> Mac OS 10.6: '''Complete''' <br> Linux: '''Complete''' | ||
|- | |- | ||
| Places (History and Bookmarks) | | Places (History and Bookmarks) <br> [http://bit.ly/gUvX7X Fixes] | ||
| | | Alex, Vlad (Waverley) | ||
| WinXP: '' | | WinXP: '''complete''' <br> Win7: '''complete''' <br> Mac OS 10.6: '''complete''' <br> Linux: '''complete''' | ||
|- | |- | ||
| | | Session Store <br> [http://bit.ly/ela6HW Fixes] | ||
| Waverley (Teodosia) | |||
| WinXP: '' | | WinXP: '''Completed''' <br> Win7: '''Completed''' <br> Mac OS 10.6: '''Completed''' <br> Linux: '''Completed''' | ||
|- | |- | ||
| | | Plugins (Flash, Silverlight, Quicktime) <br> <a href="http://bit.ly/fqmbJO">Fixes</a> | ||
| | | Marcia, Juanb | ||
| WinXP: '' | | WinXP: '''Completed''' <br> Win7: '''Completed''' <br> Mac OS 10.6: '''Completed''' <br> Linux: '''Completed''' | ||
|- | |- | ||
| | | Add-ons Manager <br> [http://bit.ly/fk16oc Fixes] | ||
| | | Waverley(Andreea, George), Henrik | ||
| WinXP: '' | | WinXP: '''complete''' <br> Win7: '''complete''' <br> Mac OS 10.6: '''complete''' <br> Linux: '''complete''' | ||
|- | |- | ||
| | | Web Console <br> <a href="http://bit.ly/gjk8cd">Fixes</a> | ||
| | | Vlad, Geo | ||
| WinXP: '' | | WinXP: '''Complete''' <br> Win7: '''Complete''' <br> Mac OS 10.6: '''Complete''' <br> Linux: ... | ||
|- | |- | ||
| Doorhanger | | Open Video <br> <a href="http://bit.ly/ey965p">Fixes</a> | ||
| | | Cameron | ||
| WinXP: '' | | WinXP: '''Complete'''<br> Win7: '''Complete''' <br> Mac OS 10.6: '''Complete''' <br> Linux: '''Complete''' | ||
|- | |||
| Graphics and Layout <br> [http://bit.ly/dQdLQl Fixes] | |||
| Waverley (Teodosia, George), Marcia | |||
| WinXP: '''Completed''' <br> Win7: '''Completed''' <br> Mac OS 10.6: '''Completed''' <br> Linux: '''Completed''' | |||
|- | |||
| Theme, Toolbar, and Doorhanger Notifications <br> [http://bit.ly/ia4qQk Fixes] | |||
| Alex (Waverley), Raymond | |||
| WinXP: '''''complete'' '''(Raymond)<br> Win7: '''complete''' <br> Mac OS 10.6: '''''complete'' '''(Raymond)<br> Linux: '''complete''' | |||
|- | |||
| General (mostly Notifications) <br> <a href="http://bit.ly/gWVCDf">Fixes</a> | |||
| George (Waverley), Vlad | |||
| WinXP: '''Complete''' <br> Win7: '''Complete''' <br> Mac OS 10.6: ''in progress'' <br> Linux: '''Complete''' | |||
|- | |||
| Location Bar and Private Browsing <br> [http://bit.ly/ejv4EP Fixes] | |||
| Anthony, Alex (Waverley) | |||
| WinXP: '''complete''' <br> Win7: '''Completed''' <br> Mac OS 10.6: '''Completed''' <br> Linux: '''Completed''' | |||
|- | |||
| Printing <br> [http://bit.ly/hgc0B0 Fixes] | |||
| Teodosia/Marcia | |||
| WinXP: '''Completed''' <br> Win7:'''Completed''' <br> Mac OS 10.6: '''Completed''' <br> Linux: '''Completed''' | |||
|} | |} | ||
= Results Notes = | = Results Notes = | ||
== Example Focus Area == | |||
* WinXP: Issues can be noted here, such as bugs filed in the area using this platform. Make notes of issues that you encountered but didn't lead to a bug being filed. '''If you file a bug''' please add '''[ | == Example Focus Area == | ||
* Mac OS 10.6: Issue... | |||
* Linux 10.4: Issue... | *WinXP: Issues can be noted here, such as bugs filed in the area using this platform. Make notes of issues that you encountered but didn't lead to a bug being filed. '''If you file a bug''' please add '''[4b10]''' in the Whiteboard field. | ||
*Mac OS 10.6: Issue... | |||
*Linux 10.4: Issue... | |||
== Sync == | |||
Look for data integrity, make sure bookmarks and history are in a good state (no duplicates, nothing missing) | |||
Look for data integrity, make sure bookmarks and history are in a good state (no duplicates, nothing missing) | |||
* Win | *Win 7: '''PASS''' | ||
* Mac | *Win XP: '''PASS''' | ||
* Linux (version): Issues... | *Mac 10.6 : '''PASS''' | ||
*Linux (version): Issues... | |||
== Panorama == | == Panorama == | ||
Pay close attention to interactions with tabbed browsing, tab drag and drop, tab previews in Win 7, and lightweight themes. | |||
* Win (version): | *Win (version): '''Complete''' | ||
* Mac (version): | ** Win XP - See bugs filed {{bug|628174}} and {{bug|628173}} | ||
* Linux (version): | *Mac (version): '''Complete - in Litmus''' | ||
** Mac os x 10.6 - {{bug|628270}} | |||
*Linux (version): '''Complete''' | |||
== Places (History-Bookmarks) == | |||
== Plugins (Flash/Silverlight/Quicktime) == | Have a basic "rich" profile with a few bookmarks, history, saved passwords, some Panorama groups, and shutdown and restart app, for example. Make sure nothing is missing as far as history and bookmarks. | ||
*Win (version): '''Complete''' | |||
*Mac (version): '''Complete''' | |||
*Linux (version): '''Complete''' | |||
== Plugins (Flash/Silverlight/Quicktime) == | |||
There were several fixes in accelerated graphics and there are interactions especially in Flash where we could see painting issues. | There were several fixes in accelerated graphics and there are interactions especially in Flash where we could see painting issues. | ||
== Web Console == | Notes: I went through the bug fixes and tried to verify them. Most of them were in Windows 7 and Mac OS X. Also went through plug-ins test suite in Litmus (BFT) using Windows XP. (juanb, Jan 22 @4pm) | ||
* Win (version): | |||
* Mac (version): | *Win (XP&7): '''PASS''' - See notes above. Verifications on Win7 and BFT on WinXP. | ||
* Linux ( | *Mac (10.6&10.5): '''PASS''' - See notes above. 10.5 was a quick spot check with Flash and QT. | ||
** marcia tested Flash Version: 10.2.152.14 on Mac 10.6 | |||
*Linux (64bit): ''in-progress'' - Spot check with Ubuntu and Flash. | |||
** Issues: Window-less (fullscreen) Flash playback not working on 64bit Linux. Investigating and looking for dupes. | |||
== Web Console == | |||
*Win xp: '''PASS''' | |||
*Windows 7: '''PASS''' | |||
*Mac 10.6: '''PASS''' | |||
*Linux Ubuntu 10.04: In progress... | |||
* [https://spreadsheets.google.com/ccc?key=0AlsgObMBZ381dEpfNzFYM3BhVHdTMHJRM0tzMTdPZ3c&hl=en#gid=6 Results from spreadsheet tests] {{bug|627674}} | |||
== Open Video == | |||
*Win (version): '''Pass''' | |||
*Mac (version): '''Pass''' | |||
*Linux (version): '''Pass''' | |||
** Unable to test Bug 616778, due to a requirement of special hardware I don't possess. | |||
== Doorhanger Notifications (ashughes) == | |||
*Win 7: PASS | |||
*Win XP: PASS | |||
*Mac 10.6: PASS | |||
*Ubuntu 10.10: PASS | |||
== Session Store (Teodosia) == | |||
Notes: Went through the bug fixes and verified them. Also run BFTs in Litmus on Session Store for Win 7, Win XP and Linux. | |||
*Win 7: {{bug|625257}} still reproducible | |||
*Win XP: {{bug|625257}} still reproducible | |||
*Mac 10.6 : {{bug|625257}} still reproducible | |||
*Linux 32: {{bug|625257}} still reproducible | |||
== Theme,Toolbar and Doorhanger Notifications == | |||
*Win (version): '''COMPLETE''' | |||
*Mac (version): ''Not Run'' | |||
*Linux (version): '''COMPLETE''' | |||
== General (Mostly notifications) == | |||
*{{Bug|627972}} - Arrow panel clipping when in maximized mode | |||
*{{Bug|627974}} - Arrow panel hidden behind Windows taskbar when browser not maximized | |||
*Win (version): '''Complete''' | |||
*Mac (version): '''Complete''' | |||
*Linux (version): {{Bug|616502}} - reproducible | |||
== Graphics and Layout (Teodosia, George, Marcia) == | |||
*Win 7: '''PASS''' | |||
*Win XP: {{Bug|627768}} | |||
*Mac 10.5: '''PASS''' | |||
*Linux (version): {{Bug|627768}} | |||
== Add-ons Manager (Andreea) == | |||
*Win 7: '''Complete''' | |||
*Win XP: '''Complete''' | |||
*Mac 10.6 : '''Complete''' | |||
*Linux (version): '''Complete''' | |||
* [https://spreadsheets.google.com/ccc?key=0AsEULNcoK-ekdDE2LVhyX1dtSGMyODcwcVVNSUdoanc&authkey=CJX29LoG&hl=en#gid=3 Results in spreadsheet ] | |||
== Location Bar & Private Browsing (ashughes, Alex) == | |||
*Win 7: '''COMPLETE''' | |||
*Mac 10.6: '''COMPLETE''' | |||
*Ubuntu 10.10: '''COMPLETE''' | |||
*[reopen] <img _fck_mw_template="true" _fckrealelement="0" _fckfakelement="true" src="https://wiki.mozilla.org/extensions/FCKeditor/fckeditor/editor/images/spacer.gif" class="FCK__MWTemplate"> - Internal URLs show up as results after RSS feed preview | |||
== Printing (Teodosia, Marcia) == | |||
Notes: | |||
Went through the bug fixes and verified them. Also ran BFTs in Litmus on Win XP, Win 7 and Linux 32. | |||
*Win XP: '''PASS''' | |||
*Win 7: '''PASS''' | |||
*Mac 10.6: '''PASS''' - Performed spot check of Printing and Save to PDF operations | |||
*Linux 32: '''PASS''' | |||
== Web GL (Marcia) == | |||
*Although Web GL is not being heavily emphasized since it is an enthusiast area, I performed some tests using http://bodybrowser.googlelabs.com/ which seems to be a pretty popular site. | |||
*Win XP: '''''' | |||
*Win 7: '''''' | |||
*Mac 10.6: '''PASS''' - Sometimes the browser froze while manipulating some of the controls on the left hand side. | |||
== | == Headline text == | ||
<br> | |||
Latest revision as of 19:40, 24 January 2011
Firefox 4 Beta 10 BFTs
In addition to running all the Mozmill BFT automated tests, we will test around areas with fixes. The end goal is to make sure the fixes in beta 10 are actually fixed and haven't introduced new problems.
Instructions:
- Download builds: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/4.0b10-candidates/
- See the list of fixes for each area
- Run the Litmus BFTs that more closely exercise those fixes
Where there are no related Litmus testcases it is up to the feature owners to test their areas.
There are a number of JavaScript fixes due to a JS engine merge. We will not focus on those. Instead we will rely on nightly testers feedback.
| Focus Area | Tester | Status (detailed results at the bottom) |
|---|---|---|
| Sync Fixes |
Andreea | WinXP: Complete Win7: Complete Mac OS 10.6: Complete Linux: ... |
| Panorama <a href="http://bit.ly/gyEqz1">Fixes</a> |
George (Waverley), Tracy/Tony Marcia, Vlad | WinXP: Complete (spot check/did not run BFT) Win7: Complete Mac OS 10.6: Complete Linux: Complete |
| Places (History and Bookmarks) Fixes |
Alex, Vlad (Waverley) | WinXP: complete Win7: complete Mac OS 10.6: complete Linux: complete |
| Session Store Fixes |
Waverley (Teodosia) | WinXP: Completed Win7: Completed Mac OS 10.6: Completed Linux: Completed |
| Plugins (Flash, Silverlight, Quicktime) <a href="http://bit.ly/fqmbJO">Fixes</a> |
Marcia, Juanb | WinXP: Completed Win7: Completed Mac OS 10.6: Completed Linux: Completed |
| Add-ons Manager Fixes |
Waverley(Andreea, George), Henrik | WinXP: complete Win7: complete Mac OS 10.6: complete Linux: complete |
| Web Console <a href="http://bit.ly/gjk8cd">Fixes</a> |
Vlad, Geo | WinXP: Complete Win7: Complete Mac OS 10.6: Complete Linux: ... |
| Open Video <a href="http://bit.ly/ey965p">Fixes</a> |
Cameron | WinXP: Complete Win7: Complete Mac OS 10.6: Complete Linux: Complete |
| Graphics and Layout Fixes |
Waverley (Teodosia, George), Marcia | WinXP: Completed Win7: Completed Mac OS 10.6: Completed Linux: Completed |
| Theme, Toolbar, and Doorhanger Notifications Fixes |
Alex (Waverley), Raymond | WinXP: complete (Raymond) Win7: complete Mac OS 10.6: complete (Raymond) Linux: complete |
| General (mostly Notifications) <a href="http://bit.ly/gWVCDf">Fixes</a> |
George (Waverley), Vlad | WinXP: Complete Win7: Complete Mac OS 10.6: in progress Linux: Complete |
| Location Bar and Private Browsing Fixes |
Anthony, Alex (Waverley) | WinXP: complete Win7: Completed Mac OS 10.6: Completed Linux: Completed |
| Printing Fixes |
Teodosia/Marcia | WinXP: Completed Win7:Completed Mac OS 10.6: Completed Linux: Completed |
Results Notes
Example Focus Area
- WinXP: Issues can be noted here, such as bugs filed in the area using this platform. Make notes of issues that you encountered but didn't lead to a bug being filed. If you file a bug please add [4b10] in the Whiteboard field.
- Mac OS 10.6: Issue...
- Linux 10.4: Issue...
Sync
Look for data integrity, make sure bookmarks and history are in a good state (no duplicates, nothing missing)
- Win 7: PASS
- Win XP: PASS
- Mac 10.6 : PASS
- Linux (version): Issues...
Panorama
Pay close attention to interactions with tabbed browsing, tab drag and drop, tab previews in Win 7, and lightweight themes.
- Win (version): Complete
- Win XP - See bugs filed bug 628174 and bug 628173
- Mac (version): Complete - in Litmus
- Mac os x 10.6 - bug 628270
- Linux (version): Complete
Places (History-Bookmarks)
Have a basic "rich" profile with a few bookmarks, history, saved passwords, some Panorama groups, and shutdown and restart app, for example. Make sure nothing is missing as far as history and bookmarks.
- Win (version): Complete
- Mac (version): Complete
- Linux (version): Complete
Plugins (Flash/Silverlight/Quicktime)
There were several fixes in accelerated graphics and there are interactions especially in Flash where we could see painting issues.
Notes: I went through the bug fixes and tried to verify them. Most of them were in Windows 7 and Mac OS X. Also went through plug-ins test suite in Litmus (BFT) using Windows XP. (juanb, Jan 22 @4pm)
- Win (XP&7): PASS - See notes above. Verifications on Win7 and BFT on WinXP.
- Mac (10.6&10.5): PASS - See notes above. 10.5 was a quick spot check with Flash and QT.
- marcia tested Flash Version: 10.2.152.14 on Mac 10.6
- Linux (64bit): in-progress - Spot check with Ubuntu and Flash.
- Issues: Window-less (fullscreen) Flash playback not working on 64bit Linux. Investigating and looking for dupes.
Web Console
- Win xp: PASS
- Windows 7: PASS
- Mac 10.6: PASS
- Linux Ubuntu 10.04: In progress...
Open Video
- Win (version): Pass
- Mac (version): Pass
- Linux (version): Pass
- Unable to test Bug 616778, due to a requirement of special hardware I don't possess.
Doorhanger Notifications (ashughes)
- Win 7: PASS
- Win XP: PASS
- Mac 10.6: PASS
- Ubuntu 10.10: PASS
Session Store (Teodosia)
Notes: Went through the bug fixes and verified them. Also run BFTs in Litmus on Session Store for Win 7, Win XP and Linux.
- Win 7: bug 625257 still reproducible
- Win XP: bug 625257 still reproducible
- Mac 10.6 : bug 625257 still reproducible
- Linux 32: bug 625257 still reproducible
Theme,Toolbar and Doorhanger Notifications
- Win (version): COMPLETE
- Mac (version): Not Run
- Linux (version): COMPLETE
General (Mostly notifications)
- bug 627972 - Arrow panel clipping when in maximized mode
- bug 627974 - Arrow panel hidden behind Windows taskbar when browser not maximized
- Win (version): Complete
- Mac (version): Complete
- Linux (version): bug 616502 - reproducible
Graphics and Layout (Teodosia, George, Marcia)
- Win 7: PASS
- Win XP: bug 627768
- Mac 10.5: PASS
- Linux (version): bug 627768
Add-ons Manager (Andreea)
- Win 7: Complete
- Win XP: Complete
- Mac 10.6 : Complete
- Linux (version): Complete
Location Bar & Private Browsing (ashughes, Alex)
- Win 7: COMPLETE
- Mac 10.6: COMPLETE
- Ubuntu 10.10: COMPLETE
- [reopen] <img _fck_mw_template="true" _fckrealelement="0" _fckfakelement="true" src="
" class="FCK__MWTemplate"> - Internal URLs show up as results after RSS feed preview
Printing (Teodosia, Marcia)
Notes: Went through the bug fixes and verified them. Also ran BFTs in Litmus on Win XP, Win 7 and Linux 32.
- Win XP: PASS
- Win 7: PASS
- Mac 10.6: PASS - Performed spot check of Printing and Save to PDF operations
- Linux 32: PASS
Web GL (Marcia)
- Although Web GL is not being heavily emphasized since it is an enthusiast area, I performed some tests using http://bodybrowser.googlelabs.com/ which seems to be a pretty popular site.
- Win XP: '
- Win 7: '
- Mac 10.6: PASS - Sometimes the browser froze while manipulating some of the controls on the left hand side.
Headline text