QA/Firefox3/TestResults/RC2: Difference between revisions

From MozillaWiki
< QA‎ | Firefox3‎ | TestResults
Jump to navigation Jump to search
Line 263: Line 263:
* Bugs Seen during testing
* Bugs Seen during testing
**[https://bugzilla.mozilla.org/show_bug.cgi?id=436497 Bug 436497] Google favicon needs to be updated.
**[https://bugzilla.mozilla.org/show_bug.cgi?id=436497 Bug 436497] Google favicon needs to be updated.
**
**[https://bugzilla.mozilla.org/show_bug.cgi?id=435485 Bug 435485] Can't reliably use "Move" command in Organize Bookmarks
 


==== Tomcat's Results ====
==== Tomcat's Results ====

Revision as of 02:03, 1 June 2008

« QA/Firefox3/TestResults

Firefox 3 RC2 Test Results

Related Pages

Links to separate test results. Some are referenced further below.

Project Release Tracking Page

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony (tchung)

Schedule

Code Freeze
5/28
Code Build
5/29
QA test signoff
ETA 6/4
  • Smoketests PASS
  • BFTs 40% completed
  • RC2 and past focused Tests (ongoing) result
  • Update snippets from Alpha X, Beta X -> RC2 betatest, result releasetest, result, rc1->rc2 beta/release, result
  • P1 L10N spotchecks PASS
  • RC2+ bug verifications (ongoing) result
  • RC2 Testday result
  • Release notes, Websites, live tests result

Release Notes

See list of release notes here

Build Location and IDs

*Note: Only Mac was respun for a second build. Windows and Linux were not affected by the bug in question.

Windows

  • Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0

Mac

  • RC2 Build 1
  • Leopard
    • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
  • RC2 Build 2
  • Leopard
    • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008053008 Firefox/3.0
  • RC2 Build 2
  • Tiger
    • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9) Gecko/2008053008 Firefox/3.0

Linux

  • Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008052912 Firefox/3.0

Test Coverage

  • Areas Covered:
    • Smoketests
    • BFTs
      • Use test profiles stored here.
    • Focused Tests
    • Update snippets from Alpha X, Beta X, RC1 -> RC2
    • P1 L10N spotchecks
    • Top Plugins (special coverage on bug 434914 for mac)
    • Bug verifications (RC2+ bugs)
    • RC2 Testday
  • Areas Not Covered:
    • FFT
    • Top Sites
    • Top Themes
    • Top Extensions
    • Accessibility Tests (litmus and automation)
    • Memory Leak Testing (handled by Tomcat on debug builds)
    • Bc's topsite spidertests against jemalloc/malloc builds
    • localization (String changes) - Localizers to cover
    • Distribution
    • Major Updates

Test Results

Types of Tests Win XP Win Vista Mac Tiger Mac Leopard Linux
Smoketest Tomcat - PASS juanb - PASS Tracy - PASS marcia - PASS Tomcat (Build 2) PASS Tracy - PASS
BFT Tomcat - PASS Juan - 90% Hasham/Marcia - result Tomcat - PASS Build 2 BFT in progress Tracy - PASS
Updates Testing Tchung - Betatest PASS, Releasetest result, Beta result Juanb - Betatest result, Releasetest result, Beta result Stephend - Betatest result, Releasetest result, Beta result Marcia - Betatest result, Releasetest result, Beta result Abillings - Betatest result, Releasetest result, Beta result
l10N Pri 1 locales Tchung - PASS N/A Tchung - PASS N/A Tchung - PASS
Top Plugins Hashem - result N/A Tracy - result N/A Tchung - PASS
Bug Verifications Everyone - result Everyone - result Everyone - result Everyone - result Everyone - result
Live Checks & Download Links Tomcat - result Stephend - result Marcia - result Juanb - result Abillings - result

Updates Testing

We will be doing updates of all Alpha and Beta users to RC2. We will be checking partial updates, complete updates, and failed status.

Will be available on betatest, releasetest, beta, and release (rc1 only) channels

  • Betatest channel - result
  • Releasetest channel - result
  • Beta channel (live) - result
  • Release channel (rc1 only) - result

Testcases and Results are tracked here.

L10N P1 Spotchecks

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Automation inspection: supported locales will be ran through Minotaur, and results will be reviewed by axel and mic.

Testcases and Results are tracked here - PASS

Top Plugins Testing

Query the top plugins supported for the web, and verify they are compatible with Firefox 3. See AMO list for recommended plugins.

Testcases

  • On a clean environment, install plugins
  • Test updating of old plugins
  • Disable plugins in Addons Manager, verify plugins stop working
  • Uninstall plugins, verify websites request a plugin to download
  • Verify websites with popular plugins:

Additional Plugin litmus cases can be found here.

Bug Verifications

Bugs that are marked RESOLVED FIXED that are checked in for RC. Please verify them.

RC2+ Bugs - http://tinyurl.com/47wglk
P1/P2 Firefox bugs - http://tinyurl.com/3bfgoa
P1/P2 Platform bugs - http://tinyurl.com/354n48

Testday

A RC2 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Testday results are posted here.

Live Spot Checks

Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

  • Check for signed installer and applications (windows only)
  • Firstrun page, whatsnew page, Home page, Search plugin
  • Install, uninstall
  • General spotchecks and browser functionality
  • Confirm Build IDs

OS assignments:

  • XP - Tomcat - result
  • Vista - Stephend - result
  • Mac Tiger - Marcia - result
  • Mac Leopard - Juanb - result
  • Linux - Abillings - result

Detailed Results

Smoketests

Tracy's Results

  • Mac (Tiger) - Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
  • Status: PASS results in Litmus
  • Linux (FC7) - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008052912 Firefox/3.0
  • Status: PASS results in Litmus

marcia's Results

  • Mac (Leopard) - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
  • Status: PASS results in Litmus
  • Confirmed Bug 436210 which was seen during testing.

Tomcat's Results

  • Mac Leopard Build 2: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008053008 Firefox/3.0
  • Status: PASS results in Litmus
  • Windows XP : Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
  • Status: PASS results in Litmus

juanb's results

  • Vista Smoketest: PASS
    • Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0

BFT

hasham/marcia Results

  • marcia: Mac 10.4 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9) Gecko/2008053008 Firefox/3.0
  • Status: in progress
  • Bugs Seen during testing
    • Bug 436497 Google favicon needs to be updated.
    • Bug 435485 Can't reliably use "Move" command in Organize Bookmarks

Tomcat's Results

  • Mac 10.5 Build 1 - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
  • Status: PASS results in Litmus
  • Confirmed Bug 432832 during the tests
  • Windows XP : Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
  • Status: PASS results in Litmus


juanb's restuls

  • BFT on Vista: 90%
    • Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0
  • Notes:

tracy's results

  • Linux (FC7) - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008052912 Firefox/3.0
  • Status: PASS results in Litmus
    • bug 436521 loop in plugin finder service says Flash is installed when it isn't

Accessibility testing

  • Windows: Pass results in Litmus.
  • Verified bug 434002 using ChatZilla 0.9.82.1, which exposed this bug reliably for me.

L10n Testing (tchung)

  • Filed bug 436589 – Search plugin order differs on Linux / Windows versus Mac.

Hendrix Feedback =

  • Bugs Filed by marcia as a result of Hendrix review:
    • Bug 436568 [Vista] DEP may impede the browsing experience on Vista
    • Bug 436356 Bullet lists does not display correctly, works in FF 2
    • Bug 436139 Top part of website renders too high
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3/TestResults/RC2&oldid=94973"