QA/Firefox3/TestResults/Beta5

From MozillaWiki
Jump to navigation Jump to search

« QA/Firefox3/TestResults

Firefox 3 Beta 5 Test Results

Related Pages

Links to separate test results. Some are referenced further below.

Project Release Tracking Page

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony (tchung)

Schedule

Code Freeze
3/18
Code Build
ETA ?
QA test signoff
ETA 3/26-3/31
  • Smoketests (3/26) result
  • BFTs (3/26-3/27) result
  • Beta 5 and past focused Tests (ongoing)
  • Update snippets from Alpha X, Beta 1,2,3,4 -> 5 (3/27-3/28) result betatest, result releasetest, result beta (4 testers)
  • P1 L10N spotchecks (3/27) result (1 tester)
  • B5 bug verifications (ongoing)
  • Beta 5 Testday (3/28) result (3 testers)
  • Release notes, Websites, live tests (3/31) result (3 testers)

Release Notes

See list of release notes here

Build Location and IDs

RC1: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0b5-candidates/rc1/

  • Linux: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5
  • Mac: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5

RC2: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0b5-candidates/rc2/

  • Windows:
  • Mac: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
  • Linux:

Strategy

  • Areas Covered:
    • Smoketests
    • BFTs
    • Focused Tests
    • Update snippets from Alpha X, Beta X -> Beta 5
    • Memory Leak Testing (handled by Tomcat on debug builds)
    • P1 L10N spotchecks
    • Bug verifications
    • Beta 5 Testday
  • Areas Not Covered:
    • FFTs
    • Top Sites
    • Top Plugins
    • Top Themes
    • Top Extensions
    • localization (String changes) - 3rd party will cover
    • Distribution
    • Major Updates

Test Results

Types of Tests Win XP Win Vista Mac Tiger Mac Leopard Linux
Smoketest Tomcat - result juanb - result Tracy - PASS marcia - PASS Tomcat - result
BFT Tomcat - result Tomcat (Vista test suite) - result juanb - result Marcia (Leopard test suite/PPC and Intel) - result Tracy - result
Updates Testing (see below) Tchung - Betatest result, Releasetest result, Beta result Juanb - Betatest result, Releasetest result, Beta result Stephend - Betatest result, Releasetest result, Beta result Marcia - Betatest result, Releasetest result, Beta result Abillings - Betatest result, Releasetest result, Beta result
l10N Pri 1 locales Tchung - PASS N/A Tchung - PASS N/A Tchung - PASS
Bug Verifications Everyone - result Everyone - result Everyone - result Everyone - result Everyone - result

Updates Testing

We will be doing updates of all Alpha and Beta users to Beta 5. Tests will include different locales for beta 1, but only en-US for all past versions. We will be checking partial updates, complete updates, and failed status.

Will be available on betatest, releasetest, and beta channels

  • Betatest channel - result
  • Releasetest channel - result
  • Beta channel (live) - result

Testcases and Results are tracked here.

L10N P1 Spotchecks

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Automation inspection: supported locales will be ran through Minotaur, and results will be reviewed by axel and mic.

Testcases and Results are tracked here - PASS.

Bug Verifications

Bugs that are marked RESOLVED FIXED that are checked in for beta. Please verify them.

P1/P2 Firefox bugs - http://tinyurl.com/3bfgoa
P1/P2 Platform bugs - http://tinyurl.com/354n48

Testday

A beta 5 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Testday results are posted here.

RC2 Detailed Results

marcia's results

  • Leopard Test Suite (Intel):
    • 10.5 Mac:
      • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5
      • Date: 27. March 2008
      • Status: PASS - Results in Litmus
      • Added one new Parental Controls test case to Litmus.
      • Noted horrible error message that renders when a site is not allowed - the text overflows the page.

tomcat's results

tracy's results

  • Mac 10.4 Smoketests (PPC) - PASS
    • Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9b5) Gecko/2008032619 Firefox/3.0b5

whimboo's results

Bugs filed:

  • Crash when a local SWF file tries to communicate with the internet and Javascript is deactivated Bug 425506
  • Undo function does not work for deleted tags in the Library Bug 425529

RC1 Detailed Results

Tomcats Results

  • Smoketests:
    • Linux:
      • Build Id: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5
      • Date: 26. March 2008
      • Status: PASS - Results in Litmus

marcia's results

  • Smoketests:
    • 10.5 Mac:
      • Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9b4) Gecko/2008032604 Firefox/3.0b5
      • Date: 26. March 2008
      • Status: PASS - Results in Litmus
      • Unable to test extension and themes since none are currently compatible.
  • Smoketests:
    • 10.4 PPC Mac:
      • Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9b5) Gecko/2008032604 Firefox/3.0b5
      • Date: 26. March 2008
      • Fun smoketest: Status: PASS - Results in Litmus
      • Unable to test extension and themes since none are currently compatible.
      • Bug filed: https://bugzilla.mozilla.org/show_bug.cgi?id=425274 -> Should be release noted if not fixed

Live Spot Checks

Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

  • Check for signed installer and applications (windows only)
  • Firstrun page, whatsnew page, Home page, Search plugin
  • Install, uninstall
  • General spotchecks and browser functionality
  • Confirm Build IDs

Windows XP -> abillings - Result
Windows Vista -> stephend - Result
Mac Tiger -> Juan - Result
Mac Leopard -> Marcia - Result
Linux -> stephend - Result

Top Issues Reported in Hendrix Against Firefox 3 Beta 5

  • Enter here

Top Issues from Reporter

  • Enter here
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3/TestResults/Beta5&oldid=88256"