QA/Firefox3/TestResults/RC2

« QA/Firefox3/TestResults

Firefox 3 RC2 Test Results

Related Pages

Links to separate test results. Some are referenced further below.

Project Release Tracking Page

• Update snippets Test Results
• L10n Spotchecks Test Results
• RC2 Testday Results
• QA RC2 Relnotes Tracking
• QA RC2 Bug Bash
• RC2 Bug Candidates Tracking Page

Contact Info

PM: Mike Beltzner (beltzner)
Build: John O'Duinn (joduinn)
QA: Tony (tchung)

Schedule

Code Freeze

5/28

Code Build

5/29

QA test signoff

ETA 6/4

• Smoketests result
• BFTs result
• RC2 and past focused Tests (ongoing) result
• Update snippets from Alpha X, Beta X -> RC2 betatest, result releasetest, result, rc1->rc2 beta/release, result
• P1 L10N spotchecks result
• RC2+ bug verifications (ongoing) result
• RC2 Testday result
• Release notes, Websites, live tests result

Release Notes

See list of release notes here

Build Location and IDs

• ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.0rc2-candidates/build1/

Windows

• Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0

Mac

• Leopard
  • Build ID: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0

Test Coverage

• Areas Covered:
  • Smoketests
  • BFTs
    • Use test profiles stored here.
  • Focused Tests
  • Update snippets from Alpha X, Beta X, RC1 -> RC2
  • P1 L10N spotchecks
  • Bug verifications (RC2+ bugs)
  • RC2 Testday

• Areas Not Covered:
  • FFT
  • Top Sites
  • Top Plugins
  • Top Themes
  • Top Extensions
  • Accessibility Tests (litmus and automation)
  • Memory Leak Testing (handled by Tomcat on debug builds)
  • Bc's topsite spidertests against jemalloc/malloc builds
  • localization (String changes) - Localizers to cover
  • Distribution
  • Major Updates

Test Results

Types of Tests	Win XP	Win Vista	Mac Tiger	Mac Leopard	Linux
Smoketest	Tomcat - result	juanb - result	Tracy - PASS	marcia - PASS	Tracy - PASS
BFT	Tomcat - result	Juan - result	Marcia - result	Tomcat - PASS	Tracy - result
Updates Testing	Tchung - Betatest result, Releasetest result, Beta result	Juanb - Betatest result, Releasetest result, Beta result	Stephend - Betatest result, Releasetest result, Beta result	Marcia - Betatest result, Releasetest result, Beta result	Abillings - Betatest result, Releasetest result, Beta result
l10N Pri 1 locales	Tchung - PASS	N/A	Tchung - PASS	N/A	Tracy - result
Top Plugins	Tchung - result	N/A	Tracy - result	N/A	Stephend - result
Bug Verifications	Everyone - result	Everyone - result	Everyone - result	Everyone - result	Everyone - result
Live Checks & Download Links	Tomcat - result	Stephend - result	Marcia - result	Juanb - result	Abillings - result

Updates Testing

We will be doing updates of all Alpha and Beta users to RC2. We will be checking partial updates, complete updates, and failed status.

Will be available on betatest, releasetest, beta, and release (rc1 only) channels

• Betatest channel - result
• Releasetest channel - result
• Beta channel (live) - result
• Release channel (rc1 only) - result

Testcases and Results are tracked here.

L10N P1 Spotchecks

Visual inspection: Go through the P1 list of locales, and run a spotcheck to see if everything visually looks fine. Refer to L10N Requirements for more specifics.

Automation inspection: supported locales will be ran through Minotaur, and results will be reviewed by axel and mic.

Testcases and Results are tracked here - result

Bug Verifications

Bugs that are marked RESOLVED FIXED that are checked in for RC. Please verify them.

RC2+ Bugs - http://tinyurl.com/47wglk
P1/P2 Firefox bugs - http://tinyurl.com/3bfgoa
P1/P2 Platform bugs - http://tinyurl.com/354n48

Testday

A RC2 testday will be held and call for the community support to assist with testing. Activities include executing litmus testcases, bug verifications, and focus on the new feature areas that are being implemented. More can be found on QMO.

Testday results are posted here.

Live Spot Checks

Check live site download links on all platforms. Please cover en-US, and 2 or 3 additional Tier 1 locales as well. Pay attention to the following areas:

• Check for signed installer and applications (windows only)
• Firstrun page, whatsnew page, Home page, Search plugin
• Install, uninstall
• General spotchecks and browser functionality
• Confirm Build IDs

OS assignments:

• XP - Tomcat - result
• Vista - Stephend - result
• Mac Tiger - Marcia - result
• Mac Leopard - Juanb - result
• Linux - Abillings - result

Detailed Results

Smoketests

Tracy's Results

• Mac (Tiger) - Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
• Status: PASS results in Litmus

• Linux (FC7) - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008052912 Firefox/3.0
• Status: PASS results in Litmus

Template:Bug 436521 loop in plugin finder service

marcia's Results

• Mac (Leopard) - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
• Status: PASS results in Litmus
• Confirmed Bug 436210 which was seen during testing.

BFT

Tomcat's Results

• Mac 10.5 - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008052903 Firefox/3.0
• Status: PASS results in Litmus
• Confirmed Bug 432832 during the tests

Accessibility testing

• Windows: Pass results in Litmus.
• Verified bug 434002 using ChatZilla 0.9.82.1, which exposed this bug reliably for me.