MozillaWiki

Releases/Firefox 21/Test Plan

< Releases
Revision as of 13:55, 7 May 2013 by VirgilD (talk | contribs) (→‎GPU Stability Dogfooding)

« Firefox 20 | Firefox 21 | Firefox 22 »

Firefox 21 Test Plan

Summary

Milestone Firefox 21
Status Currently in Beta -- merging to Release on 2013-05-14
Lead Anthony Hughes (irc: ashughes)
2nd Juan Becerra (irc: juanb)
Builds 21.0a2

The following is the test plan for Firefox 21 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 21. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Get Involved

If you'd like to help make this version of Firefox better, please don't hesitate to contact us. We accept help from people of all backgrounds, including people with no formal QA training and experience.

If you have any of the following operating systems, we need your help:

  • Windows: XP, Vista, 7, 8
  • Mac OSX: 10.6, 10.7, 10.8
  • Linux: any modern distribution

Here are some ways you can help:

  • Install the latest build, use it as your day-to-day browser, and report any issues you encounter
  • Provide general feedback on a new feature
  • Select a fixed bug and test to verify that it is truly fixed
  • Select an unconfirmed bug and test to confirm that it is truly an issue
  • Select a regressionwindow-wanted bug and test to find the regression range

If you'd like to know more, please email Anthony Hughes.

Schedule

The following is a description of the tasks which need to be completed throughout this release.

  • [DONE] 2013-01-07: Nightly Merge
  • [DONE] 2013-02-18: Aurora Merge
  • [DONE] 2013-03-29: Pre-Beta feature sign-offs
  • [DONE] 2013-04-03: Beta 1
  • [DONE] 2013-04-11: Beta 2
  • [DONE] 2013-04-18: Beta 3
  • [DONE] 2013-04-25: Beta 4
  • 2013-05-03: Beta 5
  • 2013-05-08: Beta 6
  • 2013-05-13: Release Candidate
  • 2013-05-14: Final

Nightly

Features

See the Feature Release Tracking wiki for a general overview

Feature Owner Status Tests
H264/AAC/MP3 support for Windows 7/8 Cornel Ionce On Track. Landed in Firefox 21 but may be pref'd off at some point
3-state UI for DNT Alexandra Lucinet On Track. Landed: currently tracked in bug 765398
Mixed Content Blocker Virgil Dicu

Endurance Automation

The purpose of this section is to track any potential memory regressions via spikes reported in the dashboard.

What to Do
  1. Check the Endurance Charts
  2. If you see an anomalous event (ie. spike or dip), get the report for the specific event
  3. If the event happens twice in a row, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status (Mihaela Velimiroviciu) - Last Updated - yyyy.mm.dd

Bugzilla query error

Query options must be valid JSON.1


Functional Automation

The purpose of this section is to track any potential functional regressions via results reported in the dashboard.

What to Do
  1. Check the testrun reports
  2. If you see any failing tests, check that there are not any bugs on file for it already
  3. If not, file a bug under Mozmill Tests
  4. Notify Anthony Hughes of the issue
  5. Record the event in this wiki below
Current Status - Last Updated - yyyy-mm-dd

Bug Triage

The purpose of this section is to triage UNCONFIRMED incoming bugs.

What to Do
  1. Check the UNCONFIRMED bugs
  2. Test the bugs in the latest Firefox Nightly
  3. If you can't reproduce the bug, ask for the reporter to provide more information
  4. If you reproduce the bug, add your testing information and mark the bug NEW
  5. Notify Anthony Hughes of the NEW issue
Current Status (SoftVision) - Last Updated

Result Spreadsheet

Aurora Merge

Builds nightlies
Changeset tip (pushlog)
Checklist
  • [DONE] Run automated Mozmill tests
  • [DONE] Qualify included features
  • [DONE] Verify important bug fixes
  • [DONE] Qualify updates on auroratest channel
  • [DONE] Send sign-off to release-drivers to enable updates
  • [DONE] Qualify updates on aurora channel

Bug Fix Verifications

Bug QA Contact Status
bug 841094 "extensions" directory disappeared after updating to Feb12th Nightly Anthony Hughes [DONE] VERIFIED FIXED
bug 842039 Updater removes command line option after update Anthony Hughes [DONE] VERIFIED FIXED
bug 841919 Turn off --enable-metro on Aurora after Firefox 21 uplift to Aurora Anthony Hughes [DONE] VERIFIED FIXED
bug 843933 Disable health report on Aurora 21 Anthony Hughes [DONE] VERIFIED DISABLED

Update Testing

Platform auroratest aurora
Windows 7 32-bit [DONE]
SOURCE: Aurora 20.0a2 20130123042017
TARGET: Aurora 21.0a2 20130222111254
Mac OS X 10.7 [DONE]
SOURCE: Aurora 19.0a2 20121202042013
TARGET: Aurora 21.0a2 20130222111254
Ubuntu 12.04 64-bit [DONE]
SOURCE: Aurora 20.0a2 20130201042018
TARGET: Aurora 21.0a2 20130222111254
Issues

AURORA Channel

Expected target build: Aurora 21.0a2 en-US 20130222111254

Windows 7 32-bit

  • [DONE] Aurora 20.0a2 en-US 20130123042017 -> Aurora 21.0a2 en-US 20130222111254
  • FAIL Aurora 20.0a2 it 20130123042017 -> Aurora 20.0a2 20130219042021 (see bug 844336)

Mac OSX 10.7

  • [DONE] Aurora 19.0a2 en-US 20121202042013 -> Aurora 21.0a2 en-US 20130222111254
  • [DONE] Aurora 19.0a2 fr 20121202042013 -> Aurora 21.0a2 fr 20130222111254

Ubuntu 12.04 32-bit

  • [DONE] Aurora 19.0a2 en-US 20130201042018 -> Aurora 21.0a2 en-US 20130222111254
  • [DONE] Aurora 19.0a2 es-ES 20130201042018 -> Aurora 21.0a2 es-ES 20130222111254

Feature Sign Off

Feature Owner Win32 Mac Linux
Mixed Content Blocker Virgil Dicu [DROPPED] Scoped for Fx22 [DROPPED] Scoped for Fx22 [DROPPED] Scoped for Fx22
H264/AAC/MP3 support for Windows 7/8 Cornel Ionce [DONE] Ready for Aurora N/A N/A
3-state UI for DNT Alexandra Lucinet [DONE] Ready for Aurora [DONE] Ready for Aurora [DONE] Ready for Aurora
JavaScript Profiling Mihaela Velimiroviciu [DONE] Ready for Aurora [DONE] Ready for Aurora [DONE] Ready for Aurora

Platforms: Windows 7 64bit, Windows 8 32bit, Ubuntu 12.04, Mac OSX 10.8.2.

l10n

Checklist
  • [DONE] Functional testing via mozmill-ci: win32 fr
  • [DONE] Update testing via mozmill-ci: win32 fr
  • [DONE] Manual spotcheck functional and updates for de, it, es-ES
  • [DONE] Download links verified [1]
  • [DONE] Verify fixed bug 844262, bug 844336

Beta 1

Builds FTP, HTTP
ID 20130401192816
Changeset 2e91ff229d84 (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Feature Sign-off
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Results [dashboard]
Windows XP [DONE] en-US & da
Windows Vista [DONE] en-US & it
Windows 7 32-bit [DONE] en-US & zh-TW
Windows 7 64-bit [DONE] en-US & ro
Windows 8 32-bit [DONE] en-US & es-ES
Windows 8 64-bit [DONE] en-US & pt-BR
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & fr
Mac OSX 10.8 [DONE] en-US & it
Ubuntu 12.04 32-bit [DONE] en-US & de
Ubuntu 12.04 64-bit [DONE] en-US & fr

Known test failures in bugzilla

Feature Sign Off

Regression Tests

As per the checklist we will regression test the following areas.

Test Windows 7 32-bit Windows 8 64-bit Mac OSX 10.8 Ubuntu 12.04 32-bit
Breakpad end-to-end functionality [DONE] Manuela [DONE] Alexandra [DONE] Virgil [DONE] Cornel
Mac OSX 10.8 code signing N/A N/A [DONE] Virgil N/A
HTML5 Audio/Video playback [DONE] Manuela [DONE] Alexandra [DONE] Virgil [DONE] Cornel
Feedly Add-on Compatibility [DONE] Manuela [DONE] Alexandra [DONE] Virgil [DONE] Cornel
Tab animation with D2D on/off (bug 838758) [DONE] Manuela [DONE] Alexandra N/A N/A

Bug Fix Verifications

The following bugs have been fixed and need confirmation:

  • [DONE] bug 717178 Printing stops after 73%, and browser hangs up (Paul Silaghi)
  • [DONE] bug 738952 "Save as..." File menu entry or Ctrl+S produces unexpected results when having a PDF file opened within PDF Viewer (Ioana Budnar)
  • [ON TRACK] bug 799523 [Retina] Crash on entering full screen mode in OS X (Matt Wobensmith) - Pending verification
  • [DONE] bug 803568 Nightly crashed when see pdf file in fullscreen (presentation mode) or zooming (Ioana Budnar)
  • [DONE] bug 835045 - Core Graphics stops rendering the canvas content (Ioana Budnar)
  • [DONE] bug 836044 Aurora stub installer doesn't seem to be working (Paul Silaghi)
  • [DONE] bug 837486 Reposition pinned tabs when resetting tab strip underflow (Anthony Hughes)
  • [DONE] bug 840195 Update fails if FF is installed in a non-default install path (Paul Silaghi)
  • [DONE] bug 841239 Pandora.com stops playing music due to jplayer error frequently (Stephen Donner)
  • [DONE] bug 843671 - Cannot enter comment in plugin crash UI (Anthony Hughes)
  • [DONE] bug 844466 - Tab bar doesn't render tab for link from a modal dialog until dialog is closed (Ioana Budnar)
  • [DONE] bug 844553 - Plugins disabled by Aurora 21.x (Paul Silaghi)
  • [DROPPED] bug 844561 - Firefox prompts when closing private window (Ioana Budnar) - Pending fix for bug 854926
  • [DONE] bug 844671 - Tabs sent through Sync should not open in private browsing windows (Virgil Dicu)
  • [DONE] bug 845486 webgl conformance crashes frequently on ubuntu VMs (Anthony Hughes)
  • [DONE] bug 845592 - New Private Window entry in File menu doesn't make sense in permanent private browsing (Ioana Budnar)
  • [DONE] bug 845837 - Drag and Drop features in Blackboard Learn 9.1 SP9 broken after updating to Firefox 19 (Bogdan Maris)
  • [DONE] bug 846186 - Tilt error flooding error console (TypeError: mouseLerp is undefined) (Ioana Budnar)
  • [DONE] bug 846334 - Panels are flickering when opened in Social API (Bogdan Maris)
  • [DONE] bug 846617 - Text shaping with AAT Myanmar fonts broken in Firefox 19 (Simona Marcu)
  • [DONE] bug 846843 Profile directory (containing PII) may be submitted in Firefox Health Report errors (Anthony Hughes)
  • [DONE] bug 847627 - The clear Downloads History at shutdown option should be updated (Simona Marcu)
  • [DONE] bug 848794 Can't re-run Firefox stub installer after cancelling Install (Virgil Dicu)
  • [DONE] bug 850077 - Tooltips do not work in the sidebar (Simona Marcu)
  • [DONE] bug 851436 - Pinned tabs sometimes scroll with normal tabs (Bogdan Maris)
  • [DONE] bug 851473 Restart dialogues say opposite feature enable/disable when toggling Private browsing (Ioana Budnar)
  • [DONE] bug 851495 Text shaping with AAT Khmer fonts broken in Firefox 19 (Virgil Dicu)
  • [DONE] bug 852530 - notificationBox is not visible on Linux (Bogdan Maris)
  • [DONE] bug 852569 Cannot uninstall provider when social is disabled (Ioana Budnar)
  • [DONE] bug 853387 - Crash in nsNPAPIPluginInstance::HandleEvent @ libnpgeplugin with Google Earth Plugin (Simona Marcu)

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & ru 		[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & ru 		[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 20.0b7 en-US & es-ES 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 20.0b7 en-US & es-ES 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 20.0b7 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 20.0b7 en-US & da 		[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 20.0b7 en-US & da 		[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 20.0b7 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 20.0b7 en-US & ro 		[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 20.0b7 en-US & ro 		[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 20.0b7 en-US & ro
Windows 8 32-bit [DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 20.0b7 en-US & da 		[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 20.0b7 en-US & da 		[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 20.0b7 en-US & da
Windows 8 64-bit [DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 20.0b7 en-US & zh-TW 		[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 20.0b7 en-US & zh-TW 		[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 20.0b7 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 20.0b7 en-US & es-ES 		[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 20.0b7 en-US & es-ES 		[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 20.0b7 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 20.0b7 en-US & zh-TW 		[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 20.0b7 en-US & zh-TW 		[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 20.0b7 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 20.0b7 en-US & ko 		[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 20.0b7 en-US & ko 		[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 20.0b7 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 20.0b7 en-US & ru 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 20.0b7 en-US & ru 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 20.0b7 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & it 		[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & it 		[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 20.0b7 en-US & it
Bugs
  • VERIFIED FIXED bug 857380 21b1 update tests failing with NS_ERROR_UNEXPECTED in nsIFileOutputStream.init

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 857096 Java drop down list not displayed properly

Beta 2

Builds FTP, HTTP
ID 20130408165307
Changeset c4dfe07f855c (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • {{ |}} Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Results [dashboard]
Windows XP [DONE] en-US & da
Windows Vista [DONE] en-US & it
Windows 7 32-bit [DONE] en-US & zh-TW
Windows 7 64-bit [DONE] en-US & ro
Windows 8 32-bit [DONE] en-US & es-ES
Windows 8 64-bit [DONE] en-US & pt-BR
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & fr
Mac OSX 10.8 [DONE] en-US & it
Ubuntu 12.04 32-bit [DONE] en-US & de
Ubuntu 12.04 64-bit [DONE] en-US & fr

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Regression testrun in Moztrap

Test Windows 7 64-bit Windows 8 32-bit Mac OSX 10.7 Ubuntu 12.04 32-bit
Breakpad end-to-end functionality (run test in Moztrap) [DONE] Simona [DONE] Bogdan [DONE] Manuela [DONE] Mihaela
MD5 as a hash algorithm in signatures (run test in Moztrap) [DONE] Simona [DONE] Bogdan [DONE] Manuela [DONE] Mihaela
Flash 11.7 Beta (download and test Flash websites) [DONE] Simona [DONE] Bogdan [DONE] Manuela [DROPPED] Mihaela
Exploratory testing involving UNC path names [DONE] Simona [DONE] Bogdan [DONE] Manuela [DROPPED] Mihaela (*)

(*) Cannot access Network from the Choose Folder dialog (tried on several machines and also with other SW - open file in Google Chrome)

Notes on UNC Testing
  • UNC testing can be done by using a profile located on a Windows share (use Choose Folder in the Profile Manager)
  • Also test with a Windows system configured to use Roaming Profiles if at all possible
  • Behaviours to test include basic navigation, bookmarks, history, session restore, search and search engine management, profile migration, cookies, add-ons, software update, etc
  • Include other types of network sharing (smb, afp, USB drive) to run profiles from

Bug Fix Verifications

The following bugs have been fixed and need confirmation:

  • [ON TRACK] bug 799523 [Retina] Crash on entering full screen mode in OS X (Matt Wobensmith)
    • Pending NEEDINFO? from Jonathan Kew
  • [MISSED] bug 857199 Please start offering the stub installer for English/Win32 on Beta (Cornel)
    • Issue is not yet fixed
  • [DONE] bug 846848 [OS.File] Handle UNC-formatted paths (Paul)
  • [DONE] bug 857672 Address Bar not working (Paul)
  • [DONE] bug 857588 Search engine list empty when the profile is on a network drive (Paul)
  • [DONE] bug 858704 provider manifests (ashughes)
  • [DONE] bug 858305 Need change to abouthealth.css to work around lack of flexbox support (Alexandra)
  • [DONE] bug 827979 add Growl support back (Cornel)
  • [DONE] bug 828815 SVGRectElement["onxxx"] = Function does not work, No Tooltip on SVG graphic (StatCounter) (Cornel)
  • [DONE] bug 830066 Drag & drop links into toolkit downloads manager to download files fails (TypeError: aInitiatingDocument is undefined) (Cornel)
  • [DONE] bug 827954 Firefox button overlaps the first tab in private windows (Cornel)
  • [DONE] bug 827460Copy/paste of "About Firefox" window's build version text no longer works when the window is closed (Alexandra)
  • [DONE] bug 834075 pdf.js: The PDF url in the address bar becomes wrong in a certain situation. (Alexandra)
  • [DONE] bug 831472 Crash [@ nsGlobalWindow::GetInnerScreenRect()] due to lack of null check (Alexandra)
  • [DONE] bug 829870 Appear the non-expected color gap on window that are opened from private one. (Alexandra)
  • [DONE] bug 842290 "Restart Aurora" pop up doesn't appear when selecting private browsing mode (Alexandra)
  • [DONE] bug 857291 SPNEGO / MS KRB5 no longer working. Tries to use NTLM SSP instead (reporter)
  • [DONE] bug 844406 Greasemonkey script stops working correctly in Firefox 20. 'self-hosted' builtin functions destroy js-callstack in certain case. (Manuela)

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & ru 		[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & ru 		[DONE] 18.0b5 en-US & fr
[DONE] 19.0b5 en-US & ko
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 21.0b1 en-US & es-ES 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 21.0b1 en-US & es-ES 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & pt-BR
[DONE] 20.0b4 en-US & it
[DONE] 21.0b1 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b1 en-US & da 		[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b1 en-US & da 		[DONE] 18.0b2 en-US & es-ES
[DONE] 19.0b2 en-US & zh-TW
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b1 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 21.0b1 en-US & ro 		[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 21.0b1 en-US & ro 		[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & de
[DONE] 20.0b3 en-US & ko
[DONE] 21.0b1 en-US & ro
Windows 8 32-bit [DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 21.0b1 en-US & da 		[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 21.0b1 en-US & da 		[DONE] 18.0b6 en-US & es-ES
[DONE] 19.0b6 en-US & zh-TW
[DONE] 20.0b6 en-US & ro
[DONE] 21.0b1 en-US & da
Windows 8 64-bit [DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 21.0b1 en-US & zh-TW 		[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 21.0b1 en-US & zh-TW 		[DONE] 18.0b1 en-US & fr
[DONE] 19.0b1 en-US & it
[DONE] 20.0b1 en-US & es-ES
[DONE] 21.0b1 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 21.0b1 en-US & es-ES 		[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 21.0b1 en-US & es-ES 		[DONE] 18.0b1 en-US & de
[DONE] 19.0b1 en-US & ja-JP-mac
[DONE] 20.0b1 en-US & it
[DONE] 21.0b1 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 21.0b1 en-US & zh-TW 		[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 21.0b1 en-US & zh-TW 		[DONE] 18.0b2 en-US & pt-BR
[DONE] 19.0b2 en-US & fr
[DONE] 20.0b2 en-US & ru
[DONE] 21.0b1 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko 		[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko 		[DONE] 18.0b3 en-US & it
[DONE] 19.0b3 en-US & ro
[DONE] 20.0b3 en-US & de
[DONE] 21.0b1 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 21.0b1 en-US & ru 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 21.0b1 en-US & ru 		[DONE] 18.0b4 en-US & de
[DONE] 19.0b4 en-US & ko
[DONE] 20.0b4 en-US & zh-TW
[DONE] 21.0b1 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & it 		[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & it 		[DONE] 18.0b5 en-US & es-ES
[DONE] 19.0b5 en-US & pt-BR
[DONE] 20.0b5 en-US & de
[DONE] 21.0b1 en-US & it
Bugs
  • [FIXED] bug 859977 Firefox 21.0b2 Windows 7 fallback update tests failing with "channelprefs.js not found"
    • Mitigated by manually spotchecking Windows 7 fall-back updates

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 859977 Firefox 21.0b2 Windows 7 fallback update tests failing with "channelprefs.js not found"

Beta 3

Builds FTP, HTTP
ID 20130416200523
Changeset e845a10035f2 (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & da
Windows Vista [DONE] en-US & ro
Windows 7 32-bit [DONE] en-US & pt-BR
Windows 7 64-bit [DONE] en-US & ru
Windows 8 32-bit [DONE] en-US & zh-TW
Windows 8 64-bit [DONE] en-US & ko
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & ja-JP-mac
Mac OSX 10.8 [DONE] en-US & it
Ubuntu 12.04 32-bit [DONE] en-US & fr
Ubuntu 12.04 64-bit [DONE] en-US & es-ES

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Regression testrun in Moztrap

Test Windows 7 32-bit Windows Vista Mac OSX 10.6 Ubuntu 12.04 32-bit
Breakpad end-to-end functionality (run test in Moztrap) [DONE] Manuela [DONE] Simona [DONE] Cornel [DONE] Virgil
Mac OSX 32-bit mode (run test in Moztrap) N/A Manuela N/A Simona [DONE] Cornel N/A Virgil
Web compatibility (search, webmail, video/audio, maps, cms, etc) [DONE] Manuela [DONE] Simona [DONE] Cornel [DONE] Virgil
Exploratory testing around Private Browsing and Session Restore [DONE] Manuela [DONE] Simona [DONE] Cornel [DONE] Virgil
Exploratory testing around Mozmill failures (en-US & l10n) Windows XP Windows 8 32-bit Mac OSX 10.7 Ubuntu 12.04 32-bit
Search and search selection [DONE] Simona [DONE] Manuela [DONE] Cornel [DONE] Virgil
Safe browsing warning pages [DONE] Simona [DONE] Manuela [DONE] Cornel [DONE] Virgil
Bookmarks, including default bookmarks [DONE] Simona [DONE] Manuela [DONE] Cornel [DONE] Virgil

Bug Fix Verifications

The following bugs have been fixed and need confirmation:

Unassigned:

  • n/a


[BLOCKED]

  • bug 840881 Wrong menu rendering with layout.css.devPixelsPerPx non-default (HI-DPI displays) - Blocked by bug 861270


[ON TRACK]

  • bug 799523 [Retina] Crash on entering full screen mode in OS X (Matt Wobensmith)
  • bug 857199 Please start offering the stub installer for English/Win32 on Beta (Paul Silaghi)


[DONE]

  • bug 797288 Pdf.js download behavior inconsistent (Ioana Budnar)
  • bug 822933 WebM playback regression (Paul Silaghi)
  • bug 825035 Blocklisted click-to-play notification only fades in/out once per window per session (Paul Silaghi)
  • bug 825620 Crash Reporter's locale different from the build's one (Alexandra Lucinet)
  • bug 833823 Random, unpredictable behavior with multiple YouTube embeds (Paul Silaghi)
  • bug 837845 Multi-GB memory spike when using regular expressions (Bogdan Maris)
  • bug 839031 crash in mozilla::wmf::MFCreateSourceReaderFromByteStream @ mfflac with FLAC 1.2.0 (Bogdan Maris)
  • bug 840739 Flash links in private windows are converting normal windows into private windows. (Bogdan Maris)
  • bug 842577 Remove from History is visible for a paused download (Ioana Budnar)
  • bug 843342 PDF doesn't print correctly on USPS.com with pdfjs (Alexandra Lucinet)
  • bug 845681 Closing all windows and quitting Firefox afterwards leads to about:sessionrestore to be shown (Paul Silaghi)
  • bug 852295 LW Themes do not display in Permanent Private Browsing mode post Per-Window PB (Ioana Budnar)
  • bug 854175 permanent PB mode adds "(Private Browsing)" to window title (Ioana Budnar)
  • bug 854926 Firefox prompts a warning when closing private windows with multiple tabs open (Ioana Budnar)
  • bug 856784 crash in gfxUserFontSet::UserFontCache::ForgetFont (Ioana Budnar)
  • bug 819377 Hang on https://www.mozilla.org/en-US/contribute/ when playing video (Bogdan Maris)
  • bug 844031 Fail to detect the correct version of Intel graphics driver on Windows (Alexandra Lucinet)
  • bug 839515 Tiny fonts in the remote connection screen (Ioana Budnar)
  • bug 861188 Firefox 20 makes svg-edit place opened SVG images in lower-right corner instead of center (Paul Silaghi)

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 17.0b6 en-US & fr
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b2 en-US & dr
[DONE] 20.0b2 en-US & ru 		[DONE] 17.0b6 en-US & fr
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b2 en-US & dr
[DONE] 20.0b2 en-US & ru 		[DONE] 17.0b6 en-US & fr
[DONE] 18.0b1 en-US & ko
[DONE] 19.0b2 en-US & dr
[DONE] 20.0b2 en-US & ru
Windows Vista [DONE] 17.0b5 en-US & de
[DONE] 18.0b6 en-US & pt-BR
[DONE] 19.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES 		[DONE] 17.0b5 en-US & de
[DONE] 18.0b6 en-US & pt-BR
[DONE] 19.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES 		[DONE] 17.0b5 en-US & de
[DONE] 18.0b6 en-US & pt-BR
[DONE] 19.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
Windows 7 32-bit [DONE] 17.0b3 en-US es-ES
[DONE] 18.0b4 en-US zh-TW
[DONE] 19.0b5 en-US fr
[DONE] 21.0b2 en-US da 		[DONE] 17.0b3 en-US es-ES
[DONE] 18.0b4 en-US zh-TW
[DONE] 19.0b5 en-US fr
[DONE] 21.0b2 en-US da 		[DONE] 17.0b3 en-US es-ES
[DONE] 18.0b4 en-US zh-TW
[DONE] 19.0b5 en-US fr
[DONE] 21.0b2 en-US da
Windows 7 64-bit [DONE] 17.0b4 en-US es-ES
[DONE] 18.0b5 en-US de
[DONE] 19.0b6 en-US ko
[DONE] 21.0b2 en-US ro 		[DONE] 17.0b4 en-US es-ES
[DONE] 18.0b5 en-US de
[DONE] 19.0b6 en-US ko
[DONE] 21.0b2 en-US ro 		[DONE] 17.0b4 en-US es-ES
[DONE] 18.0b5 en-US de
[DONE] 19.0b6 en-US ko
[DONE] 21.0b2 en-US ro
Windows 8 32-bit [DONE] 17.0b1 en-US es-ES
[DONE] 18.0b2 en-US zh-TW
[DONE] 19.0b3 en-US ro
[DONE] 21.0b2 en-US da 		[DONE] 17.0b1 en-US es-ES
[DONE] 18.0b2 en-US zh-TW
[DONE] 19.0b3 en-US ro
[DONE] 21.0b2 en-US da 		[DONE] 17.0b1 en-US es-ES
[DONE] 18.0b2 en-US zh-TW
[DONE] 19.0b3 en-US ro
[DONE] 21.0b2 en-US da
Windows 8 64-bit [DONE] 17.0b2 en-US fr
[DONE] 18.0b3 en-US it
[DONE] 19.0b4 en-US es-ES
[DONE] 21.0b2 en-US zh-TW 		[DONE] 17.0b2 en-US fr
[DONE] 18.0b3 en-US it
[DONE] 19.0b4 en-US es-ES
[DONE] 21.0b2 en-US zh-TW 		[DONE] 17.0b2 en-US fr
[DONE] 18.0b3 en-US it
[DONE] 19.0b4 en-US es-ES
[DONE] 21.0b2 en-US zh-TW
Mac OSX 10.6 [DONE] 17.0b1 en-US de
[DONE] 18.0b2 en-US ja-JP-mac
[DONE] 19.0b3 en-US it
[DONE] 21.0b2 en-US es-ES 		[DONE] 17.0b1 en-US de
[DONE] 18.0b2 en-US ja-JP-mac
[DONE] 19.0b3 en-US it
[DONE] 21.0b2 en-US es-ES 		[DONE] 17.0b1 en-US de
[DONE] 18.0b2 en-US ja-JP-mac
[DONE] 19.0b3 en-US it
[DONE] 21.0b2 en-US es-ES
Mac OSX 10.7 [DONE] 17.0b2 en-US pt-BR
[DONE] 18.0b3 en-US fr
[DONE] 19.0b4 en-US ru
[DONE] 21.0b2 en-US zh-TW 		[DONE] 17.0b2 en-US pt-BR
[DONE] 18.0b3 en-US fr
[DONE] 19.0b4 en-US ru
[DONE] 21.0b2 en-US zh-TW 		[DONE] 17.0b2 en-US pt-BR
[DONE] 18.0b3 en-US fr
[DONE] 19.0b4 en-US ru
[DONE] 21.0b2 en-US zh-TW
Mac OSX 10.8 [DONE] 17.0b3 en-US it
[DONE] 18.0b4 en-US ro
[DONE] 19.0b5 en-US de
[DONE] 21.0b2 en-US ko 		[DONE] 17.0b3 en-US it
[DONE] 18.0b4 en-US ro
[DONE] 19.0b5 en-US de
[DONE] 21.0b2 en-US ko 		[DONE] 17.0b3 en-US it
[DONE] 18.0b4 en-US ro
[DONE] 19.0b5 en-US de
[DONE] 21.0b2 en-US ko
Ubuntu 12.04 32-bit [DONE] 17.0b5 en-US de
[DONE] 18.0b6 en-US ko
[DONE] 19.0b1 en-US zh-TW
[DONE] 21.0b2 en-US ru 		[DONE] 17.0b5 en-US de
[DONE] 18.0b6 en-US ko
[DONE] 19.0b1 en-US zh-TW
[DONE] 21.0b2 en-US ru 		[DONE] 17.0b5 en-US de
[DONE] 18.0b6 en-US ko
[DONE] 19.0b1 en-US zh-TW
[DONE] 21.0b2 en-US ru
Ubuntu 12.04 64-bit [DONE] 17.0b6 en-US es-ES
[DONE] 18.0b1 en-US pt-BR
[DONE] 19.0b2 en-US de
[DONE] 21.0b2 en-US it 		[DONE] 17.0b6 en-US es-ES
[DONE] 18.0b1 en-US pt-BR
[DONE] 19.0b2 en-US de
[DONE] 21.0b2 en-US it 		[DONE] 17.0b6 en-US es-ES
[DONE] 18.0b1 en-US pt-BR
[DONE] 19.0b2 en-US de
[DONE] 21.0b2 en-US it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 863183 The remote debugger "Connecting..." string and animation should be larger
  • bug 863238 Not localized mozilla pages opened from the Firefox options
  • bug 863256 Report Web Forgery warning is not always displayed

Beta 4

Builds FTP, HTTP
ID 20130423212553
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & pl & ro
Windows Vista [DONE] en-US & pl & ko
Windows 7 32-bit [DONE] en-US & pl & da
Windows 7 64-bit [DONE] en-US & pl & it
Windows 8 32-bit [DONE] en-US & pl & ru
Windows 8 64-bit [DONE] en-US & pl & ja-JP
Mac OSX 10.6 [DONE] en-US & pl & de
Mac OSX 10.7 [DONE] en-US & pl & es-ES
Mac OSX 10.8 [DONE] en-US & pl & fr
Ubuntu 12.04 32-bit [DONE] en-US & pl & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & pl & zh-TW

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Regression testrun in Moztrap

Test Windows 7 64-bit Windows XP Mac OSX 10.8 Ubuntu 12.10 32-bit
Breakpad end-to-end functionality (run test in Moztrap) [DONE] Paul [DONE] Bogdan [DONE] Virgil [DONE] Manuela
Verify update applies with root owned files (use betatest channel) [DONE] Paul [DONE] Bogdan [DONE] Virgil [DONE] Manuela
Add-on compatibility (top third-party and addons.mozilla.org hosted) [DONE] Paul [DONE] Bogdan [DONE] Virgil [DONE] Manuela
  • Web compatibility exploratory testing around drop-down select menus and autocomplete forms etherpad

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 862606 [summary hidden] Matt Wobensmith [ON TRACK]
bug 861606 Add some assertions and guards for dir=auto (see also bug 861607, bug 861610) Anthony Hughes [DROPPED]
bug 861455 Daily collection when upload is disabled does not collect pull only providers Anthony Hughes [ON TRACK]
bug 861270 menu behavior is unstable depending on value of devPixelsPerPx (see also bug 840881) Cornel Ionce [DONE]
bug 860180 U+30D8 causes XML error in UTF-16BE Anthony Hughes [DROPPED]
bug 859172 Firefox now trying to mistakenly decode non-UTF8 links as UTF-8 in UTF-8 webpages Alexandra Lucinet [DONE]
bug 858459 drop-down select menu closes without selecting in Firefox 20.0 Ioana Budnar [DONE]
bug 857817 [NBV] Consumes all client resources when load is slightly increased Alexandra Lucinet [DONE]
bug 857199 Please start offering the stub installer for English/Win32 on Beta Paul Silaghi [ON TRACK] Pending web-dev fixes
bug 855024 High-level collection APIs aren't safe when run in parallel Ioana Budnar [DROPPED]
bug 853291 Add Polish (pl) to gecko localizations, central, mozilla-b2g18, mozilla-b2g18_v1_0_1 Anthony Hughes [DONE]
bug 852315 [summary hidden] Matt Wobensmith [ON TRACK]
bug 851630 Can't open popup notifications with the keyboard (focus gets stuck at the identity icon) Cornel Ionce [DONE]
bug 848237 [summary hidden] Matt Wobensmith [DROPPED]
bug 845743 Firefox 19.0 does not support BIG5-HKSCS encoding. Version 18.0 does Simona Badau [DONE]
bug 845555 Context menu item to switch to hide statistics instead of offering to show statistics again Cornel Ionce [DONE]
bug 838864 keyword.URL reset interferes poorly with some partner add-ons Manuela Muntean [DONE]
bug 838239 [hidpi] wrong devicePixelRatio in browser history Matt Wobensmith [DONE]
bug 826349 video unmutes itself after seeking Cornel Ionce [DONE]
bug 825545 crash in js_NewStringCopyN Simona Badau [DONE]
bug 816498 Table backgrounds don't print, starting from FX17 Simona Badau [DONE]
bug 799523 [Retina] Crash on entering full screen mode in OS X Matt Wobensmith [DONE]
Totals
  • 64% [DONE]
  • 18% [DROPPED]
  • 18% [ON TRACK]

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ru 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ru 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ru
Windows Vista [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b3 en-US & ru 		[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b3 en-US & ru 		[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b3 en-US & ru
Windows 7 32-bit [DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b3 en-US & da 		[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b3 en-US & da 		[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b3 en-US & da
Windows 7 64-bit [DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b3 en-US & ro 		[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b3 en-US & ro 		[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b3 en-US & ro
Windows 8 32-bit [DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b3 en-US da 		[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b3 en-US da 		[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b3 en-US da
Windows 8 64-bit [DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 21.0b3 en-US & zh-TW 		[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 21.0b3 en-US & zh-TW 		[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b2 en-US & es-ES
[DONE] 21.0b3 en-US & zh-TW
Mac OSX 10.6 [DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES 		[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES 		[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
Mac OSX 10.7 [DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b3 en-US & zh-TW 		[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b3 en-US & zh-TW 		[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b3 en-US & zh-TW
Mac OSX 10.8 [DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ko 		[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ko 		[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & ko
Ubuntu 12.04 32-bit [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b3 en-US & ru 		[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b3 en-US & ru 		[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b3 en-US & ru
Ubuntu 12.04 64-bit [DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & it 		[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & it 		[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b3 en-US & it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

  • bug 865267 Duplicate folders in bookmarks menu when one folder is dragged and dropped intro another
  • bug 865672 Number of bookmarks doesn't update in about:healthreport

Beta 5

ADVISORY: THIS IS EFFECTIVELY A RESPIN OF 21.0b4 TO ADDRESS A POTENTIAL STABILITY CONCERN

Builds FTP, HTTP
ID 20130425162858
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [ON TRACK] Update Tests (beta)
  • Manual testing reduced to basic dogfooding since this is built from the same changeset as 21.0b4

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & ja
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & zh-TW

Known test failures in bugzilla

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows Vista [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b4 en-US & ru 		[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b1 en-US & es-ES
[DONE] 21.0b4 en-US & ru 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 7 32-bit [DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b3 en-US & fr
[DONE] 21.0b4 en-US & da 		[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b3 en-US & fr
[DONE] 21.0b4 en-US & da 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 7 64-bit [DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b4 en-US & ro 		[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b4 en-US & ro 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 8 32-bit [DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b4 en-US da 		[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b1 en-US ro
[DONE] 21.0b4 en-US da 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Windows 8 64-bit [DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b4 en-US & zh-TW 		[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b3 en-US & es-ES
[DONE] 21.0b4 en-US & zh-TW 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Mac OSX 10.6 [DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES 		[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES 		[DONE] 19.0b5 en-US & fr*
[DONE] 20.0b5 en-US & ko*
[DONE] 21.0b2 en-US & de*
[DONE] 21.0b4 en-US & ru*
Mac OSX 10.7 [DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b4 en-US & zh-TW 		[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b4 en-US & zh-TW 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Mac OSX 10.8 [DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b3 en-US & de
[DONE] 21.0b4 en-US & ko 		[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b3 en-US & de
[DONE] 21.0b4 en-US & ko 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Ubuntu 12.04 32-bit [DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b4 en-US & ru 		[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b4 en-US & ru 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru
Ubuntu 12.04 64-bit [DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & it 		[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & it 		[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b2 en-US & de
[DONE] 21.0b4 en-US & ru

1. Mac OSX 10.6 results were tested manually due to a Mozmill Infrastructure issue: bug 867005

Beta 6

Builds FTP, HTTP
ID 20130430204233
Changeset changeset (pushlog)
Checklist
  • [DONE] Automated Tests
  • [DONE] Regression Tests
  • [DONE] Bug Fix Verifications
  • [DONE] Update Tests (betatest)
  • [DONE] Update Tests (releasetest)
  • [DONE] Update Tests (beta)

Stability

Beta candidate crashes per 100 ADIs on ship day

  • 21.0b3: 4.2 (2013-04-18)
  • 21.0b5: 4.5 (2013-04-29)
  • 21.0b6: 1.5 (2013-05-02)

Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results [dashboard]
Windows XP [DONE] en-US & ro
Windows Vista [DONE] en-US & ko
Windows 7 32-bit [DONE] en-US & da
Windows 7 64-bit [DONE] en-US & it
Windows 8 32-bit [DONE] en-US & ru
Windows 8 64-bit [DONE] en-US & ja-JP
Mac OSX 10.6 [DONE] en-US & de
Mac OSX 10.7 [DONE] en-US & es-ES
Mac OSX 10.8 [DONE] en-US & fr
Ubuntu 12.04 32-bit [DONE] en-US & pt-BR
Ubuntu 12.04 64-bit [DONE] en-US & zh-TW

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Windows XP Windows 8 64-bit Mac OSX 10.7 Ubuntu 13.04 32-bit
[DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Issues:
  • No issues found

Plug-in Compatibility

Goal: Ensure plug-ins and plug-in content remain compatibile and stable in the current build.

Suggested Plugins:

  • Flash 11.7, 11.6, 11.2, and 10.3
  • Java 7, 6
  • Apple QuickTime
  • Microsoft Silverlight 5
  • Microsoft Windows Media Player
  • Flip4Mac
  • Unity 3D
  • Skype Click-to-Call
  • RealPlayer


Plug-in Windows XP Windows 8 64-bit Mac OSX 10.7 Ubuntu 13.04 32-bit
Flash Player 10.3 [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Flash Player 11.2 [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Flash Player 11.6 [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Flash Player 11.7 [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Java 6 [DONE] Paul [DONE] Alexandra N/A Virgil [DONE] Mihaela
Java 7 [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Apple QuickTime [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Microsoft Silverlight 5 [DONE] Paul [DONE] Alexandra [DONE] Virgil [DONE] Mihaela
Microsoft Windows Media Player [DONE] Paul N/A Alexandra [DONE] [using flip4mac] Virgil [DONE] Mihaela
Unity 3D [DONE] Paul [DONE] Alexandra [DONE] Virgil N/A Mihaela
Skype Click-to-Call [DONE] Paul N/A Alexandra N/A Virgil N/A Mihaela
RealPlayer [DONE] Paul [DONE] Alexandra N/A Virgil N/A Mihaela
Issues:
  • [known] bug 857096 - Java dropdown list not displayed properly
  • [known] bug 767673 - windows media player plugin shows blank content
  • [known] bug 692446 - Firefox prompts to install windows media player plugin even though it is already installed
  • [new] bug 868005 - Java applet steals focus from the location bar

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 867005 Mac OSX 10.6 nodes failing on release1.qa.mtv1.mozilla.com Anthony Hughes [DONE]
bug 862606 [summary hidden] Matt Wobensmith [DONE]
bug 861455 Daily collection when upload is disabled does not collect pull only providers Anthony Hughes [DONE]
bug 861112 Flash plugin incorrectly positioned when using non-standard layout.css.devPixelsPerPx value Cornel Ionce [DONE]
bug 860897 stub installer localization Cornel Ionce [DONE]
bug 859099 crash in nsObjectLoadingContent::DoStopPlugin (also verify bug 863792) Simona Marcu, Paul Silaghi [DROPPED] see comment #57
bug 857199 Please start offering the stub installer for English/Win32 on Beta Paul Silaghi [ON TRACK]
bug 851378 Move tearing down plugin prototype from NPRuntime to content Ioana Budnar [DROPPED] see comment #26.
bug 847272 "ASSERTION: caching a font associated with no family yet" Cornel Ionce [DONE]
bug 838105 crash in gfxUserFontSet::UserFontCache::Entry::KeyEquals Anthony Hughes [DROPPED] Comment #13
bug 810726 Infinite loop in nsBlockInFlowLineIterator::FindValidLine() with specific layout Cornel Ionce [DROPPED] Comment #84
bug 831780 scrollWidth/Height of DIV containing IMG reported as 40x40px regardless of CSS sizing Ioana Budnar [DONE]
bug 743252 Don't print the URL and other information when printing PDFs Ioana Budnar [DONE]
Totals
  • 62% [DONE]
  • 31% [DROPPED]
  • 7% [ON TRACK]

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest [dashboard] Releasetest [dashboard] Beta [dashboard]
Windows XP [DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ru 		[DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ru 		[DONE] 18.0b5 en-US & it
[DONE] 19.0b5 en-US & fr
[DONE] 20.0b5 en-US & ko
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ru
Windows Vista [DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b3 en-US & it
[DONE] 21.0b5 en-US & es-ES 		[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b3 en-US & it
[DONE] 21.0b5 en-US & es-ES 		[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & pt-BR
[DONE] 21.0b3 en-US & it
[DONE] 21.0b5 en-US & es-ES
Windows 7 32-bit [DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b5 en-US & da 		[DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b5 en-US & da 		[DONE] 18.0b2 en-US & ru
[DONE] 19.0b2 en-US & es-ES
[DONE] 20.0b2 en-US & zh-TW
[DONE] 21.0b1 en-US & fr
[DONE] 21.0b5 en-US & da
Windows 7 64-bit [DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b5 en-US & ro 		[DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b5 en-US & ro 		[DONE] 18.0b3 en-US & pt-BR
[DONE] 19.0b3 en-US & es-ES
[DONE] 20.0b3 en-US & de
[DONE] 21.0b2 en-US & ko
[DONE] 21.0b5 en-US & ro
Windows 8 32-bit [DONE] 18.0b6 en-US & pl
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b3 en-US & ro
[DONE] 21.0b5 en-US & da 		[DONE] 18.0b6 en-US & pl
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b3 en-US & ro
[DONE] 21.0b5 en-US & da 		[DONE] 18.0b6 en-US & pl
[DONE] 19.0b6 en-US & es-ES
[DONE] 20.0b6 en-US & zh-TW
[DONE] 21.0b3 en-US & ro
[DONE] 21.0b5 en-US & da
Windows 8 64-bit [DONE] 18.0b1 en-US & ko
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 21.0b5 en-US & zh-TW 		[DONE] 18.0b1 en-US & ko
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 21.0b5 en-US & zh-TW 		[DONE] 18.0b1 en-US & ko
[DONE] 19.0b1 en-US & fr
[DONE] 20.0b1 en-US & it
[DONE] 21.0b4 en-US & es-ES
[DONE] 21.0b5 en-US & zh-TW
Mac OSX 10.6 [DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b5 en-US & es-ES 		[DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b5 en-US & es-ES 		[DONE] 18.0b1 en-US & da
[DONE] 19.0b1 en-US & de
[DONE] 20.0b1 en-US & ja-JP-mac
[DONE] 21.0b1 en-US & it
[DONE] 21.0b5 en-US & es-ES
Mac OSX 10.7 [DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b5 en-US & zh-TW 		[DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b5 en-US & zh-TW 		[DONE] 18.0b2 en-US & ja-JP-mac
[DONE] 19.0b2 en-US & pt-BR
[DONE] 20.0b2 en-US & fr
[DONE] 21.0b2 en-US & ru
[DONE] 21.0b5 en-US & zh-TW
Mac OSX 10.8 [DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ko 		[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ko 		[DONE] 18.0b3 en-US & es-ES
[DONE] 19.0b3 en-US & it
[DONE] 20.0b3 en-US & ro
[DONE] 21.0b4 en-US & de
[DONE] 21.0b5 en-US & ko
Ubuntu 12.04 32-bit [DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b5 en-US & ru 		[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b5 en-US & ru 		[DONE] 18.0b4 en-US & fr
[DONE] 19.0b4 en-US & de
[DONE] 20.0b4 en-US & ko
[DONE] 21.0b1 en-US & zh-TW
[DONE] 21.0b5 en-US & ru
Ubuntu 12.04 64-bit [DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b5 en-US & it 		[DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b5 en-US & it 		[DONE] 18.0b5 en-US & da
[DONE] 19.0b5 en-US & es-ES
[DONE] 20.0b5 en-US & pt-BR
[DONE] 21.0b2 en-US & de
[DONE] 21.0b5 en-US & it

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

Full Query
ID Summary Priority Status
868005 Java applet steals focus from the location bar P3 RESOLVED

1 Total; 0 Open (0%); 1 Resolved (100%); 0 Verified (0%);


Beta 7

Builds FTP, HTTP
ID
Changeset changeset (pushlog)
Checklist
  • [ON TRACK] Automated Tests
  • [ON TRACK] Regression Tests
  • [ON TRACK] Bug Fix Verifications
  • [ON TRACK] Update Tests (betatest)
  • [ON TRACK] Update Tests (releasetest)
  • [ON TRACK] Stability checkpoint
  • [ON TRACK] Update Tests (beta)


Automated Testing

The following are the detailed results of the automated Mozmill test runs. These tests are run against the en-US and one random locale candidate per platform.

Platform Results dashboard
Windows XP
Windows Vista
Windows 7 32-bit
Windows 7 64-bit
Windows 8 32-bit
Windows 8 64-bit
Mac OSX 10.6
Mac OSX 10.7
Mac OSX 10.8
Ubuntu 12.04 32-bit
Ubuntu 12.04 64-bit

Known test failures in bugzilla

Regression Tests

As per the checklist we will regression test the following areas.

Breakpad

Goal: Ensure that crash reports are submitted with the current build (use this Moztrap test as a rough guideline).

Build Windows 7 64-bit Mac OSX 10.8 Ubuntu 12.04 64-bit
Firefox 21.0b7#1 en-US [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Firefox 21.0b7#1 de [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Firefox 21.0b7#1 fr [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Firefox 21.0b7#1 it [DONE] Manuela [DONE] Bogdan [DONE] Virgil
Issues:
  • No issues found

Platform De-support

Goal: Ensure that platforms which are no longer supported do not receive software updates to this Beta.

Test Platform Tester Result
Firefox 7 Beta Mac OSX 10.5 Simona Updated to Firefox 11 beta 5
Firefox 8 Beta Windows 2000 Simona Updated to Firefox 11 beta 5
Firefox 9 Beta Mac OSX 10.5 Simona Updated to Firefox 11 beta 5
Firefox 10 Beta Windows 2000 Simona Firefox is up to date
Issues:
  • bug 869397 Firefox Update for the unsupported OSs is inconsistent

GPU Stability Dogfooding

What to Test:

  • P1 Platforms: Windows 8, Windows 7
  • P2 Platforms: Mac OSX 10.8, Mac OSX 10.7, Ubuntu 12.04
  • P3 Platforms: Windows XP, Windows Vista
  • GPUs: Any AMD Radeon (particularly 6xxx/7xxx series), Any NVidia (particularly GTX 5xx/6xx series)
  • Scenarios:
    • Video playback
    • Web gaming
    • Maps
    • Excessive tabbed browsing
    • PDF viewer
    • Web mail including instant messaging and video calling (Skype, Yahoo, GMail, etc)
    • Plug-in driven web interfaces and menus
    • Restart and Session restore
GPU Platform Tester Results
AMD Radeon 6310 Windows 7 Juan Becerra
NVIDIA GeForce GT 610 Windows 7 Manuela Muntean [DONE]
AMD Radeon HD 7700 Series Windows 8 Virgil Dicu
NVIDIA GeForce GT 610 Windows 8 Manuela Muntean [ON TRACK]
AMD Radeon HD 6450 Ubuntu 12.04 Cornel Ionce
NVIDIA GeForce GT 610 Ubuntu 12.10 Manuela Muntean [ON TRACK]
AMD Radeon HD 6630 Mac OSX 10.7 Virgil Dicu [DONE]
NVIDIA GeForce 9400 Mac OSX 10.7 Bogdan Maris
AMD Radeon HD 6630 Mac OSX 10.8 Virgil Dicu [ON TRACK]
NVIDIA GeForce 9400 Mac OSX 10.8 Bogdan Maris
AMD Radeon HD 7700 Series Windows XP Virgil Dicu
NVIDIA GeForce GT 610 Windows XP Manuela Muntean [ON TRACK]
AMD Radeon HD 6450 Windows Vista Cornel Ionce
NVIDIA GeForce GT 610 Windows Vista Manuela Muntean [ON TRACK]
  • Notes
    • NVIDIA GeForce GT 610 - Windows 7 64bit - Skype Click to Call doesn't work with Firefox or Chrome, but works with IE

Bug Fix Verifications

The following bugs have been fixed and need confirmation.

Bug Assignee Status
bug 867737 Update FHR reporting URL
bug 857199 Please start offering the stub installer for English/Win32 on Beta Paul Silaghi [DONE]
Totals

Update Tests

The following is the results of the Mozmill automated update tests.

Platform Betatest dashboard Releasetest dashboard Beta dashboard
Windows XP
Windows Vista
Windows 7 32-bit
Windows 7 64-bit
Windows 8 32-bit
Windows 8 64-bit
Mac OSX 10.6
Mac OSX 10.7
Mac OSX 10.8
Ubuntu 12.04 32-bit
Ubuntu 12.04 64-bit

New Issues

The following is a list of bugs found while testing this Beta. If you report a bug to bugzilla please list it below.

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 400 [1] => Bad Request ) ) 1


Known Issues

Full Query
ID Summary Priority Status
692446 Firefox prompts to install windows media player plugin even though it is already installed -- RESOLVED
767673 windows media player plugin shows blank content P3 RESOLVED
791195 Session is not restored after Firefox crash -- RESOLVED
842290 "Restart Aurora" pop up doesn't appear when selecting private browsing mode -- VERIFIED
842491 Source page remains active when Private Browsing is closed -- RESOLVED
842577 Remove from History is visible for a paused download -- RESOLVED
843087 Private and non private downloads shown in the same Library window -- RESOLVED
843540 [Windows] Shield icon missing in Mixed content blocker doorhanger -- VERIFIED
843559 [Ubuntu] Profiler keyboard shortcut (CTRL+SHIFT+Y) opens Library -- VERIFIED
857096 Java dropdown list not displayed properly P3 RESOLVED
863183 The remote debugger "Connecting..." string and animation should be larger -- RESOLVED
863238 Not localized mozilla pages opened from the Firefox options -- RESOLVED
863256 Report Web Forgery warning is not always displayed -- RESOLVED
868005 Java applet steals focus from the location bar P3 RESOLVED

14 Total; 0 Open (0%); 11 Resolved (78.57%); 3 Verified (21.43%);


QAWANTED Bugs

The following bugs require help investigating steps to reproduce, regression ranges, workarounds, or other conditions (like hardware or software). Select a bug from the bugzilla query and do some testing to provide the information being requested. Be sure to add your bug to the list below.

Bug Assignee Status
bug 865701 crash in mozilla::dom::DocumentBinding::CreateInterfaceObjects with AMD Radeon 6310/6320 Juan Becerra [ON TRACK]
bug 864478 Unable to upload receipt to our Health and Wellness provider Juan Becerra [DONE]
bug 863714 crash in nsCSSScanner::Next mainly with AMD Radeon HD 6310 Juan Becerra [ON TRACK]
bug 860052 Java plugins get caught in nested event loop, browser ceases repainting Ioana Budnar [DONE]
bug 859099 crash in nsObjectLoadingContent::DoStopPlugin Paul Silaghi [DONE]
bug 858655 crash in js::ScopeIter::settle Anthony Hughes [DONE]
bug 852915 [Win7] crash in mozilla::MediaDecoderReader::DecodeToFirstAudioData Anthony Hughes [DROPPED]
bug 851788 crash in JSScript::clearTraps Anthony Hughes [DROPPED]
bug 851630 Can't open popup notifications with the keyboard (focus gets stuck at the identity icon) Anthony Hughes [DONE]
bug 850571 crash in nsView::`.* deleting destructor' Juan Becerra [DROPPED]
bug 848663 [Win8] Can’t update from FF19.0 to FF19.0.1 w/custom browser Anthony Hughes [DONE]
bug 844553 Plugins disabled by Aurora 21.x Paul Silaghi [DONE]
bug 842473 Crash in mozilla::dom::FragmentOrElement::CanSkip with FoxyProxy 4.2 Anthony Hughes [DROPPED]
bug 840901 Crash in nsDocLoader::DoFireOnStateChange with Whitesky's ID Vault (bundled with HP computers?) Paul Silaghi [DROPPED]
bug 839629 Fix partner add-ons so that search pref changes comply with add-on guidelines Manuela Muntean [DONE]
bug 837370 Crash in TypeConstraintFreeze::newType @ js::types::TypeCompartment::addPendingRecompile Anthony Hughes [DROPPED]
bug 836263 Crash in AppendUTF8toUTF16 or AppendUTF16toUTF8 with Giant Savings add-on Paul Silaghi [DROPPED]
bug 832091 Crash in xpc::XPCWrappedNativeXrayTraits::resolveDOMCollectionProperty w/Java Virgil Dicu [DROPPED]
bug 830531 [Win8] crash in XPC_WN_Helper_NewResolve mainly with AMD Radeon HD Anthony Hughes [DONE]
bug 824971 [summary hidden] Matt Wobensmith [DROPPED]
bug 819028 [Win7][KB2670838] crash in gfxWindowsPlatform::UpdateRenderMode @ CResource::InvalidAlignment Marcia Knous [DROPPED]
bug 816976 [adbe 3519647][10.8] crash in glcGetIOAccelService in Flash plugin-container process Paul Silaghi [DROPPED]
bug 812695 [D2D] Text Rendering Issues due to Windows 7 Platform Update KB2670838 Juan Becerra [DROPPED]
bug 812683 [Win7][KB2670838] startup crash in _d2d_clear_surface @ CContext::RestorePipelineStateImpl Juan Becerra [DROPPED]
bug 805406 Crash in gfxContext::PushClipsToDT with d2d11.dll 6.2 Juan Becerra [DROPPED]
Totals:
  • 36% [DONE]
  • 56% [DROPPED]
  • 8% [ON TRACK]

UNCONFIRMED Bugs

The following bugs need confirmation that they are reproducible. Select a bug from bugzilla and set the status to NEW if you reproduce it. Be sure to add a comment indicating your operating system, Firefox version(s), and steps you used to reproduce the bug. Add the bug to the list below.

Bugs [bugzilla] Assignee Status
bug 852976 Messages minimised in Gmail composer automatically restore when switching tabs Paul Silaghi [DONE] RESOLVED WORKSFORME
bug 847653 SVG animation (using requestAnimationFrame) severely chokes on Firefox for Mac Paul Silaghi [ON TRACK] ASSIGNED

Release Blockers

  • FHR
    • [VERIFIED FIXED] bug 858242 [FHR] Privacy Policy isn't linked from the sharing doorhanger and clicking it targets the graph toggle instead
    • [VERIFIED FIXED] bug 856315 [FHR] Outlier startup time destroys chart usability
Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_21/Test_Plan&oldid=652808"