Releases/Firefox 23/Test Plan

From MozillaWiki
Jump to navigation Jump to search

« Firefox 22 | Firefox 23 | Firefox 24 »

Firefox 23 Test Plan

Summary

Milestone Firefox 23
Status Currently in Aurora -- merging to Beta on 2013-06-24
Lead Anthony Hughes (irc: ashughes)
2nd Juan Becerra (irc: juanb)
Builds [ftp] [http] [web]

The following is the test plan for Firefox 23 from Nightly through to Release. Use this document as a reference for what is being tested to validate the quality of Firefox 22. After the release this document will be used as an archive of what was done to validate this release.

If you have some free time, please pick a task below and get in touch with one of the leads.

Schedule

  • [DONE] 2013.04.01: Nightly Merge
  • [DONE] 2013.05.13: Aurora Merge
  • [DONE] 2013.06.21: Pre-Beta feature sign-offs
  • [ON TRACK] 2013.06.27: Beta 1
  • 2013.07.05: Beta 2
  • 2013.07.12: Beta 3
  • 2013.07.19: Beta 4
  • 2013.07.26: Beta 5
  • 2013.07.31: Beta 6
  • 2013.08.01: Release Candidate
  • 2013.08.06: Final

Nightly

Merge

Functional Automation - result
Update Spotchecks - result

Features

See the Feature Release Tracking wiki for a general overview

Aurora

Migration Sign-off

Builds nightlies
Changeset tip (pushlog)
Checklist
  • [DONE] Automated functional tests
  • [DONE] Update tests on auroratest
  • [DONE] Update tests on aurora

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Results [dashboard]
Windows XP [DONE] en-US de fr it: 0 fails
Windows Vista [DONE] en-US de fr it: 0 fails
Windows 7 32-bit [DONE] en-US de fr it: 0 fails
Windows 7 64-bit [DONE] en-US de fr it: 0 fails
Windows 8 32-bit [DONE] en-US de fr it: 0 fails
Windows 8 64-bit [DONE] en-US de fr it: 0 fails
Mac OS X 10.6 [DONE] en-US de fr it: 0 fails
Mac OS X 10.7 [DONE] en-US de fr it: 0 fails
Mac OS X 10.8 [DONE] en-US de fr it: 0 fails
Ubuntu 12.04 32-bit [DONE] en-US de fr it: 0 fails
Ubuntu 12.04 64-bit [DONE] en-US de fr it: 0 fails
Test Failures (known bugs)

Updates

Pre-migration Update Throttling
Platform Tested Version Target Version Result
Windows XP Firefox 21.0a2 en-US no update PASS
Firefox 22.0a2 ar no update PASS
Mac OS X 10.8 Firefox 20.0a2 en-US no update PASS
Firefox 22.0a2 ja-JP-mac no update PASS
Ubuntu 12.04 64-bit Firefox 19.0a2 en-US no update PASS
Firefox 22.0a2 de no update PASS
Auroratest
Platform Tested Version Target Version Result
Windows XP Firefox 22.0a2 2013-04-26 en-US Firefox 23.0a2 2013-05-15 en-US PASS
Firefox 22.0a2 2013-05-08 de Firefox 23.0a2 2013-05-15 de PASS
Mac OS X 10.8 Firefox 14.0a2 2012-06-02 en-US Firefox 23.0a2 2013-05-15 en-US PASS
Firefox 22.0a2 2013-05-10 fr Firefox 23.0a2 2013-05-15 fr PASS
Ubuntu 12.04 64-bit Firefox 21.0a2 2013-03-07 en-US Firefox 23.0a2 2013-05-15 en-US PASS
Firefox 22.0a2 2013-05-09 it Firefox 23.0a2 2013-05-15 it PASS
Issues
Aurora
Platform Tested Version Target Version Result
Windows XP Firefox 22.0a2 2013-04-26 en-US Firefox 23.0a2 2013-05-17 en-US PASS
Firefox 22.0a2 2013-05-08 de Firefox 23.0a2 2013-05-17 de PASS
Mac OS X 10.8 Firefox 14.0a2 2012-06-02 en-US Firefox 23.0a2 2013-05-17 en-US PASS
Firefox 22.0a2 2013-05-10 fr Firefox 23.0a2 2013-05-17 fr PASS
Ubuntu 12.04 64-bit Firefox 21.0a2 2013-03-07 en-US Firefox 23.0a2 2013-05-17 en-US PASS
Firefox 22.0a2 2013-05-09 it Firefox 23.0a2 2013-05-17 it PASS
Issues
Post-migration Update Unthrottling
Platform Tested Version Target Version Result
Windows XP Firefox 22.0a2 2013-04-26 en-US Firefox 23.0a2 2013-05-17 en-US PASS
Firefox 22.0a2 2013-05-08 de Firefox 23.0a2 2013-05-17 de PASS
Mac OS X 10.8 Firefox 14.0a2 2012-06-02 en-US Firefox 23.0a2 2013-05-17 en-US PASS
Firefox 22.0a2 2013-05-10 fr Firefox 23.0a2 2013-05-17 fr PASS
Ubuntu 12.04 64-bit Firefox 21.0a2 2013-03-07 en-US Firefox 23.0a2 2013-05-17 en-US PASS
Firefox 22.0a2 2013-05-09 it Firefox 23.0a2 2013-05-17 it PASS

Beta

Beta 1

Build Info

Checklist

  • [DONE] Feature Sign-off
  • [ON TRACK] Automated Tests
  • [ON TRACK] Regression Tests
  • [ON TRACK] Bug Fix Verifications
  • [ON TRACK] Update Tests (betatest)
  • [ON TRACK] Stability & Support
  • [ON TRACK] Update Tests (releasetest)
  • [ON TRACK] Update Tests (beta)

Automated Tests

  • Locales: en-US, da, de, es-ES, es-MX, fa, fi, fr, it, pt-BR, ro, zh-TW
  • Results (dashboard)
    • [ON TRACK] Windows XP
    • [ON TRACK] Windows Vista
    • [ON TRACK] Windows 7 32-bit
    • [ON TRACK] Windows 7 64-bit
    • [ON TRACK] Windows 8 32-bit
    • [ON TRACK] Windows 8 64-bit
    • [ON TRACK] Mac OSX 10.6
    • [ON TRACK] Mac OSX 10.7
    • [ON TRACK] Mac OSX 10.8
    • [ON TRACK] Ubuntu 12.04 32-bit
    • [ON TRACK] Ubuntu 12.04 64-bit
  • Issues Found (known failures)

Feature Sign Off

Regression Tests

  • Moztrap Testrun #1519 (results)
    • [ON TRACK] Windows 7 32-bit
    • [ON TRACK] Windows 8 64-bit
    • [ON TRACK] Mac OSX 10.8
    • [ON TRACK] Ubuntu 12.04 32-bit
  • Issues Found:

Bug Fix Verifications

  • bug 878449 Crash in mozilla::widget::JumpListBuilder::AddListToBuild (Mihai Morar)
  • [ON TRACK] bug 876856 Crash in mozilla::dom::MediaStreamList::Length after closing a peer connection (Manuela Muntean)
  • bug 876721 crash in mozilla::gl::SharedSurface_ANGLEShareHandle::Create (Mihai Morar)
  • bug 874090 Crash in mozilla::dom::Notification::GetPermissionInterna
  • bug 872758 [FHR] Multi delete documents
  • bug 871942 crash in nsWebShellWindow::RequestWindowClose
  • bug 868337 crash in js::ion::DoCallFallback (Mihai Morar)
  • bug 867226 Incomplete framebuffer abort in mozilla::layers::LayerManagerOGL::CreateFBOWithTexture with "error 0x8cdd" (Mihai Morar)
  • [ON TRACK] bug 866339 crash in mozilla::dom::DocumentBinding::createDocumentFragment (Manuela Muntean)
  • bug 865584 crash in js::ion::IonBuilder::createThisScriptedSingleton
  • bug 865260 crash in js::IsStandardClassResolved
  • bug 859718 crash in mozilla::image::RasterImage::DecodePool::DecodeJob::Run
  • [ON TRACK] bug 858800 crash in mozilla::plugins::MiniShmBase::GetWritePtr (Manuela Muntean)
  • [ON TRACK] bug 854319 crash in mozilla::MediaResource::`.* deleting destructor (Manuela Muntean)
  • need-info bug 868257 watchPosition only sends one location
  • [ON TRACK] bug 881991 [FHR] Duplicate aborted sessions being inserted (Ioana Budnar)
  • [ON TRACK] bug 880911 [SocialAPI] tear-off chat windows (Samvedana Gohil)
  • [ON TRACK] bug 877097 crash in mozilla::layout::ScrollbarActivity::UpdateOpacity (Marc Schifer)
  • [ON TRACK] bug 873963 Arbitrary code execution from Font Inspector (Manuela Muntean)
  • [ON TRACK] bug 864505 [SocialAPI] active chat titlebar fails to change (Samvedana Gohil)
  • [ON TRACK] bug 860149 crash in mozilla::image::Decoder::Write (tomhanksjr)
  • [ON TRACK] bug 851934 crash in js::mjit::stubs::ValueToBoolean on Outlook.com (Fredy)
  • [ON TRACK] bug 821073 [SocialAPI] full screen support for video chats (Samvedana Gohil)
  • [ON TRACK] bug 704063 Add unprefixed requestAnimationFrame (Mihai Morar)
  • [DONE] bug 880042 Multi-file form field nsIFilePicker crash if overriding component has not implemented a domfiles getter (Manuela Muntean)
  • [DONE] bug 867445 Remove test pilot from Firefox (Alexandra Lucinet)
  • [DONE] bug 863646 Start up crash in nsContentUtils::GetCurrentJSContext in profile manager (Alice White)
  • [DONE] bug 859714 File open dialog buttons not working (Simona Marcu)
  • [DONE] bug 855370 Remove the ability to not "Always show the tab bar" (Bogdan Maris)
  • [DONE] bug 851702 Remove "Enable JavaScript" checkbox from Prefs (Mihaela Velimiroviciu)
  • [DONE] bug 851701 Remove "Load images automatically" checkbox from Prefs (Mihaela Velimiroviciu)
  • [DONE] bug 847267 Enable DXVA2 on Windows Vista+ to accelerate H.264 video decoding (Bogdan Maris)
  • [DONE] bug 839206 Replace plugin installation notification bar with door hanger (Paul Silaghi)
  • [DONE] bug 331105 Allow users to set a default search engine (Ioana Budnar)
  • [DONE] bug 52500 Make regular CSS properties apply on <input type='file'> (Ioana Budnar)
  • [DROPPED] bug 678392 New swipe animation on OSX 10.7+ (Marc Schifer)
  • [DROPPED] bug 673875 New page bounce animation on OSX 10.7+ (Marc Schifer)
  • [DROPPED] bug 856917 Improve about:memory's functional UI
  • More fixed bugs

Update Tests

The following is the results of the automated Mozmill update tests.

Betatest Channel
  • Versions:
  • Locales: en-US, da, de, es-ES, es-MX, fa, fi, fr, it, pt-BR, ro, zh-TW
  • Results (dashboard)
    • [ON TRACK] Windows XP
    • [ON TRACK] Windows Vista
    • [ON TRACK] Windows 7 32-bit
    • [ON TRACK] Windows 7 64-bit
    • [ON TRACK] Windows 8 32-bit
    • [ON TRACK] Windows 8 64-bit
    • [ON TRACK] Mac OSX 10.6
    • [ON TRACK] Mac OSX 10.7
    • [ON TRACK] Mac OSX 10.8
    • [ON TRACK] Ubuntu 12.04 32-bit
    • [ON TRACK] Ubuntu 12.04 64-bit
  • Issues Found (known failures)
Releasetest Channel
  • Versions:
  • Locales: en-US, da, de, es-ES, es-MX, fa, fi, fr, it, pt-BR, ro, zh-TW
  • Results (dashboard)
    • [ON TRACK] Windows XP
    • [ON TRACK] Windows Vista
    • [ON TRACK] Windows 7 32-bit
    • [ON TRACK] Windows 7 64-bit
    • [ON TRACK] Windows 8 32-bit
    • [ON TRACK] Windows 8 64-bit
    • [ON TRACK] Mac OSX 10.6
    • [ON TRACK] Mac OSX 10.7
    • [ON TRACK] Mac OSX 10.8
    • [ON TRACK] Ubuntu 12.04 32-bit
    • [ON TRACK] Ubuntu 12.04 64-bit
  • Issues Found (known failures)
Beta Channel
  • Versions:
  • Locales: en-US, da, de, es-ES, es-MX, fa, fi, fr, it, pt-BR, ro, zh-TW
  • Results (dashboard)
    • [ON TRACK] Windows XP
    • [ON TRACK] Windows Vista
    • [ON TRACK] Windows 7 32-bit
    • [ON TRACK] Windows 7 64-bit
    • [ON TRACK] Windows 8 32-bit
    • [ON TRACK] Windows 8 64-bit
    • [ON TRACK] Mac OSX 10.6
    • [ON TRACK] Mac OSX 10.7
    • [ON TRACK] Mac OSX 10.8
    • [ON TRACK] Ubuntu 12.04 32-bit
    • [ON TRACK] Ubuntu 12.04 64-bit
  • Issues Found (known failures)

New Issues

QAWANTED Bugs

Full Query
ID Summary Priority Status
878449 Fix mismatch between CoInitialize and CoUninitialize causing crash in mozilla::widget::JumpListBuilder::AddListToBuild @ CCliModalLoop::CCliModalLoop -- VERIFIED
883222 Firefox 21 and Ember.js calls on an event throws 'Permission denied to access property' error -- RESOLVED

2 Total; 0 Open (0%); 1 Resolved (50%); 1 Verified (50%);


UNCONFIRMED Bugs

Full Query
ID Summary Priority Status
801379 "Clear recent history" does not clear recently closed tabs & windows -- RESOLVED
857938 Page flickers while scrolling using MATE desktop manager -- RESOLVED
859117 Fullscreen on Nascar raceview in nightly and aurora -- RESOLVED
861781 Very slow startup with "automatic proxy configuration URL" (*.pac) -- RESOLVED
863679 Jank on page. Profile highlights js_Date -- RESOLVED
864001 Flash player -interrupt -- RESOLVED
864635 Star in About Nightly dialog overlaps some text -- RESOLVED
865350 on about:newtab page, the bookmark star button does nothing. -- RESOLVED
866475 can't make a new tab, back button can't use -- RESOLVED

9 Total; 0 Open (0%); 9 Resolved (100%); 0 Verified (0%);


Known Issues

Full Query
ID Summary Priority Status
842566 EXC_BAD_ACCESS (SIGSEGV) after building with DMD -- NEW
885351 Google is always used in about:home and the URL bar when setting the default search engine to an invalid one -- VERIFIED

2 Total; 1 Open (50%); 0 Resolved (0%); 1 Verified (50%);

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_23/Test_Plan&oldid=670403"