Releases/Firefox 28/Test Plan

From MozillaWiki
Jump to navigation Jump to search

« Firefox 27 | Firefox 28 | Firefox 29 »

Firefox 28 Test Plan

QA Lead: Anthony Hughes

Schedule

  • 2013.10.28: Nightly
  • 2013.12.09: Aurora
  • 2014.01.17: Pre-Beta feature sign-offs
  • 2014.01.21: Beta 1
  • 2014.01.28: Beta 2
  • 2014.01.31: Beta 3
  • 2014.02.04: Beta 4
  • 2014.02.07: Beta 5
  • 2014.02.11: Beta 6
  • 2014.02.14: Beta 7
  • 2014.02.18: Beta 8
  • 2014.02.21: Beta 9
  • 2014.02.25: Beta 10
  • 2014.03.02: Release Candidate
  • 2014.03.03: Final

Nightly

Features

Feature Tracking

Tentative

Sign-off Criteria
  • All user stories covered by manual or automated tests
  • Manual testing complete with no blocking regressions
  • Automated testing complete with no blocking regressions
  • 0 untriaged unconfirmed bugs
  • 0 untriaged unverified fixes
Status
Feature Owner Documentation Tracked Bugs Aurora Merge Sign-off Pre-beta Sign-off
Australis Paul Silaghi, Cornel Ionce Wiki page Disabled NA
Shumway Paul Silaghi, Bogdan Maris Wiki page
Thumbnail Service Alexandra Lucinet Wiki page fixed bugs
Input type=color Manuela Muntean, Paul Silaghi Wiki page 1 bug
Session Restore Cornel Ionce fixed bugs
Disallow enumeration of navigator.plugins Catalin Varga 1 bug
MP4 Support No need for QA owner 1 bug NA
Enable WEBGL_draw_buffers No need for QA owner 1 bug NA NA
Port DOM Promises to Workers No need for QA owner - covered automatically 1 bug NA NA
Notes
  • Sign-off work should be done on Holly builds (nightly without Australis).

How to Help

Fixed Bug Verification

  • Download and install the latest Firefox Nightly
  • Load this list of bug reports
  • Read each bug report until you find one you understand and want to test
  • Download and install the version of Firefox used to report the bug
  • Using the older build, test the claim made in the bug report to see if you can reproduce it
    • If you cannot reproduce the bug move on to another bug or ask the reporter to verify the fix
  • Using the latest Nightly, test to see if you can reproduce the bug
    • If you cannot reproduce the bug set the STATUS to VERIFIED FIXED and add a comment describing your testing
    • If you can still reproduce the bug set the STATUS to REOPENED and add a comment describing your testing
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field

Reported Bugs

  • Download and install the latest Firefox Nightly
  • Load this list of bug reports
  • Read each bug report until you find one you understand and want to test
  • Test the claim made in the bug report and see if you can reproduce it
  • If you can reproduce the bug, set the STATUS to NEW, add a comment describing your testing, and try to find a regression window
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field

Stability Bugs

  • Review the Firefox 28 Explosiveness Report
    • Any signatures rising significantly (>= 2.0 explosiveness rating) without a bug number should be reported to Bugzilla and nominated for tracking
    • If there is already a bug report, update the report with the latest numbers
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Ensure the bug is listed below using the bug number template
  • Review the Firefox 28 Topcrash Report
    • Any signature in the top 10 without a bug number should be reported to Bugzilla and nominated for tracking
    • If there is already a bug report, update the report with the latest ranking information
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Ensure the bug is listed below using the bug number template
  • Review the Firefox 28 tracked bugs that aren't tracked in 27 or 26.
    • Each bug listed needs to have its rank updated once per week
    • Use the topcrash reports from crash-stats to get the rank for each branch
    • Add this information to the bug report
    • Ensure the bug report has the topcrash-platform keyword added (where platform is the platform where crashes occur -- ie. topcrash-win for Windows)
    • Conversely, remove the topcrash keyword if the crash is no longer in the top-10
  • For each of these tasks, ensure the etherpad is updated with your name and the date checked

QA Wanted

  • Download and install the latest Firefox Nightly
  • Review this list of bug reports to find a bug you both understand and want to test (each should contain a comment detailing what's needed from QA)
    • If the bug needs steps to reproduce, try to find a reproducible set of steps
    • If the bug needs a regression window, try to find a regression window
    • If the bug is fixed and needs verification, test to confirm you can reproduce the bug on an older build but not the latest build
  • Any tested bugs should be added to the list below using the bug number template
  • Remove the qawanted keyword from any fulfilled request
  • If you need help, send email to Anthony Hughes with the bug summary in the Subject field

Testdays

Aurora

Branch Migration

Tasks

  • [DONE] Updates verified throttled on aurora channel (QA)
  • [DONE] Set mozmill-aurora update tests to auroratest channel in MM-CI
  • Automation daily testruns reviewed for failures
  • Features qualified for uplift to Aurora by feature owners
  • All flagged bug fixes verified
  • All unconfirmed bugs triaged
  • Topcrash reports reviewed for stability concerns
  • Sign-off and verify updates on the aurora channel

Automated Testing

Review the following reports and file bugs if any failures are found:

Feature Sign-offs

Confirm that all enabled features pass smoketests for primary user stories and that disabled features are disabled. Update the status of the feature below as follows:

  • SIGNED-OFF if the feature is enabled and passes all tests
  • DISABLED if the feature is disabled and verified disabled
  • FAILED if serious issues were encountered related to the feature

Additionally, sign-off any features not listed below tracked in this list

Feature Owner Status
Australis Cornel Ionce DISABLED
Shumway Bogdan Maris DISABLED
Thumbnail Service Alexandra Lucinet
<input type=color> Paul Silaghi
Disallow enumeration of navigator.plugins Catalin Varga
Session Restore Improvements Cornel Ionce

Bug Fix Verifications

  • Triage this list of bugs
  • Attempt to reproduce the bug as described in a known broken build and the latest Aurora build
  • Update the bug with the results of your testing
  • List the bug in the table below with one of the following statuses:
    • VERIFIED if you verified the fix
    • NEED-INFO if you need follow-up on the bug
    • DROPPED if fix is unverifiable
    • REOPENED if you reopened the bug
Bug Status

Unconfirmed Bug Triage

  • Triage this list of bugs
  • Attempt to reproduce the bug as described
  • Update the bug with the results of your testing
  • List the bug in the table below with one of the following statuses:
    • CONFIRMED if you set the bug to NEW
    • DROPPED if you set the bug to RESOLVED
    • UNCONFIRMED if you left the bug UNCONFIRMED
Bug Status

Post Sign-off Tasks

  • Updates verified unthrottled on aurora channel (QA)

Beta

Bugs

Potential Blockers

UNTRIAGED: 0 bugs

TRACKED: 1 bugs

  • bug 942427 Australis: No highlights indicated on Windows 7 High Contrast visual style

FIXED: 2 bugs

  • bug 947916 "Couldn't load XPCOM" & missing pgort100.dll after updating Nightly (2013-12-09) [needs verification]
  • bug 944774 Holly is not applying nightly updates on windows [needs verification]

DROPPED: 3 bugs

  • bug 947930 Latest Nightly Firefox Build does not starts on windows platforms [dupe of bug 947916]
  • bug 944565 Browser would not restart after check update from About Holly [dupe of bug 944774]
  • bug 940364 Australis: Nightly not displaying web pages after update [WORKSFORME]

Issues Found in Testing

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Top Crash Bugs

UNTRIAGED: 0 bugs

TRACKED: 12 bugs

  • bug 947240 crash in mozilla::layers::CompositorD3D11::DrawQuad(mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::gfx::RectTyped<mozilla::gfx::UnknownUnits> const&, mozilla::layers::EffectChain const&, float, mozilla::gfx::Matrix4x4 const&)
  • bug 947213 crash in mozilla::layers::DeprecatedTextureHostBasic::UpdateImpl(mozilla::layers::SurfaceDescriptor const&, nsIntRegion*, nsIntPoint*)
  • bug 941869 `anonymous namespace::ReportErrorRunnable::ReportError(JSContext*, mozilla::dom::workers::WorkerPrivate*, bool, mozilla::dom::workers::WorkerPrivate*, nsString const&, nsString const&, nsString const&, unsigned int, unsigned int, unsigned int, unsign...
  • bug 937220 mozalloc_abort(char const* const) | NS_DebugBreak | js::AtomizeChars(js::ExclusiveContext*, wchar_t const*, unsigned __int64, js::InternBehavior)
  • bug 934533 Nightly Nov-04, crash in gfxContext::gfxContext(mozilla::gfx::DrawTarget*) if gfx.content.azure.enabled = false
  • bug 931794 startup crash in mozilla::dom::WindowBinding::get_content
  • bug 931911 crash in mozilla::a11y::SelectionManager::ProcessSelectionChanged(nsISelection*)
  • bug 883059 crash in mozilla::a11y::EventQueue::PushEvent
  • bug 880140 crash in nsDeviceContextSpecX::EndPage @ decode_data when printing [verifyme]
  • bug 869027 crash in JS_HasPropertyById - fixed, needs verification
  • bug 812683 [Win7] startup crash in _d2d_clear_surface @ CContext::RestorePipelineStateImpl w/ASUS GamerOSD
  • bug 805406 crash in gfxContext::PushClipsToDT with Direct2D 1.1 (d3d11.dll 6.2 or 6.3)
  • bug 767343 crash in nsSupportsStringImpl::SetData with abort message: "###!!! ABORT: OOM: file e:\builds\moz2_slave\m-cen-w32-ntly\build\xpcom\string\src\nsTSubstring.cpp, line 348" [VERIFYME]

FIXED: 2 bugs

  • bug 945308 crash in mozilla::a11y::HyperTextAccessible::GetBoundsInFrame(nsIFrame*, unsigned int, unsigned int)
  • bug 942819 crash in mozilla::layers::DeviceManagerD3D9::Init()


AT RISK: 0 bugs

DROPPED: 0 bugs

QA Wanted

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Verifications

How to Verify a Fix
  • Pick a bug from this list
  • Download a build that is known to reproduce the bug and test to make sure you can see the bug
  • Download a build that is said to be fixed and test to make sure you cannot see the bug
  • If the bug reproduces in the fixed build:
    • set the STATUS field to REOPENED and provide a comment detailing your testing
  • If the bug does not reproduce in the fixed build:
    • set the status flag to verified for the version you tested
    • set the STATUS field to VERIFIED FIXED if the version tested is the same as the TARGET MILESTONE
    • any bugs marked verified will automatically appear in the Verified Fixes section below

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Unconfirmed

Pick a bug from this list and try to reproduce the issue using the information provided. If you reproduce the issue, provide a description of the circumstances, set the status to NEW, and move the bug into a component which matches the issue (ie. a bookmarks issue would go in the Bookmarks component). If you can't reproduce the issue ask the reporter for more information by setting the need-info? flag to their email address.

UNTRIAGED: 0 bugs

TRACKED: 0 bugs

FIXED: 0 bugs

AT RISK: 0 bugs

DROPPED: 0 bugs

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_28/Test_Plan&oldid=787731"