Releases/Firefox 24 ESR/Test Plan

From MozillaWiki
Jump to navigation Jump to search

Firefox 24 ESR

Builds FTP, HTTP
ID 20130910201120
Changeset c5fc2f003e4d (pushlog)
Checklist
  • [DONE] Functional automation
  • [DONE] Manual tests
  • [DONE] Bug fix verifications

Automated Tests

The following are the detailed results of the automated Mozmill test runs.

Platform Results (report)
Windows XP PASS
Windows Vista PASS
Windows 7 32-bit PASS
Windows 7 64-bit PASS
Windows 8 32-bit PASS
Windows 8 64-bit PASS
Mac OS X 10.6 PASS
Mac OS X 10.7 PASS
Mac OS X 10.8 PASS
Ubuntu 12.04 32-bit PASS
Ubuntu 12.04 64-bit PASS

Manual Smoketests

Run the Firefox ESR 24 Smoketests in MozTrap and report the results below.

Platform Assignee Result: report
Windows 8 32-bit Manuela Muntean [DONE]
Windows 7 64-bit Mihai Morar [DONE]
Mac OS X 10.7 Bogdan Maris [DONE]
Ubuntu 12.04 32-bit Petruta Rasa [DONE]


Additional Spotchecks
  • Pave-over install of FF24esr over FF17esr [DONE]
    • From 17.0.x to 24
    • From 17.0.8 to 24
    • From 17.0.9 to 24
  • Firefox Desktop : Sync : Account [DONE]
    • Moztrap tests
    • Notes/Issues:
      • on Win 8 32bit, Ubuntu 32bit and Mac OS X 10.7.5: regarding "View and dismiss Sync discovery notifications" test -> there is no "browser.syncPromoViewsLeft" option in "about:config"
  • Show PDF inline Smoketests [DONE]
    • Moztrap tests
    • Notes/Issues:
      • on Win 8 32bit, Ubuntu 32bit and Mac OS X 10.7.5: "Display embedded PDF" test fails, because the PDF embedded with the object tag isn't displayed; known issue: see bug 738967
  • SocialAPI Service Management [DONE]
  • Mixed Content Blocker [DONE]
    • Moztrap tests
    • Notes/Issues:
      • on all platforms:
        • the behavior explained in bug 906190 is encountered
        • the prefs used in the test cases aren't the correct ones; these are the correct prefs: "security.mixed_content.block_active_content" and "security.mixed_content.block_display_content"

Bug Fix Verifications

  • [DONE] bug 633001 VERIFIED: SSL cannot set exceptions on IPv6 addresses
  • [DONE] bug 888820 VERIFIED: Crash in nsHtml5TreeBuilder
  • [DONE] bug 887334 NOT VERIFIED: GC hazard with default compartments and frame chain restoration
    • No test case or STR. Also, developer has created his own tests.
  • [DONE] bug 874952 VERIFIED: Always create enough last chunks for an engine, even if we don't end up calling into its ProduceAudioBlock function
  • List of all security bugs verified in FF24

Issues

New
Known


Firefox 24.1.0 ESR

Builds FTP, HTTP
ID 20131021230807
Changeset b6aa22c418f4 (pushlog)
Checklist
  • [DONE] Functional automation
  • [DONE] Manual smoketests
  • [DONE] Bug fix verifications
  • [DONE] Update automation on esrtest
  • [DONE] Conditional sign-off for push to mirrors
  • [DONE] Update automation on releasetest
  • [DONE] Final sign-off for push to esr
  • [DONE] Update automation on esr

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional [report]
Windows XP PASS
Windows Vista PASS
Windows 7 32-bit PASS
Windows 7 64-bit PASS
Windows 8 32-bit PASS
Windows 8 64-bit PASS
Mac OS X 10.6 PASS
Mac OS X 10.7 PASS
Mac OS X 10.8 PASS
Ubuntu 12.04 32-bit PASS
Ubuntu 12.04 64-bit PASS
Ubuntu 13.04 32-bit PASS
Ubuntu 13.04 64-bit PASS

Manual Smoketests

Run the Firefox ESR 24.1 Smoketests in MozTrap and report the results below.

Platform Assignee Result: report
Windows 8 32-bit Paul Silaghi [DONE]
Windows 7 64-bit Alexandra Lucinet [DONE]
Mac OS X 10.7 Cornel Ionce [DONE]
Ubuntu 12.04 32-bit Mihai Morar [DONE]

Update Testing

  • Complete: 24.0
Platform Target Version esrtest releasetest esr
Windows XP Firefox 24.0 PASS PASS PASS
Windows Vista Firefox 24.0 PASS PASS PASS
Windows 7 32-bit Firefox 24.0 PASS PASS PASS
Windows 7 64-bit Firefox 24.0 PASS PASS PASS
Windows 8 32-bit Firefox 24.0 PASS PASS PASS
Windows 8 64-bit Firefox 24.0 PASS PASS PASS
Mac OS X 10.6 Firefox 24.0 PASS PASS PASS
Mac OS X 10.7 Firefox 24.0 PASS PASS PASS
Mac OS X 10.8 Firefox 24.0 PASS PASS PASS
Ubuntu 12.04 32-bit Firefox 24.0 PASS PASS PASS
Ubuntu 12.04 64-bit Firefox 24.0 PASS PASS PASS
Ubuntu 13.04 32-bit Firefox 24.0 PASS PASS PASS
Ubuntu 13.04 64-bit Firefox 24.0 PASS PASS PASS

Bug Fix Verifications

  • [DONE] bug 913614 NO VERIFY: For lines and rectangles, skip the work when the input is invalid
    • Verification of related bug is enough
  • [DONE] bug 914667 VERIFIED: PDF viewer doesn't resolve base URIs properly
  • [DONE] bug 897678 VERIFIED: Make worker code stop using propertyops, as a step toward removing propertyops altogether.
  • [DONE] bug 918864 VERIFIED: Improve doStopDocumentLoad
  • [DONE] bug 914017 VERIFIED: Initialize members variables of the document hash correctly. (mwobensmith)
  • [DONE] bug 916580 NO VERIFY: Fix bugs related to the usage of calloc (mwobensmith)
    • No test case
  • [DONE] bug 916404 NO VERIFY: ContentIsHostIncludingDescendantOf,
    • Can't reproduce, relying on reporter's verification
  • [DONE] bug 869406 NO VERIFY: Really enforce non-parallel build of ICU.
    • Can't verify, no test case
  • [ON TRACK] bug 910881 NOT VERIFIED: Do not call RasterImage::DoError off the main thread. (Ioana)
    • Still crashes, cannot mark bug as verified
    • Developer has not responded to inquiries; we are going to proceed anyway
  • [DONE] bug 916404 NO VERIFY, relying on reporter
  • [DONE] bug 838105 VERIFIED: Don't rely on key lookup to remove items from the user font cache, as the principal could have changed (Cornel)
  • [DONE] bug 788031 VERIFIED: For object/embed/applet tags, pass baseURI to shouldProcess if mURI is null.
  • [DONE] bug 868327 VERIFIED: Only allow the drop-down menu to open above the combobox if there is room for it, otherwise open it below even though it doesn't fit.
  • [DONE] bug 911158 VERIFIED: Removal of URL Protocol from FirefoxURL causes problems for some users. (Ioana)
  • [DONE] bug 916126 VERIFIED: Move IDL addition to the end of the IDL since we're not bumping the IID, (Manuela)
  • [DONE] bug 915210 VERIFIED: Improve GetPresContext
  • [DONE] bug 860123 VERIFIED: Disallow extending or collapsing selections across documents.
  • [DONE] bug 916576 VERIFIED: Follow the com rules when calling LoadURI.
  • [DONE] bug 921090 NO VERIFY: Upgrade Firefox to NSS 3.15.2
    • All-or-nothing change, can't easily verify just this fix
  • [DONE] bug 917160 VERIFIED: AssureBlitted before switching EGLSurfaces with MakeCurrent. (Cornel)
  • [DONE] bug 914618 VERIFIED: Reimplement XML pretty printing with events
  • [DONE] bug 916685 NO VERIFY: No need to store empty event handlers no verify,
    • Can't reproduce, relying on reporter's verification

Issues

New

Full Query
ID Summary Priority Status
931791 When creating a new sync account the validity of the email is not checked. -- RESOLVED
931800 Closing the window with the generated Device code will result in a unsuccessful pairing of the said device. -- RESOLVED
931869 Missing text in Pdf P3 RESOLVED

3 Total; 0 Open (0%); 3 Resolved (100%); 0 Verified (0%);


Known

Full Query
ID Summary Priority Status
801715 Sync: "Warning: Unresponsive script" when changing recovery key / MozTrap test case #2675 -- NEW
918864 [FIX] Heap-use-after-free in nsDocLoader::doStopDocumentLoad() -- RESOLVED

2 Total; 1 Open (50%); 1 Resolved (50%); 0 Verified (0%);


Firefox 24.1.1 ESR

Builds FTP, HTTP
ID 20131112155850
Changeset 7d89d35ff7e3 (pushlog)
Checklist
  • [DONE] Functional automation
  • [DONE] Update automation on esrtest
  • [DONE] Update automation on releasetest
  • [DONE] Final sign-off for push to esr
  • [DONE] Update automation on esr

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional [report]
Windows XP PASS
Windows Vista PASS
Windows 7 32-bit PASS
Windows 7 64-bit PASS
Windows 8 32-bit PASS
Windows 8 64-bit PASS
Mac OS X 10.6 PASS
Mac OS X 10.7 PASS
Mac OS X 10.8 PASS
Ubuntu 12.04 32-bit PASS
Ubuntu 12.04 64-bit PASS
Ubuntu 13.04 32-bit PASS
Ubuntu 13.04 64-bit PASS

Regression Testing

Localized String Testing
  • See bug 932310 for details. Spot-check non-English locales to make sure that various strings are translated.
Windows XP Windows 7 64-bit Windows 8.1 32-bit Mac OSX 10.9 Ubuntu 13.04 64-bit
[DONE] fr [DONE] es-ES [DONE] de [DONE] ru [DONE] pt-BR
[DONE] th [DONE] ja [DONE] zh-CN [DONE] ko [DONE] af
[DONE] fi [DONE] bn-IN [DONE] es-MX [DONE] ar [DONE] he
[DONE] pl [DONE] tr [DONE] hi-IN [DONE] zh-TW [DONE] id
[DONE] it [DONE] sv-SE [DONE] da [DONE] kn [DONE] fa

Notes

  • fr, th, fi, pl, it - text is in English on the Save dialog (ctrl+S), Open dialog (ctrl+O), the Downloads pop-up (options - general - download files to...), Print menu (ctrl+P), about:home - the messages under the search bar.
  • th - text is in English for the privacy menu (privacy options), setting - updates (Automatically install updates string), submit feedback page and the profiler (dev tools).
  • fi - text is in English for the privacy menu (privacy options), submit feedback page, dev tools
  • pl - text is in English on the FF health report from about menu and the profiler (dev tools)
  • es-ES, ja, bn-IN, tr, sv-SE, fr, pt-BR, af, he, id, fa - not completetly localized Bookmarks menu.
  • tr, bn-IN, ja, af, he, id, fa, zh-CN, hi-IN, es-MX - Addons Manager not localized or only partially localized.
  • ja - not localized Tools -> Options - OK button.
  • bn-IN, es-MX - not localized menu option - File -> New Private Window, most of the menu options - Tools -> Web Developer and the entire Web Console.
  • tr, sv-SE, bn-IN - not localized help page - Help -> Firefox Help.
  • pt-BR, af, he, id, fa, es-MX - Library not completely localized.
  • he, fa - not localized File -> New Private Tab menu item, address bar placeholder text.
  • he, es-MX, hi-IN - about:support not fully localized.
  • id - Edit menu option not localized.
  • kn, zh-TW - text is in English on the Save dialog (ctrl+S), Open dialog (ctrl+O), Print menu (ctrl+P).
  • ru - Mozilla Crash Reporter is in english and text is out of panel.
  • ru, ko, ar, zh-TW, kn - text from Details located in Mozilla Crash Reporter is in english.
  • ko - Library -> Import and backup your bookmarks is in english.
  • ar, zh, hi-IN - Web developer partially localized.
  • zh-TW - Web developer -> Web console -> Inspector (Box Model is in english), Firefox -> Services -> (no services apply and Services Preferences are in english).
  • kn - Preferences -> General -> Choose -> Choose download folder (buttons english), Web developer -> Web Console buttons are in english and hovering messages are in english.
  • es-MX - Larry Panel -> Permissions and Tools -> Options -> Privacy are partially localized.
  • es-MX, de - Tools -> Options -> Advanced is partially localized.
  • hi-IN - Mozilla crash reporter, Bookmarks Toolbar, CTP overlay, Add-ons installer, Submit feedback page and Tools -> Set Up Sync menu option are partially localized.
  • about:license; about:buildconfig; about:cache; about:compartments; about:credits; about:memory doesn't seem to be localized at all.
  • about:crashes - zh-TW
  • about:mozilla - zh-TW, kn, de, es-MX
  • about:plugins - kn, de, es-MX
  • about:rights - kn, hi-IN, de
  • about:about - hi-IN, zh-CN
  • about:sync-progress - hi-IN
  • about:robots - de, da
  • about:downloads; about:telemetry - es-MX


SSL Topsite Testing
  • Choose a website which uses SSL and do some exploratory testing on that site
  • Be sure to log in to the website if it has that functionality
  • Report any issues with rendering and general usage, in particular if you get any SSL errors
  • Suggestions for sites to test include email, banking, social networking, shopping, video streaming, and forum websites
  • Try to test at least 25 different websites
  • Add PASS if there were no problems or FAIL if you encountered an issue
  • Be sure to note any issues in the Issues Found section below
Sites Tested Windows XP Windows 7 64-bit Windows 8.1 32-bit Mac OSX 10.9 Ubuntu 13.04 64-bit
1. https://www.facebook.com/ PASS PASS PASS PASS PASS
2. https://www.tumblr.com/ PASS PASS PASS PASS PASS
3. https://login.yahoo.com/ PASS PASS PASS PASS PASS
4. https://www.linkedin.com/ PASS PASS PASS PASS PASS
5. https://login.live.com/ PASS PASS PASS PASS PASS
6. https://twitter.com/ PASS PASS PASS PASS PASS
7. http://www.amazon.com/ PASS PASS PASS PASS PASS
8. https://accounts.google.com/ PASS PASS PASS PASS PASS
9. http://wordpress.com/ PASS PASS PASS PASS PASS
10. http://www.ebay.com/ PASS PASS PASS PASS PASS
11. http://www.aliexpress.com/ PASS PASS PASS PASS PASS
12. https://www.pinterest.com/ PASS PASS PASS PASS PASS
13. https://www.paypal.com/ PASS PASS PASS PASS PASS
14. https://instagram.com/accounts/login/ PASS PASS PASS PASS PASS
15. https://login.alibaba.com/ PASS PASS PASS PASS PASS
16. https://www.adcash.com/en/index.php PASS PASS PASS PASS PASS
17. https://vimeo.com/ PASS PASS PASS PASS PASS
18. http://emag.ro PASS PASS PASS PASS PASS
19. https://www.dropbox.com/ PASS PASS PASS PASS PASS
20. https://eu.battle.net/shop/en-us/ PASS PASS PASS PASS PASS
21. https://wiki.mozilla.org/ PASS PASS PASS PASS PASS
22. https://mozqa.etherpad.mozilla.org/ PASS PASS PASS PASS PASS
23. https://bugzilla.mozilla.org/ PASS PASS PASS PASS PASS
24. https://soundcloud.com/ PASS PASS PASS PASS PASS
25. https://drive.google.com/ PASS PASS PASS PASS PASS

Update Testing

  • Complete: 24.0
  • Partial: 24.1.0
Platform Target Version esrtest releasetest esr
Windows XP Firefox 24.0 PASS PASS PASS
Windows Vista Firefox 24.0 PASS PASS PASS
Windows 7 32-bit Firefox 24.0 PASS PASS PASS
Windows 7 64-bit Firefox 24.0 PASS PASS PASS
Windows 8 32-bit Firefox 24.0 PASS PASS PASS
Windows 8 64-bit Firefox 24.0 PASS PASS PASS
Mac OS X 10.6 Firefox 24.0 PASS PASS PASS
Mac OS X 10.7 Firefox 24.0 PASS PASS PASS
Mac OS X 10.8 Firefox 24.0 PASS PASS PASS
Ubuntu 12.04 32-bit Firefox 24.0 PASS PASS PASS
Ubuntu 12.04 64-bit Firefox 24.0 PASS PASS PASS
Ubuntu 13.04 32-bit Firefox 24.0 PASS PASS PASS
Ubuntu 13.04 64-bit Firefox 24.0 PASS PASS PASS

Issues

New

Full Query
ID Summary Priority Status
938029 Test failure "Java(TM) Platform SE 7 U45 state is Enabled in about:plugins - got 'false'" in testAddons/testPluginDisableAffectsAboutPlugins.js -- RESOLVED
938698 Java 7u45 is CTP blocked for some versions -- RESOLVED

2 Total; 0 Open (0%); 2 Resolved (100%); 0 Verified (0%);


Known

Full Query
ID Summary Priority Status
801715 Sync: "Warning: Unresponsive script" when changing recovery key / MozTrap test case #2675 -- NEW
918864 [FIX] Heap-use-after-free in nsDocLoader::doStopDocumentLoad() -- RESOLVED
931791 When creating a new sync account the validity of the email is not checked. -- RESOLVED
931800 Closing the window with the generated Device code will result in a unsuccessful pairing of the said device. -- RESOLVED
931869 Missing text in Pdf P3 RESOLVED

5 Total; 1 Open (20%); 4 Resolved (80%); 0 Verified (0%);

Firefox 24.2.0 ESR

Builds FTP, HTTP
ID xxxxxxxx
Changeset 46049da0a585 (pushlog)
Checklist
  • {{|}} Functional automation
  • {{|}} Manual smoketests
  • {{|}} Bug fix verifications
  • {{|}} Update automation on esrtest
  • {{|}} Conditional sign-off for push to mirrors
  • {{|}} Update automation on releasetest
  • {{|}} Final sign-off for push to esr
  • {{|}} Update automation on esr

Automated Testing

The following are the detailed results of the automated Mozmill test runs.

Platform Functional [report]
Windows XP
Windows Vista
Windows 7 32-bit
Windows 7 64-bit
Windows 8 32-bit
Windows 8 64-bit
Mac OS X 10.6
Mac OS X 10.7
Mac OS X 10.8
Ubuntu 12.04 32-bit
Ubuntu 12.04 64-bit
Ubuntu 13.04 32-bit
Ubuntu 13.04 64-bit

Manual Smoketests

Run the Firefox ESR 24.2 Smoke Tests in MozTrap and report the results below.

Platform Assignee Result: report
Windows 8 32-bit Cornel Ionce }}
Windows 7 64-bit Catalin Varga }}
Mac OS X 10.7 Andrei Vaida }}
Ubuntu 12.04 32-bit Petruta Rasa }}

Update Testing

  • Complete: 17.0.x, 24.1
Platform Target Version esrtest releasetest esr
Windows XP Firefox 24.0
Windows Vista Firefox 24.0
Windows 7 32-bit Firefox 24.0
Windows 7 64-bit Firefox 24.0
Windows 8 32-bit Firefox 24.0
Windows 8 64-bit Firefox 24.0
Mac OS X 10.6 Firefox 24.0
Mac OS X 10.7 Firefox 24.0
Mac OS X 10.8 Firefox 24.0
Ubuntu 12.04 32-bit Firefox 24.0
Ubuntu 12.04 64-bit Firefox 24.0
Ubuntu 13.04 32-bit Firefox 24.0
Ubuntu 13.04 64-bit Firefox 24.0

Bug Fix Verifications

  • [DONE] bug 905903 Properly trace ICTypeMonitor chains for pre-barrier when unlinking them.
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 929261 Only generate TypedArray IC stubs on in-bounds accesses.
  • [DONE] bug 886850 Clean up write barriers for ion/baseline scripts.
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 937582 Fix nsPrefetchNode
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 891904 Un-schedule full spell check when checker is disabled.
  • [DONE] bug 909078 Pretend the inner table frame is top-of-page if the outer table frame is.
  • Template:Needed bug 921045 include the new update cert and remove the no longer used Equifax cert. r=bbondy,
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 905382 register callbacks in workers.
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 909490 Add per-zone data for use by embedder.
  • [DONE] bug 891693 Make sure that huffval gets zeroed before use.
  • Template:Needed bug 910845 Add a wrapper for mmap() in order to keep the high 17-bits of JS pointers cleared on IA64.
  • [DONE] bug 938341 Clean up nsEventListenerManager
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 919021 Convert ctypes over to use JS::AutoValueVector instead of its own array class.
  • style="background:#ff9090; color:black; vertical-align: middle; text-align: center; " class="table-no" | bug 922009 MOZ_CRASH when trying to transplant objects with SCSWs.
  • [DONE] bug 930281 Use nsINode instead of nsIContent.
  • Template:Needed bug 927073 - Binary compatibility broken for maintenance releases due to strict version-script (edit)

Issues

New

Known

Full Query
ID Summary Priority Status
801715 Sync: "Warning: Unresponsive script" when changing recovery key / MozTrap test case #2675 -- NEW
918864 [FIX] Heap-use-after-free in nsDocLoader::doStopDocumentLoad() -- RESOLVED

2 Total; 1 Open (50%); 1 Resolved (50%); 0 Verified (0%);

Retrieved from "https://wiki.allizom.org/index.php?title=Releases/Firefox_24_ESR/Test_Plan&oldid=781344"