MozillaWiki

Releases/Firefox 17 ESR/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 17 ESR/Test Plan (view source)

Revision as of 22:53, 15 November 2013

6,573 bytes added ,  15 November 2013
→‎Firefox 17.0.11 ESR
 
(32 intermediate revisions by 6 users not shown)
Line 2,031: Line 2,031:


; Checklist
; Checklist
* {{|}} Functional automation
* {{done|}} Functional automation
* {{|}} Manual smoketests
* {{done|}} Manual smoketests
* {{|}} Bug fix verifications
* {{done|}} Bug fix verifications
* {{|}} Update automation on esrtest
* {{done|}} Update automation on esrtest
* {{|}} Conditional sign-off for push to mirrors
* {{done|}} Conditional sign-off for push to mirrors
* {{|}} Update automation on releasetest
* {{done|}} Update automation on releasetest
* {{|}} Final sign-off for push to esr
* {{done|}} Final sign-off for push to esr
* {{|}} Update automation on esr
* {{done|}} Update automation on esr


=== Automated Testing ===
=== Automated Testing ===
Line 2,045: Line 2,045:
  |-
  |-
  | class="header" style="width:20%" | Platform
  | class="header" style="width:20%" | Platform
  | class="header" style="width:20%" | Functional [[http://mozmill-ondemand.blargon7.com/#/functional/reports?branch=17.0&platform=All&from=2013-09-10&to=2013-09-11 report]]
  | class="header" style="width:20%" | Functional [[http://mozmill-ondemand.blargon7.com/#/functional/reports?branch=17.0&platform=All&from=2013-10-21&to=2013-10-24 report]]
  |-
  |-
  | Windows XP
  | Windows XP
  |  
  | PASS
  |-
  |-
  | Windows Vista
  | Windows Vista
  |  
  | PASS
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  |  
  | PASS
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  |  
  | PASS
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit
  |  
  | PASS (known failure {{bug|927361}})
  |-
  |-
  | Windows 8 64-bit
  | Windows 8 64-bit
  |  
  | PASS
  |-
  |-
  | Mac OS X 10.6
  | Mac OS X 10.6
  |  
  | PASS
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  |  
  | PASS
  |-
  |-
  | Mac OS X 10.8
  | Mac OS X 10.8
  |  
  | PASS
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  |  
  | PASS
  |-
  |-
  | Ubuntu 12.04 64-bit
  | Ubuntu 12.04 64-bit
  |  
  | PASS (known failure {{bug|927361}})
  |-
  |-
  | Ubuntu 13.04 32-bit
  | Ubuntu 13.04 32-bit
  |
  | PASS
  |-
  |-
  | Ubuntu 13.04 64-bit
  | Ubuntu 13.04 64-bit
  |  
  | PASS (known failure {{bug|927361}})
  |}
  |}


Line 2,096: Line 2,096:
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit
  |  
  | Petruta Rasa
  | {{|}}
  | {{done|}}
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  |  
  | Simona Marcu
  | {{|}}
  | {{done|}}
  |-
  |-
  | Mac OS X 10.7
  | Mac OS X 10.7
  |
  | Catalin Varga
  | {{|}}
  | {{done|}}
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  |  
  | Ioana Budnar
  | {{|}}
  | {{done|}}
  |}
  |}


Line 2,126: Line 2,126:
  | Windows XP
  | Windows XP
  | Firefox 17.0, 17.0.3, 17.0.9esr
  | Firefox 17.0, 17.0.3, 17.0.9esr
  |  
  | PASS
  |  
  | PASS
  |  
  | PASS
  |-
  |-
  | Windows Vista
  | Windows Vista
  | Firefox 17.0, 17.0.4, 17.0.9esr
  | Firefox 17.0, 17.0.4, 17.0.9esr
  |  
  | PASS
  |  
  | PASS
  |  
  | PASS
  |-
  |-
  | Windows 7 32-bit
  | Windows 7 32-bit
  | Firefox 17.0, 17.0.4, 17.0.9esr
  | Firefox 17.0, 17.0.4, 17.0.9esr
  |  
  | PASS
  |  
  | PASS
  |  
  | PASS
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | Firefox 17.0.2, 17.0.4, 17.0.9esr
  | Firefox 17.0.2, 17.0.4, 17.0.9esr
  |  
  | PASS
  |  
  | PASS
  |  
  | PASS
  |-
  |-
  | Windows 8 32-bit
  | Windows 8 32-bit
  | Firefox 17.0.1, 17.0.4, 17.0.9esr
  | Firefox 17.0.1, 17.0.4, 17.0.9esr
  |  
  | PASS
  |  
  | PASS
  |  
  | PASS
  |-
  |-
  | Windows 8 64-bit
  | Windows 8 64-bit
  | Firefox 17.0, 17.0.1, 17.0.9esr
  | Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Mac OS X 10.6
| Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Mac OS X 10.7
| Firefox 17.0.2, 17.0.3, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Mac OS X 10.8
| Firefox 17.0.1, 17.0.2, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Ubuntu 12.04 32-bit
| Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Ubuntu 12.04 64-bit
| Firefox 17.0.2, 17.0.4, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Ubuntu 13.04 32-bit
| Firefox 17.3, 17.0.5, 17.0.9esr
| PASS
| PASS
| PASS
|-
| Ubuntu 13.04 64-bit
| Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
| PASS
| PASS
|}
=== Bug Fix Verifications ===
* {{done|921090}} NO VERIFY: Upgrade Firefox 17.0.x to NSS 3.14.4
** All-or-nothing checkin, no verification needed
* {{done|913614}} NO VERIFY: When the input is invalid, skip the work as we won't get anything useful anyway
** Verification of related bug is sufficient
* {{done|897678}} VERIFIED: Make worker code stop using propertyops, as a step toward removing propertyops altogether
* {{done|918864}} VERIFIED: Improve doStopDocumentLoad
* {{done|914017}} VERIFIED: Initialize members variables of the document hash correctly
* {{done|916580}} NO VERIFY: Fix bugs related to the usage of calloc
** No test case
* {{done|915210}} VERIFIED: Follow the COM rules when flushing a presshell
* {{done|860123}} VERIFIED: Disallow extending or collapsing selections across documents
* {{done|916576}} VERIFIED: Follow the com rules when calling LoadURI
=== Issues ===
==== New ====
==== Known ====
<bugzilla>
{
"id":"717178, 813531, 747909, 813531, 900934, 791195 "
}
</bugzilla>
== Firefox 17.0.11 ESR ==
{| class="querytable" style="width:30%"
| style="width:10%" |
| style="width:20%" |
|-
| '''Builds'''
| [ftp://ftp.mozilla.org/pub/firefox/nightly/17.0.11esr-candidates/build1/ FTP], [http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/17.0.11esr-candidates/build1/ HTTP]
|-
| '''ID'''
| 20131113153432
|-
| '''Changeset'''
| [http://hg.mozilla.org/releases/mozilla-esr17/rev/1cda5cffe4d7 1cda5cffe4d7] ([https://hg.mozilla.org/releases/mozilla-esr17/pushloghtml?fromchange=c3750e0515f7&tochange=1cda5cffe4d7 pushlog])
|}
; Checklist
* {{done|}} Functional automation
* {{done|}} Regression testing
* {{done|}} Update automation on esrtest
* {{done|}} Update automation on releasetest
* {{done|}} Final sign-off for push to esr
* {{done|}} Update automation on esr
=== Automated Testing ===
The following are the detailed results of the automated Mozmill test runs.
{|class="querytable" style="width:60%"
|-
| class="header" style="width:20%" | Platform
| class="header" style="width:20%" | Functional [[http://mozmill-ondemand.blargon7.com/#/functional/reports?branch=17.0&platform=All&from=2013-11-13&to=2013-11-14 report]]
|-
| Windows XP
| PASS
|-
| Windows Vista
| PASS
|-
| Windows 7 32-bit
| PASS
|-
| Windows 7 64-bit
| PASS
|-
| Windows 8 32-bit
| PASS
|-
| Windows 8 64-bit
| PASS
|-
| Mac OS X 10.6
| PASS
|-
| Mac OS X 10.7
| PASS
|-
| Mac OS X 10.8
| PASS
|-
| Ubuntu 12.04 32-bit
| PASS
|-
| Ubuntu 12.04 64-bit
| PASS
|-
| Ubuntu 13.04 32-bit
| PASS
|-
| Ubuntu 13.04 64-bit
| PASS
|}
=== Regression Testing ===
* Visit sites in this table and record any regressions you might find. Mark each config PASS or FAIL
{| class="querytable"
| class="header" | Sites Tested
| class="header" | Windows XP
| class="header" | Windows 7 64-bit
| class="header" | Windows 8.1 32-bit
| class="header" | Mac OSX 10.9
| class="header" | Ubuntu 13.04 64-bit
|-
| 1. https://www.facebook.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 2. https://www.tumblr.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 3. https://login.yahoo.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Mac OS X 10.6
  | 4. https://www.linkedin.com/
  | Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
  | PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 5. https://login.live.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 6. https://twitter.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Mac OS X 10.7
  | 7. http://www.amazon.com/
  | Firefox 17.0.2, 17.0.3, 17.0.9esr
| PASS
| PASS
| PASS
| PASS
  | PASS
  |  
  |  
|-
| 8. https://accounts.google.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 9. http://wordpress.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Mac OS X 10.8
  | 10. http://www.ebay.com/
  | Firefox 17.0.1, 17.0.2, 17.0.9esr
| PASS
| PASS
| PASS
  | PASS
| PASS
  |  
  |  
|-
| 11. http://www.aliexpress.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 12. https://www.pinterest.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Ubuntu 12.04 32-bit
  | 13. https://www.paypal.com/
  | Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
  | PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 14. https://instagram.com/accounts/login/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 15. https://login.alibaba.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Ubuntu 12.04 64-bit
  | 16. https://en.wikipedia.org/wiki/Main_Page
  | Firefox 17.0.2, 17.0.4, 17.0.9esr
| PASS
| PASS
| PASS
  | PASS
| PASS
  |  
  |  
|-
| 17. https://vimeo.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 18. http://emag.ro
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Ubuntu 13.04 32-bit
  | 19. https://www.dropbox.com/
  | Firefox 17.3, 17.0.5, 17.0.9esr
| PASS
  | PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 20. https://eu.battle.net/shop/en-us/
| PASS
| PASS
| PASS
| PASS
| PASS
|-
| 21. https://wiki.mozilla.org/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 22. https://mozqa.etherpad.mozilla.org/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
  |-
  |-
  | Ubuntu 13.04 64-bit
  | 23. https://bugzilla.mozilla.org/
  | Firefox 17.0, 17.0.1, 17.0.9esr
| PASS
  | PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 24. https://soundcloud.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
| 25. https://drive.google.com/
| PASS
| PASS
| PASS
| PASS
| PASS
  |  
  |  
|-
  |}
  |}


=== Bug Fix Verifications ===
=== Update Testing ===
* TBD: List of verified bugs
* Partial: 17.0.10
* TBD: List of unverified bugs
* Complete: <=17.0.8
<bugzilla>
 
{
{|class="querytable" style="width:60%"
  "id":"918864"
|-
}
| class="header" | Platform
</bugzilla>
| class="header" | Target Version
* {{ok|}}{{bug|xxxxx}} NOT VERIFIED: description
  | class="header" | esrtest
| class="header" | releasetest
| class="header" | esr
|-
| Windows XP
| Firefox 17.0, 17.0.3, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Windows Vista
| Firefox 17.0, 17.0.4, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Windows 7 32-bit
| Firefox 17.0, 17.0.4, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Windows 7 64-bit
| Firefox 17.0.2, 17.0.4, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Windows 8 32-bit
| Firefox 17.0.1, 17.0.4, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Windows 8 64-bit
| Firefox 17.0, 17.0.1, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Mac OS X 10.6
| Firefox 17.0, 17.0.1, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Mac OS X 10.7
| Firefox 17.0.2, 17.0.3, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Mac OS X 10.8
| Firefox 17.0.1, 17.0.2, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Ubuntu 12.04 32-bit
| Firefox 17.0, 17.0.1, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Ubuntu 12.04 64-bit
| Firefox 17.0.2, 17.0.4, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Ubuntu 13.04 32-bit
| Firefox 17.3, 17.0.5, 17.0.10esr
| PASS
| PASS
| PASS
|-
| Ubuntu 13.04 64-bit
| Firefox 17.0, 17.0.1, 17.0.10esr
| PASS
| PASS
| PASS
|}


=== Issues ===
=== Issues ===
Mwobensmith
Confirmed users
651

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/735572...762342"