MozillaWiki

Releases/Firefox 24/Test Plan: Difference between revisions

Page Discussion

Releases/Firefox 24/Test Plan (view source)

Revision as of 16:02, 17 September 2013

237 bytes added ,  17 September 2013
m
→‎Details
 
(40 intermediate revisions by 6 users not shown)
Line 39: Line 39:
* 2013.09.05: {{done|}} Beta 9
* 2013.09.05: {{done|}} Beta 9
* 2013.09.10: {{done|}} Beta 10
* 2013.09.10: {{done|}} Beta 10
* 2013.09.13: {{ok|}} Release Candidate
* 2013.09.13: {{done|}} Release Candidate
* 2013.09.17: {{ok|}} Final
* 2013.09.17: {{done|}} Final


== Features ==
== Features ==
Line 95: Line 95:


=== Bug Fix Verifications ===
=== Bug Fix Verifications ===
note: this doesn't show the security fixes that have been verified.
<bugzilla>
<bugzilla>
{
{
"id": "",
"id": "867226,888839,894234,744836,883544,883562,883973,884194,871368,877390,887098,906301,907727,895285,872375,908386,871338,893308,883514,882865,888252,876129,886690,904521,880882,890085,893513,904263,876721,876332,881782,883495,889821,902375,901351,901955,907503,907657,898871,904810,881634,890690,908651,851353,880377,828284,872307,874870,888745,876080,876155,868416,904561,851606,883607,853774,888972,889504,898161,880237,891035,902532,874090,880042,873010,873012,856823,870007,894576,888510,895939,879787,875757,876458,883524,884053,886803,899447,858101,885668,857883,866915,877612,870400,876044,878606,895023,901747,881264,865806,905176,906979,888274,888304,887368,888336,888867,885351,879096,883560,883589,872455,874391,875769,874158,877164,873812,874546,874569,874572,891158,885433,888172,894007,883928,879727,886255,898832,763879,842657,873222,903475,874974,877127,881370,883748,644952,866444,873902,874053,890374,880734,873073,855221,861595,867758,878751,887466,868316,868432,738396,870216,871942,880591,864120,870192,633001,870034,877287,877826,769348,850571,871099,873926,871917,871322,871156,872479,879799,886137,821073,878702,880911,882652,887210,910741,880947,52500,882164,878705,888458,863635,866567,870529,875609,878449,900200,872466,874469,875878,876346,884334,875790,876856,865537,868405,875060,886295,906199,844606,879723,892370,880842,868498,877085,877097,881022,896443,871560,877912,877532,883541,902350,865702,891424,876313,893061,873757,888900,829954,821877,879161,881266,883554,886886,886067,862679,878406,886717,890259,889788,875252,887773,888834,876806,877351,888292,888908,905084,860490,832963,887075,892340,889193,888252,907404,884156,873966,904315,886134,888300,875777,644952,912606,873963,872605,902156,902546,875748,908824,904616,885751",
  "include_fields":"id, summary, status, resolution"
  "include_fields":"id, summary, status, resolution"
}
}
Line 105: Line 106:
<bugzilla>
<bugzilla>
{
{
  "id": "835319, 881634, 906057, 901527, 907213, 907214, 890247, 882819, 872413, 313959, 694748, 743563, 759334"
  "id": "835319, 881634, 906057, 901527, 907213, 907214, 890247, 882819, 872413, 313959, 694748, 743563, 759334, 912110, 791195"
}
}
</bugzilla>
</bugzilla>
Line 119: Line 120:
  |-
  |-
  | Build ID
  | Build ID
  | 20130909203154
  | 20130910201120
  |-
  |-
  | Changes
  | Changes
Line 125: Line 126:
  |-
  |-
  | Configs
  | Configs
  | [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx24b10 mozmill]
  | [https://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx24RC1 mozmill]
  |}
  |}


Line 134: Line 135:
  |-
  |-
  | Regression Tests  
  | Regression Tests  
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Automated Tests
  | Automated Tests
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Bug Fix Verifications
  | Bug Fix Verifications
  | {{ok| on going in beta cycles}}
  | {{done|}}
  |-
  |-
  | Update Tests (betatest)
  | Update Tests (betatest)
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Update Tests (releasetest)
  | Update Tests (releasetest)
  | {{ok|}}
  | {{done|}}
  |-  
  |-  
  | Update Tests (beta)
  | Update Tests (beta)
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Stability & Support sign-off
  | Stability & Support sign-off
  | {{ok|}}
  | {{done|}}
  |-
  |-
  |}
  |}
Line 158: Line 159:
==== Regression Tests ====
==== Regression Tests ====
Exploratory testing on the below areas:
Exploratory testing on the below areas:
* Breakpad End-to-End functionality (use this [https://moztrap.mozilla.org/manage/cases/?filter-id=2525#caseversion-id-66838 MozTrap test] for guidelines).
* Breakpad End-to-End functionality ([https://moztrap.mozilla.org/manage/cases/?filter-id=2525#caseversion-id-66838 guidelines])
*
* Partner repack update ([https://moztrap.mozilla.org/manage/cases/?filter-id=6785#caseversion-id-69632 guidelines])
* Windows share compatibility smoketests ([https://moztrap.mozilla.org/manage/cases/?filter-id=7514#caseversion-id-69661 guidelines])


{| class="querytable" style="width:35%"
{| class="querytable" style="width:35%"
Line 166: Line 168:
  |-
  |-
  | Windows 7 64-bit
  | Windows 7 64-bit
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Windows 8 32-bit
  | Windows XP
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Mac OSX 10.6
  | Mac OSX 10.8
  | {{ok|}}
  | {{done|}}
  |-
  |-
  | Ubuntu 12.04 32-bit
  | Ubuntu 12.04 32-bit
  | {{ok|}}
  | {{done|}}
  |}
  |}


Line 185: Line 187:
  |-
  |-
  | Automated Tests
  | Automated Tests
  | '''results''' ([http://mozmill-release.blargon7.com/#/functional/reports?branch=24.0&platform=All&from=2013-09-10&to=2013-09-12 details])
  | '''PASS''' ([http://mozmill-release.blargon7.com/#/functional/reports?branch=24.0&platform=All&from=2013-09-10&to=2013-09-12 details])
  | All Mozmill ''ondemand_functional'' testruns PASS with no new failures
  | All Mozmill ''ondemand_functional'' testruns PASS with no new failures
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.04
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.04
Line 202: Line 204:
  |-
  |-
  | Update Tests <br> (betatest)
  | Update Tests <br> (betatest)
  | '''results''' ([http://mozmill-release.blargon7.com/#/update/detail?branch=24.0&channel=betatest&from=2013-09-12&to=2013-09-12&target=24.0 details])
  | '''PASS''' ([http://mozmill-release.blargon7.com/#/update/detail?branch=24.0&channel=betatest&from=2013-09-13&to=2013-09-13&target=24.0 details])
  | All Mozmill ''ondemand_update'' testruns PASS with no new failures
  | All Mozmill ''ondemand_update'' testruns PASS with no new failures
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.04
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.04
* Builds: 20.0b1-24.0b9 da de en-US es-ES fr it ja ko pl pt-BR ro ru zh-TW
* Builds: 19.0-23.0.1 da de en-US es-ES fr it ja ko pl pt-BR ro ru zh-TW
  |-
  |-
  | Update Tests <br> (releasetest)
  | Update Tests <br> (releasetest)
  | '''results''' ([http://mozmill-release.blargon7.com/#/update/detail?branch=24.0&channel=releasetest&from=2013-09-12&to=2013-09-12&target=24.0 details])
  | '''PASS''' ([http://mozmill-release.blargon7.com/#/update/detail?branch=24.0&channel=releasetest&from=2013-09-17&to=2013-09-17&target=24.0 details])
  | All Mozmill ''ondemand_update'' testruns PASS with no new failures
  | All Mozmill ''ondemand_update'' testruns PASS with no new failures
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.0.4
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.0.4
* Builds: 20.0b1-24.0b9 da de en-US es-ES fr it ja ko pl pt-BR ro ru zh-TW
* Builds: 19.0-23.0.1 da de en-US es-ES fr it ja ko pl pt-BR ro ru zh-TW
  |-
  |-
  | Update Tests <br> (beta)
  | Update Tests <br> (beta)
  | '''results''' ([http://mozmill-release.blargon7.com/#/update/detail?branch=24.0&channel=beta&from=2013-09-12&to=2013-09-12&target=24.0 details])
  | '''PASS''' ([http://mozmill-release.blargon7.com/#/update/detail?branch=24.0&channel=release&from=2013-09-17&to=2013-09-17&target=24.0 details])
  | All Mozmill ''ondemand_update'' testruns PASS with no new failures
  | All Mozmill ''ondemand_update'' testruns PASS with no new failures
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.04
* Platforms: Windows XP/Vista/7/8, Mac OSX 10.6/10.7/10.8, Ubuntu Linux 12.04/13.04
* Builds: 20.0b1-24.0b9 da de en-US es-ES fr it ja ko pl pt-BR ro ru zh-TW
* Builds: 19.0-23.0.1 da de en-US es-ES fr it ja ko pl pt-BR ro ru zh-TW
  |}
  |}


Line 433: Line 435:
Failures ([http://mzl.la/11JKuu4 known])
Failures ([http://mzl.la/11JKuu4 known])
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 562: Line 556:
*  test Failure {{bug|912023}} - ' A new window has been opened' in /testPrivateBrowsing/testOpenClosePBKeyboardShortcut.js  
*  test Failure {{bug|912023}} - ' A new window has been opened' in /testPrivateBrowsing/testOpenClosePBKeyboardShortcut.js  
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 691: Line 677:
*   
*   
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 819: Line 797:
*   
*   
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 948: Line 918:
*   
*   
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 1,184: Line 1,146:
* {{bug|907101}} '''Test failure''' "Certificate's country code is displayed - '(US)' should equal '(US,postalCode=94041,STREET=650 Castro St Ste 300)'" in testGreenLarry.js was seen in every testrun.  
* {{bug|907101}} '''Test failure''' "Certificate's country code is displayed - '(US)' should equal '(US,postalCode=94041,STREET=650 Castro St Ste 300)'" in testGreenLarry.js was seen in every testrun.  
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "869866,826938,873966,852848,887319,858940,890853",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 1,314: Line 1,268:
*  
*  
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "869866,826938,873966,879172,852848,887319,886916,885768,858940,844177,900694,843497,888665,868816,890853,886156",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 1,443: Line 1,389:
*  
*  
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "869866,826938,873966,879172,872375,871338,876129,852848,887319,880882,890085,867758,886916,885768,887210,858940,878705,888458,872466,874870,844177,900694,843497,888665,868816,871322,890853,876346,886156",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Line 1,593: Line 1,531:
*  
*  
*
*
==== Bug Fix Verifications ====
<bugzilla>
{
"id": "869866,851606,891035,826938,874090,873966,856823,879172,887098,872375,871338,876129,852848,887319,880882,890085,866444,890374,876721,867758,868316,868432,886916,769348,885768,887210,52500,858940,878705,888458,878449,872466,874870,844177,900694,843497,888665,862679,887773,888834,876806,888292,888908,868816,861595,871322,890853,876346,877532,886156",
"include_fields":"id, summary, status, resolution"
}
</bugzilla>


==== Update Tests ====
==== Update Tests ====
Twalker
Confirmed users
4,072

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/707202...710372"