B2G/QA/2.2/2015-05-04Status

From MozillaWiki
< B2G | QA | 2.2
Jump to navigation Jump to search

V2.2 Highlight & Risk

V2.2 Milestone

  • 2015/06/08 V2.2 CC
  • 2015/04/29 V2.2 Updated FC
  • 2015/04/06 V2.2 FC
  • 2015/02/23 V2.2 FL

V2.2 Testing Scope

Bugzilla query error

There was a problem during the HTTP request: 400 Bad Request


V2.2 Testing by Type

Smoke Testing

Nexus-5-L 5.1 Yellow

  • Testing on PVT build from 2015/04/14
  • Cases blocking manual test
    • OTA not working, but the update channel is fine for 2.2->2.2.
  • Manual - are available on Google Drive
  • Known issues - no FM radio support, no DSDS support and no UMS support.

Flame-KK Green

  • No bug found blocking manual smoketests but we have couple automation bugs need to investigate, and the removal of 'Mozilla Mobile' has huge impact on our testing framework, we are currently trying to resolve this.
  • New Bugs Breaking the Automation smoketests
    • [Investigating] test_clock_create_new_alarm.py - Bug 1138278- [Clock] After running test_clock_create_new_alarm.py, it should set the alarm time to 1 min more than the current time
    • [Investigating] [Not reproduced manually] test_ftu_skip_tour.py - Bug 1138278 - test_ftu_skip_tour should reflect new Send Data initial state and separate SIM- and network-available conditions
    • [Investigating] test_marketplace_packaged_app.py - Bug 1151172 - [marketplace-tests-gaia] "test_marketplace_search_and_install_app" is failing on b2g v2.2
    • [Not reproduced] test_sms_with_attachments.py - Bug 1151696 - [v2.2] Investigate test_sms_with_attachments.py failure . (Cannot reproduce locally)
    • [Not reproduced manually] test_music_artist_mp3.py - Bug 1149886 - using to replace at Music app results in unable to locate <span.list-song-index> error

Regression Testing

2nd Run (2015/04/14)

  • Result
    • 1119(total), 1090(passed), 8(failed), 21(invalid)
    • 0.007% fail rate
  • bugs
    • Total 5
    • http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)
  • Result
    • ​342(total), 334(passed), 5(failed), 3(invalid)
    • ​0.01% fail rate

1st Run (2015/03/03)

Functional Testing

Red

  • General
    • Details:
    • NEXUS-5-L, FLAME-KK - will start another exploratory test from next week in TPE and in China
    • FLAME-L - RD needs time to make it work, currently it fails to boot into home screen.
    • ZTE 8909 - just for reference, DSDS, FM okay, SD not working, running MTBF now.
  • Cut, Copy, and Paste (Gerry, Edward, Teri)
    • Health Status:
    • Details: 1 blocker
  • System Platform (Gerry)
    • Health Status:
    • Details: 4 blockers
  • OTA/FOTA (Gerry)
    • Health Status:
    • Details:
      • 2.0/2.1 can’t upgrade to 2.2 via FOTA - Bug 1155035 - patch ready
      • (Don't support) 2.0 can upgrade to 2.2 via OTA with some issues
        • Bluetooth issue (resolved) - Bug 1132388 & Bug 1134821
        • Bug 1153787 & Bug 1153778
      • (Don't support) 2.1 can’t upgrade to 2.2 via OTA - Bug 1151006
      • Best way to do FOTA - comment #55 in BZ1136638
  • NFC (Alison)
    • Health Status:
    • Details:
      • 2 blockers:

https://bugzilla.mozilla.org/show_bug.cgi?id=1155873 (patch r+, wait to land code) https://bugzilla.mozilla.org/show_bug.cgi?id=1156605 (RESOLVED FIXED, wait uplift to v2.2)

      • 2 nominators:

https://bugzilla.mozilla.org/show_bug.cgi?id=1158379 https://bugzilla.mozilla.org/show_bug.cgi?id=1158388


Exploratory Test

Flame-KK & NEXUS-5-L

  • Regression bugs

Stability

NEXUS-5-L 5.1

  • Environment
    • Display DIM
    • Rest time with dummy case (30 sec)
    • Running tests ** 14 case http://bit.ly/1aWkJyY
    • NEXUS *4, ZTE *2

- Result

  • NEXUS-5-L 5.1
    • Current run - NEXUS-5-L 5.1 *4: 13.77 hrs = [17.94+17.39+10.03+51.05] / (4 devices + 3 crashes) = 13.77
    • Current run - ZTE-L 5.1 * 2: Just started
    • Previous run - NEXUS-5-L 5.1 *4: MTBF 34.42 hrs = (41.3 + 53.2 + 43.8 + 6.1)/4
    • NEXUS-5-L 5.1 Rest time: 10%
    • [AudioSystem issue] https://bugzil.la/1155490
    • [adb issue] https://bugzil.la/1154629


FLAME-KK 4.4

  • Environment
    • Display DIM
    • Rest time with dummy case (30 sec)
    • FLAME *9

- Result

  • FLAME-KK 4.4
    • FLAME-KK 4.4 * 4: MTBF 34.42 hrs

V2.2 Test Case Creation

On Track

Test Case Creation WIP

Test Case Creation DONE

  • [2.2-feature-qa+] & in-moztrap+ (19 bugs) --> WORK DONE
  • feature-b2g=2.2+ & in-moztrap+ (78 bugs) --> WORK DONE

V2.2 Testing Verification Status

Overall Verification Summary

Need Verification

Full Query
ID Qa contact Status Resolution Component Assigned to Summary
987532 npark@mozilla.com RESOLVED WORKSFORME Performance bignose1007+bugzilla@gmail.com [meta] Project Silk on B2G.
1018320 RESOLVED FIXED DOM: Core & HTML amarchesini@mozilla.com Implement RequestSync API for FirefoxOS
1020757 RESOLVED FIXED Gaia::Settings eragonj+moz@eragonj.me [FDN] Enable/Disable FDN asks to enter PUK if the enable/disable FDN failed
1043558 RESOLVED FIXED Graphics sotaro.ikeda.g@gmail.com Use gralloc buffers for WebRTC local camera preview
1061130 RESOLVED FIXED Gaia::Dialer gsvelto@mozilla.com [STK] 3GPP TS 22.030 6.6.4 An abbreviated dialling code shall be able to be read using the following procedure: N(N)(N)#
1068963 RESOLVED INCOMPLETE Audio/Video blakewu0205@gmail.com Frame drops during end-to-end playback
1071484 RESOLVED FIXED Audio/Video bobby.chien+bugzilla@gmail.com [Meta][User story] Support *.mkv container format
1073419 RESOLVED WONTFIX General nobody@mozilla.org [ALA] All gecko code needed to support adjustable location accuracy
1082268 edchen@mozilla.com RESOLVED WORKSFORME Gaia::Music bobby.chien+bugzilla@gmail.com [meta] Music app launch latency is worse compared to Android on quad core 1 GB device
1082290 RESOLVED FIXED Performance dhylands@gmail.com Implement cgroup swappiness feature for low memory target (256 MB)
1083953 RESOLVED FIXED Gaia marta@sec.t-labs.tu-berlin.de [META] Privacy Panel move from dev_apps to apps
1086963 edchen@mozilla.com RESOLVED WORKSFORME Gaia::Gallery bobby.chien+bugzilla@gmail.com [meta] Gallery app launch latency is worse than Android on quad core device
1092427 echang@mozilla.com RESOLVED FIXED Gaia::Dialer thills@mozilla.com Dialer app should follow text selection pattern.
1092437 echang@mozilla.com RESOLVED FIXED Gaia::SMS nobody@mozilla.org Messages app should follow text selection pattern
1092960 RESOLVED FIXED Gaia::Camera wilsonpage@mozilla.com Camera app to follow text selection pattern
1092966 RESOLVED FIXED Gaia::FMRadio jdarcangelo@mozilla.com FM Radio app to follow text selection pattern
1092967 RESOLVED FIXED Gaia::Gallery pdahiya@mozilla.com Gallery app to follow text selection pattern
1092969 echang@mozilla.com RESOLVED FIXED Gaia::Music dominickuo@csie.io Music app to follow text selection pattern
1092972 RESOLVED FIXED Gaia::Video rnicoletti@mozilla.com Video app to follow text selection pattern
1094121 RESOLVED FIXED General styang@mozilla.com [meta] Android L 5.0 Porting for B2G
1097865 shing.lyu@gmail.com RESOLVED WONTFIX Gaia::System marshall@mozilla.com [Telemetry] Collect metrics for voice call info
1098970 RESOLVED FIXED GonkIntegration boris.chiou@gmail.com [lollipop] Porting NativeWindow from LL to gonk
1099076 shing.lyu@gmail.com RESOLVED WONTFIX WebRTC nobody@mozilla.org Screen sharing support on B2G
1099079 RESOLVED WONTFIX Gaia nobody@mozilla.org Implement permission dialog for screen sharing
1107291 RESOLVED FIXED MTP/UMS viralwang@gmail.com SD/Storage (fuse) for L on Nexus 5
1107296 RESOLVED WORKSFORME General 6jamin@gmail.com [meta] GPS Android L Porting
1107298 RESOLVED FIXED General viralwang@gmail.com [meta] Sensors Android L Porting
1107305 RESOLVED FIXED General seinlin.maung@gmail.com [gonk-l] the error message: could not load library "libsigchain.so" from LD_PRELOAD
1109422 shing.lyu@gmail.com RESOLVED FIXED General marshall@mozilla.com AppUsage Metric collection enhancements for MonthlyActiveUsers computation
1115356 RESOLVED INVALID Gaia::Settings jocheng@mozilla.com [meta]Privacy Bugs Meta

30 Total; 0 Open (0%); 30 Resolved (100%); 0 Verified (0%);


Verification Done

Full Query
ID Qa contact Status Resolution Component Assigned to Summary
1107304 VERIFIED DUPLICATE General nobody@mozilla.org [meta] BT Android L Porting

1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);

Retrieved from "https://wiki.allizom.org/w/index.php?title=B2G/QA/2.2/2015-05-04Status&oldid=1071938"