Mobile/Testing/12 11 13

Previous Action Items

• snorp to try again to repro canvasmark failures on panda
  • see http://puppetagain.pub.build.mozilla.org/data/bmm/artifacts/panda-android-4.0.4_v3.1/ and https://bugzilla.mozilla.org/show_bug.cgi?id=914472#c3
• callek to disable canvasmark on pandas -- bug 916258
• jmaher to follow-up with blassey on talos -- bug 941155
• armenzg to send email about tegra retirement schedule, plans for armv6, android version coverage

Status reports

Rel Eng

• Disabled tcanvasmark on pandas, everywhere
• In bug 918168 Marked 44 tegras as decomissioned due to varying obvious signs of bad hardware (I can chat off-meeting with anyone who cares for more information on how I determined 'bad')
• IT Recovered a whole bunch of pandas in bug 902657 at releng urging.
  • Most common issue was failing sdcard
• IT Recovered a large swatch of tegras which fell through cracks by releng in bug 944498
• coordination continues on planning 650-castro evac of tegra infrastructure to scl3. More info as it gets locked down.

A Team

• tegra failure rate: [3.25%]
  • Possibly due to large backlog of tegra tests to run?
• panda failure rate: [20.21%]
  • excluding retries: 2.09
  • last week's failure rate was 13.6%, retries are the culprit

• Cpp unittests now scheduled and riding the trains (thanks bhearsum!)
• Landed fix for bug 945273 to get panda reftests green again

x86 automation

• All tests running hidden on Cedar, with some failures
• Working on:
  • cleaning up remaining failures: bug 936226
  • biggest remaining problem is intermittent loss of connectivity: bug 927602

Autophone

• Nexus 4 and Nexus 7
  • Received two Nexus 4s and one Nexus 7 2012 model.
  • Need to figure out how to root. Suggestions welcome.
  • Will wait for ordering Nexus 5 and Nexus 7 2013 model until I figure out the rooting.
• Fixed
  • bug 945390 Autophone - support build selection via revision
  • bug 947449 Phonedash - change tooltip to display full build date time.
  • bug 943871 Autophone - use http instead of ftp for build urls
  • bug 943873 Autophone - support inbound archive
  • bug 948512 Autophone - specify device specific tests
• Current Device assignments
  • s1s2
    • 80_96_b1_ee_55_e0_atrix
    • 40_fc_89_4c_95_3f_atrix-4g
    • c8_aa_21_ac_0c_b5_droid-pro
    • 98_4b_4a_13_01_84_droid-pro
    • a8_26_d9_93_fe_4b_htc-sensation
    • 44_a7_cf_ea_f9_73_lg-revolution
    • 00_23_76_b1_92_39_nexus-one
    • 00_23_76_96_cc_6f_nexus-one
    • 90_21_55_09_87_95_nexus-one
    • 78_d6_f0_cf_d2_17_nexus-s
    • 78_d6_f0_cf_8d_11_nexus-s
    • 98_0c_82_33_ec_8d_samsung-gs2
    • 20_64_32_4f_e9_4f_samsung-gs2
    • 20_64_32_21_35_70_samsung-gs2
    • 14_7d_c5_af_c8_e3_samsung-gs2
    • 04_46_65_fd_2f_e1_samsung-gs2
    • 5c_0a_5b_b4_31_32_samsung-gs3
    • 38_aa_3c_1e_a8_ee_samsung-gs3
    • 38_aa_3c_1c_f6_94_samsung-gs3
  • Mochitest-Canvas
    • 80_96_b1_ee_55_e0_atrix
    • 40_fc_89_4c_95_3f_atrix-4g
    • a8_26_d9_93_fe_4b_htc-sensation
    • 98_0c_82_33_ec_8d_samsung-gs2
    • 20_64_32_4f_e9_4f_samsung-gs2
    • 20_64_32_21_35_70_samsung-gs2
    • 14_7d_c5_af_c8_e3_samsung-gs2
    • 04_46_65_fd_2f_e1_samsung-gs2
    • 5c_0a_5b_b4_31_32_samsung-gs3
    • 38_aa_3c_1e_a8_ee_samsung-gs3
    • 38_aa_3c_1c_f6_94_samsung-gs3

• We can now use builds from http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/ and http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/ which extends our mozilla-central builds back to August 2013 and mozilla-inbound builds back to May 2013.
• Phonedash timezone - phonedash.mozilla.org displays dates in UTC but build date and build id values should be in Mountain View time. Currently, you need to subtract 8 hours from the displayed date to get the actual build date. Supposedly, flot 0.8.2 can handle timezones correctly. I recommend we take the time to upgrade flot so that phonedash will display build dates in Mountain View time accurately. This will help in triaging results and filing accurate bugs.

Round Table

• Increase in failure rates.
  • http://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=663657&tree=trunk&startday=2013-11-01&endday=2013-12-09
  • http://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=915449&tree=trunk&startday=2013-11-01&endday=2013-12-09
  • http://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=807230&tree=trunk&startday=2013-11-01&endday=2013-12-09
  • http://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=720452&tree=trunk&startday=2013-11-01&endday=2013-12-09
  • New, infrequent, crashes: bug 948390, bug 948360, bug 948342

• (armenzg) Re-thinking our Mobile testing
  • see Callek's post & armenzg's draft
  • Please load this URL during our call.