Mobile/Testing/12 04 13

Previous Action Items

• snorp to try again to repro canvasmark failures on panda
  • see http://puppetagain.pub.build.mozilla.org/data/bmm/artifacts/panda-android-4.0.4_v3.1/ and https://bugzilla.mozilla.org/show_bug.cgi?id=914472#c3
• rel-eng to test sut 1.20 in staging
  • No problems discovered, plan to deploy live today.
• callek to disable canvasmark on pandas -- bug 916258
• jmaher to follow-up with blassey on talos -- bug 941155

Status reports

Rel Eng

• graydon has a spot on Ash to mess with Android 2.3 testing on EC2
  • https://tbpl.mozilla.org/?tree=Ash&jobname=Android%202.3

A Team

• tegra failure rate: [1.22%]
• panda failure rate: [13.6%]
  • excluding retries: 2.03

• Working on bug 937637 to get cpp unittests scheduled on trunk trees
• Can't attend this week due to a conflict

x86 automation

• All tests running hidden on Cedar, with some failures
• Complete:
  • kvm enabled and deployed: bug 929048
  • chunks coalesced, sets coalesced: bug 935214
  • patched emulator deployed: bug 933918
  • S1 split into 2 sets and output no longer duplicated: bug 939823
  • new AVD definitions (Graydon's) deployed: bug 919812
• Working on:
  • cleaning up remaining failures: bug 936226

Round Table

• (armenzg) Re-thinking our Mobile testing
  • see http://blog.drapostles.org/archives/107 for info

We have awful wait times on Tegras

• We currently test Android 2.2 on Tegras
  • However, only less than 2% of our users are using Android 2.2
  • graydon is helping us to have an Android 2.3 story (10.82%-14.22% of our users)
• We currently test Armv6 builds (~2% of users) and no-ion (we don't ship this - only b2g18 branches)
  • I want to know if we can have a not-per-checkin model testing for Armv6
  • This would improve our current 2.2 testing on Tegras

I think we're OK with Pandas wrt to capacity, however, I want to point out that 4.1 has passed 4.0 in number of users.