Auto-tools/Meetings/2012-06-11

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

• [ON TRACK] Extend Mobile Platform Automation for B2G and Fennec Native to extend our automation systems to work with specific phone hardware and new development boards for both products.
  • [wlach] - Added some preliminary support for the Pandaboard to Eideticker (commit), used for testing performance in bug 751347.
  • [jgriffin] - Turned on mochitest-plain on the B2G continuous-integration system: bug 759887. Need to investigate some random failures which are causing chunks to fail to start.

• [MISSED] Deploy Datazilla new graph server UI into production to make it easier and simpler to track all our performance numbers across our growing sets of performance and endurance data we are collecting on our products.

• [ON TRACK] Signal From Noise Phase II - Make the same noise-reduction changes we made on Tp5 on all the other page-load tests and ensure all performance tests are sending raw observations to Datazilla.
  • [jmaher] - all changes to tests are running side by side on mozilla-central.

• [AT RISK] Enhance Bugzilla Performance by upgrading hardware and software dependencies as well as fixing a major performance bug in BMO (a bug related to how we, as Mozilla, use bugzilla)

• [ON TRACK] Reduce android test automation instability and make it easier for the web QA and desktop QA teams to write and run automated tests.
  • [jmaher] - Worked with Callek to get updated tegra-host-utils.zip, landed, waiting on deployment

In-Depth Project Discussion

This week, jeads will be covering Datazilla!

The Overview

Mozbase

• [henrik] Released MozInstall v1.0 which includes an uninstaller now

Mozmill

• [henrik] Complete test refactor of Mutt JS tests (nothing touches the net anymore!!)
• [henrik] Finally removed the JUM unit test module
• [henrik] Mozmill opens target application in the background now!
• [henrik] Enhancements for the screenshot feature

Mochitest

• [jmaher] - plan to turn on 200,000+ new mochitests for android native this week

Eideticker

• (as above) [wlach] - Added some preliminary support for the Pandaboard to Eideticker (commit), used for testing performance in bug 751347.
• [wlach] - Did some preliminary tests on Galaxy Nexus for Fennec vs. other browsers (Chrome, Dolphin, Opera, etc.) for marketing and qualification stuff.
• [wlach] Various help getting the testing setup in MV back online, dashboard should hopefully be running this week.

Peptest

• [mcote] Fixed bug causing problems in OS X runs.
• [mcote] Met with Christina and Gilbert from analytics about peptest statistics and forecasting.
  • Given their experiences with talos, they wanted more information and justification about the methods we are using. Specifically, they were concerned with the fact that we only record the duration of the event loop when it exceeds 50 ms; when it is less, we record nothing. They would love to see complete data on the event-loop duration for a test run or two. They also had concerns with the way threading may affect the results.

Sheriffing & Trees

• [edmorley] Lost Mon/Tues to public holiday.
• [edmorley] Day-to-day sheriffing of the trees.
• [edmorley] Infra/tree closure issues: https://bugzil.la/762873,762879,762869

treestatus.mozilla.org

• [edmorley] Posted to dev.platform about upcoming switch to Treestatus & added the people who responded to Treestatus with sheriff permissions. Anyone who has not yet let me know, needs to do so ASAP.
• [edmorley] Deployed latest changes to prod, fixing previous state bug & a few other papercuts.

Tinderboxpushlog

• [edmorley] Deployed a number of TBPL improvements to prod: https://bugzil.la/656732,756724,757036,757879,758680,758954,758962,759293,759313,759329
• [edmorley] Posted to dev.platform about the TBPL UX changes.
• [edmorley] Patch for bug 762120 - Reftest analyser doesn't work when using baseURL to run TBPL from the local filesystem.
• [edmorley] Profiled hangs when viewing tinderbox logs (affects sheriffs multiple times a day), filed bug 762710, seems to be making progress \o/

War on Orange

• [mcote] Still working on Orange Seed. Got the data I need and able to roughly correlate test-run and orange data.

Talos

• [jhammel] working on making results reporting more extensible and sane (needed for jetperf and talos+mozharness)
• [jmaher] - dromaeo raw results landed (not deployed), will work on xperf raw results

Marionette

• [mdas] finishing up perf test work, will move on to fixing regressions

Bughunter

• [bc] crash triaging, test case reduction and regression finding continues
• [bc] working on binary search tool using nightly opt and debug builds for beta, aurora and nightly builds.

New Graphserver (Datazilla)

• [carljm] database isolation for datazilla tests
• [carljm] investigated querying hg.mozilla.org for changeset ordering

autophone/Noah's Ark

• Autophone server and devices moved to new subnet. adb now appears to be totally crazy on the server, even after a reboot. :\

Speedtests

• Nada.

Jetperf

• [jhammel] figured out who was working on what and the last missing pieces
• [jhammel] fixed last outstanding nits; working on reporting to datazilla

Mobile Evangelism Site-Scraping Tool

• [carljm] put together and presented project proposal/estimate

Upcoming Events

All times Pacific Time, click on link to find your timezone

• Platform Meeting (all platform engineers) Tuesday at 11AM, Conf # 8605.
• Mobile Test Automation Wednesday at 10:30AM, Vidyo Warp Core
• Marionette/Eideticker Meeting Thursday at 10AM Conf # 304
• Signal from Noise Thursday at 11AM #343

Round Table

• The list of standing weekly meetings somehow got dropped from recent meeting pages, and is anyway out-of-date. Let's keep it up-to-date!

Holidays and Trips

• Clint is out from June 8 to June 17 - back on the 18th. See Jgriffin, Mcote, Jmaher, or Bmoss if you need something urgent while I'm out. I will be reachable by cell. Feel free to call.
• Jgriffin is out from June 20 to July 2. See Mdas for B2G issues or Ctalbert for anything else. I will be reachable by cell phone; ctalbert has the number!
• wlach out from June 18 to July 2. Mcote to keep eideticker dashboard running while gone.

Misc

• Udacity is offering a free course on statistics late June, which might be interesting for some people given our recent focus on more rigorous analysis in various projects (Datazilla, Talos, Eideticker, Peptest, etc.). I (wlach) took Professor Thrun's course on AI last fall and it was great: http://www.udacity.com/overview/Course/st101/CourseRev/1