Auto-tools/Meetings/2012-07-16: Difference between revisions

← Older edit

Auto-tools/Meetings/2012-07-16 (view source)

Revision as of 18:21, 16 July 2012

4,361 bytes added , 16 July 2012

→‎Goals Progress

Ctalbert

Confirmed users

3,816

edits

@@ Line 5: / Line 5: @@
 * {{ok|Support the B2G and platform team by moving existing automation to that platform as well as expanding automation on B2G and B2G devices for basecamp deliverables.}}
 * {{ok|Expand Mobile testing platforms for B2G and Fennec Native to include user devices, NEON Chipsets (panda boards), and ARMv6 (Beagle boards).}}
-** Project plan in place for AutoPhone.
+** [mcote] Project plan in place for AutoPhone.
+** [mcote] Refactoring and self-tests.
 * {{ok|Establish new Datazilla system as a useful tool for capturing and diagnosing talos performance regressions on a per-push basis, as well as capturing data for network and B2G performance tests.}}
-** ''NOTE:'' We will '''not''' be replacing existing Talos in Q3. By the end of Q3, we want to have Datazilla to a point at which it can shadow the existing Talos system and we can use Q4 to verify that the new approach captures just as many regressions as the old system does. Potential timeframe for conversion of Talos to the new Datazilla system is in Q1 2013 (earliest "thumb in the wind" estimate).
+** ''NOTE:'' This is largely a change to the data harvesting and analysis side of the Talos system (even though some changes are required to the test harness and tests themselves). We will '''not''' be replacing existing Talos data harvesting system in Q3. By the end of Q3, we want to have the new performance data harvesting and analysis system Datazilla to a point at which it can shadow the old system and we can use Q4 to verify that the new approach captures just as many regressions as the old system does. Potential timeframe for conversion of the old system to the new Datazilla system is in Q1 2013 (earliest "thumb in the wind" estimate).
-* {{ok|Implement and land Bugzilla UX Optimizations for web mail, tracking flags optimizations, secure mail, and work with UX team on other suggested improvements.}}
+* {{ok|Upgrade BMO to 4.2, deploy tracking flags optimizations, UX improvements (comment tagging, mid-air resolution)}}
+** blogged about 4.2 search changes (http://globau.wordpress.com/2012/07/13/bugzilla-4-2-whats-new-searching/), bmo/4.2 testing, and bugfixing
 * {{ok|Make TBPL More Orange aware to make it easier to file intermittent orange bugs, and so that we can more adequately track (and therefore address) issues around android-related and memory leak intermittent failures}}
+** [edmorley] Fixed {{bug|772388}} - Make ShutdownLeakLogger output the list of leaking tests in a format that can be individually starred.
 * {{ok|Enable QA and contributors to increase their skills and participation by crafting automated tests for B2G and desktop smoketests by providing examples, mentoring, and documentation}}
@@ Line 19: / Line 22: @@
 * [DBurns] Last week it was hopefully, but this week it is out! [http://www.w3.org/TR/2012/WD-webdriver-20120710/ Browser Automation Spec] is in FPWD
+* [ted] Got B2G crash reporting wired up, {{bug|761909}}. Sample crash report: https://crash-stats.mozilla.com/report/index/6f54b900-a9f0-468e-9e53-f795a2120716
+* [dkl] Got X-Headers in bugzilla implemented for tracking flag changes (x-bugzilla-tracking) (per a dev.planning thread)
+* [edmorley] Posted about TBPL/OrangeFactor changes [https://groups.google.com/d/topic/mozilla.dev.platform/QWzf-3YZCvM/discussion in dev.platform].
 == In-Depth Project Discussion ==
+* Ahal reviewed [https://developer.mozilla.org/en/Mozilla_automated_testing/Peptest peptest framework]
-''Pick one project/presenter per week who will give a detail presentation on their project.''
 == The Overview ==
@@ Line 36: / Line 41: @@
 === [[BMO|Bugzilla]] ===
+* bmo changes: http://globau.wordpress.com/2012/07/12/happy-bmo-push-day-4/
+* [glob] blogged about 4.2 search changes (http://globau.wordpress.com/2012/07/13/bugzilla-4-2-whats-new-searching/)
+* [glob+dkl] bmo/4.2 testing, and bugfixing
+* [dkl+glob] component searching enhancements (added to component browsing, pressing enter with a single result selects it)
+* [dkl] added x-bugzilla-tracking headers to bugmail (results of dev.planning discussion)
+* [glob+dkl] upstream development work and reviews
+* [glob+dkl] administration work (including rapid release 17 work)
 === [[Project_Eideticker|Eideticker]] ===
@@ Line 45: / Line 57: @@
 === [https://github.com/mozilla/Firebug-Test-Runner Firebug Testing] ===
+* [ahal] Option to specify custom database in order to move away from couchone
 === [[Auto-tools/Projects/Jetperf|Jetperf]] ===
+* [jhammel] initial config files up for feedback: https://bugzilla.mozilla.org/show_bug.cgi?id=767117
 === [[Auto-tools/Projects/Marionette|Marionette]] ===
 * [DBurns] Currently aligning client API with Selenium which allowed me to do what I did in the Open Web Apps section
+* [mdas] added a workaround for {{bug|771224}} so chrome tests can run again.
+* [mdas] readjusting performance test runner job, adding one more perf test {{bug|774278}}
 === Mobile Automation ===
@@ Line 55: / Line 71: @@
 * [jmaher] - investigating resource consumption during tests to see if that causes reboots/crashes/oranges.
 * [wlach] - various patches to SUTAgent and client library, mostly to fix eideticker-related stuff but may be useful elsewhere as well: commands now run as su by default ({{bug|763497}}), fall back to wifiman to get MAC address on older versions of Android ({{bug|768597}}), give better error when we fail to push a file ({{bug|772105}}), and don't consume tons of CPU when waiting for a response in the client ({{bug|773463}}).
+* [ahal] - b2g reftests running again after emulator fix and fix to {{bug|772307}}
+** continuing triaging process
+* [mihneadb] - got xpcshell tests running in b2g emulator
+* [mihneadb] - various fixes/refactorings to marionette and devicemanager
 === [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] ===
@@ Line 74: / Line 94: @@
 * [camd] Added storage and importing of pushlog data, merged to master
 * [camd] Adding batch processing to certain management commands for cache updates and data processing.  Added batch field to be able to assign projects to specific cron batches. (not yet merged)
+* [jeads] Pulled datazilla data for the talos test suite tp5n from 06/20/2012 to 07/05/2012 and ran t-test metrics on all of the associated pushes.  This time range contains the changeset a9ae09d4f1d8 which caused a performance regression.  Metrics is analyzing the data to further validate the t-test method.
+* [jeads] Cleaning up some loose ends to get ready for a release.  This includes adding handling to set a default branch, making some schema changes, and cleaning up how reference data is cached.
 === [[Auto-tools/Projects/Open_Web_Apps_Testing|Open Web Apps]] ===
@@ Line 79: / Line 101: @@
 === [[Auto-tools/Projects/peptest|Peptest]] ===
+* [ahal] Wrote test to catch unresponsiveness in tab switching
+** updating mozharness script to provide proxy sites
 === [[Auto-tools/Projects/Powerball|Powerball]] ===
@@ Line 108: / Line 132: @@
 === [https://tbpl.mozilla.org/ Tinderboxpushlog] ===
-* [edmorley] [https://bugzil.la/772546 Discussions about moving primary TBPL development out of a user repo and under hg.m.o/{automation,webtools,build}.
+* [edmorley] [https://bugzil.la/772546 Discussions] about moving primary TBPL development out of a user repo and under hg.mozilla.org/{automation,webtools,build}.
 * [edmorley] Fixed {{bug|772084}}.
-* Items to ease starring/identification of failures more easily:
+* [edmorley] Items to ease starring/identification of failures more easily:
 ** Fixed {{bug|772388}} - Make ShutdownLeakLogger output the list of leaking tests in a format that can be individually starred.
 ** Filed {{bug|772610}} - results.py shouldn't say "Graph server unreachable" when the test name isn't recognised.
@@ Line 116: / Line 140: @@
 === [[Auto-tools/Projects/WarOnOrange|War on Orange]] ===
+* [jgriffin,edmorley] Resolving: [https://bugzil.la/772001 OrangeFactor not updating since July 5th].
+* [edmorley] [https://bugzilla.mozilla.org/show_bug.cgi?id=669316#c15 Reviewed] mcote's [https://bugzil.la/669316 Orange Seed] patch.
+* [edmorley] Landed {{bug|769706}}.
+* [edmorley] Patch for {{bug|773312}}.
+* [edmorley] Posted about TBPL/OrangeFactor changes [https://groups.google.com/d/topic/mozilla.dev.platform/QWzf-3YZCvM/discussion in dev.platform].
 === Web QA ===
@@ Line 131: / Line 160: @@
 === Take notes [https://etherpad.mozilla.org/A-TeamMeetingNotes here] and copy and paste them to this section afterwards. ===
+* We discussed the goals, refining the language of them
+* We watched a fantastic presentation overview on peptest by Ahal
+* Held a long discussion around how to address the reftest screen size issue:
+** Reftest requires a resolution that is too large for mobile phones - need to figure out a way to reduce it so that we can perform true reftests on mobile and resolution constrained devices
+** Decided to change reftest to save all the canvases at the high resolution, then compare a subset of that canvas at our target small resolution, to determine tests that might fail (i.e. would be drawing anything out of the frame in the smaller resolution). We will generate a list of tests for further investigation from the set of tests that draw content outside the small resolution. Jmaher and ahal are going to take a crack at instrumenting reftest to support this and create the list.
 === Notes ===
@@ Line 138: / Line 172: @@
 === Next week MC ===
+* Ted will be the MC
+* Wlach will present Eideticker!!
 == Misc ==
@@ Line 144: / Line 180: @@
 * [http://www.damnyouautocorrect.com/21494/yoda-impression-fail/ Damn you Autocorrect, in the spirit of all our mobile work]
 * [wlach] hired [https://www.facebook.com/photo.php?fbid=10151108549551111&set=a.67659011110.103580.652596110&type=1&theater bicycle movers] to move apartments last weekend. Truly a sight to behold. See also: http://www.demenagementmyette.ca/videos/
+* [edmorley] A friend had this cake made for his birthday by his other half: [https://fbcdn-sphotos-a.akamaihd.net/hphotos-ak-snc7/582187_3254627337742_352238461_n.jpg pic]. Jealous! :-D