canmove, Confirmed users
1,126
edits
Auto-tools/Goals/2012Q4: Difference between revisions
Jump to navigation
Jump to search
Update sheriffing goals
(Update sheriffing goals) |
(Update sheriffing goals) |
||
| Line 35: | Line 35: | ||
* '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem''' | * '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem''' | ||
** {{done|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2G tests show up in TBPL}} | ** {{done|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2G tests show up in TBPL}} | ||
*** {{done|Add TBPL support for B2G emulator/Panda/Unagi/Otoro}} | |||
*** {{done|Make TBPL's log parser compatible with mozharness-style output}} | |||
*** {{done|File/track B2G intermittent failures, report issues with mozharness' error detection/reporting & unhide builds/testsuites on TBPL once they pass consistently}} | |||
*** {{done|Add the new B2G18 tree to OrangeFactor}} | |||
** {{done|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}} | ** {{done|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}} | ||
*** {{done|Add support for more types of failures to TBPL's log parser}} | |||
*** {{done|Improve test harness & buildbot output for failures & make the output more compatible with TBPL's parser}} | |||
*** {{done|Improve TBPL's bug suggestions for failures}} | |||
*** {{done|Make TBPL's tree open/closed status update more frequently}} | |||
*** {{done|Resolve timeouts during TBPL log parsing/summary generation}} | |||
*** {{done|Simplify the 'TBPL job -> recent history for that machine' workflow}} | |||
*** {{done|Obtain BuildVPN access & investigate BuildVPN-only tools and their use to sheriffs}} | |||
*** {{done|Simplify the TBPL -> 'viewing job on the buildbot master' workflow}} | |||
** {{done|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}} | ** {{done|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}} | ||
** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}} | ** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}} | ||
*** | *** Patch in progress, see {{bug|797811}}. | ||
** {{ | ** {{risk|Assist with getting Datazilla's UI integrated with TBPL, phase 1}} | ||
*** | *** Awaiting finalised datazilla/talos output format, see: {{bug|816634}} & {{bug|794901}}. | ||
** {{done|Reduce regression windows for tree bustage by switching off unused builds/tests to reduce coalescing}} | |||
** {{done|Reduce TBPL & OrangeFactor load on BzAPI/b.m.o, by updating use of BzAPI & transitioning to keywords rather than whiteboard annotations}} | |||
** {{done|Fix our test-harness detection of crashes on Android}} | |||
** {{done|Make it easier to differentiate open/resolved/test-disabled bugs on OrangeFactor, to improve prioritisation as well as finding potential backport candidates}} | |||
** {{done|Track down intermittent-failures missing from OrangeFactor}} | |||
** {{done|Make the tree-closure hook fail closed if treestatus is down}} | |||
** {{done|Increase the SnR of dev.tree-management regression emails by suppressing small percentage changes & cleaning up the subject lines}} | |||
* '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.''' | * '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.''' | ||
** {{ok|Re-organize and improve the automated testing documentation at MDN}} | ** {{ok|Re-organize and improve the automated testing documentation at MDN}} | ||