EngineeringProductivity/Meetings/2011-09-26

RoundTable

Let's keep notes over here as we do the discussion.

• Discussion on projects, priority and polish
  • Two phase solution - development & release
  • Having well-defined priorities
  • Having well-defined end goals for each phase of a project
  • Handling bumping projects and shifting priorities

• Outcome: What can we do (as a team) to help us with this?
• Outcome: What can Clint do (as team lead) to help with this?

Goals: These are the goals I pulled out of the "Thinking Caps" thread, plus some that I know about. The details are not necessarily complete, but listing them all helps illuminate the big picture of where we're heading.

• mobile automation
  • get from 25% failure rate to <5% rate (desktop parity that is)
  • make it easier for developers to run and debug tests with tools and hardware
    • Make targets for all tests
    • better documentation for all tests and tools
  • Investigate pandaboards for honeycomb support?

• performance
  • aid in future xperf and pageloader modifications
  • keep talos and AMO perf up to date with firefox changes (i.e. missing application.ini, etc..)
  • resolve and triage perf bugs

• specialpowers
  • finish the mochitest harness
  • ensure existing work done on tests lands (i.e. content patches from :jdm)
  • make a specialpowers-core for quit, prefs and logging. Good for talos, etc...
  • NOTE: this will make us 100% harness and about 75% test complete.

• mozbase
  • have working prototype with releng for talos, mochitest, reftest, xpcshell
  • resolve location and process for mozprocess, mozprofile, etc...
  • land any prerequisite patches and toolchains
  • NOTE: I suspect we won't get mozbase landed 100%, but we should be getting close
  • get virtualenv on m-c
  • coordiante with Release Engineering on mobase/mozharness efforts
  • get talos on mozbase packages
  • get automation.py on mozbase packages
  • update documentation
  • communicate about this!
  • continue to added needed features to mozbase packages
  • bugs are at https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=mozbase&list_id=1352142
  • [MAYBE] split up the mozmill repo
• manifests
  • mochitest-plain manifests

• MozMill:
  • get 2.0 out the door for real and support it
  • possibly help refactor mozmill-automation if desired
  • Make Mozmill e10s ready (probably working with :ahal)

• W3C browser testing work
  • TBD

• B2G
  • Get phase 1 of platform level test framework created
  • Aid with build system needs (joint with releng?)
  • Have linux agent ready to go

• Tegra Automation
  • Deploy to MV network and announce
  • Wire up with panda boards as they come available?

• SpeedTests
  • Expanding to new browsers
  • Expanding tests
  • Maintenance

• Eideticker
  • Complete test framework side
  • Investigate using another card?

• WOO
  • Shepherd through secreview

• Templeton
  • Find/fix

• bughunter
  • Greater integration with soccoro?

• User Responsiveness Regression Test
  • Complete harness for this
  • Shepherd through buildbot and/or mozharness integration

• Better community outreach
  • Have volunteers involved in community stewards effort
  • Have a blogging goal for the team?

• Better process/priority clarity for each project
  • TBD (see above)

Goals

• Work with release engineering to reduce overall end-to-end time from push to test result to an average of 2 hours.
• Ensure all harnesses and tests are E10S ready
• Improve performance testing by finalizing user-activated addons performance system, xperf-enabled performance testing, and automating a user responsiveness performance test.

Automation

• Go Faster (P1) - jgriffin, ted, mdas, bc, ctalbert, wlach
  • Some minor further improvements pushed to gofaster dashboard (see: bug 688588, bug 686034)
  • Probably easing away from further improvements to dashboard for now. Some bugs still unfixed (e.g. bug 686036, bug 687600). Thoughts?

• Mobile Support (P1) - jmaher, bmoss, ctalbert

• Bug Hunter (P1) - bc

• Mozmill 2.0/1.5.x (P1) - jhammel, harth, ctalbert
  • two bugs left!
  • will release a new beta today
  • other than that, we're in a holding pattern

• Crossweave 2 - jgriffin

Performance

• Talos/User Responsiveness (P1) - anode, ted

• Talos/Addon Testing Service (P1) - anode

• Talos/General (P1) - anode

• Talos/Xperf (P2) - jmaher

• SpeedTests (P2) - mcote
  • New results front end: http://brasstacks.mozilla.com/speedtests/results.html
  • Updating docs so that ahal can take over machine maintenance while I'm away.

• Eideticker (P2) - mcote, wlach
  • Project synopsis now on public wiki
  • Video capture prototype now up as a github project
  • Discussion of architecture issues, etc. last week
  • Going with talos for now as a harness
    • First iteration will just capture loading static pages
    • Second iteration may use more complex simulated user interactions. Some initial thoughts in this message.
  • Video capture machine (+ LG Android device with HDMI out) now at wlach's house

Web Tools

• War on Orange/Auto-log (P1) - jgriffin, mcote
  • View for Bug Correlations awaiting review.

• Bugzilla Improvements (P1) - dkl, glob
  • The decision has been made to modify the core Bugzilla code where appropriate to best suit how Mozilla uses it; this should result in faster improvements to Bugzilla with a tighter focus on our requirements
  • Tweaks to searching (comments not searched by default, google searching integrated)
  • Various bug fixes and tweaks

• Regression Hunter (P1) - harth

• DXR Automation/Support {P2} - mdas
  • Submitted automation code to jcranmer (works on DXR). When he hooks up that code, along with bug 688673 landing, then build reports should be coming into autolog on a daily basis.

• Personal Bugzilla Dashboard (P2) - harth

Tool Infrastructure

• SpecialPowers (P1) - jmaher, ted

• MozBase (P2) - jmaher, jhammel, ted

• Web UX Platform (P2) - mcote, harth
  • A couple more useful JS libraries added in.

• flyingtanks - mcote
  • WOO update complete, no security concerns.
    • Will be installing it on flyingtanks soon (but brasstacks' copy will be up until everything is reviewed & migrated).