QA/StaffMeetings/QA-Staff-05-23-2012: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(27 intermediate revisions by 10 users not shown)
Line 28: Line 28:


== [https://www.google.com/calendar/embed?src=j41a8ne525ml0heaipg0idb6is@group.calendar.google.com Travel / PTO] ==
== [https://www.google.com/calendar/embed?src=j41a8ne525ml0heaipg0idb6is@group.calendar.google.com Travel / PTO] ==
* item
* Geo on PTO 5/25 and 5/29
* Tracy PTO 5/26 to 6/3


== QA Team Updates  ==
== QA Team Updates  ==
Line 34: Line 35:
=== Desktop (Juan)  ===
=== Desktop (Juan)  ===
* Firefox
* Firefox
**  
** Firefox 13 beta 5 is being built today
* ESR (ashughes)
* ESR (ashughes)
**
** 10.0.5esr is getting close, please help with verifications using the nightlies (ping ashughes if you want to help)
* Stability (marcia)
* Stability (marcia)
** [https://wiki.mozilla.org/Platform/2012-05-22#Desktop Stability Report for this week]
** [https://wiki.mozilla.org/Platform/2012-05-22#Desktop Stability Report for this week]
** Significant spike in the trunk on Saturday after a JS landing. This resulted in over 20K in crashes for users of the trunk. The bug has been fixed and things should look better in crash stats in the next few days
** Significant spike in the trunk on Saturday after a JS landing. This resulted in over 20K in crashes for users of the trunk. The bug has been fixed and things should look better in crash stats in the next few days
* Automation (ashughes)
* Automation (ashughes)
**
** Currently working to identify priorities to improve coverage
* Sync Client (Tracy)
* Sync Client (Tracy)
**
** Sync exposed an OOM crasher ({{bug|756549}} - Sync updates cause explosive memory usage leading to OOM) caused by the fix in {{bug|723350}} - Simplify gcMallocBytes accounting
** Currently investigating {{Bug|757646}} - usernames/passwords disappear after Sync
*** That fix was backed out and confirmed to resolve the crashing.
* App WebRT (David/Jason/Aaron)
* App WebRT (David/Jason/Aaron)
** What happened this week?
** What happened this week?
***
*** On-going testing support for desktop web runtime
*** Working on getting traction on Google Mobile Web Compatibility Issues
*** Finished automation specification for desktop web runtime
*** Began drafting test plan for AITC desktop [https://etherpad.mozilla.org/aitc-manual-testing here]
** What's next?
** What's next?
***
***
Line 54: Line 60:
=== Mobile (Tony) ===
=== Mobile (Tony) ===
* Highlights
* Highlights
**  
** Fennec: trending down bugs fixes, beta 3 this week, testday brought a few new contributors
** B2G: builds close to daily generation, feature testplanning started on 3 top apps, automation tests being added to CI
* Fennec (Kevin/Aaron)
* Fennec (Kevin/Aaron)
**
** Help us [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=ALL%20status%3ARESOLVED%20resolution%3AFIXED%20blocking-fennec1.0%3Abeta%2C%2B%2C%253F%2Csoft%2CbetaN%252B;list_id=3165197 Verify Beta bugs!]
** FF14 Beta 3 this week.  Test, test, test!
** Testday results still need to be posted.  significant contributors: fredy, peeves leading the pack with bugs filed and verifications.
** 58 open release blockers; 330 closed vs. 63 open a week ago with 243 closed. Our close rate is healthy, but we need to get the incoming rate of blockers down more significantly. 
** Meeting today on dicussion of Fennec Native Tablet support in release branch
** Stability (nhirata)
** Stability (nhirata)
*** see platform and mobile meeting for the week.
*** see platform and mobile meeting for the week.
* B2G (Geo/jhammink)
* B2G (Geo/jhammink)
**
**Vetting new build process (already up on blog, with some comments:  http://johnhammink.blogspot.com ) plan is to get it on MDN; to community.  Problem is that building for emulator doesn't work on 12.04:  https://github.com/mozilla-b2g/B2G/issues/13
**Team still vetting daily builds
**John breaking down Gaia features into separate test logs
**Tony filed several new bugs, including blocker on NexusS/SGS2
**Collaborating with Telefonica/Massimo Barone's team to maintain list of open bugs: http://shrt.st/2f5g for triage
**Geo inventorizing web apis/doing gap analysis. Has begun working directly w/ dev to determine test planning requirements.
**Mochi + JS-style Marionette entering CI within next week, QA automation effort kickstarting with that.
 
* Pancake (nhirata)
* Pancake (nhirata)
** Pancake to have drawers changed for the ui to make it more intuitive.
** Pancake to have drawers changed for the ui to make it more intuitive.
Line 66: Line 84:
** Pancake progressing
** Pancake progressing
* Community Members Onboarding
* Community Members Onboarding
** item
** a few contributors from testday helped with bug verifications and filing.  fredy, peeves, and 2 others
** b2g getting tons of interest on learning to flash devices.  john adding notes to MDN this week


=== Services (jbonacci) ===
=== Services (jbonacci) ===
Line 90: Line 109:
* Affiliates
* Affiliates
** project is in maintenance mode until we do a UX update later in the summer
** project is in maintenance mode until we do a UX update later in the summer
* AMO
* AMO/Marketplace
**
** [http://cl.ly/1D2e1T2T2M3w27200M0z 2012-05-24] is scheduled for 05/24
* BrowserID
* BrowserID
**
** [https://github.com/mozilla/BrowserID-Tests/pull/13 Pull request] out there for new-user registrations
* Engagement Projects
* Engagement Projects
** [https://make-dev.mozillalabs.com/en-US/events/about/summer_campaign/ webmaker] released this week
** [https://make-dev.mozillalabs.com/en-US/events/about/summer_campaign/ webmaker] released this week
** [https://reps.mozilla.org/ ReMo 2.5] release went out today [5/23]
** [https://reps.mozilla.org/ ReMo 2.5] release went out today [5/23]
* Flightdeck
**
* Mozilla.com
* Mozilla.com
** shipped 2.6
** shipped 2.6
* Mozillians
* Mozillians
** Phonebook - currently in code freeze. Fixing db issues on dev/stage/prod
** Phonebook - currently in code freeze. Fixing db issues on dev/stage/prod
** Taskboard update to QA
** [https://wiki.mozilla.org/Mozillians/TaskBoard Taskboard] update to QA
*** the plan is for a simplified version of craigslist
*** the initial design calls for a simplified implementation of Craigslist
*** please add you user stories/features here - https://etherpad.mozilla.org/mozillians-taskboard-user-stories
*** please add you user stories/features here - https://etherpad.mozilla.org/mozillians-taskboard-user-stories
* MozTrap
* MozTrap
Line 118: Line 135:
** Continuous Deployment; smooth sailing
** Continuous Deployment; smooth sailing
* Community Members Onboarding
* Community Members Onboarding
** item
** Having Aditya look at our contributor flow; no new contributors in the past week


=== Community (Marcia)  ===
=== Community (Marcia)  ===
Line 131: Line 148:
**
**
* Community Members Onboarding
* Community Members Onboarding
** item
** Working on onboarding two community members found through contribute list


=== [[Auto-tools/Automation_Development/Meetings|Automation Development (Henrik, Dave)]] ===
=== [[Auto-tools/Automation_Development/Meetings|Automation Development (Henrik, Dave)]] ===
Line 157: Line 174:
== Basecamp & Kilimanjaro Roundtable/Discussion Items ==
== Basecamp & Kilimanjaro Roundtable/Discussion Items ==
* Basecamp & k9o Discussion Items
* Basecamp & k9o Discussion Items
** Cutting back thrashing, and focusing more on getting the job done [jsmith]
** [https://wiki.mozilla.org/Apps/newnewtab newnewtab] (k9o P1 requirement - new tab integration for apps) - Need QA owners
** Coming up with a better way for us to work jointly together for k9o - organization [jsmith]
*** Many meetings getting scheduled - Do we really need them?
*** Can everyone identify how they tie into k9o and basecamp?
*** How can we get the larger QA org to work together better for k9o and basecamp?
** k9o Test Cases in MozTrap
** k9o [https://etherpad.mozilla.org/kilimanjaro-qa-questions Open Questions]
* Team Activity
* Team Activity
** Reminder - please take a moment to look at and add your comments to the homework section of https://etherpad.mozilla.org/QA-Requirements-For-Base-Camp
** Reminder - please take a moment to look at and add your comments to the homework section of https://etherpad.mozilla.org/QA-Requirements-For-Base-Camp
Line 163: Line 186:
== Other Roundtable/Discussion Items  ==
== Other Roundtable/Discussion Items  ==
* QA Hiring Update - Managers? [marcia]
* QA Hiring Update - Managers? [marcia]
* Rapid Betas - we've identified getting "sufficient" automation coverage as a long pole for QA (ashughes)
** keeping in mind that k9o, basecamp, b2g, fennec native are all higher priority then rapid beta...
** what is our minimum comfort level of automation coverage, where are we today, how much work will it take to get there, and what's a reasonable timeline for success
** NOTE: It's okay if this can't be answered right now, but I'd like to be able to deliver a snapshot to Release Management on Friday if possible
** NOTE: Identifying and closing our coverage gaps will improve Nightly, Aurora, and Beta; irrespective of whether we do Rapid Betas or not
* Softvision bi-monthly meetings attendance has been sporatic from Mozilla (tchung)
** affects webQA, automation, mobile, desktop.  [https://wiki.mozilla.org/QA/SoftVision/Meetings/2012-05-22 Notes] have been empty from mozilla's end.


== Notes, Takeaways and Action Items  ==
== Notes, Takeaways and Action Items  ==

Latest revision as of 21:19, 23 May 2012


« previous week | index | next week »

QA Staff Meeting Notes May-23-2012

Meeting Details

 # Every Weds @ 1:30p Pacific Time - Mozilla HQ, Warp Core 
 #
 # Vidyo:  https://v.mozilla.com/flex.html?roomdirect.html&key=UK1zyrd7Vhym
 # Phone:  +1 650 903 0800 or 650-215-1282 x92 Conf# 95312 (US/INTL)
 #         +1 800 707 2533 (pin 369) Conf# 95312 (US Toll Free)
 #         +1 416 848 3114 x92 Conf# 95312 (Canada)
 # IRC:    irc://irc.mozilla.org:6697/#qa

QA Org Items

  • New Hires & Interviews
    • Aditya Jalgaonkar joined Web QA on Monday, May 21, mentored by Krupa
      • Working on all things AMO and Mozilla Marketplace -- welcome!
    • item
  • Upcoming Events/Conferences
  • Holiday reminder: Monday, May 28 is a US holiday

Travel / PTO

  • Geo on PTO 5/25 and 5/29
  • Tracy PTO 5/26 to 6/3

QA Team Updates

Desktop (Juan)

  • Firefox
    • Firefox 13 beta 5 is being built today
  • ESR (ashughes)
    • 10.0.5esr is getting close, please help with verifications using the nightlies (ping ashughes if you want to help)
  • Stability (marcia)
    • Stability Report for this week
    • Significant spike in the trunk on Saturday after a JS landing. This resulted in over 20K in crashes for users of the trunk. The bug has been fixed and things should look better in crash stats in the next few days
  • Automation (ashughes)
    • Currently working to identify priorities to improve coverage
  • Sync Client (Tracy)
    • Sync exposed an OOM crasher (bug 756549 - Sync updates cause explosive memory usage leading to OOM) caused by the fix in bug 723350 - Simplify gcMallocBytes accounting
    • Currently investigating bug 757646 - usernames/passwords disappear after Sync
      • That fix was backed out and confirmed to resolve the crashing.
  • App WebRT (David/Jason/Aaron)
    • What happened this week?
      • On-going testing support for desktop web runtime
      • Working on getting traction on Google Mobile Web Compatibility Issues
      • Finished automation specification for desktop web runtime
      • Began drafting test plan for AITC desktop here
    • What's next?
  • Community Members Onboarding
    • item

Mobile (Tony)

  • Highlights
    • Fennec: trending down bugs fixes, beta 3 this week, testday brought a few new contributors
    • B2G: builds close to daily generation, feature testplanning started on 3 top apps, automation tests being added to CI
  • Fennec (Kevin/Aaron)
    • Help us Verify Beta bugs!
    • FF14 Beta 3 this week. Test, test, test!
    • Testday results still need to be posted. significant contributors: fredy, peeves leading the pack with bugs filed and verifications.
    • 58 open release blockers; 330 closed vs. 63 open a week ago with 243 closed. Our close rate is healthy, but we need to get the incoming rate of blockers down more significantly.
    • Meeting today on dicussion of Fennec Native Tablet support in release branch
    • Stability (nhirata)
      • see platform and mobile meeting for the week.
  • B2G (Geo/jhammink)
    • Vetting new build process (already up on blog, with some comments: http://johnhammink.blogspot.com ) plan is to get it on MDN; to community. Problem is that building for emulator doesn't work on 12.04: https://github.com/mozilla-b2g/B2G/issues/13
    • Team still vetting daily builds
    • John breaking down Gaia features into separate test logs
    • Tony filed several new bugs, including blocker on NexusS/SGS2
    • Collaborating with Telefonica/Massimo Barone's team to maintain list of open bugs: http://shrt.st/2f5g for triage
    • Geo inventorizing web apis/doing gap analysis. Has begun working directly w/ dev to determine test planning requirements.
    • Mochi + JS-style Marionette entering CI within next week, QA automation effort kickstarting with that.
  • Pancake (nhirata)
    • Pancake to have drawers changed for the ui to make it more intuitive.
    • Need for automation; need to get in touch with webqa
    • Pancake progressing
  • Community Members Onboarding
    • a few contributors from testday helped with bug verifications and filing. fredy, peeves, and 2 others
    • b2g getting tons of interest on learning to flash devices. john adding notes to MDN this week

Services (jbonacci)

  • Sync Server (jbonacci/jrgm): Sync Server
    • Working with Richard S. on switching PHX1 and SCL2 Sync storage API from memcached to couchbase
    • Working with Ryan, Toby, and Tarek on AITC 1.1 in Stage
    • Working with Ryan and Rob on Sync 1.1 plus MetLog support
    • Working through some QA docs for AITC and TokenServer
    • Need to start digging into HSM
      • Bug 756664 - Metabug for HSM/signing service transition
  • BrowserID (jbonacci/jrgm): BrowserID
    • BrowserID train-2012.05.14
    • Testing of Hot Fix for Issue 2592
    • Several prod firedrills last 10 days, including push and test of emergency bug fix
    • Q2 test planning for the following: Updated API, UI flow, RP, Keywrapping, Persona Beta, BigTent, VinzClortho (Mozilla.com), KPI
    • QA Automation: https://wiki.mozilla.org/Identity/QA#Automation
  • Community Members Onboarding
    • None at this time
  • Hiring and Training
    • Resume evals and phone screens are in progress

Web (Stephen)

  • Affiliates
    • project is in maintenance mode until we do a UX update later in the summer
  • AMO/Marketplace
  • BrowserID
  • Engagement Projects
  • Mozilla.com
    • shipped 2.6
  • Mozillians
  • MozTrap
    • Automated tests are being converted to webdriver
    • We are mapping out the Selenium test strategy for the project, following the SUMO team example of a few crucial tests including user flows and tests unable to be covered by unit tests.
  • MDN
  • Socorro
    • Milestone 10 scheduled to release today
    • Bug 550538 - Provide easily-accessible list of URLs-for-signature to logged-in users
    • Bug 726385 - Replace 8 digit hex addresses with a placeholder within Java signatures
  • SUMO
    • Continuous Deployment; smooth sailing
  • Community Members Onboarding
    • Having Aditya look at our contributor flow; no new contributors in the past week

Community (Marcia)

  • Upcoming Events
  • Testdays (ashughes)
  • Video Work
  • QMO.next (Raymond):
  • Community Members Onboarding
    • Working on onboarding two community members found through contribute list

Automation Development (Henrik, Dave)

General

  • Automation Development team work week is currently happen in London
  • Mozdownload has been released - Grab it from pypi or by running 'pip install mozdownload'
    • With it you can download all versions of Firefox across platforms and specific dates
    • Dave will blog about it so we can promote it as tool for regression testing

Projects

  • Mozmill CI
    • Further improvements for stability - we handle build notifications correctly now
MozTrap (Cameron)
Community Members Onboarding
  • item

Basecamp & Kilimanjaro Roundtable/Discussion Items

  • Basecamp & k9o Discussion Items
    • newnewtab (k9o P1 requirement - new tab integration for apps) - Need QA owners
    • Coming up with a better way for us to work jointly together for k9o - organization [jsmith]
      • Many meetings getting scheduled - Do we really need them?
      • Can everyone identify how they tie into k9o and basecamp?
      • How can we get the larger QA org to work together better for k9o and basecamp?
    • k9o Test Cases in MozTrap
    • k9o Open Questions
  • Team Activity

Other Roundtable/Discussion Items

  • QA Hiring Update - Managers? [marcia]
  • Rapid Betas - we've identified getting "sufficient" automation coverage as a long pole for QA (ashughes)
    • keeping in mind that k9o, basecamp, b2g, fennec native are all higher priority then rapid beta...
    • what is our minimum comfort level of automation coverage, where are we today, how much work will it take to get there, and what's a reasonable timeline for success
    • NOTE: It's okay if this can't be answered right now, but I'd like to be able to deliver a snapshot to Release Management on Friday if possible
    • NOTE: Identifying and closing our coverage gaps will improve Nightly, Aurora, and Beta; irrespective of whether we do Rapid Betas or not
  • Softvision bi-monthly meetings attendance has been sporatic from Mozilla (tchung)
    • affects webQA, automation, mobile, desktop. Notes have been empty from mozilla's end.

Notes, Takeaways and Action Items

Today

  • Add notes here

Last Meeting

  • Action Items
Retrieved from "https://wiki.allizom.org/index.php?title=QA/StaffMeetings/QA-Staff-05-23-2012&oldid=433931"