QA/Fennec/Waverley/Postmortem/Fx5

Firefox Waverley/QA Postmortem

The purpose of the meeting is to identify any areas where we can improve our QA process for future releases, with Firefox 5 as an example. We'll focus on communication, testing approach and mechanics, and any other feedback you would like to provide.

1. Who: QA/Waverley Leads
2. What: What worked, what needs improvement, suggestions for changes that might make us more efficient

Communication

• What went well? What could improve?
  • Waverley: Desktop :
    • Week's highlights on etherpad are no longer being set
  • Waverley: Fennec :
    • Wen well : Task for each build/channel and the adaptability to the new rapid process
    • Communication can be improved
• Are we communicating frequently enough?
  • Waverley:Desktop
    • We could set another short meeting to discus specific issues (either by telephone or on the IRC)
  • Waverley: Fennec :
    • We consider that we could talk more, almost daily when a major task is given
• Was turn-round time for resolving issues or questions adequate?
  • Waverley:Desktop
    • Yes - IRC helpful with this (ie: When Litmus went down it was easy and fast to get in touch with tomcat to solve problem)
  • Waverley: Fennec :
• Are the topics we discuss in meetings the right set of things?
  • Waverley:Desktop
    • No - more focus on specific issues (what is blocking the work, what are the plans for the next period, what was discussed in other meetings and is of interest for everybody)
  • Waverley: Fennec :
    • We encountered some issues regarding teh questions posted - unanswered ones or the follow up did not arrived
• Are the media we are using appropriate for the kind of information we share? (wiki, etherpads, irc, ...)
  • Waverley:Desktop
    • Yes - wiki pages need to be updated more often (ie: most of the feature pages haven't been updated since they were created)
  • Waverley: Fennec :
    • we consider that they are appropriate

Planning

• What went well? What could improve?
  • Waverley:Desktop
    • Try to avoid 2 releases at the same time
• Are tasks well defined and in the right places when you need them?
  • Waverley:Desktop
    • Overall yes - wiki pages need more organizing and clean-up so as to have the information more centralized
• Are assignments clear?
  • Waverley:Desktop
    • Yes - more pressure on devs to set flags for Firefox 6,7... when they upload patches so as for us not to leak bugs in the bug verification process
    • More organized wiki pages could lead to clearer assignments

Testing Approach and Mechanics

• What went well? What could improve?
  • Waverley: Desktop
    • We should improve the way in which attention is drawn to issues that are validated from the unconfirmed bug triage and from our own exploratory testing
• Are we focusing on the right tasks per channel and milestone?
  • Waverley: Desktop
    • More emphasis on new features especially now that their number is increasing
    • Bug verification should be performed on all channels on which the patch has landed
• Which tasks have the most impact? Which have lesser impact?
  • Waverley: Desktop
    • Less impact: Updating testcases in Litmus and vetting results - because less and less people are using Litmus in their testing activities
    • Greater impact: Work on new features
• What should we focus on for the next releases?
  • Waverley: Desktop
    • Thorough sign-off on features
    • Offer more support for x64 builds from the QA point of view