QA/Embedded Teams: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
No edit summary
No edit summary
Line 3: Line 3:
# Provide more focus and development more technical acumen on QA
# Provide more focus and development more technical acumen on QA
# Identify strategic areas where involvement results in better overall quality across the board to best use the small number of people we have
# Identify strategic areas where involvement results in better overall quality across the board to best use the small number of people we have
== Embedded Areas ==


The areas where we are working are:
The areas where we are working are:
Line 11: Line 13:
* [[QA/Fennec| Firefox for Android (AaronMT & Kbrosnan)]]
* [[QA/Fennec| Firefox for Android (AaronMT & Kbrosnan)]]
* Overall QA Releases, Quality Programs, and Risk: Kairo & Mschifer
* Overall QA Releases, Quality Programs, and Risk: Kairo & Mschifer
== Non-Embedded Areas ==


There are of course other areas that we cannot cover to the degree we would like to and currently have no one embedded in these teams:
There are of course other areas that we cannot cover to the degree we would like to and currently have no one embedded in these teams:
Line 19: Line 23:
* JavaScript
* JavaScript


== The Hot List ==
As a result there are things that will fall through the cracks. We are working quite hard to analyze the risk of everything that falls through the cracks and ensure that risk is mitigated through some means (contributor, contractor, staff, and/or automation). We will maintain the below list for things that are currently in this zone.
As a result there are things that will fall through the cracks. We are working quite hard to analyze the risk of everything that falls through the cracks and ensure that risk is mitigated through some means (contributor, contractor, staff, and/or automation). We will maintain the below list for things that are currently in this zone.
* Reading List (desktop)
{|class="fullwidth-table" border="1"
* Password Manager (desktop)
|-
* ios product
| style="font-weight: bold; background: #DDD; width: 20%" | Project
* windows 64 bit builds
| style="font-weight: bold; background: #DDD; width: 10%" | Quality Risk
* geolocation changes
| style="font-weight: bold; background: #DDD; width: 10%" | Schedule
* addon signing
| style="font-weight: bold; background: #DDD; width: 10%" | Contributor Friendliness
* network predictor landing
| style="font-weight: bold; background: #DDD; width: 50%" | Next Step/Notes
|-
| Reading List (desktop)
| Medium
| 38?
| 0
|
* Needs assessment
* Create project wiki page
* Analyze deliverables and scope
|-
| Password Manager (desktop)
| High
| ?
| 0
|
* Needs assessment
* Marked high because for years our password system (master pass/no master pass) have caused untold headache with implementations of these kinds of things
* Create project wiki page
* Analyze deliverables and scope
|-
| ios product
| High
| ?
| -1
|
* Marked high because we are building our own browser UX for this effort
* Marked negative contributor friendly because of difficulty of beta/pre-release programs available to iOS users
* Marked negative due to requirement for devices
* Analyze how well and how much relevant QA can be done on simulators
* Create plan
|-
| windows 64 bit builds
| low
| 37/38
| 0
|
* We have excellent automation here and have had nightlies run on 64 bit builds for some time
* Our existing update automation will still work
* We might be able to do this entirely automated
* Could ship in 37 or 38 depending on where the demand lies for this
|-
| geolocation service changes
| medium
| ?
| 0
|
* Need to create the wiki page for contribution this is a prime target for contributor support because we need people in many locations testing the accuracy of the new geo system
* Medium quality risk only because there are a fair number of location dependent web sites on mobile systems that will depend on this feature working (though if we are on a mobile device do we use the device's GPS, short circuiting the problem? TODO: Investigate this
|-
| addon signing
| medium
| ?
| 0
|
* Need to create contributor wiki page for involvement
* Should be imminently testable and crowd-source-able.
* There is a long tail of addons, but we can divide and conquer by considering classes of addons
|-
| network predictor
| high
| -1
|
* This was identified late in the release cycle as a problem last time it landed
* We need eyes on this sooner rather than later to ensure that it does not regress performance
* The speed with which we need to act coupled with the low level nature of this system makes it not as attractive an option for contributor review
* Need to create test plan wiki
|}

Revision as of 14:21, 28 January 2015

For the core Firefox and Platform QA teams we are using an embedding strategy where we have embedded QA engineers alongside engineers in specific functional teams. We are doing this to:

  1. Improve the scope and depth of quality engineering in these areas
  2. Provide more focus and development more technical acumen on QA
  3. Identify strategic areas where involvement results in better overall quality across the board to best use the small number of people we have

Embedded Areas

The areas where we are working are:

Non-Embedded Areas

There are of course other areas that we cannot cover to the degree we would like to and currently have no one embedded in these teams:

  • Graphics
  • Firefox Front-end
  • Developer Tools
  • Networking
  • JavaScript

The Hot List

As a result there are things that will fall through the cracks. We are working quite hard to analyze the risk of everything that falls through the cracks and ensure that risk is mitigated through some means (contributor, contractor, staff, and/or automation). We will maintain the below list for things that are currently in this zone.

Project Quality Risk Schedule Contributor Friendliness Next Step/Notes
Reading List (desktop) Medium 38? 0
  • Needs assessment
  • Create project wiki page
  • Analyze deliverables and scope
Password Manager (desktop) High ? 0
  • Needs assessment
  • Marked high because for years our password system (master pass/no master pass) have caused untold headache with implementations of these kinds of things
  • Create project wiki page
  • Analyze deliverables and scope
ios product High ? -1
  • Marked high because we are building our own browser UX for this effort
  • Marked negative contributor friendly because of difficulty of beta/pre-release programs available to iOS users
  • Marked negative due to requirement for devices
  • Analyze how well and how much relevant QA can be done on simulators
  • Create plan
windows 64 bit builds low 37/38 0
  • We have excellent automation here and have had nightlies run on 64 bit builds for some time
  • Our existing update automation will still work
  • We might be able to do this entirely automated
  • Could ship in 37 or 38 depending on where the demand lies for this
geolocation service changes medium ? 0
  • Need to create the wiki page for contribution this is a prime target for contributor support because we need people in many locations testing the accuracy of the new geo system
  • Medium quality risk only because there are a fair number of location dependent web sites on mobile systems that will depend on this feature working (though if we are on a mobile device do we use the device's GPS, short circuiting the problem? TODO: Investigate this
addon signing medium ? 0
  • Need to create contributor wiki page for involvement
  • Should be imminently testable and crowd-source-able.
  • There is a long tail of addons, but we can divide and conquer by considering classes of addons
network predictor high -1
  • This was identified late in the release cycle as a problem last time it landed
  • We need eyes on this sooner rather than later to ensure that it does not regress performance
  • The speed with which we need to act coupled with the low level nature of this system makes it not as attractive an option for contributor review
  • Need to create test plan wiki