canmove, Confirmed users
2,041
edits
B2G/QA/Automation/UI/Strategy/Integration vs End to end: Difference between revisions
B2G/QA/Automation/UI/Strategy/Integration vs End to end (view source)
Revision as of 03:43, 11 February 2015
, 11 February 2015→Summary
| Line 11: | Line 11: | ||
== Summary == | == Summary == | ||
=== The Problem === | |||
To date, UI automation has been treated as a single type of testing, with the differences expressed largely as to whether it was running on TBPL or not, was written in JS or Python, who sheriffed, and so forth. And so much discussion, debate, and analysis paralysis has involved whether the automation should be run, written, or treated a particular way. It has been all too common to reach an impasse based on differing opinions. | To date, UI automation has been treated as a single type of testing, with the differences expressed largely as to whether it was running on TBPL or not, was written in JS or Python, who sheriffed, and so forth. And so much discussion, debate, and analysis paralysis has involved whether the automation should be run, written, or treated a particular way. It has been all too common to reach an impasse based on differing opinions. | ||
| Line 25: | Line 27: | ||
However, these are weaknesses in perception. When used for their own specific purposes, each type of automation provides great value. Further, the compromises each makes are compensated for by the other. | However, these are weaknesses in perception. When used for their own specific purposes, each type of automation provides great value. Further, the compromises each makes are compensated for by the other. | ||
=== The Solution === | |||
The solution is to separate the concerns. And since each type has a different primary stakeholder, ownership should be separated as well. | The solution is to separate the concerns. And since each type has a different primary stakeholder, ownership should be separated as well. | ||
| Line 47: | Line 51: | ||
** Must be sheriffed by QA, via a combination of alerts and result reviews | ** Must be sheriffed by QA, via a combination of alerts and result reviews | ||
** Primarily runs on-device to test full stack | ** Primarily runs on-device to test full stack | ||
=== Different Contexts === | |||
The main differences are: | The main differences are: | ||
| Line 65: | Line 71: | ||
* It is more important that UATs be complete than solid. All acceptance criteria must be tested to accept a build. | * It is more important that UATs be complete than solid. All acceptance criteria must be tested to accept a build. | ||
Ownership is separated because these differences can lead to variant needs in terms of breadth and depth of verification. In addition, tests can and should overlap between CI and UAT as it is unacceptable for QA to expect developers to always consult their needs and vice versa. Of course, each group should have an opinion on coverage for either suite, and can (and should) help expand each suite, but single point of ownership allows decisions to be made quickly as appropriate for each set of primary stakeholders. | === Ownership and Overlap === | ||
Ownership is separated because these differences can lead to variant needs in terms of breadth and depth of verification. | |||
In addition, tests can and should overlap between CI and UAT as it is unacceptable for QA to expect developers to always consult their needs and vice versa. | |||
Of course, each group should have an opinion on coverage for either suite, and can (and should) help expand each suite, but single point of ownership allows decisions to be made quickly as appropriate for each set of primary stakeholders. | |||
Via skillful code reuse, each suite can be treated as an independent target without increasing maintenance unacceptably. | Via skillful code reuse, each suite can be treated as an independent target without increasing maintenance unacceptably. | ||