B2G/QA/New Hire Doc: Difference between revisions
Jump to navigation
Jump to search
| (7 intermediate revisions by 5 users not shown) | |||
| Line 7: | Line 7: | ||
;QA Lab: ''There is a QA lab in Mountain View on the second floor (right next to the kitchen) that has a variety of hardware that we test with. All members of the QA team automatically have badged access to the room, but if you are a new hire you may still need to request access.'' | ;QA Lab: ''There is a QA lab in Mountain View on the second floor (right next to the kitchen) that has a variety of hardware that we test with. All members of the QA team automatically have badged access to the room, but if you are a new hire you may still need to request access.'' | ||
;Tinderbox Build: | ;Tinderbox Build: | ||
;[https://wiki.mozilla.org/B2G/Triage#Blocker_Triage_Guidelines Triage]: ''Bug triage is a process in which bugs are prioritized to be fixed. The bugs are sorted into categories like critical and non-critical to manage the inward flow of issues and to make sure the proper issues are being worked on at a given time.'' | |||
;[https://bugzilla.mozilla.org/ Bugzilla]: ''Mozilla’s own bug tracking software. We dogfood this application and use it to keep track of all of our issues and sometimes to manage projects as well. If the issue needs to be visible, it needs to be in Bugzilla.'' | ;[https://bugzilla.mozilla.org/ Bugzilla]: ''Mozilla’s own bug tracking software. We dogfood this application and use it to keep track of all of our issues and sometimes to manage projects as well. If the issue needs to be visible, it needs to be in Bugzilla.'' | ||
*;QA Wanted: ''This is a flag used in Bugzilla to have QA perform a branch check (they will specify if they want to check something older than the latest stable).'' | *;QA Wanted: ''This is a flag used in Bugzilla to have QA perform a branch check (they will specify if they want to check something older than the latest stable).'' | ||
*;VerifyMe: ''This flag is used in Bugzilla to have QA verify that a patch truly fixed the issue. These tasks are usually given to contractors.'' | *;VerifyMe: ''This flag is used in Bugzilla to have QA verify that a patch truly fixed the issue. These tasks are usually given to contractors.'' | ||
*;Nomination: Action of flagging a bug in order to make it triaged. On Bugzilla, it's usually defined with the question mark like in blocking-b2g:2.2?. | |||
*;Blocker: A blocker is a bug where critical functionality is broken to the point that we cannot release the product. The release is therefore "Blocked". On Buzilla, with the Firefox OS project, a blocker is defined with blocking-b2g and the + sign as in blocking-b2g:2.2+ | |||
;[https://wiki.mozilla.org/Sheriffing/TBPL TBPL]: A tool, called Tinderboxpushlog, that correlates checkins with Buildbot results so that developers and sheriffs can quickly see the results of automatic builds and tests for specific change sets. TBPL has been replaced by Treeherder. | ;[https://wiki.mozilla.org/Sheriffing/TBPL TBPL]: A tool, called Tinderboxpushlog, that correlates checkins with Buildbot results so that developers and sheriffs can quickly see the results of automatic builds and tests for specific change sets. TBPL has been replaced by Treeherder. | ||
;[https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Gaia Gaia]: ''The user interface layer of the Firefox OS Platform. Any part of the user interface that is drawn on the screen is done | ;[https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Gaia Gaia]: ''The user interface layer of the Firefox OS Platform. Any part of the user interface that is drawn on the screen is done through the Gaia. Gaia is implemented entirely in HTML, CSS and JavaScript. It interfaces with the underlying operating system through open Web APIs, which are implemented through the Gecko layer.'' | ||
;[https://developer.mozilla.org/en-US/docs/Gecko Gecko]: The Firefox OS application runtime. This is the layer that provides all of the support for HTML, CSS and | ;[https://developer.mozilla.org/en-US/docs/Gecko Gecko]: The Firefox OS application runtime. This is the layer that provides all of the support for HTML, CSS and JavaScript. It makes sure that those APIs work on every OS that Gecko supports. Gecko many layers including, but not limited to, a networking stack, graphics stack, layout engine, a JavaScript virtual machine and porting layers. | ||
;[https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Gonk Gonk]: ''Gonk is the lower level operating system of the Firefox OS platform. It consists of a Linux kernel (based on the [http://source.android.com/ Android Open Source Project]) and a userspace hardware abstraction layer (HAL). Basically, one could say that Gonk is a very simple Linux distribution. Gonk is a '''porting target''' of Gecko.'' | ;[https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Gonk Gonk]: ''Gonk is the lower level operating system of the Firefox OS platform. It consists of a Linux kernel (based on the [http://source.android.com/ Android Open Source Project]) and a userspace hardware abstraction layer (HAL). Basically, one could say that Gonk is a very simple Linux distribution. Gonk is a '''porting target''' of Gecko.'' | ||
;[https://wiki.mozilla.org/B2G B2G]: B2G is short for Boot To Gecko which is a project a complete, standalone operating system in build for the Web. It is built entirely using web technologies (HTML/CSS/JavaScript). | ;[https://wiki.mozilla.org/B2G B2G]: B2G is short for Boot To Gecko which is a project a complete, standalone operating system in build for the Web. It is built entirely using web technologies (HTML/CSS/JavaScript). | ||
| Line 19: | Line 21: | ||
;Nexus 5:''This is going to be the second generation reference device for Firefox OS. More to come.'' | ;Nexus 5:''This is going to be the second generation reference device for Firefox OS. More to come.'' | ||
;[https://developer.mozilla.org/en-US/Firefox_OS/Platform/Architecture Firefox OS]: ''Mozilla’s offering for a mobile operating system. It is built completely on web technologies creating an innovative ecosystem for developers and gives users a choice.'' | ;[https://developer.mozilla.org/en-US/Firefox_OS/Platform/Architecture Firefox OS]: ''Mozilla’s offering for a mobile operating system. It is built completely on web technologies creating an innovative ecosystem for developers and gives users a choice.'' | ||
;[https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette Marionette]: ''Marionette is an automation driver based on Selenium WebDriver for Mozilla's Gecko engine. | ;[https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette Marionette]: ''Marionette is an automation driver based on Selenium WebDriver for Mozilla's Gecko engine. It relies on 2 parts: the server (included in Gecko) and one client (either in [http://marionette-client.readthedocs.org/ Python] or in [https://github.com/mozilla-b2g/marionette-js-client/ JavaScript])'' | ||
;Buildbot: ''See [https://intranet.mozilla.org/QA:NewHireDocument New Hire Document For QA]'' | ;Buildbot: ''See [https://intranet.mozilla.org/QA:NewHireDocument New Hire Document For QA]'' | ||
;Sheriffing: | ;Sheriffing: | ||
;[https://wiki.mozilla.org/Auto-tools/Projects/Treeherder Tree Herder]: ''Treeherder is the successor to TBPL. The purpose of Treeherder is to support longer term use-cases for managing and visualizing data and rewrite the existing application from scratch, so that it can scale appropriately in the future. It was created as a way to address some pain points with TBPL.'' | ;[https://wiki.mozilla.org/Auto-tools/Projects/Treeherder Tree Herder]: ''Treeherder is the successor to TBPL. The purpose of Treeherder is to support longer term use-cases for managing and visualizing data and rewrite the existing application from scratch, so that it can scale appropriately in the future. It was created as a way to address some pain points with TBPL.'' | ||
;[https://github.com/try-server-hook Gaia-try]: A robot which is in charge of creating jobs on Treeherder against Pull requests on the Gaia repo. | |||
;Task Cluster: | ;Task Cluster: | ||
;Mozilla Central: | ;[https://hg.mozilla.org/mozilla-central/ Mozilla Central]: ''The main (a.k.a. trunk, a.k.a. master) branch of Firefox Desktop and Gecko.'' | ||
;Jenkins: The current on-device test runner. All the other tests are run on Treeherder. | |||
;Jenkins: | |||
;Image Compare: | ;Image Compare: | ||
;Wiki: ''Wiki is the type of markup tool we use to post documentation. There is a [http://en.wikiversity.org/wiki/Wiki_101 quick tutorial] that one can use to become familiar with it. For some Mozilla specific information about Wiki, see this [https://wiki.mozilla.org/Help:Editing tutorial].'' | ;Wiki: ''Wiki is the type of markup tool we use to post documentation. There is a [http://en.wikiversity.org/wiki/Wiki_101 quick tutorial] that one can use to become familiar with it. For some Mozilla specific information about Wiki, see this [https://wiki.mozilla.org/Help:Editing tutorial].'' | ||
| Line 35: | Line 35: | ||
== Glossary == | == Glossary == | ||
;Kernel Panic | ;[http://en.wikipedia.org/wiki/Kernel_panic Kernel Panic] | ||
;...On Device: This means that whatever action is being referenced is being performed on a physical Flame device. | ;...On Device: This means that whatever action is being referenced is being performed on a physical Flame device. | ||
;...On Desktop: This means that whatever action is being referenced is being performed on B2G Desktop or Web IDE. This would not be appropriate for performance or hardware specific issues since those don't have access to the hardware of the phone. | ;...On Desktop: This means that whatever action is being referenced is being performed on B2G Desktop or Web IDE. This would not be appropriate for performance or hardware specific issues since those don't have access to the hardware of the phone. | ||
;G.I.P: Gaia Integration Python tests are the automation tests that are written in Python. We, as a group, are migrating away from the Python testing suite to use in favor of GIJ. This way, both the applications and the tests are using the same tech and will have better support. | ;G.I.P: Gaia Integration Python tests are the automation tests that are written in Python. We, as a group, are migrating away from the Python testing suite to use in favor of GIJ. This way, both the applications and the tests are using the same tech and will have better support. | ||
;G.I.J: | ;G.I.J: Gaia Integration Javascript. | ||
;Shallow Flash: This process means you are flashing the Gecko and Gaia layers only. It assumes that all drivers and libraries for the hardware are in place and up to date. | ;Shallow Flash: This process means you are flashing the Gecko and Gaia layers only. It assumes that all drivers and libraries for the hardware are in place and up to date. | ||
;Full Flash': This process means that you are flashing all layers | ;Full Flash': This process means that you are flashing all layers | ||
;QAnalyst: | ;QAnalyst: | ||
;Regression Window: The timeframe between the last successful build and when it broke. The smaller the window, the better for assessing what went wrong. | ;Regression Window: The timeframe between the last successful build and when it broke. The smaller the window, the better for assessing what went wrong. | ||
;Smoke Test: | ;Smoke Test: A test suite (automated and manual) run everyday to check the basic features of the product | ||
;Nightly: | ;Nightly: A build that happens daily, usually overnight. | ||
;Functional Team: | ;Functional Team: | ||
; | ;[https://wiki.mozilla.org/Release_Management/FirefoxOS/Release_Milestones Milestones (Release)]: | ||
;Feature Landing | ;[https://wiki.mozilla.org/Release_Management/FirefoxOS/Release_Milestones#FEATURE_LANDING_.28FL.29 Feature Landing] | ||
;Feature Complete | ;[https://wiki.mozilla.org/Release_Management/FirefoxOS/Release_Milestones#FEATURE_COMPLETE_.28FC.29 Feature Complete] | ||
;Dogfood: When you hear the term "dogfooding", it is the process of using a product that you created. So, in terms of Firefox OS QA, someone would use a Flame device as their own personal phone. Thus, they are testing the phone in the realest sense of the term. | ;Dogfood: When you hear the term "dogfooding", it is the process of using a product that you created. So, in terms of Firefox OS QA, someone would use a Flame device as their own personal phone. Thus, they are testing the phone in the realest sense of the term. | ||
== Communication Channels == | |||
* IRC | |||
** #fxosqa | |||
** #fxos | |||
* Telegram App (Download and install from https://telegram.org/) | |||
** Firefox OS QA | |||
** Quality Assurance & Mozillians | |||
== Important Mailing Lists == | |||
Sign up for these to stay up-to-date with FFOS! | |||
* b2g-internal | |||
* qa-b2g-internal | |||
* CD_allstaff | |||
* dev-fxos | |||
* b2g-release-drivers | |||
* fxosqa-report-watchlist | |||
== Reading List == | |||
* Testing Computer Software (Second Edition) | |||
* | |||
Latest revision as of 06:50, 27 January 2016
New Hire Documentation for Firefox OS QA
This document was created as a supplement to the New Hire Document For QA
Tools and Processes
- QA Lab
- There is a QA lab in Mountain View on the second floor (right next to the kitchen) that has a variety of hardware that we test with. All members of the QA team automatically have badged access to the room, but if you are a new hire you may still need to request access.
- Tinderbox Build
- Triage
- Bug triage is a process in which bugs are prioritized to be fixed. The bugs are sorted into categories like critical and non-critical to manage the inward flow of issues and to make sure the proper issues are being worked on at a given time.
- Bugzilla
- Mozilla’s own bug tracking software. We dogfood this application and use it to keep track of all of our issues and sometimes to manage projects as well. If the issue needs to be visible, it needs to be in Bugzilla.
- QA Wanted
- This is a flag used in Bugzilla to have QA perform a branch check (they will specify if they want to check something older than the latest stable).
- VerifyMe
- This flag is used in Bugzilla to have QA verify that a patch truly fixed the issue. These tasks are usually given to contractors.
- Nomination
- Action of flagging a bug in order to make it triaged. On Bugzilla, it's usually defined with the question mark like in blocking-b2g:2.2?.
- Blocker
- A blocker is a bug where critical functionality is broken to the point that we cannot release the product. The release is therefore "Blocked". On Buzilla, with the Firefox OS project, a blocker is defined with blocking-b2g and the + sign as in blocking-b2g:2.2+
- TBPL
- A tool, called Tinderboxpushlog, that correlates checkins with Buildbot results so that developers and sheriffs can quickly see the results of automatic builds and tests for specific change sets. TBPL has been replaced by Treeherder.
- Gaia
- The user interface layer of the Firefox OS Platform. Any part of the user interface that is drawn on the screen is done through the Gaia. Gaia is implemented entirely in HTML, CSS and JavaScript. It interfaces with the underlying operating system through open Web APIs, which are implemented through the Gecko layer.
- Gecko
- The Firefox OS application runtime. This is the layer that provides all of the support for HTML, CSS and JavaScript. It makes sure that those APIs work on every OS that Gecko supports. Gecko many layers including, but not limited to, a networking stack, graphics stack, layout engine, a JavaScript virtual machine and porting layers.
- Gonk
- Gonk is the lower level operating system of the Firefox OS platform. It consists of a Linux kernel (based on the Android Open Source Project) and a userspace hardware abstraction layer (HAL). Basically, one could say that Gonk is a very simple Linux distribution. Gonk is a porting target of Gecko.
- B2G
- B2G is short for Boot To Gecko which is a project a complete, standalone operating system in build for the Web. It is built entirely using web technologies (HTML/CSS/JavaScript).
- Flame
- This is the first generation reference device for Firefox OS.
- Nexus 5
- This is going to be the second generation reference device for Firefox OS. More to come.
- Firefox OS
- Mozilla’s offering for a mobile operating system. It is built completely on web technologies creating an innovative ecosystem for developers and gives users a choice.
- Marionette
- Marionette is an automation driver based on Selenium WebDriver for Mozilla's Gecko engine. It relies on 2 parts: the server (included in Gecko) and one client (either in Python or in JavaScript)
- Buildbot
- See New Hire Document For QA
- Sheriffing
- Tree Herder
- Treeherder is the successor to TBPL. The purpose of Treeherder is to support longer term use-cases for managing and visualizing data and rewrite the existing application from scratch, so that it can scale appropriately in the future. It was created as a way to address some pain points with TBPL.
- Gaia-try
- A robot which is in charge of creating jobs on Treeherder against Pull requests on the Gaia repo.
- Task Cluster
- Mozilla Central
- The main (a.k.a. trunk, a.k.a. master) branch of Firefox Desktop and Gecko.
- Jenkins
- The current on-device test runner. All the other tests are run on Treeherder.
- Image Compare
- Wiki
- Wiki is the type of markup tool we use to post documentation. There is a quick tutorial that one can use to become familiar with it. For some Mozilla specific information about Wiki, see this tutorial.
Glossary
- Kernel Panic
- ...On Device
- This means that whatever action is being referenced is being performed on a physical Flame device.
- ...On Desktop
- This means that whatever action is being referenced is being performed on B2G Desktop or Web IDE. This would not be appropriate for performance or hardware specific issues since those don't have access to the hardware of the phone.
- G.I.P
- Gaia Integration Python tests are the automation tests that are written in Python. We, as a group, are migrating away from the Python testing suite to use in favor of GIJ. This way, both the applications and the tests are using the same tech and will have better support.
- G.I.J
- Gaia Integration Javascript.
- Shallow Flash
- This process means you are flashing the Gecko and Gaia layers only. It assumes that all drivers and libraries for the hardware are in place and up to date.
- Full Flash'
- This process means that you are flashing all layers
- QAnalyst
- Regression Window
- The timeframe between the last successful build and when it broke. The smaller the window, the better for assessing what went wrong.
- Smoke Test
- A test suite (automated and manual) run everyday to check the basic features of the product
- Nightly
- A build that happens daily, usually overnight.
- Functional Team
- Milestones (Release)
- Feature Landing
- Feature Complete
- Dogfood
- When you hear the term "dogfooding", it is the process of using a product that you created. So, in terms of Firefox OS QA, someone would use a Flame device as their own personal phone. Thus, they are testing the phone in the realest sense of the term.
Communication Channels
- IRC
- #fxosqa
- #fxos
- Telegram App (Download and install from https://telegram.org/)
- Firefox OS QA
- Quality Assurance & Mozillians
Important Mailing Lists
Sign up for these to stay up-to-date with FFOS!
- b2g-internal
- qa-b2g-internal
- CD_allstaff
- dev-fxos
- b2g-release-drivers
- fxosqa-report-watchlist
Reading List
- Testing Computer Software (Second Edition)