TestEngineering/Performance/Triage Process

Workflow

• Triage incoming bugs as early as possible. To be able to do that there should be at least one person who is watching the component for all changes. The triage team decides who is responsible for that task until the next triage meeting.
• Only investigate an intermittent failure if it happened more than once. Otherwise glimpse over the failure details, and if incomplete information has been added as the first comment, add the relevant part of the log as a new comment. If it’s a duplicate bug mark it as such, or if not related to the component move it immediately to the correct one. Intermittent failures should have a priority of P5 by default, unless they need investigation and a fix immediately. Then set a priority of P2 and find an owner.
• On Monday the triage owner goes through all the bugs that got updated by the intermittent failures bot. If there is a top-occurring failure make sure to assign the bug to someone familiar with the affected code.
• If it is not clear how to proceed on the bug, or if further input is necessary from stakeholders, add the whiteboard entry [perftest:triage]. Those bugs will be discussed in the next triage meeting.
• Bugs without a priority set should move to P3 by default, which means it will be fixed at some point. Only set P2 if the bug blocks current OKRs.
• Check regularly for mentored bugs, and set needinfo if there wasn’t a reply from the contributor for more than a week. Never set a person as assignee, given that this will be done by Phabricator itself when the initial patch gets submitted. Reset the assignee and set the bug to new if no further response comes in within a week.
• When you start working on a bug, set its priority to P1 and assign yourself.
• When you stop working on a bug or when it is blocked by another one, reset the priority to its original value, and unassign yourself.

Priorities

• P1 - This bug represents an OKR or an important intermittent failure, and has an assignee working on an implementation
• P2 - This bug represents an OKR or an important intermittent failure, but no-one is working on it at the moment
• P3 - This bug will be fixed eventually (non-OKR, mentoring)
• P4 - Not used (reserved for bots)
• P5 - Used for intermittent failures, or no intention to fix but will accept patches

FAQ

How to handle intermittent bugs which cover a crash of Firefox?

First make sure it doesn't stay in the Testing component but gets moved to a product that covers Firefox. Therefore check the crashing thread and find the first stack which is part of our code. Search for the associated file name in searchfox, and find the corresponding Bugzilla component within the nearest moz.build file. If not done yet, also add a comment with the link to the exact crash location. Make sure to keep the changeset id in the URL.