193
edits
Release Management/Nightly Respin: Difference between revisions
Adding WIP mobile android process for relman
Diannasmith (talk | contribs) (Adding WIP mobile android process for relman) |
|||
| Line 14: | Line 14: | ||
== Summarized process == | == Summarized process == | ||
===Desktop=== | |||
# File a bug with as much detail as possible about the regression / crash if a bug hasn't been filed yet | # File a bug with as much detail as possible about the regression / crash if a bug hasn't been filed yet | ||
# Ask releng to stop automatic nightly updates because of the bug filed in step 1 | # Ask releng to stop automatic nightly updates because of the bug filed in step 1 | ||
| Line 22: | Line 23: | ||
# Ask the patch author to investigate the regression (NeedInfo in Bugzilla) | # Ask the patch author to investigate the regression (NeedInfo in Bugzilla) | ||
# When updates are back announce that the fix is served on Twitter and IRC | # When updates are back announce that the fix is served on Twitter and IRC | ||
===Mobile=== | |||
# Find the regression as soon as possible and notify the right party about the situation | |||
# Identify the affected population | |||
# Identify best method to limit the affected population from the unstable build | |||
# Warn our users about the regression via Matrix (#Fenix or #Nightly channel) | |||
# Ask the patch author or the Mobile team to investigate the regression (NeedInfo in Bugzilla or Tag on Github) | |||
# When updates are back announce that the fix is available | |||
== When should we back out a patch? == | == When should we back out a patch? == | ||
===Desktop and Mobile=== | |||
We want to back out a patch when a significant regression is identified. This is usually either a functional regression (browser unusable, content rendering broken) or a sudden spike of crashes on the Nightly channel. | We want to back out a patch when a significant regression is identified. This is usually either a functional regression (browser unusable, content rendering broken) or a sudden spike of crashes on the Nightly channel. | ||
| Line 31: | Line 41: | ||
== How to find the patch to back out == | == How to find the patch to back out == | ||
===Desktop=== | |||
If it is a functional regression (reproducible case), then we should use [https://mozilla.github.io/mozregression/quickstart.html mozregression]. If it is a spike in crashes not necessarily reproducible (random crashes while surfing), then our crash analysis experts in the Release Management team should be contacted. The analysis of the stack trace combined with hg logs on mozilla-central often allow finding the bug number that introduced the instability. | If it is a functional regression (reproducible case), then we should use [https://mozilla.github.io/mozregression/quickstart.html mozregression]. If it is a spike in crashes not necessarily reproducible (random crashes while surfing), then our crash analysis experts in the Release Management team should be contacted. The analysis of the stack trace combined with hg logs on mozilla-central often allow finding the bug number that introduced the instability. | ||
===Mobile=== | |||
TBD | |||
== Bug filing == | == Bug filing == | ||
===Desktop=== | |||
* If the bug was already filed by a community member, then use it to track the regression and qualify it. Add the nightly-community keyword if missing. | * If the bug was already filed by a community member, then use it to track the regression and qualify it. Add the nightly-community keyword if missing. | ||
* If it is a crash, get a Crash ID from the people that reported it and file a bug via Socorro. | * If it is a crash, get a Crash ID from the people that reported it and file a bug via Socorro. | ||
| Line 47: | Line 62: | ||
== Stopping automatic background updates == | == Stopping automatic background updates == | ||
===Desktop=== | |||
If you think that a lot of people are going to be impacted by a regression, ask releng or relman to stop automatic update. | If you think that a lot of people are going to be impacted by a regression, ask releng or relman to stop automatic update. | ||
| Line 57: | Line 73: | ||
Most of the major regressions are reported immediately via our @FirefoxNightly Twitter account followers, usually when more than 2 people report a similar regression there is a high chance that it will be serious and stopping automatic updates should be done rapidly. | Most of the major regressions are reported immediately via our @FirefoxNightly Twitter account followers, usually when more than 2 people report a similar regression there is a high chance that it will be serious and stopping automatic updates should be done rapidly. | ||
== Asking for a back out of the patch | ===Mobile=== | ||
If the significant regression is due to geckoview changes, the Relman team can create a new commit to "rollback" the Geckoview bump to the last stable build. | |||
If it is a mobile change that is causing a crash spike/significant regression, relman team can backout the patch. | |||
If the issue is specific to a device or OS version, the app can be limited or blocked from those devices in the Google Play Store via the device catalog. | |||
== Asking for a back out of the patch and new nightlies == | |||
===Desktop=== | |||
You can contact sheriffs in the #sheriffs IRC channel to back out the patch that caused the regression when you have identified it. The back out commit will reference the bug number. | You can contact sheriffs in the #sheriffs IRC channel to back out the patch that caused the regression when you have identified it. The back out commit will reference the bug number. | ||
| Line 64: | Line 89: | ||
'''Note:''' Some members of the release management team have the technical knowledge and permissions to back out patches. | '''Note:''' Some members of the release management team have the technical knowledge and permissions to back out patches. | ||
===Mobile=== | |||
If the next nightly is about to be triggered, it can be canceled via taskcluster. | |||
If a new build is needed, the Relman team can trigger new mobile nightly builds via a taskcluster hook once the branch is in a stable state. | |||
== Communicating about the issue == | == Communicating about the issue == | ||
====Desktop==== | |||
We should not hesitate to communicate the issue with a reference to the bug number to our community so as to minimize the number of duplicate bugs. If the issue needs steps to reproduce which are not obvious or a specific hardware/OS combination, having all communications centralized in a single bug helps. | We should not hesitate to communicate the issue with a reference to the bug number to our community so as to minimize the number of duplicate bugs. If the issue needs steps to reproduce which are not obvious or a specific hardware/OS combination, having all communications centralized in a single bug helps. | ||
| Line 77: | Line 108: | ||
When updates are stopped, this will be automatically indicated on https://whattrainisitnow.com/release/?version=nightly with the reason message (usually linking to a bug) entered by release managers or sheriffs in Balrog when they stopped updates. | When updates are stopped, this will be automatically indicated on https://whattrainisitnow.com/release/?version=nightly with the reason message (usually linking to a bug) entered by release managers or sheriffs in Balrog when they stopped updates. | ||
====Mobile==== | |||
TBD | |||
[[Category:Release_Management]] | [[Category:Release_Management]] | ||
[[Category:Release_Management:Processes|Nightly Respin]] | [[Category:Release_Management:Processes|Nightly Respin]] | ||