|
|
Line 247: |
Line 247: |
|
| |
|
| = Throttling = | | = Throttling = |
| AUS2 supports two types of throttling, which are applied in this order of precedence
| | The throttle rate is number from 0 to 100, used in the sense of a percentage a gas pedal is pushed down. So |
| * app-version-channel specific
| |
| * global
| |
| | |
| We don't use the global throttle, so this document will not talk about it specifically.
| |
| | |
| In both cases, an update query which carries the parameter force=1, eg
| |
| https://aus3.mozilla.org/Firefox/.../update.xml?force=1
| |
| will not be throttled. Checks which are initiated by the user have this parameter, while background checks (initiated by the app itself) do not.
| |
| | |
| In both cases the throttle is number from 0 to 100, used in the sense of a percentage a gas pedal is pushed down. So
| |
| * 0 means stopped, no requests will be served an update | | * 0 means stopped, no requests will be served an update |
| * 100 means full speed, all requests will be served an update | | * 100 means full speed, all requests will be served an update |
| * 25 means quarter speed, 1 in 4 requests will be served an update (on average!) | | * 25 means quarter speed, 1 in 4 requests will be served an update (on average!) |
|
| |
|
| These are configured in [http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/webtools/aus/xml/inc/config-dist.php&rev=AUS2_PRODUCTION&cvsroot=/cvsroot config-dist.php]. The machines that serve AUS symlink/copy config-dist.php to config.php.
| | Update queries which carry the parameter force=1, eg: |
| | | https://aus3.mozilla.org/Firefox/.../update.xml?force=1 |
| == Configuring Throttling ==
| | will not be throttled. Checks which are initiated by the user have this parameter, while background checks (initiated by the app itself) do not. |
| App-version-channel throttling allows you to configure a throttle level for an application and version, and to set exceptions for channels that it should not apply to. The version comes from the request, rather than the update served for that request.
| |
| | |
| eg, Updates from Firefox 3.6.17 are turned off on the release channel, but not beta or either test channel
| |
| $productThrottling = array(
| |
| 'Firefox' => array(
| |
| '3.6.17' => 0
| |
| )
| |
| );
| |
| $throttleExceptions = array(
| |
| '3.6.17' => array (
| |
| 'betatest',
| |
| 'releasetest',
| |
| 'beta'
| |
| )
| |
| );
| |
| | |
| We always add exceptions for the two test channels, so that QA and RelEng can test updates without any randomness.
| |
|
| |
|
| == Do I need to throttle? == | | == Do I need to throttle? == |
| Generally, there's two different ways we do throttling for releases. Most of the time we ship a release partly throttled (10% served), throttle it completely 48 hours later (at the request of RelMan through release-drivers and/or a bug), once some amount of users have it in their hands. In certain cases (for example, when our release day coincides with a Microsoft patch Tuesday), we ship a release fully throttled (0% of requests are served an update). | | Generally, there's two different ways we do throttling for releases. Most of the time we ship a release partly throttled (10% served), throttle it completely 48 hours later (at the request of RelMan through release-drivers and/or a bug), once some amount of users have it in their hands. In certain cases (for example, when our release day coincides with a Microsoft patch Tuesday), we ship a release fully throttled (0% of requests are served an update). |
|
| |
|
| '''In both cases you must make throttling changes before pushing the snippets.''' | | '''In both cases you must make throttling changes before making the updates live.'' |
|
| |
|
| We're usually told explicity "please push release xxxx fully throttled" or "please push release xxxx partly throttled", but if you're unclear about whether a release should be throttled or not, ask on release-drivers prior to pushing snippets. If you're confused about anything else, ask another RelEng person. | | We're usually told explicity "please push release xxxx fully throttled" or "please push release xxxx partly throttled", but if you're unclear about whether a release should be throttled or not, ask on release-drivers prior making the updates live. If you're confused about anything else, ask another RelEng person. |
|
| |
|
| == How to throttle == | | == How to throttle == |
| Throttling is a multi step process that involves: | | Throttling is an attribute on each rule in Balrog. To change the throttle rate, simply adjust the "Rate" on the appropriate rule and save the changes: |
| # [[ Release:Release_Automation_on_Mercurial:Updates_through_Shipping#Configuring_Throttling | Changing the AUS config ]] (and getting review on it)
| | [[File:Throttling in Balrog.png|upright]] |
| # Moving the AUS2_PRODUCTION tag on config-dist.php
| |
| #* move tag '''only''' on config-dist.php: "<tt>cvs tag -F AUS2_PRODUCTION inc/config-dist.php</tt>"
| |
| # Asking IT to deploy the new code ([https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=server-ops-webops%40mozilla-org.bugs&bug_severity=blocker&bug_status=NEW&comment=fyi%2C%20instructions%20at%3A%20https%3A%2F%2Fmana.mozilla.org%2Fwiki%2Fdisplay%2Fwebsites%2Faus2.mozilla.org&component=WebOps%3A%20Product%20Delivery&form_name=enter_bug&product=Infrastructure%20%26%20Operations&rep_platform=All&short_desc=Please%20update%20AUS%20to%20AUS2_PRODUCTION%20tag%20%28REV_NUMBER%29&version=other bug template]).
| |
| #* include the new revision number of config-dist.php (''REV_NUMBER in template'' IT uses a [https://mana.mozilla.org/wiki/display/websites/aus2.mozilla.org script] that takes that rev number as input.) from: "<tt> cvs status -v inc/config-dist.php | grep AUS2_PRODUCTION</tt>"
| |
| # When the change is deployed, send email to r-d summarizing the change.
| |
| | |
| For example, when we throttled to release 23.0 [https://bug901780.bugzilla.mozilla.org/attachment.cgi?id=786107 this patch] was used. After review it was committed to the CVS repository, and the AUS2_PRODUCTION tag was moved to the new revision. Not everyone has commit access, ask aki, bhearsum, Callek, catlee, coop, kmoir, nthomas, or rail for help. {{Bug|901913}} was filed for IT to deploy the code. Note that we file such deployment bugs as blockers, otherwise they won't happen quickly enough.
| |
|
| |
|
| == Verifying Throttling == | | == Verifying Throttling == |