Confirmed users
3,990
edits
Auto-tools/Projects/Pulse actions: Difference between revisions
m
minor
(added image) |
m (minor) |
||
| (8 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
* Source code: https://github.com/ | = Description = | ||
Pulse Actions has two python workers listening to pulse messages from Treeherder in order to take actions | |||
for sheriffs (from the TH UI) and based on jobs failing. | |||
In order to disable any of the workers: | |||
* Add --dry-run to the appropiate worker on [https://github.com/mozilla/pulse_actions/blob/master/Procfile Procfile] | |||
* Push the change to the 'heroku' branch (not master) | |||
* The change will be autodeployed | |||
= Info = | |||
* Source code: https://github.com/mozilla/pulse_actions | |||
* Running on: Heroku | * Running on: Heroku | ||
* Logs on: https://papertrailapp.com/systems/pulse-actions/events | * Logs on: | ||
* Papertrail alerts sent to armenzg - https://papertrailapp.com/alerts | ** For corp access -> https://papertrailapp.com/systems/pulse-actions/events | ||
** | ** For communit access: | ||
*** https://dashboard.heroku.com/apps/pulse-actions/resources | |||
*** Click on PaperTrail icon | |||
= Reverting a deployment = | |||
* Visit the [https://dashboard.heroku.com/apps/pulse-actions/activity|activity] page. | |||
* Choose the revision you want to roll back to and click "Rollback to here" | |||
= Test locally = | |||
* Setup your Pulse credentials | |||
** If you don't have a pulse client visit [https://pulseguardian.mozilla.org/register|PulseGuardian] and create a user | |||
** export PULSE_USER | |||
** export PULSE_PW | |||
* We're going to be listening for the pulse stream of the staging version of Treeherder | |||
** Ask someone in #treeherder to make your account be of a sheriff role (since only sheriffs can trigger actions) | |||
* Start one of the workers which listens to the specified pulse exchanges and routing keys | |||
** valid topics are the keys in [https://github.com/mozilla/pulse_actions/blob/master/pulse_actions/run_time_config.json run_time_config.json] | |||
** the ones ending with "stage" will listen to TH's staging version | |||
** Use '''--dry-run''' to avoid actually scheduling anything | |||
python pulse_actions/worker.py --topic-base topic-stage --dry-run | |||
= Alerts = | |||
* Papertrail alerts sent to armenzg and others - https://papertrailapp.com/alerts | |||
** [https://papertrailapp.com/groups/853883/events?q=pulse-actions.*mem Memory issues] | |||
** [https://papertrailapp.com/groups/1671233/events?q=program%3Aapp%2Fworker1.1+AND+%28+Traceback+OR+Exception+OR+Error%29 Worker1: Errors] | |||
** [https://papertrailapp.com/groups/1671233/events?q=heroku%2Fworker1.1+%22to+crashed%22 Worker1: crashed] | |||
** [https://papertrailapp.com/groups/1671233/events?q=program%3Aapp%2Fworker2.1+AND+%28+Traceback+OR+Exception+OR+Error%29 Worker2: Errors] | |||
** [https://papertrailapp.com/groups/1671233/events?q=heroku%2Fworker2.1+%22to+crashed%22 Worker2: crashed] | |||
= Accounts = | |||
* Papertrail: | |||
** For employees, you can request a corp account to all Papertrail apps | |||
** For contributors, speak with armenzg to add you to the Papertrail add-on | |||
* Pulse: | |||
** Visit pulse guardian and create your credentials | |||
= Credentials = | = Credentials = | ||
| Line 17: | Line 61: | ||
= Workflow = | = Workflow = | ||
[[File:Pulse actions.png]] | [[File:Pulse actions.png]] | ||
This workflow was created with Gliffy: http://www.gliffy.com/go/publish/8686299 | |||
= Support = | = Support = | ||
| Line 23: | Line 69: | ||
* Team alias address: auto-tools | * Team alias address: auto-tools | ||
* Brief description: | * Brief description: | ||
** Pulse_actions listens to various [https://github.com/ | ** Pulse_actions listens to various [https://github.com/mozilla/pulse_actions/blob/master/pulse_actions/run_time_config.json pulse exchanges] and takes various scheduling actions through BuildApi. | ||
* What to do in case of issues: Send an email to auto-tools and ask someone to stop it until point of contact can look into it. | * What to do in case of issues: Send an email to auto-tools and ask someone to stop it until point of contact can look into it. | ||
* Impact: If you disable pulse_actions: | * Impact: If you disable pulse_actions: | ||
** sheriffs will not be able to backfill jobs (and other actions) from Treeherder | ** sheriffs will not be able to backfill jobs (and other actions) from Treeherder | ||
*** Re-opening trees would take longer | *** Re-opening trees would take longer | ||
** | ** developers/sheriffs would not be able to add missing jobs to pushes | ||
*** | |||
= Taskcluster scopes = | |||
In the future, we will be using TaskCluster to schedule tasks. | |||
These are the scopes which I have needed in the past: | |||
* queue:create-task:* | |||
* queue:define-task:* | |||
* queue:route:* | |||
* scheduler:create-task-graph | |||
* scheduler:extend-task-graph | |||
Scopes needed because of tasks already defined in-tree: | |||
* docker-worker:cache:* | |||
* docker-worker:capability:* | |||
* docker-worker:image:* | |||