ReleaseEngineering/How To/Restart Buildbot Masters: Difference between revisions

→‎By script: Update for bug 1340925
(Add filtered view for logs)
(→‎By script: Update for bug 1340925)
Line 24: Line 24:
Here is an example invocation:
Here is an example invocation:


  # buildduty@buildduty-tools
  # ssh -A buildduty@buildduty-tools
  $ screen -R restart_masters
  $ screen -R restart_masters
  $ /home/buildduty/restart_masters.sh
  $ /home/buildduty/restart_masters.sh
You can enter nonsense for the cltbld and root passwords because key auth is used. Forwarding your ssh agent is required for ssh  access to the masters. If you use a timeout make sure it's sufficiently long for the script to repeat, but don't leave it running indefinitely. '''NB: rebooting masters by adding the -r arg to restart_masters.py is non-functional because root logins are disabled.'''


The logs go to [https://papertrailapp.com/systems/255918104/events?q=restart_masters papertrail], outputting a progress report every 60 minutes. For really minimal view (which may hide errors) see this [https://papertrailapp.com/systems/255918104/events?q=restart_masters+-%22Connection+as+cltbld%22+-%22Attempting+to+connect+to%22+-%22Checking+shutdown+status%22+-lockfile+-Disabling+-Initiating+-%22Re-enabling%22+-%22Attempting+to+restart+master%22 filtered view].  
The logs go to [https://papertrailapp.com/systems/255918104/events?q=restart_masters papertrail], outputting a progress report every 60 minutes. For really minimal view (which may hide errors) see this [https://papertrailapp.com/systems/255918104/events?q=restart_masters+-%22Connection+as+cltbld%22+-%22Attempting+to+connect+to%22+-%22Checking+shutdown+status%22+-lockfile+-Disabling+-Initiating+-%22Re-enabling%22+-%22Attempting+to+restart+master%22 filtered view].  
Confirmed users
4,293

edits