Talk:ReleaseEngineering/Archive/Android Tegras: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Created page with "= what bhearsum did to get a bunch of tegras online = I attacked the pool of broken/dead Tegras this week, and with a lot of guidance from Callek, managed to get a bunch of them...")
 
Line 1: Line 1:
= what bhearsum did to get a bunch of tegras online =
= what bhearsum did to get a bunch of tegras online =


I attacked the pool of broken/dead Tegras this week, and with a lot of
I attacked the pool of broken/dead Tegras this week, and with a lot of guidance from Callek, managed to get a bunch of them back online. The Tegra wiki page (https://wiki.mozilla.org/ReleaseEngineering:How_To:Android_Tegras) isn't very helpful in terms of mapping "state it's in" -> "action i need to take", so I wanted to let you all know the simple steps I took:
guidance from Callek, managed to get a bunch of them back online. The
Tegra wiki page
(https://wiki.mozilla.org/ReleaseEngineering:How_To:Android_Tegras)
isn't very helpful in terms of mapping "state it's in" -> "action i need
to take", so I wanted to let you all know the simple steps I took:


1) Look for Tegras with alerts on
# Look for Tegras with alerts on http://admin1.infra.scl1.mozilla.com/nagios/cgi-bin/status.cgi?host=all&servicestatustypes=28&hoststatustypes=15&serviceprops=270346
http://admin1.infra.scl1.mozilla.com/nagios/cgi-bin/status.cgi?host=all&servicestatustypes=28&hoststatustypes=15&serviceprops=270346
# Find the Tegra on http://mobile-dashboard1.build.mtv1.mozilla.com/tegras/
2) Find the Tegra on http://mobile-dashboard1.build.mtv1.mozilla.com/tegras/
# Click the "tegra-xxx" link to find the Tegra's Buildbot page
3) Click the "tegra-xxx" link to find the Tegra's Buildbot page
# If the Tegra is offline, try to PDU reboot it (https://wiki.mozilla.org/ReleaseEngineering:How_To:Android_Tegras#Reboot_a_Tegra_through_the_PDU)
4) If the Tegra is offline, try to PDU reboot it
# If the Tegra isn't online 30min later, open its bug and block it against the "tegra-recovery" bug. You may need to file one or both of these.
(https://wiki.mozilla.org/ReleaseEngineering:How_To:Android_Tegras#Reboot_a_Tegra_through_the_PDU)
# If that doesn't work, talk to Callek ;).
5) If the Tegra isn't online 30min later, open its bug and block it
against the "tegra-recovery" bug. You may need to file one or both of these.
6) If that doesn't work, talk to Callek ;).


And that's it! More than half the Tegras I touched came back online
And that's it! More than half the Tegras I touched came back online either through the PDU reboot or IT recovery. I implore you all to keep on top of Tegras when you're on buildduty - it's not as hard as it seems.
either through the PDU reboot or IT recovery. I implore you all to keep
on top of Tegras when you're on buildduty - it's not as hard as it seems.

Revision as of 19:20, 30 August 2012

what bhearsum did to get a bunch of tegras online

I attacked the pool of broken/dead Tegras this week, and with a lot of guidance from Callek, managed to get a bunch of them back online. The Tegra wiki page (https://wiki.mozilla.org/ReleaseEngineering:How_To:Android_Tegras) isn't very helpful in terms of mapping "state it's in" -> "action i need to take", so I wanted to let you all know the simple steps I took:

  1. Look for Tegras with alerts on http://admin1.infra.scl1.mozilla.com/nagios/cgi-bin/status.cgi?host=all&servicestatustypes=28&hoststatustypes=15&serviceprops=270346
  2. Find the Tegra on http://mobile-dashboard1.build.mtv1.mozilla.com/tegras/
  3. Click the "tegra-xxx" link to find the Tegra's Buildbot page
  4. If the Tegra is offline, try to PDU reboot it (https://wiki.mozilla.org/ReleaseEngineering:How_To:Android_Tegras#Reboot_a_Tegra_through_the_PDU)
  5. If the Tegra isn't online 30min later, open its bug and block it against the "tegra-recovery" bug. You may need to file one or both of these.
  6. If that doesn't work, talk to Callek ;).

And that's it! More than half the Tegras I touched came back online either through the PDU reboot or IT recovery. I implore you all to keep on top of Tegras when you're on buildduty - it's not as hard as it seems.