View source for Build:StageMigrationNotes

<h1>Stage Migration Notes</h1>

= Current Configuration =

Currently, contributors and Tinderbox machines upload directly to <tt>surf</tt> (CNAMEed to stage.m.o). <tt>manna</tt> (CNAMEd to ftp.m.o) pulls these builds from surf via rsync at set intervals.

<tt>surf</tt> is responsible for virus scanning and verifying other build-related issues.

Both surf and manna are constantly dangerously low on diskspace; the goals of this migration are to:

* Solve this continuing disk space problem
* Ensure that bits are virus scanned and otherwise verified before reaching the mirrors, which is currently not the case. Infected bits will eventually get removed from ftp.m.o/mirrors, but it's not instantaneous.

= Migration Plan =

On the evening of Thursday, 14 September, from 6 pm to midnight PDT:

# surf/stage will be moved inside the firewall and become responsible for virus scanning and verifying all bits that go to the mirror farm/ftp.m.o. Because of this, access will be restricted to this host.
# surf/stage will have a read/write mount to shared, central storage.
# manna will be renamed stage (as a CNAME), and will mount this shared, central storage read-only. This shared, central storage will become the new definitive copy of ftp.m.o.
# Some tinderboxen will change to publish their builds to surf
# Bits in the archive.m.o rsync module will move entirely to the shared, central storage.
# Bits in the release.m.o rsync module will continue to exist on surf and the shared central storage
# surf will pull bits onto its local disk via a scheduled process and perform various verification steps on them (virus scanning, permissions checks, etc.); when these checks pass, they will be uploaded to the shared, central storage, and immediately appear on the ftp mirrors/ftp.m.o.

= How These Changes Affect External Contributors =

* External contributors may have to accept a new SSH host key; this key will be published after the switchover is complete, so contributors can verify it before accepting it
* Access to surf will be limited
* As part of this plan, we will be auditing access to manna; if access is removed, but it shouldn't have been, please email build@mozilla.org.
* Builds may be spotty on Friday; we'll do our best to ensure the builds get uploaded as expected.

= Process =

This is a rough, ordered timeline of the steps necessary to fulfill the requirements.

{| class="fullwidth-table"
! Step
! Build
! IT
|-
| Prerequ.
| 
* Initial list of rsync directories from surf -&gt; netapp
|
* Netapp mounts available to surf (r/w)
* Netapp mounts available to manna (r/o)
* LDAP up and running on both for authentication?
* Reduce TTL on stage.m.o in anticipation of switchover
|-
| 1
|
|
* Move surf inside firewall
* Connect Netapp mount to surf
|-
| 2
|
* rsync the contents of surf -&gt; the netapp
|
* Setup/verify as appropriate the http/rsync/ftp configs on manna; test with builds that begin to be available (it may make sense to make stage.m.o/ftp.m.o completely unavailable to mirrors/outsiders at this point, since we don't want the mirrors picking up incomplete directory tree)
|-
| 3
|
* Delete the contents of the archive rsync module on surf; delete ftp logs on surf
|
* Verify that ftp.m.o/stage.m.o appear as before, but accessed via manna (can diff manna's local disk vs. the Netapp to verify)
|-
| 4
|
* Point Tinderboxen at surf
|
* Start manna-upload-area -&gt; surf rsync scripts on surf
|-
| 5
|
* Verify builds in the build farm get to surf
* Verify builds outside the build farm can get to manna's upload area
|
* Setup virus scanning script
* Start surf -&gt; Netapp rsync scripts
|-
| 6
|
* Verify end-to-end tinderbox -&gt; ftp.m.o status
|
* Verify end-to-end tinderbox -&gt; ftp.m.o status
|}