ReleaseEngineering/PuppetAgain/Base Images: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(7 intermediate revisions by 3 users not shown)
Line 3: Line 3:
See [[ReleaseEngineering/PuppetAgain/Puppetization Process|Puppetization Process]] for how a base image becomes a running instance.
See [[ReleaseEngineering/PuppetAgain/Puppetization Process|Puppetization Process]] for how a base image becomes a running instance.


= CentOS 6.2 with Kickstart =
= CentOS 6.x with Kickstart =


The bulk of a kickstart script is available in [http://hg.mozilla.org/build/puppet/file/default/setup/ setup].  Run 'make' to create the script.  The top section will need a few site-specific modifications.  Most of the action is in the %post section, which:
The bulk of a kickstart script is available in [http://hg.mozilla.org/build/puppet/file/default/setup/ setup].  Run 'make' to create the script.  The top section will need a few site-specific modifications.  Most of the action is in the %post section, which:
Line 13: Line 13:
The same script supports both i386 and x86_64 ISAs; the determining factor is the kernel used to run the installer.
The same script supports both i386 and x86_64 ISAs; the determining factor is the kernel used to run the installer.


= Mac OS X 10.8 (Mountain Lion) =
= Ubuntu with Kickstart =
 
Ubuntu is very similar to CentOS.  While it doesn't use Anaconda, it reads a kickstart profile similar to, but not exactly like, that for CentOS.  Beyond those minor differences, it does the same things:
* looks for PUPPET_PASS on the kernel command line, and stores it in /root/deploypass
* installs some repos and minimal packages
* installs [http://hg.mozilla.org/build/puppet/file/default/setup/puppetize.sh puppetize.sh] and sets it to run on boot.
 
The same script supports both i386 and x86_64 ISAs; the determining factor is the kernel used to run the installer.
 
= Mac OS X 10.9 - Mavericks =
 
Reproducing with VMWare Fusion
 
* Download OS X Mavericks from the app store, but do not install it
* Using disk utility, create a new, 8GB DMG file partitioned as a CD/DVD
* Launch Terminal to show hidden files and relaunch the Finder using this defaults command string:
    defaults write com.apple.Finder AppleShowAllFiles TRUE;\killall Finder;\say Files Revealed
* Go to the /Applications/ folder to locate the “Install OS X 10.9 Developer Preview.app” file
* Right-click and choose “Show Package Contents”, then open “Contents” and open “SharedSupport”
* Double-click to mount “InstallESD.dmg”
* Open the mounted ‘OS X Install ESD’ image, and right-click “BaseSystem.dmg” choosing “Open” to mount the image (BaseSystem.dmg is invisible by default, thus why invisible files must be made visible in the earlier step)
* Go back to Disk Utility, then select “BaseSystem.dmg” from the sidebar and then click the “Restore” tab
* With “Source” set to “BaseSystem.dmg”, set “Destination” to the volume in the empty DMG by dragging the Untitled Volume in the blang DMG, then click “Restore” to begin – confirm that contents of the drive will be erased
* When finished, go to Finder and navigate within the newly created USB drive or dmg image to System > Installation > and delete the file (alias) located here named “Packages” – keep this window open
* Go back to the ‘OS X Install ESD’ mounted drive, and drag & drop the “Packages” folder into the /System/Installation/ directory you just deleted the Packages alias from, let this large folder copy
* Once that Packages folder is finished copying to the drive, unmount everything.
* Set up a new VM to boot from DVD, and point the DVD drive to your new image
* Follow the install steps until it begins installing and reboots.  You may have to shut down, remove the DVD, and re-start.
* Accept the defeault language and keyboard, skip logging with an Apple ID, and agree
* Create a new user named 'Administrator' with the kickstart password
** uncheck "Require password to unlock screen", "Set time zone..", and "Send Diagnostics.."
* Accept the default (Cupertino) timezone
* Don't register the system
* Wait a while, and you'll end up in the Finder
* Open System Preferences -> Sharing
** Enable:
*** Screen Sharing
*** Remote Login
* Open System Preferences -> Enery Saver
** Move Computer sleep slider to Never
** Move Disply sleep slider to Never
** Disable:
*** put hard disks to sleep when possible
*** Allow power button to put computer to sleep (doesn't seem to be present, at least in Fusion)
** Enable:
*** Wake for network access
* Shut down
* ..and capture
 
Capturing from VMWare Fusion
 
* Shut down the VM
* Duplicate the VM's .vmdk in Finder
* Add a new, existing hard disk to the VM, pointing it at the duplicate
* Boot the VM
* Fire up Disk Utility within the VM
* Click on the second "Mac OS X" hard drive
* File -> New -> Image from "Mac OS X"
 
= Mac OS X 10.8 (12A269) - Mountain Lion =


The current base image is built from installing 10.7 Lion via the recovery boot option (cmd + R) and then upgrading to 10.8 Mountain Lion through the App Store.
The current base image is built from installing 10.7 Lion via the recovery boot option (cmd + R) and then upgrading to 10.8 Mountain Lion through the App Store.
Line 32: Line 91:


* walk through the setup screen and create a default user
* walk through the setup screen and create a default user
** default user should be 'administrator' using the current releng root/admin password
** default user should be 'administrator' using the current releng root/admin kickstart password
* you can skip most of the screens such as apple account registration
* skip all of the apple account registration screens
* At the login screen, use the 'administrator' credentials that were entered earlier
* At the login screen, use the 'administrator' credentials that were entered earlier
* Launch the App store using your apple developer credentials
* Launch the App store using your apple developer credentials
Line 40: Line 99:
** Make sure to sign out of the app store once the download is complete
** Make sure to sign out of the app store once the download is complete
* Once the download is complete, launch the OS X Mountain Lion installer and proceed through the setup process
* Once the download is complete, launch the OS X Mountain Lion installer and proceed through the setup process
* Take another nap
* Once installed, the system will reboot
* Once installed, the system will reboot


Line 59: Line 117:
** Enable:
** Enable:
*** Wake for network access
*** Wake for network access
*** Start up automatically after power failure
* Capture the image for Deploy Studio
* Integrate into the workflows to image 10.8 systems


The system is ready to be captured as an image
= Mac OS X Server 10.7.2 (11C74) - Lion =
* Install OS X 10.7.2 on an r4 or r5 mini
* walk through the setup screen and create a default user
** default user should be 'administrator' using the current releng root/admin kickstart password
* skip all of the apple account registration screens
* At the login screen, use the 'administrator' credentials that were entered earlier
* Open System Preferences -> Sharing
** Enable Screen Sharing and Remote Login
* Open System Preferences -> Enery Saver
** Move Computer sleep slider to Never
** Move Display sleep slider to Never
* Capture the image for Deploy Studio
* Integrate into the workflows to image 10.7.2 systems


* Open System Preferences -> Startup Disk
= Mac OS X 10.6.8 (10K549) - Snow Leopard =
** select DSR-1074 Network Disk, click Restart
*** Confirm restart


This will reboot the system into the netboot image with the Deploy Studio runtime where Deploy Studio workflow will handle installing puppetize.sh and org.mozilla.puppetize.plist to be launched on first boot
* Install OS X 10.6.4 on an r4 mini from disk.
 
* walk through the setup screen and create a default user
= CentOS 6.2 on Ganeti KVM =
** default user should be 'administrator' using the current releng root/admin kickstart password
== Installed image ==
* skip all of the apple account registration screens
Install a minimal config with CentOS-6.2 install media:
* At the login screen, use the 'administrator' credentials that were entered earlier
 
* Open System Preferences -> Sharing
First, create a new OS type if necessary - add
** Enable Screen Sharing and Remote Login
IMAGE_NAME="centos-62"
* Open System Preferences -> Enery Saver
ARCH=x86_64
** Move Computer sleep slider to Never
IMAGE_TYPE=dump
** Move Display sleep slider to Never
CDINSTALL=no
* Apply the 10.6.8 update.
to <tt>/etc/ganeti/instance-image/variants/centos-62.conf</tt>, and add it to <tt>/etc/ganeti/instance-image/variants.list</tt>.
* Capture the image for Deploy Studio
 
* Integrate into the workflows to image 10.8.6 systems
Send it everywhere:
gnt-cluster copyfile /etc/ganeti/instance-image/variants/centos-62.conf
gnt-cluster copyfile /etc/ganeti/instance-image/variants.list
modify your test VM to use this new image type:
gnt-instance modify -o image+centos-62 relabs-buildbot-master
and then start the install.  I'm using a CD:
gnt-instance modify -H boot_order=cdrom,cdrom_image_path=/tmp/CentOS-6.2-x86_64-netinstall.iso relabs-buildbot-master
gnt-instance reboot relabs-buildbot-master
A few hints: /dev/vda1 = boot, /dev/vda2 = swap, /dev/vda3 = root; use ext3/4; *no* LVM!  Do a minimal install.
 
Halt this and dump it:
/usr/lib/ganeti/os/image/tools/make-dump relabs-buildbot-master
 
This is your installed image - it's just CentOS.
 
== Base Image ==
 
Now, to create the base image that automatically puppetizes.
 
reinstall, to get the ganeti-instance-image networking, console, etc. set up
gnt-instance reinstall relabs-buildbot-master
 
On another system, run 'make' in the setup subdirectory of http://hg.mozilla.org/build/puppet, to get a copy of base-image-centos6.sh. Copy this and paste it into /root/base-image-centos6.sh on the destination machine:
 
<pre>
cat >base-image-centos6.sh <<EOF
[[paste it here]]
EOF
sh base-image-centos6.sh
</pre>
 
After some action, and after asking for the deploy password, the system halts, ready to be imaged.
 
gnt-instance modify -o image+releng-base-centos-62 relabs-buildbot-master
/usr/lib/ganeti/os/image/tools/make-dump relabs-buildbot-master

Latest revision as of 15:37, 24 June 2014

Base images are OS/hardware-specific. In some cases, we can leverage existing installers; in others, we must use snapshots of installed systems. In either case, the idea is to get a minimal system set up to run puppetize.sh, which will take care of getting a certificate and running puppet in a loop until it succeeds.

See Puppetization Process for how a base image becomes a running instance.

CentOS 6.x with Kickstart

The bulk of a kickstart script is available in setup. Run 'make' to create the script. The top section will need a few site-specific modifications. Most of the action is in the %post section, which:

  • looks for PUPPET_PASS on the kernel command line, and stores it in /root/deploypass
  • installs minimal packages
    • Using a possibly site-specific yum-repo location (under http://repos/repos/)
  • installs puppetize.sh and sets it to run on boot.

The same script supports both i386 and x86_64 ISAs; the determining factor is the kernel used to run the installer.

Ubuntu with Kickstart

Ubuntu is very similar to CentOS. While it doesn't use Anaconda, it reads a kickstart profile similar to, but not exactly like, that for CentOS. Beyond those minor differences, it does the same things:

  • looks for PUPPET_PASS on the kernel command line, and stores it in /root/deploypass
  • installs some repos and minimal packages
  • installs puppetize.sh and sets it to run on boot.

The same script supports both i386 and x86_64 ISAs; the determining factor is the kernel used to run the installer.

Mac OS X 10.9 - Mavericks

Reproducing with VMWare Fusion

  • Download OS X Mavericks from the app store, but do not install it
  • Using disk utility, create a new, 8GB DMG file partitioned as a CD/DVD
  • Launch Terminal to show hidden files and relaunch the Finder using this defaults command string:
   defaults write com.apple.Finder AppleShowAllFiles TRUE;\killall Finder;\say Files Revealed
  • Go to the /Applications/ folder to locate the “Install OS X 10.9 Developer Preview.app” file
  • Right-click and choose “Show Package Contents”, then open “Contents” and open “SharedSupport”
  • Double-click to mount “InstallESD.dmg”
  • Open the mounted ‘OS X Install ESD’ image, and right-click “BaseSystem.dmg” choosing “Open” to mount the image (BaseSystem.dmg is invisible by default, thus why invisible files must be made visible in the earlier step)
  • Go back to Disk Utility, then select “BaseSystem.dmg” from the sidebar and then click the “Restore” tab
  • With “Source” set to “BaseSystem.dmg”, set “Destination” to the volume in the empty DMG by dragging the Untitled Volume in the blang DMG, then click “Restore” to begin – confirm that contents of the drive will be erased
  • When finished, go to Finder and navigate within the newly created USB drive or dmg image to System > Installation > and delete the file (alias) located here named “Packages” – keep this window open
  • Go back to the ‘OS X Install ESD’ mounted drive, and drag & drop the “Packages” folder into the /System/Installation/ directory you just deleted the Packages alias from, let this large folder copy
  • Once that Packages folder is finished copying to the drive, unmount everything.
  • Set up a new VM to boot from DVD, and point the DVD drive to your new image
  • Follow the install steps until it begins installing and reboots. You may have to shut down, remove the DVD, and re-start.
  • Accept the defeault language and keyboard, skip logging with an Apple ID, and agree
  • Create a new user named 'Administrator' with the kickstart password
    • uncheck "Require password to unlock screen", "Set time zone..", and "Send Diagnostics.."
  • Accept the default (Cupertino) timezone
  • Don't register the system
  • Wait a while, and you'll end up in the Finder
  • Open System Preferences -> Sharing
    • Enable:
      • Screen Sharing
      • Remote Login
  • Open System Preferences -> Enery Saver
    • Move Computer sleep slider to Never
    • Move Disply sleep slider to Never
    • Disable:
      • put hard disks to sleep when possible
      • Allow power button to put computer to sleep (doesn't seem to be present, at least in Fusion)
    • Enable:
      • Wake for network access
  • Shut down
  • ..and capture

Capturing from VMWare Fusion

  • Shut down the VM
  • Duplicate the VM's .vmdk in Finder
  • Add a new, existing hard disk to the VM, pointing it at the duplicate
  • Boot the VM
  • Fire up Disk Utility within the VM
  • Click on the second "Mac OS X" hard drive
  • File -> New -> Image from "Mac OS X"

Mac OS X 10.8 (12A269) - Mountain Lion

The current base image is built from installing 10.7 Lion via the recovery boot option (cmd + R) and then upgrading to 10.8 Mountain Lion through the App Store. You will need to log in to the apple developer site and redeem OS X Mountain Lion under your credentials

  • Using a R5 Mac Mini, boot into OS X Lion Recovery mode
Hold down command + R during the boot process
  • Once in recovery mode; launch the disk utility and remove any software raid configuration
  • Erase all disks. This partitions and formats all disks
  • Close disk utility and launch lion osx recovery from the menu
  • Go get coffee and take a nap
  • Once the recovery is complete, the system will reboot

At this point the system will have a fresh install of 10.7 OS X Lion

  • walk through the setup screen and create a default user
    • default user should be 'administrator' using the current releng root/admin kickstart password
  • skip all of the apple account registration screens
  • At the login screen, use the 'administrator' credentials that were entered earlier
  • Launch the App store using your apple developer credentials
  • Search 'os x mountain lion'
  • Click download under OS X Mountain Lion
    • Make sure to sign out of the app store once the download is complete
  • Once the download is complete, launch the OS X Mountain Lion installer and proceed through the setup process
  • Once installed, the system will reboot

The system is now upgraded and running OS X 10.8 Mountain Lion

  • At the login screen, use the 'administrator' credentials that were entered earlier
  • Open System Preferences -> Sharing
    • Disable:
      • Remote Management
    • Enable:
      • Screen Sharing and Remote Login
  • Open System Preferences -> Enery Saver
    • Move Computer sleep slider to Never
    • Move Disply sleep slider to Never
    • Disable:
      • put hard disks to sleep when possible
      • Allow power button to put computer to sleep
    • Enable:
      • Wake for network access
  • Capture the image for Deploy Studio
  • Integrate into the workflows to image 10.8 systems

Mac OS X Server 10.7.2 (11C74) - Lion

  • Install OS X 10.7.2 on an r4 or r5 mini
  • walk through the setup screen and create a default user
    • default user should be 'administrator' using the current releng root/admin kickstart password
  • skip all of the apple account registration screens
  • At the login screen, use the 'administrator' credentials that were entered earlier
  • Open System Preferences -> Sharing
    • Enable Screen Sharing and Remote Login
  • Open System Preferences -> Enery Saver
    • Move Computer sleep slider to Never
    • Move Display sleep slider to Never
  • Capture the image for Deploy Studio
  • Integrate into the workflows to image 10.7.2 systems

Mac OS X 10.6.8 (10K549) - Snow Leopard

  • Install OS X 10.6.4 on an r4 mini from disk.
  • walk through the setup screen and create a default user
    • default user should be 'administrator' using the current releng root/admin kickstart password
  • skip all of the apple account registration screens
  • At the login screen, use the 'administrator' credentials that were entered earlier
  • Open System Preferences -> Sharing
    • Enable Screen Sharing and Remote Login
  • Open System Preferences -> Enery Saver
    • Move Computer sleep slider to Never
    • Move Display sleep slider to Never
  • Apply the 10.6.8 update.
  • Capture the image for Deploy Studio
  • Integrate into the workflows to image 10.8.6 systems