This procedure will re-create the Windows 2003 based reference image. Internally, this VM is named win2k3sp2-vc8tools-ref-vm.

When you update the ref platform, also update the text file on the desktop of the VM (in both Admin and cltbld) to the new version number.

Things Done For You

Version 3

Configuration Details

VM Created with ISO: EN_WIN2003_ENTWITHSP1_WIN2003_STANDARDWITHSP1.ISO MD5 sum: 9f584062feca857738ac794a8bc46936 From MSDN CD: Disc 2429.4, November 2005

Includes Downloaded Software Updates

  • Security Update for Windows Server 2003 (KB917734)
  • Security Update for Windows Server 2003 (KB914389)
  • Security Update for Windows Server 2003 (KB917344)
  • Security Update for Windows Server 2003 (KB918439)
  • Security Update for Windows Server 2003 (KB911280)
  • Cumulative Security Update for Internet Explorer for Windows Server 2003 (KB916281)
  • Security Update for Windows Server 2003 (KB917953)
  • Windows Malicious Software Removal Tool - June 2006 (KB890830)
  • Security Update for Windows Server 2003 (KB908531)
  • Cumulative Security Update for Outlook Express for Windows Server 2003 (KB911567)
  • Security Update for Windows Server 2003 (KB911562)
  • Security Update for Windows Server 2003 (KB911927)
  • Security Update for Windows Server 2003 (KB908519)
  • Security Update for Windows Server 2003 (KB912919)
  • Update for Windows Server 2003 (KB910437)
  • Security Update for Windows Server 2003 (KB904706)
  • Security Update for Windows Server 2003 (KB896424)
  • Security Update for Windows Server 2003 (KB900725)
  • Security Update for Windows Server 2003 (KB905414)
  • Security Update for Windows Server 2003 (KB902400)
  • Security Update for Windows Server 2003 (KB899589)
  • Security Update for Windows Server 2003 (KB901017)
  • Security Update for Windows Server 2003 (KB893756)
  • Security Update for Windows Server 2003 (KB896428)
  • Security Update for Windows Server 2003 (KB896422)
  • Security Update for Windows Server 2003 (KB896358)
  • Security Update for Windows Server 2003 (KB899587)
  • Security Update for Windows Server 2003 (KB890046)
  • Security Update for Windows Server 2003 (KB899591)
  • Security Update for Windows Server 2003 (KB899588)
  • Security Update for Windows Server 2003 (KB901214)
  • Update for Windows Server 2003 (KB898715)

Configuration Changes

  • Turned OFF all automatic updates
  • Turned ON firewall; allowed RDP and SSH
  • Turned off all MS networking, leaving only TCP/IP
  • Created cltbld account
  • Turned on RDP; allowed cltbld and Administrator to connect
  • Turned performance settings to forground programs -> max
  • Turned off screensavers
  • Set networking to DHCP (so, will have to assign an IP)
  • Installed VMware tools; expects vmxnet networking

VC8 CONFIGURATION DETAILS

Installed from ISOs: 1cc97c96e3b037e4514b8f0fe6929862 en_vs_2005_pro_cd1.iso 7ac81a9a1c1571ed7e204fc96093e0c7 en_vs_2005_pro_cd2.iso

Includes ONLY the following VC8 components:

  • Visual C++
  • Tools for Redistributing Applications

FURTHER CONFIGURATION DETAILS

  • Add 15 gig d:\ drive
  • install cygwin into d:\cygwin
  • download cygwin packages to d:\packages (from www.cygwin.com; check CVS)
  • download vc8 moztools packages into d:\packages (from MDC; check CVS)
  • install vc8 tools into d:\moztools
  • rename link.exe to cyglink.exe (per MDC instructions)
  • Changed the following environment variables (system wide) in the System control panel:
 -- CYGWINBASE: d:\cygwin
 -- VCVARS: C:\Program Files\Microsoft Visual Studio 8\VC\bin\vcvars32.bat
  • Setup openssh to start as a service by running ssh-host-config
 (http://pigtail.net/LRP/printsrv/cygwin-sshd.html)
  • Check out mofo/release/tinderbox-configs/host/ssh in ~; ln -s .ssh/ to it.
  • Change font on Cygwin window
  • Install Blat:
 c25c74ecb8fe8e670e204e7196679190  blat-2.50/Blat250.source.zip
 66fc3b95fde52fa1887c9d8aab472033  blat-2.50/Blat250.zip
    • unzip Blat250.zip
    • mv -v full/blat.* /cygdrive/d/moztools/bin
    • Configure Blat: blat -install rheet.mozilla.org build@mozilla.org 5
  • Install 7Zip
 1c412567bb5473911a44ce17b5aa02dd  7zip-4.32/7z432.exe
 d55f567a62153bc43a04e9c52f210cdd  7zip-4.32/7z432.tar.bz2
    • Run the installer; install in d:\buildtools\7zip
  • Install UPX:
 852bd691d8abc75b52053465846fba34  upx-1.25/ucl-1.03.tar.gz
 6f20a62999a46a1864652454b3c8a5d8  upx-1.25/upx-1.25-src.tar.gz
 a092272e9e6995764adfcd4fb1e729db  upx-1.25/upx125w.zip
    • unzip upx125w.zip into d:\buildtools\upx
  • Install NSIS:
 32b1a327d3611f01053c3a2677e50d7b  nsis-2.17/nsis-2.17-setup.exe
 a3c068fc1cb613160d7202e1719d63bc  nsis-2.17/nsis-2.17-src.tar.bz2
    • Run the installer; full install; install in d:\buildtools\NSIS
    • Delete desktop icon
  • Set the perms on the installed tools correctly
    • chgrp -R users d:\buildtools
    • chmod -Rv 0755 d:\buildtools

Version 4

Was created from pre-tools installation snapshot of the above Win32 VM, so Cygwin, etc. weren't ever installed (from this VM's perspective).

  • Installed Win2k3 Service Pack 2 for x86 32-bit.
  • Installed post-SP2 updates:
    • Windows Malicious Software Removal Tool - March 2007 (KB890830)
    • Update for Windows Server 2003 Service Pack 2 (KB931836)
    • MSXML 6.0 RTM Security Update (KB927977)
    • MSXML 6.0 RTM Security Update (925673)

Version 5 (August 2007 Update/Refresh)

Configuration Changes

These are all VMware-related changes:

  • Upgrade virtual hardware in VI (ESX 3 virtual machine)
  • Change to LSI logic SCSI controller
  • Upgrade VMware tools to build 38803

Includes Automatic Updates

In Order:

  • Run Windows Updater
    • Install Microsoft Update Active X control
    • Install Microsoft Windows Component Publisher Active X control
    • Click "Update Windows"
  • Update to Windows 2k3 Service Pack 2 (32-bit x86, released 6/12/2007)

Individual Updates:

Windows 2003 Server

  • Security Update for Windows Media Player 6.4 (KB925398)
  • Windows Malicious Software Removal Tool - July 2007 (KB890830)
  • Security Update for Microsoft .NET Framework, Version 2.0 (KB928365)
  • Security Update for Microsoft .NET Framework, Version 1.1 Service Pack 1 (KB933854)
  • Security Update for Windows Server 2003 (KB926122)
  • Security Update for Windows Server 2003 (KB924667)
  • Cumulative Security Update for Internet Explorer 6 for Windows Server 2003 (KB933566)
  • Cumulative Security Update for Outlook Express for Windows Server 2003 (KB929123)
  • Security Update for Windows Server 2003 (KB935840)
  • Security Update for Windows Server 2003 (KB935839)
  • Update for Windows Server 2003 (KB927891)
  • MSXML 6.0 RTM Security Update (KB927977)
  • Security Update for Windows Server 2003 (KB930178)
  • Security Update for Windows Server 2003 (KB931784)
  • Security Update for Windows Server 2003 (KB932168)
  • Security Update for Windows Server 2003 (KB925902)
  • Update for Windows Server 2003 Service Pack 2 (KB931836)

Microsoft CAPICOM

  • Security Update for CAPICOM (KB931906)

Microsoft SQL Server Feature Pack

  • MSXML 6.0 RTM Security Update (925673)

Microsoft Visual Studio 2005

  • Security Update for Microsoft Visual Studio 2005 (KB925674)

to install

Version 6 (January 2008 Update/Refresh)

Install Microsoft® Windows® Software Development Kit Update for Windows Vista

http://www.microsoft.com/downloads/details.aspx?familyid=4377F86D-C913-4B5C-B87E-EF72E5B4E065&displaylang=en

  • Make sure you get the Web Install
    • ONLY install the 'Developer Tools' (all of them). Do NOT install Documentation or Samples.
    • Install to 'd:\sdks\v6.0'
    • Install Samples to 'd:\sdks\v6.0\Samples'
  • After installing it, it should show up in Add/Remove Programs as 'Microsoft Windows Software Development Kit for Windows Vista Update (6000.16384.10)'.

Install MozillaBuild 1.2

  • Delete everything in d:\mozilla-build
  • Uninstall previous NSIS (v2.17) installation.
  • Download MozillaBuild 1.2 and install to d:\mozilla-build.
    • MozillaBuild1.2 includes NSIS v2.22

Version 7

Installing the NRPE daemon

  • Download NSClient++ and extract to C:\Program Files\nscp
  • Checkout mozilla/tools/nagios and copy NSC.ini to C:\Program Files\nscp
  • Open up cmd.exe and do the following:
cd C:\Program Files\nscp
nsclient++ /install
nsclient++ /start
  • Added firewall exceptions for port 5666 (tcp+udp) and ICMP ping replies.
    1. Go to Control Panel -> Windows Firewall
    2. Click on the 'Exceptions' tab.
    3. Click 'Add Port'
    4. Name: 'nscp', Port: '5666', (*) TCP
    5. Repeat the above for (*) UDP
    1. Go To Control Panel -> Windows Firewall
    2. Click on the 'Advanced' tab.
    3. Click on ICMP -> Settings
    4. Check the box for 'Allow incoming echo request'
    5. Hit OK, OK.

Version 8

Move to Python 2.5

  • Delete d:\buildbot (which contains python24)
  • Append ';d:\mozilla-build\python25' to system path.
  • Install this registry key:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore\2.5\InstallPath]
@="d:\\mozilla-build\\python25"

Install Mercurial

  • Download Mercurial and install to d:\mercurial.
  • Append ';d:\mercurial' to the system path.

Version 9

Disable JIT Debugger

Per this MSDN entry:

Under the registry key: 
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current Version\AeDebug

set "Auto"=0
rename the "Debugger" variable to "VDebugger"
create a new string entry "Debugger" with an empty string

Under the registry key: 
HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting

set value to 1 (dumps stack and exits)

Version 10

Fix registry setting

In version 9, we accidentally set

 HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting

to hexidecimal 10 instead of 1. Not clear if this will fix bug 453260.

Setup environment vars

  • finish setting up environment vars by:
    • logging into machine as Administrator
    • in control panel, system, advanced tab, environment
    • set MOZ_NO_RESET_PATH=1

Downgrade twisted and apply process group patch

To fix a long running saga with win32 machines not recovering when an executable dies, we downgrade Twisted to version 2.5.0 and patch _dumbwin32proc.py so that twisted (and hence buildbot) can kill process groups properly. As Administrator, do the following:

rm -rfv /d/mozilla-build/python25/Lib/site-packages/{twisted/,Twisted-8.0.1-py2.5.egg-info/,zope/} /d/mozilla-build/python25/scripts/twistd*
wget http://tmrc.mit.edu/mirror/twisted/Twisted/2.5/Twisted_NoDocs-2.5.0.win32-py2.5.exe
./Twisted_NoDocs-2.5.0.win32-py2.5.exe
cd /d/mozilla-build/python25/Lib/site-packages/twisted/internet/
wget -O patch "http://twistedmatrix.com/trac/attachment/ticket/2726/_dumbwin32proc.patch?format=raw"
patch -p8 < patch

Version 11

Setup extra disk

  • finish disk setup by
    • logging into machine as Administator
    • right click on MyComputer, and select "Manage"
    • click "DiskManagement" and select "Rescan"
    • see unformatted partition. Assign it drive "E", and format it as FAT32 (not NTFS)
    • save (and possibly reboot?).


setup ssh keys

  • manually copy ssh keys from another staging / production VM and install into /c/Documents and Settings/cltbld/.ssh.
  • also copy them to ~/.ssh in msys

Version 12

  • Version of Ref Image is now in a txt file on the desktop for quick reference

Accept license for D:\\Utilities

The pslist.exe and pskill.exe are in D:\\Utilites and have been opened once, their licenses accepted.

Screen Resolution Set so Unittests will pass

  • Run GPEDIT.MSC;
  • Go to: Local Computer Policy/Computer Configuration/Administrative Templates/Windows Components/Terminal Services;
  • Open the policy 'Limit maximum colour depth';
  • Set this to 'Enabled and 24 bit'
  • Verify that the screen resolution is set to: 1280 x 1024, 24bit color

Alternatively, import the following registry file:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services]
"ColorDepth"=dword:00000004

Setup Slave Directory

cd /e
mkdir -p builds/moz2_slave

Accept staging SSH keys

Manually use ssh to connect to the following machines, and manually accept "yes" the first connection

# staging
$ ssh -i ~/.ssh/ffxbld_dsa staging-stage.build.mozilla.org


Version 13

Install VS9

As administrator:

  • Start -> All Programs -> Accessories -> System Tools -> Disk Cleanup. Do a default clean of C:
  • Mount the VS2008 DVD from the VI Client and do a full install of Visual Studio 2008 into D:\msvs9 and reboot.
  • Checkout Windows Mobile MSI from mofo repo:
cvs -d:ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/win32/wm6-pro-sdk.msi
cvs -d:ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/win32/wm6.1-pro-images.msi
  • Install Windows Mobile 6 Pro SDK into D:\sdks\wm6 ... This will warn you about missing ActiveSync; that's fine.
  • Install Windows Mobile 6.1 Pro Emulator Images to D:\sdks\wm6

Version 14

Java

This isn't an update per-se, just documenting an older install per a developer request.

java version "1.5.0_10"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_10-b03)
Java HotSpot(TM) Client VM (build 1.5.0_10-b03, mixed mode, sharing)

Clean rebooting setup (VNC, autologin, audio driver)

Enable the VM to use the proper resolution and screendepth in the console session:

  • Shutdown & poweroff the VM.
  • Edit the .vmx file and add the following lines:
svga.maxHeight = 1024
svga.maxWidth = 1280
svga.vramSize = 16777216
  • Start the VM back up again

Install RealVNC

  • Checkout the installer from mofo:
cvs -d:ext:cltbld@cvs.mozilla.org:/mofo co -d vnc ref-platforms/win32/vnc-4_1_3-x86_win32.exe
  • Click through the installer accepting all the defaults
  • When the setup window pops up set a password and turn off the Java viewer
  • Add a firewall exception
    • Start Menu -> Control Panel -> Windows Firewall
    • Exceptions -> Add Port -> TCP/5900

Install the virtual audio driver

  • Checkout the VAC zip from mofo:
cvs -d:ext:cltbld@cvs.mozilla.org:/mofo co -d vnc ref-platforms/win32/vac409.zip
  • Run the installer and accept the defaults

Setup cltbld autologin

  • Allow cltbld to restart the system
    • Start menu -> Run -> gpedit.msc
    • Computer Configuration -> Windows Settings -> Security Settings -> Local Policies -> User Rights Assignment
    • Double click 'Shut down the system', add cltbld to the list.
  • Add autologin registry keys
wget -Ologon.reg --no-check-certificate https://bug428123.bugzilla.mozilla.org/attachment.cgi?id=367599
  • Edit with the proper password, then import into the registry

Download Buildbot startup scripts

As cltbld, with an msys window open:

cd ~/Start\ Menu/Programs/Startup
wget -Obuildbot.bat --no-check-certificate https://bugzilla.mozilla.org/attachment.cgi?id=362741
cd /d/mozilla-build
wget -Ostart-buildbot.bat --no-check-certificate https://bugzilla.mozilla.org/attachment.cgi?id=362742

Reboot

  • Reboot and login over VNC as cltbld
  • Set screen resolution to 1280x1024x32

Version 15

Update Tar.exe

Replace tar.exe because of bug 383136

cd /
wget http://downloads.sourceforge.net/sourceforge/mingw/tar-1.19.90-MSYS-1.0.11-2-bin.tar.gz
mv /bin/tar.exe /bin/tar2.exe
tar2 zxf tar-1.19.90-MSYS-1.0.11-2-bin.tar.gz
rm /bin/tar2.exe
rm tar-1.19.90-MSYS-1.0.11-2-bin.tar.gz

Version 16

OPSI Client & Autoit

Enable File and Print Sharing

  • Enable File & Print Sharing
    • Control Panel -> Network Connections -> Local Area Connection -> Properties
    • Enable "Client for Microsoft Networks" and "File and Print Sharing", click Close.
  • Open firewall (only to opsi server)
    • Control Panel -> Windows Firewall -> Exceptions
    • Enable "File and Print Sharing", click "Edit"
    • For each entry:
      • Click "Change Scope"
      • Select "Custom List"
      • Enter the IP address of the production OPSI server (10.2.71.64)

OPSI Prep and Install

NB: This has to be done post-install too, otherwise OPSI won't register the new clone properly. See instructions in the post-install section for that.

We need to flip a few options in Windows to enable certain packages to install without prompting.

  • Do not prompt for unsigned drivers (needed for at least the windows mobile sdk)
    • Control Panel -> System -> Hardware -> Driver Signing, choose "Ignore".
  • Security Settings
    • Start Menu -> Run -> \\10.2.71.64
    • Use 'pcpatch' and the root password when prompted.
    • Right click 'opt_pcbin' and select 'Mount as Network Drive'
    • Browse to z:\install\preloginloader
    • Double click 'opsi-prep.vbs'
    • Double click 'service-setup.cmd', hit a key when prompted
    • When prompted for a username and password use 'root' and the appropriate password.

Once OPSI is finished installing it will forcibly reboot the system. In a few minutes it should come back up and the client should be listed in the OPSI GUI.

Autoit

We require Autoit to deploy many packages. To install it, do the following:

  • Checkout the installer:
cvs -d:ext:cltbld@cvs.mozilla.org:/mofo co ref-platforms/win32/autoit-v3-setup.exe
  • (as Administrator), run the installer, accept the defaults

Manual Setup Steps after VMImage

IT Stuff

Try slave firewall setup

Only for try slaves (try-win32-slaveN) We need to open up ports for the try slaves to access the OPSI server. For production slaves, the following ports need to be opened between the slave and production-opsi.b.m.o:

  • tcp/udp 137-139
  • tcp/445
  • tcp/901
  • tcp/4447
  • icmp ping

For staging slaves, the same changes need to be made, but between the slave and staging-opsi.b.m.o.

Add drive E

  • create an 80gb NTFS disk, attach to VM
  • use VNC to login as Administrator on VM
    • assign to drive letter E
    • Volume name should be "BUILDS"
    • manually right click on disk E, select properties and go to the general tab f the dialog. At the bottom of the dialog box, verify that "Compress Drive" and "Indexing service" are both unchecked (turned off).
  • start dos-cmd-shell and run the following commands. It does not matter what directory is currect directory, these changes are system wide.
c:\> fsutil behavior set disable8dot3 1
c:\> fsutil behavior query disable8dot3
disable8dot3 = 1
c:\> fsutil behavior set disablelastaccess 1
c:\> fsutil behavior query disablelastaccess
disablelastaccess = 1
  • reboot

(We followed tips outlined http://www.windowsdevcenter.com/pub/a/windows/2005/02/08/NTFS_Hacks.html and http://technet.microsoft.com/en-us/library/cc785435.aspx.)

RelEng stuff

Change the computername

  • Log in as administrator
  • Right click on My Computer -> Properties
  • In the Computer Name tab, click "Change" and enter the desired hostname and click OK. You will be warned that you need to reboot and that there is a 15 character limit for netbios.
  • Reboot. (You may as well do this after the next step, changing your display settings)
  • After reboot, the hostname will be truncated to 15 characters in your environment, but if you use the 'hostname' command in msys, it'll show the full hostname.

Reinstallation of OPSI

OPSI is installed on the reference platform so we can easily perform upgrades to it. Upon cloning a new slave it must be reinstalled so it identifies itself to OPSI correctly.

Installing OPSI on a slave that has been recloned

If a slave with OPSI on it gets recloned the OPSI server will still have a reference to it. Before the client install can happen you must remove it from the OPSI server. To do so, do the following on the correct OPSI Server (production-opsi.build or staging-opsi.build):

# as root
opsi-admin -d method deleteClient $hostname

Then check /etc/opsi/pckeys to make sure the slave was removed.

$hostname is what the hostname command would return on the slave with .uib.local appended, for example "win32-slave30.uib.local"

Uninstall OPSI

You must uninstall the OPSI client so the new slave doesn't claim to be the reference platform. To do so, browse to c:\program files\opsi.org\preloginloader and run 'deinstall.cmd'.

Firewall setup

For staging slaves only (the ref platform is already setup correctly for production slaves).

  • Open firewall (only to opsi server)
    • Control Panel -> Windows Firewall -> Exceptions
    • Enable "File and Print Sharing", click "Edit"
    • For each entry:
      • Click "Change Scope"
      • Select "Custom List"
      • Enter the IP address of the staging OPSI server (10.2.71.216).

The rest

(Applicable to staging and production).

  • Security Settings
    • Start Menu -> Run -> \\$ipOfOpsiServer (staging: \\10.2.71.216, production: 10.2.71.64)
    • Use 'pcpatch' and the root password when prompted.
    • Right click 'opt_pcbin' and select 'Mount as Network Drive'
    • Browse to z:\install\preloginloader
    • Double click 'opsi-prep.vbs'
    • Double click 'service-setup.cmd', hit a key when prompted
    • When prompted for a username and password use 'root' and the appropriate password.

Once OPSI is finished installing it will forcibly reboot the system. It will come up with an error on the first reboot. Do the following

  • Logon to the slave with VNC
  • Click the red 'X'
  • Click the 'OK' button until the dialogs go away

OPSI will work fine on the next reboot. Before rebooting, update the state table for the slave with the OPSI client as follows:

  • Mark 'java' as installed
  • Set 'passwordupdate's action to 'always'

Now, reboot again. Once the reboot is done 'passwordupdate' should be 'installed' according the OPSI client.

Installing Buildbot

Because our Buildbot code changes fairly regularly we don't keep a copy of it on the VM.

  • Login as cltbld, start an MSYS window, and run:
hg clone http://hg.mozilla.org/build/buildbot 
cd buildbot
hg up -r BUILDBOT_PRODUCTION
python setup.py build
python setup.py install

Setup Slave

cd /e/builds/slave or /e/builds/moz2_slave

...and either run "/d/mozilla-build/python25/scripts/buildbot create-slave" or else copy over buildbot.tac and info files from another win32 slave.

Edit the buildbot.tac file to have

  • umask = 002 instead of None, so that nightly update snippets have sensible permissions
  • keepalive = None so that a busy master doesn't cause us to timeout jobs
  • usepty = 1 so we can kill off failed unit test runs (definitely need this on mac and linux, not sure about win32 but here for parity)

Start Slave

$ cd 
$ /d/mozilla-build/python25/scripts/buildbot start /e/builds/slave

Note: make sure to minimize the MSYS window, or else unittests will fail... bug 465096

verify setup in staging

  • trigger a build on staging-master, to make sure all works ok

switchover from staging to production

  • stop slave process
  • switchover the .ssh keys to use production keys

Note: for a try server builder you will need to copy keys from another try machine

  • Manually use ssh to connect to the following machines, and manually accept "yes" the first connection
# production
$ ssh -i ~/.ssh/ffxbld_dsa ffxbld@aus2-staging.mozilla.org
$ ssh -i ~/.ssh/ffxbld_dsa ffxbld@dm-symbolpush01.mozilla.org
$ ssh -i ~/.ssh/ffxbld_dsa ffxbld@stage.mozilla.org
  • edit /e/builds/moz2_slave/buildbot.tac to change "staging-master" to "production-master".
  • clobber all build trees on slave
  • start slave process