Auto-tools/staging

Tools Staging Environment Booking

masters

Master	User	Testing	Notes	Dates	Interruptable	Waiting
mozilla-tests1 (9010)			-		yes	-
mozilla-tests2 (9012)	anode	bug 599162	talos needs to be able to install release builds	oct 5 - ?	yes	-
mozilla-tests3 (9012)	ctalbert		tegra testing			-

slaves

001 slaves are attached to mozilla-tests1, 002 slaves are attached to mozilla-tests2. The slaves themselves can also be removed from the pool and booked individually.

currently booked

Slave	User	Testing/Notes	Dates
-	-	-	-

known slaves

Platform	Slaves
Linux	tools-r3-fed-{001,002}
Linux64	tools-r3-fed64-{001,002}
Mac	tools-r3-leopard-{001,002}
Mac64	tools-r3-snow-{001,002}
Win32	tools-r3-xp-{001,002}, tools-r3-w7-{001,002}
Win64	tools-r3-w764-{001,002}

Notes & Best Practices

restarting buildbot

Go to your master configuration path and run

make reconfig

or

make stop; make start

restarting buildslaves

The waterfall should tell you if builders are offline

click on the builder on your waterfall to get a list of slaves
vnc into each slave

On Windows:

open a terminal
change to the c:\ directory
run `buildbot start talos-slave`

sending changes

run

/builds/buildbot/sendchanges.sh

vnc vs. rdc

only log into talos slaves using vnc
rdc gives you a remote, virtual console on the box, you end up running performance tests against your home machine's graphics display which is not desired
to restart the slave, open a terminal and run buildbot start talos-slave
you must use vnc to restart a slave; you can ssh into a slave but the tests require a display. so use vnc
don't use the startTalos icon on the desktop for reasons i don't understand

test-master1

uses 001 slaves
master path : buildbot master configuration

/builds/buildbot/mozilla-tests1

buildbotcustom path

/tools/buildbotcustom/buildbotcustom

user configs

/builds/buildbot/user-configs

waterfall: http://tools-staging-master.mv.mozilla.com:8010/waterfall
sendchange port: 9010

test-master2

uses 002 slaves
master path

/builds/buildbot/mozilla-tests2

buildbotcustom path

/tools/buildbotcustom2/buildbotcustom

user configs

/builds/buildbot/user-configs2

waterfall: http://tools-staging-master.mv.mozilla.com:8012/waterfall
sendchange port: 9012

switching users

create your own user environment

go to the user configs path for your master
create configs-{your_name}

cp -r configs-clean configs-{your_name}

create your own buildbotcustom

go to the buildbotcustom path for your master
create buildbotcustom-{your_name}

cp -r buildbotcustom-clean buildbotcustom-{your_name}

switch master to use your configs/buildbotcustom

book the master you want to use
go to the directory of your master
run

./switch-user {your_name}

A possible doc update on script name and location?:

[root@localhost mozilla-tests2]# pwd
/builds/buildbot/mozilla-tests2
[root@localhost mozilla-tests2]# ./switch_configs.sh jmaher

buildbot config files and what they do

you will be working out of

configs-{your_name}/mozilla-tests

you can switch configs from /builds/buildbot/mozilla-tests1

./switch_configs {name}

config.py

contains configuration for talos tests
contains configuration for which talos tests are run on which branches
shared between production and staging machines

staging-config.py

contains graph server configuration (points to graphs-stage)
contains slave names
- NOTE have to be updated to point to tools machine names instead of releng stage talos machine names)

staging_tests_master_stm{01,02}_localconfig.py

sets slave port number
sets master waterfall link
- NOTE has to be updated to point to correct tools set up
sets active branches + platforms
- can switch on/off branches to narrow down testing to some/all branches
- can switch on/off platforms to narrow testing to some/all OSes

how to run custom talos code

talos code is checked out per-run by default
edit

/tools/buildbotcustom{ ,2}/buildbotcustom-{your-name}/process/factory.py

#self.customTalos = customTalos
self.customTalos = 'talos.zip' #customTalos

there is already a copy of talos in your user configs that you can edit

/build/buildbot/user-configs{ ,2}/configs-{your_name}/talos

just package it into and it will be downloaded and unzipped per-run

/build/buildbot/user-configs{ ,2}/configs-{your_name}/talos.zip

userland code

If you need non shared code, make a directory

/home/cltbld/{your_name}

Landing Your Changes

Are you ready to land

Full, green staging run?
r+ review?

Talos Changes

push changes to http://hg.mozilla.org/build/talos/
pull a fresh copy
rm -rf .hg
package into talos.zip

zip -r talos.zip talos/
unzip -l talos.zip
Archive:  talos.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  10-14-10 15:21   talos/
     6618  10-05-10 15:27   talos/remote.config
     7936  10-05-10 15:27   talos/ffprocess_win32.py
    19030  10-14-10 15:20   talos/PerfConfigurator.py
      287  10-05-10 15:27   talos/.hgtags
        0  10-05-10 15:29   talos/page_load_test/
        0  10-05-10 15:27   talos/page_load_test/svg_opacity/
      177  10-05-10 15:27   talos/page_load_test/svg_opacity /svg_opacity.manifest
   191667  10-05-10 15:27   talos/page_load_test/svg_opacity/small-group-opacity-2500.svg
   120159  10-05-10 15:27   talos/page_load_test/svg_opacity/big-optimizable-group-opacity-2500.svg
    33272  10-05-10 15:27   talos/page_load_test/parray.js
        0  10-05-10 15:27   talos/page_load_test/dhtml/
    13272  10-05-10 15:27   talos/page_load_test/dhtml/replaceimages.html
     3409  10-05-10 15:27   talos/page_load_test/dhtml/slidein.html
     4800  10-05-10 15:27   talos/page_load_test/dhtml/movingtext.html
     1425  10-05-10 15:27   talos/page_load_test/dhtml/slidingballs.html
     1529  10-05-10 15:27   talos/page_load_test/dhtml/colorfade.html
      832  10-05-10 15:27   talos/page_load_test/dhtml/textslide.html
    31237  10-05-10 15:27   talos/page_load_test/dhtml/layers1.html
      716  10-05-10 15:27   talos/page_load_test/dhtml/imageslide.html
      830  10-05-10 15:27   talos/page_load_test/dhtml/diagball.html
    31198  10-05-10 15:27   talos/page_load_test/dhtml/layers4.html
     1686  10-05-10 15:27   talos/page_load_test/dhtml/mozilla.html
     2729  10-05-10 15:27   talos/page_load_test/dhtml/fadespacing.html
      995  10-05-10 15:27   talos/page_load_test/dhtml/dhtml.manifest
     1243  10-05-10 15:27   talos/page_load_test/dhtml/meter.html
    31274  10-05-10 15:27   talos/page_load_test/dhtml/layers6.html
    31212  10-05-10 15:27   talos/page_load_test/dhtml/layers2.html
      596  10-05-10 15:27   talos/page_load_test/dhtml/zoom.html
        0  10-05-10 15:27   talos/page_load_test/dhtml/images/
      472  10-05-10 15:27   talos/page_load_test/dhtml/images/element_subnavigatio.gif
     2288  10-05-10 15:27   talos/page_load_test/dhtml/images/img1.jpg
     3329  10-05-10 15:27   talos/page_load_test/dhtml/images/css-icon.jpg
     1144  10-05-10 15:27   talos/page_load_test/dhtml/images/3dball.gif
     3495  10-05-10 15:27   talos/page_load_test/dhtml/images/gecko-icon.jpg
     2401  10-05-10 15:27   talos/page_load_test/dhtml/images/moz.gif
     4016  10-05-10 15:27   talos/page_load_test/dhtml/images/bugs-icon.jpg
     2956  10-05-10 15:27   talos/page_load_test/dhtml/images/dhtml-icon.jpg
     1659  10-05-10 15:27   talos/page_load_test/dhtml/images/welcome-icon.jpg
     2221  10-05-10 15:27   talos/page_load_test/dhtml/images/img2.jpg
     2912  10-05-10 15:27   talos/page_load_test/dhtml/images/credits-icon.jpg
     1593  10-05-10 15:27   talos/page_load_test/dhtml/images/ball.gif
     <snip>
     ...
     </snip>
    19038  10-05-10 15:27   talos/ttest.py
     6615  10-05-10 15:27   talos/cmanager_linux.py
     3516  10-05-10 15:27   talos/post_file.py
 --------                   -------
 29653726                   1284 files

verification steps
- check file size
- check # of files packaged
post copy of talos.zip to your people account
- be careful here, don't get it mixed up with an old copy
file bug for releng to land your new talos.zip into http://hg.mozilla.org/build/talos/talos.zip during their next downtime
- provide the md5sum

Pageloder Changes

push changes to http://hg.mozilla.org/build/pageloader/
pull a fresh copy
package into pageloader.xpi

cd pageloader
zip -r pageloader.xpi .
$ unzip -l pageloader.xpi 
Archive:  pageloader.xpi
  Length     Date   Time    Name
 --------    ----   ----    ----
     3166  09-17-10 08:22   README
        0  09-17-10 08:23   chrome/
    12694  09-17-10 08:23   chrome/pageloader.js
     2525  09-17-10 08:22   chrome/pageloader.xul
     3344  09-17-10 08:22   chrome/quit.js
     7987  09-17-10 08:22   chrome/report.js
      300  09-17-10 08:22   chrome.manifest
        0  09-17-10 08:22   components/
     7416  09-17-10 08:22   components/tp-cmdline.js
 --------                   -------
    37432                   9 files

post copy of pageloader.xpi your people account
- be careful here, don't get it mixed up with an old copy
file bug for releng to land your new pageloader.xpi into http://hg.mozilla.org/build/talos/xpis/pageloader.xpi during their next downtime

Auto-tools/staging

Contents

Tools Staging Environment Booking

masters

slaves

currently booked

known slaves

Notes & Best Practices

restarting buildbot

restarting buildslaves

sending changes

vnc vs. rdc

test-master1

test-master2

switching users

create your own user environment

create your own buildbotcustom

switch master to use your configs/buildbotcustom

buildbot config files and what they do

config.py

staging-config.py

staging_tests_master_stm{01,02}_localconfig.py

how to run custom talos code

userland code

Landing Your Changes

Are you ready to land

Talos Changes

Pageloder Changes

Navigation menu

Auto-tools/staging

Tools Staging Environment Booking

masters

slaves

currently booked

known slaves

Notes & Best Practices

restarting buildbot

restarting buildslaves

sending changes

vnc vs. rdc

test-master1

test-master2

switching users

create your own user environment

create your own buildbotcustom

switch master to use your configs/buildbotcustom

buildbot config files and what they do

config.py

staging-config.py

staging_tests_master_stm{01,02}_localconfig.py

how to run custom talos code

userland code

Landing Your Changes

Are you ready to land

Talos Changes

Pageloder Changes

Navigation menu

Search