Auto-tools/staging: Difference between revisions

Revision as of 21:08, 28 June 2011

Tools Staging Environment Booking

masters

Master	User	Testing	Notes	Dates	Interruptable	Waiting
[1] (9012)	jmaher					-
[2] (9010)	anode					-

slaves

001 slaves are attached to master:anode, 002 slaves are attached to master:jmaher. The slaves themselves can also be removed from the pool and booked individually.

currently booked

Slave	User	Testing/Notes	Dates
-	-	-	-

known slaves

Platform	Slaves
Linux	tools-r3-fed-001
Linux64	tools-r3-fed64-001
Mac	tools-r3-leopard-001
Mac64	tools-r3-snow-001
Win32	tools-r3-xp-001, tools-r3-w7-001
Win64	tools-r3-w764-001

All 002 machines have been donated to create the addon performance testing pool

Notes & Best Practices

restarting buildbot

Go to /builds/buildbot/$MASTER

make reconfig

or

make stop; make start

restarting buildslaves

The waterfall should tell you if builders are offline

click on the builder on your waterfall to get a list of slaves
vnc into each slave

On Windows:

open a terminal
change to the c:\ directory
run `buildbot start talos-slave`

sending changes

run

/builds/buildbot/sendchanges.sh

vnc vs. rdc

only log into talos slaves using vnc
rdc gives you a remote, virtual console on the box, you end up running performance tests against your home machine's graphics display which is not desired
to restart the slave, open a terminal and run buildbot start talos-slave
you must use vnc to restart a slave; you can ssh into a slave but the tests require a display. so use vnc
don't use the startTalos icon on the desktop for reasons i don't understand

master: anode

uses 001 slaves
master path : buildbot master configuration

/builds/buildbot/anode

buildbotcustom path

/builds/buildbot/anode/buildbotcustom

user configs

/builds/buildbot/anode/buildbot-configs

waterfall: http://tools-staging-master02.mv.mozilla.com:8010/waterfall
sendchange port: 9010

master: jmaher

uses 002 slaves
master path

/builds/buildbot/jmaher

buildbotcustom path

/builds/buildbot/jmaher/buildbotcustom

user configs

/builds/buildbot/jmaher/buildbot-configs

waterfall: http://tools-staging-master02.mv.mozilla.com:8012/waterfall
sendchange port: 9012

switching users

you can currently create your own master, we no longer support user switching

buildbot config files and what they do

you will be working out of

//builds/buildbot/$MASTER/buildbot-configs/mozilla-tests

config.py

contains configuration for talos tests
contains configuration for which talos tests are run on which branches
shared between production and staging machines

staging-config.py

contains graph server configuration (points to graphs-stage)
contains slave names
- NOTE have to be updated to point to tools machine names instead of releng stage talos machine names)

staging_tests_master_stm{01,02}_localconfig.py

sets slave port number
sets master waterfall link
- NOTE has to be updated to point to correct tools set up
sets active branches + platforms
- can switch on/off branches to narrow down testing to some/all branches
- can switch on/off platforms to narrow testing to some/all OSes

how to run custom talos code

talos code is checked out per-run by default
edit

/builds/buildbot/$MASTER/buildbotcustom/process/factory.py

#self.customTalos = customTalos
self.customTalos = 'talos.zip' #customTalos

there is already a copy of talos in the $MASTER to edit

/build/buildbot/$MASTER/talos

just package it into and it will be downloaded and unzipped per-run

/build/buildbot/$MASTER/talos.zip

userland code

If you need non shared code, make a directory

/home/cltbld/{your_name}

making a new test master

first figure out some settings:

MASTER_NAME should be one of
- staging-tests_master1
  - uses staging_tests_master_stm01_localconfig.py
  - creates a master that communicates on port 9010 and waterfall on port 8010
- staging-tests_master2
  - uses staging_tests_master_stm02_localconfig.py
  - creates a master that communicates on port 9012 and waterfall on port 8012
MASTER_DIR
- location of your master
- should be a subdirectory of /builds/buildbot
then run

cd /builds/buildbot/buildbot-configs
hg pull -u

make -f Makefile.setup \
MASTER_NAME=$MASTER_NAME \
BASEDIR=/builds/buildbot/$MASTER_DIR \
PYTHON=/usr/bin/python2.6 \
VIRTUALENV=/usr/bin/virtualenv-2.6 \
BUILDBOTCUSTOM_BRANCH=default \
BUILDBOTCONFIGS_BRANCH=default \
virtualenv deps install-buildbot master master-makefile

further info available in releng doc: https://wiki.mozilla.org/ReleaseEngineering/How_To/Setup_Personal_Development_Master

Newer suites

Once in a while we add new suites and having a checklist on what to verify will help not missing steps:

Does the builder gets added to buildbot?
Does the builder show up on tinderbox and is it marked to be scrapped?
Does the builder show up on tbpl? Does the summary show the breakdown of each suite? (e.g. tp contains tp5, tp5_rss et all)
When using the "compare" feature in between changesets on tbpl does the compare-talos tool show up your new suite?
Does the Try syntax work?
Does the Try Chooser website show the new suite?

See bug to disable tp4 for specific examples.

Landing Your Changes

Are you ready to land

Full, green staging run?
r+ review?

Talos Changes

push changes to http://hg.mozilla.org/build/talos/
pull a fresh copy
rm -rf .hg
package into talos.zip

zip -r talos.zip talos/
unzip -l talos.zip
Archive:  talos.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  10-14-10 15:21   talos/
     6618  10-05-10 15:27   talos/remote.config
     7936  10-05-10 15:27   talos/ffprocess_win32.py
    19030  10-14-10 15:20   talos/PerfConfigurator.py
      287  10-05-10 15:27   talos/.hgtags
        0  10-05-10 15:29   talos/page_load_test/
        0  10-05-10 15:27   talos/page_load_test/svg_opacity/
      177  10-05-10 15:27   talos/page_load_test/svg_opacity /svg_opacity.manifest
   191667  10-05-10 15:27   talos/page_load_test/svg_opacity/small-group-opacity-2500.svg
   120159  10-05-10 15:27   talos/page_load_test/svg_opacity/big-optimizable-group-opacity-2500.svg
    33272  10-05-10 15:27   talos/page_load_test/parray.js
        0  10-05-10 15:27   talos/page_load_test/dhtml/
    13272  10-05-10 15:27   talos/page_load_test/dhtml/replaceimages.html
     3409  10-05-10 15:27   talos/page_load_test/dhtml/slidein.html
     4800  10-05-10 15:27   talos/page_load_test/dhtml/movingtext.html
     1425  10-05-10 15:27   talos/page_load_test/dhtml/slidingballs.html
     1529  10-05-10 15:27   talos/page_load_test/dhtml/colorfade.html
      832  10-05-10 15:27   talos/page_load_test/dhtml/textslide.html
    31237  10-05-10 15:27   talos/page_load_test/dhtml/layers1.html
      716  10-05-10 15:27   talos/page_load_test/dhtml/imageslide.html
      830  10-05-10 15:27   talos/page_load_test/dhtml/diagball.html
    31198  10-05-10 15:27   talos/page_load_test/dhtml/layers4.html
     1686  10-05-10 15:27   talos/page_load_test/dhtml/mozilla.html
     2729  10-05-10 15:27   talos/page_load_test/dhtml/fadespacing.html
      995  10-05-10 15:27   talos/page_load_test/dhtml/dhtml.manifest
     1243  10-05-10 15:27   talos/page_load_test/dhtml/meter.html
    31274  10-05-10 15:27   talos/page_load_test/dhtml/layers6.html
    31212  10-05-10 15:27   talos/page_load_test/dhtml/layers2.html
      596  10-05-10 15:27   talos/page_load_test/dhtml/zoom.html
        0  10-05-10 15:27   talos/page_load_test/dhtml/images/
      472  10-05-10 15:27   talos/page_load_test/dhtml/images/element_subnavigatio.gif
     2288  10-05-10 15:27   talos/page_load_test/dhtml/images/img1.jpg
     3329  10-05-10 15:27   talos/page_load_test/dhtml/images/css-icon.jpg
     1144  10-05-10 15:27   talos/page_load_test/dhtml/images/3dball.gif
     3495  10-05-10 15:27   talos/page_load_test/dhtml/images/gecko-icon.jpg
     2401  10-05-10 15:27   talos/page_load_test/dhtml/images/moz.gif
     4016  10-05-10 15:27   talos/page_load_test/dhtml/images/bugs-icon.jpg
     2956  10-05-10 15:27   talos/page_load_test/dhtml/images/dhtml-icon.jpg
     1659  10-05-10 15:27   talos/page_load_test/dhtml/images/welcome-icon.jpg
     2221  10-05-10 15:27   talos/page_load_test/dhtml/images/img2.jpg
     2912  10-05-10 15:27   talos/page_load_test/dhtml/images/credits-icon.jpg
     1593  10-05-10 15:27   talos/page_load_test/dhtml/images/ball.gif
     <snip>
     ...
     </snip>
    19038  10-05-10 15:27   talos/ttest.py
     6615  10-05-10 15:27   talos/cmanager_linux.py
     3516  10-05-10 15:27   talos/post_file.py
 --------                   -------
 29653726                   1284 files

verification steps
- check file size
- check # of files packaged
post copy of talos.zip to your people account
- be careful here, don't get it mixed up with an old copy
file bug for releng to land your new talos.zip into build.mozilla.org (in talos/zips) during their next downtime (downtime might not be required depending on the update)
- provide the md5sum

Dirty Profile Changes or other NPOTB

For "Not Part of the Build" changes that can still affect numbers for developers please follow the following steps:

Write a notification to dev.planning, dev.platform and dev.tree-management to inform them of the change and what to expect
Arrange for tree-closure with the Tinderbox sheriff of the day
Make changes
Land a patch unto mozilla-central to have a cutting point for new talos numbers
Monitor the results in case backout is needed, send email to the above-mentioned lists if this needs to happen and repeat steps 1 & 2
- Please indicate the time and changeset after the change was deployed

Pageloder Changes

push changes to http://hg.mozilla.org/build/pageloader/
pull a fresh copy
package into pageloader.xpi

cd pageloader
zip -r pageloader.xpi .
$ unzip -l pageloader.xpi 
Archive:  pageloader.xpi
  Length     Date   Time    Name
 --------    ----   ----    ----
     3166  09-17-10 08:22   README
        0  09-17-10 08:23   chrome/
    12694  09-17-10 08:23   chrome/pageloader.js
     2525  09-17-10 08:22   chrome/pageloader.xul
     3344  09-17-10 08:22   chrome/quit.js
     7987  09-17-10 08:22   chrome/report.js
      300  09-17-10 08:22   chrome.manifest
        0  09-17-10 08:22   components/
     7416  09-17-10 08:22   components/tp-cmdline.js
 --------                   -------
    37432                   9 files

post copy of pageloader.xpi your people account
- be careful here, don't get it mixed up with an old copy
file bug for releng to land your new pageloader.xpi into http://hg.mozilla.org/build/talos/xpis/pageloader.xpi during their next downtime

Auto-tools/staging: Difference between revisions

Revision as of 21:08, 28 June 2011

Contents

Tools Staging Environment Booking

masters

slaves

currently booked

known slaves

Notes & Best Practices

restarting buildbot

restarting buildslaves

sending changes

vnc vs. rdc

master: anode

master: jmaher

switching users

buildbot config files and what they do

config.py

staging-config.py

staging_tests_master_stm{01,02}_localconfig.py

how to run custom talos code

userland code

making a new test master

Newer suites

Landing Your Changes

Are you ready to land

Talos Changes

Dirty Profile Changes or other NPOTB

Pageloder Changes

Navigation menu

Auto-tools/staging: Difference between revisions

Revision as of 21:08, 28 June 2011

Tools Staging Environment Booking

masters

slaves

currently booked

known slaves

Notes & Best Practices

restarting buildbot

restarting buildslaves

sending changes

vnc vs. rdc

master: anode

master: jmaher

switching users

buildbot config files and what they do

config.py

staging-config.py

staging_tests_master_stm{01,02}_localconfig.py

how to run custom talos code

userland code

making a new test master

Newer suites

Landing Your Changes

Are you ready to land

Talos Changes

Dirty Profile Changes or other NPOTB

Pageloder Changes

Navigation menu

Search