StandaloneTalos: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
|||
| Line 1: | Line 1: | ||
= V2 ( | = How to set up Talos for testing at home = | ||
== Prerequisites == | |||
#[http://www.python.org/download/releases/2.5/ Python 2.5] | |||
#[http://pyyaml.org/ Pyyaml] | |||
#[http://sourceforge.net/project/showfiles.php?group_id=78018 PyWin32 Extensions] (if this is to be run on a windows box) | |||
#[http://people.mozilla.org/~anodelman/standaloneV2.zip Standalone V2] | |||
#[http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/ Nightly firefox builds] (grab the appropriate type for your OS) | |||
[dmandelin] When I installed PyWin32 the installer put things in my Windows installation of Python, not my mozilla-build copy. I copied the PyWin32 files from the one site-packages directory to the other and it worked. | |||
== Set Up == | |||
== In This Version == | #unzip standaloneV2.zip | ||
#unzip/install your nightly firefox build | |||
#in the talos directory | |||
#*edit sample.config, change the path to firefox to be the appropriate path to the firefox executable of the nightly build that you just installed | |||
Seriously, really use the command-line unzip instead of the Ubuntu GUI Archive Manager. The latter doesn't extract the archive properly. | |||
== Testing == | |||
#in the talos directory run 'python run_tests.py sample.config' | |||
#*for more ouput run 'python run_tests.py --noisy sample.config' | |||
#*for debug output run 'python run_tests.py --debug sample.config' | |||
#*to dump all results directly to stdout run 'python run_tests.py --screen sample.config' | |||
== Results == | |||
#results from all the tests should be found in talos/output directory. | |||
(On a MacBook Pro, the default test configuration takes about 35 minutes to run with a nightly Firefox 3 build.) | |||
== N900 notes == | |||
* Python does not get installed on your N900 by default. | |||
** Add "deb http://repository.maemo.org/extras/ freemantle free non-free" to your /etc/apt/sources.list file | |||
** "apt-get upgrade" | |||
** "apt-get install python" | |||
* You will need to download [http://pyyaml.org/ Pyyaml] and run "python setup.py install" | |||
= Release Notes = | |||
== V2 (October 7th, 2010) == | |||
=== In This Version === | |||
* now using the mercurial Talos code base. We are free from CVS! | * now using the mercurial Talos code base. We are free from CVS! | ||
= V1.9 (September 24th, 2010) = | == V1.9 (September 24th, 2010) == | ||
== In This Version == | === In This Version === | ||
* addition of permissions.sqlite to base_profiles to allow talos to bypass certain security restrictions to run tests | * addition of permissions.sqlite to base_profiles to allow talos to bypass certain security restrictions to run tests | ||
= V1.8 (August 5th, 2010) = | == V1.8 (August 5th, 2010) == | ||
== In This Version == | === In This Version === | ||
*new accessibility test | *new accessibility test | ||
*switched to pageloader as bundle (https://bugzilla.mozilla.org/show_bug.cgi?id=580698 Switch Talos pageloader to ship in distribution/bundles instead of dropping into components/ and chrome/) | *switched to pageloader as bundle (https://bugzilla.mozilla.org/show_bug.cgi?id=580698 Switch Talos pageloader to ship in distribution/bundles instead of dropping into components/ and chrome/) | ||
= V1.7.2 (June 28th, 2010) = | == V1.7.2 (June 28th, 2010) == | ||
== In This Version == | === In This Version === | ||
*another fix to how pages are loaded so that this works without installing a local web server | *another fix to how pages are loaded so that this works without installing a local web server | ||
= V1.7.1 (June 18th, 2010) = | == V1.7.1 (June 18th, 2010) == | ||
== In This Version == | === In This Version === | ||
* changed so that pages are loaded through file:// - no longer requires installation of a local web server | * changed so that pages are loaded through file:// - no longer requires installation of a local web server | ||
= V1.7 (June 11th, 2010) = | == V1.7 (June 11th, 2010) == | ||
== In This Version == | === In This Version === | ||
* new dromaeo tests | * new dromaeo tests | ||
* new --screen option (for sending all results directly to stdout | * new --screen option (for sending all results directly to stdout | ||
| Line 34: | Line 78: | ||
* talos code stability updates | * talos code stability updates | ||
= V1.6 (September 16th, 2009) = | == V1.6 (September 16th, 2009) == | ||
== In This Version == | === In This Version === | ||
* talos code stability updates | * talos code stability updates | ||
* inclusion of dirty profile tests, though they are off by default | * inclusion of dirty profile tests, though they are off by default | ||
= V1.5 (May 7th, 2009) = | == V1.5 (May 7th, 2009) == | ||
== In This Version == | === In This Version ==== | ||
* ability to run tshutdown tests | * ability to run tshutdown tests | ||
* ability to run firefox-bin (instead of firefox) on mac/linux | * ability to run firefox-bin (instead of firefox) on mac/linux | ||
= V1.4 (January 15th, 2009) = | == V1.4 (January 15th, 2009) == | ||
== In This Version == | === In This Version === | ||
* updated pageloader extension to latest (includes fix to how median is calculated) | * updated pageloader extension to latest (includes fix to how median is calculated) | ||
* tests are now run in the order that they appear in the sample.config | * tests are now run in the order that they appear in the sample.config | ||
* csv output now written out post each individual test | * csv output now written out post each individual test | ||
= V1.3.1 (September 26th, 2008 = | == V1.3.1 (September 26th, 2008 == | ||
== In This Version == | === In This Version === | ||
* updated pageloader extension to latest | * updated pageloader extension to latest | ||
* fixed errors in sample.config introduced in last version | * fixed errors in sample.config introduced in last version | ||
= V1.3 (September 25th, 2008) = | == V1.3 (September 25th, 2008) == | ||
== In This Version == | === In This Version === | ||
* removed some failing web pages from the tp test | * removed some failing web pages from the tp test | ||
* {{bug|454999}} (Standalone talos kills all running Firefox processes) | * {{bug|454999}} (Standalone talos kills all running Firefox processes) | ||
| Line 64: | Line 108: | ||
** better process management, better ability to kill frozen/crashed browsers | ** better process management, better ability to kill frozen/crashed browsers | ||
= V1.2 (May 8th, 2008)= | == V1.2 (May 8th, 2008)== | ||
== In This Version == | === In This Version === | ||
* compliance with {{bug|432321}} (Need to add the browser.EULA.override pref to all Talos profiles) | * compliance with {{bug|432321}} (Need to add the browser.EULA.override pref to all Talos profiles) | ||
= V1.1 (April 17th, 2008)= | == V1.1 (April 17th, 2008)== | ||
== In This Version == | === In This Version === | ||
* compliance with {{bug|424594}} ( have talos comply with new file:// security restrictions) | * compliance with {{bug|424594}} ( have talos comply with new file:// security restrictions) | ||
* new & better process management {{bug|419492}} ( talos needs to ensure that browser is closed at end of testing) | * new & better process management {{bug|419492}} ( talos needs to ensure that browser is closed at end of testing) | ||
= V1.0 (February 14th, 2008) = | == V1.0 (February 14th, 2008) == | ||
== In This Version == | === In This Version === | ||
* tsvg/tgfx/twinopen/ts/tdhtml/tjss/tsunspider/tp | * tsvg/tgfx/twinopen/ts/tdhtml/tjss/tsunspider/tp | ||
* includes a small, distributable page set for page load time testing | * includes a small, distributable page set for page load time testing | ||
Revision as of 22:38, 15 April 2011
How to set up Talos for testing at home
Prerequisites
- Python 2.5
- Pyyaml
- PyWin32 Extensions (if this is to be run on a windows box)
- Standalone V2
- Nightly firefox builds (grab the appropriate type for your OS)
[dmandelin] When I installed PyWin32 the installer put things in my Windows installation of Python, not my mozilla-build copy. I copied the PyWin32 files from the one site-packages directory to the other and it worked.
Set Up
- unzip standaloneV2.zip
- unzip/install your nightly firefox build
- in the talos directory
- edit sample.config, change the path to firefox to be the appropriate path to the firefox executable of the nightly build that you just installed
Seriously, really use the command-line unzip instead of the Ubuntu GUI Archive Manager. The latter doesn't extract the archive properly.
Testing
- in the talos directory run 'python run_tests.py sample.config'
- for more ouput run 'python run_tests.py --noisy sample.config'
- for debug output run 'python run_tests.py --debug sample.config'
- to dump all results directly to stdout run 'python run_tests.py --screen sample.config'
Results
- results from all the tests should be found in talos/output directory.
(On a MacBook Pro, the default test configuration takes about 35 minutes to run with a nightly Firefox 3 build.)
N900 notes
- Python does not get installed on your N900 by default.
- Add "deb http://repository.maemo.org/extras/ freemantle free non-free" to your /etc/apt/sources.list file
- "apt-get upgrade"
- "apt-get install python"
- You will need to download Pyyaml and run "python setup.py install"
Release Notes
V2 (October 7th, 2010)
In This Version
- now using the mercurial Talos code base. We are free from CVS!
V1.9 (September 24th, 2010)
In This Version
- addition of permissions.sqlite to base_profiles to allow talos to bypass certain security restrictions to run tests
V1.8 (August 5th, 2010)
In This Version
- new accessibility test
- switched to pageloader as bundle (https://bugzilla.mozilla.org/show_bug.cgi?id=580698 Switch Talos pageloader to ship in distribution/bundles instead of dropping into components/ and chrome/)
V1.7.2 (June 28th, 2010)
In This Version
- another fix to how pages are loaded so that this works without installing a local web server
V1.7.1 (June 18th, 2010)
In This Version
- changed so that pages are loaded through file:// - no longer requires installation of a local web server
V1.7 (June 11th, 2010)
In This Version
- new dromaeo tests
- new --screen option (for sending all results directly to stdout
- new tscroll test
- talos code stability updates
V1.6 (September 16th, 2009)
In This Version
- talos code stability updates
- inclusion of dirty profile tests, though they are off by default
V1.5 (May 7th, 2009)
In This Version =
- ability to run tshutdown tests
- ability to run firefox-bin (instead of firefox) on mac/linux
V1.4 (January 15th, 2009)
In This Version
- updated pageloader extension to latest (includes fix to how median is calculated)
- tests are now run in the order that they appear in the sample.config
- csv output now written out post each individual test
V1.3.1 (September 26th, 2008
In This Version
- updated pageloader extension to latest
- fixed errors in sample.config introduced in last version
V1.3 (September 25th, 2008)
In This Version
- removed some failing web pages from the tp test
- bug 454999 (Standalone talos kills all running Firefox processes)
- instead of killing off processes warns the user that existing Firefox processes are running and need to be shut down before testing can begin
- not a full fix, but more of a stop gap to stop unexpected browser killing
- updates to talos code
- better process management, better ability to kill frozen/crashed browsers
V1.2 (May 8th, 2008)
In This Version
- compliance with bug 432321 (Need to add the browser.EULA.override pref to all Talos profiles)
V1.1 (April 17th, 2008)
In This Version
- compliance with bug 424594 ( have talos comply with new file:// security restrictions)
- new & better process management bug 419492 ( talos needs to ensure that browser is closed at end of testing)
V1.0 (February 14th, 2008)
In This Version
- tsvg/tgfx/twinopen/ts/tdhtml/tjss/tsunspider/tp
- includes a small, distributable page set for page load time testing