|
|
| (6 intermediate revisions by one other user not shown) |
| Line 1: |
Line 1: |
| = How to set up Talos for testing at home =
| | '''StandaloneTalos is deprecated: https://bugzilla.mozilla.org/show_bug.cgi?id=714659 and see https://wiki.mozilla.org/Buildbot/Talos#Running_locally_-_Source_Code for how to run with modern [http://wiki.mozilla.org/Buildbot/Talos Talos]''' |
| | |
| ''StandaloneTalos is scheduled to be deprecated at some point in time: https://bugzilla.mozilla.org/show_bug.cgi?id=714659'' | |
| | |
| == Prerequisites ==
| |
| | |
| # Python >= 2.5
| |
| #[http://pyyaml.org/ Pyyaml]
| |
| #[http://sourceforge.net/project/showfiles.php?group_id=78018 PyWin32 Extensions] (if this is to be run on a windows box)
| |
| #[http://people.mozilla.org/~anodelman/standaloneV2.1.zip Standalone V2.1]
| |
| #[http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/ Nightly firefox builds] (grab the appropriate type for your OS) | |
| | |
| [dmandelin] When I installed PyWin32 the installer put things in my Windows installation of Python, not my mozilla-build copy. I copied the PyWin32 files from the one site-packages directory to the other and it worked.
| |
| | |
| pyyaml and pywin32 can be installed with:
| |
| | |
| easy_install pyyaml
| |
| easy_install pywin32
| |
| | |
| (jmaher: "easy_install pywin32" didn't work and gave "No local packages or download links found for pywin32" so I used:
| |
| | |
| easy_install http://superb-sea2.dl.sourceforge.net/project/pywin32/pywin32/Build216/pywin32-216.win32-py2.6.exe
| |
| | |
| Note that this URL is python 2.6 specific.)
| |
| | |
| == Set Up ==
| |
| | |
| #unzip standaloneV2.1.zip
| |
| #unzip/install your nightly firefox build
| |
| #in the talos directory
| |
| #*edit sample.config, change the path to firefox to be the appropriate path to the firefox executable of the nightly build that you just installed
| |
| | |
| Seriously, really use the command-line unzip instead of the Ubuntu GUI Archive Manager. The latter doesn't extract the archive properly.
| |
| | |
| == TP ==
| |
| | |
| If running tp you will need to extract the tp pageset package, pageloader.xpi to talos/page_load_test. You will want to install a webserver such as apache to serve the content, using file:// is strongly discourage as we have seen it run significantly slower. We are not sure exactly why this is the case however beware that using file:// to serve the pageset can mask regression. I've seen scores of 300 with a webserver and 500 without [BenWa].
| |
| | |
| == Testing ==
| |
| | |
| #in the talos directory run 'python run_tests.py sample.config'
| |
| #*for more ouput run 'python run_tests.py --noisy sample.config'
| |
| #*for debug output run 'python run_tests.py --debug sample.config'
| |
| #*to dump all results directly to stdout run 'python run_tests.py --screen sample.config'
| |
| | |
| == Results ==
| |
| | |
| #results from all the tests should be found in talos/output directory.
| |
| | |
| (On a MacBook Pro, the default test configuration takes about 35 minutes to run with a nightly Firefox 3 build.)
| |
| | |
| == N900 notes ==
| |
| | |
| * Python does not get installed on your N900 by default.
| |
| ** Add "deb http://repository.maemo.org/extras/ freemantle free non-free" to your /etc/apt/sources.list file
| |
| ** "apt-get upgrade"
| |
| ** "apt-get install python"
| |
| * You will need to download [http://pyyaml.org/ Pyyaml] and run "python setup.py install"
| |
| | |
| = Release Notes =
| |
| | |
| == V2.1 (April 18th, 2011) ==
| |
| === In This Version ===
| |
| * includes dirty profiles for dirty profile tests
| |
| * includes new tpaint tests
| |
| * updated to latest talos code
| |
| | |
| == V2 (October 7th, 2010) ==
| |
| | |
| === In This Version ===
| |
| * now using the mercurial Talos code base. We are free from CVS!
| |
| | |
| == V1.9 (September 24th, 2010) ==
| |
| | |
| === In This Version ===
| |
| | |
| * addition of permissions.sqlite to base_profiles to allow talos to bypass certain security restrictions to run tests
| |
| | |
| == V1.8 (August 5th, 2010) ==
| |
| | |
| === In This Version ===
| |
| | |
| *new accessibility test
| |
| *switched to pageloader as bundle (https://bugzilla.mozilla.org/show_bug.cgi?id=580698 Switch Talos pageloader to ship in distribution/bundles instead of dropping into components/ and chrome/)
| |
| | |
| == V1.7.2 (June 28th, 2010) ==
| |
| | |
| === In This Version ===
| |
| | |
| *another fix to how pages are loaded so that this works without installing a local web server
| |
| | |
| == V1.7.1 (June 18th, 2010) ==
| |
| === In This Version ===
| |
| * changed so that pages are loaded through file:// - no longer requires installation of a local web server
| |
| | |
| == V1.7 (June 11th, 2010) ==
| |
| === In This Version ===
| |
| * new dromaeo tests
| |
| * new --screen option (for sending all results directly to stdout
| |
| * new tscroll test
| |
| * talos code stability updates
| |
| | |
| == V1.6 (September 16th, 2009) ==
| |
| === In This Version ===
| |
| * talos code stability updates
| |
| * inclusion of dirty profile tests, though they are off by default
| |
| | |
| == V1.5 (May 7th, 2009) ==
| |
| === In This Version ====
| |
| * ability to run tshutdown tests
| |
| * ability to run firefox-bin (instead of firefox) on mac/linux
| |
| | |
| == V1.4 (January 15th, 2009) ==
| |
| === In This Version ===
| |
| * updated pageloader extension to latest (includes fix to how median is calculated)
| |
| * tests are now run in the order that they appear in the sample.config
| |
| * csv output now written out post each individual test
| |
| | |
| == V1.3.1 (September 26th, 2008 ==
| |
| === In This Version ===
| |
| * updated pageloader extension to latest
| |
| * fixed errors in sample.config introduced in last version
| |
| | |
| == V1.3 (September 25th, 2008) ==
| |
| === In This Version ===
| |
| * removed some failing web pages from the tp test
| |
| * {{bug|454999}} (Standalone talos kills all running Firefox processes)
| |
| ** instead of killing off processes warns the user that existing Firefox processes are running and need to be shut down before testing can begin
| |
| ** not a full fix, but more of a stop gap to stop unexpected browser killing
| |
| * updates to talos code
| |
| ** better process management, better ability to kill frozen/crashed browsers
| |
| | |
| == V1.2 (May 8th, 2008)==
| |
| === In This Version ===
| |
| * compliance with {{bug|432321}} (Need to add the browser.EULA.override pref to all Talos profiles)
| |
| | |
| == V1.1 (April 17th, 2008)==
| |
| === In This Version ===
| |
| * compliance with {{bug|424594}} ( have talos comply with new file:// security restrictions)
| |
| * new & better process management {{bug|419492}} ( talos needs to ensure that browser is closed at end of testing)
| |
| | |
| == V1.0 (February 14th, 2008) ==
| |
| === In This Version ===
| |
| * tsvg/tgfx/twinopen/ts/tdhtml/tjss/tsunspider/tp
| |
| * includes a small, distributable page set for page load time testing
| |