StandaloneTalos: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(cvsroot has been updated in revision 1.2 generate-tpcomponent.py)
No edit summary
Line 1: Line 1:
= How to set up Talos for testing at home =
= How to set up Talos for testing at home V1.0 =


== First Steps ==
== In This Version ==
# Prerequisites
* tsvg/tgfx/twinopen/ts/tdhtml/tjss/tsunspider/tp
## [http://www.python.org/download/releases/2.5/ Python 2.5]
* includes a small, distributable page set for page load time testing
## [http://pyyaml.org/ Pyyaml]
## [http://sourceforge.net/project/showfiles.php?group_id=78018 PyWin32 Extensions] (if this is to be run on a windows box)
# Get the Talos code
#* Check out talos:  <br><code> cd c:\<br> cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d talos mozilla/testing/performance/talos</code>
# Install the new pageloader (only for browsers not built with --enable-tests flag)
## <code>cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d scripts mozilla/tools/buildbot-configs/testing/talos/perfmaster/scripts/generate-tpcomponent.py</code>
## Run in talos\page_load_test directory with <br><code> python generate-tpcomponent.py </code>
## You should see that <code>chrome\</code> and <code>components\</code> directories have been created, these need to be present in your page_load_test directory for the tests to run
# Get the standalone directory
## [http://people.mozilla.org/~anodelman/standalone.zip standalone.zip]
## unzip in the talos directory
# Set up your local web page set


== How to generate your own page set for local testing ==
== Prerequisites ==
There are two ways of gathering your own web page set for testing with. The first is to use a script using wget/bash to collect and clean web pages, the second is running the page cycler script through a browser that has been proxied and then using the proxy cache as the set. They both have problems in creating a repeatable, re-playable page set as described below.
# [http://www.python.org/download/releases/2.5/ Python 2.5]
# [http://pyyaml.org/ Pyyaml]
# [http://sourceforge.net/project/showfiles.php?group_id=78018 PyWin32 Extensions] (if this is to be run on a windows box)
# [http://people.mozilla.org/~anodelman/standaloneV1_0.zip Standalone V1.0]
# [http://stage.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/ Nightly firefox builds] (grab the appropriate type for your OS)


Why can't Mozilla just give you a web page set for testing?  The re-distribution of copied web pages does not fall under fair use. It would certainly be nice to have everyone testing from the same set of web pages but, for now, the best that we can do is provide the tools to create your own test set.
== Set Up ==
# unzip standaloneV1_0.zip
# unzip/install your nightly firefox build
# in the talos directory
#* edit sample.config, change the path to firefox to be the appropriate path to the firefox executable of the nightly build that you just installed


=== Disclaimer ===
== Testing ==
As a warning, be aware that the web pages lists provided in the standalone directory (proxy_manifest.txt or wget_sitelist.txt) will result in the collection of a lot of pornographic pages.  When creating this list the goal was simply to take the Alexa top 500 for a given day.  If you are uncomfortable viewing this material feel free to create your own list of sites - as long as the format is the same as the given files.
# in the talos directory run 'python run_tests.py sample.config'


=== Using wget/bash ===
== Results ==
# Prerequisites
# results from all the tests should be found in talos/output directory.
## wget css parser
##* http://www.mail-archive.com/wget@sunsite.dk/msg09142.html
## Copy of getpages.sh
##* <code>cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d grabber mozilla/testing/tools/grabber</code>
## Apache
##* You'll need to serve the local pages from a local server
##** Why can't I just do file load?  Isn't that simpler?
##*** No - file load takes a different path through rendering than loading pages through the web.  So, you can use file load but you may not end up with the most meaningful load time results
# Grabbing the pages
## Run in a directory containing both getpages.sh and wget-1.10-css-parser
##* <code>./getpages.sh wget_sitelist.txt wget_manifest.txt</code>
##** wget_sitelist.txt is in the standalone directory
##* a testpages directory will be created, it contains the web page collection
## Change the Apache DocumentRoot to the testpages directory
## In testpages/ you'll find the wget_manifest.txt file
### Edit wget_manifest.txt to reflect correct localhost urls
###* ie, www.mop.com/www.mop.com/index.html => http://localhost/www.mop.com/www.mop.com/index.html
### Move the updated wget_manifest.txt file to your standalone directory
# Pitfalls
## Cleaning the web pages results in broken pages
##* You will have to run the page cycler over the page set to determine which pages have become broken - if the cycler stalls or is unable to continue you remove the page from the set
##** To do this follow the directions for running the tests below, on test stalling/failure remove the page that caused the failure from the wget_manifest.txt file
##* This is tedious work and for a large web page set can take a long time to figure out a series of pages that works consistently
## Pages won't look exactly as they do in the wild
##* Through the cleaning web pages can end up looking odd/incomplete, so you don't get pages looking as they do when served live
# Advantages
## Initial collection of the page set is easy, set the getpages.sh script going and walk away
 
=== Using the proxy server ===
# Prerequisites
## Copy of talos
## Copy of the standalone directory
## Copy of the proxy server
##* <code>cvs -d:pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -d proxyserver mozilla/testing/tools/proxyserver</code>
##* copy proxyserver\proxyserver.py into the standalone directory
# From the talos directory
## Set up the proxy server <br><code> python standalone\proxyserver.py -p 9001</code>
## in standalone\proxy_setup.config change the <code>firefox: firefox/firefox.exe</code> to a valid path for firefox on your machine
## Run the tests once from the talos directory <br><code> python run_tests.py standalone\proxy_setup.config</code>
# Pitfalls
## All pages in the set may not be served successfully, resulting in a stalled cycler.  You'll have to remove the page from the proxy_manifest.txt and give it another try
##* This is tedious work and for a large web page set can take a long time to figure out a series of pages that works consistently
## Some pages are not served correctly from the cache.  Again, these pages will have to be removed from the manifest
##* To do this, follow the steps to run the tests from below, on page stalling/failure remove the page from proxy_manifest.txt and start again
##* More tedious work after the set has been collected to ensure that it cycles correctly
# Advantages
## Pages are served exactly as they would look in the wild (no alterations during page set cleaning/localizing)
## No Apache dependency
 
== Start Testing ==
 
* If you are using a wget/bash generated page set...
*# in standalone\wget_run.config change the <code>firefox: firefox/firefox.exe</code> to a valid path for firefox on your machine
*# Run the tests from the talos directory <br><code> python run_tests.py standalone\wget_run.config</code>
*# All the results are found in the <code>talos\standalone\output</code> directory
*# Use your favorite spreadsheet program to examine the results
* If you are using a proxy server cache...
*# in standalone\proxy_run.config change the <code>firefox: firefox/firefox.exe</code> to a valid path for firefox on your machine
*# Flip the proxy server to only serve pages from its cache <br><code> python standalone\proxyserver.py -p 9001 -l</code>
*# Run the tests from the talos directory <br><code> python run_tests.py standalone\proxy_run.config</code>
*# All the results are found in the <code>talos\standalone\output</code> directory
*# Use your favorite spreadsheet program to examine the results

Revision as of 19:12, 15 February 2008

How to set up Talos for testing at home V1.0

In This Version

  • tsvg/tgfx/twinopen/ts/tdhtml/tjss/tsunspider/tp
  • includes a small, distributable page set for page load time testing

Prerequisites

  1. Python 2.5
  2. Pyyaml
  3. PyWin32 Extensions (if this is to be run on a windows box)
  4. Standalone V1.0
  5. Nightly firefox builds (grab the appropriate type for your OS)

Set Up

  1. unzip standaloneV1_0.zip
  2. unzip/install your nightly firefox build
  3. in the talos directory
    • edit sample.config, change the path to firefox to be the appropriate path to the firefox executable of the nightly build that you just installed

Testing

  1. in the talos directory run 'python run_tests.py sample.config'

Results

  1. results from all the tests should be found in talos/output directory.
Retrieved from "https://wiki.allizom.org/index.php?title=StandaloneTalos&oldid=83684"