SoftwareTesting:Tools:SimpleXulTestHarness: Difference between revisions

= description =

davel used this set of scripts to help verify that the default search engine plugins bundled with localised builds of firefox 2.0b2 were as expected.

The wrapper script extract_search.sh expects paths to firefox dmg files on the command line.  It installs firefox from each dmg, creates a new profile, runs the "test" to list the search engine stuff, then deletes the profile and that instance of firefox.

The wrapper script has the following assumptions:

* running on a mac

* APP_DATA_DIR set to ~/Library/Application Support/Firefox

** on my machine, I have a symlink to that dir from my home dir

* SEARCHTEST_DIR set to directory containing all these files

* WORK_DIR set to a writable directory, use to install firefox and hold the newly-created profile

* tests.manifest contains a file: url link to the SEARCHTEST_DIR directory

** hmm, we could make extract_search.sh edit the manifest file on the fly . . .

* nothing is creating or deleting profiles while this script is running

** changes to profiles.ini made by other things while this script is running will be clobbered.

= how to run =

# download contents of [http://people.mozilla.com/~davel/scripts/ this directory] to a directory on your mac

# edit extract_search.sh and tests.manifest, replacing paths with the ones on your machine

# download firefox .dmg files you want to "test"

# invoke extract_search.sh

==example==

   <code>./extract_search.sh "en-US/Firefox 2.0 Beta 2.dmg" "en-GB/Firefox 2.0 Beta 2.dmg"</code>

== output from above example ==

  <code><pre>

1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a

1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-GB:official&client=firefox-a

3|Amazon.co.uk|http://www.amazon.co.uk/s/202-2144193-3391812?ie=UTF8&tag=firefox-uk-21&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search

</pre></code>

= comments =

* in the example, extract_search.sh demonstrates how to create a profile

** 2 steps - CreateProfile makes the directory and prefs.js file, and the first run populates the profile directory with all the other stuff

* the sleep 5 is necessary because firefox (on mac at least) restarts itself during the first-run process. probably extension manager checking for stuff

* installing an extension is not covered by this example

= to do list =

* factor out the generic harness bits from the search engine lister bits

* figure out where/how to check this in to the tree

** as an extension, to populate chrome://tests/ ?

** how can test stuff from other parts of the tree get installed into chrome://tests/

* add invocation hooks

** what test to run

** auto-discovery of tests?

** command -line pointer to test manifest file, like layout reftest?

* add test result reporting

** simple dump statements

** maybe a chrome test runner with progress bar, like jsunit's test runner

* make this work for already-installed firefox

* make this work on windows and linux

* clean up output of search engine lister bits