SoftwareTesting:Tools:SimpleXulTestHarness

description

davel used this set of scripts to help verify that the default search engine plugins bundled with localised builds of firefox 2.0b2 were as expected.

The wrapper script extract_search.sh expects paths to firefox dmg files on the command line. It installs firefox from each dmg, creates a new profile, runs the "test" to list the search engine stuff, then deletes the profile and that instance of firefox.

The wrapper script has the following assumptions:

• running on a mac
• APP_DATA_DIR set to ~/Library/Application Support/Firefox
  • on my machine, I have a symlink to that dir from my home dir
• SEARCHTEST_DIR set to directory containing all these files
• WORK_DIR set to a writable directory, use to install firefox and hold the newly-created profile
• tests.manifest contains a file: url link to the SEARCHTEST_DIR directory
  • hmm, we could make extract_search.sh edit the manifest file on the fly . . .
• nothing is creating or deleting profiles while this script is running
  • changes to profiles.ini made by other things while this script is running will be clobbered.

how to run

1. download contents of this directory to a directory on your mac
2. edit extract_search.sh and tests.manifest, replacing paths with the ones on your machine
3. download firefox .dmg files you want to "test"
4. invoke extract_search.sh

example

 ./extract_search.sh "en-US/Firefox 2.0 Beta 2.dmg" "en-GB/Firefox 2.0 Beta 2.dmg"

output from above example

en-US/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-US:official&client=firefox-a
2|Yahoo|http://search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.com|http://www.amazon.com/s/002-3363096-7452819?ie=UTF8&tag=mozilla-20&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
6|eBay|http://search.ebay.com/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300
en-GB/Firefox 2.0 Beta 2.dmg
Agree Y/N? "disk1" unmounted.
"disk1" ejected.
Success: created profile 'sbtestprofile /tmp/profile'
1|Google|http://www.google.com/search?q=foofoo&ie=utf-8&oe=utf-8&rls=org.mozilla:en-GB:official&client=firefox-a
2|Yahoo.co.uk|http://uk.search.yahoo.com/search?p=foofoo&ei=UTF-8&fr=moz2
3|Amazon.co.uk|http://www.amazon.co.uk/s/202-2144193-3391812?ie=UTF8&tag=firefox-uk-21&index=blended&link%5Fcode=qs&field-keywords=foofoo&sourceid=Mozilla-search
### MRJPlugin:  getPluginBundle() here. ###
### MRJPlugin:  CFBundleGetBundleWithIdentifier() succeeded. ###
### MRJPlugin:  CFURLGetFSRef() succeeded. ###
4|Answers.com|http://www.answers.com/main/ntquery?s=foofoo&gwp=13
5|Chambers (UK)|http://www.chambersharrap.co.uk/chambers/features/chref/chref.py/main?query=foofoo&title=21st&sourceid=Mozilla-search
6|Creative Commons|http://search.creativecommons.org/?q=foofoo&sourceid=Mozilla-search
7|eBay.co.uk|http://search.ebay.co.uk/foofoo_W0QQfrppZ50QQfsopZ1QQmaxrecordsreturnedZ300

comments

• in the example, extract_search.sh demonstrates how to create a profile
  • 2 steps - CreateProfile makes the directory and prefs.js file, and the first run populates the profile directory with all the other stuff
• the sleep 5 is necessary because firefox (on mac at least) restarts itself during the first-run process. probably extension manager checking for stuff
• installing an extension is not covered by this example

to do list

• figure out where/how to check this in to the tree
  • as an extension, to populate chrome://tests/ ?
  • how can test stuff from other parts of the tree get installed into chrome://tests/
• add invocation hooks
  • what test to run
  • auto-discovery of tests?
  • command -line pointer to test manifest file, like layout reftest?
• add test result reporting
  • simple dump statements
  • maybe a chrome test runner with progress bar, like jsunit's test runner
• make this work for already-installed firefox
• make this work on windows and linux