Auto-tools/Projects/CrossWeave

Introduction

CrossWeave is a new, data-driven automation framework for testing client/server interactions in Weave using a Firefox extension. CrossWeave is distinct from, and complementary to, other test projects for Weave:

• unit tests
• manual client testing
• load testing

CrossWeave does not use the Weave UI, but does use the Weave extension code for all Weave server access; e.g., it imports resource://weave/service.js and uses Weave.Service.sync() to sync with the Weave service. In this way it tests both the Weave client and the Weave server for things such as account access, data synchronization, and conflict resolution.

The CrossWeave project is currently not tied to a specific Weave release; in particular, it will not necessarily be ready in time to be useful for Weave 1.3.

Code location: http://hg.mozilla.org/automation/crossweave/

Dependencies

From what I can glean from my history, dependencies are at least these:

easy_install pytz carrot

hg clone http://hg.mozilla.org/users/clegnitto_mozilla.com/mozillapulse
hg clone http://hg.mozilla.org/automation/pulsebuildmonitor
git clone git://github.com/jonallengriffin/mozrunner.git

cd mozillapulse
python setup.py install
cd ../mozrunner
python setup.py install
cd ../pulsebuildmonitor
python setup.py install

See Also

• CrossWeave Test Format
• Running CrossWeave

How it works

CrossWeave consists of a Firefox extension of the same name, along with a python test runner. The python test runner will read a test file (in JSON format), setup a Firefox profile with the necessary extensions and preferences, then launch Firefox and pass the test file to the CrossWeave extension. The extension will read the test file and perform a series of actions specified therein, such as populating a set of bookmarks, syncing to the weave server, making bookmark modifications, etc.

A test file may contain an arbitrary number of sections, each involving the same or different profiles, so that one test file may be used to test the effect of syncing and modifying a common set of data (from a single Weave account) over a series of different events and clients.

The easiest way to understand this is to look at a simple CrossWeave test file.

setup:
  profiles:
    * profile1
    * profile2
bookmark-list-1:
  menu:
    * uri: http://www.google.com
      title: google.com
      changes:
        title: Google
    * folder: foldera
    * folder: folderb
  menu/foldera:
    * uri: http://www.yahoo.com
      title: testing Yahoo
      changes: 
        location: menu/folderb
bookmark-list-2:
  menu:
    * uri: http://www.google.com
      title: Google
    * folder: foldera
    * folder: folderb
  menu/folderb:
    * uri: http://www.yahoo.com
      title: testing Yahoo
phase1:
  profile: profile1
  actions:
    * add:
        bookmarks: bookmark-list-1
    * sync: all
      wipe-server: true
phase2:
  profile: profile2
  actions:
    * sync: all
    * verify:
        bookmarks: bookmark-list-1
    * modify:
        bookmarks: bookmark-list-1
    * sync: all
phase3:
  profile: profile1
    actions:
      * sync: all
      * verify:
          bookmarks: bookmark-list-2

The effects of this test file will be:

1. Firefox is launched with profile1, the CrossWeave extension adds the two bookmarks specified in the "bookmarks-initial" array, then they are synced to the Weave server. The "wipe-server" property causes CrossWeave to set the firstSync=wipeServer pref, in case the Weave account already contains data. Firefox closes.
2. Firefox is launched with profile2, and all data is synced from the Weave server. The CrossWeave extension verifies that all bookmarks in the "bookmarks-list-1" list are present. Then it modifies those bookmarks by applying the "changes" property to each of them. E.g., the title of the first bookmark is changed from "google.com" to "Google". Next, the changes are synced to the Weave server. Finally, Firefox closes.
3. Firefox is launched with profile1 again, and data is synced from the Weave server. The CrossWeave extension verifies that the bookmarks in "bookmarks-list-2" list are present; i.e., all the changes performed in profile2 have successfully been synced to profile1. Lastly, Firefox closes and the tests ends.

This is a simple case, but CrossWeave will support tests of greater complexity, involving an arbitrary number of phases, actions and clients. Test actions will allow interaction with all types of data supported by Weave.

For a more complex example, see: http://hg.mozilla.org/automation/crossweave/file/f35c4dd23dea/sync.test