QA/TDAI/Gristmill/Mozmill Tutorial

This is a simple tutorial on how to create a Firefox UI automation script by hand using the Gristmill tool set. Since you're just working inside the UI, you need to install the Mozmill IDE extension (integrated development environment) to write and run the script we'll be creating.

Next, we should decide what to test. After looking over litmus briefly, testcase 5958 looks interesting. Here is the testcase:

1. Windows: Go to Tools | Options. (to exit Options, click the OK button)
   Mac: Go to FireFox | Preferences. (to exit Preferences, click the red (x) in the upper left.)
   Linux: Go to Edit | Preferences. (to exit Preferences, click the Close button)
2. In the Contents panel, uncheck Enable JavaScript, then exit the dialog.
3. Visit <a href="http://randomibis.com/coolclock/">Test Clock</a>.

Results -- There should be no clock displayed

I find it is easiest to do these tests walking backwards. So, that's how I'll present this.

Step 1

Fire up the Mozmill UI window from the Tools menu. And let's go visit the http://randomibis.com/coolclock site to take a look at it.

Once you have the clock site open, click on the "Explorers" menu of the Mozmill dialog. Highlight DOM Explorer and click "Start", now switch back to your browser window, and arrange the Mozmill IDE window and the browser window so that you can see both at the same time. Focus the browser window.

You'll notice that a list of text gets generated in Mozmill as you mouse over things. The tool is figuring out how to get to each element under your mouse pointer as you mouse around.

Mouse over the great big clock at the top of the page. The Mozmill window says: "ID: c5". Double click on the clock, and you'll be focused in the Mozmill window. Use the arrow keys and shift to copy the text from the window and paste it into some temporary document.

This ID is how we will find the big clock and ensure it doesn't exist.

Step 2

We are going to have to get into the preferences window and turn off the 'Enable Javascript' flag. So, let's open up the Preferences (Options on Windows) dialog.

We have to let the Mozmill Tool know that we've changed windows (it is still looking at the browser window). After you have the preference dialog open, click on the Explorers menu of Mozmill, Choose DOM Explorer->Stop. Then restart the explorer: Explorers->DOM Explorer->Start. It will automatically figure out that you are now working with the preference window (because it was the most recent window opened).

Click on the content tab. Hover the mouse over the "Enable Javascript" checkbox. You'll see "ID: enableJavaScript" in the Mozmill Window. Double click the checkbox to focus us back in the mozmill window and use the keyboard to copy that text into your temporary document.

Step 3

So far, we have figured out how to get to our javascript check box and our end result object. Now we need to figure out how to tell if the canvas element (the clock) is not displayed. The ID will still be available, since that is in markup and is not part of the script.

<note> It would be better to change the testcase to have javascript automatically generate content, so that we can easily check for whether or not that content exists. But, since I am hell-bent on making this specific test case work, I am going to hack around the fact that the canvas HTML element does not contain any ability to tell me if it has drawn or not</note>

So, doing some research on the canvas spec, I see that there is a toDataURL call which will output a string representation of the image. We know that the representation of the image will be shorter if the default white canvas is drawn versus the actual clock. And we'll use that property for our verification.

Step 4

Ok, we're ready to write the test. We have figured out:

• How to toggle the pref under test
• How to get to our object under test
• How to verify the expected behavior

So, let's write a test. First we start with the Boilerplate:

var elementslib = {}; 
Components.utils.import('resource://mozmill/modules/elementslib.js', elementslib);
var mozmill = {}; 
Components.utils.import('resource://mozmill/modules/mozmill.js', mozmill);

var test_foo = function(){
 var controller = mozmill.getBrowserController();
 controller.open('http://www.google.com');
 controller.sleep(2000);
 controller.type(new elementslib.Name(controller.window.content.document, 'q'), 'Mozilla');
 controller.click(new elementslib.Name(controller.window.content.document, 'btnG'));
}

We don't need the sample test in there, so let's modify it to be:

var elementslib = {}; 
Components.utils.import('resource://mozmill/modules/elementslib.js', elementslib);
var mozmill = {}; 
Components.utils.import('resource://mozmill/modules/mozmill.js', mozmill);

var test_5958 = function(){
}

That's the nugget that we need to start coding.

Step 6

Our success criteria is that the image URL with javascript on is bigger than the image URL with javascript off. So, the method for automating this is as follows:

1. Go to the page with javascript on
2. Grab the size of that image URL
3. Go turn javascript off
4. Go back to the page and get the size of that image URL

Let's write code! First, we have to get to our page. We also want to insert a "wait" statement so that we can ensure we wait for the entire page to load before proceeding with the test. We now have this:

var elementslib = {}; 
Components.utils.import('resource://mozmill/modules/elementslib.js', elementslib);
var mozmill = {}; 
Components.utils.import('resource://mozmill/modules/mozmill.js', mozmill);

var test_5958 = function(){
  var controller = mozmill.getBrowserController();
  controller.open('http://randomibis.com/coolclock/');
  controller.waitForElement(new elementslib.XPath(controller.window.content.document, '/html/body/p[22]');
}

The waitForElement at the bottom does the following:

• Waits on the element in content (i.e. web page) (that's the controller.window.content.document part).
• And uses the XPath expression /html/body/p[22] to find that element. I got the XPath using the same method we used to get the IDs up above. This element corresponds to the text of the "Note" section at the bottom of the page.

Step 7

Now, we need to grab that data URL. To do that we grab the canvas element, and call that method on its interface and store the value for later use. Here's the code.

var elementslib = {}; 
Components.utils.import('resource://mozmill/modules/elementslib.js', elementslib);
var mozmill = {}; 
Components.utils.import('resource://mozmill/modules/mozmill.js', mozmill);
var results = {};
Components.utils.import('resource://mozmill/modules/results.js', results);

var test_5958 = function(){
  var controller = mozmill.getBrowserController();
  controller.open('http://randomibis.com/coolclock/');
  controller.waitForElement(new elementslib.XPath(controller.window.content.document, '/html/body/p[22]');

  // Grab our canvas element - ID was c5, remember?
  var canvas5 = controller.window.content.document.getElementById('c5');

  // Grab the length of the dataURL
  var dataLengthBefore = canvas5.toDataURL().length;

  // Print out that value so we can see it
  results.write('The data length before turning off JS is: ' + dataLengthBefore);

Note that I also added a mechanism to write out the value too. That will write the value to the output tab in mozmill.

Step 8

Now we need to go turn off Javascript in the preferences window. We will use the mozmill.getPreferencesController shortcut to help us out here.

var elementslib = {}; 
Components.utils.import('resource://mozmill/modules/elementslib.js', elementslib);
var mozmill = {}; 
Components.utils.import('resource://mozmill/modules/mozmill.js', mozmill);
var results = {};
Components.utils.import('resource://mozmill/modules/results.js', results);

var test_5958 = function(){
  var controller = mozmill.getBrowserController();
  controller.open('http://randomibis.com/coolclock/');
  controller.waitForElement(new elementslib.XPath(controller.window.content.document, '/html/body/p[22]');

  // Grab our canvas element - ID was c5, remember?
  var canvas5 = controller.window.content.document.getElementById('c5');

  // Grab the length of the dataURL
  var dataLengthBefore = canvas5.toDataURL().length;

  // Print out that value so we can see it
  results.write('The data length before turning off JS is: ' + dataLengthBefore);

  var prefController = mozmill.getPreferencesController();

  // Click on the Content tab
  prefController.click(new elementslib.Elem( prefController.tabs.Content.button ));
  prefController.sleep(1000);

  // Turn off javascript
  prefController.click(new elementslib.ID(prefController.window.document, 'enableJavaScript'));

This is all pretty self explanatory and has been covered in many of the previous demos (TODO Link). However, there is something that should be noticed. Since the preferences dialog is a chrome window (part of the application) then the way to designate that lookup context for the element library changes. Note that for the preference window we use:

prefController.window.document

We have dropped the "content" object from that object chain since the document we want to search is in chrome space.

Step 9

Let's go back to the browser and reload our page now that we've turned off javascript. We'll actually just load a new page for the site since that's simpler. To do that, we open another page from the controller, just like we did in the beginning.

var elementslib = {}; 
Components.utils.import('resource://mozmill/modules/elementslib.js', elementslib);
var mozmill = {}; 
Components.utils.import('resource://mozmill/modules/mozmill.js', mozmill);
var results = {};
Components.utils.import('resource://mozmill/modules/results.js', results);

var test_5958 = function(){
  var controller = mozmill.getBrowserController();
  controller.open('http://randomibis.com/coolclock/');
  controller.waitForElement(new elementslib.XPath(controller.window.content.document, '/html/body/p[22]');

  // Grab our canvas element - ID was c5, remember?
  var canvas5 = controller.window.content.document.getElementById('c5');

  // Grab the length of the dataURL
  var dataLengthBefore = canvas5.toDataURL().length;

  // Print out that value so we can see it
  results.write('The data length before turning off JS is: ' + dataLengthBefore);

  var prefController = mozmill.getPreferencesController();

  // Click on the Content tab
  prefController.click(new elementslib.Elem( prefController.tabs.Content.button ));
  prefController.sleep(1000);

  // Turn off javascript
  prefController.click(new elementslib.ID(prefController.window.document, 'enableJavaScript'));

  // Get a new page
  controller.open('http://randomibis.com/coolclock/');
  controller.waitForElement(new elementslib.XPath(controller.window.content.document, '/html/body/p[22]');

  // Grab our canvas element - ID is still c5
  var canvas5pass2 = controller.window.content.document.getElementById('c5');

  // Grab the length of the dataURL now that we have turned off javascript
  var dataLengthAfter = canvas5pass2.toDataURL().length;

  // And now make sure the datalength after switching JS off is less than it was before we turned it off
  controller.assertJS(dataLengthAfter < dataLengthBefore);
}

All this part is very similar, except the very last line. On the last line, we assert that the expression 'lengthAfter < lengthBefore' is true. If that is true, then the test will pass. If that fails then the test will fail.

There is a big hole in this test. The visual test requires that nothing be drawn. This just tests that less is drawn when javascript is off. So, as I said earlier, it would be better to use a different web page for this test, but I wanted to automate the litmus test without changing it for example's sake.