Jetpack/Testing
|
THIS PAGE IS A WORKING DRAFT | 
|
| The page may be difficult to navigate, and some information on its subject might be incomplete and/or evolving rapidly. If you have any questions or ideas, please add them as a new topic on the discussion page. |
This page explains how to run the SDK's unit tests, and how to interpret the results. It's aimed at people who don't have any familiarity with the SDK, but nevertheless have to run the tests to diagnose any test failures, for example because of regressions introduced by the underlying platform.
First, follow these instructions to download, install, and activate the SDK.
You can run tests using the cfx command-line tool. There's cfx documentation here, and command line help is available by typing:
cfx --help
There are five different test commands:
test - run tests testcfx - test the cfx tool testex - test all example code testpkgs - test all installed packages testall - test whole environment
cfx test
cfx test runs the tests implemented for a single SDK "package". A package is a collection of JavaScript modules, and is defined as such by the presence of a file called "package.json" in its root directory. Among other things, "package.json" includes an string called "tests" which describes where the package's test suites live. If this is not defined, the default is the "tests" or "test" directory under the package root.
So if you navigate to a package's root directory (containing a package.json file) and execute cfx test, cfx will:
- initialize the tests directory using the "tests" value inside "package.json", or the top-level "tests/" or "test/" directory
- load every module it finds in the tests directory whose name starts with "test"
- call each function exported by that module, passing it a test object as an argument.
An add-on is a package, and the SDK's APIs are currently collected into two packages called "addon-kit" and "api-utils" (you can see them under the top-level "packages" directory).
Testing an Add-on
From the SDK root, navigate to "examples/reading-data" and explore the "tests" directory:
(addon-sdk)~/mozilla/addon-sdk/examples > cd reading-data/
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > ls tests
test-main.js
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > more tests/test-main.js
const m = require("main");
const self = require("self");
exports.testReplace = function(test) {
const input = "Hello World";
const output = m.replaceMom(input);
test.assertEqual(output, "Hello Mom");
var callbacks = { quit: function() {} };
// Make sure it doesn't crash...
m.main({ staticArgs: {} }, callbacks);
};
exports.testID = function(test) {
// The ID is randomly generated during tests, so we cannot compare it against
// anything in particular. Just assert that it is not empty.
test.assert(self.id.length > 0);
test.assertEqual(self.data.url("sample.html"),
"resource://reading-data-example-at-jetpack-dot-mozillalabs-dot-com-reading-data-data /sample.html");
};
So this add-on implements two unit tests. You can see that each test expects a test object as an argument, and that test defines functions like assert, assertEqual and timeout functionality like waitUntilDone. To learn more about what you can do with a test object check the api documentation.
If you type cfx test it will execute these two tests. We'll use the -v option to make it show us the test names:
(addon-sdk)~/mozilla/addon-sdk/examples/reading-data > cfx test -v
Using binary at '/Applications/Firefox.app/Contents/MacOS/firefox-bin'.
Using profile at '/var/folders/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmpASJfvx.mozrunner'.
Running tests on Firefox 6.0/Gecko 6.0 ({ec8030f7-c20a-464f-9b0e-13a3a9e97384}) under Darwin/x86_64-gcc3.
info: executing 'test-main.testReplace'
info: pass: a == b == "Hello Mom"
info: My ID is reading-data-example@jetpack.mozillalabs.com
info: executing 'test-main.testID'
info: pass: assertion successful
info: pass: a == b == "resource://reading-data-example-at-jetpack-dot-mozillalabs-dot-com-reading-data-data /sample.html"
3 of 3 tests passed.
OK
Total time: 6.058659 seconds
Program terminated successfully.
Testing an SDK Package
An SDK package like "addon-kit" or "api-utils" is a package too, so the procedure is the same. Executing cfx test from the root of "addon-kit" will execute all the following test suites:
(addon-sdk)~/mozilla/addon-sdk > (addon-sdk)~/mozilla/addon-sdk > cd packages/addon-kit/ (addon-sdk)~/mozilla/addon-sdk/packages/addon-kit > ls tests pagemod-test-helpers.js test-notifications.js test-request.js test-clipboard.js test-page-mod.js test-selection.js test-context-menu.html test-page-worker.js test-simple-storage.js test-context-menu.js test-panel.js test-tabs.js test-hotkeys.js test-passwords.js test-widget.js test-module.js test-private-browsing.js test-windows.js
This isn't ideal if you only want to test a single module, so you can use the -f option to cfx to filter the set of tests to run. This option takes a regex and runs only the test suites whose names match the regex. This example runs only the tests for the "clipboard" module:
(addon-sdk)~/mozilla/addon-sdk/packages/addon-kit > cfx test -f clipboard -v
Using binary at '/Applications/Firefox.app/Contents/MacOS/firefox-bin'.
Using profile at '/var/folders/me/me2B1lFDE0WZCgd33s2OTE+++TU/-Tmp-/tmpVLFIqe.mozrunner'.
Running tests on Firefox 6.0/Gecko 6.0 ({ec8030f7-c20a-464f-9b0e-13a3a9e97384}) under Darwin/x86_64-gcc3.
info: executing 'test-clipboard.testWithNoFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testWithFlavor'
info: pass: assertion successful
info: pass: a == b == "html"
info: pass: a == b == null
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testWithRedundantFlavor'
info: pass: assertion successful
info: pass: a == b == "text"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: pass: a == b == "hello there"
info: executing 'test-clipboard.testNotInFlavor'
info: pass: assertion successful
info: pass: a == b == null
17 of 17 tests passed.
OK
Total time: 2.909257 seconds
Program terminated successfully.
cfx testcfx
cfx testcfx tests the cfx tool itself. cfx is written in Python, and testcfx uses Python's unittest framework.
cfx test code is implemented by a collection of Python modules under "python-lib/cuddlefish/tests". Each module implement defines a subclass of the unittest module's TestCase class, and implements one or more test functions, each of which begins with the string "test". For example, here's what the "test_rdf" module looks like:
(addon-sdk)~/mozilla/addon-sdk > cd python-lib/cuddlefish/tests
(addon-sdk)~/mozilla /addon-sdk/python-lib/cuddlefish/tests > more test_rdf.py
import unittest
import xml.dom.minidom
from cuddlefish import rdf
class RDFTests(unittest.TestCase):
def testBug567660(self):
obj = rdf.RDF()
data = u'\u2026'.encode('utf-8')
x = '<?xml version="1.0" encoding="utf-8"?><blah>%s</blah>' % data
obj.dom = xml.dom.minidom.parseString(x)
self.assertEqual(obj.dom.documentElement.firstChild.nodeValue,
u'\u2026')
self.assertEqual(str(obj).replace("\n",""), x.replace("\n",""))
cfx testcfx builds a TestSuite out of all the test cases it finds, then uses TestTestRunner to execute them.
It also looks for DocTests in the documentation files, and executes any that it finds. But there aren't any, at the moment.
testex
cfx testcfx runs cfx test for all the add-ons which ship with the SDK under the examples/ directory.
testpkgs
cfx testpkgs finds all packages installed under the packages/ directory, and executes cfx test for each package it finds.
testall
cfx testall tests everything: it runs cfx testcfx, cfx testex and cfx testpkgs, in that order.