AMO:Developers/JavaScriptTesting: Difference between revisions
Jump to navigation
Jump to search
m (→Mock Objects) |
No edit summary |
||
(11 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
The [http://github.com/jbalogh/zamboni Zamboni Django app] has quite a bit of JavaScript now for features on the site. We currently don't have any automated tests to | ---- | ||
'''UPDATE: This was a brainstorming page that led to the creation of''' [https://github.com/kumar303/jstestnet JS TestNet] | |||
---- | |||
The [http://github.com/jbalogh/zamboni Zamboni Django app] has quite a bit of JavaScript now for features on the site. We currently don't have any automated tests to run so here are some ideas about how we can make a test suite. | |||
=== Why? === | === Why? === | ||
Line 10: | Line 17: | ||
* Quick tests, easy to run during development | * Quick tests, easy to run during development | ||
* A test environment as close as possible to production, which is mainly the Firefox web browser | * A test environment as close as possible to production, which is mainly the Firefox web browser | ||
** Really? What does that mean in practice (os/browser matrix)? A simpler goal is unit testing (pure JS environment) and integration testing (Fx or other). | |||
** If a pure JS environment was "pretty close" to production then that would be fine. A problematic scenario would be where something worked in the tests but not in production (the browser has its quirks!) --[[User:Kumar303|Kumar303]] 08:45, 16 November 2010 (PST) | |||
* A test suite that can run reliably in [https://hudson.mozilla.org/ CI] and deliver meaningful results | * A test suite that can run reliably in [https://hudson.mozilla.org/ CI] and deliver meaningful results | ||
* the ability to use a DOM since most features involve attaching behavior to the DOM | * the ability to use a DOM since most features involve attaching behavior to the DOM | ||
* We want to create small integration tests that verify one or more widgets, not large functional tests that focus on website behavior (QA writes tests for that) | |||
* User interface tests that do not rely on a server. That is, no actual Ajax requests just mock requests (if needed). | |||
=== Test Runners === | === Test Runners === | ||
==== [http://docs.jquery.com/Qunit QUnit] ==== | |||
*** popular framework, well supported | * Pros | ||
** [http://wiki.commonjs.org/wiki/Unit_Testing/1.0 CommonJS Unit Test] compliant | |||
** popular framework, well supported | |||
** very simple and easy to write tests with | |||
** Runs primarily in a web browser like production | |||
** Can provide visual feedback if necessary (useful for development) | |||
** uses a real DOM | |||
** since tests are written in HTML, Zamboni template logic can be reused in some cases | |||
** has several adapters for the command line | |||
*** [https://github.com/kof/node-qunit/ QUnit in node.js] | |||
*** QUnit with Rhino + [https://github.com/thatcher/env-js env.js] (example in [http://pypi.python.org/pypi/NoseJS#running-javascript-tests NoseJS] and [https://github.com/eroh92/nosejs this fork]) | |||
*** [http://code.google.com/p/js-test-driver/wiki/QUnitAdapter Qunit for JsTestDriver] (Cannot do async testing) | |||
* Cons | |||
** Might be tricky to get working for CI. We could load a single webpage in a web browser VM though and use builtin hooks to get test results | |||
** All user events need to be simulated by triggering the event or otherwise. | |||
** does not fit seamlessly into the current test suite (but maybe with NoseJS?) | |||
* Proof of Concept | |||
** You can clone this branch https://github.com/kumar303/zamboni/tree/new-addon-validator-609355 and open http://127.0.0.1:8000/en-US/firefox/qunit/ | |||
*** At the moment you'll need to pip install https://github.com/kumar303/django-qunit/ | |||
==== [http://ianb.github.com/doctestjs/ doctest.js] ==== | |||
* Pros | |||
** Tests are in a more readable format | |||
** Doubles as documentation | |||
** some builtin utilities for async testing (like wait()) | |||
** runs in a browser, like production | |||
* Cons | |||
** Might be tricky to integrate into CI | |||
==== [https://github.com/caolan/nodeunit nodeunit] ==== | |||
* Pros | |||
** can take advantage of the node.js world (like require statements!) | |||
** nice command line output | |||
** easy for CI | |||
* Cons | |||
** Needs a mock dom (like [https://github.com/tmpvar/jsdom jsdom]?) | |||
** might take some fiddling to get things working outside of the Zamboni template stack (loading JS libraries, etc) | |||
** will it be similar enough to production? (i.e. Firefox) | |||
* Notes | |||
** see also [https://github.com/kof/node-qunit/ node-qunit] (above) | |||
=== Mock Objects === | === Mock Objects === |
Latest revision as of 22:54, 4 August 2011
UPDATE: This was a brainstorming page that led to the creation of JS TestNet
The Zamboni Django app has quite a bit of JavaScript now for features on the site. We currently don't have any automated tests to run so here are some ideas about how we can make a test suite.
Why?
- Tests help to refactor existing code
- Tests make it easier to upgrade libraries like jQuery or external plugins
- It's easier to work on another developer's features without fear when there are tests
- A good testing environment helps to simulate errors and timeouts that can be hard or impossible to test manually
What Do We Want?
- Quick tests, easy to run during development
- A test environment as close as possible to production, which is mainly the Firefox web browser
- Really? What does that mean in practice (os/browser matrix)? A simpler goal is unit testing (pure JS environment) and integration testing (Fx or other).
- If a pure JS environment was "pretty close" to production then that would be fine. A problematic scenario would be where something worked in the tests but not in production (the browser has its quirks!) --Kumar303 08:45, 16 November 2010 (PST)
- A test suite that can run reliably in CI and deliver meaningful results
- the ability to use a DOM since most features involve attaching behavior to the DOM
- We want to create small integration tests that verify one or more widgets, not large functional tests that focus on website behavior (QA writes tests for that)
- User interface tests that do not rely on a server. That is, no actual Ajax requests just mock requests (if needed).
Test Runners
QUnit
- Pros
- CommonJS Unit Test compliant
- popular framework, well supported
- very simple and easy to write tests with
- Runs primarily in a web browser like production
- Can provide visual feedback if necessary (useful for development)
- uses a real DOM
- since tests are written in HTML, Zamboni template logic can be reused in some cases
- has several adapters for the command line
- QUnit in node.js
- QUnit with Rhino + env.js (example in NoseJS and this fork)
- Qunit for JsTestDriver (Cannot do async testing)
- Cons
- Might be tricky to get working for CI. We could load a single webpage in a web browser VM though and use builtin hooks to get test results
- All user events need to be simulated by triggering the event or otherwise.
- does not fit seamlessly into the current test suite (but maybe with NoseJS?)
- Proof of Concept
- You can clone this branch https://github.com/kumar303/zamboni/tree/new-addon-validator-609355 and open http://127.0.0.1:8000/en-US/firefox/qunit/
- At the moment you'll need to pip install https://github.com/kumar303/django-qunit/
- You can clone this branch https://github.com/kumar303/zamboni/tree/new-addon-validator-609355 and open http://127.0.0.1:8000/en-US/firefox/qunit/
doctest.js
- Pros
- Tests are in a more readable format
- Doubles as documentation
- some builtin utilities for async testing (like wait())
- runs in a browser, like production
- Cons
- Might be tricky to integrate into CI
nodeunit
- Pros
- can take advantage of the node.js world (like require statements!)
- nice command line output
- easy for CI
- Cons
- Needs a mock dom (like jsdom?)
- might take some fiddling to get things working outside of the Zamboni template stack (loading JS libraries, etc)
- will it be similar enough to production? (i.e. Firefox)
- Notes
- see also node-qunit (above)
Mock Objects
To simulate errors and not depend on a web server, it makes sense to mock out Ajax requests.
- The mockjax jQuery plugin works well for this