SoftwareTesting
In The Ongoing Revolution in Software Testing (pdf), Cem Kaner defines testing as
[...] a technical investigation of a product, done to expose quality-related information.
This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.
Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.
Projects
Testing in General
Dave Liebreich is interested in creating running code to test the mozilla codebase. Not just end-to-end automated tests, but unit tests, module tests, test harnesses, test reporting infrastructure, and anything else that lowers the barrier to creating new code that gives someone feedback.
Dave is also interested in extending and improving the existing testing process. The information received from community testing efforts can be excellent and invaluable; Dave is looking for ways in which existing volunteers can improve their testing skill and skilled testers can join the effort.
Dave needs help. The codebase is huge. The community is huge. The task is huge.
- SoftwareTesting:Watir and Firefox - Writing the code necessary for Watir to drive Firefox
- SoftwareTesting:BBST and the Mozilla Codebase - Using mozilla-based apps and libraries as coursework examples and exercises in A Course in Black Box Testing
- SoftwareTesting:TDD and the Mozilla Codebase - Infrastructure, support, and examples of using TDD in mozilla projects
Black-Box Testing
Software Update Testing
Project Ideas
- Defining a common invocation mechanism for tests, with example code
- Defining a common reporting format for tests
- perl has TAP, xUnit has textrunner (. vs F)
- fixture-based frameworks for test development
- hosting of Fit/Fitnesse to run tests against a browser instance
- tinderbox-based tests
Areas/concepts to investigate
- xulUnit
- jsUnit
- xforms tests written using jsUnit
- FireWatir
- jUnit/FIT + javaxpcom
- pyUnit/pyFIT + pyxpcom
- webdav extension test harness (precursor to some calendar tests)
- jsDriver.pl
- what can we use/re-use
- SoftwareTesting:2006-06-26 Brainstorm
Wishlist for xpcshell test harness
- Unbuffered stdout, to get a useful log file even when the test hangs
- Ability to run just a single test (with output to stdout rather than the logfile)
- In particular, ability to run a single test in a debugger
- Maybe as
make check TEST=test_authentication.js, or as./testone.sh test_authentication.js
- Ability to mark tests as XFAIL when they are known to fail at the moment
- non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one
- print log file on stdout for failed tests, especially useful when tinderbox runs the tests and one fails
Tools
- SoftwareTesting:Catalog of Automated Tests - a list of pointers to existing automated tests in the mozilla codebase
- bug test-harness tracking bug for work on mozilla test harnesses
- SoftwareTesting:Tools:Simple xpcshell test harness - project tracking page
- SoftwareTesting:Tools:JsshDriverPython
- SoftwareTesting:Tools:TestRunnerPython
- Updates to Testing Module
- Example code for simulating a click in js
- JSAN Version of Perl's Test::Simple and related modules
- bug test-suites tracking bug for adding tests to test suites
- How to shut down the browser
People
Ideas to Collect
(stuff that needs to be organised)
- example jsunit-based tests (davel's examples)
- urls in test suite pages are resolved relative to the url of the test runner page, not the url of the test suite page
- can we use chrome urls (with appropriate manifest files) to access the jsunit core js files in a path-independent manner? this means we just have to worry about the test runner page and the test suite/case pages.
- this is only a problem for packaging jsunit-based tests in our tree so that they point to the same core jsunit files