SoftwareTesting: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
mNo edit summary
 
(39 intermediate revisions by 7 users not shown)
Line 9: Line 9:
* [[SoftwareTesting:Scratchpad]]
* [[SoftwareTesting:Scratchpad]]
=Projects=
=Projects=
==Testing in General==
We need your help! There is a small but growing set of software tools for testing Firefox functionality. Please see [http://developer.mozilla.org/en/docs/Mozilla_automated_testing MDC:Mozilla_automated_testing] for details on what we have currently.
==Black-Box Testing==
* [[SoftwareTesting:BlackBoxTesting]]
==Software Update Testing==
* [[SoftwareTesting:SoftwareUpdateTesting]]
=Project Ideas=
* Defining a common invocation mechanism for tests, with example code
* Defining a common reporting format for tests
** perl has TAP, xUnit has textrunner (. vs F)
* fixture-based frameworks for test development
** hosting of Fit/Fitnesse to run tests against a browser instance
** tinderbox-based tests
==Areas/concepts to investigate==
* xulUnit
* jsUnit
** xforms tests written using jsUnit
* FireWatir
* jUnit/FIT + javaxpcom - See [http://www.sirgroane.net/2010/09/automated-javascript-unit-tests-with-xpcshell-and-hudson howto integrate xpcshell and hudson]
* pyUnit/pyFIT + pyxpcom
* webdav extension test harness (precursor to some calendar tests)
* jsDriver.pl
** what can we use/re-use
* [[SoftwareTesting:2006-06-26 Brainstorm]]
==Wishlist for xpcshell test harness==
* Ability to mark tests as XFAIL when they are known to fail at the moment
* non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one
=Tools=
=Tools=
* [[SoftwareTesting:Catalog of Automated Tests]] - a list of pointers to existing automated tests in the mozilla codebase
* {{bug|test-harness}} tracking bug for work on mozilla test harnesses
* [[SoftwareTesting:Tools:Simple xpcshell test harness]] - project tracking page
* [[SoftwareTesting:Tools:JsshDriverPython]]
* [[SoftwareTesting:Tools:TestRunnerPython]]
* [http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=&branchtype=match&dir=mozilla%2Ftesting&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=month&mindate=&maxdate=&cvsroot=%2Fcvsroot Updates to Testing Module]
* [http://developer.mozilla.org/samples/domref/dispatchEvent.html Example code] for simulating a click in js
* [http://www.openjsan.org/doc/t/th/theory/Test/Simple/0.21/ JSAN Version] of Perl's Test::Simple and related modules
* {{bug|test-suites}} tracking bug for adding tests to test suites
* [[SoftwareTesting:Tools:BrowserShutdown|How to shut down the browser]]
=People=
=People=
* [[User:Davel|Dave Liebreich]]
* [[User:Rcampbell|Rob Campbell]]
* [http://developer.mozilla.org/en/docs/User:ChrisCooper Chris Cooper]
* [[User:Bhearsum|Ben Hearsum]]
* [[User:Rhelmer|Rob Helmer]]
* [[User:Anodelman|Alice Nodelman]]
* [[User:Ctalbert|Clint Talbert]]
 
=Ideas to Collect=
(stuff that needs to be organised)
* [[SoftwareTesting:Tools:jsUnit]]
* [[SoftwareTesting:Tools:SimpleXulTestHarness]]
* example extension test using foxunit
** davel thinks xul/js-run-as-chrome is probably better
* jsunit in chrome proof-of-concept - [http://lxr.mozilla.org/mozilla/source/toolkit/mozapps/update/src/testnsUpdateService.js file] and Makefile.in in the same directory
* dbaron's [http://lxr.mozilla.org/mozilla/source/layout/tools/reftest/ layout reftest stuff]
** uses copy of domi code to handle command-line args in both firefox and seamonkey
* continuous integration/test issues
** {{bug|351968}} filed to track ENABLE_TEST issues
** build team has decided to avoid spending time getting new things to work with tinderbox
*** we need a tinderbox replacement, so this would be wasted time
** if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added
* aus2 tests using pyFit/fitnesse
** check-in pending from morgamic
[[Category:SoftwareTesting]]

Latest revision as of 23:16, 7 September 2010

In The Ongoing Revolution in Software Testing (pdf), Cem Kaner defines testing as

[...] a technical investigation of a product, done to expose quality-related information.

This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.

Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.

Projects

Testing in General

We need your help! There is a small but growing set of software tools for testing Firefox functionality. Please see MDC:Mozilla_automated_testing for details on what we have currently.

Black-Box Testing

Software Update Testing

Project Ideas

  • Defining a common invocation mechanism for tests, with example code
  • Defining a common reporting format for tests
    • perl has TAP, xUnit has textrunner (. vs F)
  • fixture-based frameworks for test development
    • hosting of Fit/Fitnesse to run tests against a browser instance
    • tinderbox-based tests

Areas/concepts to investigate

Wishlist for xpcshell test harness

  • Ability to mark tests as XFAIL when they are known to fail at the moment
  • non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one

Tools

People

Ideas to Collect

(stuff that needs to be organised)

  • SoftwareTesting:Tools:jsUnit
  • SoftwareTesting:Tools:SimpleXulTestHarness
  • example extension test using foxunit
    • davel thinks xul/js-run-as-chrome is probably better
  • jsunit in chrome proof-of-concept - file and Makefile.in in the same directory
  • dbaron's layout reftest stuff
    • uses copy of domi code to handle command-line args in both firefox and seamonkey
  • continuous integration/test issues
    • bug 351968 filed to track ENABLE_TEST issues
    • build team has decided to avoid spending time getting new things to work with tinderbox
      • we need a tinderbox replacement, so this would be wasted time
    • if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added
  • aus2 tests using pyFit/fitnesse
    • check-in pending from morgamic
Retrieved from "https://wiki.allizom.org/index.php?title=SoftwareTesting&oldid=251570"