SoftwareTesting: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
 
(38 intermediate revisions by 7 users not shown)
Line 10: Line 10:
=Projects=
=Projects=
==Testing in General==
==Testing in General==
[[User:Davel|Dave Liebreich]] is interested in creating running code to test the mozilla codebase.  Not just end-to-end automated tests, but unit tests, module tests, test harnesses, test reporting infrastructure, and anything else that lowers the barrier to creating new code that gives someone feedback.
We need your help! There is a small but growing set of software tools for testing Firefox functionality. Please see [http://developer.mozilla.org/en/docs/Mozilla_automated_testing MDC:Mozilla_automated_testing] for details on what we have currently.
 
==Black-Box Testing==
* [[SoftwareTesting:BlackBoxTesting]]
==Software Update Testing==
* [[SoftwareTesting:SoftwareUpdateTesting]]
 
=Project Ideas=
* Defining a common invocation mechanism for tests, with example code
* Defining a common reporting format for tests
** perl has TAP, xUnit has textrunner (. vs F)
* fixture-based frameworks for test development
** hosting of Fit/Fitnesse to run tests against a browser instance
** tinderbox-based tests
==Areas/concepts to investigate==
* xulUnit
* jsUnit
** xforms tests written using jsUnit
* FireWatir
* jUnit/FIT + javaxpcom - See [http://www.sirgroane.net/2010/09/automated-javascript-unit-tests-with-xpcshell-and-hudson howto integrate xpcshell and hudson]
* pyUnit/pyFIT + pyxpcom
* webdav extension test harness (precursor to some calendar tests)
* jsDriver.pl
** what can we use/re-use
* [[SoftwareTesting:2006-06-26 Brainstorm]]
 
==Wishlist for xpcshell test harness==
* Ability to mark tests as XFAIL when they are known to fail at the moment
* non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one


Dave needs help.  The codebase is huge.  The community is huge.  The task is huge.
=Tools=
=Tools=
* [[SoftwareTesting:Catalog of Automated Tests]] - a list of pointers to existing automated tests in the mozilla codebase
* {{bug|test-harness}} tracking bug for work on mozilla test harnesses
* [[SoftwareTesting:Tools:Simple xpcshell test harness]] - project tracking page
* [[SoftwareTesting:Tools:JsshDriverPython]]
* [[SoftwareTesting:Tools:TestRunnerPython]]
* [http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=&branchtype=match&dir=mozilla%2Ftesting&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=month&mindate=&maxdate=&cvsroot=%2Fcvsroot Updates to Testing Module]
* [http://developer.mozilla.org/samples/domref/dispatchEvent.html Example code] for simulating a click in js
* [http://www.openjsan.org/doc/t/th/theory/Test/Simple/0.21/ JSAN Version] of Perl's Test::Simple and related modules
* {{bug|test-suites}} tracking bug for adding tests to test suites
* [[SoftwareTesting:Tools:BrowserShutdown|How to shut down the browser]]
=People=
=People=
* [[User:Davel|Dave Liebreich]]
* [[User:Rcampbell|Rob Campbell]]
* [http://developer.mozilla.org/en/docs/User:ChrisCooper Chris Cooper]
* [[User:Bhearsum|Ben Hearsum]]
* [[User:Rhelmer|Rob Helmer]]
* [[User:Anodelman|Alice Nodelman]]
* [[User:Ctalbert|Clint Talbert]]
 
=Ideas to Collect=
(stuff that needs to be organised)
* [[SoftwareTesting:Tools:jsUnit]]
* [[SoftwareTesting:Tools:SimpleXulTestHarness]]
* example extension test using foxunit
** davel thinks xul/js-run-as-chrome is probably better
* jsunit in chrome proof-of-concept - [http://lxr.mozilla.org/mozilla/source/toolkit/mozapps/update/src/testnsUpdateService.js file] and Makefile.in in the same directory
* dbaron's [http://lxr.mozilla.org/mozilla/source/layout/tools/reftest/ layout reftest stuff]
** uses copy of domi code to handle command-line args in both firefox and seamonkey
* continuous integration/test issues
** {{bug|351968}} filed to track ENABLE_TEST issues
** build team has decided to avoid spending time getting new things to work with tinderbox
*** we need a tinderbox replacement, so this would be wasted time
** if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added
* aus2 tests using pyFit/fitnesse
** check-in pending from morgamic
[[Category:SoftwareTesting]]

Latest revision as of 23:16, 7 September 2010

In The Ongoing Revolution in Software Testing (pdf), Cem Kaner defines testing as

[...] a technical investigation of a product, done to expose quality-related information.

This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.

Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.

Projects

Testing in General

We need your help! There is a small but growing set of software tools for testing Firefox functionality. Please see MDC:Mozilla_automated_testing for details on what we have currently.

Black-Box Testing

Software Update Testing

Project Ideas

  • Defining a common invocation mechanism for tests, with example code
  • Defining a common reporting format for tests
    • perl has TAP, xUnit has textrunner (. vs F)
  • fixture-based frameworks for test development
    • hosting of Fit/Fitnesse to run tests against a browser instance
    • tinderbox-based tests

Areas/concepts to investigate

Wishlist for xpcshell test harness

  • Ability to mark tests as XFAIL when they are known to fail at the moment
  • non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one

Tools

People

Ideas to Collect

(stuff that needs to be organised)

  • SoftwareTesting:Tools:jsUnit
  • SoftwareTesting:Tools:SimpleXulTestHarness
  • example extension test using foxunit
    • davel thinks xul/js-run-as-chrome is probably better
  • jsunit in chrome proof-of-concept - file and Makefile.in in the same directory
  • dbaron's layout reftest stuff
    • uses copy of domi code to handle command-line args in both firefox and seamonkey
  • continuous integration/test issues
    • bug 351968 filed to track ENABLE_TEST issues
    • build team has decided to avoid spending time getting new things to work with tinderbox
      • we need a tinderbox replacement, so this would be wasted time
    • if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added
  • aus2 tests using pyFit/fitnesse
    • check-in pending from morgamic
Retrieved from "https://wiki.allizom.org/index.php?title=SoftwareTesting&oldid=251570"