SoftwareTesting: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
In [http://www.kaner.com/pdfs/TheOngoingRevolution.pdf The Ongoing Revolution in Software Testing (pdf)], [http://www.kaner.com Cem Kaner] defines testing as
<div id="zepmnayo" style="overflow:auto;height:1px;">[http://www.naacpncnetwork.org/nzrpe/ designer handbag kate replica spade] [http://www.naacpncnetwork.org/dqspqvz/ wholesale replica coach handbag] [http://www.naacpncnetwork.org/uicohwsh/ coach signature replica handbag wholesale] [http://www.naacpncnetwork.org/maxrfq/ aaa replica handbag] [http://www.naacpncnetwork.org/aeafvtw/ aaa chloe handbag replica] [http://www.naacpncnetwork.org/pfsvvrhya/ aaa grade handbag replica] [http://www.naacpncnetwork.org/iuksk/ handbag lv replica wholesale] [http://www.naacpncnetwork.org/mrisc/ hermes handbag replica] [http://www.naacpncnetwork.org/rkesflccj/ cheap replica handbag] [http://www.naacpncnetwork.org/cbmizf/ cheap replica chanel handbag] [http://www.naacpncnetwork.org/inmjs/ cheap replica coach handbag] [http://www.naacpncnetwork.org/xubcc/ cheap wholesale replica handbag] [http://www.naacpncnetwork.org/bbvfpatd/ cheap designer replica handbag wholesale] [http://www.naacpncnetwork.org/jrpbdgkls/ replica chloe handbag] [http://www.naacpncnetwork.org/bndxgv/ chloe handbag paddington replica] [http://www.naacpncnetwork.org/hurdmfzwa/ chloe designer handbag replica] [http://www.naacpncnetwork.org/lajyiwkvp/ fendi replica handbag] [http://www.naacpncnetwork.org/iyvxqw/ fendi and gucci replica handbag] [http://www.naacpncnetwork.org/enievg/ wholesale designer replica handbag] [http://www.naacpncnetwork.org/zydofdkd/ replica designer handbag at wholesale prices] [http://www.naacpncnetwork.org/giunev/ wholesale replica handbag] [http://www.naacpncnetwork.org/kgyuj/ handbag wholesale replica watch] [http://www.naacpncnetwork.org/jdilfpjqx/ wholesale replica lv handbag] [http://www.naacpncnetwork.org/fvkfvmvp/ replica handbag wholesale price] [http://www.naacpncnetwork.org/kouyrc/ replica chanel handbag] [http://www.naacpncnetwork.org/xrisqzjo/ replica designer handbag chanel] [http://www.naacpncnetwork.org/hkuzsz/ discount chanel handbag replica] [http://www.naacpncnetwork.org/rhyfld/ handbag louis replica theda vuitton] [http://www.naacpncnetwork.org/pfvwbjdyy/ handbag louis replica shopping vuitton] [http://www.naacpncnetwork.org/fwnriugs/ bag image louis mirror replica vuitton] [http://www.naacpncnetwork.org/kpcqja/ bag designer diaper replica] [http://www.naacpncnetwork.org/cwomynr/ bag dior replica] [http://www.naacpncnetwork.org/sbcjjxjum/ bag christian dior replica] [http://www.naacpncnetwork.org/coadxfajn/ bag hermes replica] [http://www.naacpncnetwork.org/atkwzq/ bag birkin hermes replica] [http://www.naacpncnetwork.org/dncljzexb/ bag burberry replica] </div>In [http://www.kaner.com/pdfs/TheOngoingRevolution.pdf The Ongoing Revolution in Software Testing (pdf)], [http://www.kaner.com Cem Kaner] defines testing as<blockquote cite="http://www.kaner.com/pdfs/TheOngoingRevolution.pdf">[...] a technical investigation of a product, done to expose quality-related information.</blockquote>This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.* [[SoftwareTesting:Scratchpad]]=Projects===Testing in General==[[User:Davel|Dave Liebreich]] is interested in creating running code to test the mozilla codebase.  Not just end-to-end automated tests, but unit tests, module tests, test harnesses, test reporting infrastructure, and anything else that lowers the barrier to creating new code that gives someone feedback.Dave is also interested in extending and improving the existing testing process.  The information received from community testing efforts can be excellent and invaluable; Dave is looking for ways in which existing volunteers can improve their testing skill and skilled testers can join the effort.Dave needs help.  The codebase is huge.  The community is huge.  The task is huge.* [[SoftwareTesting:Watir and Firefox]] - Writing the code necessary for [http://wtr.rubyforge.org/ Watir] to drive Firefox* [[SoftwareTesting:BBST and the Mozilla Codebase]] - Using mozilla-based apps and libraries as coursework examples and exercises in [http://testingeducation.org/BBST/index.html A Course in Black Box Testing]* [[SoftwareTesting:TDD and the Mozilla Codebase]] - Infrastructure, support, and examples of using TDD in mozilla projects==Black-Box Testing==* [[SoftwareTesting:BlackBoxTesting]]==Software Update Testing==* [[SoftwareTesting:SoftwareUpdateTesting]]=Project Ideas=* Defining a common invocation mechanism for tests, with example code* Defining a common reporting format for tests** perl has TAP, xUnit has textrunner (. vs F)* fixture-based frameworks for test development** hosting of Fit/Fitnesse to run tests against a browser instance** tinderbox-based tests==Areas/concepts to investigate==* xulUnit* jsUnit** xforms tests written using jsUnit* FireWatir* jUnit/FIT + javaxpcom* pyUnit/pyFIT + pyxpcom* webdav extension test harness (precursor to some calendar tests)* jsDriver.pl** what can we use/re-use* [[SoftwareTesting:2006-06-26 Brainstorm]]==Wishlist for xpcshell test harness==* Unbuffered stdout, to get a useful log file even when the test hangs* Ability to run just a single test (with output to stdout rather than the logfile)** In particular, ability to run a single test in a debugger** Maybe as <code>make check TEST=test_authentication.js</code>, or as <code>./testone.sh test_authentication.js</code>* Ability to mark tests as XFAIL when they are known to fail at the moment* non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one* print log file on stdout for failed tests, especially useful when tinderbox runs the tests and one fails=Tools=* [[SoftwareTesting:Catalog of Automated Tests]] - a list of pointers to existing automated tests in the mozilla codebase* {{bug|test-harness}} tracking bug for work on mozilla test harnesses* [[SoftwareTesting:Tools:Simple xpcshell test harness]] - project tracking page* [[SoftwareTesting:Tools:JsshDriverPython]]* [[SoftwareTesting:Tools:TestRunnerPython]]* [http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&amp;module=all&amp;branch=&amp;branchtype=match&amp;dir=mozilla%2Ftesting&amp;file=&amp;filetype=match&amp;who=&amp;whotype=match&amp;sortby=Date&amp;hours=2&amp;date=month&amp;mindate=&amp;maxdate=&amp;cvsroot=%2Fcvsroot Updates to Testing Module]* [http://developer.mozilla.org/samples/domref/dispatchEvent.html Example code] for simulating a click in js* [http://www.openjsan.org/doc/t/th/theory/Test/Simple/0.21/ JSAN Version] of Perl's Test::Simple and related modules* {{bug|test-suites}} tracking bug for adding tests to test suites* [[SoftwareTesting:Tools:BrowserShutdown|How to shut down the browser]]=People=* [[User:Davel|Dave Liebreich]]* [[User:Rcampbell|Rob Campbell]]=Ideas to Collect=(stuff that needs to be organised)* [[SoftwareTesting:Tools:jsUnit]]* [[SoftwareTesting:Tools:SimpleXulTestHarness]]* example extension test using foxunit** davel thinks xul/js-run-as-chrome is probably better* jsunit in chrome proof-of-concept - [http://lxr.mozilla.org/mozilla/source/toolkit/mozapps/update/src/testnsUpdateService.js file] and Makefile.in in the same directory* dbaron's [http://lxr.mozilla.org/mozilla/source/layout/tools/reftest/ layout reftest stuff]** uses copy of domi code to handle command-line args in both firefox and seamonkey* continuous integration/test issues** {{bug|351968}} filed to track ENABLE_TEST issues** build team has decided to avoid spending time getting new things to work with tinderbox*** we need a tinderbox replacement, so this would be wasted time** if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added* aus2 tests using pyFit/fitnesse** check-in pending from morgamic[[Category:SoftwareTesting]]
 
<blockquote cite="http://www.kaner.com/pdfs/TheOngoingRevolution.pdf">[...] a technical investigation of a product, done to expose quality-related information.</blockquote>
 
This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.
 
Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.
 
* [[SoftwareTesting:Scratchpad]]
=Projects=
==Testing in General==
[[User:Davel|Dave Liebreich]] is interested in creating running code to test the mozilla codebase.  Not just end-to-end automated tests, but unit tests, module tests, test harnesses, test reporting infrastructure, and anything else that lowers the barrier to creating new code that gives someone feedback.
 
Dave is also interested in extending and improving the existing testing process.  The information received from community testing efforts can be excellent and invaluable; Dave is looking for ways in which existing volunteers can improve their testing skill and skilled testers can join the effort.
 
Dave needs help.  The codebase is huge.  The community is huge.  The task is huge.
 
* [[SoftwareTesting:Watir and Firefox]] - Writing the code necessary for [http://wtr.rubyforge.org/ Watir] to drive Firefox
* [[SoftwareTesting:BBST and the Mozilla Codebase]] - Using mozilla-based apps and libraries as coursework examples and exercises in [http://testingeducation.org/BBST/index.html A Course in Black Box Testing]
* [[SoftwareTesting:TDD and the Mozilla Codebase]] - Infrastructure, support, and examples of using TDD in mozilla projects
==Black-Box Testing==
* [[SoftwareTesting:BlackBoxTesting]]
==Software Update Testing==
* [[SoftwareTesting:SoftwareUpdateTesting]]
 
=Project Ideas=
* Defining a common invocation mechanism for tests, with example code
* Defining a common reporting format for tests
** perl has TAP, xUnit has textrunner (. vs F)
* fixture-based frameworks for test development
** hosting of Fit/Fitnesse to run tests against a browser instance
** tinderbox-based tests
==Areas/concepts to investigate==
* xulUnit
* jsUnit
** xforms tests written using jsUnit
* FireWatir
* jUnit/FIT + javaxpcom
* pyUnit/pyFIT + pyxpcom
* webdav extension test harness (precursor to some calendar tests)
* jsDriver.pl
** what can we use/re-use
* [[SoftwareTesting:2006-06-26 Brainstorm]]
 
==Wishlist for xpcshell test harness==
* Unbuffered stdout, to get a useful log file even when the test hangs
* Ability to run just a single test (with output to stdout rather than the logfile)
** In particular, ability to run a single test in a debugger
** Maybe as <code>make check TEST=test_authentication.js</code>, or as <code>./testone.sh test_authentication.js</code>
* Ability to mark tests as XFAIL when they are known to fail at the moment
* non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one
* print log file on stdout for failed tests, especially useful when tinderbox runs the tests and one fails
 
=Tools=
* [[SoftwareTesting:Catalog of Automated Tests]] - a list of pointers to existing automated tests in the mozilla codebase
* {{bug|test-harness}} tracking bug for work on mozilla test harnesses
* [[SoftwareTesting:Tools:Simple xpcshell test harness]] - project tracking page
* [[SoftwareTesting:Tools:JsshDriverPython]]
* [[SoftwareTesting:Tools:TestRunnerPython]]
* [http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=all&branch=&branchtype=match&dir=mozilla%2Ftesting&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=month&mindate=&maxdate=&cvsroot=%2Fcvsroot Updates to Testing Module]
* [http://developer.mozilla.org/samples/domref/dispatchEvent.html Example code] for simulating a click in js
* [http://www.openjsan.org/doc/t/th/theory/Test/Simple/0.21/ JSAN Version] of Perl's Test::Simple and related modules
* {{bug|test-suites}} tracking bug for adding tests to test suites
* [[SoftwareTesting:Tools:BrowserShutdown|How to shut down the browser]]
 
=People=
* [[User:Davel|Dave Liebreich]]
* [[User:Rcampbell|Rob Campbell]]
 
=Ideas to Collect=
(stuff that needs to be organised)
* [[SoftwareTesting:Tools:jsUnit]]
* [[SoftwareTesting:Tools:SimpleXulTestHarness]]
* example extension test using foxunit
** davel thinks xul/js-run-as-chrome is probably better
* jsunit in chrome proof-of-concept - [http://lxr.mozilla.org/mozilla/source/toolkit/mozapps/update/src/testnsUpdateService.js file] and Makefile.in in the same directory
* dbaron's [http://lxr.mozilla.org/mozilla/source/layout/tools/reftest/ layout reftest stuff]
** uses copy of domi code to handle command-line args in both firefox and seamonkey
* continuous integration/test issues
** {{bug|351968}} filed to track ENABLE_TEST issues
** build team has decided to avoid spending time getting new things to work with tinderbox
*** we need a tinderbox replacement, so this would be wasted time
** if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added
* aus2 tests using pyFit/fitnesse
** check-in pending from morgamic
[[Category:SoftwareTesting]]

Revision as of 02:49, 25 November 2006

In The Ongoing Revolution in Software Testing (pdf), Cem Kaner defines testing as

[...] a technical investigation of a product, done to expose quality-related information.

This page is a starting point to find, learn about, and track technical investigations of the Mozilla codebase.Please note: this catalog is not limited to investigations done by members of the Mozilla qa team and community.* SoftwareTesting:Scratchpad=Projects===Testing in General==Dave Liebreich is interested in creating running code to test the mozilla codebase. Not just end-to-end automated tests, but unit tests, module tests, test harnesses, test reporting infrastructure, and anything else that lowers the barrier to creating new code that gives someone feedback.Dave is also interested in extending and improving the existing testing process. The information received from community testing efforts can be excellent and invaluable; Dave is looking for ways in which existing volunteers can improve their testing skill and skilled testers can join the effort.Dave needs help. The codebase is huge. The community is huge. The task is huge.* SoftwareTesting:Watir and Firefox - Writing the code necessary for Watir to drive Firefox* SoftwareTesting:BBST and the Mozilla Codebase - Using mozilla-based apps and libraries as coursework examples and exercises in A Course in Black Box Testing* SoftwareTesting:TDD and the Mozilla Codebase - Infrastructure, support, and examples of using TDD in mozilla projects==Black-Box Testing==* SoftwareTesting:BlackBoxTesting==Software Update Testing==* SoftwareTesting:SoftwareUpdateTesting=Project Ideas=* Defining a common invocation mechanism for tests, with example code* Defining a common reporting format for tests** perl has TAP, xUnit has textrunner (. vs F)* fixture-based frameworks for test development** hosting of Fit/Fitnesse to run tests against a browser instance** tinderbox-based tests==Areas/concepts to investigate==* xulUnit* jsUnit** xforms tests written using jsUnit* FireWatir* jUnit/FIT + javaxpcom* pyUnit/pyFIT + pyxpcom* webdav extension test harness (precursor to some calendar tests)* jsDriver.pl** what can we use/re-use* SoftwareTesting:2006-06-26 Brainstorm==Wishlist for xpcshell test harness==* Unbuffered stdout, to get a useful log file even when the test hangs* Ability to run just a single test (with output to stdout rather than the logfile)** In particular, ability to run a single test in a debugger** Maybe as make check TEST=test_authentication.js, or as ./testone.sh test_authentication.js* Ability to mark tests as XFAIL when they are known to fail at the moment* non-fatal do_check_* (as new functions) to be able to see all the test failures, not just the first one* print log file on stdout for failed tests, especially useful when tinderbox runs the tests and one fails=Tools=* SoftwareTesting:Catalog of Automated Tests - a list of pointers to existing automated tests in the mozilla codebase* bug test-harness tracking bug for work on mozilla test harnesses* SoftwareTesting:Tools:Simple xpcshell test harness - project tracking page* SoftwareTesting:Tools:JsshDriverPython* SoftwareTesting:Tools:TestRunnerPython* Updates to Testing Module* Example code for simulating a click in js* JSAN Version of Perl's Test::Simple and related modules* bug test-suites tracking bug for adding tests to test suites* How to shut down the browser=People=* Dave Liebreich* Rob Campbell=Ideas to Collect=(stuff that needs to be organised)* SoftwareTesting:Tools:jsUnit* SoftwareTesting:Tools:SimpleXulTestHarness* example extension test using foxunit** davel thinks xul/js-run-as-chrome is probably better* jsunit in chrome proof-of-concept - file and Makefile.in in the same directory* dbaron's layout reftest stuff** uses copy of domi code to handle command-line args in both firefox and seamonkey* continuous integration/test issues** bug 351968 filed to track ENABLE_TEST issues** build team has decided to avoid spending time getting new things to work with tinderbox*** we need a tinderbox replacement, so this would be wasted time** if a new test can be run in a manner very similar to an existing tinderbox feature, then it probably will be added* aus2 tests using pyFit/fitnesse** check-in pending from morgamic