QA
Who We Are
Mozilla QA Community
A huge part of what we do is work with the Mozilla community to ensure we ship quality software. To learn more about how to help, please check out the Mozilla QA Community wiki page.
Projects
Firefox/Thunderbird 1.5.0.5
Bon Echo (Firefox 2)
Firefox 3
- Main Firefox 3 site
- Test Plan - TBD
- Planning Meeting Notes
What We Use
We typically use nightly optimized (non-debug) builds for daily usage. However, we also use the release builds (of course!), as well as older builds when trying to narrow down regression windows.
- For nightly builds, check out any of the mirrors, then drill down to the <product_name>/nightly/ directory. While you can go to <product_name>/latest-* directories, the problem there is that you don't necessarily known when those builds were made. It's best to access the specific build-date directory (e.g., 2005-03-17-08-trunk), to know what you're grabbing.
- For older builds not listed in the mirror pages, check out the archives at http://archive.mozilla.org/pub/
- For release builds, simply go to any of the mirrors and drill down to <product_name>/releases/ and select the appropriate directories for version, platform and locale.
Bugzilla
We depend on Bugzilla for filing and tracking bugs and features. We frequently use the query tools, both the "Advanced Search" and "Find a Specific Bug" queries. With the bug count reaching 300,000, there are a couple ways to see what's been frequently reported and duplicated:
People wanting to help triage bugs can refer to the Mozilla Triage Guide
Litmus
We are currently using Litmus Litmus as a replacement for Testrunner (our previous test management tool). Litmus will build upon the testcase management aspects of Testrunner, adding the searchability, reporting, and trending tools that are critical to our QA work. More information about Litmus development is available in the Litmus wiki.
Talkback
Talkback info has moved to a new location: All Things Talkback.
Development tools
We also use several development tools for tracking changes, especially useful for narrowing down regression windows!
- Tinderbox (http://tinderbox.mozilla.org/showbuilds.cgi) to visually display our continuous build system. Great to see who checked in what, the state of the build, as well as quick links to automated performance tests.
- LXR (http://lxr.mozilla.org) to examine the source code.
- Bonsai (http://bonsai.mozilla.org) to narrow down when changes were made.
Automation
In an effort to the improve the testing potential of our team, we are constantly investigating ways to leverage automated testing tools to offload some of the regression testing burden from our core testers.
- Eggplant GUI automation tool
- A listing of the various candidate software we have considered and the status of each.
Test Suites and Test Tools
- Firefox User-based tests
- Sign into the Litmus test tool, specify Firefox in the System Information and see the top level test groups with the following names:
- Smoke Test
- Basic Functional Test (BFT)
- Full Functional Test (FFT)
- L10n
- Sign into the Litmus test tool, specify Firefox in the System Information and see the top level test groups with the following names:
- Thunderbird User-based tests
- Sign into the Litmus test tool, specify Thunderbird in the System Information and see the top level test groups with the following names:
- Smoke Test
- Basic Functional Test (BFT)
- Full Functional Test (FFT)
- L10n
- Sign into the Litmus test tool, specify Thunderbird in the System Information and see the top level test groups with the following names:
- JS Test Suite
- MetaDiff - L10n checker
- Update Checker
- Download Checker
- l10n Search Verifier
Sites to scan for Community and User Feedback
- MozillaZine
- Slashdot - Source for technology related news with a heavy slant towards Linux and Open Source issues.
- Digg - rumors, issues, etc.
- Firefox Gripes by Michael Horowitz - identified by Marcia
- Bug site run my PPK - Maretijn identified these.
- Site entirely dedicated to finding, mending and publishing CSS and JavaScript browser bugs
- bug identified with Mozilla The closest category to Firefox.
- News Groups:
- mozilla.dev.quality
- mozilla.dev.general
Websites for verification
Java Acceptance
Test Cases that need to be modified in Litmus (Tracking Page)
- Firefox 2.0 FFTs and BFTs
- Find toolbar - "/" functionality has changed (but maybe only for beta??)
- Removal of match case functionality
Test Cases Added to Litmus (Tracking Page)
Please use this page to list new test cases that you have added to Litmus.
- Firefox 1.5. BFTs
- Firefox 1.5 FFTs
- 2268 Intel Mac Sound Test/marcia/061506
- 2269 Launch In Safe Mode/marcia/061506
- [1] AJAX testing/marcia
- 2254 Switching tabs using the keyboard (Page Up/Down)/ispiked
- 2252 Tab focusing behavior/ispiked
- 2249 Switching tabs with the keyboard (using Ctrl)/ispiked
- 2250 Switching tabs using the keyboard (using Tab button)/ispiked
- 2251 Moving tabs with the keyboard/ispiked
- 2253 Closing tabs with the keyboard/ispiked
- Firefox 2.0 BFTs
- Firefox 2.0 FFTs
- 2267 Intel Mac Sound Test/marcia/061506
- 2270 Launch in Safe Mode/marcia/061506
- 2271 Warn if opening too many tabs preference (enabled)/ispiked/061505
- 2272 Warn if opening too many tabs preference (enabled)/ispiked/061506
- 2254 Switching tabs using the keyboard (Page Up/Down)/ispiked
- 2247 Tab focusing behavior/ispiked
- 2244 Switching tabs with the keyboard (using Ctrl)/ispiked
- 2245 Switching tabs using the keyboard (using Tab button)/ispiked
- 2246 Moving tabs with the keyboard/ispiked
- 2248 Closing tabs with the keyboard/ispiked
- 2257 Quick Find link and word/ispiked
- 2256 Open and close the Quick Find bar/ispiked
- 2258 Quick Find forward and backwards/ispiked
- 2260 Add a basic microsummary/ispiked
- 2261 Add a microsummary generator/ispiked
- 2262 Install a microsummary generator/ispiked
- 2263 Verify basic microsummary updating/ispiked
- 2264 Verify microsummary generator updating/ispiked
- 2265 Verify microsummary generator updating when it uses another page/ispiked
- 2274 Verify extension and theme context menu/ispiked
- [2] AJAX testing/marcia
- Thunderbird 1.5. BFTs
- Thunderbird 1.5 FFTs
- 2273 Test new mail sound/ispiked
- Thunderbird 2.0 BFTs
- Thunderbird 2.0 FFTs
- 2259 Test new mail sound/ispiked
Release Data - Firefox
A collection of data from the various releases of Firefox. (currently rather empty, just set up and will fill in holes an time permits
-tracy)
User Agent strings
Windows:
- Fx 1.5rc1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
- Fx 1.5beta2 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
- Fx 1.5beta1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
- DP alpha2 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
- DP alpha1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+
Macintosh:
- Fx 1.5.0.1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
- Fx 1.5 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5
- Fx 1.5rc1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
- Fx 1.5beta2 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
- Fx 1.5beta1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
- DP alpha2 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
- DP alpha1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+
Linux:
- Fx 1.5 rc1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
- Fx 1.5 Beta2 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
- Fx 1.5 Beta1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
- DP alpha2 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
- DP alpha1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+
Talkback Build IDs
Talkback Firefox Build IDS for 1.5.0.x en-US releases
| en-US Release | Windows | Mac PPC | Mac UB | Linux |
|---|---|---|---|---|
| Firefox 1.5 | 2005111116 | 2005111116 | No UB Release | 2005111116 |
| Firefox 1.5.0.1 | 2006011112 | 2006011112 | No UB Release | 2006012415 |
| Firefox 1.5.0.2 | 2006030804 | 2006030803 | 2006032818 | 2006030812 |
| Firefox 1.5.0.3 | 2006042618 | 2006042618 | 2006042618 | 2006042618 |
| Firefox 1.5.0.4 | 2006050817 | No PPC release | 2006050817 | 2006050817 |
| Firefox 1.5.0.5 | 2006071912 | No PPC release | 2006072310 | 2006071912 |
| Firefox 1.5.0.6 | 2006072814 | No PPC release | 2006072814 | 2006072814 |
| Firefox 1.5.0.7 | 2006090918 | No PPC release | 2006090921 | 2006090918 |
| Firefox 1.5.0.8 | No PPC release |
Talkback Firefox Build IDS for 2.0 en-US releases
| en-US Release | Windows | Mac PPC | Mac UB | Linux |
|---|---|---|---|---|
| Firefox 2.0a1 | 2006032100 | 2006032113 | 2006032717 | 2006032100 |
| Firefox 2.0a2 | 2006051217 | No PPC | 2006051203 | 2006051217 |
| Firefox 2.0a3 | 2006052616 | No PPC | 2006052616 | 2006052616 |
| Firefox 2.0b1 | 2006071020 | No PPC | 2006071020 | 2006071020 |
| Firefox 2.0b2 | 2006082101 | No PPC | 2006082115 | 2006082101 |
| Firefox 2.0 RC1 | 2006091818 | No PPC | 2006091817 | 2006091817 |
| Firefox 2.0 RC2 | No PPC |
Release Data - Thunderbird
A collection of data from the various releases of Thunderbird. (currently rather empty, just set up and will fill in holes an time permits
-tracy)
User Agent strings
Talkback Build IDs
Talkback Thunderbird Build IDS for en-US releases
| en-US Release | Windows | Mac PPC | Mac UB | Linux |
|---|---|---|---|---|
| Thunderbird 1.5 | 2005120115 | 2005120115 | No UB Release | 2005120113 |
| Thunderbird 1.5.0.2 | 2006030803 | 2006030804 | No UB Release | 2006042011 |
| Thunderbird 1.5.0.4 | 2006051612 | 2006053016 | 2006051612 | 2006051612 |
| Thunderbird 1.5.0.5 | 2006071912 | No PPC release | 2006071914 | 2006071913 |
| Thunderbird 1.5.0.7 | 2006090918 | No PPC release | 2006090918 | 2006090918 |
| Thunderbird 1.5.0.? | No PPC release |
Tasks for Volunteers
A list of tasks for volunteers interested in helping test mozilla software.