QA

From MozillaWiki
Revision as of 10:31, 21 September 2006 by Mw22 (talk | contribs) (Added picture of mw22)
Jump to navigation Jump to search

Who We Are

QA Blog

Mozilla QA Community

A huge part of what we do is work with the Mozilla community to ensure we ship quality software. To learn more about how to help, please check out the Mozilla QA Community wiki page.

Projects

Firefox/Thunderbird 1.5.0.5

Bon Echo (Firefox 2)

Firefox 3

What We Use

We typically use nightly optimized (non-debug) builds for daily usage. However, we also use the release builds (of course!), as well as older builds when trying to narrow down regression windows.

  • For nightly builds, check out any of the mirrors, then drill down to the <product_name>/nightly/ directory. While you can go to <product_name>/latest-* directories, the problem there is that you don't necessarily known when those builds were made. It's best to access the specific build-date directory (e.g., 2005-03-17-08-trunk), to know what you're grabbing.
  • For older builds not listed in the mirror pages, check out the archives at http://archive.mozilla.org/pub/
  • For release builds, simply go to any of the mirrors and drill down to <product_name>/releases/ and select the appropriate directories for version, platform and locale.

Bugzilla

We depend on Bugzilla for filing and tracking bugs and features. We frequently use the query tools, both the "Advanced Search" and "Find a Specific Bug" queries. With the bug count reaching 300,000, there are a couple ways to see what's been frequently reported and duplicated:

People wanting to help triage bugs can refer to the Mozilla Triage Guide

Litmus

We are currently using Litmus Litmus as a replacement for Testrunner (our previous test management tool). Litmus will build upon the testcase management aspects of Testrunner, adding the searchability, reporting, and trending tools that are critical to our QA work. More information about Litmus development is available in the Litmus wiki.

Talkback

Talkback info has moved to a new location: All Things Talkback.

Development tools

We also use several development tools for tracking changes, especially useful for narrowing down regression windows!

Automation

In an effort to the improve the testing potential of our team, we are constantly investigating ways to leverage automated testing tools to offload some of the regression testing burden from our core testers.

Test Suites and Test Tools

  • Firefox User-based tests
    • Sign into the Litmus test tool, specify Firefox in the System Information and see the top level test groups with the following names:
      • Smoke Test
      • Basic Functional Test (BFT)
      • Full Functional Test (FFT)
      • L10n
  • Thunderbird User-based tests
    • Sign into the Litmus test tool, specify Thunderbird in the System Information and see the top level test groups with the following names:
      • Smoke Test
      • Basic Functional Test (BFT)
      • Full Functional Test (FFT)
      • L10n
  • JS Test Suite
  • MetaDiff - L10n checker
  • Update Checker
  • Download Checker
  • l10n Search Verifier

Sites to scan for Community and User Feedback

Websites for verification

Java Acceptance

http://www.basemetals.com/

Test Cases that need to be modified in Litmus (Tracking Page)

Test Cases Added to Litmus (Tracking Page)

Please use this page to list new test cases that you have added to Litmus.

  • Firefox 1.5. BFTs
  • Firefox 1.5 FFTs
    • 2268 Intel Mac Sound Test/marcia/061506
    • 2269 Launch In Safe Mode/marcia/061506
    • [1] AJAX testing/marcia
    • 2254 Switching tabs using the keyboard (Page Up/Down)/ispiked
    • 2252 Tab focusing behavior/ispiked
    • 2249 Switching tabs with the keyboard (using Ctrl)/ispiked
    • 2250 Switching tabs using the keyboard (using Tab button)/ispiked
    • 2251 Moving tabs with the keyboard/ispiked
    • 2253 Closing tabs with the keyboard/ispiked
  • Firefox 2.0 BFTs
  • Firefox 2.0 FFTs
    • 2267 Intel Mac Sound Test/marcia/061506
    • 2270 Launch in Safe Mode/marcia/061506
    • 2271 Warn if opening too many tabs preference (enabled)/ispiked/061505
    • 2272 Warn if opening too many tabs preference (enabled)/ispiked/061506
    • 2254 Switching tabs using the keyboard (Page Up/Down)/ispiked
    • 2247 Tab focusing behavior/ispiked
    • 2244 Switching tabs with the keyboard (using Ctrl)/ispiked
    • 2245 Switching tabs using the keyboard (using Tab button)/ispiked
    • 2246 Moving tabs with the keyboard/ispiked
    • 2248 Closing tabs with the keyboard/ispiked
    • 2257 Quick Find link and word/ispiked
    • 2256 Open and close the Quick Find bar/ispiked
    • 2258 Quick Find forward and backwards/ispiked
    • 2260 Add a basic microsummary/ispiked
    • 2261 Add a microsummary generator/ispiked
    • 2262 Install a microsummary generator/ispiked
    • 2263 Verify basic microsummary updating/ispiked
    • 2264 Verify microsummary generator updating/ispiked
    • 2265 Verify microsummary generator updating when it uses another page/ispiked
    • 2274 Verify extension and theme context menu/ispiked
    • [2] AJAX testing/marcia
  • Thunderbird 1.5. BFTs
  • Thunderbird 1.5 FFTs
    • 2273 Test new mail sound/ispiked
  • Thunderbird 2.0 BFTs
  • Thunderbird 2.0 FFTs
    • 2259 Test new mail sound/ispiked

Release Data - Firefox

A collection of data from the various releases of Firefox. (currently rather empty, just set up and will fill in holes an time permits 

-tracy)

User Agent strings

Windows:

  • Fx 1.5rc1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
  • Fx 1.5beta2 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
  • Fx 1.5beta1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
  • DP alpha2 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
  • DP alpha1 - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+

Macintosh:

  • Fx 1.5.0.1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1
  • Fx 1.5 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051111 Firefox/1.5
  • Fx 1.5rc1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
  • Fx 1.5beta2 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
  • Fx 1.5beta1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
  • DP alpha2 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
  • DP alpha1 - Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+

Linux:

  • Fx 1.5 rc1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8) Gecko/20051025 Firefox/1.5
  • Fx 1.5 Beta2 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051006 Firefox/1.4.1
  • Fx 1.5 Beta1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b4) Gecko/20050908 Firefox/1.4
  • DP alpha2 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b3) Gecko/20050712 Firefox/1.0+
  • DP alpha1 - Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b2) Gecko/20050531 Firefox/1.0+

Talkback Build IDs

Talkback Firefox Build IDS for 1.5.0.x en-US releases

en-US Release Windows Mac PPC Mac UB Linux
Firefox 1.5 2005111116 2005111116 No UB Release 2005111116
Firefox 1.5.0.1 2006011112 2006011112 No UB Release 2006012415
Firefox 1.5.0.2 2006030804 2006030803 2006032818 2006030812
Firefox 1.5.0.3 2006042618 2006042618 2006042618 2006042618
Firefox 1.5.0.4 2006050817 No PPC release 2006050817 2006050817
Firefox 1.5.0.5 2006071912 No PPC release 2006072310 2006071912
Firefox 1.5.0.6 2006072814 No PPC release 2006072814 2006072814
Firefox 1.5.0.7 2006090918 No PPC release 2006090921 2006090918
Firefox 1.5.0.8 No PPC release

Talkback Firefox Build IDS for 2.0 en-US releases

en-US Release Windows Mac PPC Mac UB Linux
Firefox 2.0a1 2006032100 2006032113 2006032717 2006032100
Firefox 2.0a2 2006051217 No PPC 2006051203 2006051217
Firefox 2.0a3 2006052616 No PPC 2006052616 2006052616
Firefox 2.0b1 2006071020 No PPC 2006071020 2006071020
Firefox 2.0b2 2006082101 No PPC 2006082115 2006082101
Firefox 2.0 RC1 No PPC

Release Data - Thunderbird

A collection of data from the various releases of Thunderbird. (currently rather empty, just set up and will fill in holes an time permits 

-tracy)

User Agent strings

Talkback Build IDs

Talkback Thunderbird Build IDS for en-US releases

en-US Release Windows Mac PPC Mac UB Linux
Thunderbird 1.5 2005120115 2005120115 No UB Release 2005120113
Thunderbird 1.5.0.2 2006030803 2006030804 No UB Release 2006042011
Thunderbird 1.5.0.4 2006051612 2006053016 2006051612 2006051612
Thunderbird 1.5.0.5 2006071912 No PPC release 2006071914 2006071913
Thunderbird 1.5.0.7 2006090918 No PPC release 2006090918 2006090918
Thunderbird 1.5.0.? No PPC release

Tasks for Volunteers

A list of tasks for volunteers interested in helping test mozilla software.

Retrieved from "https://wiki.allizom.org/index.php?title=QA&oldid=34469"