QA/Firefox3.6/TestPlan:DLL Blocklisting: Difference between revisions

From MozillaWiki
< QA
Jump to navigation Jump to search
 
(24 intermediate revisions by 2 users not shown)
Line 24: Line 24:
*** Tests for blocking a special version of a DLL
*** Tests for blocking a special version of a DLL
**** Backup your default installation
**** Backup your default installation
**** Extract the [https://build.mozilla.org/tryserver-builds/hskupin@mozilla.com-bug524904-blocklist-special_version/ Namoroka] build into the above folder
**** Extract the [https://build.mozilla.org/tryserver-builds/hskupin@mozilla.com-blocklist-version/ Namoroka] build (GDS+npFFAddon.dll) into the above folder
**** Copy the files from the gdsv4 folder into the components folder  
**** Copy the files from the gdsv4 folder into the components folder  
**** Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
**** Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
Line 32: Line 32:
**** Check that the module exists in the profile's compreg.dat
**** Check that the module exists in the profile's compreg.dat
*** Tests for blocking all version of a DLL
*** Tests for blocking all version of a DLL
**** Extract the [https://build.mozilla.org/tryserver-builds/hskupin@mozilla.com-bug524904-testwithGDS/ Namoroka] build (w/o whitelist)
**** Extract the [https://build.mozilla.org/tryserver-builds/hskupin@mozilla.com-blocklist-all/ Namoroka] build (GDS+npFFAddon.dll)
**** Copy the files from the gdsv4 folder into the components folder  
**** Copy the files from the gdsv4 folder into the components folder  
**** Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
**** Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
Line 54: Line 54:
*** Check that no partner builds are not affected by this change
*** Check that no partner builds are not affected by this change
** Update Checks
** Update Checks
*** Add hard blocked modules to Components directory for Fx3.0 and Fx3.5,
*** Blocklist DLL
*** Check that software updates (partial/complete) replace the contents of components.list
**** Add hard blocked modules (LSP's would be good candidates) for Fx3.0 and Fx3.5
**** Does not work at the moment due to {{bug|528623}}
**** Check that none of those modules are loaded after the upgrade
*** Check minor updates from 3.5 -> 3.6
*** Components directory lockdown
*** Check major updates from 3.0 -> 3.6
**** Check that software updates (partial/complete) restores the contents of components.list ({{bug|528457}} needs to be fixed first)
**** Check that a deleted components.list gets restored and all components working as expected
*** Update types to test
**** Check minor updates from 3.5 -> 3.6
**** Check major updates from 3.0 -> 3.6
* Generate [[QA/Firefox3.6/TestPlan:DLL_Blocklisting:3rd-party|list]] of top100 3rd party tools which store files inside the components folder
* Litmus Tests
* Litmus Tests
** Check basic test, which is part of the browser for testing purposes.
** Check basic test, which is part of the browser for testing purposes.
Line 81: Line 86:
** Mac: Activity Monitor
** Mac: Activity Monitor
** Linux: lsof | grep %proc_id%
** Linux: lsof | grep %proc_id%
* Tools for tracking loaded js modules
** All: Venkman (Javascript debugger) for js modules
** All platforms: Venkman (Javascript debugger)
* Other tools
** Modify version information of a DLL: [http://www.angusj.com/resourcehacker/ ResHack]
* Relevant Bugs
* Relevant Bugs
** '''Fixed''' {{bug|524904}}: Add support for generic DLL blocklist [fixed]
** '''Fixed''' {{bug|524904}}: Add support for generic DLL blocklist [fixed]
** '''Fixed''' {{bug|519357}}: Only load known binary components from app directory
** '''Fixed''' {{bug|519357}}: Only load known binary components from app directory
** '''New''' {{bug|525103}}: Generate list of DLLs to Blocklist
** '''Fixed''' {{bug|525103}}: Generate list of DLLs to Blocklist
** '''New''' {{bug|528457}}: Always include components.list to partial/complete updates
** '''Fixed''' {{bug|528457}}: Always include components.list to partial/complete updates
** '''Assigned''' {{bug|528651}}: Component registrations not correctly cached leading to re-registering every component on every startup   
** '''Assigned''' {{bug|528651}}: Component registrations not correctly cached leading to re-registering every component on every startup   
** '''Invalid''' {{bug|528623}}: Changes to components.list are not applied (inconsistent caching in profiles compreg.dat)
** '''Invalid''' {{bug|528623}}: Changes to components.list are not applied (inconsistent caching in profiles compreg.dat)
* Some Examples:
* Some Examples:
** Note: Are there any sample add-ons or programs that are relevant for testing? Contact jorgev for help in regards to add-ons.
** Note: Are there any sample add-ons or programs that are relevant for testing? Contact jorgev for help in regards to add-ons.
** '''<FONT COLOR=RED>WARNING!</FONT>  DOWNLOADING THIS MAY SCREW UP YOUR WHOLE MACHINE!  Be sure to have a way to backup and restore your OS first.''' 
*** [http://www.gamingharbor.com/go.do?a=1849&l=4023 Install npffaddon.dll] malware.  it should install the necessary dlls you need.
*** Filenames: NPFFAddOn.dll & NPFFAddOn.xpt
*** Location: C:\Program Files\Internet Saving Optimizer\3.9.0.4780\FF\components
*** Version: 3.9.0.4780 (0x00030009000012ACULL)


= Test Results=
= Test Results=
Line 105: Line 116:
|-
|-
|Block special version
|Block special version
|32: '''PASS'''
|32: '''PASS''' 64: '''PASS'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''n/a'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''PASS'''
|-
|-
|Block all versions
|Block all versions
|32: '''PASS'''
|32: '''PASS''' 64: '''PASS'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''n/a'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''PASS'''
|-
|-
|Blocklisted modules are not registered
|Blocklisted modules are not registered
|32: '''PASS'''
|32: '''PASS''' 64: '''PASS'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''n/a'''
|32: '''result''' 64: '''result'''
|32: '''PASS''' 64: '''PASS'''
|-
|Test with other extensions (e.g. LSP's)
|32: '''wait'''
|32: '''wait''' 64: '''wait'''
|32: '''wait''' 64: '''wait'''
|}
|}


Line 130: Line 136:
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 300px;" | '''Test Description'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 300px;" | '''Test Description'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''Win XP'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''Win XP'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''Win Vista (32/64)'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 80px;" | '''Win Vista (32/64)'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''Win 7 (32/64)'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 80px;" | '''Win 7 (32/64)'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''Linux (32/64)'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 80px;" | '''Linux (32/64)'''  
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''OS X 10.5'''
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 90px;" | '''OS X 10.5'''
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 110px;" | '''OS X 10.6'''
| style="background: rgb(239, 239, 239) none repeat scroll 0% 0%; -moz-background-clip: border; -moz-background-origin: padding; -moz-background-inline-policy: continuous; width: 110px;" | '''OS X 10.6'''
|-
|-
|Only whitelisted modules are loaded
|Only whitelisted modules are loaded
|32: '''PASS'''
|32: '''PASS''' 64: '''PASS'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''n/a'''
|32: '''result''' 64: '''result'''
|32: '''PASS''' 64: '''PASS'''
|32: '''PASS''' 64: '''result'''
|32: '''PASS''' 64: '''n/a'''
|32: '''result'''
|32: '''PASS''' 64: '''n/a'''
|64: '''result'''
|32: '''n/a''' 64: '''PASS'''
|-
|Test with other extensions
|32: '''PASS'''
|32: '''PASS''' 64: '''result'''
|32: '''result''' 64: '''result'''
|32: '''result''' 64: '''result'''
|32: '''result'''
|64: '''result'''
|}
|}


== Update Testing ==
== Update Testing ==
* The following results have been checked with the Google Desktop software which installs GoogleDektopMozilla.dll inside the components folder. This module should not be loaded after an upgrade.
* Results:
** Minor Update 3.5.5 => 3.6b4: '''PASS''' (OS X, Windows)
** Major Update 3.0.15 => 3.6b4: '''PASS''' (OS X, Windows)

Latest revision as of 17:45, 30 November 2009

« Firefox 3.6 TestPlan

DLL Blocklisting

  • Development Status: - In progress (date)
  • Feature Testing: - In progress (date)
  • Team: - vlad (dev), bsmedberg (dev), tchung (QA), hskupin (QA), juanb

Overview

There have been dangerous DLLs that have found their way into the applications directory and have been causing crashiness. The solution here is to blacklist any dlls that are not expected to run with firefox. Any malware dlls should not load and be displayed in the addon blocklist site.

The second part is to whitelist any accepted components in the applications directory.

Things We Test

List the tests we have where applicable:

  • Unit tests
    • Follow-up on test results on Tinderboxen
  • Manual Tests
    • Blocklist DLL
      • Pre-Requisites
        • Install a Firefox 3.5.5 build into the default location
        • Install Google Desktop Search v5
        • Download the prepared files
        • Remember to delete the compreg.dat file from your profile between each test
      • Tests for blocking a special version of a DLL
        • Backup your default installation
        • Extract the Namoroka build (GDS+npFFAddon.dll) into the above folder
        • Copy the files from the gdsv4 folder into the components folder
        • Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
        • Check that the module does not exist in the profile's compreg.dat
        • Copy the files from the gdsv5 folder into the components folder
        • Start Firefox and check that GoogleDesktopMozilla.dll has been loaded
        • Check that the module exists in the profile's compreg.dat
      • Tests for blocking all version of a DLL
        • Extract the Namoroka build (GDS+npFFAddon.dll)
        • Copy the files from the gdsv4 folder into the components folder
        • Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
        • Check that the module does not exist in the profile's compreg.dat
        • Copy the files from the gdsv5 folder into the components folder
        • Start Firefox and check that GoogleDesktopMozilla.dll hasn't been loaded
        • Check that the module does not exist in the profile's compreg.dat
      • Test with a real extensions / software
        • We need a list of possible LSP's we could test
    • Components directory lockdown
      • Pre-Requisites
        • Download the prepared files
        • Remember to delete the compreg.dat file from your profile between each test
      • Check that only white-listed modules are loaded
        • Place a library (.dll, .so, .dylib) into the components folder and check with the Process Monitor that the library hasn't been loaded
        • Remove 'nsExtensionManager.js' from the components.list and check that Firefox doesn't start anymore (compreg.dat shouldn't list this file)
      • Check against other software which store modules under the components folder
        • We need a list
      • Check that if no components.list file is present all modules get loaded
    • Partner Builds
      • Check that no partner builds are not affected by this change
    • Update Checks
      • Blocklist DLL
        • Add hard blocked modules (LSP's would be good candidates) for Fx3.0 and Fx3.5
        • Check that none of those modules are loaded after the upgrade
      • Components directory lockdown
        • Check that software updates (partial/complete) restores the contents of components.list (bug 528457 needs to be fixed first)
        • Check that a deleted components.list gets restored and all components working as expected
      • Update types to test
        • Check minor updates from 3.5 -> 3.6
        • Check major updates from 3.0 -> 3.6
  • Generate list of top100 3rd party tools which store files inside the components folder
  • Litmus Tests
    • Check basic test, which is part of the browser for testing purposes.

Things We Don't Test

  • application software with malware dlls

Environments

  • Win XP
  • Win Vista (32bit, 64bit)
  • Win 7 (32bit, 64bit)
  • Mac OSX 10.5
  • Mac OSX 10.6
  • Linux (32bit, 64bit)

Discussion

  • Are there other real world examples of bad .dlls out there? GD4 is one to use, but we'd like to diversify.

Reference

  • Tools for tracking loaded modules
    • Windows: Process Explorer
    • Mac: Activity Monitor
    • Linux: lsof | grep %proc_id%
    • All: Venkman (Javascript debugger) for js modules
  • Other tools
    • Modify version information of a DLL: ResHack
  • Relevant Bugs
    • Fixed bug 524904: Add support for generic DLL blocklist [fixed]
    • Fixed bug 519357: Only load known binary components from app directory
    • Fixed bug 525103: Generate list of DLLs to Blocklist
    • Fixed bug 528457: Always include components.list to partial/complete updates
    • Assigned bug 528651: Component registrations not correctly cached leading to re-registering every component on every startup
    • Invalid bug 528623: Changes to components.list are not applied (inconsistent caching in profiles compreg.dat)
  • Some Examples:
    • Note: Are there any sample add-ons or programs that are relevant for testing? Contact jorgev for help in regards to add-ons.
    • WARNING! DOWNLOADING THIS MAY SCREW UP YOUR WHOLE MACHINE! Be sure to have a way to backup and restore your OS first.
      • Install npffaddon.dll malware. it should install the necessary dlls you need.
      • Filenames: NPFFAddOn.dll & NPFFAddOn.xpt
      • Location: C:\Program Files\Internet Saving Optimizer\3.9.0.4780\FF\components
      • Version: 3.9.0.4780 (0x00030009000012ACULL)

Test Results

Notes

  • Other crash found: bug 529292 - GDS causes a crash when profile manager is used

Blocklist DLL

Test Description Win XP Win Vista (32/64) Win 7 (32/64)
Block special version 32: PASS 64: PASS 32: PASS 64: n/a 32: PASS 64: PASS
Block all versions 32: PASS 64: PASS 32: PASS 64: n/a 32: PASS 64: PASS
Blocklisted modules are not registered 32: PASS 64: PASS 32: PASS 64: n/a 32: PASS 64: PASS

Components directory lockdown

Test Description Win XP Win Vista (32/64) Win 7 (32/64) Linux (32/64) OS X 10.5 OS X 10.6
Only whitelisted modules are loaded 32: PASS 64: PASS 32: PASS 64: n/a 32: PASS 64: PASS 32: PASS 64: n/a 32: PASS 64: n/a 32: n/a 64: PASS

Update Testing

  • The following results have been checked with the Google Desktop software which installs GoogleDektopMozilla.dll inside the components folder. This module should not be loaded after an upgrade.
  • Results:
    • Minor Update 3.5.5 => 3.6b4: PASS (OS X, Windows)
    • Major Update 3.0.15 => 3.6b4: PASS (OS X, Windows)
Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3.6/TestPlan:DLL_Blocklisting&oldid=186371"