Marketplace/Reviewers/FxOSAdd-ons/Testing

FirefoxOS Add-on Review guide

Accessing the Review Tools

• Device dependant
  • Desktop go to https://marketplace.firefox.com/content/ on Desktop
  • FirefoxOS open Marketplace app - if you're logged in choose 'More'|'Settings|'Develop Firefox OS Add-ons'
• Press FirefoxOS Add-ons button
• Press Review Add-ons button
• Default is pending queue
  • sort is newest first (! - its the wrong order currently)
• Add-on name or 'review this add-on' links go to review page

Device Setup

• You need a 2.5 build of FirefoxOS - the one released on branch date around 2 Nov, or as near as you can get (there were last minute features landing)
• Enable reviewer certificates in the settings menu, like with apps.
• You shouldn't need to set any other preferences or enable developer mode

Testing Procedure

• Download the zip file for the latest version (no in-site file viewer)
• Inspect the manifest to see what pages or apps are affected
  • obfuscated code - if its trivial and you can de-obfuscate then fine; if its large or still incomprehensible then we must request source code and obfuscation instructions to match output zip. See security guide obfuscation section for more details.
  • note any syntax errors or invalid properties. Common issues are in canned responses
  • Consider if pattern is appropriate, e.g. matching all urls when it only affects one app or site
• inspect the content scripts
  • read every line and establish what the scripts is doing/changing/inserting
  • if there are further regex or filtering of affected urls inside, again check for appropriateness.
    • url.indexOf('blah')!=-1 is normally bad
• check compliance with the review criteria - especially the security section.
• generally consider quality of the add-on - these will be the first few add-ons that users are able to install. We aren't setting a quality bar, but we can be picky about things that need fixing.
• once code review is done install on device
  • see device setup
  • see Login/Navigating (no 'reviewing' shortcut queue)
• if the add-on just affects websites
  • opening an affected site in the browser app, or the hosted webapp (e.g. Twitter) is enough to test.
• if the add-on affects system
  • we must check disabling:
    • toggle off in settings|Add-ons
    • We don't (currently) require an 100% removal after just disable (the user is warned they should reboot) but the device should still be usable. Either the disable should work cleanly and all add-on changes reverted; or the add-on functionality should completely work until reboot.
  • and uninstall:
    • reboot the device after uninstalling in settings|Add-ons (the disable eventListeners don't get called for uninstall)
    • this check is only for certain add-ons that make preference changes that need to be reverted (e.g. movable homescreen button)
• its good practise to have a clean boot after testing each add-on that affects system anyway.