AMO:Editors/EditorGuide/ThemeReviews

Firefox Theme Tests

Overview

These review guidelines are in addition to standard add-on review guidelines found at: https://wiki.mozilla.org/AMO:Editors/EditorGuide/AddonReviews

• Testing should be done on a new profile or a profile dedicated to testing themes. With a dedicated profile you can speed up testing by setting the following general startup options:
  1. When Firefox starts: Show my windows and tabs from the last time
  2. Check: Don't load tabs until selected
  3. Set the about:config and about:permissions settings as instructed in "Test Pages" below
• Open each of the pages listed below in the "Test Pages" section in its own tab with about:addons being a pinned tab. Do not close tabs between sessions (item #1 above).
• Install the extensions DOM Inspector and Console² in test profile.
• Disable any extensions that may be installed in test profile (other than DOM Inspector and Console²).
• Themes have limited capabilities to run code, which generally not allowed. Make sure to run the validator to see if there is any JS in the theme. If there is, it should be flagged for admin review and/or reviewed as if it were an extension.
• Make sure the theme looks good. Text should be easy to read and buttons should be clear and concise.
• As well as just looking stylish, make sure it all looks stylish in a consistent way.
• Make sure stuff "fits." This is relevant for menus and dialogs - check to make sure that all the UI elements fit within the default boundaries and do not appear "clipped". If anything's clipped and/or if resizing is required to see it completely, make a note of it and request the author to address the issue.
• While testing a theme, don't forget to check that the author has included a suitable preview image.

OS Specific Issues

Linux

• Unstyled Firefox Button: On Linux, themes sometimes have improperly or totally unstyled Firefox button, and/or the Firefox button not changing styling when in private browsing mode. The primary cause of this issue is that the Firefox button has a different ID in Linux from Windows. As a result, in the browser.css file any reference to #appmenu-button also needs a comparable reference to #appmenu-toolbar-button.
• Strong Black borders Around URL Bar Door Hangers (drop down panels): Many Windows based themes make use of the box-shadow style rule, however, it is not supported on Linux Firefox and resulting in the strong black border. If the strong black border is noticed, it should be noted to the developer and they should be encouraged to make sure box-shadow style rules only target Windows.

OS-X

If you test any theme that is supposed to support OS-X please make sure that if the following files exist in "chrome://browser/skin/" they were also copied to "chrome://browser/skin/lion/":

• keyhole-circle.png
• Toolbar.png
• toolbarbutton-dropmarker.png
• tabbrowser/alltabs-box-bkgnd-icon.png
• tabview/tabview.png
• places/toolbar.png

The other solution theme developers can do is rename the files to something non-default and then make the proper file mappings in the theme's CSS files. NOTE: This issue doesn't require OS-X to test, you only need to look in the theme contents (via the "contents" or "compare" links) to see if the files in question exist. For more information see Bug 679708 and Bug 702558

Win7 w/Aero Glass Support

Win7 Not going to full screen properly

Any theme that supports Aero Glass transparency needs to be tested going from a normal window to full screen.Often times when this is tried, the browser window will disappear or not go to full screen properly. When this happens, developers need to be provided the following note. See Bug 732757 and this MozillaZine thread for more information.

On Win7 with Aero Glass support Firefox doesn't always go to full screen mode from a normal window properly.
The resolution to this issue is to add the following code to your browser.css file somewhere below where the 
main-window is made transparent to support Aero glass. For more information about this issue please see 
https://bugzilla.mozilla.org/show_bug.cgi?id=732757 and http://forums.mozillazine.org/viewtopic.php?f=18&t=2438459.
++++++++++
@media all and (-moz-windows-compositor) {
  /* Make transition to Fullscreen mode seamlessly in Firefox 10+ */
  #main-window[inFullscreen="true"] {
    -moz-appearance: none;
     background-color: -moz-dialog!important;
  }
}
+++++++++

Win7 Missing right hand titlebar control box when menu bar is disabled

When tabs are on top and the menubar is disabled, Firefox is missing the control box (the min/max/restore/close button on the right side of the title bar). Please see the following threads on MozillaZine for solutions to this issue: http://forums.mozillazine.org/viewtopic.php?f=18&t=2131121 http://forums.mozillazine.org/viewtopic.php?f=18&t=1953371&start=60

Special Considerations

Windows and OS-X use different CSS rules to generate OS native scrollbars. As a result any cross OS theme either needs to target different scrollbar rules for each OS or use custom XUL scrollbars.

Error Console

While reviewing themes, watch the error console for any CSS warnings that are flagged for chrome paths containing the word "skin". Warnings should be copied from the error console and pasted into review notes for the developer to investigate. Generally, error console warnings about CSS issues should NOT be grounds for withholding a full review of a theme. However, providing notes to developers about CSS issues can be tremendously helpful to theme development.

Test Dialogs/Features

Please note that testing of some dialogs/features is put off until the webpage tests below. Policies on failure are for worst case failure. Minor styling issues should just get notes.

Dialog/Feature	Policy on failure*	What to test
'Edit > Find'	Preliminary Review	Make sure find toolbar is styled appropriately and functions correctly.
View > Toolbars > Customize	Preliminary Review Preliminary Review Add Note	Make sure customize toolbar dialog is formatted reasonably. With the "show" options set to "icons", check and uncheck "use small icons", toolbar icons needs to display properly in both states (ideally they will be different sizes). Repeat test using show "icons and text". With "use small icons" checked, change "show" to "text". Make sure toolbar button height is reasonable for text only buttons and that the text is vertically aligned in middle of buttons. If it is not provide this note or something similar in the review: When "use small icons is checked and "show" is set to "text" in customize toolbars, text labels in toolbar buttons are not aligning properly. The common cause of this is style rules that reference [iconsize="small"] without excluding [mode="text"]. The solution usually is to append :not([mode="text"]) to those style rules. For instance use 'toolbar[iconsize="small"]:not([mode="text"])'. A little experimenting may be required to fully address this issue with your theme.
Check 'View > Toolbars > Bookmarks Toolbar'	Preliminary Review	Make sure toolbar is formatted reasonably.
Check 'View > Toolbars > Add-on Toolbar'	Preliminary Review	Using customize toolbars, add a toolbar button to this toolbar (at the bottom of the browser window). Make sure toolbar is formatted reasonably.
Check 'View > Toolbars > Tabs on Top' and uncheck 'View > Toolbars > Tabs on Top'	Preliminary Review Preliminary Review	Make sure Firefox button is formatted reasonably. On Windows, with browser window maximized, make sure tabs display properly between Firefox button and right hand window control buttons. Go to private browsing mode (Firefox > Start Private Browsing), and verify that Firefox button changes appearance when in private browsing mode on Windows. Exit private browsing mode, enable menu bar (check Firefox > Options/Preferences > Menu Bar) and uncheck "tabs on top". NOTE: As of FF15 the "tabs on top" menu item has been removed. To do this test, go to about:config and toggle "browser.tabs.onTop" between true and false.
'View > Sidebar > Bookmarks' and 'View > Sidebar > History'	Preliminary Review	Make sure sidebar is formatted appropriately.
'History > Show All History'	Preliminary Review	Make sure history/bookmarks library is formatted appropriately. Drill in to a specific bookmark/history item. Test drop down menus and forward/back buttons.
'Tools > Set Up Sync'	Preliminary Review	Work through the sync dialog as if you are creating a new account and connecting to a device completion of account creation and connecting not necessary). Make sure Sync is formatted appropriately and things seem to work.
Web Developer Tools	Add Note	Make sure the following dialogs under Tools > Web Developer are styled and functioning correctly. Review these tools using the default theme to see their default look and learn how they should function. Developer Toolbar (FF16+) Style Inspector (aka "Inspect") Responsive Web Design (FF15+) Debugger (FF15+) Style Editor (FF11+) Page Source
Windows: 'Tools > Options' Linux: 'Edit > Preferences'	Add Note	Make sure all tabs are formatted and functioning correctly.
Help > About	Preliminary Review	Make sure it is styled appropriately and MUST use an appropriate Firefox logo.

Identity Boxes

Identity boxes are a critical user facing security feature that must be carefully inspected to make sure they function properly. Broken identity boxes are considered security failures and should result in rejection or if the failure is minor a preliminary review. As of Firefox v14.0 the identity box was significantly redesigned. As a result themes that support pre-FF14 and FF14+ need to support the appropriate identity box styling for supported versions of Firefox.

For each style of identity box, make sure to click on the identity box icon to make sure the drop down door hanger is styled reasonably and contains the appropriate icons.

Pre-FF14 (e.g. FF10 ESR)

For testing purposes, it is recommended to have Firefox 10 ESR installed.

The background color of the identity box needs to be significantly different between the two secure HTTPS states, broken HTTPS and HTTP. Also, the background colors of the two secure HTTPS identity boxes needs to be different from the background color of the rest of the URL bar for broken HTTPS and HTTP URLS. It is strongly encouraged, but not required to use green colors for extended validation HTTPS sites and blue for basic validation HTTPS sites.

Security Level	What to test
HTTPS Extended Identity	Test URL: https://addons.mozilla.org/ Default background color: Green Identity box text: Company Name Requirement: Background color must be unique from HTTPS basic identity, broken HTTPS and HTTP identity boxes. Must sufficiently stand out from rest of theme. Sample screen capture:
HTTPS Basic Identity	Test URL: https://bugzilla.mozilla.org/ Default background color: blue Identity box text: domain name Requirement: Background color must be unique from HTTPS extended identity, broken HTTPS and HTTP identity boxes. Must sufficiently stand out from rest of theme. Sample screen capture:
HTTPS Mixed Content	Test URL: https://people.mozilla.com/~bsterne/tests/62178/test.html Default background color: light grey Identity box text: none Requirement: Background color must be different from HTTPS extended identity and HTTPS basic identity. Sample screen capture:
HTTP Unsecured Connection	Test URL: http://www.mozilla.org Default background color: light grey Identity box text: none Requirement: Background color must be different from HTTPS extended identity and HTTPS basic identity. Sample screen capture:

FF14+

As of FF14 the identity box has significantly changed. The biggest change is that a website's favicon is no longer displayed in the identity box, rather they are only displayed on the tab. Although the basic colors used for HTTPS extended validation has remained the same, the way the colors are used has changed, Now instead of white text on a dark background, light backgrounds with dark text and borders are used. Another change is that the domain name text has been removed from the basic validation identity box.

Provided the appropriate identity box icon is used (e.g. a padlock for secured sites) themes may continue to use the "old style" dark backgrounds with light text in the identity box or switch to light colored backgrounds with dark text and borders. The important thing is that the identity box state for extended validation must be distinct from the other states and stand out from rest of the theme. Also if dark backgrounds are used, the identity box icons need to be light in color so as to stand out against the dark background. As before it is strongly encouraged, but not required to use green colors for extended validation HTTPS sites. As of Firefox v14.0 basic validation identity box is no longer required to be styled differently from the standard HTTP identity box other than the fact that it must utilize a padlock icon. It is permissible to continue styling the basic validation identity box with different colors from the other identity boxes if desired.

Security Level	What to test
HTTPS Extended Identity	Test URL: https://addons.mozilla.org/ Default background color: Very Light Green Default text and border color: Dark Green Identity box text: Company Name Icon: Padlock Requirement: Background, text and/or border colors must be unique from other identity boxes. Must use padlock icon. Must sufficiently stand out from rest of theme. Using the color green is strongly recommended. Sample screen capture:
HTTPS Basic Identity	Test URL: https://bugzilla.mozilla.org/ Default background color: White Default text and border color: Dark Grey Identity box text: None Icon: Padlock Requirement: Must utilize a padlock icon. Sample screen capture:
HTTPS Mixed Content	Test URL: https://people.mozilla.com/~bsterne/tests/62178/test.html Default background color: light grey Identity box text: none Icon: Globe Requirement: Must NOT use secured padlock icon (broken padlock okay). Sample screen capture:
HTTP Unsecured Connection	Test URL: http://www.mozilla.org Default background color: light grey Identity box text: none Icon: Globe Requirement: Background color must be different from HTTPS extended identity box, must NOT use padlock icon of any sort. Sample screen capture:

Test Pages

All of the following URLs should be tested. If you are testing a series of themes, you might want to open each of the links/items below into their own tab and set Firefox startup options to "show my windows and tabs from last time". You might also want to check the option "don't load tabs until selected" as this will help with checking by CSS errors by allowing you to only load "about:addons" after restarting from installing the theme, thus the only CSS errors in the error console should be related to the theme. In regards to policies on failure, just add notes for minor formatting issues rather than rejecting or granting only a preliminary review (unless developer hasn't fixed issue after fair warning).

URL	Policy on failure	What to test
about:addons	Preliminary Review Preliminary Review	Make sure formatting is reasonable. Addon type tabs need distinct icons. Make sure there is a significant difference between enabled and disabled addons. It is strongly encouraged but not required that the icons of disabled addons be greyed out. Pin this tab to make sure pinned tabs work correctly.
about:config	Add Note	Make sure it is formatted correctly. While in about config, set the following keys to aid later testing: geo.enabled=true (bool) browser.syncPromoViewsLeft=1000 (integer) security.warn_viewing_mixed=true (bool) security.warn_viewing_mixed.show_once=false (bool)
about:permissions	Add Note	Make sure it is formatted correctly. While in about:permissions, make sure that if there are entries for them, the following domains have the necessary permissions: addons.mozilla.org: Store password = Allow www.mozilla.org: Share location = Always ask
AMO Addon	Preliminary Review Preliminary Review Preliminary Review Add Note Add Note	Test RSS feed icon. Place RSS button on toolbar and make sure its appearance is different for pages that have RSS feeds from those that don't. If you don't have a live bookmark on your bookmark toolbar, add this page's feed to it. Test print preview. Make sure it is formatted reasonably. Test Fullscreen mode (F11). Make sure it is formatted reasonably and functions correctly. Log in to AMO (using a fake username/password is okay), make sure the save password panel is formatted reasonably and includes an appropriate key icon. On the save password dropdown from the identity box there is a sync "door hanger" it needs a proper border. See Bug 708797 Double click on the bookmark star in the URL bar. The edit bookmark box will appear there is a sync "door hanger" make sure the edit bookmark panel and sync door hanger are formatted reasonably
Mozilla's Mission	Preliminary Review Add Note	Use video controls in embedded HTML5 video to make sure they look reasonable and function correctly. Press play button for video and make sure the play icon is replaced with a pause icon. If it is not provide the following note: The play icon does not change to a pause icon when videos start. To address this issue, change the style rules: ------ .playButton[paused] {...} .muteButton[muted] {...} ------ To: ------ .playButton[paused="true"] {...} .muteButton[muted="true"] {...} ------ You can test this by going to: https://www.mozilla.org/en-US/mission/
Google PageSpeed extension download	Preliminary Review	On page click on "install page speed link and make sure a prompt appears in identity box verifying whether or not to install extension.
Mozilla geolocation Test	Preliminary Review	Click on link labeled "Give it a try" and then the "where am I" button to activate geolocation alert. Make sure it displays correctly.
Mozilla phishing test page	Preliminary Review	Phishing warning test page. Check notification strip after ignoring phishing warning. To reset warning go to history menu and open "show all history" then right mouse click on this page's entry and select "forget about this site".
Mozilla malware test page	Preliminary Review	Malware warning page. Check notification strip after ignoring malware warning. To reset warning go to history menu and open "show all history" then right mouse click on this page's entry and select "forget about this site".
https://63.245.217.20/	Preliminary Review	Untrusted connection alert. Click on identity box to test drop down details.
jar:https://bugzilla.mozilla.org/attachment.cgi?id=288634!/	Preliminary Review	Unsafe Remote File Type. Access denied. Make sure it is formatted reasonably.
Mozilla FTP Archive	Preliminary Review	FTP. Make sure it is formatted reasonably.
chrome://global/content/bindings/textbox.xml	Add Note	Unformatted XML Preview. Make sure it is formatted reasonably.
Additional "about" pages.	Add Note	Make sure the following "about" pages are formatted reasonably: about:sessionrestore about:memory about:license about:robots

Usability Tests

The Themes should be usable without Mouse - widgets have to change their appearance when tabbed through them with the keyboard. (e.g. checkboxes, radio buttons, tabs, should indicate key focus). Many of these tests can be conducted in conjunction with other tests above.

• Tab Strip: Tab through tab strip using Ctrl+Tab. Active tab should be visually different from other tabs.
• Group Tabs: Activate group tabs using Ctrl+Shift+E the highlighted tab should be visually different from other tabs use the up/down/left/right arrows to navigate from tab to tab and then press enter to open the desired tab.
• Menu Bar: Activate the file menu using Alt+F and then use keyboard to navigate through menus and sub-menus. The selected menu item must be visually different and the highlighted text must be legible.
• Places: In the places (History/Bookmarks) panel tab around and use keyboard arrows to navigate. Make sure it is visible what the selected item is.
• Options: Open the options panel and make sure you can navigate around the options panel and that you know what has focus. You must be able to change tabs, and settings from the keyboard.

Next: Special Add-on Types