View source for Add-ons/Reviewers/Guide/Review Decision

= Review Decision Guidelines =

This page provides guidance to support reviewers in making consistent review decisions based on our [https://developer.mozilla.org/en-US/Add-ons/AMO/Policy/Reviews Review Policies]. In addition, it explains how to best communicate with the developer when conducting add-on reviews. It is a supplement to our [[Add-ons/Reviewers/Guide/Reviewing|Reviewer Guide]].

== Completing the Review ==
Add-ons must be reviewed '''completely''', noting issues in the review comments field as you find them. We want to avoid sending multiple partial reviews to developers and give them an opportunity to address all policy violations at once in their next submission.

In the following sections we will explain how canned responses should be used for the issues you find and guide you through the review actions. Finally, you are guided on how to respond if you find issues that cover multiple review actions.

=== Use of Canned Responses ===
To ensure developers receive consistent, actionable responses from the review team, AMO makes use of canned responses that cover many of our policy issues.

If no canned response is available for the issue you have found, you may write a custom response specific to the situation. Keep in mind that when writing a response to the developer, all responses must be in line with our policies. This is especially true when using custom responses. If you are unsure your response conforms with the policy, an admin reviewer will be able to assist you.

In addition, we’d love to hear from you believe the response is useful to add to the official list of canned responses. We would much prefer that reviewers use the canned responses to ensure consistent replies, therefore adding new canned responses could be very useful.

In some cases the developer responds to your review, either via email or by uploading a new version, making clear that they did not fully understand how to act upon the canned response. In such cases, sending the same canned response again will not help resolve the situation. Here it makes sense to explain the policy issue in a different, possibly more verbose way, using your own words.

==== Example ====
While the review comments field is free-form, we have a de-facto standard approach that lists the review issues in numbered lists each with a headline. Note that your text is inserted into an email that handles all the formalities, you should not need to begin your review with “Dear Developer” or sign with your name.

Commonly, we use a numbered list item for each review issue, along with headings to separate sections as needed. Here is an example review text to give you an idea:

<pre>
This add-on didn't pass review because of the following problems:

1) We don't allow add-ons to use remote scripts because they can create serious security vulnerabilities. We also need to review all add-on code, and this makes it much more difficult. Please insert those scripts locally from your add-on code.
- pages/inject.html line 8

In addition, the following information is required to complete the review:

1) Please provide us with detailed information on how to test this add-on. If authentication to a website is necessary, give us a test username and password to facilitate our work. This can be provided in the Whiteboard field, which can be found in the Edit Listing page under the Technical Details section. This information is only available to reviewers.
</pre>

=== Reviewer Replies and Information Requests ===
If you do not have all information required to complete the review, or you have reached one of the cases detailed later on this page, you can request more information from the developer. All communication between developers and reviewers is captured on the review history page.

To do so, select the “Reviewer Reply” action. For requesting more information, please also ensure that you correctly set the “Require developer to respond” checkbox: if you have a question or comment where the developer must reply, please check the box. If it is an answer to the developer’s question that requires no further action, leave it unchecked. Doing so makes sure the correct email template is used and AMO knows to remind us when the developer has not responded to the request.

If more information is requested and the developer responds, you will receive email with the developer’s response. At this time you should return to the review page to continue reviewing using the information you received from the developer. Note that the developer may follow up with further questions about the information you are requesting. In all cases, please answer developers in a timely manner ('''within 2 days''').

If you cannot complete the review after the developer has provided information, please at least check if the developer has provided the information you have requested and leave a comment on the version so that other reviewers can continue your work.

Similarly, if you come across an add-on where the developer has provided information but the original reviewer has not followed up within that time, you are welcome to complete the review. The original reviewer may have prior knowledge that would speed up the review, but at the same time we do not want to keep developers waiting.

If you are unsure how to answer a response from a developer, or the developer is disputing your review, please get in touch with admins to make sure the developer receives a reply. Involving a second reviewer to confirm your review can also be helpful if the developer does not want to accept your reasoning.

=== Rejection ===
Rejecting an add-on’s versions can mean anything from showing an outdated version of the add-on to completely hiding the listing from addons.mozilla.org (when rejecting all public versions). Especially in the post-review model where add-ons are reviewed after initial approval, a rejection can lead to frustration from the developer. At the same time, we need to ensure that there are no policy violations that threaten user security or privacy.

'''Our general policy is to only reject when necessary'''. Rejection is necessary when an add-on has security or privacy issues, doesn't meet our content policies, or fits one of the examples described later on.

If the issues you have found are severe enough to warrant a rejection, you will need to determine when the developer started to use the code in question, so you can determine which versions to reject. Generally it is sufficient to check back until the last reviewed version, for example using a [https://en.wikipedia.org/wiki/Bisection_(software_engineering) bisection] approach. If the issue also exists in that last reviewed version, you will need to track back further.

In case of a rejection, developers may have questions on how to best resolve the policy issues, or if they have trouble understanding the message. Similar to the information request, please answer in a timely manner using the reviewer reply feature.

=== Super-Review ===
Using the “Request Super Review” action will put the add-on into the admin queue. There are few select cases where this would occur, please see the examples below for further details. If you have a concern that needs immediate admin attention, please get in touch with the admins via the channel described in the Escalation section, as there is no specific notifications to admins when you request super review.

=== Escalation ===
There are certain cases that are severe enough that the reviewer actions on addons.mozilla.org are not sufficient and you must inform the admin team. This is especially true for [[Add-ons/Reviewers/Content_Review_Guidelines#Sexual_Content|reporting child pornography]], add-ons containing abusive functionality or being of malicious nature that must be blocked in accordance with our [[Blocklisting|blocking policy]].

In such cases, you should contact the admin team via email at amo-admins [at] mozilla [dot] org. You can find more examples on when to escalate later on this page.

=== Multiple Categories of Issues ===
If you find multiple issues within the add-on where there are both information requests and rejections, you may reject the add-on. Please make sure to clearly separate info requests from rejection reasons. An example on how this can be done is shown above.

If you find either a case warranting a super-review or an escalation combined with other issues, you can leave the issues you have found as a comment, noting the review is incomplete (e.g. lead with “Before requesting super-review, I have found the following issues that may be useful to complete the review”)

== Examples ==
The following sections show a few common examples on how to respond to certain policy violations. Please note however these are merely examples intended to convey the intent we have with the policies. It should not be considered a complete list of review decisions. 

=== No Surprises ===

{| class="wikitable"
! Example !! Verdict
|-
| The add-on sends all visited URLs to a third party service without adhering to the [https://developer.mozilla.org/en-US/Add-ons/AMO/Policy/Reviews#No_Surprises no surprises requirements]. || Reject
|-
| The add-on uses means such as webRequest to circumvent the permission prompts for new tab page, homepage or search engine changes. || Reject
|-
| The add-on changes browsing behavior inhibiting user actions, such as closing or hiding about:addons or other special pages when opened || Escalate
|-
| The add-on unexpectedly makes use of redirection to block the user from visiting certain sites without providing the user an option to circumvent the redirection. The add-on is violating the [https://developer.mozilla.org/en-US/Add-ons/AMO/Policy/Reviews#No_Surprises no surprises policy]. || Reject
|-
| The add-on silently modifies web content, for example by exchanging words and images, or adding content. This feature is not part of the core functionality and is not described to the user in any way. || Reject
|-
| The add-on describes itself as e.g. “VPN Service”, while at the same time it also provides something completely unrelated to the add-on’s core function, such as altering the new tab page and providing affiliate search results.<br /><br />The additional features are not stated in the description, and there is no opt-in for the additional feature, violating the [https://developer.mozilla.org/en-US/Add-ons/AMO/Policy/Reviews#No_Surprises no surprises requirements]. || Reject
|-
| An add-on provides UI to allow the user to make a no surprises choice, but the default action is to accept the choice (hence not an opt-in), or does not make clear which add-on is requesting the user choice. || Reject
|-
| An add-on makes use of an “unexpected” feature as per no-surprises policy, but fails to indicate so in the add-on description. || Request Info
|}

=== Content Policy ===
The content review policies are detailed in separate guidelines. Here are a few select examples for the content policy:

{| class="wikitable"
|-
! Example !! Verdict
|-
| Sexual Content: An add-on contains obscene or pornographic images in the icon, screenshots, or anywhere within the add-on UI || Reject
|-
| Sexual Content: An add-on contains images of potential or actual child pornography || Escalate
|-
| Hate Speech: The add-on listing or UI  attacks a person or group based on the attributes described in the [https://www.mozilla.org/en-US/about/legal/acceptable-use/ acceptable use policy].<br /><br />If you are unsure certain phrasing is acceptable or not, please contact an admin. || Reject
|-
| Spam: The add-on clearly has the sole purpose of linking to a product or website and at the same time does not offer any functionality (e.g. “WATCH THISMOVIE ONLINE”) || Reject
|-
| Spam: The listing contains a large amount of words and links unrelated to the add-on’s functionality clearly intending to increase SEO rating || Reject
|-
| Trademarks: The add-on is named “Mozilla Frobnicator” or “Firefox Spice Dispenser”, instead of “Frobnicator for Mozilla” or “Spice Dispenser for Firefox” || Reject
|-
| The add-on’s code, functionality or service used indicates that payment is required to use the core functionality of the add-on but the developer has not selected this option in the listing || Request Info
|-
| The add-on only functions within a closed environment, such as only for employees of a specific company (“internal or private use”) || Reject
|-
| Users can only sign up to the service using a “contact us” link on the website. There is no apparent web sign-up process.<br /><br />(Note that especially on sites with foreign languages, maybe you just missed it. Best to ask the developer to provide information on how a user would sign up. If they can’t provide the information or confirm there is no web sign-up process, the add-on can be rejected) || Request Info
|-
| The add-on is clearly a fork of another add-on, while not providing a significant difference in functionality or code. (This should be a joint decision, we want to make sure not to block creativity by being too strict on “significant difference”) || Request Super Review
|-
| The add-on listing is well described, but requires knowledge of the specific system being used in combination with the add-on || Approve
|}

=== Submission Guidelines ===

{| class="wikitable"
|-
! Example !! Verdict
|-
| The add-on requires use of an external service that is only available with login credentials, and the developer has not provided them || Request Info
|-
| The add-on contains transpiled, obfuscated, minified or otherwise machine-generated code and has not provided source code. || Request Info
|-
| The add-on requests additional permissions that are not required for the add-on to function. || Reject
|}

=== Development Practices ===

{| class="wikitable"
|-
! Example !! Verdict
|-
| The add-on requests additional permissions that are not required for the add-on to function. The developer argues they will need them in a future update. || Reject
|-
| The add-on loads and executes remote code || Reject
|-
| The add-on uses a http channel to exchange sensitive information such as user credentials || Reject
|-
| The add-on contains a large amount of duplicate files, or files not loaded by the add-on || Request Info
|-
| There is a ''noticeable'' impact on performance, for example opening a new tab takes very long because the new tab page is very resource-intensive || Reject
|-
| The developer has failed to provide links to third party libraries, or the links do not point to the original maintainer’s website || Request Info
|-
| The add-on includes a deliberately modified version of a known library, e.g. adding additional code to the library. || Reject
|-
| The add-on includes a library that does not match the original checksum, and it is unclear what modification is made.<br /><br />The developer should be asked to provide the link where they received the library as per the [https://developer.mozilla.org/en-US/Add-ons/Third_Party_Library_Usage Third Party Libraries Usage guidelines]. || Request Info
|-
| The add-on makes use of nativeMessaging || Request Super Review
|}

=== Data Disclosure, Collection and Management ===

This section has a few items related to the privacy policy. We do not check the privacy policy for correctness. We do however make sure the privacy policy is more than just a link and generally about the add-on.

{| class="wikitable"
|-
! Example !! Verdict
|-
| The add-on uses a privacy policy which is merely a link to an external website || Request Info
|-
| On a quick skim, the privacy policy seems to be about a website more than it is about the add-on || Request Info
|-
| After code review it is clear that the add-on exchanges data with a third party service, but the add-on description and summary do not include a summary of the information collected || Request Info
|-
| The main purpose of the add-on is to collect and analyze form data. Therefore, the add-on collects personal data such as the name and email of the user and sends the data to the service, but without an opt-in for personal data. || Reject
|-
| An add-on collects all visited browser URLs without notice, as part of a feature that does not relate to the primary functionality of the add-on.<br /><br />This is considered collecting ancillary information not explicitly required for the add-on’s basic functionality. || Reject
|-
| The add-on collects personal data or passwords and sends it via http to a service. || Reject
|-
| The add-on exchanges data with a native application via native messaging, but the data being exchanged is not summarized in the description nor mentioned in the privacy policy. || Request Info
|-
| The add-on exchanges data via native messaging that does not belong to the primary functionality of the add-on and fails to adhere to the [https://developer.mozilla.org/en-US/Add-ons/AMO/Policy/Reviews#No_Surprises no surprises requirements]. || Reject
|-
| The add-on stores information about tabs, but fails exclude storing information from private browsing mode tabs. || Reject
|}

=== Security Vulnerabilities ===

{| class="wikitable"
|-
! Example !! Verdict
|-
| The add-on injects remote data into an extension page or web page using innerHTML or other methods without prior sanitation. || Reject
|-
| The add-on makes use of React’s ''dangerouslySetInnerHTML'' with remote unsanitized data || Reject
|-
| The add-on makes use of remote CSS scripts, which can cause security vulnerabilities in combination with libraries such as React and Angular. || Reject
|}

=== Monetization ===
Monetization follows the same data disclosure policies as for other data and includes a few extra provisions to set user expectations. We define monetization as a feature of the add-on that results in a potential monetary benefit for the developer.
{| class="wikitable"
|-
! Example !! Verdict
|-
| The add-on has a monetization feature but does not present a user control mechanism at startup. || Reject
|-
| The monetization feature sends personal data, but the user control mechanism at startup is not an opt-in (i.e. default choice is to accept) || Reject
|-
| The add-on sends data unrelated to the add-on’s function (ancillary data) specifically for monetization purposes. || Reject
|-
| The add-on monetizes by injecting ads into web pages, but fails to identify the content as belonging to the add-on || Reject
|-
| The add-on includes a crypto-mining function that mines coins in the background for the profit of the developer || Reject
|-
| The add-on contains a crypto-mining function for the profit of the user (this is still a performance issue) || Reject
|-
| The add-on shows information about crypto coins by querying a web service for information (this is not mining) || Approve
|-
| The add-on changes all Amazon links on web pages to add affiliate tags to profit the developer || Reject
|-
| The add-on has links that include affiliate tags within the browser popup of the add-on || Approve
|}