Firefox/SHIELD: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
Line 1: Line 1:
The SHIELD server hosts "recipes" that are fetched by Firefox once per day. These recipes help User Advocacy both gather feedback from users so that they can better advocate for their needs, and help them offer choices to users for things like fixing common problems with known solutions in Firefox.
SHIELD is a system that addresses user attrition and satisfaction in Firefox by providing a fast and powerful way for Firefox to respond to issues that users are experiencing.


== Status ==
== Status ==
Line 10: Line 10:
** '''Service Docs''': http://normandy.readthedocs.org/en/latest/
** '''Service Docs''': http://normandy.readthedocs.org/en/latest/


== History ==
== Rationale ==
* June 2014 - mconner files [https://bugzilla.mozilla.org/show_bug.cgi?id=1031506 a tracker bug] for enhancing FHR to proactively detect and solve problems for users.
User configuration problems cause attrition. Common issues we've seen include:
* December 2014 - bsmedberg files [https://bugzilla.mozilla.org/show_bug.cgi?id=1111016 bug 1111016] for implementing the smallest-possible set of changes necessary to support [https://support.mozilla.org/en-US/kb/rate-your-firefox-experience-heartbeat Heartbeat] (a prompt shown to users asking them to rate Firefox) and Self-Repair (a minimal version of the problem-solving functionality, described in detail in dependent bugs).
* Early 2015
** UITour is enhanced to support Heartbeat prompts, sending user responses to input.mozilla.org.
** Self-Repair is implemented as a set of static files hosted on AWS that Firefox retrieves.
* Late 2015 - Self Repair is renamed to Shield since the project covers more than just repairing Firefox. The name covers the server that deploys configuration files (recipes) and the in-product component that executes the recipe.
** Example 1: The Shield server deploys a recipe to a client indicating that the user should be included in the daily Heartbeat sample. The in-product component responds by offering the Heartbeat prompt.
** Example 2:  The Shield server matches a “preference repair recipe” to the user configuration, and returns it.  Firefox runs the recipe steps to fix the broken preference.


=== Terminology ===
* Search engine hijacking
; Action
* Homepage hijacking
: A semantic thing that can be done by the browser, like "Display a survey" or "Highlight this menu item". Actions may be paired with executable code for self-repair, but other services may read the actions and implement them differently.
* Misconfigured preferences


; Recipe
Currently, the tools that we have to respond to these issues are inadequate:
: A list of actions to execute to achieve something, paired with eligibility rules determining who they get sent to. For example, a recipe may check to see if a preference is set to a known-bad value, and offer to reset it for the user.


; Eligibility Rule
* One-off solutions aren't easily repeatable and require a lot of coordination.
: A rule describing who a recipe should be sent to. For example, a survey may only be relevant to users in India, so the eligibility rules for that survey recipe will limit it to users located in India only. Other filters available include locale, sampling rate, and start/end dates.
* Hotfixes require a lot of people and time to build, review, and ship, and are shipped to ''everyone''.
* Firefox Reset can fix many issues, but it's blunt (resets many things at once), hidden (so users don't accidentally trigger it), and user-driven.


; Bundle
Attackers causing these issues can move faster than us. To keep up, we need something new that can:
: A bundle is a collection of recipes that are sent to Firefox. Bundles are built based on eligibility rules.


== Phase 1 ==
* Ship fixes outside of the trains, with a small amount of effort and time to ship
* Measure the effectiveness of fixes
* Target only affected users, control rollout, and make reverting easy
* Provide the power to fix things, but in a constrained, secure way


=== Summary ===
=== Solution ===
Phase 1 will consist of writing a server to replace the current statically-generated self-repair scripts. The server will be capable of sampling, limiting recipes to a certain number of users, timed releases of recipes, localized recipes, and geolocation.
SHIELD addresses this problem with a client-side component that pulls potential fixes ("recipes") from a server ("recipe server") on a regular basis. The recipe server allows us to quickly control the delivery of recipes outside of the trains. The client-side component lets us constrain what actions are available to recipes for security purposes, and also provides a good point for testing the functionality of our fixes without removing our ability to ship fixes fast.


=== Requirements ===
In the future, we can use the recipe server to serve up recipes that do more than just fix known issues, such as suggesting actions that improve user experience or collecting feedback from users.  
# Serve bundles of recipes to all Firefox instances, once per day, filtered by eligibility rules.
# Present webpage for creating and managing recipes and the rules determining who sees them.
# Control rollout of recipes with composable rules:
#* Distribute recipe to approximately X users, then disable it.
#* Distribute recipe starting on X date/time and/or ending on Y date/time.
#* Distribute recipe to X% of users.
#* Distribute recipe to users in a particular country.
#* Distribute recipe to users based on the locale of Firefox (Ex: ru, de, en-GB).
#* Distribute recipe to users on a specific release channel (including non-standard channels like funnelcake).
# Expose debugging view for previewing the set of recipes being sent to a specific browser/location configuration.


=== Measuring success ===
=== Outcome ===
How do we know that we've won? What do we want to be measuring?
To be successful, SHIELD must:


=== Bugs ===
* Decrease user attrition
* Firefox client-side: https://bugzilla.mozilla.org/show_bug.cgi?id=1233804
* Increase usage
https://bugzilla.mozilla.org/show_bug.cgi?id=1234393
* Improve user experience


== Future phases ==
SHIELD will leverage the existing Telemetry system to measure our progress on these goals.
Possible future improvements ''may'' include:


* A debugging view for previewing a specific recipe's functionality.
== History ==
* Automatically modify sample rates to reach a certain minimum amount of users over time. For example, "Reach 500 users per day, spread out over the entire day".
* June 2014 - mconner files [https://bugzilla.mozilla.org/show_bug.cgi?id=1031506 a tracker bug] for enhancing FHR to proactively detect and solve problems for users.
* Chain recipes with auto-disable rules, such that when one is disabled the next one is enabled.
* December 2014 - bsmedberg files [https://bugzilla.mozilla.org/show_bug.cgi?id=1111016 bug 1111016] for implementing the smallest-possible set of changes necessary to support [https://support.mozilla.org/en-US/kb/rate-your-firefox-experience-heartbeat Heartbeat] (a prompt shown to users asking them to rate Firefox) and Self-Repair (a minimal version of the problem-solving functionality, described in detail in dependent bugs).
* Early 2015
** UITour is enhanced to support Heartbeat prompts, sending user responses to input.mozilla.org.
** Self-Repair is implemented as a set of static files hosted on AWS that Firefox retrieves.
* Late 2015 - Self Repair is renamed to Shield since the project covers more than just repairing Firefox. The name covers the server that deploys configuration files (recipes) and the in-product component that executes the recipe.  
** Example 1: The Shield server deploys a recipe to a client indicating that the user should be included in the daily Heartbeat sample. The in-product component responds by offering the Heartbeat prompt.
** Example 2:  The Shield server matches a “preference repair recipe” to the user configuration, and returns it.  Firefox runs the recipe steps to fix the broken preference.

Revision as of 19:11, 14 January 2016

SHIELD is a system that addresses user attrition and satisfaction in Firefox by providing a fast and powerful way for Firefox to respond to issues that users are experiencing.

Status

Rationale

User configuration problems cause attrition. Common issues we've seen include:

  • Search engine hijacking
  • Homepage hijacking
  • Misconfigured preferences

Currently, the tools that we have to respond to these issues are inadequate:

  • One-off solutions aren't easily repeatable and require a lot of coordination.
  • Hotfixes require a lot of people and time to build, review, and ship, and are shipped to everyone.
  • Firefox Reset can fix many issues, but it's blunt (resets many things at once), hidden (so users don't accidentally trigger it), and user-driven.

Attackers causing these issues can move faster than us. To keep up, we need something new that can:

  • Ship fixes outside of the trains, with a small amount of effort and time to ship
  • Measure the effectiveness of fixes
  • Target only affected users, control rollout, and make reverting easy
  • Provide the power to fix things, but in a constrained, secure way

Solution

SHIELD addresses this problem with a client-side component that pulls potential fixes ("recipes") from a server ("recipe server") on a regular basis. The recipe server allows us to quickly control the delivery of recipes outside of the trains. The client-side component lets us constrain what actions are available to recipes for security purposes, and also provides a good point for testing the functionality of our fixes without removing our ability to ship fixes fast.

In the future, we can use the recipe server to serve up recipes that do more than just fix known issues, such as suggesting actions that improve user experience or collecting feedback from users.

Outcome

To be successful, SHIELD must:

  • Decrease user attrition
  • Increase usage
  • Improve user experience

SHIELD will leverage the existing Telemetry system to measure our progress on these goals.

History

  • June 2014 - mconner files a tracker bug for enhancing FHR to proactively detect and solve problems for users.
  • December 2014 - bsmedberg files bug 1111016 for implementing the smallest-possible set of changes necessary to support Heartbeat (a prompt shown to users asking them to rate Firefox) and Self-Repair (a minimal version of the problem-solving functionality, described in detail in dependent bugs).
  • Early 2015
    • UITour is enhanced to support Heartbeat prompts, sending user responses to input.mozilla.org.
    • Self-Repair is implemented as a set of static files hosted on AWS that Firefox retrieves.
  • Late 2015 - Self Repair is renamed to Shield since the project covers more than just repairing Firefox. The name covers the server that deploys configuration files (recipes) and the in-product component that executes the recipe.
    • Example 1: The Shield server deploys a recipe to a client indicating that the user should be included in the daily Heartbeat sample. The in-product component responds by offering the Heartbeat prompt.
    • Example 2: The Shield server matches a “preference repair recipe” to the user configuration, and returns it. Firefox runs the recipe steps to fix the broken preference.
Retrieved from "https://wiki.allizom.org/index.php?title=Firefox/SHIELD&oldid=1112786"