Firefox/Shield
- If you are a Firefox User with questions about the Heartbeat prompt, go to Heartbeat and Shield User support page
- Engineers: see #engineering
- Product managers: see #process
- Release managers: see #launchSchedule
- University Researcher: see the #contact
SHIELD PROGRAM
The Shield platform gives us the ability to push changes in Firefox quickly and with high precision. There are two main goals for the Shield Program.
Shield Turns Superstition into Science
Shield is a Firefox user testing platform for proposed, new and existing features and ideas. Broad Applications The Shield system addon targets specific subsets of users, enables features and messaging, and measures the results of those interventions to evaluate the effect of the features/messages on measures (retention, uptake, performance, crashiness) to create insights for the Firefox product and Mozilla as an organization.
Self Repair
Make it easier for users to recover from common issues. This could be restoring a preference or removing an unwanted toolbar.
Design
Supported Study and Messaging Designs
Shield currently supports these Study and Outreach Designs:
- Preference-flip to control feature
- Preference-flip to control (disabled by default) system addon
- Add-on Based study (opt-in) Shield Studies
- Add-on Based Study (opt-out) Shield Studies
- Heartbeat Star Rating with action link
- Messaging Study with action link to external site (survey, Brain Games, interface testing, external user task tool)
- Contextual Feature Recommendation: Customized action after user event
- Pioneer: On-going opt-in cohort for deeper data collection
- Rollout preference study
Choosing A Design for Evaluating Features
Issues in design choice:
- Code deployment: in-tree (either as native code, or system addon) vs. Extension (Legacy Extension or Addon)
- Sensitivity of data / probes collected. In particular, full urls usually require an opt-in design
- Speed vs. fidelity: if possible, use Images, fake prototypes, etc FIRST to do validation. Gather data using **surveys** of Firefox users, Other Browser users or whoever.
Deciding:
- Just an Idea: Do surveys! Minimal wireframe prototypes! Pile-sort! Don't build! Identify needs, user models, and terminology before solving and building!
- if you are already in tree (mozilla-central), including your probes, use "pref-flip"
- Not in tree, use an opt-out (if data is benign).
- Not in tree, need more sensitive data, needs opt-in (at the risk of recruitment bias), choose opt-in (one-off study) or Pioneer.
Alternatives To Shield
Other methods support these designs better:
- early stage ideation: simple hallway / cafe testing and paper prototypes
- You need Chrome / IE / Edge users - we can recruit those using other methods
- You want in-depth narrative data.
Other methods for launching and testing features or messages include
- AMO addon
- Snippets.
Launch a Shield Study
Overview: File Bug, Send email to Release-Drivers, tell Shield Team to start recruiting.
Launch Schedule, Backlog, In Progress
Engineering and Building
Shield System Addon - downloads, interprets and runs recipes
The Shield system addon pulls a list potential "recipes" from the Normandy server on a regular (daily) basis. It allows us to quickly control the delivery of recipes outside of the trains. The addon then filters recipes based on client configuration (channel, locale, geo, addons, prefs). The client-side component lets us constrain what actions are available to recipes for security purposes, and also provides a good point for testing the functionality of our fixes without removing our ability to ship fixes fast.
Read details at: Shield client addon concepts
Normandy - User Profile Matching and Recipe Deployment
- Filters for Shield Targeting, included telemetry ping traits, addons list, and other "point in time" profile features.
- Server Ops Wiki
- Normandy Github Repository
- Normandy Issue Tracker
- There's a "Normandy Client" Bugzilla component and "Normandy Server" Bugzilla component if you want to file bugs that depend on other Bugzilla bugs or otherwise need to integrate with Bugzilla.
- Technical Documentation
Shield Studies Addon Utils
- Single File `.jsm`.
- Embed these inside your Legacy Addon or Embedded WebExtension addon.
- Shield-Studies-Addon-Utils on Github
- if you ARE NOT ON VERSION 4+, please stop :)
- No more Jetpack / Addon-sdk!
Experienced Shield Study Authors Include:
- Gregg Lind
- Kamyar Ardekani
- Benton Case
- Luke Crouch
- (others!)
Contact Us
Not Satisfied? For all additional questions or live support
Visit our SLACK Channel: #shield on mozilla.slack.com and say "help".
Shield Core Team
- Program Owner: Matt Grimes [:Matt_G]
- Weird Science Lead: Gregg Lind [:gregglind]
- Data Steward: Rebecca Weiss [:rweiss], Saptarshi Guha [:joy]
- Development: Michael Cooper [:mythmon], Lonnen [:lonnen], Web Engineering * Systems
- Operations: Daniel Thorn [:relud], Benson Wong [:mostlygeek]
- QA: Chris Hartjes [:chartjes], Adrian Florinescu [:aflorinescu], Krupa Raj [:krupa], Peter Dehaan [:pdehaan]
History
- June 2014 * mconner files a tracker bug for enhancing FHR to proactively detect and solve problems for users.
- December 2014 * bsmedberg files bug 1111016 for implementing the smallest-possible set of changes necessary to support Heartbeat (a prompt shown to users asking them to rate Firefox) and Self-Repair (a minimal version of the problem-solving functionality, described in detail in dependent bugs).
- Early 2015
- UITour is enhanced to support Heartbeat prompts, sending user responses to input.mozilla.org.
- Self-Repair is implemented as a set of static files hosted on AWS that Firefox retrieves.
- Late 2015 * Self Repair is renamed to SHIELD since the project covers more than just repairing Firefox. The name covers the server that deploys configuration files (recipes) and the in-product component that executes the recipe.
- Example 1: The Shield server deploys a recipe to a client indicating that the user should be included in the daily Heartbeat sample. The in-product component responds by offering the Heartbeat prompt.
- Example 2: The SHIELD server matches a “preference repair recipe” to the user configuration, and returns it. Firefox runs the recipe steps to fix the broken preference.
- February 2016 * Development begins on the SHIELD backend service, Normandy. Work on the system add-on begins as well.
- April 2016 * Normandy replaces the self-repair server at the domain self-repair.mozilla.org. Development on the system add-on continues.
- December 2016 * SHIELD System Add-on lands in mozilla-central
- 2017 * Shield adds pref-flip studies, v4 of shield utils and many other things.