Mobile/Projects/Telemetry probes for Fennec UI elements

Tracking

• Meta-bug: bug 942279

Goals

We need more data about how users actually interact with our browser, and we can create a valuable source of data by building more telemetry probes into our UI.

What data?

General usage patterns

• Need a session for each "screen" in order to facilitate heatmapping etc.
• How are people opening Firefox?
  • [how] Different telemetry session types — allows for partitioning behaviors between these usage types.
  • Tapping Firefox icon on Android homescreen
  • Link from another app
  • Android Homescreen bookmark
• How often are people opening (focusing) Firefox?
• Duration of each browsing session?
  • Number of pages visited
  • Number of tabs open
    • On restore!
• [P1] What are people doing?
  • How are users getting to a page?
    • Searching
      • Breakdown of providers used
      • Are they using suggestions?
    • Typing URLs
    • Retrieving Saved Stuff from Home
      • which parts?
    • Coming in from an outside link
    • Tapping: row, thumbnail/top site
  • How are users saving stuff? + bounce time, if applicable
    • Bookmarking
    • Sharing
    • Opening in new tab
    • Adding to reading list
  • How are tabs used?
    • Comparison?
      • Actively switching through tabs in a "short" session
        • More tab switching events than tabs
        • Tab switching interspersed (or not) by other events (loading page)
        • Also keep track of re-opened tabs
      • Read later?
      • Do they just collect over time?
        • Tab lifetime between sessions?
    • Amount of effort spent in typing
      • Searches
      • URLs
      • Other web inputs — in content
    • Device orientation (landscape vs portrait)
    • Methods of exiting Firefox (home, closing tab, etc).
  • Relative popularity of Private / Guest Browsing
    • For detailed recording within: [privacy-review-needed]
  • Extent to which people customize the browser
    • In Settings
    • with Add-ons
• Heatmap of browser UI usage
  • Context of what to overlay heatmap on
  • Are we prioritizing the right stuff?
• How does this differ on phones vs. tablets?

Technical details

bug 932092 ships in Firefox 28, and allows for UI code to record telemetry events for analysis.

The concepts are:

Sessions

Sessions are essentially scopes. They are meant to give context to events; this allows events to be more simple and reusable. Sessions are usually bound to some component of the UI. They are started when the user begins interacting with the UI component and are stopped when the user leaves. They also allow you record how long a user interacts with a given UI component. A simple use-case for sessions is the bookmarks panel in about:home. We start a session when the user swipes into the panel and stop it when they leave. This bookmarks session does two things: Firstly, it gives scope to any generic event that may occur within the panel (e.g loading a URL). Secondly, it allows us to figure out how much time users are spending in the bookmarks panel.

To start a session, call Telemetry.startUISession(String sessionName). Session names should be brief, lowercase, and should describe what UI component the user is interacting with. In certain cases where the UI component is dynamic, they could include an ID, essential to identifying that component. An example of this is with dynamic home panels, we use session names of the format "homepanel:<panel_id>" to identify home panel sessions.

To stop a session call Telemetry.stopUISession(String sessionName, String reason). sessionName is the name of the session and reason is essentially the reason the session was stopped. The reason field should capture why the user stopped interacting with the UI component; it should be brief, lowercase and generic so it can be reused in different places. Examples reasons are:

• "switched" - user transitioned to a UI element of equal level
• "exit" - user left for an entirely different element.

Events

Events capture key occurrences that of importance to us. They should be brief and simple, and should not contain sensitive or excess information. Context for events should come from the session (scope). An event can be created with four fields (via Telemetry.sendUIEvent): action, method, extras, and timestamp.

• action: name of the event. Should be brief and lowercase. If needed, you can make use of namespacing with a '-' separator. Example event names: "panel-switch", "panel-enable", "panel-disable", "panel-install".

• method (Optional): used for user actions that can be performed in many ways. This field specifies the method in which the action was performed. For e.g, say we want to record an event when a user adds an item to their reading list. This action can be performed by either long-tapping the reader icon in the address bar or done from within reader mode. We would use the same event name for both user actions but specify two methods: "addressbar" and "readermode".

• extras (Optional): for extra information that may be useful in understanding the event. It is important to keep this very brief.

• timestamp (Optional): time the event occurred. If not specified this field defaults to the realtime clock.

Clock

Times are relative to either elapsed realtime (an arbitrary monotonically increasing clock that continues to tick when the device is asleep), or elapsed uptime (which doesn't tick when the device is in deep sleep). We default to elapsed realtime.

See the documentation in http://mxr.mozilla.org/mozilla-central/source/mobile/android/base/Telemetry.java for more.

Research/references

Note: Make sure Insights & Metrics teams are aware of what we're hoping to do once we have this a bit more solidly defined.

An etherpad that needs a lot of work: https://etherpad.mozilla.org/FxATelemetry

UX design

• Not handed off yet

Quality criteria

• No user-perceptible performance degradation
• All goals are met
• All user stories are implemented & tested