Marketplace/Tarako/Technical design

The Marketplace has been placed into maintenance mode. It is no longer under active development. You can read complete details here.

The Tarako device class is a low-cost smartphone. In order to allow the Marketplace to work on such a device, a different architectural approach must be taken.

Technical requirements and constraints

Known device and environment constraints (as of 2014/04/28)

• 128MB RAM
• 30MB RAM max for an app
• 6MB available for browser cache
• 15MB a safe ceiling for local storage (non-browser)
• attempts to multitask result in OOM and app shutdowns requiring restart, reboot in some cases
• no GPS

Relevant audience and environment research highlights

• EDGE use is popular, and should be treated as a baseline
• users are aware they are getting a "smartphone" and have expectations about quantity of apps in the catalog
• users are used to slower connections
• advertising is seen as questionable / bad
• network / data will be heavily constrained

Known FxOS/other issues (as of 2014/05/23)

• we will not have feature detection API in 1.3T
• online/offline detection from platform is unavailable

Design drivers

• navigating: we should be as close to a flat page as we can be (safest to assume hierarchical navigation won't be usable for target audiences, and so they wouldn't use it); utility of search is equally unclear.
• apps need to be targeted: need to be ones that run well on Tarako class devices; need to be most relevant.
• operator shelf not necessary for launch (potentially at all).
• low bandwidth, high latency: performance issues related to network latency are likely to be seen as less of a big deal than we see them as, these slow networks are the norm in the target markets.
• lower end of the market implies lower levels of education & literacy: possibly replace category labels with icons? reduce depth/hierarchy of navigation, streamline app detail pages.
• home page content must be targeted as specifically as possible.

Technical drivers

• device / environment constraints
• design drivers
• KPIs for performance

KPIs for performance

Need to clarify these KPIs against https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines.

• TMP initial launch: <3000ms baseline, <2000ms target
• TMP load (wifi): <8000ms baseline, <5000ms target
• TMP load (edge): <10000ms target

Position

With an aggressive timeline for v0 delivery and outstanding blockers on key aspects (Persona, GA), the chosen approach for v0 was that of a separate hosted app that could be locally cached over a real packaged app or possibly futile optimizations of the existing marketplace.

This approach would allow for the development of a custom UI quickly, fast warm loads of the app, offline/transition use of the app, as well as the ability to iterate on changes to the app without deploying packages or requiring Gaia updates. The drawback is that we lose MNC/MCC access, and thus payments capability (though there are currently no paid apps anyway).

The ultimate goal is a real packaged app, re-instating billing, logins, and useful implementation of GA; because of the timeline, these will need to be all in future releases, as necessary.

Once the FxOS landing date changed and the hosted app testing showed that it would not perform against the quantity of apps anticipated for Tarako release, the approach was changed to the preferenced packaged app (a derivative fork of Fireplace) to be delivered in a v0 (and v0.5) and v1 iteration (requiring three separate uplifts).

Technical design details

The packaged marketplace app for use with Tarako will consist of pages and assets (JavaScript, CSS, and images) needed for the interface as well as initial data for the app in JSON format, excluding app icons and screenshots.

Interfaces

0. Global: Searchbox at the top of the screen, below that is a navigation bar that shows icons and labels for Home, Games, Tools, and Lifestyle. The "Home" content pulls from the collection identified as 'tarako-featured,' while the latter three categories are Tarako-specific programmatic aggregations of Marketplace categories (identified as 'tarako-games', 'tarako-tools', and 'tarako-lifestyle,' respectively).
1. Home page: shows 10 apps editorially designated for the homepage collection, and four buttons for Help, Feedback, Privacy Policy, and and Terms of use at the bottom of the page.
2. Category list view: shows all apps in that category, in pages of 10 at a time.
3. Search results: shows all apps meeting the search criteria, as well as a header below the global elements that shows a result count on one side of the screen and a toggle control on the other that switches between compact and expanded views of each app.
4. Compact / Expanded app view: shows the basic information for an app; the expanded view reveals a thumbnail of a screenshot.
5. Detail view: shows more information for the app, including a description and a thumbnail of a screenshot that the user can click on to load a full screenshot.

Content filtering

6. Filtering apps for device: short-term: tagging apps to be served to Tarako devices (v1); long-term: integrated dev submission flow and review verification (post-v1, server side).

Content caching (and offline implications)

7. Local caching of marketplace data: enables fast startup of the application while data is updated asynchronously in the background (when online), deferred until rendering is complete and using an in-memory cache to compensate for slow I/O on the device itself. When offline, the application will present a screen that tells the user they need to be online to use the marketplace and provides a "try again" button.
8. Cache contents: for performance improvements (including bootstrapping startup), the cached data will consist of: text data for the application and data (excluding images) for each app. This will be deployed with the package for the 10 apps that will appear on the homepage as well as each 10 on the first page of each of the 3 categories (this enables bootstrapping of initial pages (except search) and anticipated next-click detail pages); subsequent use of the marketplace will add additional apps and lists of apps to the cache. The application will always pull data from the cache first (if possible) and will update the cache asynchronously in the background from the network on every request, regardless of whether that request was served from the cache or not; the user will not be notified when the cache is updating or when the cached has been updated. NOTE: this means that the package includes whatever the contents of the 'tarako-featured' collection are at the time of package creation.

Home page freshness

9. The homepage experience will be very simple: it will show all of the apps from an editorially-chosen collection. This "Tarako Featured" collection will be maintained by the curation team. Target size of collection: 10 apps. Freshness of applications displayed will be driven directly by the curation of the Tarako featured collection.

INSTALLS_ALLOWED_FROM

10. For v0 and v1, we will use an iframe approach. For the future, we should consider the approach outlined in https://bugzilla.mozilla.org/show_bug.cgi?id=965964.

Workarounds

11. Caching: In implementing localForage, it became apparent that localStorage (despite needing JSON stringified data) was faster for the quantity of data TMP used than IndexedDB. Given that every page request is cached, we don't need to spend a lot of logic managing TTLs or cache freshness. However, in using localStorage, we then committed to a ceiling cache size of 5MB, well under the 15 limit deemed safe. A basic try/catch for writing to the cache is used to clear the cache if the error is "QuotaExceededError" or "NS_ERROR_DOM_QUOTA_REACHED"; at which point, the failed write is re-tried, and the cache is refreshed. Mathematically, it is unlikely that a user will reach this limit, but the try/catch is in place.
12. Online/Offline Detection: Our network connection is expected to be both slow and unreliable, but the platform does not allow a way to check if the device is online or offline. Various methods were tried (and abandoned) before the final, which is to navigator.mozTCPSocket.open port 80 on our CDN URL. This is a "best guess" that the available network is also sufficiently responsive, but it's as much as we can do in this limited scenario.

Repository

• https://github.com/mozilla/yogafire/

Summary of functionality for v0

Packaged app with a streamlined UI for performance, especially on low-memory devices; size of package not to exceed 1mb.

• no:
  • no offline functionality
  • no login
  • no review/rating submissions
  • no reviews
  • no prepopulated search
  • no payments
  • localizations are incomplete and/or not included
• JSON data for homepage (max 30 apps, not just 6 shown)
  • text data
  • icon
  • top 3 screenshot thumbnails? (maximum / optional)
• JSON data for first page of categories (10 apps each)
  • text data
  • icon
  • top 3 screenshot thumbnails? (maximum / optional)
• data is included as JSON in the package
  • apps are identified back-end and tagged (by Wil)
  • apps are filtered by tag on search endpoint
• all assets not loaded into the package will be lazy-loaded (when the user is online); none of that data will be cached/saved
• all assets loaded into the package only get updated with a new package release
• online cached to appserver according to existing Fireplace logic -- updated based on ttl (1 week) and size (4mb limit)

Summary of functionality for v1

Packaged app with a streamlined UI for performance, especially on low-memory devices; size of package not to exceed 1mb. Building upon v0 to deliver improved performance and localizations.

• include blocking notification screen that the user must be online to use the marketplace
• no:
  • no offline functionality
  • no login
  • no review/rating submissions
  • no reviews
  • no prepopulated search
  • no payments
• data is included as JSON in the package
  • apps are identified back-end and tagged (by Wil)
  • apps are filtered by tag on search endpoint
  • data is for: homepage (10 apps), first page of each category (10 apps each), and each app on those pages
• data cached for each app:
  • text data
  • icon (not initially stored in package)
  • first screenshot thumbnails (not initially stored in package)
• data requested by the user will be displayed from the cache if available, but requested from the network and cache updated after the response on every request regardless of whether it was served from the cache or not
• package includes localizations for 4 languages + English
• JSON data is cached locally (in localForage/localStorage) as of the initial (cold) load; subsequent app launches should start up with cached (not packaged) data first
• all cached data not to exceed 5mb
• reviewers and curators can identify apps that will run on Tarako:
  • reviewers are able to filter (and override) apps by "tarako" tag
  • curators are able to filter apps by "tarako" tag (via potato search, so ":tarako") and create the "featured" collection used to populate the homepage
    • Note: region should also be considered in filtering. Apps for tarako in china, for example, should search for ":tarako :region=cn"

Release schedule

Note: v0.5 was created on the chance that devices were going to ship before the date we were told for v1.0.

Builds to Gaia

• v0: April 30, 2014; no build id exposed in UA (https://github.com/mozilla-b2g/gaia/pull/18845)
• v1.0: May 25, 2014; build id 20140522150325 (https://github.com/mozilla-b2g/gaia/pull/19571)
• v1.1: May 30, 2014 -- post v1 addendums; build id 20140711093908 (https://github.com/mozilla-b2g/gaia/pull/21642)
• v1.1.0: October 31, 2014 -- fixed analytics bug; build id 20141031152223 (TBD, see bug 1092424)
• v1.1.0: November 14, 2014 -- fixed analytics bug, workaround for 1096555, build id 20141114105842 (https://github.com/mozilla-b2g/gaia/pull/26157)

Package versions

• v1.1.0 (update, also uplifted to Gaia): 20141114105842
• v1.1.0 (update, also uplifted to Gaia): 20141031152223 cancelled due to 1096555
• v1.1 for Gaia: 20140711093908
• v1.0 for Gaia: 20140522150325
• v0.5: (no build id exposed) 20140508151725
• v0: (no build id exposed) 20140430121841