Marketplace/Performance optimization: Difference between revisions

Latest revision as of 02:32, 1 April 2016

The Marketplace has been placed into maintenance mode. It is no longer under active development. You can read complete details here.

This is a page to aggregate information about Marketplace performance efforts.

Current Status

Via webpagetest from a non-US country shows:

More than one second is spent just to get '/'. Part of this is because we don't have OSCP stapling, so the initial TLS request is very slow. 1011152 was effectively wontfixed by ops (not much we can do about it).
Another second and a half is necessary to load our 2 main CSS files (splash.css, include.css).
Another second is necessary to load our JavaScript. (Note: at this point, we're at the 4 seconds mark and we haven't started requesting the API endpoint. In a packaged app, all this would have been loaded from the package almost instantaneously.)
Then we have a bunch of images, svg, fonts from the CSS loading in //. (This would also be part of the packaged app.)
Then we see the request the the API takes 2.5s (working on this now)
Then we start rendering. Marketplace is usable at this point (8 seconds passed)
Then we start loading the app images, and there are a lot of them.

All that was on a cable connection, on a desktop. It's going to be worse on mobile.

Want to contribute?

Log new bugs in Bugzilla, block 'marketplace-perf'

Ask about things in #marketplace.

Mitigation strategies

The short version is that we need to optimize things that should be optimized regardless of other constraints (device constraints, network constraints, etc). Only then can we differentiate between strategic investment and Doing Things The Right Way.

At a high level, these are the strategies being considered for marketplace.firefox.com:

Feature/Memory/Device detection (reliable) (900241)
Speed and Performance bugs, general at this point (see below)
1. Establish target benchmarks & KPIs for performance
Front and Back-end Performance adjustments (pending benchmark results -- currently the mark is "as fast as possible")

Tarako-specific:

new "low resource" packaged app (implemented)
Filtering apps for "low resource" device (implemented)
UI updates to accommodate RTL & other local language requirements (pending)

Note: We now have the ability to do NewRelic/ElasticSearch deep debugging

Open issues

These are immediate issues:

There is an open github issue with Persona about ETags... closed (but not resolved) on 2014/10/31

These are future issues:

Related, having the feature detection API (983502) as originally described -- this isn't a Marketplace bug, so we're implementing something else for 1.3 and hope to have this for 1.4, 1.5 at the latest.

Bugs

Tracking bug: 1075278.

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

The following bugs have been identified as delivering good performance increases; they are grouped here for ease of consideration.

Note that not all of these are Marketplace bugs, and thus not all are in the priority groupings at the bottom of the page.

overall speed

871683 (wontfix, Server Ops) Note: thus far, SPDY tests have not demonstrated substantial improvements, in part due to system configuration aspects outside of our control. However, these tests were run on the current Marketplace app, which is basically an iframe wrapper. SPDY tests may be more straightforward on the Tarako Marketplace, but as that app is fully packaged it may be irrelevant.

improve API response time

988557 (wontfix)
989181
991955

improve caching techniques

optimize asset delivery

874999 (wontfix)
987851 (wontfix)
988674

optimize synchronous delivery

847679 -- this one is big, mostly because it depends on...

optimize first run

897156 -- ... becoming a real packaged app.

General performance bugs (potentially obsolete)

This is a listing of Marketplace bugs only, by priority. If a priority has not been applied, you will have to find it directly. Here's a good place to start

@@ Line 1: / Line 1: @@
-This is a page to aggregate information about Marketplace performance efforts; it will be maintained as they morph into Tarako-specific issues.
+{{Marketplace_banner}}
+This is a page to aggregate information about Marketplace performance efforts.
+== Current Status ==
+Via webpagetest from a non-US country shows:
+# More than one second is spent just to get '/'. Part of this is because we don't have OSCP stapling, so the initial TLS request is very slow. {{Bugzilla|1011152}} was effectively wontfixed by ops (not much we can do about it).
+# Another second and a half is necessary to load our 2 main CSS files (splash.css, include.css).
+# Another second is necessary to load our JavaScript. (Note: at this point, we're at the 4 seconds mark and we haven't started requesting the API endpoint. In a packaged app, all this would have been loaded from the package almost instantaneously.)
+# Then we have a bunch of images, svg, fonts from the CSS loading in //. (This would also be part of the packaged app.)
+# Then we see the request the the API takes 2.5s (working on this now)
+# Then we start rendering. Marketplace is usable at this point (8 seconds passed)
+# Then we start loading the app images, and there are a lot of them.
+All that was on a cable connection, on a desktop. It's going to be worse on mobile.
 == Want to contribute? ==
-Log new bugs [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody@mozilla.org&bug_status=NEW&cc=ddurst@mozilla.com&contenttypemethod=autodetect&contenttypeselection=text/plain&form_name=enter_bug&keywords=perf&priority=--&product=Marketplace in Bugzilla, with the "perf" keyword]
+Log new bugs [https://bugzilla.mozilla.org/enter_bug.cgi?assigned_to=nobody@mozilla.org&bug_status=NEW&cc=ddurst@mozilla.com&contenttypemethod=autodetect&contenttypeselection=text/plain&form_name=enter_bug&keywords=perf&priority=--&product=Marketplace in Bugzilla, block 'marketplace-perf']
 Ask about things in #marketplace.
+== Mitigation strategies ==
+The short version is that we need to optimize things that should be optimized regardless of other constraints (device constraints, network constraints, etc). Only then can we differentiate between strategic investment and Doing Things The Right Way.
+At a high level, these are the strategies being considered for marketplace.firefox.com:
+# [https://bugzilla.mozilla.org/show_bug.cgi?id=900241 Feature/Memory/Device detection (reliable) (900241)]
+# Speed and Performance bugs, general at this point (see below)
+## Establish target benchmarks & KPIs for performance
+# Front and Back-end Performance adjustments (pending benchmark results -- currently the mark is "as fast as possible")
+Tarako-specific:
+# new "low resource" packaged app (implemented)
+# Filtering apps for "low resource" device (implemented)
+# UI updates to accommodate RTL & other local language requirements (pending)
+'''Note:''' We now have the ability to do [https://rpm.newrelic.com/accounts/315282/dashboard/5293009 NewRelic/ElasticSearch deep debugging]
 == Open issues ==
 <b>These are immediate issues:</b>
-* should change [https://bugzilla.mozilla.org/show_bug.cgi?id=957388 investigate using SPDY] to "using SPDY?" -- does this need a separate ticket?
+* [https://github.com/mozilla/persona/issues/4105 There is an open github issue with Persona about ETags... closed (but not resolved) on 2014/10/31]
-* [https://bugzilla.mozilla.org/show_bug.cgi?id=983815 Use CDN (etc)] <-- Jeremy from IT is looking at using one CDN for images, one for API?
-* [https://bugzilla.mozilla.org/show_bug.cgi?id=980081 cache API responses in localForage] has a PR open already -- worth keywording as perf at this point?
 <b>These are future issues:</b>
-* [http://bugzilla.mozilla.org/show_bug.cgi?id=897156 Becoming a real packaged app] appears to be blocked by external factors (Persona, still?) -- but is this even possible for Tarako?
+* Related, [https://bugzilla.mozilla.org/show_bug.cgi?id=983502 having the feature detection API (983502)] as [https://bugzilla.mozilla.org/show_bug.cgi?id=900241 originally described] -- this isn't a Marketplace bug, so we're implementing something else for 1.3 and hope to have this for 1.4, 1.5 at the latest.
-* Related, [https://bugzilla.mozilla.org/show_bug.cgi?id=983502 having the feature detection API] as [https://bugzilla.mozilla.org/show_bug.cgi?id=900241 originally described] -- this isn't a Marketplace bug, so we're implementing something else for 1.3 and hope to have this for 1.4, 1.5 at the latest.
-== Performance bugs ==
+== Bugs ==
-=== Open P1-P3 ===
+Tracking bug: {{Bugzilla|1075278}}.
 <bugzilla>
-{
+{ "blocks": ["1075278"], "status": ["UNCONFIRMED", "ASSIGNED", "NEW", "REOPENED"] }
-    "product": "Marketplace",
-    "status": ["NEW", "ASSIGNED"],
-    "resolution": "---",
-    "priority": ["P1", "P2", "P3"],
-    "keywords": "perf",
-    "include_fields": "id, product, component, priority, severity, status, summary, assigned_to, last_change_time"
-}
 </bugzilla>
+The following bugs have been identified as delivering good performance increases; they are grouped here for ease of consideration.
-=== Open P4-P5 ===
+Note that not all of these are Marketplace bugs, and thus not all are in the priority groupings at the bottom of the page.
-<bugzilla>
-{
+=== overall speed ===
-    "product": "Marketplace",
+* {{Bugzilla|871683}} (wontfix, Server Ops) Note: thus far, SPDY tests have not demonstrated substantial improvements, in part due to system configuration aspects outside of our control. However, these tests were run on the current Marketplace app, which is basically an iframe wrapper. SPDY tests may be more straightforward on the Tarako Marketplace, but as that app is fully packaged it may be irrelevant.
-    "status": ["NEW", "ASSIGNED"],
-    "resolution": "---",
+=== improve API response time ===
-    "priority": ["P4", "P5"],
+* {{Bugzilla|988557}} (wontfix)
-    "keywords": "perf",
+* {{Bugzilla|989181}}
-    "include_fields": "id, product, component, priority, severity, status, summary, assigned_to, last_change_time"
+* {{Bugzilla|991955}}
-}
-</bugzilla>
+=== improve caching techniques ===
+* {{Bugzilla|957383}}
+* {{Bugzilla|985291}}
+* {{Bugzilla|985292}}
+* {{Bugzilla|985293}}
+* {{Bugzilla|985804}}
-=== Resolved: Fixed ===
+=== optimize asset delivery ===
-<bugzilla>
+* {{Bugzilla|874999}} (wontfix)
-{
+* {{Bugzilla|987851}} (wontfix)
-    "product": "Marketplace",
+* {{Bugzilla|988674}}
-    "status": "RESOLVED",
-    "resolution": "FIXED",
-    "keywords": "perf",
-    "include_fields": "id, product, component, priority, severity, status, summary, assigned_to, last_change_time"
-}
-</bugzilla>
+=== optimize synchronous delivery ===
+* {{Bugzilla|847679}} -- this one is big, mostly because it depends on...
-=== Resolved: Deferred ===
+=== optimize first run ===
-<bugzilla>
+* {{Bugzilla|897156}} -- ... becoming a real packaged app.
-{
-    "product": "Marketplace",
-    "status": "RESOLVED",
-    "resolution": "WONTFIX",
-    "keywords": "perf",
-    "include_fields": "id, product, component, priority, severity, status, summary, assigned_to, last_change_time"
-}
-</bugzilla>
+== General performance bugs (potentially obsolete) ==
+This is a listing of Marketplace bugs only, by priority. If a priority has not been applied, you will have to find it directly. [https://bugzilla.mozilla.org/buglist.cgi?priority=--&keywords=perf&keywords_type=allwords&list_id=9800214&resolution=---&query_format=advanced&product=Marketplace Here's a good place to start]
-== Performance data dashboard ==
-Not sure how these apply to bugs yet, because not sure how these poll yet. But here they are:
+== Performance data dashboards ==
 * [https://metaplace.paas.allizom.org/apikiosk/ API Speed dashboard]
 * [http://eideticker.mozilla.org/b2g/#/inari/b2g-marketplace-startup/timetostableframe Time to full display of Marketplace app (warm)]
+* [http://dashboard.mktadm.ops.services.phx1.mozilla.com/graphite-api (VPN required) ops level dashboard]