Performance/Fenix: Difference between revisions

Performance/Fenix (view source)

Revision as of 16:12, 17 August 2021

491 bytes removed , 17 August 2021

→‎List of tests running in fenix

Mcomella

Confirmed users

975

edits

@@ Line 13: / Line 13: @@
 === List of tests running in fenix ===
-The perftest team is working to dynamically generate the list of tests that run on the fenix application. Some progress can be seen in [https://sql.telemetry.mozilla.org/queries/77734/source this query] and [https://treeherder.mozilla.org/perfherder/tests this treeherder page]. Until then, we manually list the tests below.
+In addition to the tests we actively look at above, there are other tests that run in mozilla-central on fenix or GeckoView example. '''We're not sure who looks at these.''' The perftest team is working to dynamically generate the list of tests that run. Some progress can be seen in [https://sql.telemetry.mozilla.org/queries/77734/source this query] and [https://treeherder.mozilla.org/perfherder/tests this treeherder page]. Until then, we manually list the tests below.
 As of Feb. 23, 2021, we run at least the following performance tests on fenix:
-* Page load duration: see the query above for a list of sites (sometimes run in automation, sometimes run manually; todo: details)
+* Additional page load duration tests: see the query above for a list of sites (sometimes run in automation, sometimes run manually; todo: details)
 * media playback tests (TODO: details; in the query above, they are prefixed with ytp)
-* Start up duration (see [[#Terminology|Terminology]] for start up type definitions)
+* Start up duration via mach perftest
-** COLD VIEW tests on mach perftest. Runs per master merge to fenix on unreleased Nightly builds so we can identify the commit that caused a regression
-** COLD MAIN & VIEW tests on FNPRMS. Runs Nightly on production Nightly builds. This is being transitioned out in favor of mach perftest.
 * Speedometer: JS responsiveness tests (todo: details)
 * tier 3 unity webGL tests (todo: details)
-There are other tests that run on desktop that will cover other parts of the platform. We also have other methodologies to check for excessive resource use including lint rules and UI tests that measure things such as
+There are other tests that run on desktop that will cover other parts of the platform.
-Notable gaps in our test coverage includes:
-* Duration testing for front-end UI flows such as the search experience
-* Testing on non-Nightly builds (does this apply outside of start up?)
 == Preventing regressions automatically ==