|
|
| Line 429: |
Line 429: |
| === Page-Load Tests === | | === Page-Load Tests === |
|
| |
|
| For all Raptor page-load tests, the pages are played back from [[https://wiki.mozilla.org/TestEngineering/Performance/Raptor/Mitmproxy Mitmproxy]] recordings. If you need the HTML page source (outside of the Mitmproxy recording) for debugging, the raw HTML can be found in our [https://github.com/mozilla/perf-automation/tree/master/pagesets perf-automation github repo].
| | Raptor page-load test documentation is generated by [https://firefox-source-docs.mozilla.org/code-quality/lint/linters/perfdocs.html PerfDocs] and available in the [https://firefox-source-docs.mozilla.org/testing/perfdocs/raptor.html Firefox Source Docs]. |
| | |
| All the pages in a test suite an be run by calling the top-level test name, i.e.:
| |
| | |
| ./mach raptor --test raptor-tp6-1
| |
| | |
| Individual test pages can be ran by calling the subtest, i.e.:
| |
| | |
| ./mach raptor --test raptor-tp6-google-firefox
| |
| | |
| Some of the page recordings contain [[https://wiki.mozilla.org/Performance_sheriffing/Raptor/Mitmproxy#Adding_Hero_Elements hero elements]]. When hero elements are measured, the value is the time until the hero element appears on the page (in MS).
| |
| | |
| All pageload tests can be found at [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/ raptor-tp6 tests ]
| |
| | |
| Below are the details for page-load suites:
| |
| | |
| ===== raptor-tp6-1 to 10 =====
| |
| * contact: :rwood, :davehunt, :bebe
| |
| * type: page-load
| |
| * browsers: Firefox desktop, Chromium, Google Chrome desktop (Windows and OSX only at the moment)
| |
| * measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, loadtime
| |
| * measuring on Chrome: first-contentful-paint, loadtime
| |
| * page-cycles: 25
| |
| * reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
| |
| * test INI's: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop raptor-tp6-1 to 10 ].
| |
| | |
| ===== raptor-tp6-cold-1 to 4 =====
| |
| * contact: :rwood, :davehunt, :bebe
| |
| * type: page-load, cold
| |
| * browsers: Firefox desktop, Chromium, Google Chrome desktop (Windows and OSX only at the moment)
| |
| * measuring on Firefox: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, loadtime
| |
| * measuring on Chrome: first-contentful-paint, loadtime
| |
| * page-cycles: 25
| |
| * reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
| |
| * test INI's: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/desktop raptor-tp6-cold-1 to 4 ].
| |
| | |
| ===== raptor-tp6m-1 to 10 =====
| |
| * contact: :rwood, :davehunt, :bebe
| |
| * type: page-load
| |
| * browsers: Firefox Android Geckoview Example App
| |
| * measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
| |
| * page-cycles: 15
| |
| * reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
| |
| * test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-1 to 10].
| |
| | |
| ===== raptor-tp6m-cold-1 to 27 =====
| |
| * contact: :rwood, :davehunt, :bebe
| |
| * type: page-load, cold
| |
| * browsers: Firefox Android Geckoview Example App
| |
| * measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
| |
| * page-cycles: 15
| |
| * reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
| |
| * test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-cold-1 to 27].
| |
| | |
| ===== raptor-tp6m-cold-1 to 9-fennec68 =====
| |
| * contact: :rwood, :davehunt, :bebe
| |
| * type: page-load
| |
| * browsers: Firefox Android Fennec ESR 68 App
| |
| * measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
| |
| * page-cycles: 15
| |
| * reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
| |
| * test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-1 to 9-fennec68].
| |
| | |
| ===== raptor-tp6m-cold-1 to 27-fennec68 =====
| |
| * contact: :rwood, :davehunt, :bebe
| |
| * type: page-load, cold
| |
| * browsers: Firefox Android Fennec ESR 68 App
| |
| * measuring: time-to-first-non-blank-paint, first-contentful-paint, dom-content-flushed, time-to-first-interactive, loadtime
| |
| * page-cycles: 15
| |
| * reporting: Each pagecycle measures all the values (in MS). The first page-cycle is dropped due to the initial extra loading time/noise. The overall result reported for each test page is the median of the values reported for each pagecycle (in MS).
| |
| * test INI: [https://searchfox.org/mozilla-central/source/testing/raptor/raptor/tests/tp6/mobile raptor-tp6m-cold-1 to 14-fennec68].
| |
|
| |
|
| === Benchmark Tests === | | === Benchmark Tests === |