Confirmed users
372
edits
TestEngineering/Performance/Talos/Tests: Difference between revisions
TestEngineering/Performance/Talos/Tests (view source)
Revision as of 16:32, 23 March 2018
, 23 March 2018→Added about-preferences to Talos Test Descriptions
(link with parent) |
|||
| Line 139: | Line 139: | ||
0;dhtml.html;1584;1637;1643;1665;1741;1529;1647;1645;1692;1647;1542;1750;1654;1649;1541;1656;1674;1645;1645;1740;1558;1652;1654;1656;1654 | 0;dhtml.html;1584;1637;1643;1665;1741;1529;1647;1645;1692;1647;1542;1750;1654;1649;1541;1656;1674;1645;1645;1740;1558;1652;1654;1656;1654 | ||
1;tablemutation.html;398;385;389;391;387;387;385;387;388;385;384;31746;386;387;384;387;389;387;387;387;388;391;386;387;388 | 1;tablemutation.html;398;385;389;391;387;387;385;387;388;385;384;31746;386;387;384;387;389;387;387;387;388;391;386;387;388 | ||
</pre> | |||
|} | |||
=== about-preferences === | |||
* contact: :jaws, :rwood | |||
* source: [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/tests/about-preferences/about_preferences_basic.manifest] | |||
* type: PageLoader | |||
* measuring: first-non-blank-paint | |||
* data: We load 5 urls 1 time each, and repeat for 25 cycles; collecting 25 sets of 5 data points | |||
* summarization | |||
** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] five data points, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the rest; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l627 source: test.py] | |||
** suite: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#geometric_mean geometric mean] of the the subtest results. | |||
* reporting: test time in ms (lower is better) | |||
This test measures the performance of the Firefox about:preferences page. This test is a little different than other pageload tests in that we are loading one page (about:preferences) but also testing the loading of that same page's subcategories/panels (i.e. about:preferences#home). | |||
When simply changing the page's panel/category, that doesn't cause a new onload event as expected; therefore we had to introduce loading the 'about:blank' page in between each page category; that forces the entire page to reload with the specified category panel activated. | |||
For that reason, when new panels/categories are added to the 'about:preferences' page, it can be expected that a performance regression may be introduced, even if a subtest hasn't been added for that new page category yet. | |||
This test should only ever have 1 pagecycle consisting of the main about-preferences page and each category separated by an about:blank between. Then repeats are achieved by using 25 cycles (instead of pagecycles). | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! Example data | |||
|- | |||
| <pre> | |||
0;preferences;346;141;143;150;136;143;153;140;154;156;143;154;146;147;151;166;140;146;140;144;144;156;154;150;140 | |||
2;preferences#search;164;142;133;141;141;141;142;140;131;146;131;140;131;131;139;142;140;144;146;143;143;142;142;137;143 | |||
3;preferences#privacy;179;159;166;177;173;153;148;154;168;155;164;155;152;157;149;155;156;186;149;156;160;151;158;168;157 | |||
4;preferences#sync;148;156;140;137;159;139;143;145;138;130;145;142;141;133;146;141;147;143;146;146;139;144;142;151;156 | |||
5;preferences#home;141;111;130;131;138;128;133;122;138;138;131;139;139;132;133;141;143;139;138;135;136;128;134;140;135 | |||
</pre> | </pre> | ||
|} | |} | ||
| Line 1,422: | Line 1,454: | ||
The values we collect during stackwalk are: | The values we collect during stackwalk are: | ||
* [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l11 'FileRead', 'FileWrite', 'FileFlush'] | * [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/xperf.config#l11 'FileRead', 'FileWrite', 'FileFlush'] | ||
== Build Metrics == | == Build Metrics == | ||