Confirmed users
3,376
edits
TestEngineering/Performance/Talos/Tests: Difference between revisions
TestEngineering/Performance/Talos/Tests (view source)
Revision as of 18:58, 22 January 2018
, 22 January 2018update page sets links
(→tp6: update) |
(update page sets links) |
||
| Line 804: | Line 804: | ||
=== tp5 === | === tp5 === | ||
* contact: :jmaher | * contact: :jmaher | ||
* source: [https:// | * source: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests#page_sets tp5n.zip] | ||
* type: [https://wiki.mozilla.org/Performance_sheriffing/Talos#Page_Load_Tests PageLoader] | * type: [https://wiki.mozilla.org/Performance_sheriffing/Talos#Page_Load_Tests PageLoader] | ||
* data: we load each of the 51 tp5o pages 25 times, resulting in 51 sets of 25 data points. | * data: we load each of the 51 tp5o pages 25 times, resulting in 51 sets of 25 data points. | ||
* '''To run it locally''', you'd need [ | * '''To run it locally''', you'd need [https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests#page_sets tp5n.zip]. | ||
* summarization: | * summarization: | ||
** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py] | ** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] '''5''' data points, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py] | ||
| Line 821: | Line 821: | ||
|} | |} | ||
Tests the time it takes Firefox to load the [https:// | Tests the time it takes Firefox to load the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests#page_sets tp5 web page test set]. The web set was culled from the Alexa top 500 April 8th, 2011 and consists of 51 pages. | ||
Here are the broad steps we use to create the test set: | Here are the broad steps we use to create the test set: | ||
| Line 921: | Line 921: | ||
=== tp5o_scroll === | === tp5o_scroll === | ||
* contact: | * contact: | ||
* source: [https:// | * source: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests#page_sets tp5n.zip] | ||
* type: [https://wiki.mozilla.org/Performance_sheriffing/Talos#Page_Load_Tests PageLoader] | * type: [https://wiki.mozilla.org/Performance_sheriffing/Talos#Page_Load_Tests PageLoader] | ||
* data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points. | * data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points. | ||
* '''To run it locally''', you'd need [[# | * '''To run it locally''', you'd need [[#page_set|tp5n.zip]]. | ||
* summarization: | * summarization: | ||
** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py] | ** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py] | ||
| Line 1,088: | Line 1,088: | ||
* reporting: | * reporting: | ||
* data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points. | * data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points. | ||
* '''To run it locally''', you'd need [ | * '''To run it locally''', you'd need [https://wiki.mozilla.org/Performance_sheriffing/Talos/Tests#page_sets tp5n.zip]. | ||
* summarization: | * summarization: | ||
** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py] | ** subtest: [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#ignore_first ignore first] data point, then take the [https://wiki.mozilla.org/Performance_sheriffing/Talos/Data#median median] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py] | ||