83
edits
TestEngineering/Performance/Talos/Tests: Difference between revisions
Jump to navigation
Jump to search
TestEngineering/Performance/Talos/Tests (view source)
Revision as of 19:38, 26 September 2019
, 26 September 2019Mass edit #page_sets entries to #Page_sets for proper linking.
(→tp5pages: Use <tt> for inline code.) |
(Mass edit #page_sets entries to #Page_sets for proper linking.) |
||
| Line 333: | Line 333: | ||
** suite: [[../Data#geometric_mean|geometric mean]] of the 36 subtest results. | ** suite: [[../Data#geometric_mean|geometric mean]] of the 36 subtest results. | ||
To run this locally, you'll need to pull down the [[# | To run this locally, you'll need to pull down the [[#Page_sets|tp5 page set]] and run it in a local web server. See the [[#tp5|tp5 section]]. | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
| Line 750: | Line 750: | ||
== tp5 == | == tp5 == | ||
* contact: :davehunt | * contact: :davehunt | ||
* source: [[# | * source: [[#Page_sets|tp5n.zip]] | ||
* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]] | * type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]] | ||
* data: we load each of the 48 tp5o pages 25 times, resulting in 48 sets of 25 data points. | * data: we load each of the 48 tp5o pages 25 times, resulting in 48 sets of 25 data points. | ||
* '''To run it locally''', you'd need [[# | * '''To run it locally''', you'd need [[#Page_sets|tp5n.zip]]. | ||
* summarization: | * summarization: | ||
** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py] | ** subtest: [[../Data#ignore_first|ignore first]] '''5''' data points, then take the [[../Data#median|median]] of the remaining 20; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l449 source: test.py] | ||
| Line 767: | Line 767: | ||
|} | |} | ||
Tests the time it takes Firefox to load the [[# | Tests the time it takes Firefox to load the [[#Page_sets|tp5 web page test set]]. The web set was culled from the Alexa top 500 April 8th, 2011 and consists of 51 pages. | ||
Here are the broad steps we use to create the test set: | Here are the broad steps we use to create the test set: | ||
| Line 867: | Line 867: | ||
== tp5o_scroll == | == tp5o_scroll == | ||
* contact: :kats | * contact: :kats | ||
* source: [[# | * source: [[#Page_sets|tp5n.zip]] | ||
* type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]] | * type: [[TestEngineering/Performance/Talos#Page_Load_Tests|PageLoader]] | ||
* data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points. | * data: we load each of the 51 tp5o pages 12 times, resulting in 51 sets of 12 data points. | ||
* '''To run it locally''', you'd need [[# | * '''To run it locally''', you'd need [[#Page_sets|tp5n.zip]]. | ||
* summarization: | * summarization: | ||
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py] | ** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 11; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l470 source: test.py] | ||
| Line 1,018: | Line 1,018: | ||
* reporting: | * reporting: | ||
* data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points. | * data: we load 50 web pages 5 times each, resulting in 50 sets of 5 data points. | ||
* '''To run it locally''', you'd need [[# | * '''To run it locally''', you'd need [[#Page_sets|tp5n.zip]]. | ||
* summarization: | * summarization: | ||
** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py] | ** subtest: [[../Data#ignore_first|ignore first]] data point, then take the [[../Data#median|median]] of the remaining 4 data points; [https://dxr.mozilla.org/mozilla-central/source/testing/talos/talos/test.py#l305 source: test.py] | ||