Confirmed users
2,177
edits
TestEngineering/Performance: Difference between revisions
no edit summary
No edit summary |
|||
| Line 1: | Line 1: | ||
{{DISPLAYTITLE:Firefox Performance Test Engineering 🔥🦊⏱}} | {{DISPLAYTITLE:Firefox Performance Test Engineering 🔥🦊⏱}} | ||
= Who we are = | |||
* Dave Hunt [:davehunt] 🇬🇧 | * Dave Hunt [:davehunt] 🇬🇧 | ||
* Stephen Donner [:stephend] 🇺🇸 | * Stephen Donner [:stephend] 🇺🇸 | ||
| Line 12: | Line 12: | ||
* Alex Irimovici [:air] 🇷🇴 | * Alex Irimovici [:air] 🇷🇴 | ||
= Where to find us = | |||
* [[IRC]]: #perftest | * [[IRC]]: #perftest | ||
* Slack: #perftest | * Slack: #perftest | ||
= What we do = | |||
We provide automated testing support for measuring the performance of Firefox products. Here are a few examples of what you can expect our team to be working on. | We provide automated testing support for measuring the performance of Firefox products. Here are a few examples of what you can expect our team to be working on. | ||
* Provide advice/troubleshooting related to performance testing or any of our tools and harnesses. | * Provide advice/troubleshooting related to performance testing or any of our tools and harnesses. | ||
| Line 23: | Line 23: | ||
* Monitoring and reporting of performance regressions. | * Monitoring and reporting of performance regressions. | ||
= What we don't do = | |||
* Own all performance tests. We work on the test harnesses and tools that are used for performance testing, but the tests themselves are often developed outside of our team. Every test should have an owner, who is responsible for responding to questions related to the test, and may be asked to assist when the test detects a regression. | * Own all performance tests. We work on the test harnesses and tools that are used for performance testing, but the tests themselves are often developed outside of our team. Every test should have an owner, who is responsible for responding to questions related to the test, and may be asked to assist when the test detects a regression. | ||
* Review all performance tests. Similar to test ownership, we enable others to contribute performance tests. We can provide advice and reviews, but do not impose this as a restriction to landing test changes. | * Review all performance tests. Similar to test ownership, we enable others to contribute performance tests. We can provide advice and reviews, but do not impose this as a restriction to landing test changes. | ||
| Line 30: | Line 30: | ||
* Maintain the reporting tools | * Maintain the reporting tools | ||
= Meetings = | |||
{{/Meetings}} | {{/Meetings}} | ||
= Onboarding = | |||
You are encouraged to improve the [[/Onboarding|onboarding page]]. If you need to ask questions that are not already covered, please update the page so that the next person has a better onboarding experience. | Welcome to the team! You are encouraged to improve the [[/Onboarding|onboarding page]]. If you need to ask questions that are not already covered, please update the page so that the next person has a better onboarding experience. | ||
= Task Workflow = | |||
Visit our [[/Task Workflow|task workflow]] page for details on how we manage tasks. | Visit our [[/Task Workflow|task workflow]] page for details on how we manage tasks. | ||
= Objectives = | |||
== [https://docs.google.com/spreadsheets/d/1P1d7rzA27t3OHljraxapNgDl-1Hsc_novoj78w8qog0/edit#gid=584732224 2019/Q3] == | |||
* KR1.3: Fenix Startup Test Development [rwood] {{notstarted|}} [%] | * KR1.3: Fenix Startup Test Development [rwood] {{notstarted|}} [%] | ||
* KR3.1: Video QoE testing for desktop [marauder] {{ok|}} [90%] | * KR3.1: Video QoE testing for desktop [marauder] {{ok|}} [90%] | ||
| Line 50: | Line 50: | ||
* KR6.6: Improve perf sheriff efficiency [igoldan] {{ok|}} [70%] | * KR6.6: Improve perf sheriff efficiency [igoldan] {{ok|}} [70%] | ||
= Projects = | |||
=== Raptor === | === Raptor === | ||
| Line 60: | Line 60: | ||
* Documentation: https://wiki.mozilla.org/Performance_sheriffing/Raptor | * Documentation: https://wiki.mozilla.org/Performance_sheriffing/Raptor | ||
== Talos == | |||
Talos is a Python performance testing framework that is usable on Windows, Mac and Linux. Talos is our versatile performance testing framework we use at Mozilla. It was created to serve as a test runner for the existing performance tests that Mozilla was running back in 2007 as well as providing an extensible framework for new tests as they were created. | Talos is a Python performance testing framework that is usable on Windows, Mac and Linux. Talos is our versatile performance testing framework we use at Mozilla. It was created to serve as a test runner for the existing performance tests that Mozilla was running back in 2007 as well as providing an extensible framework for new tests as they were created. | ||
| Line 68: | Line 68: | ||
* Documentation: https://wiki.mozilla.org/Performance_sheriffing/Talos | * Documentation: https://wiki.mozilla.org/Performance_sheriffing/Talos | ||
== WebPageTest == | |||
* Owner: Stephen Donner [:stephend] | * Owner: Stephen Donner [:stephend] | ||
* Source: https://github.com/mozilla/wpt-api | * Source: https://github.com/mozilla/wpt-api | ||
| Line 74: | Line 74: | ||
* Documentation: https://mozilla-wpt-api-docs.readthedocs.io/en/master/ | * Documentation: https://mozilla-wpt-api-docs.readthedocs.io/en/master/ | ||
== Perfherder == | |||
Perfherder is an interactive dashboard intended to allow monitoring and analysis of automated performance tests run against Mozilla products (currently Firefox and Firefox for Android). | Perfherder is an interactive dashboard intended to allow monitoring and analysis of automated performance tests run against Mozilla products (currently Firefox and Firefox for Android). | ||
Perfherder is part of the Treeherder project. | Perfherder is part of the Treeherder project. | ||
| Line 85: | Line 85: | ||
* Developer documentation: https://treeherder.readthedocs.io/ | * Developer documentation: https://treeherder.readthedocs.io/ | ||
= Dashboards = | |||
== Firefox Are We Fast Yet Dashboard == | |||
Shows a variety of benchmarks, run on a variety of platforms. Meant to be a detailed view of performance. | Shows a variety of benchmarks, run on a variety of platforms. Meant to be a detailed view of performance. | ||
| Line 94: | Line 94: | ||
* Source: https://github.com/mozilla-frontend-infra/firefox-performance-dashboard | * Source: https://github.com/mozilla-frontend-infra/firefox-performance-dashboard | ||
== Firefox Health Dashboard == | |||
The health dashboard tracks metrics and statistics important for tracking performance improvements. Meant to be a high level view. | The health dashboard tracks metrics and statistics important for tracking performance improvements. Meant to be a high level view. | ||
| Line 102: | Line 102: | ||
* Good first bugs: https://github.com/mozilla-frontend-infra/firefox-health-dashboard/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22 | * Good first bugs: https://github.com/mozilla-frontend-infra/firefox-health-dashboard/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22 | ||
= Resources = | |||
* [[/Glossary/]] | * [[/Glossary/]] | ||
* [https://docs.google.com/document/d/1SswqYIAm4h8vlwfMc0pfGEJwXpFECyubGDezRZHHPFE/edit Strategies for investigating intermittents] | * [https://docs.google.com/document/d/1SswqYIAm4h8vlwfMc0pfGEJwXpFECyubGDezRZHHPFE/edit Strategies for investigating intermittents] | ||