TestEngineering/Performance: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Redirected page to Performance/Tools)
 
(67 intermediate revisions by 9 users not shown)
Line 1: Line 1:
== What we do ==
#REDIRECT [[Performance/Tools]]


== What we don't do ==
{{DISPLAYTITLE:Firefox Performance Test Engineering 🔥🦊⏱️}}[[File:Fxperftest.png|thumb|right]]


== Onboarding ==
= New contributors =


== Objectives ==
If you are a new contributor, or would like to start contributing you can find a guide to help you [[/NewContributors|here]].
* Increase regression detection '''coverage''' to page load on Android products including WebView comparisons
* Increase quality standards by measuring and alerting on power usage for Android and ARM64
* Build dashboards to improve '''visibility''' of how we are tracking against our release criteria for Android
* Reducing noise in test results and adding annotations to provide '''clarity''' of signal for regressions
* Develop measurements and mechanisms for reporting on our tools, policies and documentation for how to improve '''clarity''' and '''efficiency''' of risk assessments


== Team ==
= Where to find us =
* Dave Hunt [:davehunt]
* [https://chat.mozilla.org/#/room/#perftest:mozilla.org #perftest]
* Rob Wood [:rwood]
* Kyle Lahnakoski [:ekyle]
* Stephen Donner [:stephend]
* Ionut Goldan [:igoldan]
* Florian Strugariu [:bebe]
* Greg Mierzwinski [:sparky]


== Projects ==
= Team purpose =
To support the infrastructure and creation of automated tests for evaluating the performance of Firefox products. This provides value by exposing gaps in coverage, revealing areas where we can make performance gains, identifying performance regressions in a timely manner, and by providing key performance metrics that assist in determining how Firefox measures against release criteria.


Trello: https://trello.com/b/gc3AlyhB/firefox-performance-test-engineering
= What we do =
* Identification of gaps in performance test infrastructure and monitoring.
* Designing and building performance test infrastructure and monitoring solutions.
* Supporting Firefox engineers on writing performance tests.
* Supporting Firefox engineers on investigating regressions identified by tests.
* Collaboration with release operations on infrastructure requirements.
* Standing up performance tests in continuous integration environments.
* Monitoring performance test results and identifying potential regressions.
* Supporting performance sheriffs with tools to assist in identifying regressions.
* Developing test plans for performance testing.
* Running adhoc manual or partially automated performance testing.


=== Raptor ===
= What we don't do =
Raptor is a Python testing framework used to run browser benchmark and browser page-load performance tests. Raptor is cross-browser compatible and is currently running in Mozilla taskcluster production on Firefox Desktop, Firefox Android, and on Google Chromium.
* Maintenance of infrastructure hardware.
* Maintain the continuous integration pipeline.
* Writing/maintaining all performance tests.


* Owner: Rob Wood [:rwood]
= Meetings =
* Source: https://searchfox.org/mozilla-central/source/testing/raptor
{{/Meetings}}
* Good first bugs: https://codetribute.mozilla.org/projects/automation?project%3DRaptor
* User documentation: https://wiki.mozilla.org/Performance_sheriffing/Raptor
* Developer documentation: https://wiki.mozilla.org/Performance_sheriffing/Raptor


=== Talos ===
= Onboarding =
Talos is a Python performance testing framework that is usable on Windows, Mac and Linux. Talos is our versatile performance testing framework we use at Mozilla. It was created to serve as a test runner for the existing performance tests that Mozilla was running back in 2007 as well as providing an extensible framework for new tests as they were created.
Welcome to the team! You are encouraged to improve the [[/Onboarding|onboarding page]]. If you need to ask questions that are not already covered, please update the page so that the next person has a better onboarding experience.


* Owner: Rob Wood [:rwood], Joel Maher [:jmaher]
= Workflow =
* Source: https://searchfox.org/mozilla-central/source/testing/talos
* [[/Triage Process/]]
* Good first bugs: https://codetribute.mozilla.org/projects/automation?project%3DTalos
* [[/Review Process/]]
* User documentation: https://wiki.mozilla.org/Performance_sheriffing/Talos
* Developer documentation: https://wiki.mozilla.org/Performance_sheriffing/Talos


=== WebPageTest ===
= Projects =
* Owner: Stephen Donner [:stephend]
* [[/Fenix/]]
* Source: https://github.com/mozilla/wpt-api
* [[/Raptor/]]
* (User and Dev) Docs: https://mozilla-wpt-api-docs.readthedocs.io/en/master/
* [[/Raptor/Browsertime/]]
* "help-wanted" labelled Issues: https://github.com/mozilla/wpt-api/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22
* [[/Talos/]]


=== Perfherder ===
= Results =
See our {{wip|[[/Results|results page]]}}.


=== Firefox Performance Dashboard ===
= Resources =
 
* [[/Glossary/]]
=== Firefox Health Dashboard ===
* [https://docs.google.com/document/d/1SswqYIAm4h8vlwfMc0pfGEJwXpFECyubGDezRZHHPFE/edit Strategies for investigating intermittents]
* [https://docs.google.com/document/d/1HV2_z8hwhI2w8EbURtkYjpikVG5g9QeKEPo9h5msuRs/edit Following up perf bugs]
* [https://docs.google.com/document/d/103SRVVcE2SZNYP3kFXGeiVQrusH2Wj2yv8SWaDvB9SM/edit Excessive Android device queue response plan]
* [[/FAQ#How_can_I_do_a_bisection.3F|Bisection Workflow]]
* [[/Sheriffing/CompareView|CompareView]]

Latest revision as of 17:57, 24 July 2023

Redirect to:

Fxperftest.png

New contributors

If you are a new contributor, or would like to start contributing you can find a guide to help you here.

Where to find us

Team purpose

To support the infrastructure and creation of automated tests for evaluating the performance of Firefox products. This provides value by exposing gaps in coverage, revealing areas where we can make performance gains, identifying performance regressions in a timely manner, and by providing key performance metrics that assist in determining how Firefox measures against release criteria.

What we do

  • Identification of gaps in performance test infrastructure and monitoring.
  • Designing and building performance test infrastructure and monitoring solutions.
  • Supporting Firefox engineers on writing performance tests.
  • Supporting Firefox engineers on investigating regressions identified by tests.
  • Collaboration with release operations on infrastructure requirements.
  • Standing up performance tests in continuous integration environments.
  • Monitoring performance test results and identifying potential regressions.
  • Supporting performance sheriffs with tools to assist in identifying regressions.
  • Developing test plans for performance testing.
  • Running adhoc manual or partially automated performance testing.

What we don't do

  • Maintenance of infrastructure hardware.
  • Maintain the continuous integration pipeline.
  • Writing/maintaining all performance tests.

Meetings

Onboarding

Welcome to the team! You are encouraged to improve the onboarding page. If you need to ask questions that are not already covered, please update the page so that the next person has a better onboarding experience.

Workflow

Projects

Results

See our 🚧 results page 🚧.

Resources

Retrieved from "https://wiki.allizom.org/index.php?title=TestEngineering/Performance&oldid=1247190"