342
edits
TestEngineering/Performance/FAQ: Difference between revisions
Jump to navigation
Jump to search
no edit summary
Alexandrui (talk | contribs) |
Alexandrui (talk | contribs) No edit summary |
||
| Line 1: | Line 1: | ||
== What is Perfherder? == | == What is Perfherder? == | ||
[https://treeherder.mozilla.org/perf.html#/graphs Perfherder] is a tool that takes data points from log files and graphs them over time. Primarily this is used for performance data from [[TestEngineering/Performance/Talos|Talos]], but also from [[AWSY/Tests|AWSY]], build_metrics, [[EngineeringProductivity/Autophone|Autophone]] and platform_microbenchmarks. All these are test harnesses and you can find more about them [[TestEngineering/Performance/Sheriffing/Alerts|here]]. | [https://treeherder.mozilla.org/perf.html#/graphs Perfherder] is a tool that takes data points from log files and graphs them over time. Primarily this is used for performance data from [[TestEngineering/Performance/Talos|Talos]], but also from [[AWSY/Tests|AWSY]], build_metrics, [[EngineeringProductivity/Autophone|Autophone]] and platform_microbenchmarks. All these are test harnesses and you can find more about them [[TestEngineering/Performance/Sheriffing/Alerts|here]]. | ||
| Line 56: | Line 54: | ||
[[File:Ph_Muting.jpg]] | [[File:Ph_Muting.jpg]] | ||
== What makes branches different from one another? == | == What makes branches different from one another? == | ||
| Line 133: | Line 129: | ||
* OSX does not do PGO builds, so we do not have -Non-PGO branches for those platforms. (i.e. we only have Mozilla-Inbound) | * OSX does not do PGO builds, so we do not have -Non-PGO branches for those platforms. (i.e. we only have Mozilla-Inbound) | ||
* PGO alerts will probably have different regression percentages, but the overall list of platforms/tests for a given revision will be almost identical | * PGO alerts will probably have different regression percentages, but the overall list of platforms/tests for a given revision will be almost identical | ||
== What alerts are displayed in Alert Manager? == | == What alerts are displayed in Alert Manager? == | ||
| Line 176: | Line 170: | ||
NOTE: for the reverse tests we take that into account, so the bar graph will know to look in the correct direction. | NOTE: for the reverse tests we take that into account, so the bar graph will know to look in the correct direction. | ||
== What is noise? == | == What is noise? == | ||
Generally a test reports values that are in a range instead of a consistent value. The larger the range of 'normal' results, the more noise we have. | Generally a test reports values that are in a range instead of a consistent value. The larger the range of 'normal' results, the more noise we have. | ||
| Line 242: | Line 235: | ||
== What is random noise? == | == What is random noise? == | ||
Random noise are the data-points that don't fit in the graph trend of the test. They happen because of various uncontrollable factors (and this is assumed) or because the test is unstable. | Random noise are the data-points that don't fit in the graph trend of the test. They happen because of various uncontrollable factors (and this is assumed) or because the test is unstable. | ||
== How do I identify the current firefox release meta-bug? == | == How do I identify the current firefox release meta-bug? == | ||