Confirmed users
513
edits
Auto-tools/Projects/Alerts: Difference between revisions
Jump to navigation
Jump to search
→Analysis
Klahnakoski (talk | contribs) |
Klahnakoski (talk | contribs) |
||
| Line 65: | Line 65: | ||
Once the test results are retrieved, and sorted by push date, several window functions are run over the data: They calculate the past stats, future stats, t-test pvalue, and median test pvalue, among other things. | Once the test results are retrieved, and sorted by push date, several window functions are run over the data: They calculate the past stats, future stats, t-test pvalue, and median test pvalue, among other things. | ||
Decision of *if* there is a regression is done with the median test. The median test is insensitive to amplitude, which allows it to ignore the intermittent non-Gaussian noise we witness in many if the time series. | Decision of *if* there is a regression is done with the median test. The median test is insensitive to amplitude, which allows it to ignore the intermittent non-Gaussian noise we witness in many if the time series. It is only good at detecting clear discontinuities in the time series, and this is fine because that is what we are looking for. But, it fails at isolating which revision is closest to the discontinuity, and picks any revision belonging to an increasing (or decreasing) slope in the vicinity. Slowly changing statistics are completely invisible to it. | ||
The process is best explained in a chart. | If the median test signals there may be a regression, the t-test is used to determine the specific revision to blame. The t-test's amplitude sensitivity allows us to detect the specific revision closest to the discontinuity. The process is best explained in a chart. | ||
[[ File:Auto-tools_Projects_Alerts_Mtest_and_ttest.png|800px|none|Eideticker time series]] | [[ File:Auto-tools_Projects_Alerts_Mtest_and_ttest.png|800px|none|Eideticker time series]] | ||