B2G/QA/2014-10-31 Performance Acceptance: Difference between revisions

B2G/QA/2014-10-31 Performance Acceptance (view source)

166 bytes added , 3 November 2014

canmove, Confirmed users

2,041

edits

@@ Line 21: / Line 21: @@
 * Gaia: f89c7b12c36572262c9ea76058694a139b1a8634
-Note when comparing to past reports that the test timings changed with Bug [[bug|1079700]] and a handful of subsequent followups. The [[../2014-10-20_Performance_Acceptance|last report]] also noted this, as it affected the 2.1 timings. At the time, it was thought that the 2.0 timings were a control, but we found that the patch actually did not fix 2.0. All fixes on both branches are now in place, hence faster 2.0 timings than previously noted.
+Note when comparing to past reports that the test timings changed with Bug [[bug|1079700]] and a handful of subsequent followups. The [[../2014-10-20_Performance_Acceptance|last report]] also noted this, as it affected the 2.1 timings.
+At the time, it was thought that the 2.0 timings were a control, but we found that the patch actually did not fix 2.0. All fixes on both branches are now in place, hence faster 2.0 timings than previously noted. These faster 2.0 timings mean that the lower numbers in the last study probably are largely due to the timing changes. However, they are also likely more accurate.
 This initial report has a limited (390 maximum) data point set for 2.0. It will be updated with full results for future baselines and any notable differences (unlikely) will be communicated.