Platform/Games/Testing/32v64bit: Difference between revisions
Jump to navigation
Jump to search
Paul.silaghi (talk | contribs) |
Paul.silaghi (talk | contribs) |
||
| Line 378: | Line 378: | ||
|- | |- | ||
| main-thread-poppler-cold | | main-thread-poppler-cold | ||
| | | 0.453 | ||
| 0.360 | | 0.360 | ||
|- | |- | ||
| main-thread-poppler-warm | | main-thread-poppler-warm | ||
| | | 0.453 | ||
| 0.351 | | 0.351 | ||
|- | |- | ||
| main-thread-sqlite-cold | | main-thread-sqlite-cold | ||
| | | 0.047 (±21%!) | ||
| 0.060 (±27%!) | | 0.060 (±27%!) | ||
|- | |- | ||
| main-thread-sqlite-warm | | main-thread-sqlite-warm | ||
| | | 0.038 (±26%!) | ||
| 0.052 (±40%!) | | 0.052 (±40%!) | ||
|- | |- | ||
| Line 396: | Line 396: | ||
|- | |- | ||
| box2d-throughput | | box2d-throughput | ||
| | | 7.161 | ||
| 4.666 | | 4.666 | ||
|- | |- | ||
| box2d-throughput-f32 | | box2d-throughput-f32 | ||
| | | 6.589 | ||
| 4.292 | | 4.292 | ||
|- | |- | ||
| lua-binarytrees | | lua-binarytrees | ||
| | | 8.364 | ||
| 7.230 | | 7.230 | ||
|- | |- | ||
| lua-scimark | | lua-scimark | ||
| | | 9.820 | ||
| 12.195 | | 12.195 | ||
|- | |- | ||
| poppler-throughput | | poppler-throughput | ||
| | | 7.596 | ||
| 5.989 | | 5.989 | ||
|- | |- | ||
| sqlite-throughput | | sqlite-throughput | ||
| | | 11.282 | ||
| 8.583 | | 8.583 | ||
|- | |- | ||
| Line 422: | Line 422: | ||
|- | |- | ||
| poppler-cold-preparation | | poppler-cold-preparation | ||
| | | 1.035 | ||
| 0.844 | | 0.844 | ||
|- | |- | ||
| poppler-warm-preparation | | poppler-warm-preparation | ||
| | | 0.199 | ||
| 0.177 | | 0.177 | ||
|- | |- | ||
| sqlite-cold-preparation | | sqlite-cold-preparation | ||
| | | 1.232 | ||
| 1.236 | | 1.236 | ||
|- | |- | ||
| sqlite-warm-preparation | | sqlite-warm-preparation | ||
| | | 0.112 | ||
| 0.137 | | 0.137 | ||
|- | |- | ||
| Line 440: | Line 440: | ||
|- | |- | ||
| box2d-variance | | box2d-variance | ||
| | | 1.967 | ||
| 1.297 | | 1.297 | ||
|- | |- | ||
| poppler-variance | | poppler-variance | ||
| | | 5.149 | ||
| 4.086 | | 4.086 | ||
|- | |- | ||
| class="header" | Overall Score | | class="header" | Overall Score | ||
| class="header" | | | class="header" | 3,762 | ||
| class="header" | 4,211 | | class="header" | 4,211 | ||
|} | |} | ||
Revision as of 15:34, 4 June 2014
Summary
The following documents A-B testing comparing performance of 32-bit Firefox builds to 64-bit Firefox builds, utilizing the Massive benchmark.
Methodology
- Test using the same system, make note of the system profile
- Install the latest Firefox Nightly 32-bit build and start with a new profile
- Load http://kripken.github.io/Massive/ and click "Run the benchmark now!"
- Let the benchmark run
- Record the results below when the benchmark completes
- Include a screenshot of the result
- Repeat the benchmark 5 times on the same system and record the result separately
- Install the latest Firefox Nightly 64-bit and repeat the tests with a new profile
System Profile
| Operating System | Windows 7 64-bit |
| Firefox | (user agent string with build ID) Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:32.0) Gecko/20100101 Firefox/32.0, 20140604030202 |
| Processor | (make, model, speed) AMD FX(tm)-8320 Eight-Core Processor 3.50 Ghz |
| Memory | (type, speed, amount) 8 GB DDR3 1033 Mhz |
| GPU | (make, model, speed, memory, driver version) AMD Radeon R7 200 Series, Core Clock 780 Mhz, 2GB DDR3 900 Mhz, Catalyst version 13.12 |
Results
Testrun #1
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.452 | 0.729 (±22%!) |
| main-thread-poppler-warm | 0.428 | 0.523 |
| main-thread-sqlite-cold | 0.053 (±26%!) | 0.872 |
| main-thread-sqlite-warm | 0.033 (±24%!) | 0.043 (±27%!) |
| Throughput | ||
| box2d-throughput | 6.838 | 4.623 |
| box2d-throughput-f32 | 6.316 | 4.258 |
| lua-binarytrees | 8.067 | 7.271 |
| lua-scimark | 10.545 | 12.160 |
| poppler-throughput | 7.008 | 5.988 |
| sqlite-throughput | 10.891 | 8.404 |
| Preparation | ||
| poppler-cold-preparation | 0.939 | 0.864 |
| poppler-warm-preparation | 0.185 | 0.183 |
| sqlite-cold-preparation | 1.084 | 1.260 |
| sqlite-warm-preparation | 0.098 | 0.123 |
| Variance | ||
| box2d-variance | 1.466 | 1.216 |
| poppler-variance | 4.843 | 4.074 |
| Overall Score | 3,967 | 3,377 |
Testrun #2
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.461 | 0.367 |
| main-thread-poppler-warm | 0.427 | 0.361 |
| main-thread-sqlite-cold | 0.065 | 0.064 |
| main-thread-sqlite-warm | 0.040 (±30%!) | 0.043 (±28%!) |
| Throughput | ||
| box2d-throughput | 6.987 | 4.666 |
| box2d-throughput-f32 | 6.317 | 4.277 |
| lua-binarytrees | 7.899 | 7.182 |
| lua-scimark | 10.555 | 12.055 |
| poppler-throughput | 7.023 | 5.959 |
| sqlite-throughput | 10.733 | 8.485 |
| Preparation | ||
| poppler-cold-preparation | 0.953 | 0.846 |
| poppler-warm-preparation | 0.186 | 0.180 |
| sqlite-cold-preparation | 1.103 | 1.209 |
| sqlite-warm-preparation | 0.109 | 0.115 |
| Variance | ||
| box2d-variance | 1.758 | 1.290 |
| poppler-variance | 4.830 | 4.036 |
| Overall Score | 3,839 | 4,287 |
Testrun #3
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.456 | 0.338 |
| main-thread-poppler-warm | 0.445 | 0.356 |
| main-thread-sqlite-cold | 0.050 | 0.063 |
| main-thread-sqlite-warm | 0.034 (±24%!) | 0.042 (±28%!) |
| Throughput | ||
| box2d-throughput | 6.806 | 4.643 |
| box2d-throughput-f32 | 6.316 | 4.284 |
| lua-binarytrees | 8.002 | 7.239 |
| lua-scimark | 10.250 | 12.185 |
| poppler-throughput | 7.034 | 6.003 |
| sqlite-throughput | 10.759 | 8.366 |
| Preparation | ||
| poppler-cold-preparation | 0.937 | 0.842 |
| poppler-warm-preparation | 0.183 | 0.173 |
| sqlite-cold-preparation | 1.071 | 1.203 |
| sqlite-warm-preparation | 0.110 | 0.121 |
| Variance | ||
| box2d-variance | 1.467 | 1.210 |
| poppler-variance | 4.836 | 4.083 |
| Overall Score | 3,942 | 4,323 |
Testrun #4
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.448 | 0.359 |
| main-thread-poppler-warm | 0.451 | 0.362 |
| main-thread-sqlite-cold | 0.056 (±22%!) | 0.071 (±31%!) |
| main-thread-sqlite-warm | 0.033 (±22%!) | 0.042 (±26%!) |
| Throughput | ||
| box2d-throughput | 6.834 | 4.674 |
| box2d-throughput-f32 | 6.317 | 4.304 |
| lua-binarytrees | 8.219 | 7.226 |
| lua-scimark | 9.925 | 12.005 |
| poppler-throughput | 7.021 | 5.951 |
| sqlite-throughput | 10.768 | 8.414 |
| Preparation | ||
| poppler-cold-preparation | 0.953 | 0.840 |
| poppler-warm-preparation | 0.184 | 0.173 |
| sqlite-cold-preparation | 1.091 | 1.212 |
| sqlite-warm-preparation | 0.120 | 0.140 |
| Variance | ||
| box2d-variance | 1.546 | 2.258 |
| poppler-variance | 4.832 | 4.067 |
| Overall Score | 3,869 | 4,225 |
Testrun #5
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.453 | 0.360 |
| main-thread-poppler-warm | 0.453 | 0.351 |
| main-thread-sqlite-cold | 0.047 (±21%!) | 0.060 (±27%!) |
| main-thread-sqlite-warm | 0.038 (±26%!) | 0.052 (±40%!) |
| Throughput | ||
| box2d-throughput | 7.161 | 4.666 |
| box2d-throughput-f32 | 6.589 | 4.292 |
| lua-binarytrees | 8.364 | 7.230 |
| lua-scimark | 9.820 | 12.195 |
| poppler-throughput | 7.596 | 5.989 |
| sqlite-throughput | 11.282 | 8.583 |
| Preparation | ||
| poppler-cold-preparation | 1.035 | 0.844 |
| poppler-warm-preparation | 0.199 | 0.177 |
| sqlite-cold-preparation | 1.232 | 1.236 |
| sqlite-warm-preparation | 0.112 | 0.137 |
| Variance | ||
| box2d-variance | 1.967 | 1.297 |
| poppler-variance | 5.149 | 4.086 |
| Overall Score | 3,762 | 4,211 |
Overall
| Build | Lowest Score | Highest Score | Median Score | Average Score |
| Firefox 32.0a1 32-bit | ||||
| Firefox 32.0a1 64-bit |
Conclusions
Anthony will fill this in later