Platform/Games/Testing/32v64bit: Difference between revisions
Jump to navigation
Jump to search
Paul.silaghi (talk | contribs) |
Paul.silaghi (talk | contribs) |
||
| Line 211: | Line 211: | ||
| main-thread-poppler-cold | | main-thread-poppler-cold | ||
| | | | ||
| | | 0.338 | ||
|- | |- | ||
| main-thread-poppler-warm | | main-thread-poppler-warm | ||
| | | | ||
| | | 0.356 | ||
|- | |- | ||
| main-thread-sqlite-cold | | main-thread-sqlite-cold | ||
| | | | ||
| | | 0.063 | ||
|- | |- | ||
| main-thread-sqlite-warm | | main-thread-sqlite-warm | ||
| | | | ||
| | | 0.042 (±28%!) | ||
|- | |- | ||
| colspan="3" class="header" | Throughput | | colspan="3" class="header" | Throughput | ||
| Line 229: | Line 229: | ||
| box2d-throughput | | box2d-throughput | ||
| | | | ||
| | | 4.643 | ||
|- | |- | ||
| box2d-throughput-f32 | | box2d-throughput-f32 | ||
| | | | ||
| | | 4.284 | ||
|- | |- | ||
| lua-binarytrees | | lua-binarytrees | ||
| | | | ||
| | | 7.239 | ||
|- | |- | ||
| lua-scimark | | lua-scimark | ||
| | | | ||
| | | 12.185 | ||
|- | |- | ||
| poppler-throughput | | poppler-throughput | ||
| | | | ||
| | | 6.003 | ||
|- | |- | ||
| sqlite-throughput | | sqlite-throughput | ||
| | | | ||
| | | 8.366 | ||
|- | |- | ||
| colspan="3" class="header" | Preparation | | colspan="3" class="header" | Preparation | ||
| Line 255: | Line 255: | ||
| poppler-cold-preparation | | poppler-cold-preparation | ||
| | | | ||
| | | 0.842 | ||
|- | |- | ||
| poppler-warm-preparation | | poppler-warm-preparation | ||
| | | | ||
| | | 0.173 | ||
|- | |- | ||
| sqlite-cold-preparation | | sqlite-cold-preparation | ||
| | | | ||
| | | 1.203 | ||
|- | |- | ||
| sqlite-warm-preparation | | sqlite-warm-preparation | ||
| | | | ||
| | | 0.121 | ||
|- | |- | ||
| colspan="3" class="header" | Variance | | colspan="3" class="header" | Variance | ||
| Line 273: | Line 273: | ||
| box2d-variance | | box2d-variance | ||
| | | | ||
| | | 1.210 | ||
|- | |- | ||
| poppler-variance | | poppler-variance | ||
| | | | ||
| | | 4.083 | ||
|- | |- | ||
| class="header" | Overall Score | | class="header" | Overall Score | ||
Revision as of 14:56, 4 June 2014
Summary
The following documents A-B testing comparing performance of 32-bit Firefox builds to 64-bit Firefox builds, utilizing the Massive benchmark.
Methodology
- Test using the same system, make note of the system profile
- Install the latest Firefox Nightly 32-bit build and start with a new profile
- Load http://kripken.github.io/Massive/ and click "Run the benchmark now!"
- Let the benchmark run
- Record the results below when the benchmark completes
- Include a screenshot of the result
- Repeat the benchmark 5 times on the same system and record the result separately
- Install the latest Firefox Nightly 64-bit and repeat the tests with a new profile
System Profile
| Operating System | Windows 8.1 64-bit |
| Firefox | (user agent string with build ID) Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:32.0) Gecko/20100101 Firefox/32.0, 20140604030202 |
| Processor | (make, model, speed) AMD FX(tm)-8320 Eight-Core Processor 3.50 Ghz |
| Memory | (type, speed, amount) 8 GB DDR3 1033 Mhz |
| GPU | (make, model, speed, memory, driver version) AMD Radeon R7 200 Series, Core Clock 780 Mhz, 2GB DDR3 900 Mhz, Catalyst version 13.12 |
Results
Testrun #1
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.729 (±22%!) | |
| main-thread-poppler-warm | 0.523 | |
| main-thread-sqlite-cold | 0.872 | |
| main-thread-sqlite-warm | 0.043 (±27%!) | |
| Throughput | ||
| box2d-throughput | 4.623 | |
| box2d-throughput-f32 | 4.258 | |
| lua-binarytrees | 7.271 | |
| lua-scimark | 12.160 | |
| poppler-throughput | 5.988 | |
| sqlite-throughput | 8.404 | |
| Preparation | ||
| poppler-cold-preparation | 0.864 | |
| poppler-warm-preparation | 0.183 | |
| sqlite-cold-preparation | 1.260 | |
| sqlite-warm-preparation | 0.123 | |
| Variance | ||
| box2d-variance | 1.216 | |
| poppler-variance | 4.074 | |
| Overall Score | result | 3.377 |
Testrun #2
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.367 | |
| main-thread-poppler-warm | 0.361 | |
| main-thread-sqlite-cold | 0.064 | |
| main-thread-sqlite-warm | 0.043 (±27%!) | |
| Throughput | ||
| box2d-throughput | 4.666 | |
| box2d-throughput-f32 | 4.277 | |
| lua-binarytrees | 7.182 | |
| lua-scimark | 12.055 | |
| poppler-throughput | 5.959 | |
| sqlite-throughput | 8.485 | |
| Preparation | ||
| poppler-cold-preparation | 0.846 | |
| poppler-warm-preparation | 0.180 | |
| sqlite-cold-preparation | 1.209 | |
| sqlite-warm-preparation | 0.115 | |
| Variance | ||
| box2d-variance | 1.290 | |
| poppler-variance | 4.036 | |
| Overall Score | result | 4.287 |
Testrun #3
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | 0.338 | |
| main-thread-poppler-warm | 0.356 | |
| main-thread-sqlite-cold | 0.063 | |
| main-thread-sqlite-warm | 0.042 (±28%!) | |
| Throughput | ||
| box2d-throughput | 4.643 | |
| box2d-throughput-f32 | 4.284 | |
| lua-binarytrees | 7.239 | |
| lua-scimark | 12.185 | |
| poppler-throughput | 6.003 | |
| sqlite-throughput | 8.366 | |
| Preparation | ||
| poppler-cold-preparation | 0.842 | |
| poppler-warm-preparation | 0.173 | |
| sqlite-cold-preparation | 1.203 | |
| sqlite-warm-preparation | 0.121 | |
| Variance | ||
| box2d-variance | 1.210 | |
| poppler-variance | 4.083 | |
| Overall Score | result | 4.323 |
Testrun #4
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | ||
| main-thread-poppler-warm | ||
| main-thread-sqlite-cold | ||
| main-thread-sqlite-warm | ||
| Throughput | ||
| box2d-throughput | ||
| box2d-throughput-f32 | ||
| lua-binarytrees | ||
| lua-scimark | ||
| poppler-throughput | ||
| sqlite-throughput | ||
| Preparation | ||
| poppler-cold-preparation | ||
| poppler-warm-preparation | ||
| sqlite-cold-preparation | ||
| sqlite-warm-preparation | ||
| Variance | ||
| box2d-variance | ||
| poppler-variance | ||
| Overall Score | result | 4.225 |
Testrun #5
| Benchmark | Firefox 32-bit | Firefox 64-bit |
| Main-thread Responsiveness | ||
| main-thread-poppler-cold | ||
| main-thread-poppler-warm | ||
| main-thread-sqlite-cold | ||
| main-thread-sqlite-warm | ||
| Throughput | ||
| box2d-throughput | ||
| box2d-throughput-f32 | ||
| lua-binarytrees | ||
| lua-scimark | ||
| poppler-throughput | ||
| sqlite-throughput | ||
| Preparation | ||
| poppler-cold-preparation | ||
| poppler-warm-preparation | ||
| sqlite-cold-preparation | ||
| sqlite-warm-preparation | ||
| Variance | ||
| box2d-variance | ||
| poppler-variance | ||
| Overall Score | result | result |
Overall
| Build | Lowest Score | Highest Score | Median Score | Average Score |
| Firefox 32.0a1 32-bit | ||||
| Firefox 32.0a1 64-bit |
Conclusions
Anthony will fill this in later