Summary
The following documents A-B testing comparing performance of 32-bit Firefox builds to 64-bit Firefox builds, utilizing the Massive benchmark.
Methodology
- Test using the same system, make note of the system profile
- Install the latest Firefox Nightly 32-bit build and start with a new profile
- Load http://kripken.github.io/Massive/ and click "Run the benchmark now!"
- Let the benchmark run
- Record the results below when the benchmark completes
- Include a screenshot of the result
- Repeat the benchmark 5 times on the same system and record the result separately
- Install the latest Firefox Nightly 64-bit and repeat the tests with a new profile
System Profile
Operating System | Windows 7 64-bit |
Firefox | (user agent string with build ID) Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:32.0) Gecko/20100101 Firefox/32.0, 20140604030202 |
Processor | (make, model, speed) AMD FX(tm)-8320 Eight-Core Processor 3.50 Ghz |
Memory | (type, speed, amount) 8 GB DDR3 1033 Mhz |
GPU | (make, model, speed, memory, driver version) AMD Radeon R7 200 Series, Core Clock 780 Mhz, 2GB DDR3 900 Mhz, Catalyst version 13.12 |
Results
Testrun #1
Benchmark | Firefox 32-bit | Firefox 64-bit |
Main-thread Responsiveness | ||
main-thread-poppler-cold | 0.452 | 0.729 (±22%!) |
main-thread-poppler-warm | 0.428 | 0.523 |
main-thread-sqlite-cold | 0.053 (±26%!) | 0.872 |
main-thread-sqlite-warm | 0.033 (±24%!) | 0.043 (±27%!) |
Throughput | ||
box2d-throughput | 6.838 | 4.623 |
box2d-throughput-f32 | 6.316 | 4.258 |
lua-binarytrees | 8.067 | 7.271 |
lua-scimark | 10.545 | 12.160 |
poppler-throughput | 7.008 | 5.988 |
sqlite-throughput | 10.891 | 8.404 |
Preparation | ||
poppler-cold-preparation | 0.939 | 0.864 |
poppler-warm-preparation | 0.185 | 0.183 |
sqlite-cold-preparation | 1.084 | 1.260 |
sqlite-warm-preparation | 0.098 | 0.123 |
Variance | ||
box2d-variance | 1.466 | 1.216 |
poppler-variance | 4.843 | 4.074 |
Overall Score | 3,967 | 3,377 |
Testrun #2
Benchmark | Firefox 32-bit | Firefox 64-bit |
Main-thread Responsiveness | ||
main-thread-poppler-cold | 0.367 | |
main-thread-poppler-warm | 0.361 | |
main-thread-sqlite-cold | 0.064 | |
main-thread-sqlite-warm | 0.043 (±28%!) | |
Throughput | ||
box2d-throughput | 4.666 | |
box2d-throughput-f32 | 4.277 | |
lua-binarytrees | 7.182 | |
lua-scimark | 12.055 | |
poppler-throughput | 5.959 | |
sqlite-throughput | 8.485 | |
Preparation | ||
poppler-cold-preparation | 0.846 | |
poppler-warm-preparation | 0.180 | |
sqlite-cold-preparation | 1.209 | |
sqlite-warm-preparation | 0.115 | |
Variance | ||
box2d-variance | 1.290 | |
poppler-variance | 4.036 | |
Overall Score | result | 4.287 |
Testrun #3
Benchmark | Firefox 32-bit | Firefox 64-bit |
Main-thread Responsiveness | ||
main-thread-poppler-cold | 0.338 | |
main-thread-poppler-warm | 0.356 | |
main-thread-sqlite-cold | 0.063 | |
main-thread-sqlite-warm | 0.042 (±28%!) | |
Throughput | ||
box2d-throughput | 4.643 | |
box2d-throughput-f32 | 4.284 | |
lua-binarytrees | 7.239 | |
lua-scimark | 12.185 | |
poppler-throughput | 6.003 | |
sqlite-throughput | 8.366 | |
Preparation | ||
poppler-cold-preparation | 0.842 | |
poppler-warm-preparation | 0.173 | |
sqlite-cold-preparation | 1.203 | |
sqlite-warm-preparation | 0.121 | |
Variance | ||
box2d-variance | 1.210 | |
poppler-variance | 4.083 | |
Overall Score | result | 4.323 |
Testrun #4
Benchmark | Firefox 32-bit | Firefox 64-bit |
Main-thread Responsiveness | ||
main-thread-poppler-cold | 0.359 | |
main-thread-poppler-warm | 0.362 | |
main-thread-sqlite-cold | 0.071 (±31%!) | |
main-thread-sqlite-warm | 0.042 (±26%!) | |
Throughput | ||
box2d-throughput | 4.674 | |
box2d-throughput-f32 | 4.304 | |
lua-binarytrees | 7.226 | |
lua-scimark | 12.005 | |
poppler-throughput | 5.951 | |
sqlite-throughput | 8.414 | |
Preparation | ||
poppler-cold-preparation | 0.840 | |
poppler-warm-preparation | 0.173 | |
sqlite-cold-preparation | 1.212 | |
sqlite-warm-preparation | 0.140 | |
Variance | ||
box2d-variance | 2.258 | |
poppler-variance | 4.067 | |
Overall Score | result | 4.225 |
Testrun #5
Benchmark | Firefox 32-bit | Firefox 64-bit |
Main-thread Responsiveness | ||
main-thread-poppler-cold | 0.360 | |
main-thread-poppler-warm | 0.351 | |
main-thread-sqlite-cold | 0.060 (±27%!) | |
main-thread-sqlite-warm | 0.052 (±40%!) | |
Throughput | ||
box2d-throughput | 4.666 | |
box2d-throughput-f32 | 4.292 | |
lua-binarytrees | 7.230 | |
lua-scimark | 12.195 | |
poppler-throughput | 5.989 | |
sqlite-throughput | 8.583 | |
Preparation | ||
poppler-cold-preparation | 0.844 | |
poppler-warm-preparation | 0.177 | |
sqlite-cold-preparation | 1.236 | |
sqlite-warm-preparation | 0.137 | |
Variance | ||
box2d-variance | 1.297 | |
poppler-variance | 4.086 | |
Overall Score | result | 4,211 |
Overall
Build | Lowest Score | Highest Score | Median Score | Average Score |
Firefox 32.0a1 32-bit | ||||
Firefox 32.0a1 64-bit |
Conclusions
Anthony will fill this in later