Firefox/Content Performance Program/Observations

Windows Desktop

Reference system

• HP Pavilion 15" laptop
• Touchscreen
• CPU: i3-5010u
• GPU: (integrated): HD5500

Experiments

Windows 8 vs Windows 10 (Firefox)

• Performed by: aklotz, avih

TBD

GFX and e10s configurations (Firefox and others)

• Performed by: aklotz, avih
• Reproduced partially: vladan

TBD

Navigation from google search results (between browsers)

• performed by: vladan, avih
• Test how quickly different browsers navigate links from a search results page.
• System: HP reference laptop, Firefox Nightly 44.
• Procedure:
  • Navigate to google.com, type in the search query, press enter, wait for results.
  • Click the first non-ad link
  • Observe navigation speed
• Observations:
  • Searching "facebook":
    • Bug 1213434 With a clean browser cache, Chrome navigates almost instantly - quite faster than Firefox and IE.
    • With warm cache, all browsers navigate at similar speeds.
  • Searching "cnn"
    • Clean cache: Chrome faster than Firefox/IE, but it's not as instant as with "facebook".
    • Warm cache: All browsers navigate at similar speeds.
  • Searching "mozilla" or "edge":
    • Both Clean/warm cache: all browsers are similar.
  • Note that if a user routinely uses google search to access facebook (or cnn etc), it's highly likely that the main page will already be cached by the browser (or at least many components which this page uses), therefore the "facebook instant effect" which chrome has but Firefox doesn't is not likely to be a meaningful differentiator in practice.

Two monitors configurations (between browsers)

• performed by: avih
• Test how well browsers use the correct refresh rate in a two monitors configuration.
• tested using: http://www.vsynctester.com/index.html
• Procedure - with a single browser window, check all combinations of the following variables:
  • Browser: Firefox release (40) / nightly (44) / Chrome / IE / Edge (win10 only)
  • OS: Windows 8.1 / Windows 10
  • Windows "main" display: laptop's built-in / external monitor
  • The browser window is at: laptop's built-in / external monitor
  • Laptop's built in display refresh rate: lower/identical/higher than that of the external monitor
• Observe both the reported animation rate and the smoothness on screen - on all cases the reported value matched the visual observation.
• Observations:
  • When both monitors have the same rate - all browsers behave optimally.
  • Firefox is the best, with only a single suboptimal case (and not worse than other browsers in this case):
    • When the Windows "main" display has lower rate than the extended-to display (regardless if the main is the built-in/external, or windows 8.1/10), then the animation rate on the extended-to display is the refresh rate of the "main" display (therefore lower than it should be). I suspect this to be an inherent Windows issue (need quote).
  • Chrome comes second:
    • Same issue as Firefox, plus few other cases where it incorrectly uses the rate of the "other" monitor.
  • IE/Edge are last:
    • Same issues as Firefox and Chrome, plus when the main display is the laptop, and it has a lower rate than the external display, on both win8/10 -> it uses weird rAF rate (67hz, 64hz) which doesn’t match any monitor.

Memory usage of e10s, non e10s (Firefox and others)

• Performed by: vladan, avih

TBD

Google maps zoom in/out animation (between browsers)

• Performed by: vladan, avih
• Test how well the google maps zoom in/out animation is (using mouse wheel scroll over the map).
• We tested this zoom animation since vladan thought Firefox might perform worse than chrome.
• System:
  • vladan: Lenovo W520 laptop + NVidia GPU + 3 monitors (1 built in, two external), Windows 7.
  • avih: Reference HP laptop, Windows 10.
• Procedure:
  • Tested browsers: Firefox release (40) / Nightly (44) / Chrome / IE / Edge
  • Load maps.google.com, cue it to a city (e.g. Toronto), then use mouse-wheel to zoom in and out (stayed zoomed in within city limits at all times).
• Observations (avih):
  • TL;DR: Firefox is not worse than others and all browsers are quite similar.
  • All browsers initially zoom in/out in discrete steps, but once enough data has loaded, the zoom change switches to smooth.
  • While in discrete steps: Edge and IE update slightly slower than nightly/release/chrome.
  • _maybe_ Firefox release and Edge/IE take longer to load the data than nightly/chrome.
  • While in smooth zoom mode, all browsers behave pretty much the same, and very nicely at that.
• TODO:
  • Settle the disagreement: vladan: Firefox is worse, avih: all browsers are similar enough.

Fennec

Reference Device

• Samsung Galaxy S4
• Android Lollipop 5.0.1 (official update)
• Not rooted

Experiments

Alexa top 20 - Navigation and manual scroll (between browsers)

• Performed by: avih
• Reproduced partially: vladan
• Tested using:
  • Fennec Aurora 43 API11 2015-09-30 (chosen together with :kats)
  • Chrome
  • The device default "Internet" browser
• Procedure:
  • Visit the top 20 alexa sites
  • On each site scroll manually and observe the behavior
  • Visit some internal links and observe the navigation behavior
  • Take notes of the relative performance diffs between browsers
• Observations:
  • "Internet" and Chrome are similar, but between them Chrome is slightly better. We'll focus on Fennec vs Chrome.
  • Page load is similar between browsers
  • Bug 1217415 Page navigation is faster in Chrome on some sites (e.g. Wikipedia, ebay) but similar on other sites.
  • Bug 1217372 Fennec has text input lag in autocomplete boxes (google, bing) which Chrome doesn't.
  • In general, scroll is better in chrome, but on "light" pages Fennec gets pretty close.
  • Scroll _while_ the page is loading is more responsive in Fennec (but still not very good)
  • Bug 1217370 On fast scroll swipes, sometimes the momentum is less than expected.
  • Scroll on some pages (e.g. yahoo.com) is considerably worse in Fennec:
    • Bug 1217364 Inconsistent progression (momentum) without user inputs.
    • Bug 1217366 Visible low resolution rendering, especially for fast swipes.