Firefox/Content Performance Program/Observations: Difference between revisions

← Older edit

Firefox/Content Performance Program/Observations (view source)

Revision as of 20:58, 22 October 2015

3,813 bytes added , 22 October 2015

minor

Avih

Confirmed users

95

edits

@@ Line 7: / Line 7: @@
 ==== Experiments ====
-===== Windows 8 vs Windows 10 (Firefox) =====
+===== Scroll perf - e10s configs and Windows 8 / 10 (Firefox) =====
 * Performed by: aklotz, avih
-TBD
+* Compare Firefox scrolling performance between Windows 8.1 / 10 and e10s modes, using different power modes, test pages, and tabs
+* Tested using the scroll-test bookmarklet from bug {{Bugzilla|894128}} ([https://bug894128.bmoattachments.org/attachment.cgi?id=776049 this])
+* On each config, run the bookmarklet few times and register the result (average and stddev) of a single run which seems typical
+* Base configuration:
+** System: HP reference laptop, Windows 8.1 and Windows 10 - always connected to power.
+** Firefox Nightly 42 2015-08-02 with APZ disabled
+** Bookmarklet config: 7000 (ms scroll), 10 (pixels/step), 0 (don't ignore first frames)
+* Tested configurations: all the combinations of the following:
+** Page: [http://facebook.com/llbean facebook] / [http://twitter.com/Lord_Voldemort7 twitter] / [https://search.yahoo.com/yhs/search?p=mozilla&ei=UTF-8&hspart=mozilla&hsimp=yhs-002 Yahoo search results]
+** OS: Windows 8.1 / Windows 10
+** Power mode: balanced / power saver
+** Active background tab (gmail while it reloads): Yes / No
+* Observations:
+** It's really hard to take/get consistent measurements while gmail loads in the bg, due to network latency, gmail load timing, etc.
+** Facebook scrolls worse than yahoo/twitter on most configs (except single tab + balanced where the system is fast enough and all pages reach ~60fps)
+** Bug {{Bugzilla|1213425}} Power saver shows consistently worse results than balanced
+** No major differences between win8 and 10 across the board
+** No major differences between non-e10s and e10s
+** Single tab - no major differences between non-e10s/e10s/e10s-multi(3)
+** Bug {{Bugzilla|1213413}} Gmail loading in bg - affects most* configurations badly
+*** Except e10s-multi(3) which improves performance on this case dramatically
-===== GFX and e10s configurations (Firefox and others) =====
+===== Scroll perf - GFX and e10s configurations (Firefox) =====
 * Performed by: aklotz, avih
 * Reproduced partially: vladan
-TBD
+* Compare scrolling performance between different Firefox gfx configurations
+* System: HP reference laptop, Windows 8.1
+* Firefox nightly 42 2015-07-27, with APZ disabled
+* Tested using the scroll-test bookmarklet from bug {{Bugzilla|894128}} ([https://bug894128.bmoattachments.org/attachment.cgi?id=776049 this])
+** Bookmarklet config: 3000 (ms scroll), 10 (pixels/step), 1 (ignore the first frame)
+** On each config, run the bookmarklet few times and register the result (average and stddev) of a single run which seems typical
+* Tested configurations: all the combinations of the following:
+** Page: [http://facebook.com/llbean facebook] / [https://search.yahoo.com/yhs/search?p=mozilla&ei=UTF-8&hspart=mozilla&hsimp=yhs-002 Yahoo search results] / [https://www.google.com/search?q=spectrum&tbm=isch google image search results]
+** e10s: Yes / No
+** Power mode: balanced / power saver
+** Firefox gfx configs:
+*** D3D11, D2D1_1 (Default)
+*** D3D11, D2D1_1, OS scaling 125%
+*** D3D11, D2D1_1, Firefox scaling 125%
+*** D3D11, D2D1_1, on an external 1080p display
+*** D3D11, D2D1_0
+*** D3D11, no-D2D
+*** HW accel off, no-D2D
+*** D3D9, no-D2D
+*** D3D11-warp, no-D2D
+* Observations:
+** The google image search results scrolls best across configurations (and typically perfectly), yahoo results scroll somewhat worse, and facebook scrolls the worst of these pages.
+*** For this reason, it's probably better to use facebook for scroll benchmarks.
+** Bug {{Bugzilla|1213440}} D2D1_1 scrolls worse than other D2D configs (including no-D2D) on different pages, different GFX configs, and e10s yes/no.
+** facebook scrolls quite badly with D3D11-warp (bug {{Bugzilla|1213432}}) and D3D9 (bug {{Bugzilla|1213429}}), especially with e10s.
+** Except for D2D1_1, D3D11-warp and D3D9, most other configs perform similarly.
+** With e10s, the performance differences between gfx configs seem more extreme than without e10s.
-===== Navigation from google search results (between browsers) =====
+===== Navigation speed from search results (between browsers) =====
 * performed by: vladan, avih
 * Test how quickly different browsers navigate links from a search results page.
@@ Line 35: / Line 81: @@
 ** Note that if a user routinely uses google search to access facebook (or cnn etc), it's highly likely that the main page will already be cached by the browser (or at least many components which this page uses), therefore the "facebook instant effect" which chrome has but Firefox doesn't is not likely to be a meaningful differentiator in practice.
-===== Two monitors configurations (between browsers) =====
+===== Refresh rates correctness - two monitors configurations (between browsers) =====
 * performed by: avih
 * Test how well browsers use the correct refresh rate in a two monitors configuration.
@@ Line 55: / Line 101: @@
 *** Same issues as Firefox and Chrome, plus when the main display is the laptop, and it has a lower rate than the external display, on both win8/10 -> it uses weird rAF rate (67hz, 64hz) which doesn’t match any monitor.
-===== Memory usage of e10s, non e10s (Firefox and others) =====
+===== Memory usage comparison (Firefox in few e10s configs and others) =====
 * Performed by: vladan, avih
 * Compare browsers memory usage with different number of tabs
@@ Line 78: / Line 124: @@
 *** Firefox no-e10s is similar to chrome (more than chrome for the "no URL tabs" case)
 *** IE is second with ~60-70% more memory than chrome (less than chrome for the "no URL tabs" case)
-*** Edge and Firefox e10s-multi with roughly twice the memory usage of Chrome (Edge is similar to chrome for the "no URL tabs" case).
+*** Bug {{Bugzilla|1213435}} Firefox e10s-multi uses roughly twice the memory which Chrome uses
+*** Edge slightly more memory than Firefox e10s-multi (though less than Firefox for the "no URL tabs" case)
 ** E.g. for 15 tabs:
 *** Chrome: 720M, 18 processes
@@ Line 88: / Line 135: @@
 *** Edge: 1550M, 8 processes
-===== Google maps zoom in/out animation (between browsers) =====
+===== Google maps zoom animation smoothness (between browsers) =====
 * Performed by: vladan, avih
 * Test how well the google maps zoom in/out animation is (using mouse wheel scroll over the map).
@@ Line 114: / Line 161: @@
 ==== Experiments ====
-===== Alexa top 20 - Navigation and manual scroll (between browsers) =====
+===== Navigation and scroll using top 20 sites (between browsers) =====
 * Performed by: avih
 * Reproduced partially: vladan
@@ Line 122: / Line 169: @@
 ** The device default "Internet" browser
 * Procedure:
-** Visit the top 20 alexa sites (if the main page is too plain, use a meaningful page instead, e.g. facebook.com/barackobama):
+** Visit the top 20 sites (if the main page is too plain, use a meaningful page instead, e.g. facebook.com/barackobama):
 *** google, facebook, youtube, baidu, yahoo, amazon, wikipedia, qq, twitter, taobao, sina, linkedin, ebay, yandex, bing, msn, instagram, aliexpress, apple, ask.
 ** On each site scroll manually and observe the behavior