MozillaWiki

Mobile/AsyncSubframePanning

< Mobile
Revision as of 19:51, 5 June 2013 by Bgirard (talk | contribs) (→‎Design)

Goals

Phase 1

  • land in 23 (not likely)
  • B2G subframe scrolling
  • Performance is
    • as good as main frame
  • subframes can't zoom
  • seamed scrolling

Phase 2

  • Fennec subframe scrolling
  • seamless scrolling

Not doing

  • scroll indicators are a non-goal
  • not fixing page drift
  • concurrently scrolling parents and ancestors
  • subframe zooming

Plan

To dos
  • merge B2G APZC into fennec
  • multiple display ports
  • bind multiple display ports
  • hit detection
  • transform equation cleanup
    • unit tests
  • tuning/profiling
  • concurrently scrolling subframes

Design

Step 1

Performed in Layout module, Main Gecko thread

  • (Bug 866232) Layout (FrameLayerBuilder et al.) build an initial simple layer tree with the position of the subframes tagged to the best fitting layer
  • (See diagram for sample page & layer tree)

Step 2

Performed in the compositor's main thread

  • Layer tree update/transaction is received by the compositor.
  • Touch event is delivered at (6,51).
  • (Bug 775452) Hit detection is performed on a layer tree in reverse depth-first order and matches 1st subframe.
  • APZC decides of the appropriate buffer/pre-rendering and sends an IPC message to the main thread with an updated FrameMetrics/display port request.
  • (See diagram for sample page & layer tree)

Step 3

Performed in Layout module, Main Gecko thread

  • Display port is requested on the subframe in Layout
  • (Bug 864447) Layer tree is grown to make room for a layer of the 1st subframe.
  • Subframe is painted with a display port allowing Async scrolling. FrameMetrics are updated.
  • (See diagram for sample page & layer tree)

Step 4

Performed in the compositor's main thread

  • (Bug 866265) APZC can now perform Async panning on new sublayer.

Step 5

  • After timeout APZC resets FrameMetrics/display port on the subframe removing the display port, the layer and releasing the memory usage.

Notes and diagram

   

Bugs

Tracking bug for async subframe scrolling: bug 775452


Full Query
ID Priority Summary Status Assigned to
775459 -- Implement (more) hit-testing logic for async/synchronously scrolled frames on UI thread RESOLVED Benoit Girard (:BenWa)
833795 -- Content jitters when URL bar hidden RESOLVED Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)
864441 -- Add APZC tests RESOLVED Benoit Girard (:BenWa)
864447 -- Support drawing display port on nested APZC frames RESOLVED Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)
865735 -- Disambiguate between Point instances in different coordinate systems RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
866232 -- Support having multiple APZC instances attached to the layer tree RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
866265 -- Transformation equation cleanups RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
872911 -- Find a better solution for handling touch gestures with moving frames RESOLVED
883646 -- Distinguish between CSSPixel and "device pixel" for Fennec RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
890279 -- Create ScrollInfoLayers for frames we want APZC support RESOLVED Benoit Girard (:BenWa)
890280 -- APZC input event transformations should consist of the async transform relative to the correct coordinate system RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
891384 -- APZC Hit Testing doesn't support 3D transform RESOLVED
893872 -- Add TransformShadowTree unit tests RESOLVED Benoit Girard (:BenWa)
894288 -- nsLayoutUtils::FindContentFor aborts when loading any tab RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
894619 -- TransformShadowTree only transforms root APZC RESOLVED
896045 -- Do input events go to the right place with off-main-thread animations? RESOLVED
898443 -- Figure out a good focus model for input events to multi-APZC RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
898444 -- Figure out why overflow:scroll elements aren't scrolling properly RESOLVED Robert O'Callahan (:roc) (email my personal email if necessary)
900638 -- Fix event-fluffing to work with APZC RESOLVED Dale Harvey (:daleharvey)
903460 -- AsyncCompositionManager applies async transform and CSS transform in the wrong order RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
907754 -- Pages with iframes that are less than screen width are not rendering correctly RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
912144 -- Hit testing for subframes is sometimes not right RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
913659 -- Strange behaviour when zooming page with iframe a lot RESOLVED
916125 -- When displayports are set on elements, their layers are not always clipped correctly RESOLVED Timothy Nikkel (:tnikkel)

24 Total; 0 Open (0%); 24 Resolved (100%); 0 Verified (0%);


Tracking bug for allowing Fennec to use APZC: bug 776030


Full Query
ID Priority Summary Status Assigned to
777075 -- Extract an interface for the PanZoomController -> LayerController dependency RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
777351 -- Merge LayerController and GeckoLayerClient RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
777468 -- Extract an interface for dependencies on PanZoomController RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
777896 -- Break dependencies from PanZoomController -> other java code RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
839641 -- Wire up the AsyncPanZoomController to be usable from java RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
859929 -- AsyncPanZoomController doesn't play well with progressive tile painting RESOLVED Botond Ballo [:botond] [away until July 9]
859935 -- AsyncPanZoomController assumes the "UI/controller" has a message loop RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
859939 -- AsyncPanZoomController has some type wrongness RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
859951 -- Refactor the code to convert from MOTION_EVENT to nsTouchEvent instances RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
859962 -- BrowserApp assumes LayerView.getPanZoomController never returns null RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
860162 -- AsyncPanZoomController doesn't apply proper scroll compensation if the root layer has a scale RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
860455 -- AsyncPanZoomController::WidgetSpaceToCompensatedViewportSpace is not fully implemented RESOLVED
860613 -- Add a way to bind fennec's APZC to the primary scrollable layer RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
860940 -- Replace AndroidGeckoEvent constructors with static factory methods RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
867269 -- SyncFrameMetrics override is disconnected and breaks compiles on gcc 4.7+ RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
880024 -- Create a local version of GeckoContentController RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
880578 -- APZC: Scroll events are only sent when repainting is needed RESOLVED Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)
880932 -- APZC: Page stops repainting and zoom/pan stops updating RESOLVED Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)
881451 -- ThebesLayerComposite::GetCompositionBounds() needs to be fixed up in various ways RESOLVED Chris Lord [:cwiiis]
890280 -- APZC input event transformations should consist of the async transform relative to the correct coordinate system RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
895904 -- NativePanZoomController.abortAnimation() needs to be hooked up RESOLVED Botond Ballo [:botond] [away until July 9]
943557 -- Use modifier keys for click events generated by single-tap gestures in Android APZC code RESOLVED
1046344 -- Unbitrot APZ glue code for Fennec RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1049136 -- Hook up touch event flow for APZ usage on Fennec RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1055548 -- With native APZ enabled clicking on stuff sometimes doesn't work quite right RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1055557 -- Properly notify the APZ of root zoom constraints and changes to zoom constraints RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1137872 -- Make the urlbar hiding stuff work with native pan zoom controller RESOLVED
1146020 -- Make APZCCallbackHandler extend from ChromeProcessController RESOLVED Danilo Cesar Lemes de Paula
1146024 -- Fix up input routing for APZ on Fennec RESOLVED Danilo Cesar Lemes de Paula
1156392 -- Remove MOZ_ANDROID_APZ condition from ifdef in ScrollFrameHelper::WantAsyncScroll RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1156401 -- Fix ContextMenu handling when building Fennec with C++ APZ enabled RESOLVED Danilo Cesar Lemes de Paula
1158424 -- Clean up FrameMetrics::mIsRoot and related notions RESOLVED Botond Ballo [:botond] [away until July 9]
1159405 -- Fix "Found a non-root APZ with no handoff parent" warning when root process has a root XUL document with scrollable subframes RESOLVED Danilo Cesar Lemes de Paula
1162701 -- Fix panning on Fennec when built with C++ APZ RESOLVED
1163572 -- Override RequestContentRepaint to detect root-frame updates RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1175282 -- [Fennec-APZ] Unable to scroll outside scrollable subframe after zooming on a non-scrollable document RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1180267 -- Update Fennec code to use MobileViewportManager instead of the big pile of browser.js code RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1188225 -- Implement ChromeProcessController::HandleDoubleTap RESOLVED Randall Barker [:rbarker]
1190979 -- Text is selected when pinch zooming RESOLVED
1190988 -- Hit testing appears to return incorrect positions when --enable-android-apz is specified. RESOLVED Randall Barker [:rbarker]
1195553 -- Double tap gesture fails on Fennec when C++ APZ is enabled. RESOLVED Randall Barker [:rbarker]
1197824 -- Update Fennec code to use ZoomConstraintsClient instead of the big pile of browser.js code RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1200393 -- Java APZ appears more responsive during page load compared to C++ APZ. RESOLVED
1200397 -- Pinch zoom gesture does not always appear to work when C++APZ is enabled. RESOLVED
1200399 -- The dynamic tool bar in Fennec will some times permanently hide RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1201100 -- Perma-checkerboarding when zoomed in with apz-fennec RESOLVED Randall Barker [:rbarker]
1201130 -- Switching tabs in Fennec-APZ sometimes doesn't update the metrics properly RESOLVED
1201217 -- Switching tabs in Fennec-APZ sometimes renders the page at an incorrect zoom RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1201529 -- When APZ is enabled, all browser.js codepaths that set a resolution or displayport should be disabled RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1201581 -- Fixed-position margins/clip are not updated properly in fennec-apz RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1202020 P5 Remove dependencies on async pan/zoom information from Java code RESOLVED
1206620 -- Zoom out does not work at double tap action in Landscape mode on Tablet devices RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1206872 -- [meta] Enable APZ on fennec on nightly builds RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1206874 -- [meta] Let APZ on Fennec ride the trains RESOLVED
1229349 -- Tapping on things in the zoomed view is broken with C++ APZ RESOLVED
1229439 -- Horizontal Scrolling of elements is not registered during page load or janky RESOLVED
1229462 -- Use Android Scroller class for fling animation RESOLVED Randall Barker [:rbarker]
1229840 -- Reader mode toolbar seems to show/hide a lot with C++ APZ RESOLVED
1230081 -- Scrolling on partially rendered pages causes judder and other issues RESOLVED
1230085 -- White checkerboarding visible (instead of page background) RESOLVED
1230176 P3 Weirdness at end of Fennec fling. Don't send duplicate DOM 'scroll' event for sub-pixel scrolling REOPENED
1240065 -- Reconsider viewport update policy when pinch zooming RESOLVED Randall Barker [:rbarker]
1248014 -- Allow horizontal scrolling from a pointer device when positioned over a horizontally scrolling layer. RESOLVED
1260534 P3 Checkerboarding in reader view is white instead of page background color RESOLVED
1260905 -- Fennec APZ flywheel physics result in very strange behavior when you do sequence of quick and short pans RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1262807 P1 Lots of repainting while scrolling around in Fennec RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1265372 -- Scrolling in reader mode enters/exits fullscreen mode too easily RESOLVED
1272478 -- Cannot scroll about:memory horizontally VERIFIED Randall Barker [:rbarker]
1274597 P1 Clicking on links sometimes doesn't work, long-pressing results in text selection instead of context menu RESOLVED Kartikaya Gupta (email:kats@mozilla.staktrace.com)
1278390 P2 Fennec doesn't display Gitter.im content, after scrolling up/down RESOLVED
1280666 P2 Use modified OverScroller to work around bugs RESOLVED James Willcox (:snorp) (jwillcox@mozilla.com) (he/him)
1281042 P3 Displayport expiry can result in perma-checkerboarding RESOLVED
1291678 P3 Some posts on GoT (gathering of tweakers)-forum are scrollable for fractions of pixels NEW
1297148 P3 Crash in mozilla::CrossProcessMutex::CrossProcessMutex RESOLVED
1302150 P2 Dynamic toolbar doesn't reappear after scrolling to bottom of long page VERIFIED Randall Barker [:rbarker]

75 Total; 2 Open (2.67%); 71 Resolved (94.67%); 2 Verified (2.67%);


Meetings

4/26/13

  • initial meeting, created phases, goals and overall plan

5/21/13

  • Chris will take the display port work

5/28/13

  • Kats put up patch for disambiguated point classes
    • will continue propegating the use of the classes through the
    • should be up for review this week
  • Chris should start on display ports per scroll frame this week

6/4/13

  • Kats
    • propagating type changes to frame metrics and APZC code
  • BenWa
    • unit tests
    • hit testing
  • ajones
    • multiple display ports
    • code already largely supports it
  • does metro need subframe scrolling?
  • next items:
    • hit testing (in progress, owned by BenWa)
    • creating and managing APZCs from the frame loader builder (need owner)
Retrieved from "https://wiki.allizom.org/index.php?title=Mobile/AsyncSubframePanning&oldid=663906"