Firefox OS/Performance/App Performance Validation

This page outlines a simple process for manually auditing and improving the performance of an app.

Section listed first have a higher priority. This means that for example fixing over-invalidation and responsiveness will lead to vastly improved checker-boarding thus should be performed first when possible.

1. Over-invalidation

Platform Contact: Layout team

What does it mean: Over-invalidation means that we're repainting content that isn't changing. This leads to higher CPU usage & bandwidth.

Verification Steps:

• In the Developer menu turn on 'Flash repainted area'.
• Perform all common interactions with the app. Look for areas of the page that are flashing but where the content hasn't changed.
• Examples:

Good: GIF invalidates every frame, links invalidates on mouse over because of style change. Static content is not repainted	Good: When scrolling, new content is painted but it doesn't repaint once on the screen.
Bad: Clicking repaints the whole page.

• Common issues: Please add any common issues to this list. Currently empty.

Debugging:

• Attach with the App Manager and tweak the structure and styles of the page to trigger the desired behavior.

Report:

• Report as completed once all excessive painting is resolved.

2. Reflows/Restyles

Platform Contact: Layout team

What does it mean:

NOTE: All the following information applies for Restyles. Consider minimizing restyles, avoiding sync style queries and batching all style changes in the frame (or event if possible).

Reflowing the page is the process of deciding where all the DOM elements will be positioned the page. Any changes to the page that has the potential of effecting the flow of the page will have to reflow the page again. The larger the DOM the longer this can potentially take.

Reflowing is expected as the page/app is being loaded. Once the page is ready, app authors should carefully consider what interactions should make changes that can affects the flow of the document. If changes to the page are required they should happen in the same frame/refresh tick as all changes will be validated at the same time. Reflows should not happen on scrolling unless using a virtual list approach.

Sync reflow is when the position information (like scrollLeft/scrollTop/clientWidth/clientHeight) of a DOM node is queried when there are pending changes made to the flow of the document. This will prevent reflow from being delayed to the next refresh tick, block all current execution and reflow the document.

Sometimes reflows can be avoided by using features like CSS transforms that don't affect the flow of the page.

Verification Steps:

• In the Developer menu -> Developer HUD, turn on the 'Reflow' counter.
• Perform all common interactions with the app. Perform reflows only when necessarily i.e. the structure of the app must change.

More information: http://paulrouget.com/e/fxoshud/

3. Event loop responsiveness

Platform Contact: Varies, use profilers to narrow down the issue or contact the Performance team.

What does it mean: See http://paulrouget.com/e/fxoshud/ for an excellent description. Keeping the event loop delay is important to having a smooth and responsive app.

This step is very important. It's only listed 3rd to make sure the simplest problems have been resolved first.

Verification Steps:

• In the Developer menu -> Developer HUD, turn on the 'Jank' counter.
• Perform all common interactions with the app. Track the 'Jank' number for each of the interaction. When that number is above 10ms your app' scrolling is affected. Once that number is 100ms painting will fall behind and input will be delayed. 500ms+ indicate that this interaction will be noticeable jerky.
• Understand the worse case responsiveness for each interaction of your app. Consider various workload for your app (like 5000 pictures in the Gallery App).

Debugging:

• Profile the app' main thread. This is useful if you can't make an educated guess: https://developer.mozilla.org/en-US/docs/Performance/Profiling_with_the_Built-in_Profiler#Profiling_Boot_to_Gecko_%28with_a_real_device%29
• For JavaScript related slowdown: Add function duration logging throughout the code.
• Disable some functionality of your app to see if that reduces the event loop lag (jank).

Report:

• Report the numbers for various interaction and the workload used.

4. Load Time

Platform Contact: Varies, use profilers to narrow down the issue or contact the Performance team.

What does it mean: The duration, in millisecond, required to load the app.

Verification Steps:

Report:

5. Layer Tree

Platform Contact: Layout & Graphics

What does it mean:

Having a good layer tree means we can most changes without having to repaint the page. Think of a cartoons on the television like The Simpsons: Most cartoons will divide their scene into a few layer and will move the layers themselves instead of redrawing the scene for every frame of the cartoon. Using layers to perform animations means we can avoid redrawing the page every frame and can simple move these layers around.

Verification Steps:

NOTE: Verifying an optimal layer tree is complex task that require a lot of internal knowledge of Gecko. Be ready to ask for help.

• In the Developer menu, turn on 'Layer border' and/or 'Flash repainted area'.
• Perform all common interactions with the app.
• Look for borders around the content that is animating.
• Verify that paint flashing is kept to a minimum as done in the 'Over-invalidation' section.

and

• Enable 'layers.dump'. Each frame the exact layer structure will be printed to 'adb logcat'.
• Perform all common interactions with the app.

Look for the following (Note that they -may- not be bugs):

• Layers that are created around content that isn't animating.
• Layers that are created and destroyed because something changes once.
• Large layers that are created at the start or during an important user interaction.
• Excessive number of layers.
• RGBA layers in the layers.dump when the animated content is opaque

Debugging:

• Attach with the App Manager and tweak the structure and styles of the page to trigger the desired behavior.

6. Checkerboarding

Platform Contact: Graphics, verify previous steps first

What does it mean:

NOTE: Ideally at this point this section is just a verification. If the over-invalidation, reflows & event loop responsiveness is good then checkerboarding should be minimal to none.

Checkerboarding is when the app fails to keep up with the panning happening in the main process. When this happens a solid background color will be shown temporary until the page catches up.

Verification Steps:

• Perform all common interactions with the app including heavy scrolling.
• Looks for the background color.
• Consider first repeating some of the previous steps to rule out over-invalidation, reflows and non responsive event loop.

7. Compositor FPS

Platform Contact: Graphics

What does it mean:

Check that the generated layer tree can composited by the GPU every frame.

Verification Steps:

• Turn on 'Frames per second' from the Developer HUD.
• Watch the left most FPS counter during animations and scrolling. This number should remain very close to the refresh rate of the display (~60 FPS for most devices). Large layer transformations (check with layer border/layers.dump) will cause a dip in this counter.

and

• Turn on 'Frames per second' from the Developer HUD.
• Set layers.offmainthreadcomposition.frame-rate to 0.
• Watch the left most FPS counter during idle at various stage of your app. This number should remain very close to the refresh rate of the display (~60 FPS for most devices). There should be little to no dips below 60 FPS here.

8. Memory usage

Platform Contact: Memshrink

What does it mean: Check that memory consumption is not excessive.

Verification Steps:

• Use tools/get_about_memory.py to get a memory reports dump while the app is running -- both shortly after start-up, and after using the app for a while.
• Load the dumps in about:memory in desktop Firefox. Check for any measurements that seem to be excessive, particularly "heap-unclassified".
• If "heap-unclassified" is high, re-run tools/get_about_memory.py with a DMD-enabled build, which will produce data that can be used to understand where additional memory reporters need to be added.

9. Profiling

Platform Contact: Performance Team & Graphics

What does it mean:

At this point most common issues should have been fixed. It's important to check some of the other things first because bugs like over-invalidation will incorrectly report that the page is expensive to paint where instead we're simply painting too much.

With sampling profiling we can get a good approximation of where the remaining CPU time is spent and look for stages in the pipeline that are exceeding their budget or finding things that are running that simply shouldn't be.

See: https://developer.mozilla.org/en-US/docs/Performance/Profiling_with_the_Built-in_Profiler#Profiling_Boot_to_Gecko_%28with_a_real_device%29

Verification Steps:

• Start by profiling only the main thread at a high resolution.
• Profile the app and the compositor: ./profile.sh start b2g -t Compositor && ./profile.sh start APP_NAME_OR_PID

10. Power usage

Platform Contact: Performance Team

What does it mean:

Power usage for a given change is difficult to measure. Power usage is directly correlated with CPU usage, and also with the use of various hardware features of the phone.

Verification Steps:

If you have a power harness available to you, you can do before and after power measurements by following the instructions on this page. If you don't have a power harness, please contact the performance team (#fxos-perf on IRC) for help.