Hasal

A performance measurement project for Google Doc, Google Sheet, and Google Slide.

State of this project

We are currently in the middle of phase 2. Execution summary as below:

Execution Summary

• Priority: The priority of test cases
• Script Complete: test script is implemented and tested
• Execution Complete: test script run over 30 times and running time is recorded
• Profiler Data Complete: Gecko profiler, har profiler and performance timing object upload to bugzilla
• Analyze Data Complete: Analyze the gecko profiler data and find the most running time of different time unit (between navigationStart and domLoading)

Priority	Script Complete	Execution Complete	Profiler Data Complete	Analyze Data Complete
High	27/27	27/27	15/24	15/24
Medium	0/25	0/25	0/25	0/25
Low	0/14	0/14	0/14	0/14

Testing Result

You can find all testing result here Testing Result And the analysis diagram for module consume the most running time in each test cases could generate automatically like below:

Or you can download the XMind file here.

About this project

We are trying to measure the difference between Firefox and Chrome on high visibility web apps (Google Doc, Google Sheet, Google Slide, etc.)

We use the video recording to record the user actions and extract the frames to get the running time from user perspective. Based on same user actions, we measure the difference between Firefox and Chrome.

Project Plan

• • Resource Plan
  • Test Plan
  • Meeting minutes

Google Doc

Phase 1 (Done)

• Build up a workable framework that can run some manual created test cases
  • Trigger actions on browser
  • Record video on the background
  • Enable profiler on firefox automatically
  • Convert the video to image automatically
  • Compare images with sample images
• Measure the difference between Firefox and Chrome for sample user actions (40% found between Firefox and Chrome)

Phase 2

• Create test cases based on test matrices
• Create test scripts for high priority cases and execute them
• Generate profiler data for each testing result
• Find out the module and method which cost the most running time
• Go through the whole process from analyzing to provide information
• Framework improvement
  • Replace Selenium with Sikuli
  • Get the medium value and other statistics
• Support windows and mac

Phase 3

• Create scripts for medium and low priority cases and execute them
• Find out the module and method which cost the most running time

Team

Name	Email address
Shako Ho	sho@mozilla.com	You can contact him to get more project detail.
Askeing Yen	askeing@mozilla.com
Cynthia Tang	ctang@mozilla.com
Farmer Tseng	fatseng@mozilla.com
Walter Chen	wachen@mozilla.com

Working Items

Full Query

ID	Component	Summary	Status	Assigned to	Target milestone	Depends on
1278554	Graphics: Canvas2D	CanvasRenderingContext2D::DrawImage takes lots of time when scrolling Google Spreadsheet	NEW		---	1237058
1307024	General	[Perf] [google docs] Firefox will get 3~9% faster when running gdoc test cases with chrome user agent	NEW		---	1321623

2 Total; 2 Open (100%); 0 Resolved (0%); 0 Verified (0%);

Testing Tool

Testing Framework

Following the steps here to setup the framework. For the time being, only Ubuntu is officially supported as a testing environment; you can use OS X but you'll have to manually solve some package dependency issue. Note that Ubuntu 12.04 and 14.04 is the best-supported testing OS.

Testing Data Summarize Tool

Following the steps here to setup the tool. For the time being, only Ubuntu is officially supported as a testing environment; you can use OS X but you'll have to manually solve some package dependency issue. Note that Ubuntu 12.04 and 14.04 is the best-supported testing OS.

Add-on

Performance mark on profiling results at pageMod start and end

You can install this add-on from Here

Sending Performance Mark by Hotkeys

You can install this add-on from Here