Many Projects, all Complementary
- Special Powers
- Going Faster
- Project to refactor base libraries of test harnesses into a coherent set of tools and scripts.
- MozProcess, MozRunner, MozInfo done
- MozLog - fragmented
- MozProfile, MozEnv, MozOptions - not complete
- Goal: Tie into a generic low-level harness library that can be easily wired into MozHarness.
- API Implemented in Mochitest
- Needed in Reftest, Mozmill
- Needed in XPCShell?
- Porting Mochitests to API
- Will have some performance impact
- Goal: Enable test harnesses to run in out of process architecture and still have access to chrome privileges/APIs as required
- Goal: Reducing the end to end time as measured from pushing a change to getting results.
- Early Profiling results
- packed.js seems to average about 110ms per load.
- There are around 3100 test files in mochitest. So, eliminating packed.js would save almost 341 seconds (5 minutes) per run.
- In the makeIframe script, I'm seeing the window.scrollTo call take 8-10ms per call. Not sure how much we can streamline that, but best case that's 24 seconds on every run.
Related to GoFaster and MozBase
- httpd.js could be a bottleneck.
- How to profile?
- How to go about trying out replacements to it?
- Even if we stay with httpd.js, we need a better story for packaging it so that you download one item and it "just works" (tm)
- How to best work together without stepping on toes?
- How can we monitor test harness performance so we know if we check in something that blows our gains?
- Cleanup harness
- Many tools touched (unit tests, talos etc.)
- Moving tests to the right harnesses
- Update all the tests
- need to contact other devs to fix their tests
- Mochitest monitoring to see if Special Powers will slow us down, need to check end to end time
- Getting rid of packed.js would shave ~5 mins from test run of mochitests. We only use a small bit of it, we don't need to load all.
- Write a Mochitest profiling test suite so we can run to profile our progress.
- Is our JS webserver a bottleneck? Should we switch to another server? Suggestions: nginx, lighttpd, etc.
- Profiling: Run a suite that doesn't use .sjs or special content headers and test against current httpd.js server and the server we're testing
- There should be profilers for each harness, but start with mochitests.
- Should we combine some Mochitest test files into one huge generated test instead of a directory of many test files so we can load the overhead once?