canmove, Confirmed users
2,041
edits
Firefox OS/Performance/Automation: Difference between revisions
Jump to navigation
Jump to search
→Deliverables
| Line 55: | Line 55: | ||
; Validity Review : Review by test stakeholders for whether the test case design meets requirements and will measure the right thing. Tests that measure the wrong thing can still give repeatable results, so it is important to review the initial design. | ; Validity Review : Review by test stakeholders for whether the test case design meets requirements and will measure the right thing. Tests that measure the wrong thing can still give repeatable results, so it is important to review the initial design. | ||
; Workload : Data (a fixture) to be preloaded into the system under test prior to execution. There are default workflows described at [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads MDN], but some tests may need special-case workflows to be developed. | ; Workload : Data (a fixture) to be preloaded into the system under test prior to execution. There are default workflows described at [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads MDN], but some tests may need special-case workflows to be developed. | ||
; Instrumentation : Code either injected | ; Instrumentation : Code either injected into or permanently added to Firefox OS or its applications in order to let automation monitor performance details. Some tests do not need this. | ||
; On-Demand Test : Test that can be run by a developer on command line, try server, or some other way to test a set of arbitrary code. | ; On-Demand Test : Test that can be run by a developer on command line, try server, or some other way to test a set of arbitrary code. | ||
; Results Review : Review by test stakeholders of a set of results of the implemented test. This is a sanity check for whether the results look like something valid (a result frame rate of 70fps on a 60Hz screen would fail this, for example) and whether they are essentially repeatable (reasonably grouped, assuming no code changes). | ; Results Review : Review by test stakeholders of a set of results of the implemented test. This is a sanity check for whether the results look like something valid (a result frame rate of 70fps on a 60Hz screen would fail this, for example) and whether they are essentially repeatable (reasonably grouped, assuming no code changes). | ||