QA/Automation/Projects/MTBF

Overview

The MTBF tests are a suite of Firefox OS tests built on top of the Gaiatest (Gaia UI Tests) Framework. The tests run on real Firefox OS devices, and use Marionette to drive the device's UI.

Mean time between failures (MTBF) is the predicted elapsed time between inherent failures of a system during operation. MTBF can be calculated as the arithmetic mean (average) time between failures of a system. The MTBF is typically part of a model that assumes the failed system is immediately repaired (mean time to repair, or MTTR), as a part of a renewal process. This is in contrast to the mean time to failure (MTTF), which measures average time to failures with the modeling assumption that the failed system is not repaired (infinite repair time). Assuming T(i) is ith running time between two failures, MTBF will be sum(T(n))/n

Name:	Mean Time Between Failures (MTBF)
Leads:	Device QA Team
Contributors:	Paul Yang, Walter Chen
MDN:	https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/MTBF_tests
Git Repository:	https://github.com/Mozilla-TWQA/MTBF-Driver/

Pass Criteria

Currently we set 100 hours as our pass criteria. Crash, hang are regarded as failure.

Test Scope

The scope of V2.1 is still under determination. We might still change it slightly.

Version	Test Cases
V2.0	Dummy Tests, Keyboard * 4, Music * 3, Message * 1
V2.1	Dummy Tests, Keyboard * 4, Music * 3, Message * 1, FMRadio * 1, Dialer * 1, Cards View * 1, Camera * 1, Settings * 1, Browser * 6, Video * 1

Backlog

• Crash info detection
• Report enhancement
• Hang detection
• Integrating mozbase b2g related modules
• Datazilla integration

Current Bugs

MTBF Meta Bug

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

MTBF Marionette Meta Bug

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Bugzilla query error

Invalid type (count) and display (table) combination1

MTBF B2G Meta Bug

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Bugzilla query error

Invalid type (count) and display (table) combination1

MTBF-2014Q3

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Bugzilla query error

Invalid type (count) and display (table) combination1

Documentation

MTBF Reports History

MTBF Trend

Moztrap MTBF Test Cases

References

• Getting Started
  • Javascript
    • http://www.codecademy.com/tracks/javascript (online course)
    • http://bonsaiden.github.io/JavaScript-Garden/ (good practices)
    • http://eloquentjavascript.net/ (book - accessible as HTML)
    • https://developer.mozilla.org/en-US/docs/Web/JavaScript (reference)
  • Python
    • http://www.codecademy.com/tracks/python
    • https://wiki.python.org/moin/BeginnersGuide
  • HTML / CSS
    • http://www.codecademy.com/tracks/web