QA/Automation/Projects/MTBF
Overview
The MTBF tests are a suite of Firefox OS tests built on top of the Gaiatest (Gaia UI Tests) Framework. The tests run on real Firefox OS devices, and use Marionette to drive the device's UI.
Mean time between failures (MTBF) is the predicted elapsed time between inherent failures of a system during operation. MTBF can be calculated as the arithmetic mean (average) time between failures of a system. The MTBF is typically part of a model that assumes the failed system is immediately repaired (mean time to repair, or MTTR), as a part of a renewal process. This is in contrast to the mean time to failure (MTTF), which measures average time to failures with the modeling assumption that the failed system is not repaired (infinite repair time).
Assuming T(i) is ith running time between two failures, MTBF will be SUM ( T ( n ) ) / n
Below is information for MTBF in Mozilla
| Name: | Mean Time Between Failures (MTBF) |
| Leads: | Device QA Team |
| Contributors: | Paul Yang, Walter Chen |
| MDN: | https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/MTBF_tests |
| Git Repository: | https://github.com/Mozilla-TWQA/MTBF-Driver/ |
Pass Criteria
Currently we set 100 hours as our pass criteria. Crash, hang are regarded as failure.
Test Scope
The scope of V2.1 is still under determination. We might still change it slightly.
| Version | Test Cases |
| V2.0 | Dummy Tests, Keyboard * 4, Music * 3, Message * 1 |
| V2.1 | Dummy Tests, Keyboard * 4, Music * 3, Message * 1, FMRadio * 1, Dialer * 1, Cards View * 1, Camera * 1, Settings * 1, Browser * 6, Video * 1 |
Test Suite
- Keyboard - 1. number keyboard 2. add contact 3. email keyboard
- Music - play 10 sec music in song view
- Message - send a message to itself
- FMRadio - play and see radio is enable
- Dialer - make a call from call log
- Cards View - Open two apps and kill one
- Camera - take two pictures
- Settings - turn on and turn off wifi
- Browser - 1. browse a website 2. add a bookmark 3. browse from bookmark
- Video - play a very short video
Backlog
- More test suites (on-going)
- bt on/off
- airplane mode on/off
- Camcorder
- Crash info detection
- Report enhancement
- Hang detection
- Integrating mozbase b2g related modules
- Datazilla integration
Current Bugs
MTBF Meta Bug
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
MTBF Marionette Meta Bug
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Bugzilla query error
Invalid type (count) and display (table) combination1
MTBF B2G Meta Bug
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Bugzilla query error
Invalid type (count) and display (table) combination1
MTBF-2014Q3
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
Bugzilla query error
Invalid type (count) and display (table) combination1
Documentation
References
- Getting Started
- Javascript
- http://www.codecademy.com/tracks/javascript (online course)
- http://bonsaiden.github.io/JavaScript-Garden/ (good practices)
- http://eloquentjavascript.net/ (book - accessible as HTML)
- https://developer.mozilla.org/en-US/docs/Web/JavaScript (reference)
- Python
- HTML / CSS
- Javascript