QA/Desktop Firefox/WebRTC: Difference between revisions
< QA | Desktop Firefox
Jump to navigation
Jump to search
| Line 1: | Line 1: | ||
== | See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing. | ||
== Routine Manual Testing Tasks == | |||
The following tasks are essentially mechanical, but at least for now the | |||
need a person. | |||
The basic topology should include two separate machines on separate networks, | |||
each with their own NAT. We have some flexibility about which NAT, but | |||
for concreteness, I suggest a pair of Apple Airport Express NATs. I.e., the | |||
following topology: | |||
[[File:topoplogy.png]] | |||
* Platforms to include all Windows, Mac OSX, Linux, Android, FirefoxOS | * Platforms to include all Windows, Mac OSX, Linux, Android, FirefoxOS | ||
* Daily regression testing with Firefox Nightly across all platforms to include: | * Daily regression testing with Firefox Nightly across all platforms to include: | ||
| Line 7: | Line 20: | ||
** [http://mozilla.github.com/webrtc-landing WebRTC-Landing] tests pass | ** [http://mozilla.github.com/webrtc-landing WebRTC-Landing] tests pass | ||
** ''Daily might be too ambitious, can it be done weekly?'' | ** ''Daily might be too ambitious, can it be done weekly?'' | ||
== QA Tasks == | |||
* Daily verification of whatever fixes landed in that day's Nightly | * Daily verification of whatever fixes landed in that day's Nightly | ||
* Weekly triage and attempted confirmation of unconfirmed/regression bugs | * Weekly triage and attempted confirmation of unconfirmed/regression bugs | ||
Revision as of 21:39, 23 July 2013
See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.
Routine Manual Testing Tasks
The following tasks are essentially mechanical, but at least for now the need a person.
The basic topology should include two separate machines on separate networks, each with their own NAT. We have some flexibility about which NAT, but for concreteness, I suggest a pair of Apple Airport Express NATs. I.e., the following topology:
- Platforms to include all Windows, Mac OSX, Linux, Android, FirefoxOS
- Daily regression testing with Firefox Nightly across all platforms to include:
- Calling with apprtc using various combinations of Firefox and Chrome
- Calling with webrtcme using various combinations of Firefox and Chrome
- Evaluation of audio/video delay
- WebRTC-Landing tests pass
- Daily might be too ambitious, can it be done weekly?
QA Tasks
- Daily verification of whatever fixes landed in that day's Nightly
- Weekly triage and attempted confirmation of unconfirmed/regression bugs
- Monthly re-evaluation of regression rate to inform adjustments to the above approach
Testing
- Manual
- What must be done manually?
- Automated
- What can be automated?