QA/Desktop Firefox/WebRTC: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(80 intermediate revisions by 6 users not shown)
Line 1: Line 1:
See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.
See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.


== Routine Manual Testing Tasks ==
== Basic Topology ==
 
The following tasks are essentially mechanical, but at least for now the
need a person.
 
The basic topology should include two separate machines on separate networks,
The basic topology should include two separate machines on separate networks,
each with their own NAT. We have some flexibility about which NAT, but
each with their own NAT. We have some flexibility about which NAT, but
Line 13: Line 9:
[[File:topoplogy.png]]
[[File:topoplogy.png]]


* Platforms to include all Windows, Mac OSX, Linux, Android, FirefoxOS
The following test series should be run, with one machine on each network with Nightly.
* Daily regression testing with Firefox Nightly across all platforms to include:
 
** Calling with [http://apprtc.webrtc.org apprtc] using various combinations of Firefox and Chrome
=== Endpoints ===
** Calling with [http://webrtcme.herokuapp.com/r/ webrtcme] using various combinations of Firefox and Chrome
* Firefox Nightly <-> Firefox Nightly
** Evaluation of audio/video delay
* Firefox Nightly <-> Chrome Stable
** [http://mozilla.github.com/webrtc-landing WebRTC-Landing] tests pass
* Firefox Nightly <-> Chrome Canary
** ''Daily might be too ambitious, can it be done weekly?''
 
=== Platforms ===
* Desktop (Windows, Mac, Linux)
* B2G (Unagi, Peak)
* Android
 
=== AppRTC ===
Conduct a call for no less than 5 minutes with [http://apprtc.webrtc.org apprtc] resulting in no lag, no freezing, no audio or video delays, no degradation in quality from the previous revision.


=== WebRTC.ME ===
Conduct a call for no less than 1 minute using [http://webrtcme.herokuapp.com/r/ webrtcme] resulting in minimal lag and freezing.


== QA Tasks ==
=== WebRTC Landing ===
Run the [http://mozilla.github.com/webrtc-landing WebRTC-Landing tests] with no failures.


* Daily verification of whatever fixes landed in that day's Nightly
== Known Issues ==
* Weekly triage and attempted confirmation of unconfirmed/regression bugs
* {{bug|936034}} - Sound is distorted on one side on 1:1 call
* Monthly re-evaluation of regression rate to inform adjustments to the above approach


=== Testing ===
== Meetings ==
* Manual
* [https://etherpad.mozilla.org/qa-webrtc-meeting-20130814 August 14, 2013]: Kick-off meeting with WebRTC Devs
** What must be done manually?
* Automated
** What can be automated?

Latest revision as of 19:27, 7 February 2014

See: https://wiki.mozilla.org/Media/WebRTC/Testing for background on automated testing.

Basic Topology

The basic topology should include two separate machines on separate networks, each with their own NAT. We have some flexibility about which NAT, but for concreteness, I suggest a pair of Apple Airport Express NATs. I.e., the following topology:

Topoplogy.png

The following test series should be run, with one machine on each network with Nightly.

Endpoints

  • Firefox Nightly <-> Firefox Nightly
  • Firefox Nightly <-> Chrome Stable
  • Firefox Nightly <-> Chrome Canary

Platforms

  • Desktop (Windows, Mac, Linux)
  • B2G (Unagi, Peak)
  • Android

AppRTC

Conduct a call for no less than 5 minutes with apprtc resulting in no lag, no freezing, no audio or video delays, no degradation in quality from the previous revision.

WebRTC.ME

Conduct a call for no less than 1 minute using webrtcme resulting in minimal lag and freezing.

WebRTC Landing

Run the WebRTC-Landing tests with no failures.

Known Issues

  • bug 936034 - Sound is distorted on one side on 1:1 call

Meetings

Retrieved from "https://wiki.allizom.org/index.php?title=QA/Desktop_Firefox/WebRTC&oldid=920114"