Media/WebRTC/Testing: Difference between revisions

From MozillaWiki
< Media‎ | WebRTC
Jump to navigation Jump to search
Line 72: Line 72:
* mediapipeline_unittest &mdash; unit tests for the media pipelines that plumb between the Media Stream Graph, GIPS, and RTP
* mediapipeline_unittest &mdash; unit tests for the media pipelines that plumb between the Media Stream Graph, GIPS, and RTP
* signaling_unittests &mdash; full system tests for the PeerConnection API.
* signaling_unittests &mdash; full system tests for the PeerConnection API.
Note that the media pipeline and signaling_unittests do not test video, due to the linkage reasons mentioned below.


=== Mochitests and Crashtests ===
=== Mochitests and Crashtests ===

Revision as of 23:51, 28 April 2013

WebRTC Test System

We currently test WebRTC with two sets of tests:

  • Standalone C++ unit and system tests.
  • Full system mochitests.


C++ Unit Tests

The WebRTC code has extensive C++ unit and system tests based on the Google Test (https://code.google.com/p/googletest/) framework. These tests live in two locations:

  • media/mtransport — the mtransport generic transport subsystem and associated utilities
  • media/webrtc/signaling/test — the signaling system and the media handling system

As usual, you can run them individually by doing:

  make -C <directory>

Alternately, you can run a single C++ unit test by name, e.g.,

  media/mtransport/test/ice_unittest

Tinderbox runs all the tests in a given directory but for a variety of reasons some of the tests aren't run on Tinderbox. This is controlled by requiring certain environment variables to be set.

A number of tests have been disabled on Tinderbox because they either take a long time or have excessive oranges. To enable these, set:

   MOZ_WEBRTC_TESTS=1


Additionally, the TURN tests require you to specify a TURN server. However, we don't provide one for Tinderbox because we don't want to operate a public server and releng hasn't set one up in the test environment yet ( https://bugzilla.mozilla.org/show_bug.cgi?id=865296)

If you have a TURN server, you can point the tests at it by setting:

   TURN_SERVER_ADDRESS
   TURN_SERVER_USER
   TURN_SERVER_PASSWORD


Currently we have the following test programs, each of which consists of one or more individual test cases. Note that you can enumerate the test cases like so:

    obj-x86_64-apple-darwin10.8.0/media/mtransport/test/ice_unittest --gtest_list_tests

Individual tests can be run with --gtest_filter, like so:

    obj-x86_64-apple-darwin10.8.0/media/mtransport/test/ice_unittest '--gtest_gilter=*Turn*'


media/mtransport/test

  • sockettransportservice_unittest — tests various uses of the Socket Transport Service
  • nrappkit_unittest — tests Mozilla implementations of the nrappkit primitives (needed for ICE)
  • runnable_utils_unittest — tests the runnable utils in runnable_utils.h
  • turn_unittest — unit tests for TURN
  • ice_unittest — overall tests for ICE
  • sctp_unittest — tests for the SCTP code running on top of transport
  • transport_unittests — system tests for the mtransport subsystem including the DTLS wrapper and ICE wrappers


media/webrtc/signaling/test

  • mediaconduit_unittests — unit tests for the audio/video conduit wrappers around GIPS
  • mediapipeline_unittest — unit tests for the media pipelines that plumb between the Media Stream Graph, GIPS, and RTP
  • signaling_unittests — full system tests for the PeerConnection API.

Note that the media pipeline and signaling_unittests do not test video, due to the linkage reasons mentioned below.

Mochitests and Crashtests

We also have a series of Mochitests which live in dom/media/test/mochitest and dom/media/test/crashtest. These can be run via the usual Mochitest mechanisms.

These tests mostly cover:

  • getUserMedia.
  • End-to-end calling scenarios
  • That the system responds correctly to various kinds of API misuse (e.g., wrong arguments, etc.)