Media/WebRTC/WebRTC Debugging
Reporting WebRTC Call Issues
The best way to report an issue is through Bugzilla using this link. Describing the issue you've run into, and include a URL along, with the details of the call setup.
For simple issues, the first place to look is to check the web developer console for error messages related to media format issues. If you see messages here related to WebRTC, getUserMedia, or getDisplayMedia, please add this information to your bug.
Share Your about:webrtc Contents
- While your call is still ongoing, open a tab and visit about:webrtc
- Click "Clear History" to clear the stats from other recent calls which are no longer ongoing.
- At the bottom of the page click 'Save Page', and save this file.
- Add this file as an attachment to your bug.
This file contains statistics about your call, the signalling that was used to setup your call, and information about the network transports.
Common Call Quality Issues
Audio/Video Delay
Delayed media is commonly caused by long physical paths between endpoints, though anything that slows down inter-hop delivery of packets can be at fault. Note that this is different than the bandwidth of the network path, and a high latency will not be fixed by reducing the video resolution or audio sample rate. Round trip time, or RTT, is the time it takes for a packet to get from the sender to the receiver and then for a packet to get from the receiver back to the sender. If the path is symmetric between the two endpoints one can assume that the one way delay is half the delay of the round trip.
The second common cause of A/V delay is jitter, the magnitude of variability in packet inter-arrival times. In order to smoothly play out of the incoming stream a receiver experiencing jitter will have to buffer (delay) incoming packets.
Using [about:webrtc] to Diagnose Delay
The key metrics in [about:webrtc] are RTT (round-trip-time) and jitter. They can be found in the RTP stats section of the PeerConnection.
In this image we can see that there are 0 milliseconds of jitter, and 32 milliseconds of round trip delay. This call should not be experiencing any noticeable delay. For all intents and purposes jitter and RTT are additive in nature. If there was 25ms of jitter reported and a RTT of 272ms, one could estimate the expected delay from transmission at the send side to play out on receive side to be 25ms + (272ms / 2) = 161ms If the perceived delay is larger than the estimated delay that could indicate a problem within Firefox that requires debugging. Under these circumstances it would be helpful to grab a JSON copy of the current stats by pressing the "Copy Report" button, pasting those stats into your Bugzilla bug report.
Logging & Profiling
Logging can be enabled through the "Enable WebRTC Log Preset" button at the bottom of [about:webrtc]. Alternatively one can set the following environment variable:
MOZ_LOG="jsep:5,sdp:5,signaling:5,mtransport:5,RTCRtpReceiver:5,RTCRtpSender:5,RTCDMTFSender:5,VideoFrameConverter:5,WebrtcTCPSocket:5,CamerasChild:5,CamerasParent:5,VideoEngine:5,ShmemPool:5,TabShare:5,MediaChild:5,MediaParent:5,MediaManager:5,MediaTrackGraph:5,cubeb:5,MediaStream:5,MediaStreamTrack:5,DriftCompensator:5,ForwardInputTrack:5,MediaRecorder:5,MediaEncoder:5,TrackEncoder:5,VP8TrackEncoder:5,Muxer:5,GetUserMedia:5,MediaPipeline:5,PeerConnectionImpl:5,WebAudioAPI:5,webrtc_trace:5,RTCRtpTransceiver:5,ForwardedInputTrack:5,HTMLMediaElement:5,HTMLMediaElementEvents:5"
Note that webrtc_trace will not be active until "Enable WebRTC Log Preset" is pressed.
Capturing a Firefox Profiles
- obtain profiler
- enable webrtc logging profile
- starting the profiler
- stopping the profiler
- uploading a profile
Standard Logging Modules
| Module | Component | Function | Notes |
|---|---|---|---|
| jsep | signalling | JSEP state machine | |
| sdp | signalling | SDP parsing | |
| mtransport | networking | network transports | |
| RTCRtpReceiver | JS API | JS API related to receiving media and media control packets | |
| RTCRtpSender | JS API | JS API related to sending media and media control packets | |
| RTCDMTFSender | JS API | JS API related to sending DTMF messages | |
| VideoFrameConverter | ? | ? | |
| WebrtcTCPSocket | networking | ? | |
| CamerasChild | media capture | Content process end of IPC channel for receiving frames from media capture devices | |
| CamerasParent | media capture | Parent process end of IPC channel for sending frames from media capture devices | |
| VideoEngine | media capture | Orchestrates capture of frames from media capture devices in the parent process | |
| ShmemPool | media capture | Object pool of shared memory frame buffers for transferring media capture frames from parent to child process | |
| TabShare | media capture | ? | |
| MediaChild | media | ? | |
| MediaParent | media | ? | |
| MediaManager | media | ? | |
| MediaTrackGraph | media | ? | |
| cubeb | media | ? | |
| MediaStream | media | ? | |
| MediaStreamTrack | media | ? | |
| DriftCompensator | media | ? | |
| ForwardInputTrack | media | ? | |
| MediaRecorder | media | ? | |
| MediaEncoder | media | ? | |
| TrackEncoder | media | ? | |
| VP8TrackEncoder | media | ? | |
| Muxer | media | ? | |
| MediaPipeline | network | Glue code between transport, media, and libwebrtc components | |
| PeerConnectionImpl | JS API | implements the RTCPeerConnection object | |
| WebAudioAPI | ?? | ? | |
| webrtc_trace | webrtc | libwebrtc logging | needs to be enabled from [about:webrtc] |
| RTCRtpTransceiver | JS API | implements the RTCRtpTransceiver object | |
| ForwardedInputTrack | ?? | ? | |
| HTMLMediaElement | ?? | ? | |
| HTMLMediaElementEvents | ?? | ? |
Non-standard Logging Modules
| Module | Component | Function | Notes |
|---|---|---|---|
| RTPLogger | network | See Debugging Encrypted Packets |
Profiling
One can use the "WebRTC" preset on the [about:logging] page with the Firefox Performance Profiler.
Examining Call Performance Issues
Enabling Call Stats History
Call stats history is enabled by default in Nightly. To enable in release builds open [about:config], and change "media.aboutwebrtc.hist.enabled" to true. This will keep a history windows of stats for a number of recent calls, allowing for inspection in [about:webrtc] after a call has completed.
Dumping Call Stats
One can dump a JSON blob of call stats for an active call, or a recent call if call stats history is enabled. There are two buttons in [about:webrtc] to do this, "Copy Report" and "Copy Report History". The former will create a copy of the most recent stats for the PeerConnection. The later will copy all the history of stats reports that [about:webrtc] has accumulated for that PeerConnection, this can be up to several minutes of stats.
Debugging Encrypted Packets
There is a blog post covering dumping unencrypted partial RTP and RTCP packets in the logs. This allows one to manipulate those log statements into something that Wireshark can read. The command to extract the packet data in the blog is out of date. The logs produced by this module can be quite large, making it easy to identify by file size which child log file(s) contains the dump(s). Use the following command instead
TODO
.
Running WebRTC Tests
Mochitests
Running the WebRTC mochitests can be done through mach. On try WebRTC mochitests are part of the media sub- suite. They can be run as follows:
./mach try fuzzy --query 'mochitest-media'
Web Platform Tests
Web Platform Tests can be run locally from mach.
./mach wpt testing/web-platform/tests/webrtc
These tests are synced from the main Web Platform Test repository, and likewise our changes are synced from our in-tree copy back to that repository. To run those tests from try one can use the following
./mach try fuzzy --query 'wpt'
One can run those same tests in Chromium if one needs to compare behavior between browsers.