MozillaWiki

Media/WebRTC/Architecture: Difference between revisions

Page Discussion

Media/WebRTC/Architecture (view source)

Revision as of 21:58, 6 July 2017

4,004 bytes added ,  6 July 2017
Added diagram illustrating internal vs external code
(Added diagram illustrating internal vs external code)
 
(43 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<h2> Overall Architecture </h2>
<h2> Overall Architecture </h2>
<p>Note: see [[Media/WebRTC/WebRTCE10S]] for the architecture on B2G with E10S</p>
<p>https://github.com/mozilla/webrtc/raw/master/planning/architecture.png
<p>https://github.com/mozilla/webrtc/raw/master/planning/architecture.png
</p><p><br />
</p><p><br />
Line 43: Line 44:
==== PeerConnection vs. CC_Call ====
==== PeerConnection vs. CC_Call ====


ISTM that the natural design is to have PeerConnection be 1:1 with CC_Call, so that when you do CreateOffer, it's effectively kicking off the offer process on an existing call. Then we can reuse the SIPCC state machine to some extent to manage the call state. Subsequent calls to the other JSEP API's such as setRemoteDescription and localDescription will run on the same call and use the same state machine.
The PeerConnection is 1:1 with CC_Call, so that when you do CreateOffer, it's effectively kicking off the offer process on an existing call. Then we can reuse the SIPCC state machine to some extent to manage the call state. Subsequent calls to the other JSEP API's such as setRemoteDescription and localDescription will run on the same call and use the same state machine. There is a global singleton PeerConnectionCtx which handles callbacks/notifications from SIPCC.


==== Mtransport Interface ====
==== Mtransport Interface ====


The mtransport (ICE, DTLS) subsystem is pretty independent of SIPCC. We have two main options for wiring them up:
The mtransport (ICE, DTLS) subsystem is pretty independent of SIPCC. Roughly speaking, they are wired up as follows:


* Have Webrtc{Audio,Video}Provider instantiate the mtransport objects and manage them. This is roughly the way things are constructed now, but we would need to modify the existing code to talk to mtransport and to be prepared to receive and handle ICE candidate updates (both internally generated and received from the JS).  
* The PeerConnection creates ICE objects (NrIceCtx) as soon as it starts up. It creates as many as the number of m-lines we expect to need.
* Have the PeerConnection set up the mtransport objects, manage ICE and DTLS, and then provide handles downward to SIPCC.  The major obstacle here is that we somehow have to figure out how to get this data into and out of the SDP. The advantage is that it avoids tight coupling in WebRTC{Audio,VideoProvider}.
* When SIPCC (lsm) determines that a new media flow is required it stands up a MediaPipeline (containing the MediaConduit [codecs], SRTP contexts, and a TransportFlow (DTLS, ICE, etc.)


I was originally leaning towards the second of these approaches, but now that I have thought about it a while, I think the first is probably better, and that's what's shown in the diagram above.
Note that each MediaPipeline is one-way, so a two-way audio flow has two media pipelines. However, since you are doing symmetric RTP, you likely have two MediaPipelines for each TransportFlow, though there may be two Transportflows for each MediaPipeline if RTCP is not muxed.
 
==== Internal vs 3rd party code ====
 
<p>https://github.com/nils-ohlmeier/firefox-webrtc-documentation/raw/master/Firefox-WebRTC-internal-3rdparty.png
</p><br>
Colors in the diagram:
* White: Mozilla's own code
* Orange: 3rd party code
* Green: webrtc.org code shared with Google Chrome
 
== List of Components ==
 
The system has the following individual components, in no particular order
 
* PeerConnection
** PeerConnection.js -- shim translation layer to let us do API adaptation to the C++
** PeerConnectionImpl -- C++ implementation of the PeerConnection interface.
** SIPCC -- handles SDP and media negotiation. Provided by Cisco but not a downstream.
 
* Media
** Webrtc.org/GIPS -- handles media encoding and decoding. Downstream from Google.
** MediaConduit -- Generic wrapper around Webrtc.org
** MediaPipeline -- Wrapper to hold the MediaConduit, mtransport subsystem, and the SRTP contexts, as well as interface with MediaStreams.
 
* Transport
** mtransport -- generic transport subsystem with implementations for ICE, DTLS, etc.
** NSS -- new DTLS stack. Mentioned because we need to land the new version of NSS
** nICEr -- ICE stack; downstream from reSIProcate project
** nrappkit --portable runtime, utility library; downstream from nrappkit.sourceforge.net
 
* DataChannel
** DataChannel implementation in the DOM
** libsctp -- SCTP implementation; downstream from the BSD SCTP guys




Line 93: Line 127:


Internally, <tt>cprSendMessage()</tt> is a write to a unix domain socket, and the receiving thread
Internally, <tt>cprSendMessage()</tt> is a write to a unix domain socket, and the receiving thread
needs to loop around <tt>cprGetMessage()</tt>, as in <tt>ccap_task.c</tt>:
needs to loop around <tt>cprGetMessage()</tt>, as in <tt>ccapp_task.c</tt>:


     /**
     /**
Line 170: Line 204:
=== Signaling System: CreateOffer ===
=== Signaling System: CreateOffer ===


http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENyZWF0ZU9mZmVyKQpwYXJ0aWNpcGFudCAiRE9NACAHIiBhcyBET00AEw1QQyBhcyBQQwAnDkNDQVBQVGFzawAvBQAJBQBJDUdTTVRhc2sgYXMgR1MAQw5TVFMKCkRPTSAtPiBQQzogRGlzcGF0Y2gAgRUOUEMgLT4AUwY6IElQQyhDQ19DYWxsRmVhdHVyZV8AgUMNAIEJBSAtPiBHU00AJQlNU0dfQ1JFQVRFT0ZGRVIpCkdTAG8FAB8FZnNtZGVmX2V2X2MAghQFb2ZmZXIAFQ1nc21zZHAAGAdfbG9jYWxfc2RwABAUZW5jb2RlX3NkcF9hbmRfdXBkYXRlX3ZlcnNpb24Aag0AZwYgd2VicnRjLm9yZyBmbG93cwAQFG10cmFuc3BvcnQAFg5TVFMAgikLABwKW0lDRV0tPkNvbm5lY3QoKSkKU1RTAIIHDQCBPwUgSUNFIGNhbmRpZGF0ZXMAgg0JAIJfCwCCKgUgU0RQAIJOCwCDHA1vbkNhbGxFdmVudAAiCwCDLAhET00Ag0oLAIRtBQBHBgpub3RlIGxlZnQgb2YAgyUGU3Vic2VxdWVudACBEwV0cmlja2xpbmcAHQ4ATQVKUyBjYWxsYmFja1xud2l0aCAAgy4GAIFDJACBUhRDABMKAIFOGElDRQCBXwYAIgoAgU8WAF4PAIEZJACCegkKCg&s=default
http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENyZWF0ZU9mZmVyKQpwYXJ0aWNpcGFudCAiRE9NACAHIiBhcyBET00AEw1QQyBhcyBQQwAnDkNDQVBQVGFzawAvBQAJBQBJDUdTTVRhc2sgYXMgR1MAQw5TVFMKCkRPTSAtPiBQQzogRGlzcGF0Y2gAgRUOUEMgLT4AUwY6IElQQyhDQ19DYWxsRmVhdHVyZV8AgUMNAIEJBSAtPiBHU00AJQlNU0dfQ1JFQVRFT0ZGRVIpCkdTAG8FAB8FZnNtZGVmX2V2X2MAghQFb2ZmZXIAFQ1wcm9jZXNzIGNvbnN0cmFpbnRzADUNADIGIGxvY2FsIHNkcAAJFHdlYnJ0Yy5vcmcgZmxvdwAzDnF1ZXJ5AAUeZ2VuZXJhdGUgU0RQIHN0cmluZwoAcRRtdHJhbnNwb3J0AGEHAIFkCFNUUwCCVgsAHQpbSUNFXS0-Q29ubmVjdCgpKQpTVFMAgjQNAIFSBklDRSBjYW5kaWRhdGVzAII6CQCDDAsAglcFIFNEUACCewsAg0kNb25DYWxsRXZlbnQAIgsAg1kIRE9NAIN3CwCFGgUARwYKbm90ZSBsZWZ0IG9mAINSBlN1YnNlcXVlbnQAgRMFdHJpY2tsaW5nAB0OAE0FSlMgY2FsbGJhY2tcbndpdGggAINbBgCBQyQAgVIUQwATCgCBThhJQ0UAgV8GACIKAIFPFgBeDwCBGSQAgnoJCgo&s=default


[TODO: EKR. (1) Should we do the ICE allocation in PC and just have SIPCC ask it. (2) first offer creation is asynchronous internally because ICE may be asynchronous.]
[TODO: EKR. Parallel forking and cloning.]
[TODO: EKR. Parallel forking and cloning.]


The sequence for CreateOffer is shown in the link above.
The sequence for CreateOffer is shown in the link above.


As described above, most of the heavy lifting for the API happens off the DOM thread. Thus, when the JS invokes CreateOffer(), this turns into a Dispatch to the PeerConnection thread which (after some activity of its own) invokes SIPCC's FEATURE_CREATE_OFFER via a SIPCC IPC message to CCAPP_Task, which using the GSMTask thread creates the appropriate transport flows (mtransport) and constructs the SDP. Eventually, it constructs the SDP offer and then Dispatches it to the PeerConnection thread, which Dispatches the result to the DOM thread which eventually calls the JS CreateOffer callback. [Question: is this actually not Dispatch but some fancier call.]
As described above, most of the heavy lifting for the API happens off the DOM thread. Thus, when the JS invokes CreateOffer(), this turns into a Dispatch to the PeerConnection thread which (after some activity of its own) invokes SIPCC's FEATURE_CREATE_OFFER via a SIPCC IPC message to CCAPP_Task, which using the GSMTask thread constructs the SDP based on local streams added using AddStream. The appropriate transport flows (mtransport) are then constructed. (Note thast they may already exist if some have been pre-created). The SDP offer is then Dispatched back all the way to the DOM thread which eventually calls the JS CreateOffer callback. [Question: is this actually not Dispatch but some fancier call.]


In the meantime, the ICE gathering process has been running on the STS thread. As each candidate is gathered, the STS thread does an IPC call to the GSMTask thread to return the ICE candidate. These candidates are then Dispatched back to the CCAPP_Task thread and then to the PC thread and then eventually to the DOM thread where the ICE candidate events fire.
In the meantime, the ICE gathering process has been running on the STS thread. As each candidate is gathered, the STS thread does an IPC call to the GSMTask thread to return the ICE candidate. These candidates are then Dispatched back to the CCAPP_Task thread and then to the PC thread and then eventually to the DOM thread where the ICE candidate events fire.


[Question: Enda. (1) During function gsmsdp_create_local_sdp the media streams that were added by AddStream are interrogated, how can I represent that in this diagram or is it implied that GSMTask ownes these streams and there is no outside interaction?   (2) I am not sure we do 'create webrtc flows' here, I thought this was the responsibility of setLocalDescription or setRemoteDescription.  I am happy that we create SIPCC internal representation of media streams but not sure we call out to webrtc at this point.]
At this point it may be necessary to create some WebRTC.org streams in order to determine what the acceptable media parameters are for the SDP. For instance, if
 
that is required to determine whether a given codec is supported. [TODO: Enda?] However, they would not be hooked up to anything at this point.


=== Signaling System: SetLocal(Caller) ===
=== Signaling System: SetLocal(Caller) ===


http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlcjogU2V0TG9jYWxEZXNjcmlwdGlvbikKCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHxVQQyAtPgBiBjogSVBDKEZFQVRVUkVfU0VUTE9DQUxFU0MpCgCBEgUgLT4gR1NNACEGQ0NfTVNHAB0JRAAiBQoKR1MAcwUAIgVDb21wYXJlIFNEUCB0byBPZmZlciBTRFAAHQkAbwcAgRcKRFAgIT0AIgc_IFN0YXRlPUVycm9yAHcLAIFFDgAXDACBRQZET00ACRgAXxoAZQVDYWxsIFNlbnQpIABSHQAeCgBYG05lZ290aWF0aW5nKQoK&s=default
http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlcjogU2V0TG9jYWxEZXNjcmlwdGlvbikKCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHxVQQyAtPgBiBjogSVBDKEZFQVRVUkVfU0VUTE9DQUxFU0MpCgCBEgUgLT4gR1NNACEGQ0NfTVNHAB0JRAAiBQpHUwByBQAhBUNvbXBhcmUgaW5wdXQgU0RQIHRvIE9mZmVyZWQgU0RQACUJAHILU3RhdGU9Q2FsbCBTZW50KSAAcAoAgT0OABkPAIFAB0RPTQAUEU5lZ290aWF0aW5nKQoK&s=default


Once the caller has generated the offer the JS can call SetLocalDescription(). This just bubbles its way down to the GSMTask which verifies that the SDP is correct (currently this means matches the outstanding SDP) and if so partially assembles the appropriate media streams and transport objects. Note that we cannot actually plug them together until we have the DTLS fingerprint from the other side (or we will need some clever UI). See the SetRemote(Caller) section below.
Once the caller has generated the offer the JS can call SetLocalDescription(). This just bubbles its way down to the GSMTask which verifies that the SDP is correct (currently this means matches the outstanding SDP) and if so partially assembles the appropriate media streams and transport objects. Note that we cannot actually plug them together until we have the DTLS fingerprint from the other side (or we will need some clever UI). See the SetRemote(Caller) section below.
Line 192: Line 225:
=== Signaling System: SetRemote(Callee) ===
=== Signaling System: SetRemote(Callee) ===


http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlZTogU2V0UmVtb3RlRGVzY3JpcHRpb24pCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHhZQQyAtPgBjBjogSVBDKEZFQVRVUkVfU0VUUkVNT1RFU0MpCgCBEwUgLT4gR1NNACEGQ0NfTVNHABwKRAAjBQoKR1MAdQUAIwVnc21zZHBfbmVnb3RpYXRlX29mZmVyX3NkcAAfCFNUUwCBFwtJQ0UgQ2FuZGlkYXRlcykARAkAgRcHAIFACnRhdGU9QWxlcnRpbmcpIACBFQoAgWMOABkOAIFkB0RPTQA6EU4AgRsHACIFCgo&s=default
http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlZTogU2V0UmVtb3RlRGVzY3JpcHRpb24pCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHhZQQyAtPgBjBjogSVBDKEZFQVRVUkVfU0VUUkVNT1RFU0MpCgCBEwUgLT4gR1NNACEGQ0NfTVNHABwKRAAjBQpHUwB0BQAiBXBhcnNlIGFuZCB2ZXJpZnkgb2ZmZXIAFg1jcmVhdGUgcgCCRwUgU3RyZWFtcwA9CFNUUwCBNAtJQ0UgQ2FuZGlkYXRlcwBhCgCBMAtNZWRpYQA_BgCBKAsAgXgNABYNAIF2BkRPTQAJGABRE1N0YXRlPUFsZXJ0aW5nKSAAURcAGA8AVhUARQZOZWdvdGlhACEGCg&s=default
 
 
The callee receives an offer through its signaling mechanism and calls the JS API setRemoteDescription passing in the SDP. Internally in the GSMTask the offer SDP is parsed, the remote media streams are constructed and events are fired back to the DOM Thread signaling the addition of remote media streams.  If there are ICE candidates in the offer SDP they are dispatched to the STS thread for ICE processing.
 
[TODO: the question of what WebRTC remote streams can be constructed here is tricky. You need to construct some sort of representation in order to tell the generate onaddstream events, but you don't know what the codecs will be until setLocal is called. Enda, you will need to figure this out.]


=== Signaling System: CreateAnswer(Callee) ===
=== Signaling System: CreateAnswer(Callee) ===


http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENyZWF0ZUFuc3dlcikKcGFydGljaXBhbnQgIkRPTQAhByIgYXMgRE9NABMNUEMgYXMgUEMAJw5DQ0FQUFRhc2sALwUACQUASQ1HU01UYXNrIGFzIEdTAEMOU1RTCgpET00gLT4gUEM6IERpc3BhdGNoAIEVD1BDIC0-AFQGOiBJUEMoQ0NfQ2FsbEZlYXR1cmVfAIFEDgCBCwUgLT4gR1NNACYJTVNHX0NSRUFURUFOU1dFUikKR1MAcgUAIAVmc21kZWZfZXZfYwCCGAVhAIIYBQAWDWdzbXNkcF9uZWdvdGlhdGVfb2ZmZXJfc2RwABMUZW5jb2RlX3NkcF9hbmRfdXBkYXRlX3ZlcnNpb24AcwhTVFMAgWgLbXRyYW5zcG9ydFtJQ0VdLT5Db25uZWN0KCkpClNUUwCBRA1sb2NhbCBJQ0UgY2FuZGlkYXRlcwCBSQkAgh0LAIFmBiBTRFAAggwLAIJcDW9uQ2FsbEV2ZW50ACIMAIJsCERPTQCDCwsAhC4GAEkGCm5vdGUgbGVmdCBvZgCCZQZTdWJzZXF1ZW50AIEWBXRyaWNrbGluZwAdDgBOBUpTIGNhbGxiYWNrXG53aXRoIACCbAcAgUckAIFWFEMAEwoAgVEYSUNFAIFiBgAiCgCBURYAXg8AgRokAIJ-CQ&s=default
http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENyZWF0ZUFuc3dlcikKcGFydGljaXBhbnQgIkRPTQAhByIgYXMgRE9NABMNUEMgYXMgUEMAJw5DQ0FQUFRhc2sALwUACQUASQ1HU01UYXNrIGFzIEdTAEMOU1RTCgpET00gLT4gUEM6IERpc3BhdGNoAIEVD1BDIC0-AFQGOiBJUEMoQ0NfQ2FsbEZlYXR1cmVfAIFEDgCBCwUgLT4gR1NNACYJTVNHX0NSRUFURUFOU1dFUikKR1MAcgUAIAVmc21kZWZfZXZfYwCCGAVhAIIYBQAWDXByb2Nlc3MgY29uc3RyYWludHMANg0AMwYgbG9jYWwgc2RwAAkUd2VicnRjLm9yZyBmbG93ADMOcXVlcnkABR5nc21zZHBfbmVnb3RpYXRlX29mZmVyXwBeEFVwZABNIWdlbmVyYXRlIFNEUCBzdHJpbmcKAIILCFNUUwCDAAttdHJhbnNwb3J0W0lDRV0tPkNvbm5lY3QoKSkKU1RTAIJcDQCBeAZJQ0UgY2FuZGlkYXRlcwCCYQkAgzULAIJ-BiBTRFAAgyQLAIN0DW9uQ2FsbEV2ZW50ACIMAIQECERPTQCEIwsAhUYGAEkGCm5vdGUgbGVmdCBvZgCDfQZTdWJzZXF1ZW50AIEWBXRyaWNrbGluZwAdDgBOBUpTIGNhbGxiYWNrXG53aXRoIACEBAcAgUckAIFWFEMAEwoAgVEYSUNFAIFiBgAiCgCBURYAXg8AgRokAIJ-CQ&s=default


The JS calls CreateAnswer passing in the offer SDP to create the answer SDP.  This calls down to the GSMTask thread passing in the offer SDP. The local SDP wil now get created, for this to occur local streams must have been added by the JS API AddStream. If the local SDP is successfully created then the offer SDP will be negotiated, the output of this will be the answer SDP. To create the SDP webrtc.org flows will need to be created and queried. If there are ICE candidates to be added to this answer SDP they will get added. This answer SDP will then get dispatched back to the DOM thread.


=== Signaling System: SetLocal(Callee) ===
=== Signaling System: SetLocal(Callee) ===


http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlZTogU2V0TG9jYWxEZXNjcmlwdGlvbikKCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHxVQQyAtPgBiBjogSVBDKEZFQVRVUkVfU0VUTE9DQUxFU0MpCgCBEgUgLT4gR1NNACEGQ0NfTVNHAB0JRAAiBQoKR1MAcwUAIgVDb21wYXJlIFNEUCB0byBBbnN3ZXIgU0RQAB4IU1RTAIEUC0lDRSBDYW5kaWRhdGVzKQAPFlN0YXJ0IElDRSBoYW5kc2hha2UAZxBjcmVhdGUgd2VicnRjLm9yZyBmbG93cwAQFG10cmFuc3BvcnQAHQcKbm90ZSBvdmVyAIE7CG5uZWN0IAAfCVxuVG8gTWVkaWFTdHJlYW1zAIFqCQCCOAtTdGF0ZT0ANQdlZCkgAII2CgCDAw4AGQ8AgwYHRE9NABQRQWN0aXZlKQCBGgdyaWdodCBvZgCCPAZJQ0UvRFRMUyBjb21wbGV0ZQpTVFMAgyANAIEsBQCBOwogcmVhZHkpAIFdEACCVgZzZW5kaW5nL3JlY2VpdmluZyBtZWRpYQo&s=default
http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlZTogU2V0TG9jYWxEZXNjcmlwdGlvbikKCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHxVQQyAtPgBiBjogSVBDKEZFQVRVUkVfU0VUTE9DQUxFU0MpCgCBEgUgLT4gR1NNACEGQ0NfTVNHAB0JRAAiBQpHUwByBQAhBXZlcmlmeSBTRFAACg1Db21wYXJlIGlucHV0IFNEUCB0byBBbnN3ZXIAJwUAPAgAgQgLU3RhdGU9Q29ubmVjdGVkKSAAgQYKAIFTDgAZDwCBVgdET00AFBFBY3RpdmUpCgpub3RlIHJpZ2h0IG9mIFNUUzogSUNFL0RUTFMgY29tcGxldGUKU1RTAIFwDU1lZGlhIHRyYW5zcG9ydCByZWFkeSkARgZvdmVyAIIhBlN0YXJ0IHNlbmRpbmcvcmVjZWl2aW5nIG1lZGlhCg&s=default
 
Invoked by the JS SetLocalDescription must be called after CreateAnswer and takes as input the answer SDP which is passed down to the GSMTask thread.  Here the SDP is compared to the insernal answer SDP, if there is a difference an error is reported back to the DOM Thread. Following this if ICE completes processing successfuly then the media can start sending and receiving on the webrtc.org flows that were created in CreateAnswer.


=== Signaling System: SetRemote(Caller) ===
=== Signaling System: SetRemote(Caller) ===


http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlcjogU2V0UmVtb3RlRGVzY3JpcHRpb24pCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHhZQQyAtPgBjBjogSVBDKEZFQVRVUkVfU0VUUkVNT1RFU0MpCgCBEwUgLT4gR1NNACEGQ0NfTVNHABwKRAAjBQoKR1MAdQUAIwVnc21zZHBfbmVnb3RpYXRlX2Fuc3dlcl9zZHAAGw1jcmVhdGUgd2VicnRjLm9yZyBmbG93cwAQFG10cmFuc3BvcnQAFg5TVFMAgWALSUNFIENhbmRpZGF0ZXMpAIENCQCBXAtTdGF0ZT1Db25uZWN0ZWQpIACBWgoAgigOABkPAIIqB0RPTQAUEUFjdGl2ZSkKCgpub3RlIHJpZ2h0IG9mAIEWBklDRS9EVExTIGNvbXBsZXRlClNUUwCCRQ1NZWRpYSAAgVYKcmVhZHkpAIEoEwCBWQUAPwcAgxALAINgDQAWDgCBIBQAOA4AgS0Fb3ZlcgCDXAZTdGFydCBzZW5kaW5nL3JlY2VpdmluZyBtZWRpYQoK&s=default
http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlcjogU2V0UmVtb3RlRGVzY3JpcHRpb24pCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHhZQQyAtPgBjBjogSVBDKEZFQVRVUkVfU0VUUkVNT1RFU0MpCgCBEwUgLT4gR1NNACEGQ0NfTVNHABwKRAAjBQpHUwB0BQAiBWNyZWF0ZSByAIIkBSBzdHJlYW1zABoJAGwHAIEWCU1lZGlhUwAgBQBlCwCBNQ0AFg0AgTMGRE9NACwYAIEECFNUUwCBewtJQ0UgQ2FuZGlkYXRlcwCBIw9nc21zZHBfbmVnb3RpYXRlX2Fuc3dlcl9zZHAAgU0NdXBkYXRlIHdlYnJ0Yy5vcmcgZmxvdwCBQhpTdGF0ZT1Db25uZWN0ZWQpIACBShcAGBAAgVAVAEcGQWN0aXZlKQoKbm90ZSByaWdodCBvZgCBZwZJQ0UvRFRMUyBjb21wbGV0ZQpTVFMAgzENAIJoBSB0cmFuc3BvcnQgcmVhZHkpAEYGb3ZlcgCDYgZTdGFydCBzZW5kaW5nL3JlY2VpdmluZyBtZWRpYQoK&s=default


The above diagram shows the caller's SetRemote sequence.
The above diagram shows the caller's SetRemote sequence.


The process starts with receiving the remote description and the JS calling SetRemoteDescription(). This is Dispatched() onto the PC thread and then to CCAPP_Task API thread which passes it to the Internal SIPCC GSMTask thread for processing. Assuming everything is OK, it negotiates this answer with its internal local SDP used in the earlier offer. It also Dispatches the ICE candidates down to the STS thread so that ICE can proceed. At this point a Connected event is returned to CCAPP_Task thread and then to the PC thread where it gets translated to a JSEP ReadyState of Active. Once ICE and DTLS handshaking are complete, a message is sent up to the GSMTask, which can then start sending and receiving media on the working mtransport.
The process starts with receiving the remote description and the JS calling SetRemoteDescription(). This is Dispatched() onto the PC thread and then to CCAPP_Task API thread which passes it to the Internal SIPCC GSMTask thread for processing. Assuming everything is OK, it parses the SDP and creates the remote media stream which are bubbled up to the DOM Thread. The ICE candidated in the answer SDP are parsed and passed to the STS thread.  Next the answer SDP is negotiated with the internal local SDP used in the earlier offer. At this point a Connected event is returned to CCAPP_Task thread and then to the PC thread where it gets translated to a JSEP ReadyState of Active. Once ICE and DTLS handshaking are complete, a message is sent up to the GSMTask, which can then start sending and receiving media on the working mtransport that was initilized in CreateOffer earlier.
 
=== Signaling System: localDescription ===
 
Here are two options for implementing this API.
 
(1). This like the other JSEP API's will be a function on the call object. It will send feature CC_FEATURE_LOCALDESC to the CCAPP thread which will send the message CC_MSG_SETLOCALDESC to GSMTask. The event handler in GSM will return the local SDP string stored with that call's DCB (Dial Control Block).  AddStream and CreateOffer or CreateAnswer will need to have been called for this to work successfully. This approach is asynchronous so the SDP will need to be returned similarly to how it is returned for the other JSEP API's, via the UI interface and back through the CCAPP thread.
 
(2). Another approach would be to not use the the SIPCC APIs and to store the SDP returned from CreateOffer or CreateAnswer in a string in the PeerConnection Interface backend code.  This would allow this API to be asynchronous and highly performant. One downside would be that if AddStream or RemoveStream is called the SDP we now store in the PC would not get dynamically updated.
 
=== Signaling System: remoteDescription ===
 
Much the same applies to this API as localDescription and also the two possible approaches to implementing it.
 
remoteDesctiption would require setRemoteDescription to be called first, that is how the remote SDP is set into the PeerConnection.
 
 
=== Signaling System: AddStream ===
 
There are probably many ways to implement the AddStream API but I present this one for discussion.
 
AddStream would be an API on the PeerConnection backend interface that takes as a parameter a MediaStream pointer.
 
When called the MediaStreams are stored in a container in the PeerConnection backend. When CreateOffer or CreateAnswer are called to generate the local SDP then the GSMTask thread that is generating the SDP will interrogate the MediaStream container and assemble the media lines based on information it reads from the media streams.


=== Thread diagram for transport subsystem ===
=== Thread diagram for transport subsystem ===
Line 223: Line 287:


Note that real care will need to be taken to make sure that the lifetime of objects shared across threads is right. We should be using nsRefPtr with thread-safe ref count objects to assist this process.
Note that real care will need to be taken to make sure that the lifetime of objects shared across threads is right. We should be using nsRefPtr with thread-safe ref count objects to assist this process.
== Open Issues ==
* Do we have one set of worker threads (PC, Media In, Media Out) per call? One per origin? One total?
* Who is responsible for cleaning up the various objects? My thought is that SIPCC creates and deletes most things, but that makes using ref counted pointers a bit harder (Though not impossible).
* Are the media input and output streams separate or distinct?
* Where is DTLS processing (which is interlocked with the DTLS state machine) done? It's easiest to do this on STS as is ordinarily done with SSL/TLS. Is this too expensive. Note that this is only for Data Channels.
* I have mostly elided the difference between various SIPCC threads (Calling them all CCAPP). Should we really have inner threads talking to mtransport and PC or should we bounce signals lal the way back to CCAPP?
Nohlmeier
Confirmed users
147

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/434453...1175188"