Media/WebRTC/Architecture: Difference between revisions

ISTM that the natural design is to have PeerConnection be 1:1 with CC_Call, so that when you do CreateOffer, it's effectively kicking off the offer process on an existing call. Then we can reuse the SIPCC state machine to some extent to manage the call state. Subsequent calls to the other JSEP API's such as setRemoteDescription and localDescription will run on the same call and use the same state machine.

The mtransport (ICE, DTLS) subsystem is pretty independent of SIPCC. We have two main options for wiring them up:

* Have Webrtc{Audio,Video}Provider instantiate the mtransport objects and manage them. This is roughly the way things are constructed now, but we would need to modify the existing code to talk to mtransport and to be prepared to receive and handle ICE candidate updates (both internally generated and received from the JS).  

* Have the PeerConnection set up the mtransport objects, manage ICE and DTLS, and then provide handles downward to SIPCC.  The major obstacle here is that we somehow have to figure out how to get this data into and out of the SDP. The advantage is that it avoids tight coupling in WebRTC{Audio,VideoProvider}.

I was originally leaning towards the second of these approaches, but now that I have thought about it a while, I think the first is probably better, and that's what's shown in the diagram above.

needs to loop around <tt>cprGetMessage()</tt>, as in <tt>ccap_task.c</tt>:

http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENyZWF0ZU9mZmVyKQpwYXJ0aWNpcGFudCAiRE9NACAHIiBhcyBET00AEw1QQyBhcyBQQwAnDkNDQVBQVGFzawAvBQAJBQBJDUdTTVRhc2sgYXMgR1MAQw5TVFMKCkRPTSAtPiBQQzogRGlzcGF0Y2gAgRUOUEMgLT4AUwY6IElQQyhDQ19DYWxsRmVhdHVyZV8AgUMNAIEJBSAtPiBHU00AJQlNU0dfQ1JFQVRFT0ZGRVIpCkdTAG8FAB8FZnNtZGVmX2V2X2MAghQFb2ZmZXIAFQ1nc21zZHAAGAdfbG9jYWxfc2RwABAUZW5jb2RlX3NkcF9hbmRfdXBkYXRlX3ZlcnNpb24Aag0AZwYgd2VicnRjLm9yZyBmbG93cwAQFG10cmFuc3BvcnQAFg5TVFMAgikLABwKW0lDRV0tPkNvbm5lY3QoKSkKU1RTAIIHDQCBPwUgSUNFIGNhbmRpZGF0ZXMAgg0JAIJfCwCCKgUgU0RQAIJOCwCDHA1vbkNhbGxFdmVudAAiCwCDLAhET00Ag0oLAIRtBQBHBgpub3RlIGxlZnQgb2YAgyUGU3Vic2VxdWVudACBEwV0cmlja2xpbmcAHQ4ATQVKUyBjYWxsYmFja1xud2l0aCAAgy4GAIFDJACBUhRDABMKAIFOGElDRQCBXwYAIgoAgU8WAF4PAIEZJACCegkKCg&s=default

http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlcjogU2V0TG9jYWxEZXNjcmlwdGlvbikKCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHxVQQyAtPgBiBjogSVBDKEZFQVRVUkVfU0VUTE9DQUxFU0MpCgCBEgUgLT4gR1NNACEGQ0NfTVNHAB0JRAAiBQoKR1MAcwUAIgVDb21wYXJlIFNEUCB0byBPZmZlciBTRFAAHQkAbwcAgRcKRFAgIT0AIgc_IFN0YXRlPUVycm9yAHcLAIFFDgAXDACBRQZET00ACRgAXxoAZQVDYWxsIFNlbnQpIABSHQAeCgBYG05lZ290aWF0aW5nKQoK&s=default

http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlZTogU2V0UmVtb3RlRGVzY3JpcHRpb24pCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHhZQQyAtPgBjBjogSVBDKEZFQVRVUkVfU0VUUkVNT1RFU0MpCgCBEwUgLT4gR1NNACEGQ0NfTVNHABwKRAAjBQoKR1MAdQUAIwVnc21zZHBfbmVnb3RpYXRlX29mZmVyX3NkcAAfCFNUUwCBFwtJQ0UgQ2FuZGlkYXRlcykARAkAgRcHAIFACnRhdGU9QWxlcnRpbmcpIACBFQoAgWMOABkOAIFkB0RPTQA6EU4AgRsHACIFCgo&s=default

[Question: Enda. I have not shown the interaction where the remote media streams are created, I am not sure what GSMTask talks to here.]

http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENyZWF0ZUFuc3dlcikKcGFydGljaXBhbnQgIkRPTQAhByIgYXMgRE9NABMNUEMgYXMgUEMAJw5DQ0FQUFRhc2sALwUACQUASQ1HU01UYXNrIGFzIEdTAEMOU1RTCgpET00gLT4gUEM6IERpc3BhdGNoAIEVD1BDIC0-AFQGOiBJUEMoQ0NfQ2FsbEZlYXR1cmVfAIFEDgCBCwUgLT4gR1NNACYJTVNHX0NSRUFURUFOU1dFUikKR1MAcgUAIAVmc21kZWZfZXZfYwCCGAVhAIIYBQAWDWdlbmVyYXRlX2xvY2FsX3NkcAARDnNtc2RwX25lZ290aWF0ZV9vZmZlcgAPGGVuY29kZV9zZHBfYW5kX3VwZGF0ZV92ZXJzaW9uAIESCFNUUwCCBwttdHJhbnNwb3J0W0lDRV0tPkNvbm5lY3QoKSkKU1RTAIFjDQCBHgUgSUNFIGNhbmRpZGF0ZXMAgWgJAII8CwCCBQYgU0RQAIIrCwCCew1vbkNhbGxFdmVudAAiDACDCwhET00AgyoLAIRNBgBJBgpub3RlIGxlZnQgb2YAgwQGU3Vic2VxdWVudACBFgV0cmlja2xpbmcAHQ4ATgVKUyBjYWxsYmFja1xud2l0aCAAgwsHAIFHJACBVhRDABMKAIFRGElDRQCBYgYAIgoAgVEWAF4PAIEaJACCfgk&s=default

The JS calls CreateAnswer passing in the offer SDP to create the answer SDP.  This calls down to the GSMTask thread passing in the offer SDP. The local SDP wil now get created, for this to occur local streams must have been added by the JS API AddStream. If the local SDP is successfully created then the offer SDP will be negotiated, the output of this will be the answer SDP. If there are ICE candidates to be added to this answer SDP they will get added. This answer SDP will then get dispatched back to the DOM thread.

http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlZTogU2V0TG9jYWxEZXNjcmlwdGlvbikKCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHxVQQyAtPgBiBjogSVBDKEZFQVRVUkVfU0VUTE9DQUxFU0MpCgCBEgUgLT4gR1NNACEGQ0NfTVNHAB0JRAAiBQoKR1MAcwUAIgVDb21wYXJlIFNEUCB0byBBbnN3ZXIgU0RQAB4JAHAHAIEYCnRhdGU9RXJyb3IgPwAzBSE9ADEHKSAAewoAgUgOACoKAIFGCERPTQAJGACBFA1jcmVhdGUgd2VicnRjLm9yZyBmbG93cwAQFG10cmFuc3BvcnQAHQcAgSUeQ29ubmVjdGVkAIEYHwAeCgCBIBtBY3RpdmUpCgpub3RlIHJpZ2h0IG9mIFNUUzogSUNFL0RUTFMgY29tcGxldGUKU1RTAIMcDU1lZGlhIACBNQpyZWFkeSkARgZvdmVyAINNBlN0YXJ0IHNlbmRpbmcvcmVjZWl2aW5nIG1lZGlhCg&s=default

Invoked by the JS SetLocalDescription must be called after CreateAnswer and takes as input the answer SDP which is passed down to the GSMTask thread.  Here the SDP is compared to the insernal answer SDP, if there is a difference an error is reported back to the DOM Thread.  If there are no errors then the webrtc.org streams can be created along with the mtransport flows. Following this if ICE completes processing successfuly then the media can start sending and receiving.

[Question: Enda.  I am not sure what the interaction looks like for creating the webrtc.org flows and the mtransport flows.]

=== Signaling System: SetRemote(Caller) ===

http://www.websequencediagrams.com/?lz=dGl0bGUgU2lnbmFsaW5nIFRocmVhZHMgKENhbGxlcjogU2V0UmVtb3RlRGVzY3JpcHRpb24pCnBhcnRpY2lwYW50ICJET00AMQciIGFzIERPTQATDVBDIGFzIFBDACcOQ0NBUFBfVGFzawAwBQAKBQBKDUdTTVRhc2sgYXMgR1MARA5TVFMgYXMgU1RTCgpET00gLT4gUEM6IERpc3BhdGNoKACBHhZQQyAtPgBjBjogSVBDKEZFQVRVUkVfU0VUUkVNT1RFU0MpCgCBEwUgLT4gR1NNACEGQ0NfTVNHABwKRAAjBQpHUwB0BQAiBWNyZWF0ZV9yAIIkBV9TdHJlYW1zABoJAGwHAIEWCU1lZGlhAB8GAGULAIE1DQAWDQCBMwZET00ALBgAgQQIU1RTAIF7C0lDRSBDYW5kaWRhdGVzAIEjD2dzbXNkcF9uZWdvdGlhdGVfYW5zd2VyX3NkcACBRxMgd2VicnRjLm9yZyBmbG93cwAQFG10cmFuc3BvcnQAHQcAgWgYU3RhdGU9Q29ubmVjdGVkKSAAgW4XABgQAIF0FQBHBkFjdGl2ZSkKCm5vdGUgcmlnaHQgb2YAggsGSUNFL0RUTFMgY29tcGxldGUKU1RTAINVDQCDDAUgAIE1CnJlYWR5KQBGBm92ZXIAhAYGU3RhcnQgc2VuZGluZy9yZWNlaXZpbmcgbWVkaWEKCg&s=default

The above diagram shows the caller's SetRemote sequence.

The process starts with receiving the remote description and the JS calling SetRemoteDescription(). This is Dispatched() onto the PC thread and then to CCAPP_Task API thread which passes it to the Internal SIPCC GSMTask thread for processing. Assuming everything is OK, it parses the SDP and creates the remote media stream which are bubbled up to the DOM Thread. The ICE candidated in the answer SDP are parsed and passed to the STS thread.  Next the answer SDP is negotiated with the internal local SDP used in the earlier offer. The webrtc.org and mtransport flows are created. At this point a Connected event is returned to CCAPP_Task thread and then to the PC thread where it gets translated to a JSEP ReadyState of Active. Once ICE and DTLS handshaking are complete, a message is sent up to the GSMTask, which can then start sending and receiving media on the working mtransport.