Talk:MediaStreamAPI

Some basic feedback:

• There's no way for workers to signal latency to the scheduler.

If the worker just writes out timestamped buffers, it can dynamically signal it's latency, and the scheduler can make its own decision about what constitutes and underrun. However, in realtime contexts (conferencing and games) it is helpful to optimize for the lowest possible latency. To assist this, it's helpful if the workers (and internal elements like codecs and playback sinks) can advertise their expected latency. The scheduler can then sum these over the pipeline to determine a more aggressive minimal time separation to maintain between the sources an sinks.

One possible resolution is just to use an aggressive schedule for all realtime streams, and leave it to developers to discover in testing what they can get away with on current systems.