1,295
edits
MediaStreamAPI: Difference between revisions
→Stream mixing and processing
| Line 106: | Line 106: | ||
[Constructor] | [Constructor] | ||
interface StreamProcessor | interface StreamProcessor : Stream { | ||
readonly attribute Stream[] inputs; | readonly attribute Stream[] inputs; | ||
void addStream(Stream input); | void addStream(Stream input); | ||
| Line 125: | Line 125: | ||
While 'worker' is null, the output is produced simply by adding the streams together. Video frames are composited with the last-added stream on top, everything letterboxed to the size of the last-added stream that has video. While there is no input stream, the StreamProcessor produces silence and no video. | While 'worker' is null, the output is produced simply by adding the streams together. Video frames are composited with the last-added stream on top, everything letterboxed to the size of the last-added stream that has video. While there is no input stream, the StreamProcessor produces silence and no video. | ||
While 'worker' is non-null, the results of mixing (or the default silence) are fed into the worker by dispatching onstream callbacks. Each onstream callback takes a StreamEvent as a parameter. A StreamEvent provides audio sample buffers and a list of video frames for each input stream; the event callback can write audio output buffers and a list of output video frames. If the callback does not output audio, default audio output is automatically generated as above; ditto for video. Each StreamEvent contains the inputParams for each input stream contributing to the StreamEvent. | While 'worker' is non-null, the results of mixing (or the default silence) are fed into the worker by dispatching onstream callbacks. Each onstream callback takes a StreamEvent as a parameter. A StreamEvent provides audio sample buffers and a list of video frames for each input stream; the event callback can write audio output buffers and a list of output video frames. If the callback does not output audio, default audio output is automatically generated as above; ditto for video. Each StreamEvent contains the inputParams for each input stream contributing to the StreamEvent. | ||
Note that 'worker' cannot be a SharedWorker. This ensures that the worker can run in the same process as the page in multiprocess browsers, so media streams can be confined to a single process. | |||
An ended stream is treated as producing silence and no video. (Alternative: automatically remove the stream as an input. But this might confuse scripts.) | An ended stream is treated as producing silence and no video. (Alternative: automatically remove the stream as an input. But this might confuse scripts.) | ||
// XXX need to figure out the actual StreamEvent API: channel formats, etc. | // XXX need to figure out the actual StreamEvent API: channel formats, etc. | ||
==== Graph cycles ==== | ==== Graph cycles ==== | ||