MozillaWiki

Media/getUserMedia: Difference between revisions

< Media
(Add note on FF32-37)
 
(23 intermediate revisions by 5 users not shown)
Line 1: Line 1:
= getUserMedia Implementation Roadmap =
= getUserMedia =


== Goal ==
== Goal ==
* Implement the getUserMedia specification on 3 platforms: Desktop, Android and B2G.
* Define and implement navigator.mediaDevices.getUserMedia() and associated functions with the W3C's WebRTC WG and Media Capture Task Force
* Spec is at Candidate Recommendation. Current editor's draft: [http://dev.w3.org/2011/webrtc/editor/getusermedia.html W3 getUserMedia Editor's Draft].
* Scenarios: [https://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/scenarios.html W3 MediaStream Capture Scenarios]


* Includes the following uses of getUserMedia:
== Status ==
<pre>
* Implemented getUserMedia on Desktop (Windows, Mac and Linux), Android, and B2G. Enabled in FF20+.
  getUserMedia({picture:true}, onsuccess(Blob blob), onerror);
* Implemented new and old API as of FF36:
  getUserMedia({video:true,audio:true}, onsuccess(MediaStream stream), onerror);
** <b><tt>navigator.mediaDevices.getUserMedia</tt></b> - modern promise-returning version (unprefixed).
  video.src = stream;
** <b><tt>navigator.mozGetUserMedia()</tt></b> - legacy backwards-compatible version (prefixed).
  audio.src = stream;
* Capture resolution can be controlled with standard constraints for width, height, frameRate and (on mobile) facingMode can be used to choose between front/back camera. Supported in FF32+.
</pre>
* Promise-based mediaDevices.getUserMedia API is supported in FF36+.
* Full constraints syntax with plain values and ideal-algorithm supported in FF38+.
** Note on FF32-37: Plain values and ideal are not supported. However, values are not mandatory unless you add a non-spec [http://jsfiddle.net/34qxx5w1 require] keyword.
* Input device enumeration through mediaDevices.enumerateDevices() is supported in FF39+.
* Can be turned off by setting <b><tt>media.navigator.enabled</tt></b> and <tt><b> media.peerconnection.enabled</b></tt> to false.
* Permissions UI is supported, allowing selecting devices or rejecting permission, with notifications that a tab is using a mic or camera, and a global indicator on the desktop to locate tabs using mics/cameras.
* Persistent permissions are supported.
* Explicit revocation of permission is supported.
* Supports multiple tabs getting data from the same mic or camera (if the user allows)
* Audio rate is fixed at 16000Hz.
* Echo cancellation is on by default, but connecting a captured audio stream directly to an audio element on the same system may still cause feedback unless a headset is used.
* Limited screen-sharing support under a pref.


* Appropriate permission, notification and status UIs.
== Examples ==


== Phase 1 ==
* See [https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia mediaDevices.getUserMedia on developer.mozilla.org] for examples.


* Implement image capture {picture:true} API
== Demos ==
** Android: {{bug|738528}} [Anant]
** B2G: {{bug|749886}} [Fabrice]


* Define common abstraction "MediaEngine" for device access across multiple platforms.
* See [http://mozilla.github.io/webrtc-landing  WebRTC demos including mozGetUserMedia]
** Cross-platform: {{bug|750943}} [Suhas/Anant]
*** (crypt's version in {{bug|739566}})


* Define "chrome" extensions to MediaStreams for privileged JS code (and B2G's) use. This includes CameraControl features (https://wiki.mozilla.org/WebAPI/CameraControl).
== Differences from Chrome webkitGetUserMedia() ==
** Cross-platform: {{bug|752352}} [Fabrice/Anant?]
Includes, but not limited to:
* Name :-)
* Firefox does not support Chrome's outdated constraints syntax ('mandatory', 'minWidth' etc.)
* Firefox does not support MediaStreamTrack.getSources(), an outdated version of enumerateDevices.
* Firefox leaves the mic/camera active (light on, etc) until the application explicitly calls mediastream.stop().  Chrome turns them on when assigned to a media element or PeerConnection, and off again when removed.
** Note that if an application drops all references to a MediaStream but does not call stop(), the camera will remain active for some period of time (or until you navigate to another page in the tab or close the tab).
** This behavior may change in the future
* Both now support window.URL.createObjectURL(stream), though Mozilla recommends against using it unless you have to.  We expect Chrome to support srcObject = stream soon (perhaps prefixed as we are currently)
* Use the [https://github.com/webrtc/adapter adapter.js] shim to bridge cross-browser differences (constraints polyfill should be forthcoming).


* Implement a fallback backend for MediaEngine (when hardware support not available, returnMediaStream with white noise, for example).
== Todo ==
** Cross-platform: {{bug|752351}} [???]


* Write DOM bindings for getUserMedia (backed with Fallback MediaEngine).
* mediaDevices.getSupportedConstraints().
** Cross-platform: {{bug|752353}} [Anant]
* More constraints to implement, like deviceId.
*** (split code from {{bug|691234}} into this one)
 
== Phase 2 ==
* Implement MediaEngine backends:
** Android: bug ??? (either based on Android NDK/SDK, or, webrtc.org code).
** B2G: bug ??? [Fabrice] (based on Gonk).
** Desktop: {{bug|691234}} [Anant] (based on webrtc.org code).
 
* Implement image capture {picture:true} API
** Desktop: {{bug|749887}} [???]
*** ({{bug|692955}} tracking <input> on Desktop, reuse possible. {{bug|748835}} tracking UI).
 
== Phase 3 ==
* Implement permissions, notification & status UI
** Android: bug ???
** B2G: bug ???
** Desktop: {{bug|729522}} [???]
 
Once UI lands, getUserMedia may be pref'ed on.
 
== Phase 4 and beyond ==
* Implement file backend for MediaEngine.
* Refactor getUserMedia{picture:true} implementations to use MediaEngine + privileged MediaStreams and CameraControl APIs.

Latest revision as of 22:05, 2 April 2015

getUserMedia

Goal

Status

  • Implemented getUserMedia on Desktop (Windows, Mac and Linux), Android, and B2G. Enabled in FF20+.
  • Implemented new and old API as of FF36:
    • navigator.mediaDevices.getUserMedia - modern promise-returning version (unprefixed).
    • navigator.mozGetUserMedia() - legacy backwards-compatible version (prefixed).
  • Capture resolution can be controlled with standard constraints for width, height, frameRate and (on mobile) facingMode can be used to choose between front/back camera. Supported in FF32+.
  • Promise-based mediaDevices.getUserMedia API is supported in FF36+.
  • Full constraints syntax with plain values and ideal-algorithm supported in FF38+.
    • Note on FF32-37: Plain values and ideal are not supported. However, values are not mandatory unless you add a non-spec require keyword.
  • Input device enumeration through mediaDevices.enumerateDevices() is supported in FF39+.
  • Can be turned off by setting media.navigator.enabled and media.peerconnection.enabled to false.
  • Permissions UI is supported, allowing selecting devices or rejecting permission, with notifications that a tab is using a mic or camera, and a global indicator on the desktop to locate tabs using mics/cameras.
  • Persistent permissions are supported.
  • Explicit revocation of permission is supported.
  • Supports multiple tabs getting data from the same mic or camera (if the user allows)
  • Audio rate is fixed at 16000Hz.
  • Echo cancellation is on by default, but connecting a captured audio stream directly to an audio element on the same system may still cause feedback unless a headset is used.
  • Limited screen-sharing support under a pref.

Examples

Demos

Differences from Chrome webkitGetUserMedia()

Includes, but not limited to:

  • Name :-)
  • Firefox does not support Chrome's outdated constraints syntax ('mandatory', 'minWidth' etc.)
  • Firefox does not support MediaStreamTrack.getSources(), an outdated version of enumerateDevices.
  • Firefox leaves the mic/camera active (light on, etc) until the application explicitly calls mediastream.stop(). Chrome turns them on when assigned to a media element or PeerConnection, and off again when removed.
    • Note that if an application drops all references to a MediaStream but does not call stop(), the camera will remain active for some period of time (or until you navigate to another page in the tab or close the tab).
    • This behavior may change in the future
  • Both now support window.URL.createObjectURL(stream), though Mozilla recommends against using it unless you have to. We expect Chrome to support srcObject = stream soon (perhaps prefixed as we are currently)
  • Use the adapter.js shim to bridge cross-browser differences (constraints polyfill should be forthcoming).

Todo

  • mediaDevices.getSupportedConstraints().
  • More constraints to implement, like deviceId.
Retrieved from "https://wiki.allizom.org/index.php?title=Media/getUserMedia&oldid=1066229"