JoystickAPI: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
(Add a redirect to GamepadAPI page)
 
Line 1: Line 1:
== Web Joystick API (Draft Recommendation) ==
#REDIRECT [[GamepadAPI]]
 
=== Abstract ===
 
The HTML5 specification introduces many of the necessary components for rich interactive and game development, from <canvas> and WebGL to <audio> and <video>.  At the same time, JavaScript implementations have matured to the point where they can now support many tasks which previously required native code.  The Web Joystick API presents a new way for web and game developers, as well as interaction designers, to access and use joysticks and other controllers for games.
 
NOTE: this API is experimental, and is not part of Firefox or HTML5.
 
=== Authors ===
 
* Ted Mielczarek ([https://twitter.com/#!/TedMielczarek @TedMielczarek])
* Jason Orendorff ([https://twitter.com/#!/jorendorff @jorendorff])
* David Humphrey ([http://twitter.com/humphd @humphd])
* Kevin Gadd ([https://twitter.com/#!/antumbral @antumbral])
 
== API Tutorial ==
 
This API introduces new events on the window object for reading joystick and controller (hereby referred to as ''joystick'') state.  In addition to these events, the API also adds a Joystick object, which can be used to query the state of connected joysticks.
 
=== Connecting to a Joystick ===
 
When a new joystick is connected to the computer, the focused page will first receive a '''MozJoyConnected''' event.  If a joystick was already connected when the page loaded, the '''MozJoyConnected''' event will be dispatched to the focused page when the user presses a button or moves an axis.  Developers can use '''MozJoyConnected''' like this:
 
<pre>
&lt;div id="joysticks"&gt;&lt;/div&gt;
&lt;script&gt;
function joystickConnected(e) {
  var joysticks = document.getElementById("joysticks"),
    joystickId = e.joystick.id;
 
  joysticks.innerHTML += " Joystick Connected (id=" + joystickId + ")";
}
 
window.addEventListener("MozJoyConnected", joystickConnected, false);
&lt;/script&gt;
</pre>
 
Each joystick will have a unique ID associated with it, which is available on the event's '''joystick''' property.
 
=== Disconnecting a Joystick ===
 
When a joystick is disconnected, and if a page has previously received data (e.g., MozJoyConnected), a second event is dispatched to the focused window, '''MozJoyDisconnected''':
 
<pre>
&lt;div id="joysticks"&gt;&lt;/div&gt;
&lt;script&gt;
function joystickDisconnected(e) {
  var joysticks = document.getElementById("joysticks"),
    joystickId = e.joystick.id;
 
  joysticks.innerHTML += " Joystick Disconnected (id=" + joystickId + ")";
}
 
window.addEventListener("MozJoyDisconnected", joystickDisconnected, false);
&lt;/script&gt;
</pre>
 
The Joystick's ID will be the same for '''MozJoyConnected''' and '''MozJoyDisconnected''' events, making it a suitable key for storing joystick device information:
 
<pre>
var joysticks = {};
 
function joystickHandler(event, connecting) {
  var joystick = event.joystick;
 
  if (connecting) {
    joysticks[joystick.id] = joystick;
  } else {
    delete joysticks[joystick.id];
  }
}
 
window.addEventListener("MozJoyDisconnected", function(e) { joystickHandler(e, true); }, false);
window.addEventListener("MozJoyDisconnected", function(e) { joystickHandler(e, false); }, false);
</pre>
 
This previous example also demonstrates how the '''joystick''' property can be held after the event has completed--a technique we will use for device state querying later.
 
=== Joystick Button Events ===
 
Joysticks can have one or more buttons, and similar to a mouse button, this API provides events for buttons being pressed and released.  When a joystick button is pressed a '''MozJoyButtonDown''' event is dispatched to the currently focused page.  Similarly a '''MozJoyButtonUp''' event is dispatched when it is released.  Both events provide the same '''joystick''' property, which indicates the joystick (i.e., it's ID) that triggered the event.  The button itself (i.e., whichever of the 2, 4, etc. buttons the joystick has) is available on the event's '''button''' property.
 
<pre>
&lt;div id="joysticks"&gt;&lt;/div&gt;
&lt;script&gt;
function buttonHandler(event, pressed) {
  var joysticks = document.getElementById("joysticks"),
    joystickId = event.joystick.id,
    button = event.button,
    text = pressed ? " Joystick button pressed" : " Joystick button released";
 
  joysticks.innerHTML += text + " (id=" + joystickId + ", button=" + button + )";
}
 
window.addEventListener("MozJoyButtonDown", function(e) { buttonHandler(e, true); }, false);
window.addEventListener("MozJoyButtonUp", function(e) { buttonHandler(e, false); }, false);
&lt;/script&gt;
</pre>
 
=== Joystick Axis Events ===
 
Similar to the button events, the axis events provide a way for developers to know when a user has moved one or more of a joystick's axises (i.e., left-right or up-down).  Just as a joystick can have multiple buttons, so to can there be many axes.  Each physical stick on a joystick provides two axes for changes in X and Y positioning.  The '''MozJoyAxisMove''' event indicates that a joystick's axis has changed, and its new value.  The axis is numbered, and its value is given, which is a float between -1.0 (the lowest possible value) and 1.0 (the highest possible value):
 
<pre>
&lt;div id="joysticks"&gt;&lt;/div&gt;
&lt;script&gt;
function axisHandler(event, pressed) {
  var joysticks = document.getElementById("joysticks"),
    joystickId = event.joystick.id,
    axis = event.axis,
    value = event.value;
 
  joysticks.innerHTML += " Joystick Axis Move (id=" + joystickId +
                                              ", axis=" + axis +
                                              ", value=" + value + ")";
}
 
window.addEventListener("MozJoyAxisMove", axisHandler, false);
&lt;/script&gt;
</pre>
 
=== Querying the Joystick Object ===
 
All of the '''MozJoy*''' events discussed above included a '''joystick''' property on the event object.  We used this in order to determine which joystick (i.e., it's ID) had caused the event, since multiple joysticks might be connected at once.
 
We can do much more with this Joystick object, including holding a reference to it and querying it instead of using '''MozJoyButtonUp''', '''MozJoyButtonDown''', and '''MozJoyAxisMove''' events.  Doing so is often desirable for games or other interactive web pages that need to know the state of a joystick now vs. the next time an event fires.
 
As we have seen, the Joystick object for a given joystick is available on the '''MozJoyConnected''' event.  For security reasons, it is not available as a property of the window object itself.  Once we have a reference to it, we can query its properties for information about the current state of the joystick.  Behind the scenes, this object will be updated every time the joystick's state changes.
 
The Joystick object's properties include:
 
* '''id''': a unique id (string) for the joystick.
* '''connected''': true if the joystick is still connected to the system.
* '''buttons''': an array of the buttons present on the device. Each entry in the array is 0 if the button is not pressed, and non-zero if the button is pressed.
* '''axes''': an array of the axes present on the device. Each entry in the array is a float value in the range -1.0..1.0 representing the axis position from the lowest value (-1.0) to the highest value (1.0).
 
The Joystick object is often used in conjunction with an animation loop (e.g., requestAnimationFrame), where developers want to make decisions for the current frame based on the state of the joystick or joysticks.
 
=== Complete Example: Displaying Joystick State ===
 
This example shows how to use the Joystick object, as well as the '''MozJoyConnected''' and '''MozJoyDisconnected''' events in order to display the state of all joysticks connected to the system:
 
<pre>
&lt;html&gt;
&lt;head&gt;
&lt;script type="text/javascript"&gt;
var joys = {};
 
function createDiv(joystick) {
  var d = document.createElement("div");
  d.setAttribute("id", joystick.id);
  var t = document.createElement("h1");
  t.appendChild(document.createTextNode("joystick: " + joystick.id));
  d.appendChild(t);
  var b = document.createElement("div");
  b.className = "buttons";
  for (var i=0; i&lt;joystick.buttons.length; i++) {
    var e = document.createElement("span");
    e.className = "button";
    e.innerHTML = i;
    b.appendChild(e);
  }
  d.appendChild(b);
  var a = document.createElement("div");
  a.className = "axes";
  for (var i=0; i&lt;joystick.axes.length; i++) {
    var e = document.createElement("progress");
    e.className = "axis";
    e.setAttribute("max", "2");
    e.setAttribute("value", "1");
    e.innerHTML = i;
    a.appendChild(e);
  }
  d.appendChild(a);
  document.getElementById("start").style.display = "none";
 
  return d;
}
 
function connectHandler(e) {
  var joy = e.joystick;
  joys[e.joystick.id] = joy;
 
  var div = createDiv(joy);
  document.body.appendChild(div);
 
  window.mozRequestAnimationFrame(updateStatus);
}
 
function disconnectHandler(e) {
  var d = document.getElementById(e.joystick.id);
  document.body.removeChild(d);
  delete joys[e.joystick.id];
}
 
function updateStatus() {
  for (j in joys) {
    var joy = joys[j],
      d = document.getElementById(j),
      buttons = d.getElementsByClassName("button"),
      axes = d.getElementsByClassName("axis");
 
    for (var i=0; i&lt;joy.buttons.length; i++) {
      var b = buttons[i];
      if (joy.buttons[i]) {
        b.className = "button pressed";
      }
      else {
        b.className = "button";
      }
    }
 
    for (var i=0; i&lt;joy.axes.length; i++) {
      var a = axes[i];
      a.innerHTML = i + ": " + joy.axes[i].toFixed(4);
      a.setAttribute("value", joy.axes[i] + 1);
    }
  }
 
  window.mozRequestAnimationFrame(updateStatus);
}
 
window.addEventListener("MozJoyConnected", connectHandler);
window.addEventListener("MozJoyDisconnected", disconnectHandler);
&lt;/script&gt;
&lt;style&gt;
.buttons, .axes {
  padding: 1em;
}
 
.button {
  padding: 1em;
  border-radius: 20px;
  border: 1px solid black;
}
 
.pressed {
  background-color: green;
}
&lt;/style&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;h2 id="start"&gt;Press a button on your controller to start&lt;/h2&gt;
&lt;/body&gt;
&lt;/html&gt;
</pre>
 
==DOM Implementation==
 
===nsIDOMJoystick===
 
All '''MozJoy*''' events include a '''joystick''' property, which contains the state of the joystick.  This so-called Joystick object is only available via '''MozJoy*''' events, and is not available on window.  It has the following interface:
 
<pre>
interface nsIDOMJoystick : nsISupports {
  readonly attribute DOMString id;
  readonly attribute boolean connected;
  readonly attribute nsIVariant buttons;
  readonly attribute nsIVariant axes;
};
</pre>
 
The '''id''' attribute is a string containing the USB vendor and product ID as well as a name.  This string is unique per device type.  Two joysticks of the same type will share the same string.
 
The '''connected''' attributed indicates whether or not the joystick is connected to the system.  After a '''MozJoyConnected''' event, joystick.connected will be true.  It is false once '''MozJoyDisconnected''' fires.
 
The '''buttons''' attribute is an array of all buttons on the joystick and their state.  The first button (e.g., buttons[0]) will be 0 (zero) if the button is not pressed, and non-zero if it is.  The second button is buttons[1] and so on.
 
The '''axe''' attribute is an array of all the axes present on the device. Each entry in the array is a float value in the range -1.0 to 1.0, representing the axis position from the lowest value (-1.0) to the highest value (1.0).  Each physical stick on a joystick will expose two axes, one for changes in X (left-right), and the other for changes in Y (up-down) positioning.
 
The lifetime of an nsIDOMJoystick object is more than the lifetime of a '''MozJoy*''' event.  Web content may save a reference to a '''MozJoy*''' event's '''joystick''' property and refer to it at any time to determine the current state of the device.  The state will not be updated while the content script is executing, only between script executions.  For example, the state will not be updated during the course of a setTimeout callback, but it may be updated the next time the callback is called.
 
===nsIDOMJoystickConnectionEvent===
 
The '''MozJoyConnected''' and '''MozJoyDisconnected''' events use nsIDOMJoystickConnectionEvent:
 
<pre>
interface nsIDOMJoystickConnectionEvent : nsIDOMEvent
{
  readonly attribute nsIDOMJoystick joystick;
};
</pre>
 
The single '''joystick''' attribute provides access to an associated nsIDOMJoystick object for this device.  As was previously mentioned, this object can be held after the event callback has completed, and be used to query the state of the device at any point.
 
===nsIDOMJoystickButtonEvent===
 
Joystick button state information is made available via nsIDOMJoystickButtonEvent:
 
<pre>
interface nsIDOMJoystickButtonEvent : nsIDOMEvent
{
  readonly attribute unsigned long button;
  readonly attribute nsIDOMJoystick joystick;
};
</pre>
 
The '''button''' attribute indicates the index of the button that was pressed or released.
 
The '''joystick''' attribute provides access to the associated nsIDOMJoystick object, with full button state information available via '''joystick.buttons'''.
 
===nsIDOMJoystickAxisMoveEvent===
 
Joystick axes state information is made available via nsIDOMJoystickAxisMoveEvent:
 
<pre>
interface nsIDOMJoystickAxisMoveEvent : nsIDOMEvent
{
  readonly attribute unsigned long axis;
  readonly attribute float value;
  readonly attribute nsIDOMJoystick joystick;
};
</pre>
 
The '''axis''' attribute indicates the index of the axis that was moved.
 
The '''value''' attribute is a float indicating the position of the axis, between -1.0 and 1.0.
 
The '''joystick''' attribute provides access to the associated nsIDOMJoystick object, with full axes state information available via '''joystick.axes'''.
 
==Resources==
 
Implementation and discussion on this API is happening in {{bug|604039}}.  Feedback and suggestions are welcome.
 
===Obtaining Builds===
 
The latest information about custom builds for testing the API is in {{bug|604039}}.  However, the most recent try-server builds are available at:
 
http://people.mozilla.com/~tmielczarek/gamepad/
 
===Demos, Libraries, Other Code===
 
Examples of the Web Joystick API should be added here.  NOTE, the API is evolving, and some of the following examples may be based on earlier versions.
 
* Simple API demo page - https://bug604039.bugzilla.mozilla.org/attachment.cgi?id=549214
* Another API demo page - https://bug604039.bugzilla.mozilla.org/attachment.cgi?id=550638
* Example of using MozJoy* listeners - https://gist.github.com/58b0f9ed647f72da9173
* Joystick API controlling &lt;video&gt; - http://weblog.bocoup.com/javascript-firefox-nightly-introduces-dom-joystick-events, http://jsfiddle.net/rwaldron/wPuGB/
 
===Design Considerations===
 
* Should feel web-native.
** DOM Events were chosen for the first prototype implementation because of the similarity to keyboard/mouse input.
** Whatever API is chosen, it should use standard web idioms.
* Should not expose the user to unnecessary fingerprinting.
** Web pages should not be able to query for the number and type of attached devices without explicit user input.
** Ideally we wouldn't have to use a permission infobar like with some other APIs, the user explicitly interacting with their device while viewing a webpage ought to be enough "permission" to send device data to that page.
** The prototype implementation falls down a bit here, it sends events to all pages that ask for them, should really be just the foreground page.
* Needs to be performant for web games that want to run at 60FPS
** This will need experimentation and testing
** There are assertions that DOM events won't scale to this level, it's possible that the API will need to be changed if that is true
* To be useful, will need some way to map buttons/axes to meaningful values.
** Even the same gamepad looks different on different OSes!
** Across multiple devices this is insanity
** Should try to expose enough info to let content handle this, Kevin Gadd has an "Input Device API" that is higher-level and might be a good fit here.
 
===Discussion===
 
* Pages will want instantaneous state of device, we might as well expose it (may alleviate polling concerns as well)
** Expose a joyState property on each event, content can hold on to that object and it will update live as input happens
* Not sure about mapping various devices to a sane set of controls
** Going to try exposing a device ID for testing, collect some data and see what it looks like
** Probably a fingerprinting concern, may not want to ship this
** How much do we push to content vs. try to handle in the browser?
** We'll get new try builds and put up a web page to gather data from users with various devices
* Need timestamps per event (using milliseconds like requestAnimationFrame)
* sicking is going to schedule a security review to get some more people thinking about it
* Proposal for a polling based API (soliciting feedback): https://sites.google.com/a/chromium.org/dev/developers/joystick-api

Latest revision as of 18:07, 15 November 2011

Redirect to:

Retrieved from "https://wiki.allizom.org/index.php?title=JoystickAPI&oldid=369677"