Platform/AreWeFunYet

From MozillaWiki
< Platform
Revision as of 20:51, 27 February 2012 by Bjacob (talk | contribs)
Jump to navigation Jump to search

This page tracks the status of Gecko as a games platform.

Also see HTML5 Games.

Tracking Bug for Game Related Issues: https://bugzilla.mozilla.org/show_bug.cgi?id=gecko-gaming

Fast code

Feedback from Games Work Week

Here is a benchmark comparing ActionScript to several JavaScript engines. It shows that as of June 2011, JavaScript is faster than Flash 10.3 in all major browsers. Here's a runnable copy of the v8 flash benchmark.

Box2D Benchmark - This is a great blog post about some experiments done in Dec 2011 comparing different ways of getting Box2D to run in a browser and seeing it performs relative to the C version.

TODO: Find/generate more benchmarks? At least update that one to include Flash 11 and the latest JS engines (Firefox 9 with TI) ?

Known Issues

Garbage collection pauses

An important question is what's an acceptable length for a GC pause. There is no magic value there that will ensure that we maintain 60 FPS: an arbitrarily short pause might make us skip a frame. But there is an important value that will ensure that we don't skip more than one frame: 1s/60 = 16 ms, so as long as GC pauses are no longer than 16 ms, they don't cause us to skip more than one frame. Thus 16 ms seems like the first goal to aim for, allowing us to maintain an effective 30 FPS when GC occurs.

Incremental GC (bug 641025) is on track to achieve this goal. The current version keeps mark phase pause times down to 10ms. The Larch project branch has the latest work in this direction, and nightly builds (on ftp) show already better results than Nightly. Potential 10ms+ pauses left over after bug 641025:

  • The sweep phase (mostly because of objects with finalizers, e.g., DOM objects) still runs all at once, and can be 20ms+. Incrementalizing this is much easier that incrementalizing the mark phase and is planned follow-on work.
  • We throw away compiled code on GC, requiring recompilation after the GC is done, which can be a pause. We're aware of the issue and have talked a bit about solutions.
  • The JS program can run more slowly during an incremental GC. It depends on what the JS program does: writing to heap locations (e.g., object properties, array elements) is slowed down, so programs that do a lot of that can be slowed down substantially, but other programs are hardly affected at all.

Generational GC (bug 619558), now underway and expected to finish around mid-year, should further alleviate GC issues. With generational GC, short-lived objects are collected with negligible cost, so a regular incremental GC needs to be started only when there is garbage from long-lived objects that are no longer used.

A further improvement would be to provide an API that means "don't GC now". This would combine well with generational GC: with generational GC, major collections don't need to be done very often so delaying them is unlikely to cause a problem.

Impact of other tabs

See bug 710359. In the web-gaming era, it will not be tolerable that just having a GMail tab on the side severely harms FPS in a game in another tab. This should have been solved by electrolysis ( = process separation), but it's on hold. There doesn't seem to be much else to do about that. Also, this isn't JavaScript specific.

Performance issues with large JS files

See bug 644244. Large JavaScript files (which are not uncommon in games) can in some cases run slowly. Some new benchmarks with large files might help here.

Typed Arrays for Managing Assets

It seems that game developers would like to store assets in binary form in Typed Arrays. How possible/feasible is that? For example, can I say 'take this typed array and interpret it as a JPEG image' ? Another orthogonal question: can one cache on disk (IndexedDB?) the contents of a Typed Array?

Currently if you want to take data in a Typed Array and display in an <img> or otherwise end up interacting with the DOM, you'll have to copy the data at least once. One solution might be the ability to create a blob-url representing the contents of a Typed Array.

Compression of JavaScript code

Game developers want to minimize the size of code to deliver it faster to the client. Current mechanisms for that are JS minification, and gzip compression at the level of the HTTP server. Some game developers expressed concerns with that: they wanted better compression algorithms (LZMA) and they wanted something that can be used without knowing how to configure a HTTP server.

Speed of first-run

Game developers are concerned about the time taken to compile large JavaScript codebases. Note that there are two separate compilation stages: bytecode compilation (JS->bytecode), and native compilation (bytecode->native). Currently SpiderMonkey compiles all bytecode immediately when it is loaded by a page, but functions are native-compiled only after they have run a few times (or for a few loop iterations).

We need data to show which compilation(s) are a problem.

If it's bytecode, one solution is to compile the code as it is downloaded (bug on file?). Another is to parse only enough to get early errors, but not actually compile, making the front-end faster.

Native compilation is difficult to make faster: there aren't necessarily any quick fixes.

Ahead-of-time compilation or caching compiled code would address both issues.

Management of large JavaScript codebases

Does JavaScript need to evolve to make large codebases more manageable? Some things that were mentioned:

  • modules (better includes) - ES6 modules should be available around mid-year in Firefox and Chrome. It is planned for Q2 for JS team.
  • namespaces - this is probably taken care of by ES6 modules.
  • add more OO features (classes, inheritance) - not coming soon to the JS language. There are libraries, though, e.g. prototype.js.

Low-level or 3D Graphics

[Feedback from Games Work Week: https://games.etherpad.mozilla.org/8]

The biggest missing feature in WebGL is compressed textures, however there's already a bug for that.

High-level 2D Graphics

Feedback from Games Work Week

Web technology: Canvas 2D. Well, there is also SVG, but somehow it's less popular for game development. Someone should add an explanation of why.

SVG is a retained mode API which means the developer has less control over whats painted when. SVG also requires using the clumsy DOM APIs which means it's harder to get started.

Canvas 2D is a traditional 2D API, modeled after Apple's CoreGraphics, itself modeled after PostScript. Its main advantages over WebGL are ease of use, availability of important 2D features (including text) and ubiquity: it's supported in all modern browsers and is always available, regardless of possible GPU acceleration. Its downsides are that it's more specific to certain kinds of graphics (2D and some casual 3D), offers less low-level control to the programmer, and is relatively hard for the browser run fast on GPUs.

This is key to understanding the discrepancies in Canvas 2D performance across browsers, and within a single browser, across platforms.

Gecko's implementation of Canvas 2D is currently transitioning from an old model to a new one, with the new model already in use on some platforms.

The old model used Cairo as the internal 2D graphics API on all platforms, then dispatched to platform-specific Cairo back-ends. Some were GPU-accelerated, like the Cairo Direct2D back-end on Windows.

The new model uses a new lightweight API called Azure, that carries minimal overhead and maps perfectly well to Direct2D, which was its first backe-nd. More Azure back-ends are being developed as we transition to using Azure everywhere.

The current and near-future status of Gecko for Canvas 2D is summarized in this table:

Platform Implementation Performance comments Version comments
Windows 7 and Vista Azure / Direct2D Very fast GPU-accelerated Since Firefox 7
Windows XP cairo gdi/software rectangular unscaled blitting should be quite fast, transformed blitting is slower than it could be Since Firefox 3
Mac Azure / (CoreGraphics or Skia ???) much improved blitting performance Since Firefox 12
Linux/X11 cairo X11 performance depends the users drivers. This can range from good to bad. Since Firefox 3
Android cairo software rectangular unscaled blitting should be quite fast, transformed blitting is slower than it could be

As always, notice that GPU acceleration is subject to driver blacklisting.

Audio

Audio is an area that doesn't currently have a universally accepted standard.

Miscellaneous:

FullScreen

This feature allows you to build a web application that runs full screen. This includes any HTML element so you can build full screen games, full interactive video experiences, presentation software or anything else that should dominate the experience.

MouseLock

Web technology: Mouse Lock API

The Mouse Lock API is currently under development by David Humphrey and his students at Seneca College. It is also under development in Chrome. This API is currently in W3C Draft Status.

  • This is targeted to land in the first quarter 2012

Relevant links:

Gamepad

Web technology: GamepadAPI.

The Gamepad API is currently under development in Firefox and Chrome. Firefox builds are available for download here. Chromium builds need to have --enable-gamepad (?) passed on start-up. This API is currently in W3C draft status

  • This is targeted to land in the first quarter 2012

Relevant links:

Keyboard input that ignores keyboard layouts

Many games use the WASD keys for direction control. However, depending on the active keyboard layout, their usual (US-layout) key places can be occupied by different letters. In the AZERTY keyboard layout, for example, these keys have the letters ZQSD, and in Dvorak {comma}AOE.

Users of those keyboard layouts shouldn't need to reconfigure their key mapping or change the system's keyboard layout. Instead, the game should be able to get a pressed key's absolute position on the keyboard, or its "scan code". Key events don't provide that information yet.

I'm not aware of any standardization effort on this.

Competition:

  • unknown

Relevant links:

Local file storage

TODO: fill this section

IndexedDB is an API for client-side storage of significant amounts of structured data and for high performance searches on this data using indexes. While DOM Storage is useful for storing smaller amounts of data, it is less useful for storing larger amounts of structured data. IndexedDB provides a solution.

IndexedDB provides separate APIs for synchronous and asynchronous access. The synchronous API is intended to be used inside workers.

WebSockets

WebSockets is a bi-directional connection-oriented reliable-transport protocol designed to give an option to developers for low-latency communications between browsers and servers. It isn't a raw socket protocol for security reasons, and is initiated on HTTP ports via an HTTP upgrade.

You can actually do most of what WebSockets does over normal HTTP with long-hanging gets. However, the overhead of doing that over HTTP means that for small transactions - think key strokes, or a few bytes at a time - can have quite a bit of overhead because of HTTP headers. So WebSockets is really built for applications that require low-latency communications. Think live games or keystroke-based interactive applications.

The low-level WebSockets protocol is RFC 6455 and is fully implemented in Firefox 11. The protocol is TCP-based and can use SSL or operate in the clear. Messages are framed with length and can be either binary or UTF-8. Messages sent by the browser are obfuscated on the wire with a key known to the browser and the server, but is not available to JavaScript. This feature exists to prevent some attacks against transparent proxy servers which may intercept the initial WebSocket connection.

There is a separate WebSockets API maintained at the W3C. (The editor's draft is the best source of information and contains the most recent API. The API is message-based. That is, individual WebSocket frames are sent as messages via an event handler in JavaScript. Messages can be handled as strings, ArrayBuffers or Blobs.

As of Firefox 11 we've unprefixed the WebSockets API. This means that we don't expect the protocol or the API to change.

Asset Management

Feedback from Games Work Week

From a game developer: "A robust resource/asset loading/unloading/streaming system - The browser is capable of doing a whole lot on this end, particularly streaming and fetching from remote sources. However, a game developer's problem isn't just obtaining and loading the assets, it's managing them from a memory and "readyness" point of view. This is especially key in mobile spaces where you are severely constrained by memory and the traditional mindset of "load all of a level's assets at start and play" begins to break down."

Gladius is actually working on code for this, and our current theory is that we expect to split it out into a standalone library before too long.

IndexedDB will write separate Blobs as separate files in the OS file system. Is OS file system too slow? Especially when scaling up to lots of files in a single OS directory.

How big are these files usually. We've been talking about making the IndexedDB back-end collapse small files into a big OS-file. This would reduce reliance of OS large-directory handling.

We're working on file writing. That might work as a back-stop for the most advanced use cases.

Multithreaded programming

We currently have WebWorkers as solution for multithreaded programming. WebWorkers use a shared-nothing message passing model. I.e. there are no mutexes, monitors etc.

Is this going to work for game developers?

We also haven't done a lot of performance testing, should get feedback from developers if there are bottlenecks. For example is the postMessage code fast enough?

We're also missing a large number of APIs. The following APIs are known to be missing:

  • IndexedDB
  • WebSockets
  • Asynchronous FileReader (is this important given that we have FileReaderSync?)
  • Transferrable ArrayBuffers between threads
  • Shared Workers

We also know that our XMLHttpRequest performance is pretty bad, especially for large resources.

Device orientation control

Ability to block, define and get informed when the device orientation changes. Basic use case is for a game that has to be played in landscape mode. The game should lock the orientation to landscape mode even if the device is currently in portrait mode.

TODO: fill this section

  • link to a spec for the web
  • equivalent in Flash, Silverlight, iOS, Android?

Game-oriented documentation

TODO: fill this section

  • Tutorials
  • Example games
  • Articles in Gamedev.net, Gamasutra, etc.

Converting C/C++ games to Javascript

Emscripten can be used for this. It already supports SDL which is used by many games. Work on supporting OpenGL is under way.

High Resolution Timers

Feedback from Games Work Week

Mobile

Feedback from Games Work Week

Miscellaneous

Feedback from Games Work Week about missing APIs.

Regarding developer tools, see these etherpads from the Games work week:

Retrieved from "https://wiki.allizom.org/index.php?title=Platform/AreWeFunYet&oldid=401456"