User:Asqueella/JEP 107

This is an alternative version of Labs/Jetpack/Reboot/JEP/107 (page mods).

Here's in progress implementation (currently 'Model "B"').

---

JEP 107 - Page Mods

• Champion: Nickolay Ponomarev <asqueella@gmail.com>
• Status: ?
• Bug Ticket: bug 546739
• Type: API

Proposal

Introduce an API allowing jetpacks to run script whenever a content page the jetpack is interested in loads.

This way of enhancing functionality of web sites was popularized by the Greasemonkey extension.

Note: Some parts of this API are generally agreed on (include, global add()/remove() methods), but there are two execution models under consideration for the scripts actually working with the page (see #Discussion - e10s).

The initial proposal described a model (called "B" in the e10s discussion), but current thinking is that another model, "C" is overall better. The difference between the two models are noted throughout the rest of this proposal.

[Model "B"] Unlike Greasemonkey scripts, in this proposal the pagemod's scripts all share the same Jetpack context and have the same privileges as the jetpack itself (but see #Discussion - e10s below).

Use Cases

• Porting Greasemonkey-style scripts to Jetpack (see Bugzilla tweaks for Jetpack prototype for example)
• Implementing scripts that enhance certain sites, while also having access to higher-privileged APIs.
• Adding methods and properties to the global window. For example, window.geolocation could have been implemented as a jetpack, and we could implement window.camera or window.microphone as a jetpack in the future. (via bsmedberg)

Non-Use Cases

• Porting Stylish-like CSS-based modifications to Jetpack (this could be added in a later version of the API)

Dependencies & Requirements

• onWindowCreate and current implementation of the module in general requires bug 549539's fix, which (as of 2010-05-26) has only landed on mozilla-central (Firefox versions after 3.6.x). The plans are to land it on 3.6.x too, though.
• Making this API work in Electrolysis/Jetpack requires additional effort from the e10s team (see details on that page)

API Methods

Here's an example of how the ScriptMod API can be used in a jetpack:

var ScriptMod = require("page-mod").ScriptMod;
var myMod = new ScriptMod({
  include: ["*.example.com",
            "http://example.org/a/specific/url",
            "http://example.info/*"],

  // [Model "B"] The callbacks are specified in the jetpack context:
  onWindowCreate: function(wrappedWindow) {
    // this runs each time a new content document starts loading, but
    // before the page starts loading, so we can't interact with the
    // page's DOM here yet.
    wrappedWindow.wrappedJSObject.newExposedProperty = 1;
  },
  onDOMReady: function(wrappedWindow) {
    // at this point we can work with the DOM
    wrappedWindow.document.body.innerHTML = "<h1>Jetpack Page Mods</h1>";
  },

  // [Model "C"] The content script is run in a separate context, thus
  // it's specified in a separate file. The specific syntax is not final!
  script: require("self").data.url("my-example-org-mod.js"),
  // we'll also need a way to receive messages from the content
  // script here and maybe track the new pages getting loaded.
});

In this proposal, a script mod specifies the pages it might modify, and the scripts to run on these pages.

The onWindowCreate callback function gets called at the earliest possible moment (before the page even started loading, which is made possible by the notifications added in bug 549539).

The onDOMReady callback is called as soon as the page's DOM is ready (on the DOMContentLoaded event)

ScriptMod constructor

The ScriptMod constructor takes a single options parameter which is an object that may define the following properties:

• include: a required parameter specifying the pages the scripts in this script mod should run on.
  • Providing a string value str is equivalent to providing a single-item array [str].
  • The mod's scripts run on pages, matching any of include rules.
  • Each include rule is a string using one of the following formats (see discussion below):
    1. * (a single asterisk) - any page
    2. *.domain.name - pages from the specified domain and all its subdomains, regardless of their scheme.
    3. http://example.com/* - any URLs with the specified prefix.
    4. http://example.com/test - the single specified URL
• [Model "B"] onWindowCreate, onDOMReady: optional parameters specifying the code to run on the matched pages.
  • No code is run if these parameters are not specified.
  • Providing a single function func is equivalent to providing a single-item array [func]
  • When the provided value is an array, its items are expected to be functions. Non-function values are ignored.
  • The specified functions are called in order:
    • for onWindowCreate - when a page matching the include rules starts to load (but before any content is loaded in the page -- i.e. when the content-document-global-created notification implemented in bug 549539 is issued)
    • for onDOMReady - when a DOMContentLoaded event fires for the matching page.
  • An exception thrown from one of the functions does not stop the rest of functions from executing.
  • The specified callbacks are called with a single wrappedWindow parameter -- the content's window object wrapped in an XPCNativeWrapper. The callback's this is the page mod object (TBD not currently implemented). It goes without saying that with this syntax the callbacks are run in the calling module's scope, not in the content page's scope.

Creating a ScriptMod instance does not automatically add (activate) it.

add()

require("page-mod").add(scriptMod)

• add() makes the specified script mod take effect on any matching pages that start to load after the call. Adding a script mod does not apply it to existing matching pages.
• scriptMod must be a ScriptMod instance.
• Trying to add the same script mod twice throws an exception.
• This method does not have a return value.

remove()

require("page-mod").remove(scriptMod)

• Call remove() to stop a script mod from running on further pages. This does not undo the mod's effects on already loaded pages.
• scriptMod must be a ScriptMod instance, added earlier.
• Trying to remove a script mod, that has not been added, throws.
• This method does not have a return value.

Discussion

Extracted from this thread.

Discussion - include option

A short survey of existing formats:

• Greasemonkey scripts specify include and exclude URLs, each may contain wildcards ("*") in any location and may use a special ".tld" domain. These rules get compiled to a regular expression (see convert2RegExp), which is then matched against every URL loaded in the browser.
• Match patterns for Google Chrome's content scripts are similar to Greasemonkey's, but force to specify domain (either fully, any domain, or *.domain) and don't have the magic tld domain.
  • Also of interest: [1], [2]
• When specifying CSS styling Using the Stylesheet Service, which is an easy and robust way to apply CSS to all content and is also what Stylish uses, you have to describe the filters using CSS, i.e. @-moz-document rule. It allows to specify domain, exact URL, or the URL prefix.

Comments: Myk #1 Myk #2 Brian

The 'include' option is made required to make the mods clearly specify which pages they apply to (for easier auditing).

It was suggested to restrict the schemes of URLs page mods can run on, since letting a page mod run on chrome://, for example, can have security consequences we have not thought through.

Discussion - e10s

Context: in the 0.5 timeframe it is planned to move jetpacks to their own processes, as described on the Electrolysis/Jetpack page. In the long term, content tabs will run in their own processes as well ("out-of-process tabs").

Communication between different processes is not entirely transparent [3]: while the jetpack process will be able to call content functions, reference content objects and pass primitive values to content, content won't be able to hold references to jetpack objects. This means it won't be possible to pass a jetpack-defined callback to content functions (with a few possible exceptions).

From the discussion referenced above, there are different possible models of pagemods execution (implying different implementation requirements):

A. run a script in the web page context, and let it communicate with the
jetpack via postMessage-style APIs. [...]

B. run a script in the jetpack context and pass it the window/document for a
page being loaded. This requires CPOW wrappers, which have some limitations[...]

C. Run page mods in the content processes (to avoid getting involved with CPOWs
and their limitations), but in a separate context from the page (to make it
possible to write page mods that can do things that we don't want to expose
to regular pages). My understanding is that it is similar to what Google
Chrome does and similar to what Greasemonkey does.

This proposal currently specifies (B). Comments collected from the discussion:

• on A:
  • [Benjamin] This is trivially straightforward to do in a multi-process world[...]. But there are issues with polluting the content script namespace (e.g. if the jetpack needs to define functions).
  • [Nickolay] this means we can't give it [the jetpack script in content] any additional privileges (e.g. by listening for postMessage'd requests asking to do something that requires chrome permissions or by providing additional APIs like GM_* in Greasemonkey). It's fine for simple scripts, but not in general, I think.
• on B:
  • [Myk] Despite the limitations imposed by the requirement for CPOW wrappers, its developer ergonomics appeal to me. It's not yet clear what the relative security implications are, however.
  • [Nickolay] thinks that inability to register a callback is a major flaw for those who need it (cited the case of using Gmail's Greasemonkey API to get notified of changes in the web app)
• on C:
  • [Nickolay] suggested this as an optional addition to (B) for scripts that need transparent interaction with content.
  • [Benjamin] That's attractive in some ways, but it breaks the normal jetpack behavior of being a single script that does everything. I'm not sure it's worth breaking that programming model.
  • [Myk] given the technical limitations inherent to [jetpacks and content in separate processes], the question then becomes what feasible model best approximates [the] ideal experience. I think the answer to that question is in fact your suggested model C.
  • Requires additional code in the single-process case, additions to the platform in the e10s case.

Discsussion - comparison to the original JEP

This JEP has three main differences from the original JEP 107:

• CSS-based mods were deferred to a later version of the API.
• This JEP doesn't promise enabling/disabling page mods "instantly", since I don't see a way to implement it.
• Scripts in the original JEP run in the context of the page, while in this JEP they run in the jetpack context. Although it's an important feature, I think it can be implemented separately, since it requires substantially more effort and additional coordination for e10s.
• add/remove/empty methods on the page mod object were not included, since there's no clear use case for them, especially if the changes are not applied instantly, as in this proposal.

Discsussion - script context in Model C

Background: Chrome's Content Scripts and Message Passing.

A few issues here:

• [RESOLVED] At which point does the separate script run, how does it declare it wants to do something on-window-created or on-DOM-ready.
  • (Chrome has "run_at" option in the content script's manifest, which defaults to "document_idle" meaning "sometime between DOMReady and soon after onload" with other options being DOMReady and WindowCreated. It's not clear how important this is for performance and why.)
  • Since we want to make it possible to run code on-window-created to install APIs, we'll run the pagemod script before load starts and provide onReady callback or have a DOMContentLoaded example in documentation.
• [UNRESOLVED] How do we let the script include common libraries / modularize its code
  • [Nickolay] Chrome lists the scripts to be loaded (in the single content script context) in order in the manifest. Simple and similar to web pages, but different from jetpack execution model.
  • Possible option: provide require() to content scripts, like in the main jetpack
    • Pros
      • [Myk/Brian] we do need a way for the mod context to access self.data.url, JavaScript libraries like jQuery, and probably some other functionality. And modules are our hammer.
      • [Myk] there is a long tail of built-in functionality that page mods might want to access [..] We could design another mechanism to expose APIs to page mod modules, but that mechanism would either be [..] limited [..] duplicate what "require" already provides.
        • [Benjamin] strongly disagree here. Page mods, if they wish to access all these other bits, should use message-passing to the main addon code.
    • Cons
      • [Myk] a bit worried about the potential for confusion due to conflating the two spaces by providing both with the same interface for importing functionality but not allowing one to import the same functionality as the other.
      • [Nickolay] we should also remember that the content scripts (and the related CommonJS machinery) will reload every single page load. I think that while the CommonJS hammer is attractive, the content scripts should generally not be as complex as to require it. We can add it later if there's need.
  • [Brian] Perhaps the first release will not provide a require() function, and then later (once we figure out our story for the "search path" for this context and how it differs from the other modules), we can make it available.
  • [Brian] I'm vaguely thinking that the PageMod() constructor, next to the script: argument, could provide a list of libraries that are made available to that script. Maybe a mapping, like: scriptlibs: { jquery: data.url("jquery.js") } }. Allowing my-example-org-mod.js to use: var jq = require("jquery");
• [RESOLVED?] What is the script's global, how does it access page's Window and Document
  • [Nickolay] Both GreaseMonkey and Chrome create a clean object as the script's global with __proto__ set to XPCNativeWrapper(contentWindow) and necessary globals added to it (GM_*, chrome.extension.*). I think we should implement a scheme like GM's/Chrome's, since it will be the most familiar and intuitive. [Myk and Benjamin agreed this is a good solution for jetpack as well. Other options are listed in Benjamin's message.]
  • [Brian] Passing the window as an argument [to a callback] (versus providing it to the whole module as a global) seems more in keeping with the "The Number Of Globals In A CommonJS Module Shall Be Two: require and exports" pattern. [...]
    • [Myk,Nickolay] given that the sole purpose of "page mod" modules is to access the pages they are modifying, it's worth simplifying access to the "window" object by defining it globally
• How does the script communicate with the main jetpack
  • GreaseMonkey defines several GM_* globals to provide additional functionality to GM scripts.
  • [Nickolay] Chrome implements bidirectional asynchronous message passing via chrome.extension.sendReqest(json, responseCallback) and another pipe (Port) based API for long-lived connections.
  • [Nickolay] If we are going to allow exporting APIs (e.g. window.microphone) via this mechanism, we might need sync content->jetpack messaging. bsmedberg also mentioned this as a possibility.
  • [Brian] suggested a similar pipe-based mechanism: onNewPage: function (pipe) { in the ScriptMod options. "Instead of a "pipe" argument, maybe the onNewPage function should get a "control" object, from which it can manipulate the pipe, ask about the URL from which the target page was loaded, and register to hear about the page going away. The latter would be necessary for the all-volume-control jetpack to remove closed pages from its list."
  • [Benjamin] Start with something like addon.postMessage(JSON.stringify(messageobj), '*'); to re-use existing DOM mindshare (this doesn't do RPC).

TODO

• For SDK 0.5:
  1. Rename constructor to PageMod [4]
  2. Finalize the format for "include" rules and implement the necessary changes.
     • Restrict "*" to only match HTTP(S)+FTP [5] [6]
  3. Identify changes required for Electrolysis/Jetpack and implement them.
  4. Fix the remaining XXX:
     • minor tweaks
     • disable test on not supported Firefox versions (e.g. 3.6.3) -- is it needed or will jetpack drop support for 3.6.x with e10s anyway?
     • (maybe) figure out leak report in tests if the test tab is not closed before stopping tests.
     • Do we pass an XPCNativeWrapper to pagemod callbacks and do we advertise it in the docs?
       • Myk: "the argument the callback functions are passed should be called simply "window" in the documentation rather than wrappedWindow, as developers are unlikely to encounter the differences"
       • Nickolay: disagreed - XPCNW are very visible, "I'm keeping wrappedWindow for now, pending the decision to just pass an unwrapped value to the callback."
     • [docs] encourage addon developers to clean up after their page mods via the unload module. (The clean up actions should also run when the script mod is removed).
     • [docs] Should provide an example of using jQuery in a script mod.
     • If we keep model "B"'s APIs make sure to implement the latest naming changes suggested by Myk.
• Post-0.5:
  1. Possible API enhancements:
     • implement helper functions for common actions (insert <style>s, <script>s, etc.)
     • filtering functions as part of include ([7])
     • CSS-based mods
  2. Provide a way to run scripts in separate context for each page (i.e. in the content process for out-of-process tabs)