View source for WebExtensions/Implementing APIs out-of-process

This document is intended for WebExtension developers (as in implementers in Firefox, not addon
developers) and gives an overview of how new WebExtensions APIs should be implemented in a way
that is compatible with out-of-process addons.

The reader is assumed to be familiar with the layout of WebExtensions code in the tree, explained at
https://wiki.mozilla.org/WebExtensions/Hacking#Code_layout

== Process types ==

We distinguish the following process types.

* main - The main process. Hosts the browser UI, privileged chrome pages and tab browsers.
* addon - Hosts an addon's background page, popup, extension tabs, ...
* content - Hosts web content (from the internet).

There is only one browser process, and any number of addon and content processes. In practice, they
do not necessarily run in separate processes:

* At first there was only one process.
* With e10s, one process was added for content ([https://bugzil.la/e10s bugzil.la/e10s]).
* With e10s-multi, multiple processes may be used for content ([https://bugzil.la/e10s-multi bugzil.la/e10s-multi]).
* With webext-oop, addon code runs in a separate process ([https://bugzil.la/webext-oop bugzil.la/webext-oop]).
* In the far future, addon code may also run in multiple processes.

WebExtensions code should be compatible with these process models. This means that code should be
asynchronous where reasonably possible, especially when logic involves the use of information from
different processes. The canonical representation of addons is stored in the main process.

== Execution environments ==

Scripts in the following contexts can make use of addon APIs:

* Extension tabs, background page, pageAction/browserAction popup.
* Content scripts, extension frames in non-extension tabs.
* PAC scripts ([https://bugzil.la/1295807 bugzil.la/1295807]).
* Devtools pages ([https://bugzil.la/1291737 bugzil.la/1291737]).

For each of these, a subclass of <code>BaseContext</code> should be created, each with a unique <code>envType</code>. This
<code>envType</code> is used for determining which scripts should be *activated* in the context.
E.g. the next ext-script registers an API with the given namespace for the "content_child" context.

 // an example of an ext-*.js script:
 extensions.registerSchemaAPI("myNamespace", "content_child", (context) => {
   return {
     myNamespace: {
       syncMethod() { return "some return value"; },
       asyncMethod() { return Promise.reject({message: "Some error"}); },
     },
   };
 });

<code>extensions</code> in the ext-script is an instance of the <code>SchemaAPIManager</code>. This class is responsible
for maintaining the sandbox containing the ext-scripts. There is only one instance per process type.
ext-scripts run in their own scope but they can share state via the <code>global</code> object.
This design prevents inadvertent leakage of global state between supposedly independent ext-scripts,
which reduces the risk of divergent behavior if the independent scripts move to different processes.

=== envType ===
The first implemented values for envType are <code>addon_parent</code>, <code>addon_child</code>, <code>content_parent</code> and
<code>content_child</code>. The envTypes with the "child" suffix should be used by addon APIs that run in the
same process as the caller, usually because the method synchronously returns a value or because of
dependencies on DOM APIs and such. The "parent" suffix refers to APIs that run in the main process.

When adding a new <code>envType</code>, check in how many process types the API may run. There is always at
least one type, namely the process containing the addon code. If the addon code runs in the main
process, odds are that you only need one <code>envType</code>. If the addon code runs in another process, and
your API implementation requires chrome-specific functionality, you probably need two <code>envType</code>s.


=== BaseContext, ExtensionContext, ProxyContext and subclasses ===
In the above snippet, the <code>context</code> parameter is an instance of <code>BaseContext</code> (or a subclass). This
context is created whenever an addon runtime is created (e.g. a background page or content script).

<code>BaseContext</code> contains common functionality that can be relied upon by all ext-scripts, whereas
some other functionality is only available in some specific contexts. E.g. <code>context.contentWindow</code>
is only available to the contexts that are close to the content, such as the contexts with envType
"content_child" or "addon_child".

An example of <code>content_child</code> is <code>ExtensionContext</code> in ExtensionContent.jsm. This context is created
together with a <code>ProxyContext</code> of envType <code>content_parent</code> in the main process. Together they form
the implementation of content script addon APIs.

New classes should derive from <code>BaseContext</code>, unless you are absolutely certain that your use case
is compatible with the semantics of the superclass that you are using.


== Generating APIs ==
The previous section explained the runtime environment of ext-scripts. This section explains how
the ext-scripts are used to compose the <code>chrome.</code> and <code>browser.</code> objects that can be used by addons.

=== Initialization ===
API generation is completely based on JSON schemas, which specify the API and constraints. These
schemas are registered in the category manager and loaded in the main process and shared with child
processes before any addon is started.

The ext-scripts (which implement the APIs) are also declared in the category manager, and loaded via
subscripts.

Both tasks are performed by <code>SchemaAPIManager</code> (or its subclasses), and APIs are only generated
once the above steps complete. Since extensions (and its pages and content scripts) are only loaded
once the schemas are loaded, and subscripts are loaded synchronously, this effectively means that
any addon API can be generated when a <code>BaseContext</code> instance is created.

=== Internal API generation (untyped) ===

The first step towards API generation is to generate a context-specific instance of the APIs that
were registered through the ext-scripts. This is done by calling the <code>generateAPIs</code> method of
<code>SchemaAPIManager</code> (or on its subclasses). The <code>generateAPIs</code> method enumerates all registered APIs
with a matching <code>envType</code> and deeply merges the result. So it is possible for one file to provide a
partial implementation of a namespace, and complement it with another file.

The resulting object looks like the final API, but appearances are deceiving: It is is only an
intermediate API, to be used by the automatically generated APIs based on the JSON schemas.

=== Public API generation (checked) ===
The <code>chrome.</code> or <code>browser.</code> objects for addons is completely generated by <code>Schemas.inject</code>. This
method takes a destination object (e.g. <code>Cu.createObjectIn(window)</code>) and a wrapper object, and
adds all addon APIs with parameter validation to the destination object. These generated APIs
themselves do not provide any API functionality. The wrapper is the binding between the internal
API (providing the actual functionality) and the generated API (which only performs validation).
The wrapper has to implement the <code>InjectionContext</code> interface as documented in Schemas.jsm.

This interface has two points of interest: <code>shouldInject</code> and <code>getImplementation</code>.

=== shouldInject ===
The first line of eliminating API access is through (manifest) permissions. For fine-grained access
control in addition to permissions, <code>shouldInject</code> can be used.
<code>shouldInject</code> receives parameters to identify which function is being called, and an optional
array of "allowedContexts". These "allowedContexts" can be declared in a JSON schema to annotate certain
APIs and enforce access control. The next example shows how content scripts are withheld of all APIs
unless enabled explicitly by adding "content" to "allowedContexts" in the JSON schema.

   shouldInject(namespace, name, allowedContexts) {
     // Do not generate content script APIs, unless explicitly allowed.
     if (this.context.envType === "content_child" &&
         !allowedContexts.includes("content")) {
       return false;
     }
     return true;
   }

With the above check, schema APIs are completely disabled unless the JSON schema specifies the context in "allowedContexts".
With "permissions", schema APIs are enabled, unless the schema specifies a permission that the
extension lacks.

Any API node can be annotated with "allowedContexts". To cater for the use case where a full namespace
should be made available, the "defaultContexts" key can be used. Note that when "allowedContexts" is
used to opt into an API (like the above example with "content"), then the namespace declaration in
the JSON schema must also be annotated with the "allowedContexts" key.

Here are some examples of code that migrated from being untyped to using schemas to enforce types (note that "allowedContexts" / "defaultContexts" was initially called "restrictions" / "defaultRestrictions", but that changed in [https://bugzil.la/1302898 bug 1302898]):

* i18n API - https://hg.mozilla.org/mozilla-central/rev/e4ce08beaf74
* runtime API - https://hg.mozilla.org/mozilla-central/rev/2427f8eb4e83
* extension API - https://hg.mozilla.org/mozilla-central/rev/598895fae31d

=== getImplementation ===
<code>getImplementation</code> is called once <code>shouldInject</code> has returned true for a given API method, event,
property, etc. It should return an implementation of the <code>SchemaAPIInterface</code>. Currently, two
implementations exist, <code>LocalAPIImplementation</code> and <code>ProxyAPIImplementation</code>.

<code>LocalAPIImplementation</code> is a thin wrapper that delegates any calls directly to the internal API.
<code>ProxyAPIImplementation</code> implements invocation of APIs from a child process that are implemented in
the main process (remote APIs).

The following wrapper can be used to generate an API that is fully implemented locally:

   getImplementation(namespace, name) {
     let pathObj = ...;  // Somehow find the API with the given name in the given namespace.
     return new LocalAPIImplementation(pathObj, name, context);
   },

To implement remote APIs (optionally mixed with local APIs), see <code>ChildAPIManager</code>.

=== Remarks ===
* The set of available addon APIs is the intersection between available internal APIs and injected schema-gemerated APIs.
* Depending on the implementation of <code>shouldInject</code>, it is possible that a schema API is generated while no internal API exists. The behavior of this situation is unspecified, but odds are that some exception is thrown.
* It is also possible that an internal API is provided, but inaccessible to addons because of incomplete schema definitions. It has no real consequences, except wasted resources on generating unnecessary objects. Incidentally, this is also why <code>registerSchemaAPI</code> takes an <code>envType</code> as an argument, because it would be a huge waste to generate unused API bindings for all extension APIs in content scripts, because content scripts only use a very limited subset.


== Writing new APIs ==

The existing ext-*.js files [[WebExtensions/Hacking#Code_layout|in the tree]] are good examples of API implementations. This section shows more insight in why some things are implemented in the way they are.

=== Naming conventions ===

* File name: Historically, the API implementation in ext-*.js runs in the main process. To distinguish between code running in content processes and the main process, a new prefix was introduced (ext-c-*.js). This c initially stood for "content" (content processes), but was later generalized to "child" because of the big overlap in the implementation of APIs for content processes (content scripts) and addon processes (privileged addon APIs).

=== Cross-process API implementations ===
There are multiple ways to implement APIs:

* In the child only (e.g. ext-i18n.js)
* In the parent process only (e.g. ext-history.js)
* In the parent and child process where some methods run in the main process and others in the child process (e.g. ext-tabs.js and ext-c-tabs.js)
* In the parent and child process where a method is partially implemented in the child process and partially in the main process (e.g. ext-storage.js and ext-c-storage.js)

If a method is not implemented in the child process, the ChildAPIManager will automatically route the API call to the parent process. Sometimes the method's arguments or return value is so special that the default behavior is insufficient, then you can choose to implement (part of) the implementation in the child.

The ChildAPIManager (available as <code>context.childManager</code> for ExtensionContexts) has the following helper methods (with JSDoc documentation at their implementations in ExtensionUtils.jsm). All of these methods require at least the API path as an argument, which is is used to find the implementation in the parent process.

* <code>callParentAsyncFunction</code> - Call a method in the parent process, and resolve the promise/callback when that function returns.
* <code>callParentFunctionNoReturn</code> - Call a method in the parent process without expecting a return value.
* <code>getParentEvent</code> - Create a proxy for an event listener in the parent. See [[#Cross-process events]].

==== Cross-process events ====
All event objects with the same API path and the same context (=an instance of BaseContext) share the same set of listeners. The corresponding listener in the parent process is registered once on the first <code>addListener</code> call (and re-used for subsequent calls). When the last listener in the child is removed via <code>removeListener</code>, the listener is also unregistered in the parent process. Here is an example:

 let eventObj = context.getParentEvent("contextMenus.onClicked");
 eventObj.addListener(listener);     // Add one listener (and register the event in the parent)
 eventObj.addListener(listener);     // Listener was already registered, not added again.
 eventObj.removeListener(listener);  // Remove the listener (and removes the registration from the parent).
 context.getParentEvent("contextMenus.onClicked").addListener(listener); // Identical to eventObj.addListener
 eventObj.addListener(otherListener);// Add another listener, re-using existing parent event registration.

=== Timing issues ===

There are subtle timing issues related to the order of execution of asynchronous APIs. If in doubt, design the API in such a way that the order of execution is not relevant. Failing to recognize the problems will may result in intermittent tests.

* The order of running API implementations in the parent is guaranteed to match the order of API invocations in the child.
* An API call must ''not'' unconditionally assume that the previous API call has finished. This is because the previous API call may have an asynchronous implementation.
* There is a subtle timing issue with event registrations that is especially visible when the add-on controls the trigger of the event and registers an event in a callback. Async APIs that trigger events may trigger the events before the callback is called. This is especially important in unit tests. For instance, <code>tabs.create</code> creates a tab which triggers the <code>tabs.onUpdated</code> event. If the add-on registers calls <code>tabs.onUpdated.addListener</code> in the callback, then it is possible that the onUpdated event has fired before the listener registration is processed by the parent. Due to the fact that the event listeners are shared, if a ''(different)'' listener is registered before the <code>tabs.create</code> call, the result may be different: When the "onUpdated" event is triggered in the parent, the event is fired and (asynchronously) sent to the child. Meanwhile the callback was invoked, the event was registered and the new event listener is also invoked.