Breaking the grip JS has on the DOM

We want to change the grip JS has on the DOM and on XUL. We will do this in 2 steps:

• Change the framework to be language neutral - https://bugzilla.mozilla.org/show_bug.cgi?id=255942
• Create a Python implementation.

Ideally, the first step could be done without consideration for the second, in the assumption at the implementation should be truly language neutral. However, this first real implementation is likely to impact the design decisions made, so there will be some iteration involved

Original specification - stage 1

This is the high-level task list as specified at the start of the project:

• Extend nsScriptLoader using the XPCOM category manager to handle arbitrary <script type="...">
• MIME types, loading extension component mapped by MIME type through a category
• Abstract and multiplex nsIScriptContext, generalizing it (changing its IID of course) away from the JS API, making a nsIJavaScriptContext for JS and an nsIPythonContext for Python, fixing various places that assume nsIScriptContext "does JS"
• Fix default script language selection and event handlers, which are script-language-typed by the selected default, so that you can write Python event handlers.

High-level design decisions

Decisions made/ratified by Brendan etc:

• No sharing of language namespaces. Only the "global DOM namespace" (ie, the nsIScriptGlobalObject) will be shared.

• A nsIScriptGlobalObject will maintain multiple nsIScriptContexts, one for each language supported. There will be no sharing among nsIScriptContexts - ie, each language is completely independent, sharing only the nsIScriptGlobalObject.

Identified task list

These are the tasks identified during the analysis and implementation phases of the project.

Overview

The existing nsIScriptContext interface will remain tied to a specific language. The "void *" params in its interface will remain. This means that a "void *" and a suitable nsIScriptContext must always be treated as a pair (ie, given just a "void *", there is no way to determine an appropriate nsIScriptContext suitable for it - although "assume JS" is likely to remain for existing code that does exactly this)

nsIScriptContext will grow new methods relating to:

• language specific cleanup and "default language context" type code. For example, nsGlobalWindow.cpp has a number of calls to ::JS_ClearScope/::JS_ClearWatchPointsForObject etc - these need to be hidden behind one of the interfaces.
• The "WrapNative" process (or possibly this can be hidden behind JS 'SetProperty' functions?)

The existing nsIScriptGlobalObject interface will move towards a model where there is a global nsIScriptContext per language. GetGlobalJSObject and GetContext would be deprecated, with a new method allowing the desired language to be specified. Thus, nsIScriptGlobalObject may have many nsIScriptContexts associated with it (one per supported and initialized language)

A new 'context stack' will be invented:

• nsIScriptContexts to be used directly on the stack, rather than JSContext (hmm - is this necessary? I guess a "void *" would still work so long as it was associated with the language)
• Contexts for *all* languages are pushed and poped as a single operation - a new context applies to all languages in the environment, not just a single one. This is so when calls cross multiple language boundaries, they always have a correct context, not just the last one that happened to be pushed for their language.

nsDOMClassInfo will need a fair bit of work, and significant help from Mozilla resources. It may be able to be split into a language neutral nsDOMClassInfo, and a JS specific nsIXPCScriptable. The JS specific code in this file will need to be re-written for Python, but this should allow all DOM knowledge to be reused by the Python implementation.

nsIProgrammingLanguage constants will identify a language. Existing occurances of a char * identifying a language or version may also be changed to also use an integer, allowing bitmasks etc.

Below are specific implementation notes:

new interface nsIScriptLanguage

• This is really only a factory for nsIScriptContext objects. The nsIScriptContext itself becomes the factory for the language specific, void * "global object".

nsIScriptContext

• Existing JSObject replaced with void

• Existing "void *aScopeObject" (ExecuteScript/EvaluateStringWithValue/CompileScript), is *not* replaced with nsIScriptGlobalObject. This is a change from earlier versions of this document - nsXBLProtoImplField and a few others pass an explicit aScriptObject which is not the same as associated with the nsIScriptGlobal.

• FinalizeClasses method - JS does the ClearScope/ClearWatchpointsForObject/ClearRegExpStatistics?

• nsIArray to be used in place of JSVal argc/argv used now.

• Maybe some kind of "SetProperty" function - as needed by nsGlobalWindow - "arguments", "navigator" etc. http://lxr.mozilla.org/seamonkey/source/dom/src/base/nsGlobalWindow.cpp#871 [Not sure this is needed. IMO we should make "arguments" be XPCOM objects, i.e. [nsIArray -- markh] and we'd finally be able to get to the arguments passed to window.open() from *any* language. "navigator" is just a property of a window like all others, except that there's some JS:isms around it that I think we can easily ignore for Python -- jst]

• SetTerminationFunction needs thinking through - this does *not* seem to be a per-language thing. Maybe could be on the nsIDOMContextStackItem proposed below? Only few callers though.

• New method:

 void *GetNativeGlobal() to return the "global object" used by nsIScriptGlobalWindow (previously stored in mJSObject) for this context.  nsIScriptGlobalObject calls this method during language init to setup its environment.

nsIScriptGlobalObject / nsGlobalWindow

http://lxr.mozilla.org/seamonkey/source/dom/public/nsIScriptGlobalObject.h

nsGlobalWindow is the main implemention of nsIScriptGlobalObject, but XUL and XBL have a few.

• nsIScriptGlobalObject remains a single object (ie, not per language). It will keep a list of "nsIScriptContext *ctx, void *global" items, one for each language. New methods:

 nsIScriptContext *GetLanguageContext(PRUint32 lang_id);
 void *GetLanguageGlobal(PRUint32 lang_id);

The concept of a single context/global will be deprecated - callers will be encouraged to use the new method. GetContext()/GetGlobalJSObject() remain for b/w compat. They are equivalent to "GetLanguageContext('js', ...)" SetContext replaced with SetLanguageContext(). New nsIScriptContext::GetNativeGlobal() used to populate the "native global" for each language.

• New method to ensure the nsIScriptGlobal is initialized for a specific language. This may be called at any time - whenever someone needs to run a script in a language.

• New method to set and get "properties" - nsGlobalWindow does this for JS. Setting a property should presumably set it in all languages (via the global attached to the context). Getting is tricker - what if the property is in a different language? Or in multiple languages?
  • Properties set include "arguments" and "navigator". Presumably other code also sets additional properties?
  • Properties fetched seem arbitrary.

• SetNewArguments uses nsIArray

• New method to push all current contexts onto the context stack.

• Serializing scripts and fast-load needs thought. Python could benefit from caching its compiled "code objects" to avoid parser overhead on each reload. These "code objects" are just another "PyObject *", so abstracting and delegating fast-load to nsIScriptContext may be possible.

• CompileScript - We should consider fixing this:

  nsXulElement.  Line 3666:
  // XXXbe violate nsIScriptContext layering because its version parameter
  // is mis-typed as const char * 
  

nsDOMClassInfo

nsDOMClassInfo provides 2 key functions:

• standard nsIClassInfo implementations for DOM.

• Enhanced type info for DOM which can not be expressed using nsIClassInfo. These are the "scriptable helpers"

The extra nsIClassInfo implementation is suitable for all languages. However, the scriptable helpers have lots of DOM namespace magic, and are very JS specific. Its not clear how much if this is needed for other languages, but presumably some is. This section discusses only that JS specific magic.

The implementation is now at a point where Python requires this enhanced class info (ie, Python is being called with a DOM object - at the moment it can only use what is reflected in standard nsIClassInfo.

Note: Work on nsDOMClassInfo could be done by a Mozilla resource in parallel with this other work. We will not need to use this enhanced class info until we already have Python code being executed and that code trying to work with a "dumb" DOM object.

One radical alternative: Invent a new "nsIDynamicClassInfo" or similar interface that allows the additional class info to be expressed. Modify the JS XPConnect code to also use that new interface is supported regardless of implementor. Python's xpcom code would do likewise. nsIDOMClassInfo is responsible only for implementing this new interface. It is possible this could also subsume your existing IDispatch support.

It is not clear to MarkH how to make the existing scriptable helpers language agnostic

Context Stack

[MarkH - I've managed to completely ignore this for the time being]

Existing nsIJSContextStack "@mozilla.org/js/xpc/ContextStack;1" service is replaced with language generic ContextStack. This is almost identical to the JS version, but all operations work for the entire set of languages in use, not just the language about to be executed.

Probably need a new interface for each language to do their thing. The DOM will simply ask for a new context, and the nsIScriptGlobal will push the context for each language it is initialized with.

Thus, we will have one ContextStack, with each holding an array of nsIScriptContexts - one for each language. You can not push a context for only a single language on the stack.

If a language is initialized even after items are already on the context stack, only new items pushed will include a context item for that language. Once the stack pops back past where a language has no entries, the language will not be able to run (but this should be impossible - there can be no stack entries for that language higher in the context stack, and attempting to start a new script in that language will simply re-push a context entry which will again include the language)

As mentioned above re nsIScriptContext, we may need to handle SetTerminationFunction functionality on this stack.

Tricky: We need to interoperate with code that is not multi-language aware, and may not be for some time. Eg: http://lxr.mozilla.org/seamonkey/source/extensions/pref/autoconfig/src/nsJSConfigTriggers.cpp#216 http://lxr.mozilla.org/seamonkey/source/extensions/webservices/security/src/nsWebScriptsAccess.cpp#767 - pushes a NULL context? This probably will mean that the existing JS Context Stack will be a wrapper around the new cross-language one.

Generic "GetCallerDocShell" or similar will avoid some JS specifics - eg, http://lxr.mozilla.org/seamonkey/source/docshell/base/nsDocShell.cpp#5898

What about http://lxr.mozilla.org/seamonkey/source/dom/public/nsIScriptContext.h#356 - it checks flags. Will there be flags for a single stack entry, or flags for each language in a single stack entry, or both?

New interface - nsIDOMContextStackItem

An item on the context stack - stores one nsIScriptContext for every language initialized for this item.

nsIDOMContextStackItem : nsISupports {

 nsIScriptContext getLanguageContext(in PRInt32 langId);
 void setLanguageContext(in PRInt32 langId, in nsIScriptContext cx);

};

New interface - nsIDOMContextStack

interface nsIDOMContextStack : nsISupports {

 readonly attribute PRInt32     count;
 nsIDOMContextStackItem      peek();
 nsIDOMContextStackItem      pop();
 void                                   push(in nsIDOMContextStackItem cx);

 /* what is a "safe context" anyway??? */
 nsIScriptContext getLanguageSafeContext(in PRInt32 langId);
 
 /* A helper for code that wants the most recent nsIDocShell on
 the context stack - any/all languages on the stack can provide it */
 nsIDocShell GetCallerDocShell()

};

Notes on callers of existing JS Context Stack

nsJSConfigTriggers.cpp: pushed and pops its own private JSContext. Not nsIScript* aware at all.

nsWebScriptsAccess.cpp: pushes and pops an explicitly null context

nsXMLHttpRequest.cpp: appears to just push the context from its nsIScriptContext *mScriptContext, and fetch an nsIScriptContext from the context stack - can use the language independent function. In NotifyEventListeners/ChangeState/OnChannelRedirect

nsScriptSecurityManager: Very JS specific already and will probably remain so.

nsDocShell: Use new GetCallerDocShell as described above.

LegacyPlugin.cpp: Does some private magic with context stacks. Filename implies we can ignore this. ns4xPlugin.cpp: some private js context work.

nsWindowWatcher: Generally looks at the top of the stack to try and locate a dynamic or static script global. Some JS specifics though.

js/src/*: Lots of JS specific stuff as you expect

ProxyClassLoader.cpp: Peeks at the stack to get a context, to get a property. Could use new property fetch functions.

nsProfile.cpp: Calls GetSafeJSContext just to force a GC.

nsContentUtils: See nsContentUtils discussion below.

nsRange.cpp::CreateContextualFragment - seems to push/pop a JS Context, and compares principals and contexts. // If there's no JS or system JS running, push the current document's context on the JS context stack What does that mean???

content/html - a few uses ignored.

nsLocation: Looks for existing context. Some DynamicContext.

JS_GetContextPrivate ???

[JS_GetContextPrivate() is how one gets to the nsIScriptContext given a JSContext. All JSContext's are obviously not nsIScriptContext's, so flags need to be checked etc, as you saw in GetScriptContextFromJSContext() -- jst]

nsIScriptSecurityManager

All JSContext* can probably be replaced with nsIScriptContext. Use of simple JObject* may need to be replaced with nsIScriptContext* and matching void*.

nsIScriptSecurityManager is caps' interface, and caps will most likely remain JS specific. Caps is also extremely performance critical, and having to go through a virtual function to get to the JSContext may not be what we want, if it can be avoided... -- jst

nsScriptLoader

nsScriptLoader's interface is language neutral.

• nsScriptLoadRequest will have either a nsIScriptContext or language ID as a member.

nsScriptLoaderObserver

Presumably will need to become script language aware. eg, scriptAvailable() method has text of the script passed - not much use without also knowing the language. Possibly replace with nsIScriptElement, which does have the language ID

nsXULPrototypeScript

• Change JSObject *mJSObject -> void *mScriptObject.

• Add language_id reference - caller must provide their own nsIScriptContext.

• Deserializing etc needs to support multi-languages. May require serialize/deserialize functions on the context?

nsContentUtils

Uses native JSContext for ReparentContentWrapper

GetDocShellFromCaller/GetDocumentFromCaller - uses JS context stack to get the nsIScriptGlobal

nsCxPusher helper class: Peeks at JSContextStack to determine if a script is running (the simple fact an item is on the stack is all it checks). Pushes and pops a context.

nsContentUtils::NotifyXPCIfExceptionPending(JSContext* aCx) - 2 callers:

• content/xbl/src/nsXBLProtoImplMethod.cpp - nsXBLProtoImplAnonymousMethod::Execute() - very JS specific [Do we plan to keep XBL JS-specific, then? Or allow binding implementations done in other languages? I'd assume the "very JS specific" comment refers to the latter, right? If so, we'll need a lot of work on this front, especially to keep our existing brutal sharing; perhaps we need some nsIScriptContext methods for that sort of thing? I'd love for Execute() to not rely on jsapi.h --Bzbarsky 18:55, 15 Aug 2005 (PDT)]
• dom/src/base/nsJSEnvironment.cpp

nsAppShellService

SetXPConnectSafeContext - seems to create and set the "safe context" for the context stack.

Event listeners

http://lxr.mozilla.org/seamonkey/source/content/events/src/nsEventListenerManager.cpp#1160

Changes required:

• AddScriptEventListener() needs a "language_id" passed in
• 'context' still comes from ScriptGlobal
• Use of "@mozilla.org/js/xpc/ContextStack;1" needs to be abstracted into the nsIScript interfaces.

nsDOMScriptObjectFactory

nsDOMScriptObjectFactory:::NewScriptContext may be unnecessary - just allow the ScriptGlobalObject to auto-create its own somehow. There are multiple implementations, so may not be reasonable.

Exceptions

Exceptions should be chained across languages. This should just mean diligent use of nsIExceptionService?

Identified list of things we will *not* do

• We will make no attempt to have abstract the security interfaces - Python has no concept of "untrusted code". This means Python will be restricted to running from trusted XUL.
• Principals (obviously we must not change existing semantics, but Python will ignore them.)
• Allowing a language version to be specified.

Stuff Mark still needs to grok

• Still a little gray on the relationship between a JSContext, a "scope" and a "global". [ One could say that "scope" is the 'this' property when running code, "global" is what's at the top of any object's scope chain. A JSContext always has a global object (in mozilla at least), but there's not always a 1:1 mapping. In fact now with the split window work landed, there's generally more than one global object per JSContext. -- jst]
  • JSContext a DOM specific concept, so script_language -> C++ -> script_language preserves the state of the globals (correct?) [Not sure what you mean by this -- jst]
  • What is in a "context" beyond the global object? [For one, the stack frame for the running JS is in the context, very important for security code -- jst]

• WrapNative and what it really means to this

• Best way to tackle nsDOMClassInfo??? [ I think the harder problem is the scriptable helpers, the nsDOMClassInfo class is fairly language neutral already -- jst]

• nsBindingManager - very JS specific and not using nsIScript interfaces

• Need to understand the desired 'undefined' semantics for the return value. Python's builtin None is closer to a JS null, so may not be suitable for 'undefined'. However, a simple 'return' will return None.

See nsXBLProtoImplField.cpp

• Timeouts look tricky. [Yeah, somewhat. Especially the management of timeout objects is tricky due to reentrancy of all sorts being possible there. nsTimeout probably needs now to know the language, and have code to execute different languages... -- jst]

• Events look tricky [Shouldn't be too bad. There's some JS:isms in the event code, but mostly to support node.onclick=... style event handlers. Whether we want to support those in Python is a desision that'll make this potentially much harder to do :) -- jst]

Random Notes

Some notes about existing callers of certain nsIScriptContext.

nsDOMParser:

• Gets "native call context" to end up with nsIScriptContext - apparently all just to get the script URI nsDocShell
  • http://lxr.mozilla.org/seamonkey/source/extensions/xmlextras/base/src/nsDOMParser.cpp#476
  • Will need to know the language_id that should process the data from the stream
• Security related functions

nsXMLHttpRequest.cpp

* GetCurrentContext uses the JS ContextStack.

mozXMLTermUtils.cpp:

 * mozXMLTermUtils::ExecuteScript called nsIScriptContext::ExecuteScript -      but with NULL "void *" pointers.  Needs to have nsIScriptContext passed?

nsScriptSecurityManager.cpp/nsSecurityManagerFactory.cpp:

• Almost all functions convert a JSContext to an nsIScriptContext. Most just use to GetGlobalObject - GetPrincipalAndFrame has JS assumptions
• nsSecurityNameSet::InitializeNameSet - Lots of JS specific code.

docshell/base/nsDocShell.cpp: 1 use nsDocShell::CheckLoadingPermissions() - use of JF ContextStack to fetch nsIScriptContext - just to GetGlobalObject

embedding/components/windowwatcher/src/nsWWJSUtils.cpp

• Converts JSContext to nsIScriptContext. Mainly for GetGlobalObject, but nsIScriptGlobalObject *nsWWJSUtils::GetStaticScriptGlobal has JS deps.

embedding/components/windowwatcher/src/nsWindowWatcher.cpp:

   JSObject * nsWindowWatcher::GetWindowScriptObject
       not used???

   nsWindowWatcher::AttachArguments
   AddSupportsTojsvals
       2038       rv = xpc->WrapNative(cx, ::JS_GetGlobalObject(cx), data,
       2039                            *iid, getter_AddRefs(wrapper));
 

content/base/src/nsContentUtils.cpp:

   Convert nsIScriptContext -> JS Context

content/base/src/nsDocument.cpp:

   Convert nsIScriptContext -> JS Context -> CanExecuteScripts
   (can CanExecuteScripts take an nsIScriptContext?)

content/base/src/nsScriptLoader.cpp:

       nsScriptLoader::ProcessScriptElement
           Does the actual loading of the stream.
       EvaluateScript calls nsIScriptContext->EvaluateScript - with most args

content/events/src/nsEventListenerManager.cpp:

   Quite JS specific
   

content/xbl/src/ nsXBLBinding

   ::JSSetPrototype for a new context?

nsXBLDocumentInfo:

 implementation of nsIScriptGlobalObject.
  JSObject *mJSObject;    // XXX JS language rabies bigotry badness
  mJSObject = ::JS_NewObject - abstraction?

nsXBLProtoImpl:

 Lots of JS Specific code.
 Fairly generic compile function: http://lxr.mozilla.org/seamonkey/source/content/xbl/src/nsXBLProtoImpl.cpp#192

content/xul: nsXULElement

   event handlers remain JS specific
   serialization too

nsXULDocument:

   Calls ExecuteScript with mScriptGlobalObject->GetGlobalJSObject() as an arg
    case nsXULPrototypeNode::eType_Script: {
      else if (scriptproto->mJSObject) {
                    // An inline script
                    rv = ExecuteScript(scriptproto->mJSObject);
                    if (NS_FAILED(rv)) return rv;
                }

   XUL Script cache

dom/src/base/nsDOMClassInfo.cpp

 Lots of JS - dynamic DOM not exposed by XPCOM class info?
 

dom/src/base/nsGlobalWindow.cpp

   JS specific "scope" code
    nsIScriptContext *currentCX = nsJSUtils::GetDynamicScriptContext(cx);
    argv handling
    Creating a window - calls WindowWatcher::OpenWindowJS
    

dom/src/events/nsJSEventListener.cpp:

layout/generic/nsObjectFrame.cpp

xpfe/appshell/src/nsAppShellService.cpp

plugins/activex/embedding/crypto: ignoring for now.

Any idea what we're doing with setTimeout and setInterval? Namely, in their string argument form. --Hixie 01:43, 28 Aug 2005 (PDT)