JavaScript:New to SpiderMonkey

Tutorial: your first patch

The first step to getting involved with SpiderMonkey is to make your first patch. This guides you through it, and at the end you should have learnt a lot of the procedures and formalisms involved in getting things done here.

We'll assume you're on a Unix-y platform, and that you know what you're doing. We'll ignore nearly all details.

Get the code

Spidermonkey development happens in the "mozilla-central" mercurial repository:

hg clone http://hg.mozilla.org/mozilla-central spidermonkey

Build the js shell

Most of the time, you'll be working with the Javascript shell, instead of the full Firefox browser. So build the shell:

cd spidermonkey/js/src
autoconf2.13 # or autoconf-2.13
mkdir build_DBG.OBJ 
cd build_DBG.OBJ 
../configure --enable-debug --disable-optimize
make
cd ..

If you're having trouble or are missing dependencies, refer to Building SpiderMonkey Tip.

Fix something

At this point, you're ready to make your first fix.

TODO: something useless. Maybe adding a datemicrosecond function.

Building your changes

Having made the change, build the shell again.

make -C build_DBG.OBJ

Testing your changes

It builds. Hurray. Time to run the tests to check you haven't broken anything.

jit-test/jit_test.py build_DBG.OBJ/dist/bin/js

(The location of the shell has changed, it used to be build_DBG.OBJ/js)

The jit-tests are pretty quick, and should give you an idea if you've gotten something wrong. Assuming nothing goes wrong, try the ref-tests:

tests/jstests.py build_DBG.OBJ/dist/bin/js

Note that some of these tests fail depending on the timezone and locale they're run in, so if you get any errors that don't seem related to your changes, re-run the tests with a shell build without your changes applied and compare the results.

Benchmark your changes

Tests all pass. Congrats, you didn't break anything. Now, did you make it faster or slower? We benchmark using the v8 and SunSpider benchmarks. Get the benchmarks:

svn checkout http://svn.webkit.org/repository/webkit/trunk/PerformanceTests/SunSpider

And now run them:

cd SunSpider 
./sunspider --shell=../build_DBG.OBJ/dist/bin/js --run=30 --suite=sunspider-0.9.1 
cd ..

Optimized build

Whoops, we benchmarked the debug version. Let's make an optimized build to test instead.

mkdir build_OPT.OBJ 
cd build_OPT.OBJ 
../configure --disable-debug --enable-optimize
make 
cd ..

Repeat the sunspider steps above, and take note of the line that looks like:

Results are located at sunspider-0.9.1-results/sunspider-results-2010-08-07-17.56.48.js

You'll need that later.

Baseline version / Mercurial Queues

We need to time our optimized version against the baseline version. This calls for a brief introduction to mercurial queues, which most people think is a pretty good way of managing their SpiderMonkey workflow:

hg qinit
hg qnew my_first_patch -f
hg qrefresh

This puts your current work into a patch, managed by Mercurial, symbolically called my_first_patch. To pop the patch off:

hg qpop

And we're back to our pristine version.

Compare

Build again and rerun SunSpider again. You should now have two files like:

sunspider-0.9.1-results/sunspider-results-2010-08-07-17.56.48.js

Compare them using the compare script:

cd SunSpider 
./sunspider-compare-results --shell=../build_DBG.OBJ/js --suite=sunspider-0.9.1 FILE1-withoutPatch FILE2-withPatch

Get a real bug

At this point, you've seen nearly everything you need to do hack on SpiderMonkey. So it's time to get a real bug to work on. You can get a bug on Bugs Ahoy.

Fix the bug, updating the bug report with your progress, and asking questions as you go (either in the bug comments, or in #jsapi). When it's done, repeat all the steps above. Then it's time to get your patch into the tree.

Submit a patch

To get the patch from mercurial, use:

hg qdiff # if you're using queues or
hg diff  # if you're not

Add it to the bug as an attachment.

Get a review

Nothing gets into the tree without a review, so you'll need one. The SpiderMonkey hackers list is a good place to start: if your patch changes something listed as an area of expertise for someone there, that's a good person to ask for a review.

Alternatively run hg blame on the files you've changed, and check who has been changing related code recently. They're likely to be good candidates.

The review will consist of comments on your changes, suggesting or requesting alternative ways to do something and asking you to make changes where needed. They might also request additional changes, for example tests. Fix what they ask, resubmit the patch to bugzilla, and ask for another review. After you repeat this step a few times, they're mark the patch as "r+" meaning it's now good to commit.

Commit

You can't commit to mozilla-central / mozilla-inbound until you have "level 3" access, so you'll need someone to do this for you. Try asking in #jsapi, or add the checkin-needed keyword to the bug. After you have been contributing for a while, you can get level 3 access by applying for it.

After committing, a large series of tests will be run to make sure you didn't break anything. You'll need to hang around to make sure you didn't break something. Check the Firefox tree for failures. Sometimes failures will be spurious: ask for help determining when this is the case, to start. (Over time you'll figure out when this is and isn't the case yourself.)

Overview of the JS engine

The JS engine is a swiftly moving target. The most detailed information is available at https://developer.mozilla.org/en/SpiderMonkey. Here are some particularly interesting, mostly up-to-date resources:

High level overviews

(Outdated) http://hacks.mozilla.org/2010/03/a-quick-note-on-javascript-engine-components/

https://developer.mozilla.org/En/SpiderMonkey/Internals

Medium level documentation

jsapi.h: http://hg.mozilla.org/mozilla-central/file/tip/js/src/jsapi.h

Frequently used coding recipes and mappings from JS idioms to SpiderMonkey code: https://developer.mozilla.org/En/SpiderMonkey/JSAPI_Cookbook

Detailed documentation

Build: https://developer.mozilla.org/en/SpiderMonkey/Build_Documentation

Testing: https://developer.mozilla.org/en/SpiderMonkey/Running_Automated_JavaScript_Tests

Shell: https://developer.mozilla.org/En/SpiderMonkey/Introduction_to_the_JavaScript_shell

Function reference: https://developer.mozilla.org/en/SpiderMonkey/JSAPI_Reference

Collaboration and teamwork

Communication (in descending order of information content)

Nearly all communication is handled through Bugzilla. All bugs, feature requests, issues, enhancements, etc, are all referred to as bugs. No code may enter the Mozilla repository without being a patch attached to a bugzilla bug first. To follow all SpiderMonkey related bugs:

Go to email preferences
Watch the user general@spidermonkey.bugs

SpiderMonkey contributors generally hang out in the very active #jsapi IRC channel.

The js-internals mailing list is used for communicating with other SpiderMonkey hackers. (Internals as in discussing the internals of the SpiderMonkey engine. All are welcome on the list.).

Reading Planet Mozilla is the best way to keep up with the Mozilla project, which includes some SpiderMonkey related blogs:

David Anderson
Rob Sayre
Jason Orendorff
David Mandelin
Nicholas Nethercote
Jeff Walden
Jim Blandy
Chris Leary

The Mozilla platform team has a public weekly meeting where SpiderMonkey progress is (occasionally) discussed.

The js-engine mailing list is generally used for communicating with people who embed SpiderMonkey, such as asking questions and announcing API changes. No actual development happens on the list.

Code considerations

Repository

Most active work on SpiderMonkey is done in the mozilla-central branch of the mozilla repository.

Coding Style

For many years, SpiderMonkey was written in C, and is gradually moving to C++. We still avoid many features such as run-time type information and virtual functions, but have come around to the glory of templates and namespaces relatively recently. Read the Coding Style. Do NOT read the portability guidelines, which I will not link to, since they are very out of date.

Workflow

Everything goes through bugzilla. We find a bug or have an idea, then submit it to bugzilla, then file patches to solve it. When it is solved, the patch is reviewed by team members, and is then committed to the mozilla-central repository. A link to the commit is then added as a comment to the bug.

As well as committing to mozilla-central, we also have the option of committing to mozilla-inbound, which is automatically merged to mozilla-central by a Sheriff. This exempts us from watching tinderbox to check for errors in our patches, and commits are automatically backed out if errors are found.

Sample Workflows

(Outdated) Nicholas Nethercote: How I work on Tracemonkey

Policy

The following docs are for the Mozilla project, and differ ever so slightly from what you should do for SpiderMonkey. For example, you should find reviewers in #jsapi rather than #developers.

How to get commit access

How to make patches

Submitting a patch

Supported Platforms

.hgrc file

This .hgrc file contains a lot of the wisdom distilled through the wiki:

[extensions]
mq =

[ui]
username = First Last <email@domain.tld>
 
[alias]
qfulldiff = diff --rev qparent:.

[defaults]
diff = -p -U 8
qdiff = -p -U 8
qnew = -U
commit = -v

[diff]
git = true
showfunc = true
unified = 8

[paths]
try = ssh://email@domain.tld@hg.mozilla.org/try/

Try server

Often, you may be a little wary of breaking things. Mozilla has a Try Server where you can send a patch, and it'll be built and tested on tons of machines, which report to TBPL. Although you don't have access to TryServer just yet, you can get someone else to push it there for you. Just attach a patch to your bug, and ask in a comment, or on #jsapi. To be able to push to try-server yourself, you will need "level 1" access, which you can request once you've contributed a patch or two.

Mercurial Queues

Since most of our lives revolve around patches, and we use Mercurial, nearly everybody uses Mercurial queues.

Queues are based on the idea of patch management. Each queue consists of a series of patches, applied sequentially. The aim of the game is to split potential commits into simple, bite-sized chunks which are easy to review. This also makes it simple to experiment without polluting your existing work on a bug, to spin parts off into new bugs, and to rapidly apply and unapply patches (as demonstrated in the tutorial above).

Enabling

Add the following snippet to your .hgrc file to enable MQ

[extensions]
mq =

Example MQ workflow

Clone the repository to deal with a bug:

 hg clone http://hg.mozilla.org/mozilla-central

Initialize your queue:

 hg qinit -c

Create a new patch, with some name:

 hg qnew first_attempt

Work on the patch, try to fix the bug, test, compile, etc.

Refresh (save your work into the patch):

hg qrefresh

Repeat a few times.

Rename the patch (because your first name wasn't appliable):

hg qrename refactor_the_whatsit

Create a new patch to try a logically separate part of the same bug:

hg qnew rip_out_the_old_thing

Repeat the process a few times, until you have solved the problem. During this time, it can often be useful to go back and forth between patches:

hg qpop # unapply a patch
hg qpush # reapply a patch
hg qpop -a # unapply all patches
hg qpush -a # reapply all patches

Combine all the patches into a single patch, which you submit to bugzilla:

hg qpush -a
hg qdiff --rev qparent:. > my_wonderful_patch.patch

Commit the patches to your local patch repository, in case you make a mistake (You might do this periodically):

hg qcommit -m "Some message. Doesn't have to be good, this won't be committed to the repository, it's just for you"

Go back to the old patches and fiddle with them based on feedback:

hg qpop
hg qrefresh
etc

There are some more complex techniques that we won't go into in detail. You can enable only some patches using qguard and qselect, and you can reorder the patches by manually editing the .hg/patches/series file. But be careful!