Performance/MemShrink/DMD

DMD (short for "dark matter detector") is a tool for that tracks which allocations have been reported by memory reporters. It's designed to help us reduce the "heap-unclassified" value in Firefox's about:memory page, and also detect if any memory is being counted twice.

There are two versions of DMD.

• The original is an external, Valgrind-based version; it was originally called DMD but is now called DMDV.
• The newer version is integrated into Firefox and so doesn't require Valgrind. It has taken over the DMD name, and hopefully it will eventually replace DMDV entirely.

DMDV

DMDV is built with Valgrind, so it's slow. You should run it on the most powerful machine you can.

It runs on Linux and Mac, but I get nonsensical results on Mac -- I suspect the stack traces are completely bogus.

Setting up Valgrind

First, get a trunk build of Valgrind:

 svn co svn://svn.valgrind.org/valgrind/trunk $VGDIR

where $VGDIR is where you want to put Valgrind's source and build it. (Valgrind doesn't allow a separate build dir.)

Apply the "DMD" patch from $MOZSRCDIR/tools/dmdv/dmdv.patch in the Mozilla repository:

 cd $VGDIR
 patch -p0 < $MOZSRCDIR/tools/dmdv/dmdv.patch

Then build DMDV:

 cd $VGDIR
 ./autogen.sh
 ./configure --prefix=...    # see below
 make
 make install

If your system already has a valgrind.h installed somewhere like /usr/include/valgrind/valgrind.h (as is the case on Ubuntu 11.10, for example), you can skip the "make install" step. (It needs to be from a fairly recent version of Valgrind, though, e.g. 3.6.1.) Otherwise (e.g. on Mac) you'll need the "make install" step; you can probably omit the "--prefix" option to ./configure.

Check it runs on a simple program:

 $VGDIR/vg-in-place --tool=exp-dmdv ls

You should get a start-up message and 'ls' should run.

Setting up Firefox to run with DMDV

Add these line to the mozconfig file of your choice:

 ac_add_options --enable-optimize='-O0'  # for better stack traces
 ac_add_options --enable-dmdv

Build Firefox with that mozconfig.

Running DMDV

Run Firefox under DMDV with a command line like this:

$VGDIR/vg-in-place --smc-check=all-non-file --tool=exp-dmdv <normal-firefox-invocation>

On Mac, you need --dsymutil=yes as well.

Firefox will run normally, but much slower (10--20x) than usual.

DMD doesn't do anything notable until you invoke it. You do this by entering

 javascript:DMDV()

as a URI. Note that Firefox doesn't let you do this directly in the address bar, so you have to either (a) create a bookmark for it, or (b) run it from the error console. The latter is discouraged because opening the error console allocates a lot of memory that will skew your results.

When you do that, it prints (after a pause) a lot of records like this:

 ==7163== Unreported: 335 block(s) in record 18 of 15052
 ==7163==  171,520 bytes (171,520 requested / 0 slop)
 ==7163==  0.40% of the heap (22.26% cumulative unreported)
 ==7163==    at 0x402A063: malloc (vg_replace_malloc.c:263)
 ==7163==    by 0x403C0A4: moz_malloc (mozalloc.cpp:113)
 ==7163==    by 0x7A439E8: PL_DHashAllocTable (pldhash.cpp:114)
 ==7163==    by 0x7A43D44: PL_DHashTableInit (pldhash.cpp:269)
 ==7163==    by 0x769D800: nsTHashtable<nsBaseHashtableET<nsCStringHashKey, nsCOMPtr<nsIVariant> > >::Init(unsigned int) (nsTHashtable.h:396)
 ==7163==    by 0x769D01B: nsBaseHashtable<nsCStringHashKey, nsCOMPtr<nsIVariant>, nsIVariant*>::Init(unsigned int) (nsBaseHashtable.h:99)
 ==7163==    by 0x769BD66: mozilla::storage::AsyncBindingParams::AsyncBindingParams(mozIStorageBindingParamsArray*) (mozStorageBindingParams.cpp:163)
 ==7163==    by 0x769E7E1: mozilla::storage::AsyncStatement::newBindingParams(mozIStorageBindingParamsArray*) (mozStorageAsyncStatement.cpp:380)

Each record tells you about live heap blocks allocated from a particular place in Firefox's code that aren't covered by memory reporters. All heap blocks that have the same stack trace are combined into a single record. (335 heap blocks are covered by this one report.)

Each record tells you the number of requested bytes in all the heap blocks, and the number of "slop" bytes (which are caused by jemalloc rounding up allocation requests). The slop byte calculation used is the one from jemalloc (even though jemalloc is disabled when you specify --enable-dmd).

The records are sorted so that the ones representing the most unreported bytes are first.

The default stack trace size is 8; you can adjust this with the --num-callers option. Many records that conceptually describe the same piece of code may be separate because entries low down in the stack trace differ; reducing --num-callers will cause these entries to be merged, which can be a good thing. On the other hand, sometimes you need a higher --num-callers value in order to get enough context to understand what a record means.

After the unreported blocks, DMD prints a similar list of records for each reported record. These records include the name of the reporter and the stack trace that shows where it was reported.

At the end of the output will be a summary of what was and was not reported:

 ==7163== SUMMARY:
 ==7163==   Total:         42,768,456 bytes
 ==7163==   Reported:      15,353,326 bytes ( 35.89%)
 ==7163==   Unreported:    22,591,866 bytes ( 52.82%)
 ==7163==   Suppressed:     4,823,264 bytes ( 11.27%)

You can also suppress some blocks, but that should only be done if you really know what you are doing.

If you invoke javascript:DMD() again, you won't get additional data; DMD currently only tracks enough information for the analysis to work once. This limitation could be lifted, but it would require non-trivial effort and I haven't needed it so far.

Note: don't open about:memory before invoking javascript:DMD(). If you do, you'll get lots of warnings from DMD about double-reported heap blocks.

Annotating Firefox

At this stage you're probably wondering how DMDV knows which allocations have been reported and which haven't. It only knows if Firefox tells it, and so Firefox must be annotated with this information. An annotation looks like this:

 VALGRIND_DMDV_REPORT(addr, len, name)

where 'addr' and 'len' specify a block of memory, and 'name' is a name associated with this reporting.

However, naked VALGRIND_DMDV_REPORT annotations only appear in the NS_MEMORY_REPORTER_MALLOC_SIZEOF_FUN macro. This macro is used to create a function with type nsMallocSizeOfFun which can be used in memory reporters. See Platform/Memory_Reporting for more details.

Not all Firefox memory reporters are currently annotated, which means that DMDV's output will list allocations that it should not. nnethercote is gradually rectifying this situation.

Troubleshooting DMDV

This page has more info about using Valgrind on Firefox, which may be useful if you have trouble. Otherwise, contact Nick Nethercote ("njn" on IRC).

DMD

The integrated version of DMD is currently being developed. Follow along in Bug 717853 if you are interested.