Platform/InfallibleMalloc

This page will be read by people who cannot attend the meeting. Thanks for minuting carefully!

General Agenda:

• Discuss infallible malloc risks and payoffs.
• Propose next steps and assign ownership and allocate resources if necessary.

Meeting Details:

• To be held in Warp Core if available. Meet there.
• Wed, Dec 1, 2:30PM PDT
• 650-903-0800 x92 Conf# 8605 (US/INTL)
• 1-800-707-2533 (pin 369) Conf# 8605 (US)
• join irc.mozilla.org #planning for back channel

Jesse's notes from platform meeting:

• Infallible malloc (added to agenda by Jesse)
  • The problem, which has only recently come to light:
    • We haven't actually made malloc infallible yet. We already made new infallible for Firefox 4, but many data structures (including arrays and strings) can still misbehave on OOM because malloc can still return NULL.
    • Many exploitable OOM bugs are in system libraries rather than Firefox itself. Replacing malloc may be the only way for us to work around all of these bugs.
    • Security researchers have been reporting many OOM bugs lately.
      • We often can't reproduce what they see because we OOM slightly earlier or later.
      • This is a recipe for embarrassment when "old" testcases find "new" bugs, as Alex Miller discovered.
  • The plan of record:
    • Fix nsTArray
  • Proposal:
    • Make malloc infallible
    • Fix too-large-allocation crashes (distinct from OOM), which will be obvious in crash-stats.
      • Will require exposing a fallible malloc somehow
      • May require adding methods or variants to string classes, for example.
  • We must run GC and purge caches before giving up or we will have false positive OOMs when tons of memory is not actually live.
    • Need to monitor OS memory pressure and avoid getting close to these false OOMs.
    • This was part of E10s, is it ready for Firefox 4?
  • Damon will organize a smaller meeting later today to discuss this. Email him if you want to be part of that meeting.
• Crash reporter
  • integral part of any proposal: need to get dumps and backtraces from aborts to know where we've missed sites that should use fallible allocation
  • have evidence that suggests it might not be dealing well with low memory
    • bug 615798: No symbols from OOM-abort backtrace
    • bug 527095: Breakpad produces zero-byte or malformed dumps for some crashes
  • ergo, need to test breakpad in low-memory environments, and might need bug 493779, bug 587729