MemShrink progress, week 105–108

This is the first of the every-four-weeks MemShrink reports that I’m now doing. The 21 bugs fixed in the past four weeks include 11 leak fixes, which is great, but I won’t bother describing them individually. Especially when I have several other particularly impressive fixes to describe…

Lazy Bytecode Generation

Brian Hackett finished implementing lazy bytecode generation. This change means that JavaScript functions don’t have bytecode generated for them until they run. Because lots of websites use libraries like jQuery, in practice a lot of JS functions are never run, and we’ve found this can reduce Firefox’s memory consumption by 5% or more on common workloads! That’s a huge, general improvement.

Furthermore, it significantly reduces the number of things that are allocated on the GC heap (i.e. scripts, strings, objects and shapes that are created when bytecode for a function is generated). This reduces pressure on the GC which makes it less likely we’ll have bad GC behaviour (e.g. pauses, or too much memory consumption) in cases where the GC heuristics aren’t optimal.

It’s obvious that Tree Style Tabs is taking up 2.7 MB. What about the other two entries? It’s not immediately obvious, but if I look in about:support at the “extensions” section I can see that they are AdBlock Plus and ChatZilla.

If you’re wondering why those add-ons are reported as hex strings, it’s due to a combination of the packaging of each individual add-on, and the fact that the memory reporting code is C++ and the add-on identification code is JS and there aren’t yet good APIs to communicate between the two. (Yes, it’s not ideal and should be improved, but it’s a good start.) Also, not all add-on memory is reported, just that in JS compartments; old-style XUL add-ons in particular can have their memory consumption under-reported.

Despite the shortcomings, this is a big deal. Users have been asking for this information for years, and we’ve finally got it. (Admittedly, the fact that we’ve tamed add-on leaks makes it less important than it used to be, but it’s still cool.) This will also be in Firefox 24.

b2g

Gregor Wagner has landed a nicecollection of patches to help the Twitter and Notes+ apps on B2G.

While on the topic of B2G, in today’s MemShrink meeting we discussed the ongoing problem of slow memory leaks in the main B2G process. Such leaks can cause the phone to crash or become flaky after its been running for hours or days or weeks, and they’re really painful to reproduce and diagnose. Our partners are finding these leaks when doing multi-hour stress tests as part of their QA processes. In contrast, Mozilla doesn’t really have any such testing, and as a result we are reacting, flat-footed, to external reports, rather than catching them early ourselves. This is a big problem because users will rightly expect to have their phones run for weeks (or even months) without rebooting.

Those of us present at the meeting weren’t quite sure how we can improve our QA situation to look for these leaks. I’d be interested to hear any suggestions. Thanks!

about:addons-memory seems more accurate in one case (see below about https-everywhere). Both implementations are by Nils Maier. I do prefer looking at the built-in interface, since it shows everything and the add-ons I keep are low on the list (though they are easily actionable, unlike a lot of the report).

https-everywhere does have a compartment (…/components/https-everywhere.js), but it doesn’t get reparented to the add-on hierarchy. After a bit of testing I see it’s because I’m running it from git (https-everywhere@eff.org is a plain file containing the path to the git clone).

I’m not a developer and I don’t know what’s possible or not possible, so I’m just going to throw out stupid stuff that would hopefully spur thinking…

You said you guys weren’t “quite sure how we can improve our QA situation to look for these leaks.” I was going to say that our partners found them, so let’s do whatever they did. But did you mean that they merely observed that there *were* leaks? Not really identifying the roots?

Say, it wouldn’t be possible to emulate the hardware and then watch/record everything, would it?
I was thinking the other day that maybe it’d be interesting/helpful if I could create a virtual machine in VirtualBox for B2G hardware.
Might be nice if you could just freeze everything and then take a sort of snapshot, then wait a bit and do it again. Would that even be helpful? If so maybe it could be build it into the OS itself.
Perhaps what’s necessary would seem extreme so it’s hard to think of. Like, maybe attaching a secondary storage to a device and writing out massive logs.

Thanks as always for the update
It’s particularly good news that some of the big items will be in FF24 as that will be the basis of the next enterprise release. There will be a lot of people using that release until the middle of next year.