MemShrink progress, week 31

Necko Buffer Cache

I removed the Necko buffer cache. This cache used nsRecyclingAllocator to delay the freeing of up to 24 short-lived 32KB chunks (24 x 16KB on Mobile/B2G) in the hope that they could be reused soon. The cache would fill up very quickly and the chunks wouldn’t be freed unless zero re-allocations occurred for 15 minutes. This idea is to avoid malloc/free churn, but the re-allocations weren’t that frequent, and heap allocators like jemalloc tend to handle cases like this pretty well, so performance wasn’t affected noticeably. Removing the cache has the following additional benefits.

nsRecyclingAllocator added a clownshoe-ish extra word to each chunk, which meant that the 32KB (or 16KB) allocations were being rounded up by jemalloc to 36KB (or 20KB), resulting in 96KB (24 x 4KB) of wasted memory.

This cache wasn’t covered by memory reporters, so about:memory’s “heap-unclassified” number has dropped by 864KB (24 x 36KB) on desktop and 480KB (24 x 20KB) on mobile.

I also removed the one other use of nsRecyclingAllocator in libjar, for which this recycling behaviour also had no noticeable effect. This meant I could then remove nsRecyclingAllocator itself. Taras Glek summarized things nicely: “I’m happy to see placebos go away”.

Last week I mentioned bug 703427, which held up the possibility of a simple, big reduction in SQLite memory usage. Marco Bonardo did some analysis, and unfortunately the patch caused large increases in the number of disk accesses, and so it won’t work. A shame.

Quote of the week

a year ago, FF’s memory usage was about 10x what chrome was using in respect to the sites we were running…

so we have switched to chrome…

i just tested FF 9.0.1 against chrome, and it actually is running better than chrome in the memory department, which is good. but, it’s not good enough to make us switch back (running maybe 20% better in terms of memory). a tenfold difference would warrant a switch. in our instance, it was too little, too late.

If you don’t have anything constructive to say, don’t say anything at all.

When you feel the need to verbally piss into the wind, open a text file, not a web browser.
Criticizing is not contributing. It baffles me why you come here. Your negativity has only proven toxic.
Either try to be more positive and constructive or just, please, go away.

I guess it’s not surprising that people have a hard time switching back. There’s a pretty big opportunity cost to switching in terms of getting your browsing history and add-ons into a new browser. That being said, it is annoying to hear people who switched previously bash Firefox for issues that no longer exist. I guess all we can do is continue to make improvements and make a better browser.

Each nsRecyclingAllocator had a timer, and so timers have been removed, not added. We had at most two nsRecyclingAllocators alive at any one time. I expect the removal of at most two timers is negligible in terms of performance or memory consumption.

If i do understand correctly, memshrink doesn’t use compression to reduce memory usage.
I was wondering why. Is that out of scope ? or technically impossible ?

Note : I understand that compression is only applicable to “cached data”, not to “live” data currently being processed. But then, my understanding is that Firefox has a hell of lot of cached data, typically for background tabs, or even undisplayed elements of current tab…

Probably too much of a performance hit. Compression for data stored on an HD isn’t generally a problem any more, but DRAM is ~10,000 times faster. I doubt it’d be feasible to do it without a major performance hit.

However, there are 2 catches :
– Compression only works on “large enough” data blocks. Large enough is about a few KB.
– You have to decode data before using it, hence it is “twice in memory” while using it. Therefore, you want your data blocks to remain “small enough”. Small enough is presumably below 1MB.

Hence, there is a big difference between cached data (stored for later use), and active data (being used).
However, i’m not sure if it is within Memshrink requirement list to make a difference between cached data and active data.