Ramblings from the mind of Stuart Parmenter

Leaks? Memory? We never forgot about you.

I’ve seen quite a few posts lately based on the memory fragmentation work that I’m doing with titles such as “Fixing Firefox’s memory issue becomes a priority.” Others have claimed that this work is a result of Mozilla’s new focus on mobile. While I’m glad that people are paying attention to our memory work and offering great suggestions, let me say: Memory issues have always been a priority.

Since I started working on the project in 1998 we’ve always had a focus on keeping our memory footprint small and keeping leaks to a minimum. Early in the development cycle for each release we’ve set goals for memory and performance. We always set our bar under the previous release. I’ve found that developing desktop software is a pretty constant balancing act between performance and memory use. We’re always making trade-offs and we try our best to chose the things that will work best for the largest set of users:

Some examples:

Back in 2001 when I rebuilt our imaging library, I made several decisions to use more memory to store images results in faster rendering; optimizing for memory use reduces the speed at which was can display pages. We’ve looked at these issues many times over the years to make sure they were still correct. Recently we’ve adjusted that behavior to not keep full uncompressed images around as long which will result in memory savings but will cause initial scrolling to be a bit slower on documents you haven’t accessed in a while.

In Firefox 1.5 we added a feature called the back/foward cache which keeps documents in your recent history that you’ve navigated from in memory. This was done to significantly speed up hitting the back button. It worked great but caused us to use a bit more memory. We made sure that it was only using memory your computer wasn’t already using, but again, it’s an example of a trade-off. We started off with a pretty high number of pages that we kept in the cache and have continued to adjust that number to keep a more limited set of pages to help prevent unnecessary bloating. We’ve also started expiring these pages over time so that you don’t keep pages around you probably aren’t going to use.

With the popularity of extensions rising, we started hearing complaints about memory leaks. We took these reports pretty seriously and have spent a lot of time investigating what is going on. A lot of work has gone in over the last few years to reduce these leaks. Most of our early testing was around the browser, without extensions. Our investigations showed that certain extensions in caused a pretty bizarre class of leaks that were pretty difficult to fix given the architecture in Firefox 2. We fixed as many of them as we could in Firefox 2 but some we were unable to fix. In Firefox 3 some Really Smart People (graydon, peterv, dbaron, etc) have built this thing called the cycle collector in to the core which addresses many of the leaks that we were seeing from extensions (as well as leaks from other places that were of the same class). Our extensive testing shows an occasional leak here and there and we are working to fix those, but in general we aren’t seeing many leaks anymore.

It is only after we’ve gone through so many leak fixes and done so many other memory reduction fixes that we’ve needed to a deeper look at what is going on under the hood. We’ve long had suspicions that we were being hurt by memory fragmentation, but it wasn’t until recently that we had built good tools to fully diagnose the problem.

I’ll assert here that the way people use their browsers has changed. When Gecko was originally designed back around 1998 people had one, maybe two browser windows open without tabs, and they certainly didn’t have any extensions installed. I look at my browser windows now and I’ve got 3 browser windows open with a total of about 20 tabs open. That is 10x the number of documents open at once!

With the change in how people use their browsers, there is no doubt that they’re going to use more memory. We’re doing everything we can to minimize the impact of having lots of documents open. Many people are trying to shave off bytes here and there. Just in the last week we’ve removed over 200 thousand allocations just from startup and first page load. We’ve got a great community and people eager to solve these problems. We’re now equipped with data and ready to fight this battle.

Post navigation

28 thoughts on “Leaks? Memory? We never forgot about you.”

Nothing personal, but if FF has been concerned with memory usage ever, you sure couldn’t tell. People, AFAIK, don’t complain about memory footprint because they have a bunch of tabs open; they complain because they have a bunch of tabs open, close them, and memory keeps getting used up.

I have to reboot ff almost daily to keep memory usage under 500MB. And once it hits that 500MB, I can close all tabs but one, navigate to something simple like google.com, and it *still* holds on to 500MB of VM. Color me unimpressed.

While the intensity of your work around memory problems may not have changed recently, the frequency of blog entries about this topic within the Mozilla community certainly has. I’m not an active contributor to Mozilla, but posts like your past ones (or the ones about architectural changes towards Mozilla 2) are highly worth reading even for outside developers.

Earl: Right, and I’m saying that is due a lot to fragmentation. Given the amount of fragmentation shown in my earlier post, we’re so fragmented that it’ll be hard to reduce the space and do new allocations earlier. With the things we’re working on now this should be significantly better.

The number of posts on this topic certainly has increased as of late, and like many others I have enjoyed them. I just wanted to give a few suggestions for future posts on this topic:

1) How many improvements are likely to make their was into FF3.
2) Pretty graphs with Google’s tcmalloc.
3) A search of long outstanding bugs that may have a generic keyword like footprint or incorrectly labeled mlk that would be more appropriately labeled as “fragment” bugs.

As one of those who has been applauding the recent awakening in focus on the memory/performance issue I have to say that maybe the issue has never been forgotten but there’s a big diff between remembered in the back of everyone’s mind and a priority.

I commented against Mitchell’s latest fluff post that the performance/memory bloat must be fixed now. A real leader – especially one on half a mill – should not be letting this critical issue just meander along in the back of developer’s minds.

Whatever has prompted this latest upsurge in interest about the memory/performance bloat issue, any genuine leader should pounce on that interest and decide to fix the nightmare once and for all.

That people are still experiencing the type of scenarios outlined by Earl is just a disgrace. It really is a joke considering the resources now available to mozilla and the length of time this problem has lasted.

On a more constructive note, I gave the builds with low fragmentation a run. Short, relatively informal testing did indicate what you seem to suggest – more memory used initially, but more memory freed up more often. For example I saw a 20 megabyte drop at one point, or 25% of memory used, by simply closing a tab with a heavy focus on Flash content.

On the other hand, RAMback didn’t seem to do very much at all.

Keep up the great work.

Can I donate to support your work if Mitchell will not pay you enough?

pd, basically anyone talking about “the memory/performance bloat issue” is confused. There are multiple issues. They all need to be addressed. They would include leaks(of which we have pretty few now, so we’re focusing on some of the other classes of issues), holding on to memory for the lifetime of the app (not the same as a leak in terms of identifying and debugging, but the same as far as users are concerned), fragmentation issues (again, not a single issue but a whole class here), data structures being bigger than they need to be, and issues with the allocator not releasing memory back to the OS even when it could (a problem with some allocators).

There is no magic “fix this and it gets better” fix that people are somehow hiding or not finding or whatnot. There’s just a lot of work that needs to be done with instrumentation, profiling, and fixing the hundreds (if not more) of separate issues that contribute to high overall memory usage.

And I don’t see what Mitchell’s salary has to do with any of this, frankly.

I don’t think I’m confused or even using an inappropriate generalisation (the memory/performance bloat). I do think that I could explain that I am aware this issue is by no means non-trivial. I should also state that I am confident there are a lot of people doing a lot of relatively tedious fine grained work on the problem(s).

However I’m not aware to what extent this fundamental issue is being addressed and my comments are merely aimed at trying to express the level of importance I think should be applied to this issue.

Unfortunately reading Planet Mozilla I tend to see a fair bit of work being done on bits and pieces like one-click bookmarking which seem, to me, to be of little importance compared to rock-solid reliability and performance. On the flipside I agree that some visual and experience elements of any software need to be developed in order for users who cant see improvements in the ‘back end’ to see (literally) anything worth a new version number.

As I posted on Shaver’s blog just now, I think the best approach to resolve frustration from those who don’t see a clear direction/progression/leadership/focus from ‘the Mozilla development team’ would be a regularly updated collective report or newsletter on the devmo wiki or somewhere.

In short, when I discuss Firefox with people, all some of them want to know is when the overall bloat/memory/performance issues are going to be fixed. I know there may not be a single simple answer to that question, but to the layman out there, I should be able to refer to something other than irregular blog posts loosely focusing on the subject from individual developers.

pd: “Looks like some interesting new features in Firefox on the way”
colleague: “have they fixed the memory issue yet?”
pd: “well memory is just one issue … reliability has improved, performance as well (rendering speed, etc) …”
colleague: “yeah but have the fixed the memory issue yet?”
pd: “are you still having problems”
colleague: “yeah I still get occasions where it eats up a truckload of RAM making my machine slow to a crawl”
pd: “ah shit, that sucks … I wish I had an answer for you”

>>>I’ve got 3 browser windows open with a total of about 20 tabs open. That is 10x the number of documents open at once!

That is so little as to astonish me. At times I’ve had over 40 tabs in one window. And then had the tabs freeze. Everything grind to a halt so that I couldn’t minimize or even close the browser. Sometimes calling up Task Manager will “wake up” Fox. Other times, I just have to nuke the process and send my dendritic collection of linkages to Electron Hell. That. Is. Not. Fun.

And believe me, based on exchanging emails with others, my 40 tabs is now considered a baby step! Why don’t you ASK people how many tabs they have open? Don’t guess. Get some real data to work with!

I’m using Fox 2.0.0.9 with Download Helper extension, Video Downloader extension, and some damned extension I remved that has left behind a gnarly graphic in the icon bar next to the URL Address field. (Oh, I see it’s Subtile, apparently left behind after that extension. And no instructions on how to get rid of the damned thing!)

pd, the people working on things like one-click bookmarking generally aren’t the same people that’ll be making changes to things like memory allocation.

Unless you’re suggesting that every single Mozilla engineer drops what they’re doing to 100% solve memory fragmentation and that Firefox 3.0 should be delayed until this is the case, I don’t know what you think Mozilla can be doing better.

Blog posts on memory problems might have been a bit sparse, but following Bugzilla and nightly builds shows a different story, it’s been highly visible work even prior to the 1.5 release.

pd, anyone who wants to can look at the checkin logs and create a “what’s going on” list based on that. Furthermore, there are public Gecko and leak meetings, both of which post meeting summaries weekly.

So it seems to me that the information you are looking for is already there….

Monchester: His extension really just slows down Firefox by forcing it to page memory out and then right back in. It doesn’t really free your memory and certainly won’t make Firefox any faster after a long period of time. I wouldn’t recommend anyone actually install his extension. We tried this approach a long time ago and found that it had a negative impact. On Windows by default minimizing an application does something similar to what his extension does — we disable that because it can sometimes take quite a long time to become restored.

iNsuRRecTiON: Given that Opera isn’t open source, it is hard to say why they are sometimes smaller than Firefox. My guess would be that their primary focus on mobile has taught them many things and that they’ve been able to use what they’ve learned with their desktop browser. They’ve probably done a bunch of the work that we’re looking at doing now. That said, there are plenty of examples where Firefox is smaller than Opera after loading some set of pages.

well.. I think that you are trying to justify yourself and your company, the truth is that firefox does an awfully job handling memory and it would be more respectfully for you and your company just admit it and say that you made a mistake.

1 thing I really like in IE over firefox is the ability to have multiple process instances of IE running at the same time. When I used to use IE, I used to have >40 IE windows at the same time (horrible without tabs), but I never had memory problems. It’s because I open new windows (and IE instances) and close existing windows, so any memory leaks or fragmentation goes away as I browse.

Firefox has made my habit worse, I routinely have >100 tabs open across multiple windows. But the fact that I could not have multiple instances means that I have firefox running almost all the time. Over time firefox memory usage will grow until firefox starts becoming unresponsive at around 700MB, when I’m forced to close everything and reopen firefox (which is painful). Viewing images on firefox is especially bad, almost ballooning memory usage immediately.

Now I know that the reason why multiple instances cannot be done currently is because of writing to the profile. I remember there were some articles about profile sharing, but I guess that didn’t go anywhere. Although I’m not sure what’s the big problem there; relational databases has been handling concurrent writes for a long time, and a profile is basically a database.

Sorry if I went on a tangent. It’s good to hear that memory fragmentation is being handled better in version 3, so that should alleviate my problems quite a bit.

My goodness, some of these people are dumb (no offence). What kind of benefit do you get from having 100+ tabs open at once? The architecture (Windows, DOM, HTML, …) just isn’t designed for that kind of thing. Just from the GUI perspective I find it offensive!
For your information, I run FF 2.0.0.10 on a 256MB RAM PC at work, and it is a little slow – but acceptable – with AdBlock, Netcraft and McAfee SiteAdvisor addons (security is a thing for me, as you can tell). However, I wouldn’t try more than 8 tabs without some more RAM. At home I have a Core2Duo 2GB RAM PC and have no problem with 12 tabs. But – and this is the big BUT – I will usually avoid having more than that. Why? Because it clutters up my screen, and there’s much more efficient ways of working/browsing than loading up a ton of pages at once.
Instead of having to scan the screen every time you want to switch to another tab try the following:
1) Use a feed reader (Google Reader is a good example – just 1 FF tab) to aggregate all your latest news etc into one place – no more trawling dozens of websites for news.
2) Focus on just a few websites at a time. Probably 8 is too many. Try just 3-4 – you might find you’re more productive as a result! Fact – we can only take in a few items of info at a time – can’t read 20 tab titles in one glance. More importantly we can’t think about more than 2 things at a time – don’t tell me you think about the other 3 news articles on the other tabs when viewing the current one?
3) Use keyboard shortcuts – e.g. rather than middle-clicking everything you can find, just middle click a couple of them, and use CTRL+Tab to switch between them – then switch back to the original tab to middle-click a couple more links
4) When you are getting too many tabs (i.e. 12 of them) and you don’t know which ones to close, use CTRL+Shift+D (or Bookmarks > Bookmark all tabs) to save your session for later. Then find the most important tab and use Close Other Tabs. Usually I don’t even bother bookmarking, as when I think about it the other stuff really isn’t that important!
5) If you bookmark a block of sites, you can then use “Open All in Tabs” to load them simultaneously to get back to where you left off. However, don’t try this with 100 tabs as you will be sat there all afternoon waiting for the sites to download!
6) Manage your expectations. Modern websites aren’t designed with RAM usage in mind. If you really want less memory usage, then turn off images and Flash etc. Due to the architecture of HTML/XML (inefficient scripting languages, dependency on text rather than binary data, etc) it is usual for a single webpage to take anything from 10-60MB depending on number of images. That means use a reasonably modern PC if you love 20-30 tabs, and for any more you need a blazingly fast Front Side Bus and DDR3 (I believe that’s the bottleneck when an application uses over 1GB RAM – correct me if I’m wrong!). No magic bullet browser is going to solve these problems.
7) If you really need 100 websites at a time, you should probably be using a plain text web browser (Lynx – for ‘multi-tabs’ use XTerm console session tabs!).
8) Windows probably isn’t designed to handle 100s of HTML pages in a secure browser. The GUI doesn’t make it easy for a start! Perhaps someone needs to develop their own OS for the single (stupid) task of filling main memory with 1000s of web pages and flicking quickly and easily between them – how’s that for a PhD anyone…? Personally I would use Bookmarks!!
No axe to grind by the way – I’m just an IT worker who uses IE7 more often than FF, and I’ve never even tried a FF beta.

Yes, anyone who doesn’t follow my way of doing things is dumb. I can call you dumb too, what do you think? Saying no offense while calling people dumb is self-contradictory.

The problem as stated is not even because of having too many tabs opened. I can open 100 tabs and memory usage is fine initially. The problem is the longer firefox is used, the more memory it consumes. This can happen with 1 tab or 100 tabs, it’s just a matter of (usage) time. As explained, it’s because of memory fragmentation, which some developers are trying to reduce.

It’s also stated that it’s very hard to completely eliminate memory fragmentation, which as a software developer myself, I do agree. It’s just that I prefer the inevitable restart of firefox to be less painful. Maybe I’ll try to fix it myself if it bugs me enough to overcome my laziness =p

Just a quick post back to Pavlov for posting on my blog. I think it’s a memory fragmentation issue as well. My usage starts fine but leave it running for a while and BAM it just starts creeping upward. I just un-installed all my extensions and themes and will see how things look.