There appears to be quite a bad memory management problem with Calibre (windows version).

I downloaded 0.8.64, set it up and left it running on my server. When I came back from holiday about two weeks later the memory usage had grown to over 650MB from an initial 70MB. This is with 50 books and doing nothing active.

On my return I installed 0.8.67. Again, initial memory usage is about 70MB and about every five seconds it grabs another 4K.

Are there any tools available which might help to get this issue laid to rest, perhaps something that could identify where in calibre.exe it happens? I know that it's fairly trivial but it would be nice to get it tidied up.

My guess: (I have never succeeded getting the development environment working so I have not looked at most of the code)
1) there is a polling process that gets fired off after a time (it does not (just) start at startup with no device attached. There are times it does start very close to startup.
2a) I suspect that private variable declaration is being done inside the loop and until the poll process ends, does not (never) goes to garbage cleanup

2b) There is a log, string (array) variable that gets appended rather than overwritten, continuing to grow each loop

@cybmole - you will get abuse if you make silly statements like that. Kovid can't replicate it, you can't fix what you can't replicate. It only happens on a minority of users systems, it certainly doesn't happen on mine.

If you *really* want the problem fixed, then just make the effort yourself to figure out what causes it. Its not rocket science, but it can be time consuming - and why should Kovid waste his previous time trying to guess if users aren't prepared to help themselves?

You don't have to be a software developer to do it. For instance try the following:

Put a clean install of calibre on a completely clean alternative machine - change no settings from defaults, and start with a simple library. Verify that it does or does not happen.

It only happens on a minority of users systems, it certainly doesn't happen on mine.

I'm not sure whether the threads on this topic actually give enough information to be sure that it's only a minority of systems. I guess that since it's such a minor issue that most users haven't checked. It certainly happens on my Windows 7 x64 installation.

I will try out some of your suggestions but on re-reading the original thread I think this may be something of a needle-in-a-haystack exercise and consume more time than it's worth investing.

@Agama - Over 6 million calibre users, probably 85%+ on Windows, and I see a grand total of 4 people reporting having a problem. Yes a lot of people won't notice if they do have it, but there are enough regular users visiting these forums you can be darn sure will have checked to see whether it also happens to them when it was first reported. I have Windows 7 x64 too with no issues - and if neither I, Kovid or any of the other "calibre regulars" are seeing the issue, then I call it a minority .

And yes it can be a needle in a haystack exercise - which is precisely why only people like yourself experiencing the problem can provide the input necessary to resolve it.

I'm not saying that there isn't a problem "somewhere" btw - but speaking as a software developer this sort of issue is almost impossible to resolve without *quality* input from the users. It could literally be almost anything as the cause - some misbehaving plugin, some feature of calibre that only a subset of users are using, some conflict with other software on your machine causing calibre code to go awry, issues within third party libraries that calibre uses and so on.

@kiwidude: Yes, you're probably right here. Interesting that some Win 7 x64 show the issue and others don't. This seems to point to something other than calibre.

With 4GB of memory on my machine, (and only 1.35GB in use), a 4K per 5 sec increase in memory is less than 3MB an hour. At this rate my machine will grind to a halt in ... Ah! It won't grind to a halt: it gets switched off regularly.

I won't be trying to track down the problem after all - it's not worth the effort.

1) This is not a problem that I personally can reproduce. That does not mean I disbelieve it exists for some people, just that I cannot fix it, since for this kind of thing the ability to reproduce is critical.

2) One developer, chaley, was able to reproduce it, but he could find no issues in calibre code, and both firefox and thunderbird showed the same behavior on his system, and he found that interacting with parts of his system unrelated to calibre caused the leak to stop, so he gave up.