If this is your first visit, be sure to
check out the FAQ by clicking the
link above. You may have to register
before you can post: click the register link above to proceed. To start viewing messages,
select the forum that you want to visit from the selection below.

WinForms Memory Leak

I'm trying to get to grips with how Windows Forms applications manage memory allocation. I'll give you an illustration of the problem. Take this simple winforms app which is a main form with two buttons: one that opens a form containing some random data, and another button that closes all open forms (except the main form)

If you run this and click away opening lots of forms, 50 at a time, 100 at a time, and then closing them all, the memory goes up and down over time but gradually gets higher and higher, never returning to anything like its initial state. Now I assume part of this is the .NET runtime being clever and assuming that as the app needed a lot of memory previously, it will need more again and better to keep some allocated (I am running this on a machine with 4GB RAM).

You can see the memory dropping at intervals (I've marked some of them above) which is good but it still slowly has a minimum that creeps up and up.

This is a very basic illustration of a much bigger production issue with a huge application with 100,000s lines of code and users that keep the app open all day. Over time they open and close a lot of forms and the memory creeps up.

Is there any way to force the allocation down? I know I can force GC but this won't do anything as it's the memory post-GC that is remaining high as far as I can see.

This isn't a problem with references as far as I can see because I have spent some time experimenting with just one form in the main application opening and closing it and there are no references after it is closed and disposed and yet the memory still creeps up in a similar way to that logged above. Also if it was a problem with references, I'm not sure how that would explain my test app above.

Re: WinForms Memory Leak

You RandomDataHelper class has a Shared List of RandomData objects. A shared object like that list can't go out of scope while the program is running, because it belongs to the class not to an instance. So every time you instantiate a GridForm, the call to RandomDataHelper adds another 10,000 RandomData items to the list. I wonder if that is the cause of your memory leak?

Maybe you could fix that by using List.Clear before adding the new items, although a better design could be to get rid of the Shared keywords in RandomDataHelper. Then you could instantiate and dispose of it every time you need it, for example with a Using block in GridForm.Load:

Code:

Using RDH As New RandomDataHelper
_randomData = RDH.GetRandomData()
End Using

Re: WinForms Memory Leak

What is the ultimate problem? Does the program crash, or are you simply disturbed by the memory usage?

After all, you made a key point early on when you suggested that part of this was .NET cleverly recognizing that the program used lots of memory before and might do so again. That's not necessarily .NET, but it is effectively correct. If the system has lots of RAM, any application can keep asking for more, and the OS will provide...up to a point. The OS can also recover RAM, but it isn't going to do that without sound reason. If there is no other demand on the RAM, why bother recovering it? What would even be the trigger for such a thing?

This kind of question used to come up more often, and the basic answer is: Unless it is causing trouble, ignore it. Memory is managed for efficiency, not for aesthetics. This means that there will be times when it will appear to be using more and more memory simply because it is easier to give the app more memory than to re-manage the memory the app already has.

Re: WinForms Memory Leak

Originally Posted by boops boops

You RandomDataHelper class has a Shared List of RandomData objects. A shared object like that list can't go out of scope while the program is running, because it belongs to the class not to an instance. So every time you instantiate a GridForm, the call to RandomDataHelper adds another 10,000 RandomData items to the list. I wonder if that is the cause of your memory leak?

Maybe you could fix that by using List.Clear before adding the new items, although a better design could be to get rid of the Shared keywords in RandomDataHelper. Then you could instantiate and dispose of it every time you need it, for example with a Using block in GridForm.Load:

Code:

Using RDH As New RandomDataHelper
_randomData = RDH.GetRandomData()
End Using

BB

Thanks for the reply. I don't think this is the cause because of the massive amount of memory that is sometimes cleared. The static (shared) class was to keep the GUIDs in memory and generate lists from them because generating GUIDs every time took too long. I should really do it properly and read from a file or a database to eliminate this but I wanted to exclude unmanaged resources. I will make some changes and see what effect they have

Re: WinForms Memory Leak

Originally Posted by Shaggy Hiker

What is the ultimate problem? Does the program crash, or are you simply disturbed by the memory usage?

After all, you made a key point early on when you suggested that part of this was .NET cleverly recognizing that the program used lots of memory before and might do so again. That's not necessarily .NET, but it is effectively correct. If the system has lots of RAM, any application can keep asking for more, and the OS will provide...up to a point. The OS can also recover RAM, but it isn't going to do that without sound reason. If there is no other demand on the RAM, why bother recovering it? What would even be the trigger for such a thing?

This kind of question used to come up more often, and the basic answer is: Unless it is causing trouble, ignore it. Memory is managed for efficiency, not for aesthetics. This means that there will be times when it will appear to be using more and more memory simply because it is easier to give the app more memory than to re-manage the memory the app already has.

Thanks for this. I will reply to this in some more detail a little later - rushed for time now

Re: WinForms Memory Leak

Originally Posted by Shaggy Hiker

What is the ultimate problem? Does the program crash, or are you simply disturbed by the memory usage?

After all, you made a key point early on when you suggested that part of this was .NET cleverly recognizing that the program used lots of memory before and might do so again. That's not necessarily .NET, but it is effectively correct. If the system has lots of RAM, any application can keep asking for more, and the OS will provide...up to a point. The OS can also recover RAM, but it isn't going to do that without sound reason. If there is no other demand on the RAM, why bother recovering it? What would even be the trigger for such a thing?

This kind of question used to come up more often, and the basic answer is: Unless it is causing trouble, ignore it. Memory is managed for efficiency, not for aesthetics. This means that there will be times when it will appear to be using more and more memory simply because it is easier to give the app more memory than to re-manage the memory the app already has.

OK, thanks again for the reply and the information. The code posted above is a bit contrived but in the production environment we get some users who have the win forms app open all day and in constant use and it involves a lot of opening and closing of forms with large grids and a lot of data.

It is a problem for them and .NET slows down and usually throws an IO out of memory exception. We can see that their memory creeps over time by monitoring using perf mon and also logging using GC.GetTotalMemory.

I stripped down one form in the app and then opened it programmatically multiple times to push the memory up, and then closed all the open forms in a similar way to that seen above. The memory does get freed up but despite this, over time, it slowly has a minimum level that creeps up and up.

I'm pretty sure if I ran that test app above for long enough I would see a similar thing but I can't find any evidence of references that are being held onto - and if there were references you would expect the memory never to come down at all until the whole app was closed. What we see is up-down-up-down trend where the "down" gets higher and higher over time and we end up with memory pressure. These users have high spec new machines, 4GB/6GB RAM and the app starts out about 20MB and creeps up throughout the day (with some "downs") to about 1.2GB and this is usually when they start to get issues (obviously depending on what else they are doing)

I hoped to write a test app to prove that there was something wrong with the production app and that just opening and closing data-rich forms in a winforms application shouldn't push up the memory over time but I seem to be showing that this happens with even the most basic applications if you do enough opening and closing of forms.

The minimize window thing is only temporary and as soon as the app has the focus again the memory jumps back up to where it was before you minimized.

Ideally, what I would like is to tell the runtime never to use more than 500MB and if it ever reaches this level, to release the allocation (or just the 500MB or already has reserved because there should be no way the app ever needs this much at any point in time). I can't seem to do anything within the application to prevent it creeping up over time but I'm open to suggestions.

Re: WinForms Memory Leak

Where do you feel the problem lies? Do you feel that there is a flaw in your program that isn't releasing the memory, or do you feel that there is something inherently wrong with .NET that causes a slow, steady, memory leak? I would say that this is more of a gut feeling kind of thing rather than a logical conclusion, and that's all I'm asking for. The choice you make seems likely to influence the direction you take with this. You appear to be looking for an external, .NET method that will fix what sounds likely to be a design problem. If the real problem comes from resources not being freed, then placing a cap of any sort on the maximum memory usage would only serve to crash the program sooner rather than later. The only way that seems like a viable solution is if you feel that .NET is just being selfish and that if you had the right tool to limit it, it would mend its ways.

As far as I can see, the problem, while it may be VERY tricky, is one of something not being released properly such that some block of technically unused memory is still theoretically accessible, and therefore the GC will never release it. Where that would be in a large app I can't say. Fortunately, you have a simple test app that shows the same problem (or at least a similar manifestation), so you can tinker around with that.

Re: WinForms Memory Leak

Originally Posted by Shaggy Hiker

Where do you feel the problem lies? Do you feel that there is a flaw in your program that isn't releasing the memory, or do you feel that there is something inherently wrong with .NET that causes a slow, steady, memory leak? I would say that this is more of a gut feeling kind of thing rather than a logical conclusion, and that's all I'm asking for. The choice you make seems likely to influence the direction you take with this. You appear to be looking for an external, .NET method that will fix what sounds likely to be a design problem. If the real problem comes from resources not being freed, then placing a cap of any sort on the maximum memory usage would only serve to crash the program sooner rather than later. The only way that seems like a viable solution is if you feel that .NET is just being selfish and that if you had the right tool to limit it, it would mend its ways.

As far as I can see, the problem, while it may be VERY tricky, is one of something not being released properly such that some block of technically unused memory is still theoretically accessible, and therefore the GC will never release it. Where that would be in a large app I can't say. Fortunately, you have a simple test app that shows the same problem (or at least a similar manifestation), so you can tinker around with that.

I assume it's with our app somewhere 100%, not generally in winforms. I was just surprised and confused by the memory graph in a such a small basic test app. Fair enough that the timing can seem random but great that memory is reclaimed and I know memory allocation is a dark art. What makes no sense to me is the lower bound trend upwards over time. I will tweak the test app and remove the static class to rule this out. I've used a .NET Memory Profiler in the past so may revisit this but it's excruciating because of the amount of data you have to sift through

Re: WinForms Memory Leak

The lower bound trending upwards is certainly disturbing, but, as you say, memory allocation is a dark art, so it isn't entirely implausible. I would suggest getting a good understanding of how the Garbage Collector works in .NET. It's kind of an interesting topic, though I know of no single source to suggest. The key, in my mind, is understanding how the GC recognizes that some object is unreachable. It seems to me that the most likely problem here is that some resource is unreachable in your mind (you certainly believe it to be discarded), yet the GC object map suggests that the item really isn't unreachable, so it never feels safe in recovering that memory. Of course, that can be quite the subject in itself.

Re: WinForms Memory Leak

Originally Posted by sdfax_work

I've used a .NET Memory Profiler in the past so may revisit this but it's excruciating because of the amount of data you have to sift through

Actually it is really easy to find leaks using a profiler. The thing to look at is delta, you can see the difference between created and removed, the ones that are not removed have a really high delta, and those are the ones that are causing your leak. Open your base form, take a snapshot to get a baseline, open loads and loads and loads of forms, leaving them open take another snapshot, close them all back to the base form, take another snapshop. Diff the first and last, they should be similar. A decent profiler will show you the link to the object and code that is keeping it referenced.

Re: WinForms Memory Leak

Originally Posted by sdfax_work

It is a problem for them and .NET slows down and usually throws an IO out of memory exception. We can see that their memory creeps over time by monitoring using perf mon and also logging using GC.GetTotalMemory.

If it's throwing an OOM exception, that means that GC is unable to collect "unused" memory because a reference to it exists still. (That's why unused is in quotes).

If there was truly unused memory, the GC would kick in before allocation and make it available for use.

Originally Posted by sdfax_work

I hoped to write a test app to prove that there was something wrong with the production app and that just opening and closing data-rich forms in a winforms application shouldn't push up the memory over time but I seem to be showing that this happens with even the most basic applications if you do enough opening and closing of forms.

If you are allocating memory, your memory usage will go up. It will only go back down again when GC kicks in. If you don't allocate enough memory that you trigger a GC, then that memory will never be reclaimed until you exit the application. This is not a problem. However, your app is hitting those thresholds, which is why you see the dips in your memory usage.

As to the "minimum memory" creeping up: this I would hazard a guess is due to some objects surviving a garbage collection and being promoted to Gen1 objects. The .NET GC is a generational GC algorithm. It has three generations (Gen0, GEn1 and Gen2). A Gen1 collection (collects from Gen0 and Gen1) is more costly than a Gen0 collection (only collects from Gen0), so you would expect it to happen every tenth run of a Gen0 collection (approx, ish). A Gen2 (Full) collection is even more costly still, so would run every ten runs of a Gen1 collection (again: ish).

So, if you have some transient objects that are still alive when you do a Gen0 collection (that you will stop referencing very soon), they will be promoted to the next generation and a) won't be eligible for collection until a Gen1 collection occurs b) won't count towards the Gen0 threshold of memory usage and c) will still show up in the total memory count.

This is what I suspect is happening in your test app. I hope you can see how it explains the memory usage you are seeing.

Re: WinForms Memory Leak

Originally Posted by sdfax_work

Code:

Private Sub UltraButton2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles UltraButton2.Click
Try
Dim closeForms As List(Of Form) = New List(Of Form)()
For Each frm As Form In My.Application.OpenForms
If frm.Name = "GridForm" Then
closeForms.Add(frm)
End If
Next
For Each frm As Form In closeForms
frm.Close()
frm.Dispose()
Next
Catch ex As Exception
MessageBox.Show(ex.Message)
End Try
End Sub

This is just an example that sticks out at me. Suppose a Gen0 collection happens whilst in this Sub (I think this is unlikely, as I can't really see what might trigger it by inspection, but it is possible and serves a handy illustration of some of the problems you might face).

There is a List(Of Form) that is being kept alive by the closeForms variable, so that list survives the collection and is promoted to the next generation, and not collected until the next Gen1 collection. Okay, not too bad on its own, it only contains as many references as the number of your "GridForm"s open at the time. BUT, by referencing those forms, those are also alive and promoted to the next generation. You call Dispose on the forms, true, but that does not force them to be Garbage Collected! You don't override Form's implementation of Dispose to remove the reference to all the random data, so by virtue of the Form objects being alive, the random data on each of those forms are still alive as well, and get promoted to the next generation also!

So, that's just one example of how objects can stay alive longer than you'd think.

Re: WinForms Memory Leak

Originally Posted by boops boops

You RandomDataHelper class has a Shared List of RandomData objects. A shared object like that list can't go out of scope while the program is running, because it belongs to the class not to an instance. So every time you instantiate a GridForm, the call to RandomDataHelper adds another 10,000 RandomData items to the list. I wonder if that is the cause of your memory leak?

Not sure if anyone has responded to this yet (can't see anything but I may have missed it).

I don't think that is an issue. When the RandomDataHelper.GetRandomData method is called, it only allocates the list object and populates it with data the first time through. Subsequent calls to the method create a new List with new Data objects that get returned, but are not stored in the Shared member.

This will allocate a bunch of memory that is never released only on the first time the method is called. Subsequent to that, the memory allocated will be collectible before the end of the program.

Re: WinForms Memory Leak

For your test program only: I think this is one of the few times where you (the programmer) are smarter than the GC. You know that when you close all those forms, you are done with those objects. You know that the user may very well tolerate some fractional delay in the application because they are "done" with an immediate task. Therefore, this is one of the very few times where you could probably get away with forcing a collection. I would try adding a GC.Collect() in the UltraButton2_Click handler outside the try/catch block (such that the closeForms variable has gone out of scope! (see #13)).

Compare the memory usage you see with and without that call.

NOTE: I only suggest this approach for this particular scenario and if you do it you must test the effect (compare to without doing so) to ensure that you are actually getting something for your money here. Forcing a collection isn't cheap, and worse, the GC adapts to your app's memory profile as it runs so it can perform better - forcing a Collection wipes out those internal counters and resets the GC to dumb I-don't-know-what-this-app-looks-like mode.

As to your production app: you should find the memory leak that I strongly suspect is there before you try anything like this.

Re: WinForms Memory Leak

Originally Posted by Grimfort

Actually it is really easy to find leaks using a profiler. The thing to look at is delta, you can see the difference between created and removed, the ones that are not removed have a really high delta, and those are the ones that are causing your leak. Open your base form, take a snapshot to get a baseline, open loads and loads and loads of forms, leaving them open take another snapshot, close them all back to the base form, take another snapshop. Diff the first and last, they should be similar. A decent profiler will show you the link to the object and code that is keeping it referenced.

I will revisit the memory profiling options. We have some RedGate tools so I'll look to get their profiler. The size of the app and the number of Infragistics controls we use makes it much more difficult than it would be for most apps but to be fair I didn't do much complex delta analysis other than looking for a class that only got referenced in one of the forms and identifying the number of object instances of that class still in memory

Re: WinForms Memory Leak

Originally Posted by Shaggy Hiker

The lower bound trending upwards is certainly disturbing, but, as you say, memory allocation is a dark art, so it isn't entirely implausible. I would suggest getting a good understanding of how the Garbage Collector works in .NET. It's kind of an interesting topic, though I know of no single source to suggest. The key, in my mind, is understanding how the GC recognizes that some object is unreachable. It seems to me that the most likely problem here is that some resource is unreachable in your mind (you certainly believe it to be discarded), yet the GC object map suggests that the item really isn't unreachable, so it never feels safe in recovering that memory. Of course, that can be quite the subject in itself.

I had a good look at garbage collection in the past (and yes I agree that it is interesting). We also profiled the gen0,1,2 object collection counts but couldn't see anything untoward in the ratios between the generations (following some MS guidelines).

I also agree on the most likely problem being items identified as reachable. Each time I explain this issue or read up about it I get an idea about what may be the cause and you've just given me another idea. We have the equivalent of an Alt-Tab application switched internally in the app so that you can see all of the open forms visually and switch between them. When the form dies I'll check what is happening to this list collection

Re: WinForms Memory Leak

Originally Posted by Evil_Giraffe

For your test program only: I think this is one of the few times where you (the programmer) are smarter than the GC. You know that when you close all those forms, you are done with those objects. You know that the user may very well tolerate some fractional delay in the application because they are "done" with an immediate task. Therefore, this is one of the very few times where you could probably get away with forcing a collection. I would try adding a GC.Collect() in the UltraButton2_Click handler outside the try/catch block (such that the closeForms variable has gone out of scope! (see #13)).

Compare the memory usage you see with and without that call.

If do a GC.Collect() for 0,1,2 after closing all of the forms, it has no immediate effect, but if you then open just one form and repeat the close all forms code (including the the GC.Collect()) the memory seemed to be consistently compacted back to about 15MB. I need some more time to test but this is a huge improvement in the test app - I realise this is not a good idea in production

Re: WinForms Memory Leak

Originally Posted by sdfax_work

I'm trying to get to grips with how Windows Forms applications manage memory allocation. I'll give you an illustration of the problem. Take this simple winforms app which is a main form with two buttons: one that opens a form containing some random data, and another button that closes all open forms (except the main form)

If you run this and click away opening lots of forms, 50 at a time, 100 at a time, and then closing them all, the memory goes up and down over time but gradually gets higher and higher, never returning to anything like its initial state. Now I assume part of this is the .NET runtime being clever and assuming that as the app needed a lot of memory previously, it will need more again and better to keep some allocated (I am running this on a machine with 4GB RAM).

You can see the memory dropping at intervals (I've marked some of them above) which is good but it still slowly has a minimum that creeps up and up.

This is a very basic illustration of a much bigger production issue with a huge application with 100,000s lines of code and users that keep the app open all day. Over time they open and close a lot of forms and the memory creeps up.

Is there any way to force the allocation down? I know I can force GC but this won't do anything as it's the memory post-GC that is remaining high as far as I can see.

This isn't a problem with references as far as I can see because I have spent some time experimenting with just one form in the main application opening and closing it and there are no references after it is closed and disposed and yet the memory still creeps up in a similar way to that logged above. Also if it was a problem with references, I'm not sure how that would explain my test app above.

Any help or advice on what to use to troubleshoot greatly received.

you can use debuggers - Valgrind (for Linux) or Deleaker (for Windows). With Deleaker you can detect and localize resource leaks such as memory, gdi and user objects.

Re: WinForms Memory Leak

If you have a problem with memory leaks, you need to be more careful. If you allocate memory, you have the duty to liberate. Do not forget this important aspect.
The debugger only helps to look for leaks, but it does not work for you.

Re: WinForms Memory Leak

If you allocate memory, you have the duty to liberate. Do not forget this important aspect.

Um, not really the case in a managed memory language like VB. And when you do need to take care, there are different considerations to unmanaged languages.

Essentially, if there is no pressure for the memory to be released, why on earth should any time be spent on releasing the memory? This can make things look much worse than they are if you just look at 'memory usage' of a .NET process (the same with Java).

Re: WinForms Memory Leak

Re: WinForms Memory Leak

Thank you to everyone who contributed to this thread. The recent posts have served as a reminder that am I remiss in not updating what happened in this particular case.

The memory leak was traced to the way a third party component, Infragistics UltraGrid and UltraComboBox held onto a reference. In summary:

1. Instantiate and show a new form containing a grid where one column is bound to a combobox.
2. Close the form when you're done
3. A reference survives related to the combobox bound to the column (this was detected using Redgate's ANTS memory profiler)
4. Because of the surviving reference the form and its memory are never garbage collected

It was fixed by introducing something like this is the class that serves as a base to all the data bound forms

Code:

Private Sub frmBoundForm_FormClosed(ByVal sender As Object, ByVal e As System.Windows.Forms.FormClosedEventArgs) Handles Me.FormClosed
Try
ClearDownGridCombos(Me)
Catch ex As Exception
HandleException(ex)
End Try
End Sub
Private Sub ClearDownGridCombos(ByVal ctl As Control)
If TypeOf ctl Is UltraGrid Then
ClearDownGridCombos(CType(ctl, UltraGrid))
End If
If ctl.HasChildren Then
For Each child As Control In ctl.Controls
ClearDownGridCombos(child)
Next
End If
End Sub
Private Sub ClearDownGridCombos(ByVal ug As UltraGrid)
For Each band As UltraGridBand In ug.DisplayLayout.Bands
For Each col In band.Columns.Cast(Of UltraGridColumn)().Where(Function(c) c.EditorComponent IsNot Nothing)
col.EditorComponent.Dispose()
col.EditorComponent = Nothing
Next
Next
End Sub

There is still underlying strange behaviour and a general increase in memory over time in a very basic windows forms application that just opens and closes 100s of forms, but some memory does get released and it's not as big an issue as the problem above in the production app.

As has been stated, the best way to trace this kind of thing is to hunt for the root of surviving references using a tool like Redgate's ANTS memory profiler. This proved very useful in my case