 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
.NET Forum / Languages / C# / May 2007Tracking a memory leak.
I have an object tree that is pretty gigantic and it holds about 100mb of data. When I set the top object to null, I expect that the .NET framework will clean up the memory at some point. However, I am looking at the Task Manager and I don't see the MemUsage column decreasing even after an hour or two. I know that TaskManager may not be the best place to see what is the true way to gauge memory usage and/or presense of memory leaks. So I am looking for a pointer on what I should use to see whether setting an object to null really will at some point free up memory. Regards Frank, You should be looking at the performance counters here to see what the memory consumption of .NET is. You should look at the ".NET CLR Memory" category.  Signature- Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com >I have an object tree that is pretty gigantic and it holds about 100mb of >data. When I set the top object to null, I expect that the .NET framework [quoted text clipped - 6 lines] > > Regards >I have an object tree that is pretty gigantic and it holds about 100mb of >data. When I set the top object to null, I expect that the .NET framework [quoted text clipped - 4 lines] >am looking for a pointer on what I should use to see whether setting an >object to null really will at some point free up memory. Process Explorer may be of some help to you. http://www.microsoft.com/technet/sysinternals/utilities/ProcessExplorer.mspx You can right-click in the upper pane on a process and select Properties. You can right-click in the lower-pane too. > I have an object tree that is pretty gigantic and it holds about 100mb > of data. When I set the top object to null, I expect that the .NET [quoted text clipped - 4 lines] > memory leaks. So I am looking for a pointer on what I should use to see > whether setting an object to null really will at some point free up memory. Some reminders, and a question. Remember that the garbage collector will reclaim memory only when you put memory pressure on it. That is, only when you try to allocate more memory and the GC detects that you're short. Also remember that if the objects in your tree have been around for a while, they have likely been promoted into higher generations of the GC, and are therefore less likely to be reclaimed rapidly. The longer instances stay around, the less likely they are to be reclaimed quickly. Of course, if your program is _running_ for an hour or two, and keeps allocating new objects during that time, and the contents of the object tree _still_ don't go away, then you have a problem. Now the question: does any of the objects in the tree subscribe to events generated by objects outside the tree? > Now the question: does any of the objects in the tree subscribe to > events generated by objects outside the tree? Bruce, the objects in the tree do not do subscribe or produce an events. > > Now the question: does any of the objects in the tree subscribe to > > events generated by objects outside the tree? > > Bruce, the objects in the tree do not do subscribe or produce an events. Hmm. Then I'm at a loss. Events are usually the type of reference that everyone forgets about. Can you post a stripped-down program that exhibits the behaviour? One that we can all compile and run, and see it happen? >>> Now the question: does any of the objects in the tree subscribe to >>> events generated by objects outside the tree? [quoted text clipped - 5 lines] > Can you post a stripped-down program that exhibits the behaviour? One > that we can all compile and run, and see it happen? Unfortunately not, the program is pretty complicated and has a dependency on a 30gb database. >I have an object tree that is pretty gigantic and it holds about 100mb of >data. When I set the top object to null, I expect that the .NET framework [quoted text clipped - 6 lines] > > Regards You should take a look at the "CLR Memory performance" counters, notably the Gen0, 1, 2 and Large Object heap counters are of interest. What is shown in Taskman is the size of your process Working Set or the Private Bytes (depending the counter you are looking at), the GC allocates and de-allocates object space from a Private Heap holding the above generational heaps, this Private Heap,w which is part of the Private Bytes and the Working Set, grows, by requesting new segments from the OS, whenever the Generational heap 0 is full and the GC needs more space to allocate objects from. However, this heap will not necessarily decrease when objects are getting GC'd, all the GC does is free the object space and compact the generational heaps (0, or 0 and 1, or 0, 1 and 2). The CLR "Memory Allocator" de-commits (after compactation) the pages that were previously occupied by object data, and returns the Segment to the OS when *all pages* from that segment (but not the default segments) are decommited. Note that pinned objects can spoil the party, as these cannot move to another segment and as such prevents the Segment to become de-committed completely. Another thing that you need to watch for is the LOH, as this heap is never compacted, keeping objects alive in one of the possible extra segment for this LOH, will prevent the segment to return to the OS. Willy. > of data. When I set the top object to null, I expect that the .NET > framework will clean up the memory at some point. However, I am looking Why do you expect this? A tree of n nodes where (n + n1 + n2 + n3...) with n set to null is still linked at n1 + n2 + n3...). These are valid roots which won't be garbage collected. I fully expect this to leak memory. SignatureRegards, Alvin Bruney ------------------------------------------------------ Shameless author plug Excel Services for .NET is coming... OWC Black book on Amazon and www.lulu.com/owc Professional VSTO 2005 - Wrox/Wiley >I have an object tree that is pretty gigantic and it holds about 100mb of >data. When I set the top object to null, I expect that the .NET framework [quoted text clipped - 6 lines] > > Regards On May 12, 5:08 pm, "Alvin Bruney [MVP]" <some guy without an email address> wrote: > > of data. When I set the top object to null, I expect that the .NET > > framework will clean up the memory at some point. However, I am looking > > Why do you expect this? A tree of n nodes where (n + n1 + n2 + n3...) with n > set to null is still linked at n1 + n2 + n3...). These are valid roots which > won't be garbage collected. I fully expect this to leak memory. Alvin, perhaps I don't understand your post. If I have a tree of objects, with a single root pointer pointing to the root node, and that pointing to n other nodes, which in turn point to n other nodes, etc, and I set the root pointer to null, then entire tree becomes available for collection, even though parts of itself refer to other parts. Whether the GC bothers to collect it, and when, is a whole other question. Why do you think that a structure like this would "lead memory"? Yes, I see what you are saying. I don't know how trees work internally either, however I suspect that they are linked lists of sorts. That would mean that even though the root is null, the third node or n+1 still has a bonafide root which is the node n. The second node does not have a root as you correctly pointed out. Can this be collected? I think it depends on which node is examined first. If the GC examines node 0, a collection for the entire tree is possible, however for any other node examined first, these have valid roots which won't be collected. The best way to resolve this is to write a short program that forces a collection on a tree of length n and examine the heap - then we know for sure. Any body got time on their hands? SignatureRegards, Alvin Bruney ------------------------------------------------------ Shameless author plug Excel Services for .NET is coming... OWC Black book on Amazon and www.lulu.com/owc Professional VSTO 2005 - Wrox/Wiley > On May 12, 5:08 pm, "Alvin Bruney [MVP]" <some guy without an email > address> wrote: [quoted text clipped - 20 lines] > > Why do you think that a structure like this would "lead memory"? You don't need to write a short program. If you have a tree where the nodes are only referenced by the parent, and then the reference to the root is let go, then the whole tree is eligible for collection. Eligibility for GC doesn't matter if something is holding a reference to an object or not, it's whether or not something is holding a reference to that, and so on. Basically, the CLR performs a mark and sweep, where it starts with the stack on every thread (for the most part, there are other places it looks which are not thread specific) and looks to see what references are held, and then what references they hold, and so on, and so on. Once all of those references are marked, anything else that is not marked is GC'ed. So in this case, if you have a tree which has been let go, and nothing is referencing it, or it's nodes, even though the parent references the child nodes, the whole thing is eligible for GC. Signature- Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com > Yes, I see what you are saying. I don't know how trees work internally > either, however I suspect that they are linked lists of sorts. That would [quoted text clipped - 35 lines] >> >> Why do you think that a structure like this would "lead memory"? > You don't need to write a short program. > [quoted text clipped - 15 lines] > is referencing it, or it's nodes, even though the parent references the > child nodes, the whole thing is eligible for GC. What if the child nodes reference its parent? Is it still eligible for collection? > > You don't need to write a short program. > [quoted text clipped - 18 lines] > What if the child nodes reference its parent? Is it still eligible for > collection? Yes. It doesn't matter if an object instance has references to other (live) objects. All that matters is whether any live object has a reference to the object instance being considered for collection. An instance can have dozens of pointers to live objects and still be collected, so long as no live object has a reference to IT. (Where "live" means that it is referenced from the stack, from a static variable, or one of a few other "root" places, or from another "live" object.) Hi Frank, Thanks for your feedback. Actually, whether there is still any reference to an object is not the key point for identifying an object as eligible for collecting. The GC finds all the *live* objects through the *ROOT*. All the global and static object pointers in an application are considered part of the application's roots. In addition, any local variable/parameter object pointers on a thread's stack are considered part of the application's roots. Finally, any CPU registers containing pointers to objects in the managed heap are also considered part of the application's roots. The list of active roots is maintained by the just-in-time (JIT) compiler and common language runtime, and is made accessible to the garbage collector's algorithm. So in your scenario, although parent node references the child nodes and child nodes reference the parent node, there is no root points to any object in the tree, so the GC will not identify them as *live* object. So they are eligible for collecting. Jeffrey Richter's famous article below talks about GC algorithm in details: "Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework" http://msdn.microsoft.com/msdnmag/issues/1100/gci/ Hope it helps. Best regards, Jeffrey Tan Microsoft Online Community Support ================================================== Get notification to my posts through email? Please refer to http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif ications. Note: The MSDN Managed Newsgroup support offering is for non-urgent issues where an initial response from the community or a Microsoft Support Engineer within 1 business day is acceptable. Please note that each follow up response may take approximately 2 business days as the support professional working with you may need further investigation to reach the most efficient resolution. The offering is not appropriate for situations that require urgent, real-time or phone-based interactions or complex project analysis and dump analysis issues. Issues of this nature are best handled working with a dedicated Microsoft Support Engineer by contacting Microsoft Customer Support Services (CSS) at http://msdn.microsoft.com/subscriptions/support/default.aspx. ================================================== This posting is provided "AS IS" with no warranties, and confers no rights. Nicholas: Your assumption is that the initial sweep starts at the top most node in a tree. There is nothing in the docs that I have found that indicates that this is so. Perhaps you can point it out. SignatureRegards, Alvin Bruney ------------------------------------------------------ Shameless author plug Excel Services for .NET is coming... OWC Black book on Amazon and www.lulu.com/owc Professional VSTO 2005 - Wrox/Wiley > You don't need to write a short program. > [quoted text clipped - 55 lines] >>> >>> Why do you think that a structure like this would "lead memory"? > Nicholas: > Your assumption is that the initial sweep starts at the top most node > in a tree. There is nothing in the docs that I have found that > indicates that this is so. Perhaps you can point it out. If you re-read what he posted >> Basically, the CLR performs a mark and sweep, where it starts with >> the stack on every thread (for the most part, there are other places >> it looks which are not thread specific) and looks to see what >> references are held, and then what references they hold, and so on, >> and so on. Once all of those references are marked, anything else >> that is not marked is GC'ed. you will find that he made no such assumption. Unless there is a live chain of references that lead to these nodes...They are eligible for collection >> You don't need to write a short program. >> [quoted text clipped - 16 lines] >> nothing is referencing it, or it's nodes, even though the parent >> references the child nodes, the whole thing is eligible for GC. <snip> No, he didn't make that assumption. He correctly stated that the initial sweep doesn't start in the tree at all. It starts on the _stack_ (and in other places mentioned by Jeffrey Tan in his post). It then proceeds to follow all references that fan out from that point. Any object not thus marked is eligible for collection. IF the root of the tree is referenced by a "live" object (that has been found by starting from the stack, or a static object, or a register, etc), then YES, the GC will follow the pointers from the tree root to its child nodes, and their child nodes, and will traverse the tree in some fashion, marking all of its nodes as ineligible for collection. If the only thing referenced by an object outside the tree is the tree's root, AND you remove that reference, then no object in the tree will be found during the mark phase, because nothing "live" points to any part of the tree. I think that the root of your misunderstanding (pardon the pun) is that you seem to think that objects are garbage collected if they are "marked for collection"... that is, you seem to be reasoning from the angle that says that objects are collected if they are positively marked. The opposite is true: objects are collected if they are _not marked_ as "live". Mark-and-sweep finds everything that should _not_ be collected, then collects everything else. On May 14, 6:51 pm, "Alvin Bruney [MVP]" <some guy without an email address> wrote: > Nicholas: > Your assumption is that the initial sweep starts at the top most node in a [quoted text clipped - 83 lines] > > >>> Why do you think that a structure like this would "lead memory"? I think that the root of your misunderstanding (pardon the pun) is that you seem to think that objects are garbage collected if they are "marked for collection"... Not true, and I don't see how you got that from my argument either. SignatureRegards, Alvin Bruney ------------------------------------------------------ Shameless author plug Excel Services for .NET is coming... OWC Black book on Amazon and www.lulu.com/owc Professional VSTO 2005 - Wrox/Wiley > No, he didn't make that assumption. > [quoted text clipped - 130 lines] >> >> >>> Why do you think that a structure like this would "lead memory"? On May 15, 3:59 pm, "Alvin Bruney [MVP]" <some guy without an email address> wrote: > I think that the root of your misunderstanding (pardon the pun) is > that you seem to think that objects are garbage collected if they are > "marked for collection"... > > Not true, and I don't see how you got that from my argument either. Well, it was the only conclusion I could think of. Perhaps it's my poor imagination. :-) You said that Nicholas was implying that "the initial sweep starts in the topmost node of a tree." In fact, if there is no way to get from the stack / static variable / etc to any part of the tree, then the mark phase ("initial sweep") doesn't start anywhere in the tree. It never sees the tree, which is what makes the tree eligible for collection. If the topmost node of the tree is the only object in the tree referenced by anything outside the tree, and it is in fact referenced by an object that is marked as not eligible for collection, then yes the mark phase ("initial sweep") will traverse the tree starting at the topmost node, because that's the first object reachable from the collection of objects that has so far been marked as ineligible for collection. Anyway, everything that Nicholas has said so far has been accurate. I'm trying to figure out where the disagreement is, and not having much luck. :-( >I think that the root of your misunderstanding (pardon the pun) is > that you seem to think that objects are garbage collected if they are > "marked for collection"... > > Not true, and I don't see how you got that from my argument either. >> >> I think it depends on which node is examined first. If the GC >> >> examines >> >> node 0, a collection for the entire tree is possible, however for >> >> any >> >> other node examined first, these have valid roots which won't be >> >> collected. That's the statement that made me think you misunderstood the process. Since, as Bruce explains, if the root node is visited, it means that the tree is NOT eligible for garbage collection. Bill I think that even if the top object is null, if something is referencing the .Nodes, they will not be cleaned up, so the tree structure is still pretty well intact. >I have an object tree that is pretty gigantic and it holds about 100mb of >data. When I set the top object to null, I expect that the .NET framework [quoted text clipped - 6 lines] > > Regards >I have an object tree that is pretty gigantic and it holds about 100mb >of data. When I set the top object to null, I expect that the .NET [quoted text clipped - 6 lines] > >Regards Hi, This may sound dumb but is there a problem? I recently watched an app which was suspected of a memory leak. (The app runs 24/7) The priv bytes and working set climbed in a series of plateaus over a a number of hours. My initial reaction was 'memory leak' but it finally stabilised after about a day. This did not fit the perceived operational model of the application. Closing the application reduced the two counters but did not immediately send them to zero. I didn't have time to see how long it would take to zero them. Restarting put the counters back at the start of their plateau climb and the pattern was roughly repeated. regards Bob Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.