 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
.NET Forum / .NET Framework / New Users / March 2006Large object heap and OutOfMemory Exception
hello there, I have developped a windows services in vb.net (framework 1.1 sp1). I read PDF files stored in a database and put them in an arraylist to create a unique document at the end of the loop. Every single document is neary 40kb my arraylist can stored 150 elements maximum. On a windows 2003 server I have problems (OutOfMemory) after few days but this doesn't happen in WinXp. I read nearly 200 post in different forums talking about "large object heap" and the fact it's not compacted. Most people suggest not to use these objects. Other gurus say you should never call GC.Collect. Well, nobody seem to have a real solution to avoid my application blow up. Is this .NET Framework so stupid when dealing with objects in memory?! What do we have to do if we need to manage few files, in memory? Where are the M$ guys when we need them? Regards Alberto [The Large Object Heap] > On a windows 2003 server I have problems (OutOfMemory) after few days > but this doesn't happen in WinXp. What makes you think the Large Object Heap is the culprit? With the .NET Memory Performance Counters loaded, does the LOH show that it's growing continually? My experience is that it's much more common to have a Generation 2 memory leak cause this problem than an LOH issue. Are you running the Server or Workstation CLR? Does either of these machines have more than one processor? > Most people suggest not to use these objects. Other gurus say you > should never call GC.Collect. How do you think calling GC.Collect is going to help the issue? > Well, nobody seem to have a real solution to avoid my application blow > up. Nobody should. It's your application. You haven't exactly helped "us" track down the problem either at this point. > Is this .NET Framework so stupid when dealing with objects in memory?! I would venture to say the .Net framework is the smartest object and memory manger in production across any system or platform. There might be a few things that give it a run for it's money, but it's gonna be close. > Where are the M$ guys when we need them? The MS guys (or anyone who does this for a living) is going to tell you: 1) Look at the performance counters for member to determine where the problem likley is. 2) Capture a Minidump when you get OOM, load it into WinDebug and Son of Strike, and look at the heap. This is all work you are capable of doing. I understand your frustration, having been there myself, but by far the most likley canidate here is the code that you wrote has a memory leak in it. It's not going to be a framework bug. It's unlikley to be a LOH issue. It's very likley to be you not understanding how Garbage Collection works yet, so you're leaking memory. If it does turn out to be the Large Object Heap, you're going to want to switch to .Net 2.0, as it does perform compaction of the Large Object Heap.  SignatureChris Mullins Hello Chris, yes the server is a win2003 biprocessor and I have installed the framework 1.1sp1 and I have my directive <gcServer enabled="true" /> in the application cofiguration file. I have made a profiling and it seems the Gen 2 heap size and are growing progressively. I have the problem after few days the windows services works. With the GC.Collect I have solved my problem cause I can see the memory is released after the execution of the "hard work". I don't know what's going to happen in 20 days, though. I don't think the garbage collector is a good guy (I am talking about framework 1.1) cause I have read nearly 100 post in different newsgroups and many people have the same problems when dealing with datasets or binary objects. I don't understand why on my WinXp everything works great and on a server like that I do have to worry about those stuff? Are there any differences if I install my windows services in DEBUG or RELEASE? thank regards Alberto [Memory Leaks] > yes the server is a win2003 biprocessor and I have installed the > framework 1.1sp1 and I have my directive <gcServer > enabled="true" /> in the application cofiguration file. The first thing you want to do is turn that off. The gargage collection and heap management algorithms are very different between the server and workstation CLR, and that's only serving to confuse the issue at this point. You want to use the same runtime on your XP workstation and your 2003 Server so that you get the same behavior on both machines. There are lots of resources on this available on the web. Here's a very consice one... http://blogs.msdn.com/clyon/archive/2004/09/08/226981.aspx > I have made a profiling and it seems the Gen 2 heap size and are > growing progressively. This means that you're leaking memory. This isn't a garbage collection bug, or a Large Object Heap bug, but a programming bug on your part. This is good news, as it means you can fix the problem. Go and download the SciTech Memory profiler and learn how to use it. Once your comfortable with the tool, and have fixed the obvious low hanging fruit, you're going to want to run your app for an hour, collect a heap snapshot, let it run for a few more hours, and collect another heap snapshot. Save this to a file somewhere, and spend the next 2 or 3 days figuring out what's being leaked. It will take you a full day or two just to learn how to interpret the data given to you by the tool. This first thing you'll want to do is make sure Dispose Tracking is turned on, get the report of "Should have been disposed but wasn't" and fix these - hopefully fixing tje, will probably be pretty quick. Then Lather, Rinse, Repeat. > With the GC.Collect I have solved my problem cause I can see the memory > is released after the execution of the "hard work". It's really not. It's looking like fix, but it really hasn't solved the problem at all. Trust me, I've been down this path before. > I don't think the garbage collector is a good guy (I am talking about > framework 1.1) cause I have read nearly 100 post in different > newsgroups and many people have the same problems when dealing > with datasets or binary objects. Yea, well, don't believe everything what you read on the internet. People become frustrated and confused when lacking the GC Gnosis.... and often find a good scapegoat in the Garbage Collector. They're no more right than the Greeks who believed Thunder was caused by the gods. The Garbage Collector isn't the problem here. It doesn't have a bug. You have legit memory leaks in your app. > I don't understand why on my WinXp everything works great and on a > server like that I do have to worry about those stuff? Your problem is that you read a post somewhere that said to use the line: <gcServer enabled="true" /> so that your application will run better. You did this without taking the time to understand what this means - and it has changed the way your application behaves in such a way at to expose your memory management bugs much more quickly. You're using the Server CLR on the server, and the workstation CLR on your workstation. They are very different. You can go read all about it on MSDN. There's nothing wrong with the server CLR GC Algorithms - they're not buggy either. It's just that the way they work makes your problem surface more quickly, as they split the manged heaps up across processors. > Are there any differences if I install my windows services in DEBUG or > RELEASE? At this point in time? Not really. You need to fix the memory leaks, and they're going to exist in both Debug and Release mode. SignatureChris Mullins Hello Chris, thank you for your reply. You're right I should understand more about the garbage collector mechanism. Now I am going to try to use these profilers to see what's going on in the application even if I could say my application is nothing special. This services fetch binary data on a database, create a collection of binaries and build a PDF document with all these stuff. I am using iTextSharp to generate the PDF file. I think I am going to debug that component as well. Anyway, I would like to say that I don't read whatever I get on the internet and think it's true but I try to understand. I found the <gcServer enabled="true" /> directive cause I found a problem with the application. I discovered it and studied the implication of its use. At the end I realized nothing had changed. regards Alberto Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.