OutOfMemoryException exception

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Sjors - 09 May 2007 08:15 GMT

Hi,

I'm debugging some code running on Windows 2003 R2, .Net 1.1 as windows
service. The code parses medium sized (ca 1,5MB) csv files. Recently this
code starts throwing OOM exceptions running a loop that, for as far as I can
determine, uses short living large objects. The file is read into a DataTable
and for every row of the table 1 or 2 DataTable.Select(..)'s are called. The
result (array of DataRow) consists of a few rows that are examined and if
some condition is met are deleted from the DataTable.
This process takes about 2 sec. on my development machine (XP Pro, 512MB,
2,8GHz CPU) but on the server it takes 45 mins. and after that time the OOM
exception is thrown.

I'm thinking about LOH fragmentation resulting in excessive RAM use and thus
excessive paging and for that very poor performance. I added some
GC.Collect() instructions to the loop, but to no effect. I also set the
resulting array to Nothing (no effect).

A lengthy post. I hope someone sees a pattern.

Thanks for any reaction.

Sjors.

Reply to this Message

Laura T. - 17 May 2007 13:12 GMT

Hmm.. I don't think that memory problems can explain the difference between
2 sec and 45 minutes.. It's just too much. The data is the same?
If you are generating LOH objects in a loop, it might be useful to set the
vars =null immediately when they are not needed.
This can help GC to keep some order in LOH.
You can also try to use this (at least in debugging code) instead of just
GC.Collect():

GC.Collect(); // Put dead or dying in finalizer queue
GC.WaitForPendingFinalizers(); // Hopefully this clears things
GC.Collect(); // Collect also resources released by finalizers, if any

And then attach perf monitor to chec GC times%, heap sizes and other memory
values. They can give some more indications what's going on.
Comparing them with dev machine results might be interesting.

> Hi,
>
[quoted text clipped - 24 lines]
>
> Sjors.

Reply to this Message

Sjors - 18 May 2007 11:23 GMT

Thanks Laura T.,

I allready tried your suggestions before posting the question. To no no
effect obviously.
I was able to solve this thing by rewriting a loop that doesn't use the
DataTable.Select() method. Parsing the file on the production server after
the modification took a few secs (comparable to my dev. machine with the
Select()). My conslusion is that .Net Framework 1.1 for OS W2003 has a memory
leak in method DataTable.Select().

If you can come up with a different explanation I'd very much like to hear it.

Regards,
Sjors.

> Hmm.. I don't think that memory problems can explain the difference between
> 2 sec and 45 minutes.. It's just too much. The data is the same?
[quoted text clipped - 40 lines]
> >
> > Sjors.

Reply to this Message

Rate this thread:

OutOfMemoryException exception

Free Magazines