Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / Languages / Managed C++ / August 2007

Tip: Looking for answers? Try searching our database.

best memory allocation function for dealing with large (possibly in the 100s of MBs) byte arrays?

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Jonathan Wilson - 18 Aug 2007 13:17 GMT
I am working on some software which has to deal with data that could be as
large as 500mb or so. Currently I am using new[] and delete[] to manage
this memory but I find it is not ideal and sometimes gives out of memory
errors if I open one large data item then free that data item then another
large data item even though I have enough memory (including 2GB of physical
RAM and 100GB of free disk space for swap file). Are there any functions
(either in the CRT or in the win32 API) that would be better for this job
(specifically functions that are guaranteed to return the memory to the OS
and make task manager show that the memory is no longer in use as well as
having as little overhead as possible when using it)

The app in question runs only on windows > XP and only has one thread
touching this memory at all.
Johannes Passing - 18 Aug 2007 14:37 GMT
Hi Jonathan,

assuming that you are talking about a 32 bit process here, you only have
(on XP) 2 GB of virtual memory accessible in user mode. Even if you have
more physical RAM and plenty of paging file space, your process will not
be able to use more than 2GB of VM.

Your modules, stacks and heaps are sprinkled throughout these 2 GB of
VM, so you have to assume that the 2 GB are already quite a bit
fragmented. Given these constraints, it is pretty obvious that
allocating 500 MB of contiguous VM is very likely to fail.

So first you should ask yourself if you really need such large blocks of
contiguous virtual memory or if you can split them into chains of
multiple, much smaller sized blocks (of which maybe you do not even need
all in memory at once).

Finally, if you cannot avoid using large contiguous blocks of VM,
consider reserving a fair amount of VM early in the lifecycle of your
process and commit parts of it as you need them.

--Johannes

> I am working on some software which has to deal with data that could be
> as large as 500mb or so. Currently I am using new[] and delete[] to
[quoted text clipped - 10 lines]
> The app in question runs only on windows > XP and only has one thread
> touching this memory at all.

Signature

Johannes Passing - http://int3.de/


Rate this thread:







Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.