File compression in .NET - for very large files

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Amit Dedhia - 21 Jun 2007 08:12 GMT

Hi

I want to compress and decomress files. There is GZipStream class
in .Net. However this class requires to allocate buffer equal to the
file length to zip it (at least all sample code I saw were like that).
The file size in my case can be really very large and hence I want
something which does not require to load entire file in memory. Is
there any way?

Thanks and best regards
Amit Dedhia

Reply to this Message

Mark Rae - 21 Jun 2007 08:20 GMT

> Is there any way?

http://www.chilkatsoft.com/zip-dotnet.asp

Signature

http://www.markrae.net

Reply to this Message

Jon Skeet [C# MVP] - 21 Jun 2007 08:39 GMT

> > Is there any way?
>
> http://www.chilkatsoft.com/zip-dotnet.asp

There's no need to go into commercial software here - the only problem
for the OP is that all the samples read the whole of the data into
memory before pumping any of it through the output stream. That's easy
to fix and worth knowing for other situations too :)

Jon

Reply to this Message

Jon Skeet [C# MVP] - 21 Jun 2007 08:21 GMT

> I want to compress and decomress files. There is GZipStream class
> in .Net. However this class requires to allocate buffer equal to the
> file length to zip it (at least all sample code I saw were like that).
> The file size in my case can be really very large and hence I want
> something which does not require to load entire file in memory. Is
> there any way?

You don't need to read the whole file in. Just copy the data from the
input stream to the GZipStream a chunk (eg 32K) at a time.

I've got a utility class which makes this easier:
http://pobox.com/~skeet/csharp/miscutil
and
http://pobox.com/~skeet/csharp/miscutil/usage/streamutil.html
in particular.

Jon

Reply to this Message

Rate this thread:

File compression in .NET - for very large files

Free Magazines