Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / Languages / C# / September 2006

Tip: Looking for answers? Try searching our database.

Concatenate numerous large binary or text files into one

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
ffimbel@gmail.com - 30 Sep 2006 21:40 GMT
Hi,

I would like to find out the fastest way to concatenate large (200 or
300MB) and numerous (500 - 700) files (Postscript files) into a single
one (which can ends up being several GigaB) using csharp within a
windows form application.
One of the requirement also is to make sure the output file is closed
only when the concatenation is done so that the next application using
the file (printer spool) does not assume it is finished unledd it is
actually finished.

Thanks in advance for your help.
Arne Vajhøj - 30 Sep 2006 21:47 GMT
> I would like to find out the fastest way to concatenate large (200 or
> 300MB) and numerous (500 - 700) files (Postscript files) into a single
[quoted text clipped - 4 lines]
> the file (printer spool) does not assume it is finished unledd it is
> actually finished.

Efficient reading and writing of files should just be a
matter of using huge buffers.

I would be more worried about whether PDF files
can just be concatenated.

Arne
David Browne - 30 Sep 2006 22:22 GMT
> Hi,
>
[quoted text clipped - 6 lines]
> the file (printer spool) does not assume it is finished unledd it is
> actually finished.

Well, you don't really need the fastest way.  You need a reasonably fast
way.  Just use System.IO.FileStream, open one for your output file, and
iterate over your input files.  For each input file, read a buffer from the
input file, write the buffer to the output file, repeat.

You don't need to use huge buffers, and it's probably not worth while to use
async IO or double buffering or any of that.

Here's a simple file concatenation program:

David

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;

namespace FileConcat
{
 class Program
 {
   static int Main(string[] args)
   {
     if (args.Length != 2)
     {
       Console.WriteLine("usage FileConcat [SourceDir] [DestinationFile]");
     }
     try
     {
       Run(args[0],args[1]);
       return 0;
     }
     catch (Exception ex)
     {
       Console.WriteLine(ex);
       return 1;
     }
   }
   static void Run(string SourceDir, string OutputFileName)
   {
     string[] inputFiles = Directory.GetFiles(SourceDir);

     int bufSize = 1024 * 64;

     byte[] buf = new byte[bufSize];

     using (FileStream outFile =
       new FileStream(OutputFileName, FileMode.OpenOrCreate,
FileAccess.Write, FileShare.None, bufSize))
     {
       foreach (string inputFile in inputFiles)
       {
         using (FileStream inFile =
           new FileStream(inputFile, FileMode.Open, FileAccess.Read,
FileShare.Read, bufSize))
         {
           int br = 0;
           while ((br = inFile.Read(buf,0,buf.Length))> 0)
           {
             outFile.Write(buf,0,br);
           }
         }
       }
     }

   }
 }
}

Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.