Hello There,
Can anybody help me in reading through large csv files size of about
200mb, with 200 columns.
I have tried with streamreader object parsing line by line but its bit
slow, does anybody have a better solution.
I dont want to landup using 3rd party component.
Regards,
Saravanan K
Steve Barnett - 21 Dec 2006 15:57 GMT
I've used this - very easy to use and very fast:
http://www.codeproject.com/cs/database/CsvReader.asp
It seems to be supported (they fixed a bug for me) and has source if you
want to "mess". Performance wise, they quote:
"To give more down to earth numbers, with a 45 MB CSV file containing 145
fields and 50,000 records, the reader was processing about 30 MB/sec. So all
in all, it took 1.5 seconds! The machine specs were P4 3.0 GHz, 1024 MB."
HTH
Steve
> Hello There,
>
[quoted text clipped - 8 lines]
> Regards,
> Saravanan K
RobinS - 21 Dec 2006 21:40 GMT
> Hello There,
>
[quoted text clipped - 8 lines]
> Regards,
> Saravanan K
Assuming each line has a carriage-return/line-feed at the end
(and it must, or readLine wouldn't work), here's an easy way
to read in a file and split the lines by Crlf into an array
in one fell swoop. I don't know what this would do with your
file, but it's free to try it. :-)
Dim crlfs() as String = {ControlChars.CrLf}
Dim lines() as String = _
File.ReadAllText("c:\data.txt").Split(crlfs, StringSplitOptions.None)
Dim numOfLines = lines.Length
Thanks to Francesco Balena; I got this out
of one of his books, probably the VB2005 Core Reference.
Robin S.