Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / .NET Framework / ADO.NET / May 2005

Tip: Looking for answers? Try searching our database.

Insert Performance

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Andrew Robinson - 16 May 2005 19:39 GMT
I need to insert roughly 10,000 rows from a text log file into SQL. Ideally,
I will use ADO as there is a bit of pre-processing before the inserts. I am
guessing this will eliminate Bulk Copy.

In general, what is the fastest method of inserting rows into SQL?

SqlCommand ("INSERT INTO...." cn) or build a DataTable / DataSet and do an
update with a DataAdaptor?

Should I look at Async in Whidbey?

-Andrew
David Browne - 16 May 2005 21:55 GMT
>I need to insert roughly 10,000 rows from a text log file into SQL.
>Ideally,
[quoted text clipped - 8 lines]
>
> Should I look at Async in Whidbey?

DataAdapter.Update uses a plan SqlCommand for inserts, so there's no real
difference.  BULK INSERT is difficult do since you need to push the file
somewhere where the SQL Server can see it.

On a good network you can get close to 1000 inserts/sec using plain old
INSERT INTO ... VALUES.  Remember to wrap the inserts in a transaction to
prevent SQL Server from having to flush the log after each row.

In Whidbey you would use System.Data.SqlClient.SqlBulkCopy.

David
W.G. Ryan eMVP - 16 May 2005 22:01 GMT
Bulk copy is going to be  faster, as is any mechanism that cuts ADO.NET out
of the loop.  The reason I say this is that as a d loading mechanism, much
of hte benefits that ADO.NET affords are irrelevant (for instance, you
aren't going to have 25k clients connecting simultaneously doing bulk loads
when you are loading data like you are)

A data adapter calls Insert Into when it sees a rowstate that's added so
there isn't going to be  a difference here other than coding simplicity. In
Whidbey there are batch updates which enhance performance, but the
asncy/mars stuff isn't going to have any impact that I know of and I really
doubt that it will.

Signature

W.G. Ryan, MVP

www.tibasolutions.com | www.devbuzz.com | www.knowdotnet.com

> I need to insert roughly 10,000 rows from a text log file into SQL. Ideally,
> I will use ADO as there is a bit of pre-processing before the inserts. I am
[quoted text clipped - 8 lines]
>
> -Andrew
Andrew Robinson - 16 May 2005 23:55 GMT
Wrapping my data inserts inside of a transaction improve performace about
300%. My guess is that the log file is not being flushed after each insert?

-Andre

> I need to insert roughly 10,000 rows from a text log file into SQL. Ideally,
> I will use ADO as there is a bit of pre-processing before the inserts. I am
[quoted text clipped - 8 lines]
>
> -Andrew
Kevin Yu [MSFT] - 17 May 2005 02:42 GMT
Hi Andrew,

Yes, as you know, log file is not being flushed each time data is inserted
if you wrap your inserts inside a transaction.

Kevin Yu
Signature

=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.