Yes, bulk insert. Originally, I was using vb6 and pulling
data from an external data source (non rdbms) and writing
each record to sql server record by record. It started
taking too long (several hours) and was becoming
unreliable, connection open too long. So with vb.net, I
read each record and write it to a series of text files
(each text file will contain at most 20,000 records). I
write close to 100 textfiles (nearly 2,000,000 records).
Then I suck em all up with DTS - takes only 2-3 minutes
with vb.net. The suggestion of using the DTS Agent is
that I have nearly 100 text files to import. I loop
through my DTS package in the vb.net app. Much easier
than using the agent. The new system takes between 1 to 1
½ hrs. Way more reliable because writing text files is a
snap with vb.net. Plus, vb.net runs the DTS package 3-4
times faster than vb6
But I understand that ado.net2 will have bulk insert
capabilities similar to DTS. I sure hope that is true. I
believe that DTS packages are hard to deal with because
they are basically com based, I mean I have to make a
reference to DTS package Object Library from the com tab
in references.
We did something similiar at work (Bulk Insert doing about 140,000 records)
and was running through the same issues. I was originally trying to use the
BCP program and shell out to that, but became too unreliable and couldn't
manage events that would happen (out of 140,000 rows 2 would fail, I know
you can keep going but I needed to inform the user immediatly).
What we ended up doing was using the COM object for SQL DMO (I *think*
that's the library) and this worked AWESOME! Good event model on when it
inserts rows, good documentation and you run it in process so you could keep
an eye on everything going on. 140,000 records took roughly 30 seconds
doing it this way. The whole procedure took 15 minutes, 13 of that to
generate the XML file from this proprietary language called guru, 1.5
minutes for the XML DOM parser to read it in, and 2 seconds for the XSLT
transform, and 30 for the insert.
So check out the COM object, you'll find it really easy to use and easier to
debug/work with.
HTH,
CJ
Yes, bulk insert. Originally, I was using vb6 and pulling
data from an external data source (non rdbms) and writing
each record to sql server record by record. It started
taking too long (several hours) and was becoming
unreliable, connection open too long. So with vb.net, I
read each record and write it to a series of text files
(each text file will contain at most 20,000 records). I
write close to 100 textfiles (nearly 2,000,000 records).
Then I suck em all up with DTS - takes only 2-3 minutes
with vb.net. The suggestion of using the DTS Agent is
that I have nearly 100 text files to import. I loop
through my DTS package in the vb.net app. Much easier
than using the agent. The new system takes between 1 to 1
? hrs. Way more reliable because writing text files is a
snap with vb.net. Plus, vb.net runs the DTS package 3-4
times faster than vb6
But I understand that ado.net2 will have bulk insert
capabilities similar to DTS. I sure hope that is true. I
believe that DTS packages are hard to deal with because
they are basically com based, I mean I have to make a
reference to DTS package Object Library from the com tab
in references.
>-----Original Message-----
>What kind of package are you running? Bulk insert?
>"Ron" <anonymous@discussions.microsoft.com> wrote in
message
>news:066801c4b88b$498f9cd0$a501280a@phx.gbl...
>> OK. I think I fixed (I hope!). I moved the Me.Close
>> statement out of the DTS package procedure and into a
>> Timer procedure. Apparently, the DTS package is very
>> touchy and does not want to perform anything else that
is
>> not part of the DTS Package. Man, I can't wait for
>> ADO.net2 (note: I use DTS because I am moving a few
gigs
>> of data - daily - of course, for the test purpose, each
>> textfile DTS is reading only contains 100 records right
[quoted text clipped - 18 lines]
>>>
>>>Then the DTS app is supposed to close itself. But I
keep
>>>getting the error above when I run the DTS package
>>>procedure. If I don't run the DTS package then the DTS
[quoted text clipped - 10 lines]
>>>closing
>>>>> if I run a procedure in the app. I can't even trap
the
>>>>> error with try/catch ex As Exception. Is there a way
>> to
>>>>> completely shut down the app through code? I am
using
>>>>> End
>>>>>
[quoted text clipped - 5 lines]
>>>choice. Instead,
>>>>simply close the main form by calling its 'Close'
method.
>>>.
>
>.