Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / ASP.NET / Caching / April 2005

Tip: Looking for answers? Try searching our database.

Help with Caching problem

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Keith F. - 02 Apr 2005 15:27 GMT
Hi,
I'm having a serious problem with an ASP.NET web application that relies
heavily on several cache objects. One of the cache objects is a datatable
with approximately 50,000 records. From time to time, the cache object
becomes unsearchable. There are checks in the components that reference the
cache to see if it's NOTHING, if it is a DataTable and if it has rows, and
all those tests seem to say the object is ok. But when we try to execute the
.Select method on the datatable, an exception is thrown :
System.NullReferenceException: Object reference not set to an instance of an
object.
  at System.Data.Select.FindClosestCandidateIndex()
  at System.Data.Select.SelectRows()
  at System.Data.DataTable.Select(String filterExpression, String sort)
  at NCSBizComponents.ncsTill.GetStoreTillsByName_FromCache(Int32
piStoreId, String psTillTypeDescription)

When this happens, we manually force are refresh of Cache object and we're
ok for a while.
This seems to happen more often when the app is under a heavier load.

Our web server is running Win2K3 server, running on an IBMx440 under vmware.

I've read that ADO.NET creates it's own indexes when you create a datatable.
I'm wondering if somehow these indexes are getting messed up, causing the
Select method to fail. Can we/should we create indexes manually on the
datatable in cache?

Has anyone experienced this or have any ideas how to correct it?

Thank you,
Keith F
Brock Allen - 02 Apr 2005 16:49 GMT
I suppose it depends on your code to access the Cache, but ASP.NET will purce
Cache entries if the server is low on memory. Caching 50000 records feels
a bit heavy handed... so you might be seeing this purge behavior. Though
it's impossible to tell without seeing your GetStoreTillsByName_FromCache
method

-Brock
DevelopMentor
http://staff.develop.com/ballen

> Hi,
> I'm having a serious problem with an ASP.NET web application that
[quoted text clipped - 34 lines]
> Thank you,
> Keith F
Keith F. - 03 Apr 2005 14:39 GMT
Our web server has 1 gig of memory and the perf logs don't usually show the
memory going about 600 MB. It's usually at around 400-500 MB.
Here's the code where we're searching the cache:

Public Function GetStoreTillsByName_FromCache(ByVal piStoreId As Integer,
ByVal psTillTypeDescription As String) As ncsStoreTills
       Dim lsMethodName As String = "GetStoreTillsByName_FromCache"
       Try
           Dim loTransMgr As ncsTransactionManager = New
ncsTransactionManager
           'Get the Tills from Cache
           Dim ldtTills As DataTable =
loTransMgr.GetTillTranTypes_FromCache
           Dim ldrStoreTills() As DataRow
           Dim ldrRow As DataRow
           Dim lsFilter, lsSort As String
           Dim loTill As ncsStoreTill
           Dim loTillsCollection As ncsStoreTills = New ncsStoreTills
           Dim liHoldTillId As Integer = 0

           If Not ldtTills Is Nothing Then

               'Set filter expression to select by StoreId and Description
               lsFilter = "iStoreId = " & piStoreId.ToString & " AND
TillType LIKE '%" & Replace(psTillTypeDescription, "'", "''") & "%'"
               'Set sort expression to TillNbr
               lsSort = "tiTillNum"
               'Invoke the Select method on the Tills DataTable we got from
Cache
               '   to get a collection of DataRows meeting the filter
criteria
               ldrStoreTills = ldtTills.Select(lsFilter, lsSort)

               'Loop through the DataRows
               If Not ldrStoreTills Is Nothing Then 'pjo 02/21/05

                   For Each ldrRow In ldrStoreTills
                       'Note:Due to how the Cache is setup, we'll likely
have multiple records with the same till
                       '   in our DataRows collection.
                       'We'll test for duplicate TillIDs and skip those.
                       If Not ldrRow Is Nothing Then 'pjo 02/21/05

                           If ldrRow("iStoreTillId") <> liHoldTillId Then
                               'We have a new TillId
                               'create a new instance of a store till
                               loTill = New ncsStoreTill
                               'set the properties from the current DataRow
                               With loTill
                                   .TillDescription = ldrRow("TillType")
                                   .TillId = ldrRow("iStoreTillId")
                                   .TillNumber = ldrRow("tiTillNum")
                                   .TillTypeId = ldrRow("iTillTypeId")
                               End With
                               'Add the till to the store tills collection
                               loTillsCollection.Add(loTill)
                               'Save the TillId we just added to our
collection
                               liHoldTillId = ldrRow("iStoreTillId")
                           End If
                       End If

                   Next
               End If

           End If

           ldtTills = Nothing 'pjo 02/21/05
           ldrRow = Nothing 'pjo 02/21/05
           'Return the StoreTills Collection
           Return loTillsCollection

       Catch ex As Exception
           Dim loExpMgr As ExceptionMgr = New ExceptionMgr
           loExpMgr.Publish(ex, Err.Number, Err.Description, gmodName,
lsMethodName)
       End Try

   End Function

Thank you,
Keith

> I suppose it depends on your code to access the Cache, but ASP.NET will purce
> Cache entries if the server is low on memory. Caching 50000 records feels
[quoted text clipped - 44 lines]
> > Thank you,
> > Keith F
Brock Allen - 03 Apr 2005 16:44 GMT
Hmm.. well, I can't tell much from this; Sorry. Do you log the call stack
when you have unhandled exceptions? This might help in diagnosing the problem.

-Brock
DevelopMentor
http://staff.develop.com/ballen

> Our web server has 1 gig of memory and the perf logs don't usually
> show the
[quoted text clipped - 132 lines]
>>> Thank you,
>>> Keith F
Keith F. - 03 Apr 2005 18:47 GMT
I guess the answer to if we log the call stack for unhandled exceptions is no
(I'm not sure how to do that).
We email the exception info to system admins when we catch an error. That's
what I included in my original post.

This morning we identified the exact line of code that's throwing the
exception:
ldrStoreTills = ldtTills.Select(lsFilter, lsSort)
It's happening when we invoke the Select method of our DataTable object.
We added code to our exception handler to include the contents of the
lsFilter and lsSort varibles to make sure we're not passing invaid or empty
arguments to the Select method.
We have checks in the method that retrieves the DataTable from cache to make
sure it is a DataTable object, it actually has rows in it, and the HasErrors
property is false. I put a page out on the site that lists these properites.
I went to it when our app was throwing errors and all these properties looked
normal. However the cache wasn't searchable (apparently).

Would you have any thoughts as to why the Select method on the datatable
would be throwing an exception?

Or do you think the main problem is that a 50,000 row data table is just to
much to keep in cache?

One last question: When our code gets the DataTable object from Cache, and
puts it in a DataTable variable, is this a copy of the DataTable in Cache, or
a reference to the actual cache object?

Thank you for your thoughts.

Keith

> Hmm.. well, I can't tell much from this; Sorry. Do you log the call stack
> when you have unhandled exceptions? This might help in diagnosing the problem.
[quoted text clipped - 139 lines]
> >>> Thank you,
> >>> Keith F
Brock Allen - 03 Apr 2005 19:00 GMT
> Would you have any thoughts as to why the Select method on the
> datatable would be throwing an exception?

Hmm, yeah looking back at the callstack you provided from the first post...
the exception isn't that the DataTable is null, it's further down... so yeah,
hard to diagnose in the bowels of their code.

> Or do you think the main problem is that a 50,000 row data table is
> just to much to keep in cache?

Well, my sense is that the DataTable wasn't designed for this; Hundreds,
maybe a couple of thousand rows, but 50K seems like you're pushing the envelope.
I've not done a ton of stress testing on the DataTable to know what it's
breaking point is though, so this is all just my sense of things not backed
by any hard data or testing. So, YMMV :)

> One last question: When our code gets the DataTable object from Cache,
> and puts it in a DataTable variable, is this a copy of the DataTable
> in Cache, or a reference to the actual cache object?

It's a reference back to the same object in the cache.

> Thank you for your thoughts.

NP -- sorry I've not been more helpful.

-Brock
DevelopMentor
http://staff.develop.com/ballen
Ben Strackany - 26 Apr 2005 15:28 GMT
Maybe try creating a new DataView off of the DataTable after you pull it out
of cache, & using that DataView to access the data? Seems like some parts of
the DataTable are getting killed off.

Signature

Benjamin Strackany
http://www.developmentnow.com

> I guess the answer to if we log the call stack for unhandled exceptions is no
> (I'm not sure how to do that).
[quoted text clipped - 171 lines]
> > >>> Thank you,
> > >>> Keith F

Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.