Hi everyone, I think that I'm not using my SQL Caching Dependency
approach correctly. I use .NET 2.0 and SQL Server 2000. I'm
unexpectedly getting too many events fired for my
CacheItemRemovedCallbacks and I'd like your inputs. I'll provide a
summary of the issue first then some detail that I hope might raise
some red flags for people.
First some summary context:
1. My SQL Caching is at least working - that's validated - I used
the .NET 2.0 Caching strategy with SQL Server 2000
2. When I run SQL Server Trace, I unexpectedly see that the same query
is run over and over. For example, I may see this in my trace:
SELECT * FROM CountryCodes ORDER BY ShortName
Perhaps 100, 200 or more times. I don't have a clue why and I would
expect the query to get called one time only, since the Cache entry is
populated using the result of this query. The same thing happens with
all items that I'm caching that have a dependency with a cache item
removed callback. It also happens "later on" at some interval decided
upon by the .NET framework. I have CacheItemRemovedCallbacks defined
whenever a cached item is removed. The reason is usually
CacheItemRemovedReason.DependencyChanged.
Besides that I'm not sure if there's any other meaningful information
that I can provide other than code snippets. They follow, I really
hope someone can help me figure out what's going on here...thanks!
Relevant web.confg stuff. Note that I've added the <cache ... /> stuff
only today to debug based on some ideas I saw thrown out there on the
web. I've renamed the database <add ... /> elements below for security
reasons:
Web.config:
<caching>
<cache disableMemoryCollection="true" disableExpiration="true"
percentagePhysicalMemoryUsedLimit="100"/>
<sqlCacheDependency enabled="true">
<databases>
<add name="db1" connectionStringName="connStr1"
pollTime="60000"/>
<add name="db2" connectionStringName="connStr2"
pollTime="60000"/>
<add name="db2" connectionStringName="connStr3"
pollTime="60000"/>
<add name="db4" connectionStringName="connStr4"
pollTime="60000"/>
<add name="db5" connectionStringName="connStr5"
pollTime="60000"/>
</databases>
</sqlCacheDependency>
</caching>
Relevant Global.asax stuff. I use the Application_Start(...) event to
initial the caching. This gets called of course:
Global.asax code:
protected void Application_Start(object sender, EventArgs e)
{
// Code to initial all caches...snippet (for all other
db's similar code is utilized)
SqlCacheDependencyAdmin.EnableNotifications(
ConfigurationManager.ConnectionStrings[
"connStr1"].ConnectionString);
SqlCacheDependencyAdmin.EnableTableForNotifications(
ConfigurationManager.ConnectionStrings[
"connStr1"].ConnectionString,
new string[] { "table1", "table2", "table3"});
com.Web.Caching.CacheUtilities.Instance().InitializeCaches();
}
com.Web.Caching.CacheUtiilties is a class that I've created. I have an
example snippet from it relevant to country codes which are cached:
public static CacheItemRemovedCallback onCountryCodeRemove =
null;
public static CacheItemRemovedCallback
onLocalizedCountryCodeRemove = null;
public const string CACHE_KEY_COUNTRY_CODES =
"App:CountryCodes";
public void InitializeCaches()
{
CacheUtilities.onCountryCodeRemove =
new
CacheItemRemovedCallback(this.RemovedCountryCodeCallback);
this.CacheCountryCodes();
}
The code snippet above is called from my global.asax. I define my
callbacks and then explicitly initialize the cached data.
The next code snippet is the callback that occurs when the .NET
framework has invalidated the cache. It calls the same Cache method:
public void RemovedCountryCodeCallback(string cacheKey,
object cacheObject, CacheItemRemovedReason reason)
{
this.CacheCountryCodes();
}
private void CacheCountryCodes()
{
this.CacheDBCodes(CACHE_KEY_COUNTRY_CODES, "connStr1",
"SELECT * FROM CountryCodes ORDER BY ShortName",
"CountryCodes",
"db1", CacheUtilities.onCountryCodeRemove);
}
private void CacheDBCodes(string cacheKey, string
connectionStringName,
string sql, string entity, string sqlCacheDepName,
CacheItemRemovedCallback callback)
{
if (CacheInstance[cacheKey] == null)
{
SqlConnection dbConn = null;
SqlDataAdapter selectStatement;
DataSet cachedTables = new DataSet();
try
{
dbConn = new SqlConnection(
ConfigUtilities.GetConnectionString(
connectionStringName));
dbConn.Open();
selectStatement = new SqlDataAdapter(
sql, dbConn);
selectStatement.Fill(cachedTables, entity);
SqlCacheDependency cacheDep = new
SqlCacheDependency(
sqlCacheDepName, entity);
HttpRuntime.Cache.Insert(cacheKey,
cachedTables.Tables[0], cacheDep,
Cache.NoAbsoluteExpiration,
Cache.NoSlidingExpiration,
CacheItemPriority.NotRemovable, callback);
}
finally
{
// Only the db connection is required to be
closed.
// SqlDataAdapter instances are automatically
closed
dbConn.Close();
}
}
}
I think that's the major parts. Looking forward to hearing back from
you!
Alvin Bruney [MVP] - 07 Mar 2007 01:06 GMT
I think you are running into an agressive cache scavenger. The underlying
behavior was changed to be overly aggressive in 2.0 which would mean you
would get a lot more cache purges. Trying bumping the priority of the purge.

Signature
Regards,
Alvin Bruney
------------------------------------------------------
Shameless author plug
Excel Services for .NET is coming...
OWC Black book on Amazon and
www.lulu.com/owc
Professional VSTO 2005 - Wrox/Wiley
> Hi everyone, I think that I'm not using my SQL Caching Dependency
> approach correctly. I use .NET 2.0 and SQL Server 2000. I'm
[quoted text clipped - 153 lines]
> I think that's the major parts. Looking forward to hearing back from
> you!
Shan Plourde - 07 Mar 2007 03:19 GMT
Hi Alvin, thanks for your response. When you suggest bumping the
priority of the purge, re you suggesting that I set the
CacheItemPriority to a value that suggests to the .NET FW that it
perform as few purges as possible? Currently I set the
CacheItemPriority for all items that I cache as follows:
HttpRuntime.Cache.Insert(cacheKey,
cachedTables.Tables[0], cacheDep,
Cache.NoAbsoluteExpiration,
Cache.NoSlidingExpiration,
CacheItemPriority.NotRemovable, callback);
I've had CacheItemPriority.NotRemovable set pretty much since I
developed this. My tests were only done on my local development
machine, which is a Centrino core duo 2GB RAM laptop. I am speculating
that the same thing is happening on the dev and production servers for
the website.
Thanks for your suggestion and thoughts,
Shan
On Mar 6, 8:06 pm, "Alvin Bruney [MVP]" <some guy without an email
address> wrote:
> I think you are running into an agressive cache scavenger. The underlying
> behavior was changed to be overly aggressive in 2.0 which would mean you
[quoted text clipped - 166 lines]
> > I think that's the major parts. Looking forward to hearing back from
> > you!
Shan Plourde - 08 Mar 2007 13:23 GMT
Alvin / anyone - any ideas regarding this? I am still scratching my
head over this one - it would be great to hear from people that are
using the Cache API and CacheRemove callbacks in their solutions!
Shan
> Hi Alvin, thanks for your response. When you suggest bumping the
> priority of the purge, re you suggesting that I set the
[quoted text clipped - 194 lines]
> > > I think that's the major parts. Looking forward to hearing back from
> > > you!
Shan Plourde - 09 Mar 2007 13:32 GMT
Just bumping this up, hoping to hear from people that may have some
ideas regarding my Caching issue...thanks!
> Alvin / anyone - any ideas regarding this? I am still scratching my
> head over this one - it would be great to hear from people that are
[quoted text clipped - 199 lines]
> > > > I think that's the major parts. Looking forward to hearing back from
> > > > you!
Shan Plourde - 09 Mar 2007 17:13 GMT
There are so many hooks into the cache that when I run the debugger, I
get all kinds of funky things happening like the debugger freezing
because of blocking database operations. So I basically have to wait
until the debugger unfreezes itself and then I can continue...
> Just bumping this up, hoping to hear from people that may have some
> ideas regarding my Caching issue...thanks!
[quoted text clipped - 202 lines]
> > > > > I think that's the major parts. Looking forward to hearing back from
> > > > > you!
Shan Plourde - 12 Mar 2007 17:19 GMT
Microsoft...someone...anyone...? :(
> There are so many hooks into the cache that when I run the debugger, I
> get all kinds of funky things happening like the debugger freezing
[quoted text clipped - 207 lines]
> > > > > > I think that's the major parts. Looking forward to hearing back from
> > > > > > you!
Alvin Bruney [MVP] - 22 Mar 2007 01:35 GMT
If bumping the cache priority does not help you, i'd suggest you either not
use the cache or use something else. There isn't much you can do to fix a
cache scavenger because it is out of your control.
How about replace the cache with a static variable. Keep all your code the
same, just use a static variable as the store. Modify your routine to check
the static variable before you hit the database. That will definitely fix
it. The downside is that your object may not get garbage collected for a
while.

Signature
Regards,
Alvin Bruney
------------------------------------------------------
Shameless author plug
Excel Services for .NET is coming...
OWC Black book on Amazon and
www.lulu.com/owc
Professional VSTO 2005 - Wrox/Wiley
> Microsoft...someone...anyone...? :(
>
[quoted text clipped - 244 lines]
>> > > > > > from
>> > > > > > you!
Shan Plourde - 11 Apr 2007 15:55 GMT
Hi Alvin, well, basically I found that the issue was when adding an
item to the cache. If I add the item and specify a callback, the
callback simply gets invoked whether you want it to or not. After not
specifying a callback, everything is fine an dandy, but of course none
of the features of automatic cache refresh work. Anytime cached data
is updated, the caches need to be forcefully updated. Oh well...thanks
for your thoughts.
On Mar 21, 8:35 pm, "Alvin Bruney [MVP]" <some guy without an email
address> wrote:
> If bumping the cache priority does not help you, i'd suggest you either not
> use the cache or use something else. There isn't much you can do to fix a
[quoted text clipped - 226 lines]
>
> read more »