 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
.NET Forum / .NET Framework / CLR / June 2004Multi-threading article finally "finished" - reviewers welcome
Please excuse the cross-post - I'm pretty sure I've had interest in the article on all the groups this is posted to. I've finally managed to finish my article on multi-threading - at least for the moment. I'd be *very* grateful if people with any interest in multi-threading would read it (even just bits of it - it's somewhat long to go through the whole thing!) to check for accuracy, effectiveness of examples, etc. Feel free to mail me directly with comments or post them here. The article is at: http://www.pobox.com/~skeet/csharp/multithreading.html  SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too > I've finally managed to finish my article on multi-threading - at least > for the moment. I'd be *very* grateful if people with any interest in [quoted text clipped - 3 lines] > > Feel free to mail me directly with comments or post them here. I scanned through your article and it seemed very well written. I have a little bit of feedback on a few sections that you may be interested in: The section on volitile and stale data, I though the example and text was good for beginners, but didn't touch on what was (is!) the biggest issue there for me - that is the problem of "it runs great in debug mode, but when I turn on optimizations my code doesn't work any more." issue. The way the various optimizers can and do isolate variables inside loops and other very subtle things has caught me by surprise on more than one occasion. Your suggestion of wrapping things in a lock in order to avoid the 'volitile' issue is excellent - especially as it'll work in VB.NET, which lacks the volitile keyword. The code samples in the "Stopping" section, I don't agree with. First the Nitpicks: You are using variables "stopping" and "stopped", and properties "Stopping" and "Stopped". This naming convention shoud, I believe, be frowned upon. Relying on case to differentiate members and properties is just asking for trouble. It also violates the naming convention MS spells out in the MSDN documentation for .NET. I typically write this code as: Private _stop As New Threading.ManualResetEvent(False) Private _runningThread As Threading.Thread Public Sub StartTheThread() _stop.Reset() _runningThread = New Threading.Thread(AddressOf MyThread) _runningThread.Start() End Sub Public Sub StopTheThread() _stop.Set() _runningThread.Join() End Sub Private Sub MyThread() While Not _stop.WaitOne(0, False) ' Do Work Here End While End Sub I prefer this over your sample for a few reasons 1 - It is my belief (I have not benchmarked it, although I should) that testing an event for Set/Not-Set is much cheaper than acquiring and releasing a monitor (potentially) several millions times per second. 2 - If my thread loop doesn't have to run "as fast as it can", I can easilyl change the to "While Not _stop.WaitOne(1000, False)" to make my loop run once per second, while at the same time being responsive to shutdown events. In your sample code, I would need to use a Sleep to slow down the loop, which makes the thread unresponsive to shutting down. 3 - Your code is, in essece, creating a Manual Reset Event. Why create something that's already there? There were a few sections that I would have liked to see in there - mostly these are questions that I've had for a while, and haven't ever been able to get good ansers to.. 1 - Something that I've looked for information on, and found absolutly nothing, is the "Synchronized" version of the various collection classes. I have ended up having to roll my own collections in many cases, as I need to access them from multiple threads (I am sooooo looking forward to Generics!). With the lack of documentation on the synchronized collection classes, I was never comfortable using them. What about them is thread safe? I'm pretty sure Iteration is not (although it could be - the iterator could grab a lock, and prevent other iterators from proceeding until it's complete). Hashtables say (somewhere) that "one writer and multiple readers" is thread safe, but I've never been quite sure of that. What about the other collections? 2 - Some of the Win32 threading constructs that people have been using for so long are not present in .NET. Specifically Semaphores and (I think) Named Events. Any idea why? 3 - Some suggestions on how to start debugging threaded code. These could be simple suggestions like "Give all your threads unique names, so that then you're looking at the Threads window in the debugger, you can quickly tell which thread is which", or complex suggestions like "Use SoS to determine which thread is holding a lock". 4 - You don't have a section on Events (either Manual or AutoReset). Along with Monitors, these are the most used threading constructs. 5 - ReaderWriterLocks are handy at times, but are probably beyond the scope of your article. 6 - Another advanded topic that I've seen little on, but seems as if it could be really usefull, is the ability of the Threadpool to have a Handle bound to it, and provide callbacks when this handle changed. Can I bind standard wait handles to the thread pool so that I can callbacks whenever a manual or auto-reset even is changed? It seems as if there would probably be some fascinating use cases for this... SignatureChris Mullins > > I've finally managed to finish my article on multi-threading - at least > > for the moment. I'd be *very* grateful if people with any interest in [quoted text clipped - 16 lines] > issue is excellent - especially as it'll work in VB.NET, which lacks the > volitile keyword. I can't work out whether I'm surprised that VB.NET lacks volatile or not - I think I *may* have tried to find an equivalent and failed, but I'm not sure. It's a good point though - I think I've specified somewhere that "it works for me" isn't a good enough reason to believe the code is valid, but I'll emphasise it much more. Good catch. > The code samples in the "Stopping" section, I don't agree with. First the > Nitpicks: [quoted text clipped - 3 lines] > trouble. It also violates the naming convention MS spells out in the MSDN > documentation for .NET. It doesn't - the naming convention MS uses specifically doesn't mention private members. It's not a problem for VB users because anyone outside the class doesn't see the variables at all. I've used the convention for years in both Java and C# and *never* had a problem with it as far as I can remember. I find code written with my convention significantly easier to use than code which uses underscores all over the place - they just look horribly ugly to me, and disturb my train of thought when I'm reading. I'm afraid I'll stick with my convention as I do for all my code samples. > I typically write this code as: > [quoted text clipped - 22 lines] > testing an event for Set/Not-Set is much cheaper than acquiring and > releasing a monitor (potentially) several millions times per second. That's a heck of an assumption, in my view. Rather than making any assumptions, I decided to test it. The main code in both my tests was the same: using System; using System.Threading; public class LockTest { static void Main() { Worker worker = new Worker(); new Thread(new ThreadStart(worker.Run)).Start(); Thread.Sleep(10000); worker.Stop(); } } The worker object for my version is as on the website (except with stopLock declared as object, not bool - oops! Fixed the website now) but with a long counter variable which is just incremented in the loop in the same way as with the code below, which is a slightly simplified version of what you posted (just for the purposes of this test): public class Worker { long counter=0; ManualResetEvent stop = new ManualResetEvent(false); public void Stop() { stop.Set(); } public void Run() { DateTime start = DateTime.Now; while (!stop.WaitOne(0, false)) { counter++; } DateTime end = DateTime.Now; Console.WriteLine ("Count per second: {0}", counter/(end-start).TotalSeconds); } } Results (3 tests each): Using a lock: Count per second: 28038249.614792 Count per second: 28113891.7558685 Count per second: 27467033.4397496 Using a manual reset event: Count per second: 1087057.6 Count per second: 1086289.82785602 Count per second: 1082087.3 So on average, using a lock we're able to count about 25 times as fast. Just another example of threading not being a good thing to guess about :) I'd be very interested to see whether you get the same kind of results - it may *very* well depend to a very large extent on the CPU(s) of the test machines. Mine is a P4/3.06GHz laptop; no hyperthreading or anything like that. I'd be particularly interested to know the numbers on a dual processor machine... > 2 - If my thread loop doesn't have to run "as fast as it can", I can easilyl > change the to "While Not _stop.WaitOne(1000, False)" to make my loop run > once per second, while at the same time being responsive to shutdown events. > In your sample code, I would need to use a Sleep to slow down the loop, > which makes the thread unresponsive to shutting down. Well, I'd probably use Monitor.Wait in that case, myself. I'm happier with Monitor than reset events, but either would work fine here. I wouldn't usually limit it artificially like that though - normally if there's a reason to slow something down, that's because it hasn't got work the whole time - and there I'd use a work queuing system. If I want a thread to essentially be at a lower priority, I'll just lower the priority. (Ah - another thing I haven't covered :) > 3 - Your code is, in essece, creating a Manual Reset Event. Why create > something that's already there? It's not, really. There's much more to events than just testing and setting, as you've already shown with WaitOne - there are also calls to wait for potentially many events at a time, etc. I believe testing a boolean (even in a lock) is significantly simpler and easier to understand than ManualResetEvents, which is why I stick with them in general. > There were a few sections that I would have liked to see in there - mostly > these are questions that I've had for a while, and haven't ever been able to > get good ansers to.. I'll present quick answers here (to a couple of them), and see if there are other comments in this thread (no pun intended). I'll then see which bits are appropriate for inclusion in the article itself. > 1 - Something that I've looked for information on, and found absolutly > nothing, is the "Synchronized" version of the various collection classes. I [quoted text clipped - 7 lines] > is thread safe, but I've never been quite sure of that. What about the other > collections? I believe the idea of synchronized collections is that each individual operation automatically takes out a lock, but that in order to synchronize a whole series of operations, you need to lock on the SyncRoot of the collection. I've seen the same statement about multiple readers and a single writer for hashtable, but I haven't tried to verify it with stress testing or anything like that. > 2 - Some of the Win32 threading constructs that people have been using for > so long are not present in .NET. Specifically Semaphores and (I think) Named > Events. Any idea why? I really don't know. Semaphores are handy constructs, and fairly easily built out of monitors (so long as you're careful :) I may present a sample semaphore class for those who are interested. I wouldn't be at all surprised if they made it into .NET v2 anyway though. > 3 - Some suggestions on how to start debugging threaded code. These could be > simple suggestions like "Give all your threads unique names, so that then > you're looking at the Threads window in the debugger, you can quickly tell > which thread is which", or complex suggestions like "Use SoS to determine > which thread is holding a lock". Right, that's a good idea. > 4 - You don't have a section on Events (either Manual or AutoReset). Along > with Monitors, these are the most used threading constructs. Funnily enough, I've virtually never used them - except to build a fuller Monitor implementation for the Compact Framework :) I suspect it's my Java background showing through to some extent, but I more naturally think in terms of monitors, and they tend to do everything I need them to. I think they're generally simpler, which is another reason I use them :) (I do use Events when I want to wait for multiple conditions etc.) > 5 - ReaderWriterLocks are handy at times, but are probably beyond the scope > of your article. Not sure - I think that's a very good idea. I haven't had to use them myself, but almost certainly will at some time. I'll take a bit of a breather and then probably add a section about them. > 6 - Another advanded topic that I've seen little on, but seems as if it > could be really usefull, is the ability of the Threadpool to have a Handle > bound to it, and provide callbacks when this handle changed. Can I bind > standard wait handles to the thread pool so that I can callbacks whenever a > manual or auto-reset even is changed? It seems as if there would probably be > some fascinating use cases for this... Mmm... I really don't know *anything* about that topic. Sounds like something it would be worth investigating... SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too > > Your > > suggestion of wrapping things in a lock in order to avoid the 'volitile' [quoted text clipped - 4 lines] > not - I think I *may* have tried to find an equivalent and failed, but > I'm not sure. I've looked several times for it, and each time been quite shocked not to find it. I always feel if I dig "just a bit deeper" I'll find it. Ah well. One more thing that occasionally forces me to write code in C#... > > You are using variables "stopping" and "stopped", and properties "Stopping" > > and "Stopped". This naming convention shoud, I believe, be frowned upon. [...] > I've used the convention for years in both Java and C# and *never* had > a problem with it as far as I can remember. > I'm afraid I'll stick with my convention as I do for all my code > samples. 'Fair Nuff. I still think it's evil though. > > I prefer this over your sample for a few reasons > > 1 - It is my belief (I have not benchmarked it, although I should) that [quoted text clipped - 3 lines] > That's a heck of an assumption, in my view. Rather than making any > assumptions, I decided to test it. [...] > Results (3 tests each): > [quoted text clipped - 9 lines] > > So on average, using a lock we're able to count about 25 times as fast. Wow. I should know better by now than to make assumptions on these things. That's what Compuware's profiler is for. > > 1 - Something that I've looked for information on, and found absolutly > > nothing, is the "Synchronized" version of the various collection classes. I [quoted text clipped - 12 lines] > synchronize a whole series of operations, you need to lock on the > SyncRoot of the collection. I'm pretty sure this is what they do too, but I've never seen this actually documented anywhere. Their complete non-existance on the CF has meant that I needed to write my own thread-safe collections anyway, so I've never looked into them in great detail. (But for a threading article on the standard CLR, I think they're a good topic) > > 2 - Some of the Win32 threading constructs that people have been using for > > so long are not present in .NET. Specifically Semaphores and (I think) Named [quoted text clipped - 4 lines] > sample semaphore class for those who are interested. I wouldn't be at > all surprised if they made it into .NET v2 anyway though. The one thing that Semaphores have, that I don't think you can duplicate with Monitors, is that they're cross-process. They're really handy for "Is my application already running?" checks, as well as a wide variety of other computer-wide resource management tricks. > > 4 - You don't have a section on Events (either Manual or AutoReset). Along > > with Monitors, these are the most used threading constructs. > > Funnily enough, I've virtually never used them - except to build a > fuller Monitor implementation for the Compact Framework :) That's funny - I used a combination of Win32 calls and Monitors to implement a full ManualResetevent on the compact framework. For some silly reason there is no WaitHandle.WaitOne(Timeout, false) overload on the CF, which makes the class pretty useless for a wide vareity of useses.. It's either wait forever, or don't wait at all. If you want the source to this, drop me an email some time. (That's the least I can do for you writing that UTF32String class a few weeks back) SignatureChris Mullins > > I can't work out whether I'm surprised that VB.NET lacks volatile or > > not - I think I *may* have tried to find an equivalent and failed, but [quoted text clipped - 4 lines] > well. > One more thing that occasionally forces me to write code in C#... Reflector decompiles a volatile variable as having ModReq(IsVolatile) after it, which I believe is what's in IL. It doesn't look like you can apply that within VB though. <big snip> > > I believe the idea of synchronized collections is that each individual > > operation automatically takes out a lock, but that in order to [quoted text clipped - 6 lines] > never looked into them in great detail. (But for a threading article on the > standard CLR, I think they're a good topic) Right. I might see what else I can dig up about them - without the source code being available, it's hard to say. I can always look at the Rotor implementation, I suppose. > > I really don't know. Semaphores are handy constructs, and fairly easily > > built out of monitors (so long as you're careful :) I may present a [quoted text clipped - 5 lines] > my application already running?" checks, as well as a wide variety of > other computer-wide resource management tricks. Ah - I've been thinking in terms of the computer science idea of a counting semaphore, rather than the Win32 specific concept. For cross-process "am I running" checks, however, a Mutex works pretty well. See http://www.pobox.com/~skeet/csharp/faq/#one.application.instance I dare say there are other uses which Mutex wouldn't cut it for though. > > Funnily enough, I've virtually never used them - except to build a > > fuller Monitor implementation for the Compact Framework :) [quoted text clipped - 6 lines] > vareity of useses.. It's either wait forever, or don't wait at all. If > you want the source to this, drop me an email some time. Have you thought about contributing it to OpenNETCF? That's what I did with what is now called MonitorEx. I had the same problem as you did though - I can't implement Wait with a timeout. I suspect if either of us could implement one of them, we could use it to implement the other :) (I had a few ideas about how to do it using timers, but the ideas became pretty horrible.) > (That's the least I can do for you writing that UTF32String class a few > weeks back) Ah, that was for you, was it? I couldn't remember. Did you ever use it, or were you able to make do with a couple of bits of it? It was fun writing it either way :) (I dread to think what that says about me...) SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too "Jon Skeet [C# MVP]" <skeet@pobox.com> wrote in message: [UTF32 String Class] > > (That's the least I can do for you writing that UTF32String class a few > > weeks back) > > Ah, that was for you, was it? I couldn't remember. Did you ever use it, > or were you able to make do with a couple of bits of it? It was fun > writing it either way :) (I dread to think what that says about me...) Unfortunatly I ended up not being able to use it, although I spent quite some time with it before coming to that conclusion. I needed it to perform StringPrep for some XMPP stuff, and some of the required steps for that are pretty ugly. The two steps that specifically made me decide not to implement the RFC myself were the cannocial form transforms, and the KC Normalization step. I couldn't for the life of me figure out a reliable and space-efficient way to get all the Unicode Data tables required for that into a usable form. They're all available for download on the Unicode site, but it's like 60 megs of data, and I really, really wasn't looking forward to writing a perl script of some kind to transform them into .NET resource files and the like. It was a nightmare. I did end up learning more about Unicode than I ever wanted to know, that's for sure! I think I've finally settled on wrapping 'libidn' (which is a LGPL licensed project) in a Win32 DLL, and calling that. Ick. SignatureChris Mullins > > I can't work out whether I'm surprised that VB.NET lacks volatile or > > not - I think I *may* have tried to find an equivalent and failed, but [quoted text clipped - 4 lines] > well. > One more thing that occasionally forces me to write code in C#... See Thread.VolatileRead and Thread.VolatileWrite. > Results (3 tests each): > [quoted text clipped - 18 lines] > anything like that. I'd be particularly interested to know the numbers > on a dual processor machine... The numbers are a bit better on my dual processor machine, but not substantially. Looks like lock is about 10x faster on this machine. I don't think that these results would be sufficent to state that the differences are due to multiple processor machines so much as difference in speeds between the processors I used and the one you used. I ran this on a dual 2.2 ghz xeon, with hyperthreading. I was able to verify that the speed of locks and events are as identical as benchmarks can be when affinity targets one or two physical processors, or all four virtual processors, however. I would guess that the speed of these sync mechanisms are pretty much neutral to the number of processors as long as their is only one thread. A more interesting, but more complicated, test would be to test lock and event speed across multiple threads. Using a lock: Count per second: 5581104.27543036 Count per second: 5615276.9 Count per second: 5500714.86697966 Using a manual reset event: Count per second: 500654.6 Count per second: 489649.3771518 Count per second: 494306.7 This is not surprising at all. The distinction between Monitor and Mutex in .Net is the same as the distinction between critical sections and mutexes in Win32. The critical section will in many cases completely bypass the underlying lock. If you have multiple processors, you increase concurrency and therefore lock contention, forcing the crit sect to actually enter its underlying lock more often. > > Results (3 tests each): > > [quoted text clipped - 41 lines] > Count per second: 489649.3771518 > Count per second: 494306.7 > This is not surprising at all. The distinction between Monitor and Mutex in > .Net is the same as the distinction between critical sections and mutexes in > Win32. The critical section will in many cases completely bypass the > underlying lock. If you have multiple processors, you increase concurrency > and therefore lock contention, forcing the crit sect to actually enter its > underlying lock more often. FWIW, here is a simple critical section in c#. A lot more state could be added, but this is basic. using System; using System.Threading; namespace SyncPrimitives { /// <summary> /// Simple CriticalSection Implementation in c#. /// Note we don't use a "busy wait" but a test and sleep method. /// The current thread gives up it cpu slice to allow another thread to run. /// The idea is that owner thread will run and release the lock and we will get /// the lock when we wake and test again. This should be the normal behavior /// if critical sections are only used to protect very short sections /// of code. Two threads will only rarely attempt to enter the critical section /// at the same time, and even then, the loser will only have to spin/sleep for /// a brief time. We could also upgrade to waiting on an event, but that would be /// another version and we only want to show simple implementation. We could /// also add thead "state" so that only owner can Exit and owner can reenter lock. /// </summary> public sealed class CriticalSection { private int spinLock = 0; // Not locked. public CriticalSection() { } /// <summary> /// Enter the CriticalSection exclusively or wait until we do. /// A CriticalSection is intended to be Entered and Exited quickly. /// </summary> public void Enter() { while( Interlocked.Exchange(ref spinLock, 1) == 1 ) Thread.Sleep(0); // We got the lock as it was 0 (unlocked) before Exchange. } public bool Enter(int milliseconds) { if ( Interlocked.Exchange(ref spinLock, 1) == 0 ) return true; Thread.Sleep(milliseconds); if ( Interlocked.Exchange(ref spinLock, 1) == 0 ) return true; return false; } /// <summary> /// Non-blocking version of Enter method. /// </summary> /// <returns></returns> public bool TryEnter() { if ( Interlocked.Exchange(ref spinLock, 1) == 0 ) return true; return false; } /// <summary> /// Release the lock. Note that any thread could call Exit with this /// implementation, which resembles Semaphore Release behavior. /// </summary> public void Exit() { Interlocked.Exchange(ref spinLock, 0); } public bool IsLocked() { if ( Interlocked.CompareExchange(ref spinLock, 0, 0) == 0 ) return false; return true; } } } Hi, Spinlock could be fine if it is held only occasionally (i.e. once in a while) and only for a few loops. Otherwise it is quite bad for laptop battery power. Nowadays processor architectures/technologies such as Intel's SpeedStep/Clock Modulation, AMD's PowerNow and Transmeta Crusoe Longrun support dropping of processor's clock frequency based on processor's usage pattern. Ie. if processor finds that it uses lots of time in idle, then it is free to drop frequency for saving battery power (both less power consumption by processor itself and better thermal control - i.e. less power to fans). Spinlock is actually an attempt to simulate thread's idle wait but from processor point of view it is indistinguishable from any other processing intensive operations (definitely not an idle). Every spinlock resets processor's last contiguous idle time slice and reduces overall weight of idle time. Frequent use of spinlocks could cause that processor will be always running on high frequency without any reasons, burning battery power just for running dumb spins and fans that has to work double for cooling overloaded processor. If someone sits on the plain with the laptop and run program that relies on spinlocks - he/she would not really appreciate what this program does for his/her batteries (not speaking of extreme circumstances when people burns genitalia by their laptops see: http://www.wired.com/news/business/0,1367,56552,00.html). -Valery http://www.harper.no/valery >> This is not surprising at all. The distinction between Monitor and Mutex > in [quoted text clipped - 89 lines] > } > } > Spinlock could be fine if it is held only occasionally (i.e. once in a > while) and only for a few loops. Correct. I think I pointed that out in the code. > to fans). Spinlock is actually an attempt to simulate thread's idle wait but > from processor point of view it is indistinguishable from any other > processing intensive operations (definitely not an idle). Every spinlock Note this is not a busy-wait. The spin sleeps or gives up it slice so the owner can free it. If the owner is busy, the cpu is working anyway. If the owner is done, the next thread gets the lock, does its thing, and exits the lock. This issue can arise if the owner blocks for long time while owning the lock. That is not good and another lock should be used. It should only be protecting a few lines in most cases. AFIK, every primitive is ultimately based on a spin lock at the lowest level. So spins are mostly used for creating higher sync primitives. If this is not the case, please inform. > If someone sits on the plain with the > laptop and run program that relies on spinlocks - he/she would not really > appreciate what this program does for his/her batteries (not speaking of > extreme circumstances when people burns genitalia by their laptops see: http://www.wired.com/news/business/0,1367,56552,00.html). Actually that kinda happened to me once, so I feel the pain :-) Cheers. SignatureWilliam Stacey, MVP > AFIK, every primitive is > ultimately based on a spin lock at the lowest level. If we talk about kernel mode Win32 synchronization objects - than it's not quite so. Yes, spin lock is used by code, but it is different in two ways: - When thread scheduler sees that thread is waiting on Win32 synchronization objects it immediately schedules context switch to another runnable thread instead of wasting time spinning. - Additionally, kernel mode spin lock sets idle bit on interrupt command register (on Intel) or use other means (on other processor types) to notify processor that thread is running idle operation. Interrupt command register and this bit is not accessible from the user mode. -Valery. http://www.harper.no/valery >> Spinlock could be fine if it is held only occasionally (i.e. once in a >> while) and only for a few loops. [quoted text clipped - 26 lines] > > Actually that kinda happened to me once, so I feel the pain :-) Cheers. Thanks. You have a c implementation example handy? SignatureWilliam Stacey, MVP > > AFIK, every primitive is > > ultimately based on a spin lock at the lowest level. [quoted text clipped - 43 lines] > > > > Actually that kinda happened to me once, so I feel the pain :-) Cheers. Here is another version I did that goes right to an event wait if lock is locked. Please review for errors and comments as this is a bit tricky. You could cheat and use a Monitor, but that would not be very interesting. Cheers! using System; using System.Threading; namespace SyncPrimitives { /// <summary> /// Simple CriticalSection Implementation in c#. /// Note we don't use a "busy wait" but a test and wait. /// This version uses event wait if lock is owned and keeps state of current owner. /// </summary> public sealed class CriticalSection { private volatile Thread owner = null; private int lockCount = 0; private readonly AutoResetEvent lockedEvent = new AutoResetEvent(false); private const int maxSpinCount = 400; public CriticalSection() { } public bool Enter(int milliseconds) { DateTime startTime; int spinCount = 0; //Block while locked (unless we are the owner). while( Interlocked.CompareExchange(ref lockCount, 1, 0) > 0 ) { // It is locked, so check if we are the owner. if ( Object.ReferenceEquals(owner, Thread.CurrentThread) ) { // We already own the lock, so just increment count. lockCount++; return true; } // Note if we get to this point, we are not the owner. if ( milliseconds == 0 ) return false; if ( spinCount < maxSpinCount ) { spinCount++; Thread.Sleep(0); // Don't WaitOne() until we have tried maxSpinCount Times. continue; // The idea is we will get the lock very quickly and don't want overhead of wait in the fast path. } startTime = DateTime.Now; try { if ( ! lockedEvent.WaitOne(milliseconds, false) ) return false; // Did not get a signal within timeout. } catch { lockedEvent.Set(); // Could be got signalled before the exception, so signal next thread. throw; } milliseconds = (int)(milliseconds - ((TimeSpan)(DateTime.Now - startTime)).TotalMilliseconds); if ( milliseconds < 0 ) milliseconds = 0; } // We got the lock and set to 1 as prev lockCount was zero. owner = Thread.CurrentThread; return true; } public bool TryEnter() { return Enter(0); } public void Exit() { if ( Interlocked.CompareExchange(ref lockCount, 0, 0) == 0 ) throw new SynchronizationLockException("Can not exit lock that has not been entered."); // It is locked. See if we are the owner. If not, throw exception. if ( ! Object.ReferenceEquals(owner, Thread.CurrentThread) ) throw new SynchronizationLockException("Not current owner of lock."); // We are the owner, so decrement count. lockCount--; if ( lockCount == 0 ) { owner = null; lockedEvent.Set(); } } public bool IsLocked() { if ( Interlocked.CompareExchange(ref lockCount, 0, 0) == 0 ) return false; return true; } } } SignatureWilliam Stacey, MVP Interesting point Valery. It's hard to write spinlock at the level of abstraction we are at in .NET. Ideally the JIT compiler could choose a spinlock implementation based on the environment. On a single processor machine the spinning is a waste of processor time, and on a laptop the spinlock causes additional power consumption - both good reasons to switch to a different implementation in these environments. -- Scott http://www.OdeToCode.com >Hi, >Spinlock could be fine if it is held only occasionally (i.e. once in a [quoted text clipped - 20 lines] >-Valery >http://www.harper.no/valery It's best to rely on Win32 synchronization objects, because they know how to handle idle operations. For example: when thread scheduler sees that thread is waiting on Win32 synchronization objects it immediately schedules context switch to another runnable thread instead of wasting time spinning. Additionally, kernel mode spin lock sets idle bit on interrupt command register (on Intel) or use other means (on different processor types) to notify processor that thread is running idle operation. Interrupt command register and this bit is not accessible from the user mode. -Valery. http://www.harper.no/valery > Interesting point Valery. > [quoted text clipped - 36 lines] >>-Valery >>http://www.harper.no/valery My second implementation uses system autoresetevent for the wait, so this should not be an issue. SignatureWilliam Stacey, MVP > It's best to rely on Win32 synchronization objects, because they know how to > handle idle operations. [quoted text clipped - 51 lines] > >>-Valery > >>http://www.harper.no/valery > On a single processor machine the spinning > is a waste of processor time, Not if you do a wait or sleep(0) it is not. That gives the other thread a chance to give up the lock, is which is exactly what you want. SignatureWilliam Stacey, MVP >> On a single processor machine the spinning >> is a waste of processor time, > >Not if you do a wait or sleep(0) it is not. That gives the other thread a >chance to give up the lock, is which is exactly what you want. I guess it depends on your definition of spinlock. If spinning means "do not preempt me because I know the lock will be free any nanosecond now", then spinlocks are about locking CPUs, not threads, and there wouldn't be any point in chewing up cycles on a uni-proc system. If spinning means "try once then yield and repeat", then I'm not sure what benefit there is actually. I'd have to test this one out quite a bit to be comfortable with it! -- Scott -- Scott http://www.OdeToCode.com > If spinning means "do not preempt me because I know the lock will be > free any nanosecond now", then spinlocks are about locking CPUs, not > threads, and there wouldn't be any point in chewing up cycles on a > uni-proc system. Agreed. On a single cpu, there is no benefit in a *busy-wait as you will spin for rest of slice as you can't regain the lock anyway until you give up your slice. Event on a mult-proc, I would not use busy-wait but would still sleep unless you are tuning for something very specific. > If spinning means "try once then yield and repeat", then I'm not sure > what benefit there is actually. I'd have to test this one out quite a > bit to be comfortable with it! Benefit is the idea that in general, the lock will be held for only a few instruction, so do upgrade to monitor or event until you try X times (sleeping each time to give other thread a chance to release the lock.) In the general case, there there is lock contention, then you should get at very next cycle. If you spin/sleep X times, then owner must be blocked on long operation so upgrade to waiting on a monitor or event. Please test and reply if bug or error. More of a thing to learn from as using a Monitor is probably better in most cases. SignatureWilliam Stacey, MVP Jon Skeet, Its some good work u have in there. you have almost covered all the technical aspect that goes in with handling the thread. I have already give a print!!. Having said this, I'll like to add that, I generally like to treat a Thread as an fully self managed Object rather than a method of an object that is used by thread(so to say...). OK now I am confusing... what I mean is(more of a design issue), in every example I see, there are code written like this: ThreadStart job = new ThreadStart(myobj.ThreadJob); Thread thread = new Thread(job); thread.Start(); well there is nothing wrong with this, but I like to see something like MyThread thread = new MyThread(); this makes my design lot cleaner to understand, How I do this is something like this. class abstract ThreadBase { << events for OnThreadStart, OnThreadDone, OnXXX>> void abstract Run(); protected virtual void OuterRun() { << trigger OnStartEvent>> try { Run(); } catch(...){<<handle Exception & raise events>>} << trigger OnThreadDoneEvent>> } public ThreadBase()//constructor { ThreadStart job = new ThreadStart(OuterRun); Thread thread = new Thread(job); thread.Start(); } } this way the method that is run in the thread & the thread itself are treated as from single object. and I have lot of common functionality in the ThreadBase so that I don't have to re write it for every time I want to use a thread. all I have to do for creating a new thread functionality is derive a class from ThreadBase and override its Run() method and other OnXXX method. I may be biased here with this approach as I come form Delphi background, but I have not found much issue with it so far. Regards Kiran > Jon Skeet, > [quoted text clipped - 18 lines] > this makes my design lot cleaner to understand, How I do this is something > like this. <snip> > this way the method that is run in the thread & the thread itself are > treated as from single object. Whereas I would discourage that approach. It's the one that Java took until it gained the Runnable interface, and these days most people will strongly discourage people from subclassing Thread in Java. I see no advantage in conflating the two ideas, to be honest. There's nothing wrong with having a general purpose class which can be used to start a thread and is still provided with a ThreadStart delegate to actually run, but to restrict things by derivation is a bad idea, in my view. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Very nice article; had a quick look through it, its an easy way to explain not-so-easy issues :) Back in the VB days everybody wanted multi-threads (even did it with api *highly unstable*) but now I don?t see people using it as much on .NET as it would seem natural to. I myself had a time and half on a multi-threading application on .NET where I learned just how messy can it get without some sort of mutex. > Very nice article; had a quick look through it, its an easy way to explain > not-so-easy issues :) Cool, thanks :) As with most of the articles, it was born out of laziness: I don't want to spend ages typing the same kinds of answer out again and again! > Back in the VB days everybody wanted multi-threads (even did it with api > *highly unstable*) but now I don´t see people using it as much on .NET as it > would seem natural to. Interesting. I certainly cringe when Application.DoEvents is recommended for something which naturally lends itself to multiple threads. > I myself had a time and half on a multi-threading application on .NET where > I learned just how messy can it get without some sort of mutex. I would laugh, but the memory of similar situations is too painful ;) SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too A good addition might be a discussion of ManualResetEvent and AutoResetEvent especially the differences between them and when they are appropriate ... I get questions on that all the time. Greg > Please excuse the cross-post - I'm pretty sure I've had interest in the > article on all the groups this is posted to. [quoted text clipped - 9 lines] > The article is at: > http://www.pobox.com/~skeet/csharp/multithreading.html > A good addition might be a discussion of ManualResetEvent and AutoResetEvent > especially the differences between them and when they are appropriate ... I > get questions on that all the time. Right. That's two requests for them: they're definitely going in. This is a good opportunity to get myself more thoroughly familiar with them, too :) SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Please please please split it out into multiple html pages - one long one is horrible. Do you keep your .cs code files like that? No, I didn't think so! HTML deserves the same respect and for exactly the same reason, same goes for anyone reading it. > Please please please split it out into multiple html pages - one long > one is horrible. Agreed. > Do you keep your .cs code files like that? No, I didn't think so! > HTML deserves the same respect and for exactly the same reason, same > goes for anyone reading it. The trouble is, it started off being a relatively short article, which was fine as one page. It's definitely too long now. I'll work out exactly how to split it up soon. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too > Please please please split it out into multiple html pages - one long one is horrible. > > Do you keep your .cs code files like that? No, I didn't think so! HTML deserves the same respect and for exactly the same reason, same goes for anyone reading it. It's easier to print it that way, though. Do you split your WinWord documents page per page? SignatureWildHeart'2k4 Stefano "WildHeart" Lanzavecchia <stf+msnews@apl.it> wrote: > > Do you keep your .cs code files like that? No, I didn't think so! HTML > > deserves the same respect and for exactly the same reason, same goes for > > anyone reading it. > > It's easier to print it that way, though. Do you split your WinWord > documents page per page? I can always provide a print version which is just the concatenation of all the other pages - I think that's what most sites do. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too "triple-new" syntax is one of the great features of C# - check out: (new Thread(new ThreadStart((new HistoricalCatchup( accessdb, drive)).StartCatchup))).Start(); an example of mine that starts a thread on a member function of a class (due to the fact that it has to have the parameters accessdb and drive) > Please excuse the cross-post - I'm pretty sure I've had interest in the > article on all the groups this is posted to. [quoted text clipped - 9 lines] > The article is at: > http://www.pobox.com/~skeet/csharp/multithreading.html > "triple-new" syntax is one of the great features of C# - check out: > [quoted text clipped - 4 lines] > class (due to the fact that it has to have the parameters accessdb > and drive) Of course, there's nothing *forcing* you to do that all on one line - you could easily split it up. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Hi Jon, I did not go deep in your page code, however what I see often in the newsgroups that in my opinion people start using multithreading on the wrong places. To say it simple, I miss the chapter (or did not see it), "Why using multithreading and when using multithreading". The "why" is for me to use processor power when there is a process stopped because that it needs user or another acknowledge on a process to let it go on. Another process can than run in the time that waiting is done. For the when it is for me. -There has to be more processes, which are independent from a user, or any other processes running, which can have interrupts because of IO or things like that (and independent means even not need an end acknowledge going on). -When this is not the issue however there are more processors and known that there is only one application running there can be used optimistic multithreading. A nice example is printing which is by default because it is done by the OS already a multithreading process. (For you because you are in my opinion so on words, that is multiprocessing (or even a better classic word for it multiprogram) in fact, however when it not has been, it would have been nice to do it multithreading). This is only to give you some ideas about this. Cor > I did not go deep in your page code, however what I see often in the > newsgroups that in my opinion people start using multithreading on the wrong > places. > > To say it simple, I miss the chapter (or did not see it), "Why using > multithreading and when using multithreading". Yes, that wouldn't be a bad idea. > The "why" is for me to use processor power when there is a process stopped > because that it needs user or another acknowledge on a process to let it go > on. Another process can than run in the time that waiting is done. Not necessarily. It also allows things like a UI updating while doing something else in the background as well. > For the when it is for me. > -There has to be more processes, which are independent from a user, or any > other processes running, which can have interrupts because of IO or things > like that (and independent means even not need an end acknowledge going on). They don't need to be indepedent, or use interrupts. It's quite possible to have a CPU-bound background thread (preferably at a low priority), and have it update the UI (using Control.Invoke) to give the end user information about what's happening. > -When this is not the issue however there are more processors and known that > there is only one application running there can be used optimistic > multithreading. Not sure what you mean here. Could you try rewriting that sentence? > A nice example is printing which is by default because it is done by the OS > already a multithreading process. (For you because you are in my opinion so > on words, that is multiprocessing (or even a better classic word for it > multiprogram) in fact, however when it not has been, it would have been nice > to do it multithreading). (The disadvantage with doing it in threads rather than using a separate spooler is that you'd have to keep the application running while you were printing, of course.) I already draw a bit of a parallel between multi-tasking and multi- threading, but I agree that it would be a good idea to write more about it, with other examples. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Hi Jon, And again starts a discussion. > Not necessarily. It also allows things like a UI updating while doing > something else in the background as well. This gives no advantages only slower UI updating. There is no advantage when other processors do not take things over, only slower UI, however when other processors are taking things over (what can be in painting), than it is again IO and you can do it in more threads. As far as I know does, although the hyper threading optimizes the multithreading process, a processor only one thing in a time. > They don't need to be indepedent, or use interrupts. It's quite > possible to have a CPU-bound background thread (preferably at a low > priority), and have it update the UI (using Control.Invoke) to give the > end user information about what's happening. This is in my opinion only interesting when there is an actif foreground thread, than it completly as I tried to write. More processorrs: What I wrote above is not always true when there are more processors, however this is not really a multithreading isue. When there is only one application running, you can use the not used processors to do processing in an optimistic way. When there is a reason it can not be used you can garbage it. However do never forget that 2 processors have to be processed. I hope this gives some more explanations about my thoughts about this? Cor > And again starts a discussion. > > > Not necessarily. It also allows things like a UI updating while doing > > something else in the background as well. > This gives no advantages only slower UI updating. Yes it does! 1) You can give the user information as to progress 2) If windows are dragged over the UI, it can repaint rather just leaving a white rectangle 3) You can allow buttons etc for cancelling the other thread's work, and the app can be closed in the normal way 4) The user may be able to do other things while the processing is going on (depending on the app) > There is no advantage when > other processors do not take things over, only slower UI, however when other > processors are taking things over (what can be in painting), than it is > again IO and you can do it in more threads. The point is that you have a *working* UI. It will be slower than if there isn't the other process, but it's better than a completely unresponsive UI. > As far as I know does, although the hyper threading optimizes the > multithreading process, a processor only one thing in a time. No - hyperthreading makes the processor look like two processors as far as the OS is concerned, so two threads can run on it at the same time, using different parts of the processor. > > They don't need to be indepedent, or use interrupts. It's quite > > possible to have a CPU-bound background thread (preferably at a low [quoted text clipped - 3 lines] > This is in my opinion only interesting when there is an actif foreground > thread, than it completly as I tried to write. The UI *is* an active foreground thread. If you want the UI to remain responsive *at all* then either you need to call Application.DoEvents (urgh!) or you need another thread. > More processorrs: > What I wrote above is not always true when there are more processors, > however this is not really a multithreading isue. Well, it's something you need to be aware of when threading - you may wish to tailor the number of threads you use depending on the number of processors, in some special cases. (I could mention that in the article, actually.) > When there is only one application running, you can use the not used > processors to do processing in an optimistic way. When there is a reason it > can not be used you can garbage it. However do never forget that 2 > processors have to be processed. > > I hope this gives some more explanations about my thoughts about this? Yes, but I disagree very strongly. It would be easy to come up with a couple of example programs, one of which uses threading and one of which doesn't, if you're still not convinced. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Jon, Please no constructive quoting otherwise we can better stop this, I am sure you do it not express, however read first the complete message and add than your comments? > > > Not necessarily. It also allows things like a UI updating while doing > > > something else in the background as well. [quoted text clipped - 10 lines] > 4) The user may be able to do other things while the processing is > going on (depending on the app) I wrote 4. The others can be done in another way while there is than a higher throughput, I get very much the idea that you forget that the OS has to process the threads, which is not measurable inside a program. > > There is no advantage when > > other processors do not take things over, only slower UI, however when other [quoted text clipped - 4 lines] > there isn't the other process, but it's better than a completely > unresponsive UI. See my previous sentence. > > As far as I know does, although the hyper threading optimizes the > > multithreading process, a processor only one thing in a time. > > No - hyperthreading makes the processor look like two processors as far > as the OS is concerned, so two threads can run on it at the same time, > using different parts of the processor. You write, "No", what is the difference with what I wrote? You write "run" which is true, I wrote "process" in my opinion are that two different things. > > > They don't need to be indepedent, or use interrupts. It's quite > > > possible to have a CPU-bound background thread (preferably at a low [quoted text clipped - 7 lines] > responsive *at all* then either you need to call Application.DoEvents > (urgh!) or you need another thread. see my previous sentence > > More processorrs: > > What I wrote above is not always true when there are more processors, [quoted text clipped - 15 lines] > couple of example programs, one of which uses threading and one of > which doesn't, if you're still not convinced. I do use multithreading and there are a lot of advantages, however I have seen that it is used in some circumstances not needed. Processing multithreading takes time, so it can have a negative influence on the througput of the program. Cor > Please no constructive quoting otherwise we can better stop this, I am sure > you do it not express, however read first the complete message and add than > your comments? I really don't know what you mean. I haven't done any creative snipping or anything here... > > > > Not necessarily. It also allows things like a UI updating while doing > > > > something else in the background as well. [quoted text clipped - 12 lines] > > I wrote 4. Where, exactly? Maybe it was in one of the sentences where I didn't fully understand your meaning. > The others can be done in another way while there is than a > higher throughput, I get very much the idea that you forget that the OS has > to process the threads, which is not measurable inside a program. No, the others are *definitely* better done using threads. How else would you do it? Using Application.DoEvents()? That leads to horrible re-entrancy problems, in my opinion, and is really a hack dating back to a time when threads weren't easily available. It also involves guesswork as to where is a good place to call DoEvents - the more often you call it, the more throughput you lose, but the less often you call it, the less responsive the UI is. > > > There is no advantage when > > > other processors do not take things over, only slower UI, however when [quoted text clipped - 7 lines] > > > See my previous sentence. You've never said how you'd achieve a working GUI, as far as I've seen. > > > As far as I know does, although the hyper threading optimizes the > > > multithreading process, a processor only one thing in a time. [quoted text clipped - 6 lines] > which is true, I wrote "process" in my opinion are that two different > things. You didn't actually write any verb in that clause - you just wrote "a processor only one thing at a time". However, a hyperthreading processor really *can* be processing two threads at a time. For instance, an instruction for one thread might be being decoded while the instruction for another thread runs in the floating point unit. Given the deep pipelining in many processors, this is only a more complicated version of what's been happening for ages, with many instructions being processed at the same time anyway. It's just that in the past, those instructions all had to be from the same thread. > > > > They don't need to be indepedent, or use interrupts. It's quite > > > > possible to have a CPU-bound background thread (preferably at a low [quoted text clipped - 11 lines] > > see my previous sentence I really don't see how that covers it. Or are you advocating UIs which are unresponsive? Note that "unresponsive" doesn't just mean that the user needs to be able to do other things - an app which doesn't even repaint is unresponsive, in my view. <snip> > > Yes, but I disagree very strongly. It would be easy to come up with a > > couple of example programs, one of which uses threading and one of [quoted text clipped - 4 lines] > multithreading takes time, so it can have a negative influence on the > througput of the program. It certainly can - but not in the examples presented here. Or rather, the throughput *will* be very slightly diminished, but at the gain of a *much* better user experience. I would rather have an interface which took 4 hours to do something but kept me informed of what was going on, let me cancel it, etc than an app which took 3 hours and 55 minutes but never bothered to repaint itself. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Hi Jon, I stop with this thread, I wrote you why I did it and was not interested starting a discussion about this. The differences are so thin, that it makes no sense the only thing I would bring to you is that a thread is not forever the solution as I often get the idea that some people are thinking, it can be a sollution and you have to think twice before you use it. I gave you some samples to help you to add to your page, when you do not agree that with me is for me no problem, do it your way, it are your pages. Do not think I am angry or something, I am not at all. However this are parts where in my opinion I can be right and where you can be right depending on the circumstances, so discussing using a newsgroup makes it to difficult. Cor <snip> > However this are parts where in my opinion I can be right and where you can > be right depending on the circumstances, so discussing using a newsgroup > makes it to difficult. If you say so. I don't think it's *ever* right (in anything but a throwaway test program) to perform long-running operations in the UI thread, unless they absolutely *have* to due to them making a lot of changes to the UI controls themselves. Almost anything not directly related to the UI shouldn't occur on the UI thread, in my view. I think you'll find most people would agree with me on that, too. I'll certainly add something to the page about some times when it's *not* worth creating lots of threads, or when you create a thread but then don't use the fact that you've created it (e.g. by calling BeginInvoke and then calling EndInvoke on the next line, which effectively makes it synchronous anyway). SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too Hi Jon, Not about the rest of this thread, only about your answer and not agreeing with you all the rest you stated, however to give you "one" sample to think about. > Almost anything not directly > related to the UI shouldn't occur on the UI thread, in my view. I think > you'll find most people would agree with me on that, too. Every process shorter than the human reaction time of the client is useless to place in an extra thread. Cor > Not about the rest of this thread, only about your answer and not agreeing > with you all the rest you stated, however to give you "one" sample to think [quoted text clipped - 6 lines] > Every process shorter than the human reaction time of the client is useless > to place in an extra thread. Yes, I'd agree with you on that one, if you can really guarantee it'll take that short a time. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too You might consider including some discussion of ReaderWriterLock -- kind of an advanced feature, but could be useful in certain scenarios. > Please excuse the cross-post - I'm pretty sure I've had interest in the > article on all the groups this is posted to. [quoted text clipped - 9 lines] > The article is at: > http://www.pobox.com/~skeet/csharp/multithreading.html > You might consider including some discussion of ReaderWriterLock -- kind of > an advanced feature, but could be useful in certain scenarios. Yup, it's on my wishlist. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet If replying to the group, please do not mail me too > - When thread scheduler sees that thread is waiting on Win32 > synchronization objects it immediately schedules context switch to another > runnable thread instead of wasting time spinning. shoud read as: When thread scheduler sees that thread is waiting on non-signaled Win32 synchronization objects... -Valery. http://www.harper.no/valery "Valery Pryamikov" <Valery@nospam.harper.no> wrote in message news:... >> AFIK, every primitive is >> ultimately based on a spin lock at the lowest level. [quoted text clipped - 12 lines] > -Valery. > http://www.harper.no/valery Any good links on this detail? SignatureWilliam Stacey, MVP > > - When thread scheduler sees that thread is waiting on Win32 > > synchronization objects it immediately schedules context switch to another [quoted text clipped - 24 lines] > > -Valery. > > http://www.harper.no/valery It was described in several different sources (among others I belive you can find it in Richter's book and Russinovich's book). For online link, check this: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndllpro/html/m sdn_scalabil.asp I don't have C sample of setting ICR. It could only be done in kernel development and is processor related (+it would not be C, but inline assembler). If you are interesting in information about ICR on X86 Intel processor you can see it in developers Intel Manual. (btw, Itanium uses another way of signalling idle) For X68 info see links in Manuals section of this page: http://www.intel.com/design/Pentium4/documentation.htm -Valery. http://www.harper.no/valery > Any good links on this detail? > [quoted text clipped - 31 lines] >> > -Valery. >> > http://www.harper.no/valery Thanks. Cheers. SignatureWilliam Stacey, MVP > It was described in several different sources (among others I belive you can > find it in Richter's book and Russinovich's book). For online link, check > this: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndllpro/html/m sdn_scalabil.asp ...
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.