 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
.NET Forum / Languages / C# / March 2008Service terminating silently
Thought I'd throw this out and see if anyone's seen anything similar. I have a Windows service written in C#, which occasionally stops silently. The process goes away, Windows notices and shows it as unstarted in the service control manager. This always happens (as far as we've seen) after another specific kind of error (an external resource being unavailable), but the time between the previous error and the service failing can be hours. The previous error is logged nicely, but there's nothing in the log just before the service dies. I've tried reproducing this by writing deliberately bad code, and seen the following: 1) An exception in a finalizer is logged (not by our code; it's logged as a CLR error, effectively) 2) Throwing an exception is logged by our infrastructure code 3) Running out of memory just throws an exception 4) Calling Environment.Exit reproduces the behaviour - but we don't have that in our code. The only feasible explanation I currently have is that some buggy third-party library code has a call to Environment.Exit in it - but that seems pretty unlikely, and doesn't properly explain the time delay, unless it's occurring in a finalizer which somehow didn't get executed for over 4 hours in one case (gen 2 is feasible, I guess). Anyone seen anything like this? Hints? Oh, by the way I can't reproduce this on a test system... ;(  SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk > Thought I'd throw this out and see if anyone's seen anything similar. > [quoted text clipped - 26 lines] > Anyone seen anything like this? Hints? Oh, by the way I can't reproduce > this on a test system... ;( Stack overflow? I think one of the ways stack overflows manifests themselves is that there is no more room to even pop up a message box, nor would it perhaps be room to call a logging method at that point. We've certainly seen our share of disappearing apps due to stack overflows so that was my first reaction, where no hint was left anywhere of the fact that the application was just moments ago running fine, and now apparently has crashed. Could be a red herring though... SignatureLasse Vågsæther Karlsen mailto:lasse@vkarlsen.no http://presentationmode.blogspot.com/ PGP KeyID: 0xBCDEA2E3 Are any of your 3rd party libraries native or doing anything natively? There's rare situations where a stack overflow cannot be handled and Windows will simply silently terminate the application. I don't think that can happen with the managed stack... SignatureBrowse http://connect.microsoft.com/VisualStudio/feedback/ and vote. http://www.peterRitchie.com/blog/ Microsoft MVP, Visual Developer - Visual C# > Thought I'd throw this out and see if anyone's seen anything similar. > [quoted text clipped - 26 lines] > Anyone seen anything like this? Hints? Oh, by the way I can't reproduce > this on a test system... ;( > Are any of your 3rd party libraries native or doing anything natively? Wouldn't like to say for sure, to be honest - but quite possibly. > There's rare situations where a stack overflow cannot be handled and Windows > will simply silently terminate the application. I don't think that can > happen with the managed stack... Right. Both you and Lasse have pointed at the stack, which is a good point. I'll see what I can provoke along those lines (although probably sticking within managed code for the moment). Another thing I'm going to try is deadlocking the finalizer thread. Given the timing, I reckon there's a finalizer involved *somewhere*... SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk If there is 3rd party unmanaged code it may be calling Win32 SetErrorMode > Thought I'd throw this out and see if anyone's seen anything similar. > [quoted text clipped - 26 lines] > Anyone seen anything like this? Hints? Oh, by the way I can't reproduce > this on a test system... ;( > Thought I'd throw this out and see if anyone's seen anything similar. > [quoted text clipped - 26 lines] > Anyone seen anything like this? Hints? Oh, by the way I can't reproduce > this on a test system... ;( What do you mean exactly with "unstarted"?, AFAIK there is no such Service state. Calling Environment.Exit(..), from within a service should set the Service state to stopped with a Win32 exit code = 1067. Also (on XP and higher) you should see an Error message in the System log with Event ID=7034. Willy. > What do you mean exactly with "unstarted"?, AFAIK there is no such Service > state. Sorry, I was being imprecise. No status is shown, and it can be started. No doubt this is the Stopped state. > Calling Environment.Exit(..), from within a service should set the Service > state to stopped with a Win32 exit code = 1067. > Also (on XP and higher) you should see an Error message in the System log > with Event ID=7034. Not in this case - and not in my test service either. SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk >> What do you mean exactly with "unstarted"?, AFAIK there is no such >> Service [quoted text clipped - 10 lines] > > Not in this case - and not in my test service either. Weird, what OS are you running? The normal behavior is as such that when a Service Process dies, that the SCM logs this as an Error event in the System Eventlog. When this happens, the SCM also sets the process exit code to 1067 (ERROR_PROCESS_ABORTED) which means: The process terminated unexpectedly. Every Windows Service process is connected with the SCM via a named pipe IPC channel, the SCM considers the Service as being dead, whenever the NP end-point gets closed at the service side, this without being ordered by the SCM. Willy. > Weird, what OS are you running? XP SP2 on my dev box. Not sure about the servers. > The normal behavior is as such that when a Service Process dies, that the > SCM logs this as an Error event in the System Eventlog. Hmm. I'll double check tomorrow, but I'm pretty sure that I wasn't seeing anything. Of course, it's perfectly possible I was looking in the wrong event log, in which case I apologise for wasting everyone's time! (Whichever log I was looking in did get various bits of information, but not this one.) SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk <snip> > Weird, what OS are you running? > The normal behavior is as such that when a Service Process dies, that the [quoted text clipped - 5 lines] > end-point gets closed at the service side, this without being ordered by the > SCM. Just checked, and on my dev box if the service uses Environment.Exit we do indeed get an event in the System event log. Unfortunately on the server where we're seeing the odd behaviour, there's nothing in the event log at all around the time of death :( SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk > <snip> > [quoted text clipped - 14 lines] > server where we're seeing the odd behaviour, there's nothing in the > event log at all around the time of death :( That means that Environment.Exit was most certainly not called. I just checked this on XP SP2, W2K3 SP2, Vista and WS2008, and I can say that the behavior of E.E is consistent across these OS versions. It's pretty weird in your case (on the server), normally the SCM logs this message whenever he finds a Service process going away without being *notified*, for whatever reason, even killing the service process cannot prevent this. What puzzles me, is that, every Stopped<-> Running transition is also automatically logged as 'Informational' in the Application event, however, you don't see such messages, what makes me believe that the service did not stop itself ..... Willy. On Mar 26, 9:52 am, "Willy Denoyette [MVP]" <willy.denoye...@telenet.be> wrote: > > Just checked, and on my dev box if the service uses Environment.Exit we > > do indeed get an event in the System event log. Unfortunately on the [quoted text clipped - 4 lines] > checked this on XP SP2, W2K3 SP2, Vista and WS2008, and I can say that the > behavior of E.E is consistent across these OS versions. Fair enough - it seemed unlikely that a library would include a call to Environment.Exit anyway. > It's pretty weird in your case (on the server), normally the SCM logs this > message whenever he finds a Service process going away without being [quoted text clipped - 3 lines] > however, you don't see such messages, what makes me believe that the service > did not stop itself ..... Exactly. It's completely bizarre. I'd expect an orderly shutdown to log an Application event log message, and a suicide to log in the System event log. To not get anything is just weird... and very hard to diagnose, given that we can't reproduce it :( Jon > On Mar 26, 9:52 am, "Willy Denoyette [MVP]" > <willy.denoye...@telenet.be> wrote: [quoted text clipped - 26 lines] > System event log. To not get anything is just weird... and very hard > to diagnose, given that we can't reproduce it :( Hard but not impossible, some techniques are more elegant than the other, all depends on the version of the OS. What OS version is there sitting on this server? Willy. <snip> > > Exactly. It's completely bizarre. I'd expect an orderly shutdown to > > log an Application event log message, and a suicide to log in the [quoted text clipped - 4 lines] > all depends on the version of the OS. > What OS version is there sitting on this server? 2K3. Oh, and I don't have easy access to it... SignatureJon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet World class .NET training in the UK: http://iterativetraining.co.uk
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
Reply to this Message
|
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.