CLR "fatal execution engine error" messages in event log

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Lucvdv - 16 Mar 2006 10:19 GMT

Do these messages in the event log look familiar to anyone?
Could they be caused by calling Thread.Abort() on a System.Threading.Thread
without giving it time to clean up?

They are generated when a service written in .Net 2005 is stopped.

Nothing abnormal seems to happen as long as the service is running, but
stopping it causes an error for which no exception is raised in the
application itself.

- The service seems to stop immediately when it's asked to.
"Immediately" means the services control panel indicates the service is no
longer running within 1 second after clicking "Stop", or if it's done
trough "net stop {servicename}" at a command prompt, the prompt returns
almost immediately.

- Despite that, the .exe file remains active with 100% CPU use (seen in
task manager) for about half a minute after that.

- The following two messages appear in the [Application] event log:

Source: .NET Runtime
Category: None
Event ID: 1023
Description:

| .NET Runtime version 2.0.50727.42 - Fatal Execution Engine Error
| (7A05E2B3) (80131506)

Source: .Net Runtime 2.0 Error
Category: None
Event ID: 1000
Description:

| Faulting application a2ssvc.exe, version 3.0.2259.15264, stamp 440fe9e5,
| faulting module mscorwks.dll, version 2.0.50727.42, stamp 4333e7ec,
| debug? 0, fault address 0x00007a7f.

Some directional info on what the service uses/does:

It creates two worker threads, which are killed by means of Thread.Abort()
when the service is asked to stop. I've heard that calling Abort() can
have some nasty consequences for the rest of the application (something
about messing up the application domain), but I didn't worry too much about
that because it's exiting anyhow.

Each thread should be seen as a separate application on its own: the only
thing they access in common is a database (MSDE or SQL Express); each
thread uses its own connection objects for that, nothing cross-thread.
There is no communication between the threads at all, they work fully
independent of one another.

One of the threads creates and uses net sockets (Net.Sockets.TcpClient),
the other uses some COM objects and communicates with a UI app through
named pipes.

Reply to this Message

Willy Denoyette [MVP] - 17 Mar 2006 21:41 GMT

If the process is still running, it means that your service did not stop
(yet), issuing the "net stop" command is asynchronous, it always returns
immediately, whether your service has stopped or not.
The fact that you are seeing 100% CPU consumption probably means that the
finalizer thread is active running the finalizers, you may have some
problems cleaning up unmanaged resources if it takes that long (guess your
COM objects, are you sure they run in the correct apartment?). If the it
takes longer than 30 seconds to terminate the process, the CLR will issue a
rude thread abort.
And yes, calling Abort on an other thread than the current thread is
something you should never do, signal the other threads that they should
exit, for instance by using a ManualReset Event.
Is there any particular reason why you would stop a service other than
shutting down the system?
Willy.

| Do these messages in the event log look familiar to anyone?
| Could they be caused by calling Thread.Abort() on a System.Threading.Thread
[quoted text clipped - 49 lines]
| the other uses some COM objects and communicates with a UI app through
| named pipes.

Reply to this Message

Lucvdv - 18 Mar 2006 11:52 GMT

> If the process is still running, it means that your service did not stop
> (yet), issuing the "net stop" command is asynchronous, it always returns
> immediately, whether your service has stopped or not.

I thought it was synchrounous up to a certain time-out, but I don't have a
non-functional service ready to test it ;)

I copied this from a console window on a test system:

A short delay (between 1 and 2 seconds) expired between the "stopping" and
"stopped" messages. The service executable remained active in task manager
for a considerable time after that.

If it hadn't told the SCM it stopped, the services control panel applet
would list the service as "Stopping". It doesn't, and it's even possible
to start the service again while the previous instance is still shown in
task manager (so two instances of the process are running at the same
time).

I think that means the CLR is running finalizers and garbage collection
_after_ the service control function has returned (to the SCM) from
processing the SERVICE_CONTROL_STOP message.

> The fact that you are seeing 100% CPU consumption probably means that the
> finalizer thread is active running the finalizers, you may have some

Not always 100% actually, and the 'fatal execution engine error' doesn't
occur every time either (but >5 times out of 10, and more likely if the
service has been running for a longer time).

> problems cleaning up unmanaged resources if it takes that long (guess your
> COM objects, are you sure they run in the correct apartment?).

As far as I can tell - there's a single STA thread that creates them and
does all processing related to them.

There are a few cases (certain actions started by UI commands) where that
thread will spawn a new working thread that accesses them through Invoke,
but whether that has been done or not seems to make no difference.

> If the it
> takes longer than 30 seconds to terminate the process, the CLR will issue a
> rude thread abort.

Could that be reflected by a 'fatal execution engine error' in the event
log?

> And yes, calling Abort on an other thread than the current thread is
> something you should never do, signal the other threads that they should
> exit, for instance by using a ManualReset Event.
> Is there any particular reason why you would stop a service other than
> shutting down the system?

For installing an update for example, if you want to avoid a reboot.

Reply to this Message

Willy Denoyette [MVP] - 18 Mar 2006 16:53 GMT

| > If the process is still running, it means that your service did not stop
| > (yet), issuing the "net stop" command is asynchronous, it always returns
| > immediately, whether your service has stopped or not.

I mean your process has stopped, sorry. When the SCM receives the STOPPED
status it considers the service as stopped, even if the process is still
running.

| I thought it was synchrounous up to a certain time-out, but I don't have a
| non-functional service ready to test it ;)
[quoted text clipped - 20 lines]
| _after_ the service control function has returned (to the SCM) from
| processing the SERVICE_CONTROL_STOP message.

| > The fact that you are seeing 100% CPU consumption probably means that the
| > finalizer thread is active running the finalizers, you may have some
[quoted text clipped - 8 lines]
| As far as I can tell - there's a single STA thread that creates them and
| does all processing related to them.

Ok, but initializing a thread to run in an STA is not enough, that thread
needs to pump the message queue. Remember you run in a non UI thread that
means that COM has created an invisible window and an associated window
queue, you need to pump this queue, else the finalizer (running in an MTA)
will not be able to call the underlying RCW's finalizers, resulting in a
blocked finalizer thread. The CLR will release this thread at AD unload time
and will try to run the other pending finalizers, but chances are that there
are so many that there is no time left to run them all before the CLR
terminiates the threads (rude abort).

| There are a few cases (certain actions started by UI commands) where that
| thread will spawn a new working thread that accesses them through Invoke,
| but whether that has been done or not seems to make no difference.

Possibly, if the thread was running unmanaged code all bad things can
happen.

| > And yes, calling Abort on an other thread than the current thread is
| > something you should never do, signal the other threads that they should
[quoted text clipped - 3 lines]
|
| For installing an update for example, if you want to avoid a reboot.

I see.

Willy.

Reply to this Message

Marcus - 27 Mar 2006 18:12 GMT

I have a similair problem. The event-log of the web-server is full of these
errors:

Faulting application w3wp.exe, version 6.0.3790.1830, stamp 42435be1,
faulting module mscorwks.dll, version 2.0.50727.42, stamp 4333e7ec, debug? 0,
fault address 0x000e9f96.

.NET Runtime version 2.0.50727.42 - Fatal Execution Engine Error (7A05E2B3)
(80131506)

We installed .NET2.0 on the server (running 1.1 apps) and installed a 2.0
app-project (converted from 1.1)

Any suggestions?

/Marcus

> Do these messages in the event log look familiar to anyone?
> Could they be caused by calling Thread.Abort() on a System.Threading.Thread
[quoted text clipped - 49 lines]
> the other uses some COM objects and communicates with a UI app through
> named pipes.

Reply to this Message

amir.alfoly@gmail.com - 03 Apr 2006 10:15 GMT

We Have a similar problem in our system
it is intensive in work and use multithreads

*** I think it's mainly a multithreading problem, when work is
intensive ***

i will suggest some precautions to take:
1- check if you have any Threads that may have unhandled exceptions
2- check if you have System.Windows.Forms.Timer Object and if you are
running MultiThreading, it's not safe to use this but you can use
safely System.Timers.Timer Object instead
3- check that there is no Cross Threading Operation, Which mean
accessing Controls from another thread than Thread Which created them
4- check somehow that all the Event Messages are pumped periodically
and there is no

--------------------------------------------------------------------------
Amir AlFoly
Software Developer

> I have a similair problem. The event-log of the web-server is full of these
> errors:
[quoted text clipped - 66 lines]
> > the other uses some COM objects and communicates with a UI app through
> > named pipes.

Reply to this Message

CLR "fatal execution engine error" messages in event log

Free Magazines