Is there a way to gain access to managed references list then?
What I need to do is to allow an object to know when all reference to it are
gone.

The core of the problem is Async IO & Garbage collection; if a socket or a
network stream initiated Async io call, until it completes Garbage collector
will ignore it. Which makes it very problematic for objects with lost
references.

I.E. If i loose a reference to an object (or object tree) which is utilizing
IOCP, there is no way for me to get the object closed other then kill the
process.
If I were to have access to references to a particular object (socket in my
case) then I would be able to check them periodically, and if refrence is
lost I can initiate a call to socket shutdown.

I suspect any device bound to an operating system handle will experience the
same issue.