Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / Languages / C# / September 2007

Tip: Looking for answers? Try searching our database.

Pinvoke question: Am I causing this stack overflow?

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Tom Allen - 31 Aug 2007 18:52 GMT
Hi,  TIA for any help.
(Thanks, Mattias Sjögren for your help yesterday! )

I am developing a C# wrapper for a 3rd party API.  It processes buffers from
a video stream (file) and is structured where the initialization function
passes in two callback methods (my code) that the API calls to successively
read and write buffers.  The C++ sample I am modeling this after works and
processes thousands of buffers, but when interface to my C# wrapper the API
gets a stack overflow every time the 18th buffer is read and the SUCCCESS
status is returned from the callback.

Q:  Is there somethng in the way I am calling/being called by pinvoke that
is causing this stack overflow?  I am attaching two posts
 1) Facade class that calls the native methods (TpFacade.cs)
 2) Driver class that calls the facade class. (ProcessStream.cs)

The stack overflow occurs when the GetStreamBuf method returns SUCCESS for
the 18th time.

Thanks for any suggestions!

+tom allen
Cupertino, CA.
Tom Allen - 31 Aug 2007 18:54 GMT
//File:  TpFacade.cs
using System;
using System.Collections.Generic;
using System.Text;
using System.Runtime.InteropServices; // DllImport

namespace TableProcessorFacade
{
   public class TpFacade
   {
       public const int PACKET_SIZE = 188;
       public const int SDT_SECTION_SIZE = 4096 * 100;  

       // TYPE OF THE TRANSPORT STREAM, Actual -> Actual TS, Other -> Other
TS (DVB SI ONLY)
       public enum TS_TYPE { ActualTS, OtherTS } ;

       // TYPE OF THE EIT TABLE (DVB SI ONLY)
       public enum EIT_TYPE { EventSchedule, PresentFollowing } ;

       //Interop Overview
       //http://msdn2.microsoft.com/en-us/library/eaw10et3(VS.80).aspx
       //
       //Marshaling Data with Platform Invoke
       //http://msdn2.microsoft.com/en-us/library/fzhhdwae(VS.80).aspx
       //
       //Article: Using P/Invoke to Call Unmanaged APIs from Your Managed
Classes
       //http://msdn2.microsoft.com/en-us/library/aa719104(vs.71).aspx
       //
       //Default Marshaling for Delegates  (Ensure that delegates are not
garbage collected after the InitProcessor call returns.)
       //http://msdn2.microsoft.com/en-us/library/7esfatk4(VS.80).aspx
       //
       //Managed and Unmanaged Threading  
       //http://msdn2.microsoft.com/en-us/library/5s8ee185(VS.80).aspx
       //
       //Passing structures
       //http://msdn2.microsoft.com/en-us/library/awbckfbz(VS.80).aspx
       //
       /////////////////////////
       //HOWTO: DLL method names & ordinal location
       //To find ordinal:
       //     dumpbin /exports tableprocessor.dll
       //ordinal for InitProcessor is 33:
       //     33   20 000193D0 ?InitProcessor@@YAHP6AHPAEH@Z1HHPAD@Z
       //
       /////////////////////////
       //Newsgoup
       
//http://msdn.microsoft.com/newsgroups/default.aspx?&guid=&sloc=en-us&dg=microsoft
.public.dotnet.languages.csharp&p=1&tid=c2c8e936-c9aa-41dd-9ea0-a7aa6da613ef&mid
=dc6d7478-bce3-45ad-8ec0-b77f95141f93

       //
       ////////////////////////
       //Types
       //  Note: UCHAR (in Wtypes.h) maps to SByte (signed byte) which
represents an 8-bit signed integer.
       //  The SByte type is not CLS-compliant. The CLS-compliant
alternative type is Int16.
       //  Warning: In some other contexts the name "UCHAR" refers to
"Unicode character", which is a 16-bit structure.
       //
       //  Here substituting Byte array for UCHAR array.
       //
       // UCHAR  maps to Byte (for our purposes here)
       // BOOL   maps to Booean (or Int32)
       // int    maps to Int32
       // char*  maps to String (decorate with Ansi, not Unicode)

       //TableProcessor API
       //File tableprocessor.h
       // Defines the TableProcessor API functions exported from
tableprocessor.dll.
       //  #ifdef TPROCESSOR_EXPORTS
       //  #define TPROCESSOR_API __declspec(dllexport)
       //  #else
       //  #define TPROCESSOR_API __declspec(dllimport)
       //  #endif

       #region //Managed function prototypes

       // C++ signature
       // InitProcessor - Intialize and start the Table Processor.
       // Accepts two user defined callback functions to read and write TS
packets.
       // Returns 0 on success and 1 on failure. NOTE: To exit the
tableprocessor module
       // return 1 (failure) on any of the callback functions else return 0
       // \param GetStreamBuf - function pointer to read the TS packet
       // \param PutStreamBuf - function pointer to write the TS packet
       // \param enableLogging - Set to TRUE if logging is to be enabled in
the API else FALSE
       // \param logLevel - Level of the logging information to be
displayed, defaults to 0 (0 to 10)
       // \param logFile - file to save the logging information to */
       ////int TPROCESSOR_API InitProcessor(int (*GetStreamBuf)(UCHAR *Buf,
int buf_len),
       ////                                 int (*PutStreamBuf)(UCHAR *Buf,
int buf_len),
       ////                                 //int streamType,
       ////                                 BOOL enableLogging=FALSE,
       ////                                 //BOOL displayLog,
       ////                                 int logLevel=0,
       ////                                 char* logFile=NULL );

       public delegate Int32 GetStreamBufCallback(
           [Out][MarshalAs(UnmanagedType.LPArray,SizeParamIndex = 1)]
Byte[] Buf, Int32 buf_len);
       public delegate Int32 PutStreamBufCallback(
           [In][MarshalAs(UnmanagedType.LPArray,SizeParamIndex = 1)] Byte[]
Buf, Int32 buf_len);

       [DllImport("tableprocessor.dll", CharSet = CharSet.Ansi, EntryPoint
= "#33", CallingConvention=CallingConvention.StdCall)]
       private static extern int InitProcessor(
           [MarshalAs(UnmanagedType.FunctionPtr)] GetStreamBufCallback GsCb,
           [MarshalAs(UnmanagedType.FunctionPtr)] PutStreamBufCallback PsCb,
           Boolean enableLogging,
           Int32 logLevel,
           String logFile
           );

       // C++ signature
       // GetActualPresentEit - Returns the present/following EIT sections
for actual Transport stream - DVB.
       //  present/followig EIT Sections.
       // Returns 0 on success and 1 on failure.
       // param buf - char buffer containing the EIT sections.
       //int TPROCESSOR_API GetActualPresentEit(UCHAR *buf) ;

       [DllImport("tableprocessor.dll", CharSet = CharSet.Ansi, EntryPoint
= "#33", CallingConvention=CallingConvention.StdCall)]
       private static extern int GetActualPresentEit(
           [Out] [MarshalAs(UnmanagedType.LPArray, SizeConst =
SDT_SECTION_SIZE)] Byte[] Buf);

       #endregion //Managed function prototypes

       #region //Facade functions

       public static bool InitProcessorFacade(
           GetStreamBufCallback GsCb,
           PutStreamBufCallback PsCb,
           Boolean enableLogging,
           Int32 logLevel,
           String logFile
           )
       {
           //Return "0 -> SUCCESS"
           return (0 == InitProcessor(
            GsCb,
            PsCb,
            enableLogging,
            logLevel,
            logFile
           ));
       }//InitProcessorFacade

       //TODO   [In, Out] vs ref
       public static bool GetActualPresentEitFacade([In, Out] Byte[] Buf)
       {
           //Return "0 -> SUCCESS")
           return (0 == GetActualPresentEit(Buf));
       }//GetActualPresentEitFacade

       #endregion //Facade Functions

   }//Class TpFacade

}//namespace TableProcessorFacade

-- Tom Allen
Tom Allen - 31 Aug 2007 18:56 GMT
//File:  ProcessStream.cs
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;       //AutoEvent
using System.Timers;        //ModTimer
using System.IO;            //FileStream

using TableProcessorFacade; //Facade for TableProcessor API

namespace MicrosoftCorp_MSTV_Metadata
{
   public class ProcessStream
   {
       //Processing flags
       Boolean done;
       Boolean pModsAllCompleted, fModsAllCompleted;

       const int SUCCESS = 0;              //return callback status to API
       const int FAILURE = 1;              //return callback status to API
       static TpFacade.TS_TYPE TsType = TpFacade.TS_TYPE.ActualTS;
       static TpFacade.EIT_TYPE EitType = TpFacade.EIT_TYPE.PresentFollowing;

       System.IO.FileStream fsInput;
       System.IO.FileStream fsOutput;

       static TpFacade.GetStreamBufCallback GetStreamBufCb;
       static TpFacade.PutStreamBufCallback PutStreamBufCb;

       //Progress indicators
       short ReadProgressRollover65535 = 0;
       short WriteProgressRollover65535 = 0;
       char CheckEitTablesRollover256 = (char)0;

       //Threading objects
       //lock for thread sync with API processing
       
//ms-help://MS.VSCC.v80/MS.MSDN.v80/MS.NETDEVFX.v20.en/cpref12/html/T_System_Threading_AutoResetEvent.htm
       static AutoResetEvent m_autoEventProcessingDone;
       static System.Timers.Timer ModTimer;

       public void Process(RunParmsDataset.RunFileSetRow fSet,
ProposedEventPair evtPair, StreamEventsDataset.EventServiceRow EventTriplet,
bool WriteStreamEventsLog, bool WritePacketsBeforeModifiedEitEvent,
System.Int64 SkipBytesBeforeEitCheck)
       {
           //processing flags
           done = false;
           pModsAllCompleted = false;
           fModsAllCompleted = false;

           //Disable API logging
           string apiLogFileString = String.Empty;
           bool m_EnableLogging = false;
           int m_LogLevel = 0;

           //Open files
           //TODO Validate input file path better.
           string InputPath = Path.GetFullPath(fSet.InputFilePath);
           string OutputPath = Path.GetFullPath(fSet.OutputFilePath);
           fsInput = File.OpenRead(InputPath);
           fsOutput = File.OpenWrite(OutputPath);

           //Maintain reference to callback delegates to avoid garbage
collection following return from InitProcessorFacade.
           GetStreamBufCb = GetStreamBuf;
           PutStreamBufCb = PutStreamBuf;

           //TODO Exception handling, reporting, logging.

           //prep, then call API InitProcessor
           //The "done" clause of GetBuffer releases this lock.
           m_autoEventProcessingDone = new AutoResetEvent(false);

           //TODO: Ensure that delegates are not garbage collected after
the InitProcessor call returns.
           //  "If, however, the callback function can be invoked after the
call returns,
           //  the managed caller must take steps to ensure that the
delegate remains uncollected
           //  until the callback function finishes. For detailed
information about preventing garbage collection,
           //  see Interop Marshaling with Platform Invoke. "
           //HandleRef Sample
           //http://msdn2.microsoft.com/en-us/library/hc662t8k(VS.80).aspx

           BeginProcessing(apiLogFileString, m_EnableLogging, m_LogLevel);

           m_autoEventProcessingDone.WaitOne();

           //Processing completed for current file set
       }//Process

       private void BeginProcessing(string apiLogFileString, bool
m_EnableLogging, int m_LogLevel)
       {
           // initialize the API
           if (TpFacade.InitProcessorFacade(GetStreamBufCb, PutStreamBufCb,
m_EnableLogging, m_LogLevel, apiLogFileString))
           {
               return;
           }
       }//BeginProcessing

       #region TableProcessor API callbacks

       // Callback function used by the API to read packet from the
application
       public Int32 GetStreamBuf(Byte[] buf, Int32 bufSize)
       {
           if (fsInput != null)
           {
               try
               {
                   if (fsInput.Read(buf, 0, bufSize) == TpFacade.PACKET_SIZE)
                   {
                       Console.Write("->{0}", ReadProgressRollover65535);
                       Console.Out.Flush();

                       //stability - wait for API threads
                       System.Threading.Thread.Sleep(200);

                       //progress indicator
                       if (ReadProgressRollover65535 == 0)
                       {
                           Console.Write("XX->");
                           Console.Out.Flush();
                       }
                       ReadProgressRollover65535++;
                       return SUCCESS;                 //return callback
status to API
                   }
                   else
                   {
                       Console.Write("Done reading input (failed read).");
                       Console.Out.Flush();
                       done = true;
                       return FAILURE;                 //return callback
status to API
                   }
               }
               catch (Exception ex)
               {
                   Console.WriteLine(ex.Message);
                   done = true;
                   throw;
               }
           }
           else
           {
               done = true;
               Console.Write("Done reading input (eof).");
               Console.Out.Flush();
               return FAILURE;
           }
       }//GetStreamBuf

       // Callback function used by the API to write packet to the
application
       public Int32 PutStreamBuf(Byte[] buf, Int32 bufSize)
       {
           try
           {
               if (!(buf == null))
               {
                   Console.Write("<-");
                   Console.Out.Flush();
                   fsOutput.Write(buf, 0, bufSize);

                   //TODO  Stack overflow occurs regardless of this call.
                   //ModifyEit();
                   return SUCCESS; //Success == 0
               }
               m_autoEventProcessingDone.Set();
               return FAILURE; //stop API thread?
           }
           catch (Exception ex)
           {
               Console.WriteLine(ex.Message);
               //Processing completed.  Release sync lock.
               m_autoEventProcessingDone.Set();
               throw;
           }
       }

       private void ModifyEit()
       {
           if (!done)
           {
               //TODO Why allocate this here?
               byte[] buf = new byte[TpFacade.SDT_SECTION_SIZE]; //4096*100
               //Accumulate EIT event tables
               //Get the ActualPresentEit table
               if (TpFacade.GetActualPresentEitFacade(buf))
               {
                   Console.Write("\nFound GetActualPresentEit\n");
                   Console.Out.Flush();
               }
           }
       }//PutStreamBuf
       #endregion //TableProcessor API callbacks

   }//class ProcessStream
}

--Tom Allen
Ben Voigt [C++ MVP] - 04 Sep 2007 23:35 GMT
> Hi,  TIA for any help.
> (Thanks, Mattias Sjögren for your help yesterday! )
[quoted text clipped - 19 lines]
>
> Thanks for any suggestions!

You didn't specify a calling convention for the delegate types as far as I
can see, so you might be trashing your caller's stack (actually trashing his
EBP, ESP registers).

> +tom allen
> Cupertino, CA.
Tom Allen - 05 Sep 2007 05:42 GMT
Ben,

Thanks much for the reply, and for taking the time to scan the code.  Your
suggestion makes sense.

Sadly, today was my last day on this contract.  I have forwarded the
information to my boss and I hope she will be able to test it out and post a
reply to this thread.  Thanks again.

+tom allen
Cupertino, CA.

> > Hi,  TIA for any help.
> > (Thanks, Mattias Sjögren for your help yesterday! )
[quoted text clipped - 26 lines]
> > +tom allen
> > Cupertino, CA.

Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.