Bug in Win32file WaitCommEvent ???

[Grant Edwards]

In article <zLyC9.17461$nK4.48560 at news-server.bigpond.net.au>, Mark Hammond wrote:

>> I'm trying to figure out how WaitCommEvent() works in Python, and it looks
>> like there's a problem when it's used in overlapped mode.
>> 
>> My understanding is that (in C) you:
>> 
>>  1) Call WaitCommEvent() and give it three parameters:
>>      * File handle
>>      * a pointer to where the output mask value is to be stored
>>      * a pointer to an overlapped struct
>>     
>>  2) WaitCommEvent() returns immediately.
>> 
>>  3) Call one of the WaitXXX() functions to wait on the event in
>>     the overloapped struct.
>> 
>>  4) When the event is triggered WaitXXX() returns. Check the
>>     value stored via the output mask pointer passed in 1)
>>     above.
>> 
>> [I've gleaned most of this from
>> http://msdn.microsoft.com/library/en-us/devio/base/waitcommevent.asp]
> 
> I don't think this last step is correct.

Perhaps.  If it isn't correct, where does the event mask get returned? [I'd
try to generate a test case in C, but I don't have a Win32 C compiler.]
Remember that in overlapped mode, the call to WaitCommEvent() can return
_before_ the event actually happens, just as a call to ReadFile() or
WriteFile() can return before data has been transferred.

> Certainly the link you posted doesn't say this is true.

Here's the part I was reading:

    Remarks

    The WaitCommEvent function monitors a set of events for a specified
    communications resource. To set and query the current event mask of a
    communications resource, use the SetCommMask and GetCommMask functions.

    If the overlapped operation cannot be completed immediately, the
    function returns FALSE and the GetLastError function returns
    ERROR_IO_PENDING, indicating that the operation is executing in the
    background.

This means to me that WaitCommEvent can return immediately (before the event
has happened).  Therefore, the "mask" variable can not contain valid info
upon return from WaitCommEvent. IOW, the event which "mask" is to identify
hasn't happened yet.

    When this happens, the system sets the hEvent member of the OVERLAPPED
    structure to the not-signaled state before WaitCommEvent returns, and
    then it sets it to the signaled state when one of the specified events
    or an error occurs. The calling process can use one of the wait
    functions to determine the event object's state
  
Since WaitCommEvent returned immediately (none of the masked events has
happened yet) you have to use WaitXXX() if you want to know when one of the
selected events does happen.
  
    and then use the GetOverlappedResult function to determine the results
    of the WaitCommEvent operation.
  
To me the phrase "and then use the GetOverlappedResult function" implies
that this happens _after_ WaitXXX() has returned.  The reasons for calling
GetOverlappedResult are unclear.  The docs for GetOverlappedResult state
explicitly that the "lpNumberOfBytesTransferred" return value is undefined,
so the call appears to be useless in this context.  [Indeed I've never seen
it used following WaitCommEvent()/WaitForXXX()].
  
    GetOverlappedResult reports the success or failure of the operation, and
    the variable pointed to by the lpEvtMask parameter is set to indicate
    the event that occurred.

lpEvtMask was the pointer passed to WaitCommEvent().  This last sentance
says to me that the variable to which lpEvtMask points isn't written to
until _after_ WaitForXXX() has returned.

> Looking further, I can't find *anything* that implies this is true.

I don't see how the above section can be interpreted any differently, and I
don't see how WaitCommEvent() can return a mask value describing an event
that hasn't happened yet.

> KB Q175551 has sample code for Windows CE that uses overlapped IO, and it
> examines the mask immediately after the call.

FWIW, The WinCE serial driver doesn't work the same way as serial devices on
other Win32 platforms.  Notice that this example doesn't pass an overlapped
structure pointer to WaitCommEvent().  One of my companies projects had to
use Embedded NT rather than WinCE due to the differences in I/O on WinCE. At
least that's what the Win32 guys claimed.

> Further, if what you say is correct, I would expect Python COM port 
> based programs to regularly crash!  In your scenario, by the time the 
> overlapped function ended up writing to our mask variable, that address 
> would be invalid.

I don't see any other interpretation.  If WaitCommEvent() can return
immediately, the mask variable obviously isn't valid at that point. Perhaps
in the overlapped case, the output mask value information is discarded, but
that's not what the above text says.

> However, it would be pretty close to the top of the stack, so I would expect
> it to corrupt whatever was running when the mask was actually written.  I
> know a number of people have used these APIs to do serious serial port
> control - either directly, or via another wrapper.  I would be very
> surprised we survived this long.
>
> Can you find an explicit reference to this?

I think the text quoted above is pretty explicit. It's from
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/devio/base/waitcommevent.asp

Overlapped read/write work in similar ways, you pass a pointer to the
source/destination when you initiate the operation, then after doing
WaitForXXX(), you check the contents of the destination or the byte count.

Here's the example program from the Win32All Demo directory:

    88	
    89	    def _ComPortThread(self):
    90	        overlapped = OVERLAPPED()
    91	        overlapped.hEvent = CreateEvent(None, 1, 0, None)
    92	        while 1:
    95	            rc, mask = WaitCommEvent(self.handle, overlapped)
    
If WaitCommEvent() always waits for the event to happen so that mask is
valid, then we know that receive data is ready.  Why not just call
ReadFile() at this point?  Because the WaitCommEvent is overlapped, the
WaitCommEvent returns immediately whether or not data is ready, and we have
to do the WaitForXXX() below:
    
    96	            if rc == 0: # Character already ready!
    97	                SetEvent(overlapped.hEvent)
    98	            rc = WaitForMultipleObjects([overlapped.hEvent, self.eventStop], 0, INFINITE)
    99	            if rc == WAIT_OBJECT_0:
   100	                # Some input - read and print it
   101	                flags, comstat = ClearCommError( self.handle )
   102	                rc, data = ReadFile(self.handle, comstat.cbInQue, overlapped)
   103	                WaitForSingleObject(overlapped.hEvent, INFINITE)
   104	                sys.stdout.write(data)

-- 
Grant Edwards                   grante             Yow!  I represent a
                                  at               sardine!!
                               visi.com