Embedded Threading Question

[Tim Dietz]

hansgeunsmeyer at earthlink.net (hg) wrote in message news:<95959723.0212041851.494f3479 at posting.google.com>...
> tpdietz at hotmail.com (Tim Dietz) wrote in message news:<32355ee1.0212040738.3e7360ea at posting.google.com>...
> 
> > > How are you calling these C functions? Do the C functions call back into Python
> > > (or do they have access to variables in your embedded Python app)?
> > 
> > Using SWIG, we create a Python module and a C++ wrapper class.  The Python
> > module then has a class Foo, which we use to call methods that interface
> > into the C++ Foo class.  So, in Python, I call Foo.SetValue(val) and the
> > SetValue() method of the C++ Foo class is called.
> > 
> > Does that help to explain things better?
> 
> A little bit, it's still rather mysterious to me :) If the
> non-threaded version works fine, and the threaded version doesn't, and
> if the SetValue call really is the culprit, then it may be that you
> have to protect the access to that value you are setting (I suppose
> it's a class member, or else some global variable). I don't think SWIG
> does this automatically.  So, in this case, you need to modify the
> SWIG generated code, by using a lock around the access of the
> variable.
> 
> 
> Hans

OK, I'm getting closer.  I've trimmed down my Python code so that it
has a thread to monitor the socket (ReadThread) as well as the main
thread.

So, to simplify, there is a function (called SendMsg) that sends text
to a TCP/IP server using socket.send().  The server sends a response
back to the client.  The response is picked up in ReadThread and
processed from there.  However, my problem occurs whether or not I
process the data read from the socket, so I've eliminated that code. 
In addition, the server can send asynchronous updates to this client,
which are also processed in the ReadThread.

So, below is the streamlined version of my Python module (client.py):

def SendMsg(outmsg):
   mysock.send(outmsg)

def ReadThread():
   done = 0
   buf = ''
   while not done:
      rcv,wr,err = select.select([mysock],[],[],0.1)
      if mysock in rcv:
         buf += rcv[0].recv(1000)

If I run Python from a DOS prompt, import this file and connect to the
server, the server can send me as many asynchronous updates as it
wants and the ReadThread handles them without a problem.  Fast and
efficient.

When I run my C app, it starts Python, performs a few imports and
execfiles (one of which starts the ReadThread) and displays a GUI
which has a memo in which Python commands can be entered and sent
using PyRun_String().  From this app, the user can initiate an action
that will call the SendMsg function (via the PyRun_SimpleString
function) above.  The server handles the message, sends a response and
the ReadThread gets the data and all is fine.  I can perform this
action over and over and over again, with no problems.

However, when the server sends an asynchronous update, the ReadThread
does not seem to notice.  As far as I can tell, the select function
doesn't even return.  In fact, I've verified that when this thread is
executing in DOS Python, the select function returns every .1 seconds,
as it should, given that value as the timeout parameter.  But, when
running the C app, I've determined that the select function doesn't
even return unless woken up as described in the next paragraph.

I've found two ways to wake up the thread in this situation.  If I
enter ANY command in the memo on the GUI, the select fuction returns
the update.  Or, if I issue another SendMsg from the GUI, the update
is processed, then the new message is processed.

It's as if the ReadThread gets into some kind of limbo and doesn't
really wake up unless some action is performed in the main thread.

So, any ideas as to why the ReadThread acts so strangely in the C
application?


Thanks again for the help.

Tim