Threaded GUI slowing method execution?

[Dave Angel]

Aaron Hoover wrote:
> <div class="moz-text-flowed" style="font-family: -moz-fixed">I have a 
> wx GUI application that connects to a serial port in a separate 
> thread, reads from the port, and then is supposed to put the data it 
> finds into a queue to be used by the main GUI thread. Generally 
> speaking, it's working as expected.
>
> However, one method (that's part of a library I've written to parse 
> the packet structure of the data that's coming over the serial port) 
> executes approximately 1000 times slower (50ms vs. 50us) when called 
> from the serial management thread in the GUI as compared to calling 
> the same function from within a command line Python script. I checked 
> it by wrapping the call as follows in both cases  (GUI and command 
> line script):
>
> tic = time.time()
> <method call>
> print time.time() - tic
>
> All the thread is doing most of the time is sitting around checking 
> the serial port for waiting data, reading it, and appending it to a 
> list when it finds it. Then, in the same thread, the method that seems 
> to be remarkably slow works its way through that list looking for 
> packets of data and appending the packet payloads it finds to a queue 
> to be handled in some way by the GUI.
>
> My question is, what am I missing about either threading or the fact 
> that this is running in a GUI that might explain such a huge slowdown. 
> I'm sending data over the serial at a true rate of about 24k bytes per 
> second, or approximately 2 packets per ms. Is it too much to ask to be 
> able to process this data in realtime from within a GUI (I'm not 
> talking about plotting or anything - just read it and find packets)? 
> Like I said, the process pretty much runs in realtime from a command 
> line script.
>
> This packet parsing needs to happen continuously, so it seems calling 
> join() to ensure it's not interrupted by the GUI thread, won't work.
>
> Thanks in advance for your help.
>
> Aaron
>
> </div>
>
Threading in any language is tricky, so it may be just that.  But in 
Python in particular, you have the GIL (Global Interpreter Lock, I 
believe), which usually makes writing threaded code easier, but can 
really hurt efficiency.   If you do a search of this list for GIL, 
you'll see lots of discussion.  Or you can google for "GIL python".  
Look also at http://www.dabeaz.com/python/GIL.pdf

I'm no expert here, but let me try to summarize what I recall.  The GIL 
makes writing background threads easier when all but one of the threads 
are frequently blocked for system calls.  But as soon as you have two 
threads doing "busy work," instead of them getting 50% each, the 
threading overhead goes way up, and you spend much of your time just 
deciding what not to do next.  And it gets enormously worse than that 
when there are more than one CPU available (eg. multicore).

Many people have concluded that (in Python) much of what threads are 
used for should be done with processes.

DaveA