[Chicago] Watch out for Queue on cygwin

Phil Robare verisimilidude at gmail.com
Tue Oct 7 23:38:24 CEST 2008


I created a simple threaded app - control thread, network thread,
output thread and communication between the threads was done using the
synchronized queue from the Queue module.  After I got it working it
would run for a while and then crash with:

sem_init: Resource temporarily unavailable
Error while running: (<class 'thread.error'>, error("can't allocate
lock",), <traceback object at 0x7ff41eb4>)
  File "SoupReader.py", line 252, in run
    line=self.in_queue.get(timeout=1.0)
  File "/tmp/python.6884/usr/lib/python2.5/Queue.py", line 174, in get
    self.not_empty.wait(remaining)
  File "/tmp/python.6884/usr/lib/python2.5/threading.py", line 208, in wait
    waiter = _allocate_lock()
Shutting down on error...

After spending time with the Queue.py and threading.py source (and
quickly figuring out my problem is in the C code) I googled sem_init,
and found it is the low-level Unix API for semaphore creation.  Since
I am forced by work situation to use Windows I was actually running
Python under cygwin.  Googling sem_init and cygwin found this post
from 2002 on the Python-Dev mailing list:
http://mail.python.org/pipermail/python-dev/2002-February/020492.html

The discussion there is that cygwin's sem_init, as called in the
Python libraries, in inherently subject to a race condition.  And that
fits the symptoms I was seeing.  Running the real windows python I
have no problems (other than those inherent in using Idle rather than
my preferred Vim + command line).

As I am unsure what to do with this hard won knowledge I have decided
to post it here, spamming it out to the teeming millions on this list
that don't care in the hope that I can save someone the pain I have
gone through.

Phil


More information about the Chicago mailing list