semaphores and Rlocks

[Jive Dadson]

Thanks for the thoughtful reply.  

"Martin v. Löwis" wrote:
> 
> Jive Dadson wrote:
> > Now I am even confuseder than I was.  I found threading.py, which looks like it
> > implements semaphores and condition variables using polling and sleeping.  Yes?  No?
> 
> Yes on condition variables, if they have a timeout. No on semaphores and
> condition variables without timeout.

Right.  Timeouts are the problem.  I will have an application where a process may wait quite a long time, a second or more, but when it does need to wake up, it needs to wake up pretty quickly.  It also needs a timeout of several seconds in case things are fubar.

For now I can just use a threading-module condition variable with no timeout.  In the production release, I'll probably need to rig something else up.  (And, shhh don't tell anyone, but the truth is, I'm kind of paranoid about some particular kinds of things that are NIH.)

> 
> > But I also found thread.c in the C sources, which appears to define semaphores
> > and locks by means of the MS Windows API.
> 
> Where did you find this?

It's in a source distribution labeled 2.2b1. 

> ... thread_nt.h is what is used on Windows, and it
> uses Win32 semaphores only in thread creation. thread.lock is
> implemented using
> InterlockedCompareExchange/InterlockedIncrement/WaitForsingleObject.

Yeah.  Win32 CRITICAL_SECTION is implemented the same way, with an interlocked increment wrapped around a win32 Mutex.  But CRITICAL_SECTION allows a thread to acquire the a lock repeatedly.

By the way, I noticed some comments about not having InterlockedIncrement or something like that available on Win98.  I can furnish an assembly language version in a C "__asm" thingy if anyone is interested.  It would be just about as "portable" as the win32 API.

> 
> > If I had Python locks (like CRITICAL_SECTION in win32 land)
> > and true non-polling semaphores (like Win32 CreateSemaphore, WaitForSingleObject,
> > and ReleaseSemaphore), I could roll my own condition variables portably in Python.
> 
> In threading.py, the relationship is reverse: semaphores are implemented
> using condition variables, not vice versa.


Yeah, that's kind of weird.  If you have condition variables already, why would you want a semaphore?  Whatever.


> They are non-polling, as they
> don't give a timeout to the condition variable.

Right.

> 
> > But I look in the thread module per se and all I find is allocate_lock,
>  > no semaphores.
> 
> Correct.
> 
> > It takes a "LockType" parameter
> 
> Where did you find this?


Gosh, I don't know.  It was in some official looking documentation I found googling around the net.  It might be for a pre-alpha release or something.  Since I wrote that, I've read quite a bit more, and I've written some simple programs using locks.  I'm working on my newbie first class badge now.

> ...
> The thread module is purposefully limited in the primitives it provides
> primarily to be able to provide it on all systems. On top of these
> primitives, more elaborate synchronization objects are implemented
> in threading.py.

I understand.  The only timed blocking call that I can find is time.sleep().  I'll betcha
I could implement condition variables with non-polling timeouts using time.sleep() and auxillary alarm clock threads.  Sure, why not?

> 
> If you want threading objects implemented directly using the Win32 API,
> see http://python.org/sf/500447.

I would prefer a portable solution.

> If you find the time to revise this
> patch so that it transparently get those primitives instead of the
> pure-Python implementations on Windows, please do contribute.

I would be happy to contribute something in this area.  I would need an experienced Python programmer to prepare it for release, though.  I don't know all the conventions, idioms, and gotchas.

> 
> Regards,
> Martin


Merry holidays,
Jive