[Python-Dev] POSIX thread code

[Tim Peters]

[Gerald S. Williams]
> I recently came up with a fix for thread support in Python
> under Cygwin. Jason Tishler and Norman Vine are looking it
> over, but I'm pretty sure something similar should be used
> for the Cygwin Python port.
>
> This is easily done--simply add a few lines to thread.c
> and create a new thread_cygwin.h (context diff and new file
> both provided).
>
> But there is a larger issue:
>
> The thread interface code in thread_pthread.h uses mutexes
> and condition variables to emulate semaphores, which are
> then used to provide Python "lock" and "sema" services.

Please use current CVS Python for patches.  For example, all the "sema" code
no longer exists (it was undocumented and unused).

> I know this is a common practice since those two thread
> synchronization primitives are defined in "pthread.h". But
> it comes with quite a bit of overhead. (And in the case of
> Cygwin causes race conditions, but that's another matter.)
>
> POSIX does define semaphores, though. (In fact, it's in
> the standard just before Mutexes and Condition Variables.)

Semaphores weren't defined by POSIX at the time this code was written; IIRC,
they were first introduced in the later and then-rarely implemented POSIX
realtime extensions.  How stable are they?  Some quick googling didn't
inspire a lot of confidence, but maybe I was just bumping into early bug
reports.

> According to POSIX, they are found in <semaphore.h> and
> _POSIX_SEMAPHORES should be defined if they work as POSIX
> expects.

This may be a nightmare; for example, I don't see anything in the Single
UNIX Specification about this symbol, and as far as I'm concerned POSIX as a
distinct standard is a DSW (dead standard walking <wink>).  That's one for
the Unixish geeks to address.

> If they are available, it seems like providing direct
> semaphore services would be preferable to emulating them
> using condition variables and mutexes.

They could be hugely better on Linux, but I don't know:  there's anecdotal
evidence that Linux scheduling of threads competing for a mutex can get
itself into a vastly unfair state.  Provided Linux implements semaphores
properly, sempahore contention can be tweaked (and Python should do so), as
befits a realtime gimmick, to guarantee fairness (SCHED_FIFO and SCHED_RR).

> thread_posix.h.diff-c is a context diff that can be used
> to convert thread_pthread.h into a more general POSIX
> version that will use semaphores if available.

I believe your PyThread_acquire_lock() code has two holes:

1. sem_trywait() is not checked for an error return.

2. sem_wait() and sem_trywait() can be interrupted by signal, and
   that's not an error condition.

So these calls should be stuck in a loop:

	do {
		... call the right one ...
	} while (status < 0 && errno == EINTR);

	if (status < 0) {
		/* an unexpected exceptional return */
		...
	}

> ...
> Does this sound like a good idea?

Yes, provided it works <wink>.

> Should I create a more thorough set of patch files and submit them?

I'd like that, but please don't email patches -- they'll just be forgotten.
Upload patches to the Python patch manager instead:

    http://sf.net/tracker/?group_id=5470&atid=305470

Discussion about the patches remains appropriate on Python-Dev.