threading.py -> winthreading.c : Some results

[Tim Peters]

[posted & mailed]

[Jason Orendorff]
> I've reimplemented the synchronization objects of threading.py
> in non-portable, Win32-only C.

Sorry I haven't had time to follow up on this -- it's interesting and I want
to encourage you (so I don't have to do it later <wink>).

> The C versions of RLock, Event, Semaphore, and Condition perform
> at least 5x faster than their standard counterparts in threading.py
> in every situation I've tried.  Acquiring an RLock runs 25x faster.
> Condition.wait(timeout) runs 500x faster.
>
> This benchmark:
>   http://www.bagley.org/~doug/shootout/bench/prodcons/prodcons.python
> runs 6.3 times as fast under the C version of Condition.
> That's fast enough to post a reasonably non-humiliating benchmark
> number; it closes the gap between Python and Java considerably.
> ...If you care about such things.  ...As of course no sane person
> does.
>
> Whether this C code would mean a significant performance increase
> in a real-life program remains to be proven.  Comments?

It depends on the program, of course.

You could also consider solving a simpler problem, which would definitely
increase performance of real-life programs:  "Python locks" are also used
internally by Python, most notoriously for the global interpreter lock.  A
Python lock has the unusual property that it's neither "owned" nor
reentrant:  any thread can release a Python lock, whether or not it was the
thread that acquired it.  And it's legit for any thread to try to acquire a
Python lock, even if the thread has already acquired it (and in that case,
the thread blocks until some other thread releases the lock -- the point is
that this is defined and reliable behavior of a Python lock, but of few
other flavors of lock).

However, Python's internal uses of locks-- and particularly the GIL --don't
need these unusual (relative to what platforms usually offer in the way of
native synch gimmicks) semantics.  The GIL is "owned":  a thread T that
acquires the GIL won't try to acquire it again while it's acquired, T will
eventually release it itself, and no other thread will attempt to release it
while T has it held.  Most platforms have a more efficient synch
implementation for this simple usage, but we never got around to creating an
internal lock type to exploit this.  Just about *any* mutex gimmick would
suffice except for a spin lock (the GIL can be held by a thread for an
arbitrarily long time):  the GIL exists solely to enforce serialization.

c'mon-you've-been-working-on-this-for-a-year-already<wink>-ly y'rs  - tim