threading.RLock not subclassible?

[David Bolen]

Rick Lee <rwklee at home.com> writes:

> It seems to me that OS-specific behaviour of thread switching is _very_
> different among platforms.  For example, if the parent thread starts up 100
> different threads each with some long loop to run:
> 
>     for i in xrange(100):
>         try:
>             t = threading.Thread(target=runObj, args=())
>             t.start()
>         except:
>             break
>     snapshot = threading.activeCount()
> 
> If I run it in different platforms, snapshot is consistently
> different, anywhere between very close to 100, or just 1.  You may
> argue it depends on how long runObj takes to run; but I can change
> runObj to run for a very long time, say seconds, and the results are
> consistently different across the platforms (NT, Mac, Linux).

I suppose the question is why would the difference in shapshot be a
problem to an application?  What you're seeing is that how threads are
dispatched at creation with respect to the currently executing thread
can differ across platforms (and presumably whether the thread support
is native or how deep in the OS it is supported).  But it's not really
a problem, per se, except that it's something that's part and parcel
of multi-threaded development.

By it's nature, when you attempt to start threads their execution with
respect to the starting thread is asynchronous and unrelated.  I'm
guessing the two major cases you're seeing are either (a) when each
new thread gets some execution time right at creation and (b) when the
main thread runs to completion before any of the created threads
really got started.  Thus having nearly a full count versus just a
count of your main thread.  Changing the runtime of runObj won't
likely change that since it's the the scheduling aspect of the
separate threads with respect to the main thread that is causing the
behavior.  You might find though that adding a delay to your main
thread in between (or just after) the creation of the other threads
gives them all a chance to start.

If it's important to you that all of the threads actually begin
executing before you continue the main thread, you'll need to use some
sort of event or other notification mechanism (the quickest way being
a slow loop that waits until the activeCount shows all threads).  You
can't assume that the execution timeframe of the main thread will have any
specific correlation (technically even on the same platform, much less
cross-platform) to the execution timeframe of the other threads.

> I have some even weirder differences, if I use say time.sleep(.001)
> inside a thread, as the FAQ says.  And I have a few other examples I
> ran across.  (The one that bugs me the most is that some blocked
> threads in the Mac just don't get to run, when the blocking
> condition is removed.)

Can you be more specific as to the other wierder differences - they
may just be natural behavior in a multi-threaded environment.  Adding
a time.sleep() within a thread will typically cause that thread to
yield it's control of the processor, and some other thread will get
dispatched, but that's about it.  Within Python, time.sleep() will
also release the global interpreter lock, thus permitting other Python
threads to execute.  That can be important if you have a thread that
otherwise would loop rapidly (without any natural blocking point),
since it relieves some of the unnecessary CPU load.  The best case is
having a thread block until some event occurs that needs to be
serviced.

For your Mac case, how are you blocking the threads?  If you have
multiple threads all using wait() with a timeout on the same event,
then the threading module implements the timeout wait as a loop around
a non-blocking wait, and (as discussed in this thread) that can result
in a thread never being woken up, since the looping construct is not
atomic.  But I believe that uncertainty is cross-platform :-)

But if you use non-timeout waits, then I believe it should eventually
wake up, although the order may be non-deterministic, although someone
else may know more about the Mac thread implementation to comment
differently.

--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/