[Python-Dev] Last chance!

[Tim Peters]

[Tim]
>> Still, the possibility to switch threads across generator
>> resumptions seems darned hard to view as "a feature".  I'd advise
>> people not to rely on it <wink>.

[Christian Tismer]
> Well, this might only work for generators if it is guaranteed
> that the thread switch doesn't happen while the generator
> is executing. I think this will give an exception. But most
> probably the result will be hard to predict.

I'm not sure I'm following, but nothing in Python now *stops* you from
resuming a suspended generator from another thread.  Here's an extreme
example of that:

"""
import threading

def myid():
    while True:
        yield threading.currentThread()
next = myid().next

mut = threading.Lock()

class Worker(threading.Thread):
    def run(self):
        while True:
            mut.acquire()
            print next()
            mut.release()

for i in range(3):
    Worker().start()
"""

Then the body of the single generator is executed by 3 different threads,
more or less randomly, although never by the thread that created the
generator.

If the body of the generator were relying on thread-local state (like, e.g.,
the mass of settings in the decimal arithmetic package's per-thread default
"numeric context" object), the result would sure be unpredictable.  It
wouldn't crash Python, it would just be a bitch to debug.  Then again, lots
of bad threading practices lead to bitch-to-debug code, so I'm not really
inclined to offer extra protection here.

> Other for my "tasklet" tiny threads. Their first purpose is
> to have concurrent small tasks without interaction (or if
> with interaction, then via channels and blocking). A tasklet
> being interrupted by a thread switch will not be auto-schedulable
> until the thread is resumed, but other tasklets might be, and
> they don't need to care about which real thread is actually in
> charge. This is one of my reasons why I want to get rid of
> f_tstate -- it does not really apply to most of my frames.
> Being tied to a thread is a property of the tasklet, and it
> does not apply all the time, only when I have to keep C state.

If a tasklet can be affected by thread-local state (like numeric context --
Python has little of this nature now, but it's bound to increase), then a
tasklet will also start to care "who's in charge".  So I expect a patch to
reintroduce f_tstate in about a year <wink>.