[Python-ideas] @run_as_thread decorator

[Jesse Noller]

On Sat, Mar 5, 2011 at 7:55 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Mar 6, 2011 at 3:16 AM, Jesse Noller <jnoller at gmail.com> wrote:
>> I've long wanted to put something into the stdlib like this, but as
>> others in the thread have pointed out - there's some semantics that
>> remain to be hashed out and the behavior is dangerous (imo), and
>> magical to have in the stdlib right now.
>>
>> In this case, I would recommend building out a library that contains
>> these decorators (both threads and processes) building from the
>> futures (concurrent.futures.Executor ABC) library as possible, and
>> let's see how it pans out. I've struggled with really liking/wanting
>> this and the fact that it's dangerous, and surprising.
>
> Well said, especially the last line :)
>
> However, I suspect this is one of those things where:
> - rewriting it yourself is easier than finding a library for it, so a
> PyPI module would gather little interest or feedback
> - doing it "right" in the stdlib would eliminate the temptation to
> develop custom not-quite-right implementations (e.g. ones where the
> decorator actually *creates* and starts the thread)
>
> It would make a good topic for a PEP, IMO.
>
> Cheers,
> Nick.

Well, why stop at one-function in a thread - why not have:

@threading.pool(10):
def func(me):
    .... runs func in a threadpool of 10

Or

with threading.pool(10) as pool:
    pool.in.put(blah)
    pool.out.get(blah)

And so on - I can totally buy that it's pep fodder to do it "right"
once, for the stdlib, I've just struggled and debated with a lot of
people about how helpful this really is and the right way of doing it.
The original proposal seems OK at first - just run this function in a
thread - at first glance it seems like it's a harmless decorator to
throw a function into a thread or process. But then what about the
"surprise" that a function call forked a new thread - or new process?

Maybe you're right - maybe this is good PEP territory. If nothing more
we could all hash out the various way of biting yourself in the butt
with these :)

jesse

ps: i love controlling thread/process pools with context managers. I'm ill.