[Python-ideas] @run_as_thread decorator

[Jesse Noller]

On Sat, Mar 5, 2011 at 10:31 PM, Guido van Rossum <guido at python.org> wrote:
> On Sat, Mar 5, 2011 at 7:25 PM, Jesse Noller <jnoller at gmail.com> wrote:
>> On Sat, Mar 5, 2011 at 7:55 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> On Sun, Mar 6, 2011 at 3:16 AM, Jesse Noller <jnoller at gmail.com> wrote:
>>>> I've long wanted to put something into the stdlib like this, but as
>>>> others in the thread have pointed out - there's some semantics that
>>>> remain to be hashed out and the behavior is dangerous (imo), and
>>>> magical to have in the stdlib right now.
>>>>
>>>> In this case, I would recommend building out a library that contains
>>>> these decorators (both threads and processes) building from the
>>>> futures (concurrent.futures.Executor ABC) library as possible, and
>>>> let's see how it pans out. I've struggled with really liking/wanting
>>>> this and the fact that it's dangerous, and surprising.
>>>
>>> Well said, especially the last line :)
>>>
>>> However, I suspect this is one of those things where:
>>> - rewriting it yourself is easier than finding a library for it, so a
>>> PyPI module would gather little interest or feedback
>>> - doing it "right" in the stdlib would eliminate the temptation to
>>> develop custom not-quite-right implementations (e.g. ones where the
>>> decorator actually *creates* and starts the thread)
>>>
>>> It would make a good topic for a PEP, IMO.
>>>
>>> Cheers,
>>> Nick.
>>
>> Well, why stop at one-function in a thread - why not have:
>>
>> @threading.pool(10):
>> def func(me):
>>    .... runs func in a threadpool of 10
>>
>> Or
>>
>> with threading.pool(10) as pool:
>>    pool.in.put(blah)
>>    pool.out.get(blah)
>>
>> And so on - I can totally buy that it's pep fodder to do it "right"
>> once, for the stdlib, I've just struggled and debated with a lot of
>> people about how helpful this really is and the right way of doing it.
>> The original proposal seems OK at first - just run this function in a
>> thread - at first glance it seems like it's a harmless decorator to
>> throw a function into a thread or process. But then what about the
>> "surprise" that a function call forked a new thread - or new process?
>>
>> Maybe you're right - maybe this is good PEP territory. If nothing more
>> we could all hash out the various way of biting yourself in the butt
>> with these :)
>
> I'm thinking that since all of this is all pretty simple it's
> something that applications or 3rd party libraries can easily have in
> a "utilities" module of their own. The bar is a lot lower that way.

True, they're trivial to implement, and Nick may be right that there's
no way for a package to be able to gain traction that's solely based
on this idea. But something that builds on decorator and context
manager usage of threads and pools / futures might be able to gain a
fair amount of it.

>> ps: i love controlling thread/process pools with context managers. I'm ill.
>
> Sick, you mean. :-)

Yes, this is true as well.