[Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On 26 May 2010, at 18:09, Glyph Lefkowitz wrote:

>
> On May 24, 2010, at 5:36 AM, Brian Quinlan wrote:
>> On May 24, 2010, at 5:16 AM, Glyph Lefkowitz wrote:
>>> On May 23, 2010, at 2:37 AM, Brian Quinlan wrote:
>>>> On May 23, 2010, at 2:44 PM, Glyph Lefkowitz wrote:
>
>> ProcessPoolExecutor has the same serialization perils that  
>> multiprocessing does. My original plan was to link to the  
>> multiprocessing docs to explain them but I couldn't find them listed.
>
> Linking to the pickle documentation might be a good start.

Will do.

>> Yes, the execution context is Executor-dependent. The section under  
>> ProcessPoolExecutor and ThreadPoolExecutor spells this out, I think.
>
> I suppose so.  I guess I'm just looking for more precise usage of  
> terminology. (This is a PEP, after all.  It's a specification that  
> multiple VMs may have to follow, not just some user documentation  
> for a package, even if they'll *probably* be using your code in all  
> cases.)  I'd be happier if there were a clearer term than "calls"  
> for the things being scheduled ("submissions"?), since the done  
> callbacks aren't called in the subprocess for ProcessPoolExecutor,  
> as we just discussed.
>
>>> Sure.  Really, almost any contract would work, it just needs to be  
>>> spelled out.  It might be nice to know whether the thread invoking  
>>> the callbacks is a daemon thread or not, but I suppose it's not  
>>> strictly necessary.
>>
>> Your concerns is that the thread will be killed when the  
>> interpreter exits? It won't be.
>
> Good to know.  Tell it to the PEP though, not me ;).

Will do.

>>> No reaction on [invoker vs. future]?  I think you'll wish you did  
>>> this in a couple of years when you start bumping into application  
>>> code that calls "set_result" :).
>>
>> My reactions are mixed ;-)
>
> Well, you are not obliged to take my advice, as long as I am not  
> obliged to refrain from mocking you mercilessly if it happens that I  
> was right in a couple of years ;-).

I was looking for your reasoning rather than trying to negotiate the  
circumstances under which you would mock me.
>
>> Your proposal is to add a level of indirection to make it harder  
>> for people to call implementation methods. The downside is that it  
>> makes it a bit harder to write tests and Executors.
>
> Both tests and executors will still create and invoke methods  
> directly on one object; the only additional difficulty seems to be  
> the need to type '.future' every so often on the executor/testing  
> side of things, and that seems a cost well worth paying to avoid  
> confusion over who is allowed to call those methods and when.
>
>> I also can't see a big problem in letting people call set_result in  
>> client code though it is documented as being only for Executor  
>> implementations and tests.
>>
>> On the implementation side, I don't see why an Invoker needs a  
>> reference to the future.
>
> Well, uh...
>
>> class Invoker(object):
>>   def __init__(self):
>>     """Should only be called by Executor implementations."""
>>     self.future = Future()
>          ^ this is what I'd call a "reference to the future"

I said exactly the opposite of what I meant: futures don't need a  
reference to the invoker.

Cheers,
Brian