[Python-Dev] PEP 3148 ready for pronouncement

[Glyph Lefkowitz]

On May 24, 2010, at 5:36 AM, Brian Quinlan wrote:
> On May 24, 2010, at 5:16 AM, Glyph Lefkowitz wrote:
>> On May 23, 2010, at 2:37 AM, Brian Quinlan wrote:
>>> On May 23, 2010, at 2:44 PM, Glyph Lefkowitz wrote:

> ProcessPoolExecutor has the same serialization perils that multiprocessing does. My original plan was to link to the multiprocessing docs to explain them but I couldn't find them listed.

Linking to the pickle documentation might be a good start.

> Yes, the execution context is Executor-dependent. The section under ProcessPoolExecutor and ThreadPoolExecutor spells this out, I think.

I suppose so.  I guess I'm just looking for more precise usage of terminology. (This is a PEP, after all.  It's a specification that multiple VMs may have to follow, not just some user documentation for a package, even if they'll *probably* be using your code in all cases.)  I'd be happier if there were a clearer term than "calls" for the things being scheduled ("submissions"?), since the done callbacks aren't called in the subprocess for ProcessPoolExecutor, as we just discussed.

>> Sure.  Really, almost any contract would work, it just needs to be spelled out.  It might be nice to know whether the thread invoking the callbacks is a daemon thread or not, but I suppose it's not strictly necessary.
> 
> Your concerns is that the thread will be killed when the interpreter exits? It won't be.

Good to know.  Tell it to the PEP though, not me ;).

>> No reaction on [invoker vs. future]?  I think you'll wish you did this in a couple of years when you start bumping into application code that calls "set_result" :).
> 
> My reactions are mixed ;-)

Well, you are not obliged to take my advice, as long as I am not obliged to refrain from mocking you mercilessly if it happens that I was right in a couple of years ;-).

> Your proposal is to add a level of indirection to make it harder for people to call implementation methods. The downside is that it makes it a bit harder to write tests and Executors.

Both tests and executors will still create and invoke methods directly on one object; the only additional difficulty seems to be the need to type '.future' every so often on the executor/testing side of things, and that seems a cost well worth paying to avoid confusion over who is allowed to call those methods and when.

> I also can't see a big problem in letting people call set_result in client code though it is documented as being only for Executor implementations and tests. 
> 
> On the implementation side, I don't see why an Invoker needs a reference to the future.

Well, uh...

> class Invoker(object):
>   def __init__(self):
>     """Should only be called by Executor implementations."""
>     self.future = Future()
         ^ this is what I'd call a "reference to the future"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100526/44d901e1/attachment.html>