[Python-Dev] Please reconsider PEP 479.

[Nick Coghlan]

On 27 Nov 2014 06:35, "Guido van Rossum" <guido at python.org> wrote:
>
> On Wed, Nov 26, 2014 at 3:24 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> After thinking about that concern for a while, I'd like to suggest the
>> idea of having a new builtin "allow_implicit_stop" decorator that
>> swaps out a GENERATOR code object that has the new "EXPLICIT_STOP"
>> flag set for one with it cleared (attempting to apply
>> "allow_implicit_stop" to a normal function would be an error).
>>
>> Then the updated version of the above example would become:
>>
>>     @allow_implicit_stop
>>     def my_generator():
>>         ...
>>         yield next(it)
>>         ...
>>
>> Which would be semantically equivalent to:
>>
>>     def my_generator():
>>        try:
>>            ...
>>             yield next(it)
>>             ...
>>         except StopIteration
>>             return
>>
>> but *much* faster (especially if used in a producer/consumer pipeline)
>> since it would allow a single StopIteration instance to propagate
>> through the entire pipeline, rather than creating and destroying new
>> ones at each stage.
>
>
> I think we can put a number to "much faster" now -- 150 nsec per
try/except.
>
> I have serious misgivings about that decorator though -- I'm not sure how
viable it is to pass a flag from the function object to the execution
(which takes the code object, which is immutable) and how other Python
implementations would do that. But I'm sure it can be done through sheer
willpower. I'd call it the @hettinger decorator in honor of the PEP's most
eloquent detractor. :-)

I agree with everything you wrote in your reply, so I'll just elaborate a
bit on my proposed implementation for the decorator idea.

What I was thinking is that to implement the __future__ import, we're going
to need a code object flag. For the sake of discussion, let's assume that
flag is called "EXPLICIT_STOP". When StopIteration escapes from a generator
frame, we'll check for that flag and if it's set, StopIteration will be
converted to RuntimeError. The details will differ for other
implementations, but they're going to need something at least approximately
equivalent to handle the transition period.

The implicit stop decorator would then check the flags on the code object
attached to the passed in function. If GENERATOR wasn't set, that would be
an immediate ValueError, while if EXPLICIT_STOP wasn't set, the generator
function would be passed through unmodified. However, if EXPLICIT_STOP
*was* set, the generator function would be replaced by a *new* generator
function with a *new* code object, where the only change was to clear the
EXPLICIT_STOP flag.

The core part of the idea is actually keeping the runtime check for the
EXPLICIT_STOP flag on generator code objects even *after* setting the flag
becomes the default compile time behaviour. The builtin (or itertools?)
decorator to revert to the current behaviour on a per-generator basis then
becomes a convenient way to access (and document!) that functionality
without messing about with your own dynamic code object creation, and
without coming up with new syntax for it (since new syntax can't be
emulated on older versions of Python, or made implicit in other decorators
like contextlib.contextmanager).

Regards,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141127/e499c79e/attachment.html>