[Python-Dev] Please reconsider PEP 479.

[Guido van Rossum]

On Thu, Nov 27, 2014 at 3:04 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 27 November 2014 at 11:15, Guido van Rossum <guido at python.org> wrote:
> > On Wed, Nov 26, 2014 at 2:53 PM, Nick Coghlan <ncoghlan at gmail.com>
> wrote:
> >>
> >> On 27 Nov 2014 06:35, "Guido van Rossum" <guido at python.org> wrote:
> >>
> >> [...]
> >>
> >> > I think we can put a number to "much faster" now -- 150 nsec per
> >> > try/except.
> >> >
> >> > I have serious misgivings about that decorator though -- I'm not sure
> >> > how viable it is to pass a flag from the function object to the
> execution
> >> > (which takes the code object, which is immutable) and how other Python
> >> > implementations would do that. But I'm sure it can be done through
> sheer
> >> > willpower. I'd call it the @hettinger decorator in honor of the PEP's
> most
> >> > eloquent detractor. :-)
> >>
> >> I agree with everything you wrote in your reply, so I'll just elaborate
> a
> >> bit on my proposed implementation for the decorator idea.
> >
> > This remark is  ambiguous -- how strongly do you feel that this decorator
> > should be provided? (If so, it should be in the PEP.)
>
> I think it makes sense to standardise it, but something like
> "itertools.allow_implicit_stop" would probably be better than having
> it as a builtin. (The only reason I suggested a builtin initially is
> because putting it in itertools didn't occur to me until later)
>
> Including the decorator provides a straightforward way to immediately
> start writing forward compatible code that's explicit about the fact
> it relies on the current StopIteration handling, without being
> excessively noisy relative to the status quo:
>
>     # In a module with a generator that relies on the current behaviour
>     from itertools import allow_implicit_stop
>
>     @allow_implicit_stop
>     def my_generator():
>         ...
>         yield next(itr)
>         ...
>
> In terms of code diffs to ensure forward compatibility, it's 1 import
> statement per affected module, and 1 decorator line per affected
> generator, rather than at least 3 lines (for try/except/return) plus
> indentation changes for each affected generator. That's a useful
> benefit when it comes to minimising the impact on version control code
> annotation, etc.
>
> If compatibility with older Python versions is needed, then you could
> put something like the following in a compatibility module:
>
>     try:
>         from itertools import allow_implicit_stop
>     except ImportError:
>         # Allowing implicit stops is the default in older versions
>         def allow_implicit_stop(g):
>             return g
>

I understand that @allow_import_stop represents a compromise, an attempt at
calming the waves that PEP 479 has caused. But I still want to push back
pretty hard on this idea.

- It means we're forever stuck with two possible semantics for
StopIteration raised in generators.

- It complicates the implementation, because (presumably) a generator
marked with @allow_stop_import should not cause a warning when a
StopIteration bubbles out -- so we actually need another flag to silence
the warning.

- I don't actually know whether other Python implementations have the
ability to copy code objects to change flags.

- It actually introduces a new incompatibility, that has to be solved in
every module that wants to use it (as you show above), whereas just putting
try/except around unguarded next() calls is fully backwards compatible.

- Its existence encourage people to use the decorator in favor of fixing
their code properly.

- The decorator is so subtle that it probably needs to be explained to
everyone who encounters it (and wasn't involved in this PEP discussion).
Because of this I would strongly advise against using it to "fix" the
itertools examples in the docs; it's just too magical. (IIRC only 2
examples actually depend on this.)

Let me also present another (minor) argument for PEP 479. Sometimes you
want to take a piece of code presented as a generator and turn it into
something else. You can usually do this pretty easily by e.g. replacing
every "yield" by a call to print() or list.append(). But if there are any
bare next() calls in the code you have to beware of those. If the code was
originally written without relying on bare next(), the transformation would
have been easier.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141127/eba4571c/attachment.html>