[Python-ideas] A real life example of "given"
steve at pearwood.info
Thu May 31 20:42:01 EDT 2018
On Thu, May 31, 2018 at 11:03:56AM -0700, Brendan Barnwell wrote:
> What I don't understand is this: if we believe that, then why was
> comprehension-leaking EVER removed? Everything that I've seen
> advocating for this kind of leaking seems to me like it is much more
> logically consistent with allowing all comprehension variables to leak
> than it is with the current behavior, in which they don't leak.
Originally list comprehensions ran in the same scope as their
surroundings. In Python 2:
py> [1 for x in ("spam", "eggs")]
but when generator expressions were introduced a few years later, they
ran in their own sub-local scope. This was, I believe, initially
introduced to simplify the common situation that you return a generator
from inside a function:
x = "something big"
return (x for x in seq)
would require holding onto a closure of the factory locals, including
the "something big" object. Potentially forever, if the generator
expression is never iterated over.
(I hope someone will correct me if I have misunderstood.)
To avoid that, the x in the generator was put into a distinct scope from
the x in factory. Either way, the PEP introducing generator expressions
makes it clear that it was a deliberate decision:
The loop variable (if it is a simple variable or a tuple of
simple variables) is not exposed to the surrounding function.
This facilitates the implementation and makes typical use
cases more reliable.
In the case of loop variables, there is an argument from "practicality
beats purity" that they ought to run in their own scope. Loop variables
tend to be short, generic names like "i", "x", "obj", all the more
likely to clash with names in the surrounding scope, and hard to spot
when they do. They're also likely to be used inside loops:
for x in [1, 2, 3, 4]:
alist = [expr for x in range(50)]
# oops, x has been accidentally overridden (in Python 2)
I don't *entirely* buy that argument, and I occasionally find it useful
to inspect the loop variable of a list comprehension after it has run,
but this is one windmill I'm not going to tilt against. Reversing the
decision to put the loop variables in their own sublocal scope is *not*
part of this PEP.
But this is less likely to be a problem for explicit assignment. Outside
of toy examples, we're more likely to assign to descriptive names and
less likely to clash with any surrounding loop variable:
for book in library:
text = [content for chapter in books.chapters()
if (content := chapter.get_text(all=True)
and re.match(pattern, content)]
Given an obvious and explicit assignment to "chapter", say, we are more
likely to realise when we are reusing a name (compared to assigning to
a loop variable i, say). At least, we should be no more likely to mess
this up than we are for any other local-level assignment.
More information about the Python-ideas