On Mar 12, 2020, at 08:53, Eric Wieser <wieser.eric+numpy@gmail.com> wrote:
The nth reference is not removed until the n+1th reference has been created. This is unavoidable in a for loop without breaking existing code,
First, is it unavoidable? What could it break if the loop unbound the variable before nexting the iterator? Unless the iterator knew what variable was being used and explicitly looked it up in its __next__ (which I think you could only do with non-portable frame hackery), there would be no visible difference inside the iterator except for the one you’re explicitly looking for here. And there wouldn’t be any visible difference in the loop code. In fact, you could probably test this with a bytecode-hacking decorator that just replaces any GET_ITER/FOR_ITER/STORE_FAST n with GET_ITER/DELETE_FAST n/FOR_ITER/STORE_FAST n, and use that to prove that it improves performance in your case, only slightly worsens it in other cases (a real implementation could modify the behavior of the bytecodes instead of inserting a new one, but this proof of concept can’t), and doesn’t break any code (write an import hook that implies this transformation to all code, and then run the test suite with the hook injected).
but I think could (and should?) be changed in a list comprehension
I think that would actually be confusing. Comprehension loops are exactly the same as for statements in this way, which means one thing to learn instead of two. In fact, if I were designing a new language, the loop variable would be a new binding (a separate variable) each time through, because I think that’s what people actually expect. (Everyone runs into the problem of capturing loop variables in closures —not just in Python, but in radically different languages like C++. And everyone not only has a hard time understanding this the first time, they also keep running into it over and over even after they’ve learned it.) But it’s obviously too late to do that for Python. And the fact that Python explicitly leaves the binding after the loop makes the behavior that most languages have less confusing in Python than in most languages (but still somewhat confusing). Unbinding at the start of the loop instead of the end doesn’t have that problem—there is no place that user code can see where anything is changed. And that works just as well in for statements as in comprehensions, so there’s no difference to learn, understand, and remember.