Why does "except Ex as x" not restore the previous value of x?
Hi, I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ``` does not restore "value" to x after the `except` block. There doesn't seem to be an explanation for this behavior in the docs or PEPs, that I can find. Nor does there seem to be any good technical reason for doing it this way. Anyone know why, or is just one of those unintended consequences like `True == 1`? Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
Cheers, Mark.
On 17Nov2020 09:55, Mark Shannon <mark@hotpy.org> wrote:
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
Because the except is not a new scope. So it is the same "x". Here: https://docs.python.org/3/reference/compound_stmts.html#try it says: When an exception has been assigned using as target, it is cleared at the end of the except clause. This is as if except E as N: foo was translated to except E as N: try: foo finally: del N This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs.
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
In the Python 3.8.5 I don't see this: Python 3.8.5 (default, Jul 21 2020, 10:48:26) [Clang 11.0.3 (clang-1103.0.32.62)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... >>> f(3) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 6, in f UnboundLocalError: local variable 'x' referenced before assignment and the same outside a function. Cheers, Cameron Simpson <cs@cskk.id.au>
On 17/11/2020 10:22 am, Cameron Simpson wrote:
On 17Nov2020 09:55, Mark Shannon <mark@hotpy.org> wrote:
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
Because the except is not a new scope. So it is the same "x".
Here:
https://docs.python.org/3/reference/compound_stmts.html#try
it says:
When an exception has been assigned using as target, it is cleared at the end of the except clause. This is as if
except E as N: foo
was translated to
except E as N: try: foo finally: del N
This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs.
Sorry, I should have made it clearer. I'm not asking what are the semantics of the current version of Python. I'm asking why they are that way.
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
In the Python 3.8.5 I don't see this:
Python 3.8.5 (default, Jul 21 2020, 10:48:26) [Clang 11.0.3 (clang-1103.0.32.62)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... >>> f(3) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 6, in f UnboundLocalError: local variable 'x' referenced before assignment
and the same outside a function.
But why have we chosen for it do this? Wouldn't restoring the value of x be a superior option? Cheers, Mark.
On Tue, Nov 17, 2020 at 3:57 AM Mark Shannon <mark@hotpy.org> wrote:
On 17/11/2020 10:22 am, Cameron Simpson wrote:
On 17Nov2020 09:55, Mark Shannon <mark@hotpy.org> wrote:
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
Because the except is not a new scope. So it is the same "x".
Here:
https://docs.python.org/3/reference/compound_stmts.html#try
it says:
When an exception has been assigned using as target, it is cleared at the end of the except clause. This is as if
except E as N: foo
was translated to
except E as N: try: foo finally: del N
This means the exception must be assigned to a different name to be able to refer to it after the except clause. Exceptions are cleared because with the traceback attached to them, they form a reference cycle with the stack frame, keeping all locals in that frame alive until the next garbage collection occurs.
Sorry, I should have made it clearer.
I'm not asking what are the semantics of the current version of Python. I'm asking why they are that way.
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
In the Python 3.8.5 I don't see this:
Python 3.8.5 (default, Jul 21 2020, 10:48:26) [Clang 11.0.3 (clang-1103.0.32.62)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... >>> f(3) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 6, in f UnboundLocalError: local variable 'x' referenced before assignment
and the same outside a function.
But why have we chosen for it do this?
Because that solution didn't occur to anyone at the time? Because we didn't realize anyone would want those semantics? Because the current semantics evolved from previous semantics that didn't erase the variable at the end of the except block? Why do you need this question answered?
Wouldn't restoring the value of x be a superior option?
That depends. It would be more work to implement, which (at the time that feature was designed, probably more so than today) was a concern. But "superior" is rather subjective. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
On Tue, Nov 17, 2020 at 09:55:46AM +0000, Mark Shannon wrote:
Hi,
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
Because try, if, for, while, with etc. don't create a new scope. Only two statements create a new scope: `def` and `class`. I presume you don't also expect these to restore x after the blocks are ended: x = "value" for x in range(100): pass with open('filename') as x: text = x.read() `try...except` is no different. If you search the archives on Python-List, going back two or four years, you will find extensive discussion between those who think that every statement with an indent should create a new scope, and those who don't. Apologies in advance to supporters of the "new scope" philosophy who might feel I am giving the idea short shrift, but my recollection is that they were unable to give any better justification than "but that's what C does" and "well, it might be useful someday to use a name inside a block without clobbering its existing value". (My response to that is, just use a new variable name, it's not like there is a shortage.) The only wrinkle in the case of `try...except` is that the error variable is deleted, even if it wasn't actually used. If you look at the byte-code generated, each compound try...except with an exception variable is followed by the equivalent of: err = None del err There really ought to be a FAQ about this, but it has something to do with the exception object forming a long-lasting reference cycle. To avoid that, the error variable is nuked on leaving the compound block. This is documented here: https://docs.python.org/3/reference/compound_stmts.html#the-try-statement although only briefly and without much detail.
Nor does there seem to be any good technical reason for doing it this way.
I expect the reason is the usual cost-benefit of "good enough". The effort involved in storing the old name binding and then restoring it afterwards is more than the benefit gained (almost zero). This has been part of Python since Python 3.0, so if you are only just noticing it, that gives an idea of how rarely it comes up.
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
What Python interpreter are you using for that? This is not what happens in Python 3: # raises an exception UnboundLocalError: local variable 'x' referenced before assignment or Python 2.7: # returns the exception object ZeroDivisionError('integer division or modulo by zero',) So unless you are using some modified or alternative interpreter, I don't know how you got that output. -- Steve
On Tue, Nov 17, 2020 at 9:44 PM Steven D'Aprano <steve@pearwood.info> wrote:
`try...except` is no different. ... The only wrinkle in the case of `try...except` is that the error variable is deleted, even if it wasn't actually used. If you look at the byte-code generated, each compound try...except with an exception variable is followed by the equivalent of:
err = None del err
There really ought to be a FAQ about this, but it has something to do with the exception object forming a long-lasting reference cycle. To avoid that, the error variable is nuked on leaving the compound block.
That's a much bigger wrinkle than it might seem at first, though, and I agree, this is a quite literal frequently-asked-question and should be made clear somewhere. The except clause is special in that, if you want the exception afterwards, you have to reassign it to another variable; but it doesn't ACTUALLY introduce a subscope, despite kinda looking like it does. Interestingly, Python 3.10 has a very odd disassembly:
def f(): ... try: g() ... except Exception as e: ... print(e) ... import dis dis.dis(f) 2 0 SETUP_FINALLY 10 (to 12) 2 LOAD_GLOBAL 0 (g) 4 CALL_FUNCTION 0 6 POP_TOP 8 POP_BLOCK 10 JUMP_FORWARD 44 (to 56)
3 >> 12 DUP_TOP 14 LOAD_GLOBAL 1 (Exception) 16 JUMP_IF_NOT_EXC_MATCH 54 18 POP_TOP 20 STORE_FAST 0 (e) 22 POP_TOP 24 SETUP_FINALLY 20 (to 46) 4 26 LOAD_GLOBAL 2 (print) 28 LOAD_FAST 0 (e) 30 CALL_FUNCTION 1 32 POP_TOP 34 POP_BLOCK 36 POP_EXCEPT 38 LOAD_CONST 0 (None) 40 STORE_FAST 0 (e) 42 DELETE_FAST 0 (e) 44 JUMP_FORWARD 10 (to 56) >> 46 LOAD_CONST 0 (None) 48 STORE_FAST 0 (e) 50 DELETE_FAST 0 (e) 52 RERAISE >> 54 RERAISE >> 56 LOAD_CONST 0 (None) 58 RETURN_VALUE
Reconstructing approximately equivalent Python code, this would mean it looks something like this: def f(): try: g() except Exception as e: try: print(e) e = None del e raise finally: e = None del e except: raise return None I don't understand why (a) the "e = None; del e" part is duplicated, nor (b) why the RERAISE opcodes are there in two branches, but I guess it works out best to be explicit in there? Anyhow. You say that this can't come up very often because people can go a long time without asking, but the trouble is that there are two false interpretations that are both extremely close - either that "except E as e:" is similar to "with E as e:", or that the except clause creates its own scope. It's entirely possible to see supporting evidence for your own wrong assumption and never actually know the truth. Maybe this is going to be the next "Python has call-by-value" vs "Python has call-by-reference" debate? ChrisA
Hi Chris, On 17/11/2020 11:03 am, Chris Angelico wrote:
On Tue, Nov 17, 2020 at 9:44 PM Steven D'Aprano <steve@pearwood.info> wrote:
`try...except` is no different. ... The only wrinkle in the case of `try...except` is that the error variable is deleted, even if it wasn't actually used. If you look at the byte-code generated, each compound try...except with an exception variable is followed by the equivalent of:
err = None del err
There really ought to be a FAQ about this, but it has something to do with the exception object forming a long-lasting reference cycle. To avoid that, the error variable is nuked on leaving the compound block.
That's a much bigger wrinkle than it might seem at first, though, and I agree, this is a quite literal frequently-asked-question and should be made clear somewhere. The except clause is special in that, if you want the exception afterwards, you have to reassign it to another variable; but it doesn't ACTUALLY introduce a subscope, despite kinda looking like it does.
Interestingly, Python 3.10 has a very odd disassembly:
def f(): ... try: g() ... except Exception as e: ... print(e) ... import dis dis.dis(f) 2 0 SETUP_FINALLY 10 (to 12) 2 LOAD_GLOBAL 0 (g) 4 CALL_FUNCTION 0 6 POP_TOP 8 POP_BLOCK 10 JUMP_FORWARD 44 (to 56)
3 >> 12 DUP_TOP 14 LOAD_GLOBAL 1 (Exception) 16 JUMP_IF_NOT_EXC_MATCH 54 18 POP_TOP 20 STORE_FAST 0 (e) 22 POP_TOP 24 SETUP_FINALLY 20 (to 46)
4 26 LOAD_GLOBAL 2 (print) 28 LOAD_FAST 0 (e) 30 CALL_FUNCTION 1 32 POP_TOP 34 POP_BLOCK 36 POP_EXCEPT 38 LOAD_CONST 0 (None) 40 STORE_FAST 0 (e) 42 DELETE_FAST 0 (e) 44 JUMP_FORWARD 10 (to 56) >> 46 LOAD_CONST 0 (None) 48 STORE_FAST 0 (e) 50 DELETE_FAST 0 (e) 52 RERAISE >> 54 RERAISE >> 56 LOAD_CONST 0 (None) 58 RETURN_VALUE
Reconstructing approximately equivalent Python code, this would mean it looks something like this:
def f(): try: g() except Exception as e: try: print(e) e = None del e raise finally: e = None del e except: raise return None
The equivalent Python is closer to this: def f(): try: g() except Exception as e: try: print(e) finally: e = None del e
I don't understand why (a) the "e = None; del e" part is duplicated, nor (b) why the RERAISE opcodes are there in two branches, but I guess it works out best to be explicit in there?
The reason for the seeming verbosity of the bytecode is that try: body finally: final compiles to roughly: try: body: except: final raise else: final Which is why you see the duplicated sequences. Cheers, Mark.
Anyhow. You say that this can't come up very often because people can go a long time without asking, but the trouble is that there are two false interpretations that are both extremely close - either that "except E as e:" is similar to "with E as e:", or that the except clause creates its own scope. It's entirely possible to see supporting evidence for your own wrong assumption and never actually know the truth. Maybe this is going to be the next "Python has call-by-value" vs "Python has call-by-reference" debate?
ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6NKGXWLR... Code of Conduct: http://python.org/psf/codeofconduct/
17.11.20 11:55, Mark Shannon пише:
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
There doesn't seem to be an explanation for this behavior in the docs or PEPs, that I can find. Nor does there seem to be any good technical reason for doing it this way.
Others already said that it is because "except" does not create a new scope. But why it does not create a new scope? Because it is a design of Python. In general, since in Python local variables do not have declarations, inner scopes do not work. For example: y = 1 if x: y = 2 If "if" create a new scope, "y" after "if" would be restored to 1. But what if make an exception for "except" and make a special rule for it? First, "Special cases aren't special enough to break the rules". Second, how would you use it? This feature would only encourage to write hard-to-read and errorprone code. This error is so common, that in some other programming languages which support inner scopes shadowing local variable causes compiler warning or even syntax error. So this feature is a misfeature. There is also historical reason. In Python 2 the variable was not deleted after leaving the except block. Now it is deleted, and if the code uses the variable after this it would raise a NameError. It is a clear indication of program error. With your proposition it would have some other value, and program error would not be caught so easily.
Hi, It turns out that implementing the save and restore semantics in the example I gave is not that difficult. I was motivated to find out by the DLS 2020 paper on pattern matching. It claims that introducing small scopes for variables would have to be implemented as a function preventing the use of normal control flow. Since restoring the variable after an except block is a similar problem, I thought I'd see how difficult it was. If anyone's interested, here's a prototype: https://github.com/python/cpython/compare/master...markshannon:fix-exception... (This only saves globals and function-locals, class-locals and non-locals are unchanged. I'd probably want to emit a syntax warning for non-locals, as the semantics are a bit weird). Cheers, Mark. On 17/11/2020 9:55 am, Mark Shannon wrote:
Hi,
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
There doesn't seem to be an explanation for this behavior in the docs or PEPs, that I can find. Nor does there seem to be any good technical reason for doing it this way.
Anyone know why, or is just one of those unintended consequences like `True == 1`?
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KGYRLITE...
Code of Conduct: http://python.org/psf/codeofconduct/
One comment about having the exception handler 'save state' and restore it is that frequently the purpose of the exception handler is TO make changes to outer variable to fix things up based on the exception. I could see the desire of a way to create an internal scope of sorts and create names limited to that scope, but wouldn't want such a scope being anywhere automatic, and likely what would make more sense is defining a particular name (or names) to have a limited scope.
On Wed, Nov 18, 2020 at 4:08 AM Richard Damon <Richard@damon-family.org> wrote:
One comment about having the exception handler 'save state' and restore it is that frequently the purpose of the exception handler is TO make changes to outer variable to fix things up based on the exception.
I could see the desire of a way to create an internal scope of sorts and create names limited to that scope, but wouldn't want such a scope being anywhere automatic, and likely what would make more sense is defining a particular name (or names) to have a limited scope.
The ONLY variable that would be special is the one named in the "except Exc as e:" clause. Everything else would follow the normal rules. This is the only variable that is currently special, and ti'd be the only one that ever needs to be special (since it has the refloop that makes memory management harder). ChrisA
On Wed, Nov 18, 2020 at 4:08 AM Richard Damon <Richard@damon-family.org> wrote:
One comment about having the exception handler 'save state' and restore it is that frequently the purpose of the exception handler is TO make changes to outer variable to fix things up based on the exception.
I could see the desire of a way to create an internal scope of sorts and create names limited to that scope, but wouldn't want such a scope being anywhere automatic, and likely what would make more sense is defining a particular name (or names) to have a limited scope. The ONLY variable that would be special is the one named in the "except Exc as e:" clause. Everything else would follow the normal rules. This is the only variable that is currently special, and ti'd be the only one that ever needs to be special (since it has the refloop that makes memory management harder).
ChrisA My main thought on that variable, is I would expect it to have a good name that implies it is an exception, not something like e, unless you are reserving e for exceptions, so would be unlikely to shadow unless
On 11/17/20 12:16 PM, Chris Angelico wrote: this is from a try block inside an except block that then return to more processing in the outer except block -- Richard Damon
On Tue, Nov 17, 2020 at 12:36:31PM -0500, Richard Damon wrote:
My main thought on that variable, is I would expect it to have a good name that implies it is an exception,
Something like "e" or "err" then, which are the two most common choices I have seen in exception handling code.
not something like e,
There are some very common conventions for single character names, such as i,j,k for loop variables, n for integers, d for dicts, and e for an exception or error is one of them. If you need nested try blocks, and each needs an error variable that have to co-exist, e1, e2, e3 etc are obvious choices. As far as shadowing other variables, if someone has so much code in their function, or at the top level, that they are at risk of inadvertantly shadowing variables, they have far more serious problems than the use of the conventional "e for exception". To start with, what else are they using "e" for? Surely it would be more important to use a better name *there* rather than change the exception variable. -- Steve
On Wed, Nov 18, 2020 at 08:45:10AM +1100, Chris Angelico wrote:
On Wed, Nov 18, 2020 at 8:38 AM Steven D'Aprano <steve@pearwood.info> wrote:
To start with, what else are they using "e" for? Surely it would be more important to use a better name *there* rather than change the exception variable.
2.718281828?
eulers_number = 2.718281828 eulers_constant = 0.57721566490153 https://en.wikipedia.org/wiki/List_of_things_named_after_Leonhard_Euler#Numb... No chance of confusion there :-) -- Steve
On 18Nov2020 08:34, Steven D'Aprano <steve@pearwood.info> wrote:
As far as shadowing other variables, if someone has so much code in their function, or at the top level, that they are at risk of inadvertantly shadowing variables, they have far more serious problems than the use of the conventional "e for exception".
To start with, what else are they using "e" for? Surely it would be more important to use a better name *there* rather than change the exception variable.
Aye, that was my thought too. Avoiding a collision is so easy that adding a magic special scope to Python seems overkill. I'm also against lots of scopes. They cause pain. I wrote a post in (I thought) a Python discussion about adding more scopes citing what I consider a horrible flaw in Go's scope design which shadows the common "err" return from various calls (Go doesn't use exceptions), and provided a code example of how easy it was to accidentally break one's error handling because of that design. Annoyingly, I cannot find that post now. But basicly the if-statement introduces a scrope (which starts at the test, before the {...} brackets) while shadows an earlier variable, and on exiting the scrope the common "err" variable is False again, indicating no error. Really irritating. Cheers, Cameron Simpson <cs@cskk.id.au>
I think we've fallen in the trap of assuming we all agree on the meaning of "scope". In Python, that word is typically used to refer to a bunch of behaviors that go together. In particular, "the current scope" is where assignment creates new variables. (It is also linked to the semantics of "return" and "yield", and to "global" and "nonlocal".) The paper uses it in this sense. The problem with using scopes *in this sense* for case blocks is that all assignments to variables inside a case block would create the new variable in the scope of that case block. But consider this example: ``` match arg: case Point(x, y): p = (x, y) case (x, y): p = (x, y) case _: raise TypeError print(p) ``` There are no assignments to p outside the case blocks. Giving each case block a traditional "scope" would prevent the print(p) call outside the match statement from seeing the variable p. (Please don't argue that this should be refactored by putting the print call in a helper function.) Now, of course we could change the language so that this all works. But it would require a lot of changes to the compiler, since it gives scopes a bundle of semantics that would have to be separated out. We could also agree to keep the language the same and use "scope" with a different meaning (the meaning that you seem to want). But you can't blame the paper for using it in the way it is used, since that is the conventional usage of the term for Python. On Tue, Nov 17, 2020 at 7:03 AM Mark Shannon <mark@hotpy.org> wrote:
Hi,
It turns out that implementing the save and restore semantics in the example I gave is not that difficult.
I was motivated to find out by the DLS 2020 paper on pattern matching. It claims that introducing small scopes for variables would have to be implemented as a function preventing the use of normal control flow.
Since restoring the variable after an except block is a similar problem, I thought I'd see how difficult it was.
If anyone's interested, here's a prototype:
https://github.com/python/cpython/compare/master...markshannon:fix-exception...
(This only saves globals and function-locals, class-locals and non-locals are unchanged. I'd probably want to emit a syntax warning for non-locals, as the semantics are a bit weird).
Cheers, Mark.
On 17/11/2020 9:55 am, Mark Shannon wrote:
Hi,
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
There doesn't seem to be an explanation for this behavior in the docs or PEPs, that I can find. Nor does there seem to be any good technical reason for doing it this way.
Anyone know why, or is just one of those unintended consequences like `True == 1`?
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KGYRLITE...
Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IAY4XHLO... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
I think Python's rules about what constitutes a new scope are a touch inconsistent--or, as the Zen might put it, "practical". First, simple statements never create their own scope. That's easy. Second, compound statements that create a new function, "def" and "lambda", always create a new scope. "class" also creates a new scope, but that's intuitive--and if you understand the language you know that under the hood it's also creating a code object like "def" and "lambda" do. Anyway, none of the other "compound" statements create scopes. That's true even for compound statements that themselves make assignments, like "for", "with", and "except". But you'll note they also don't create new code objects. Finally, we note that modules always have their own scopes. And again, if you understand the language, you know they're also implemented using code objects. If that was the whole list, then the mental model would be super simple: code objects always create their own scopes, and nothing else does. But then we get to generator expressions and list/dict/set comprehensions. I think of those as doing an "assignment", but I have to remember the assignment "doesn't leak": x = 3 y = list(x**2 for x in range(5)) print(f"{x=}") This code prints "x=3". Why this exception to the rule? IIRC from the discussions back when they were added, this one of those "it just seemed best at the time" things. "for" was added to the language long before I got there, but it "leaking" its assignment was judged useful; if you broke out of the loop, you could continue to examine the last value from the loop. But, the thinking went, you'd never want to examine the last value from a list generator, so it was more convenient if it behaved as if it had its own scope. //arry/ p.s. Maybe I should be using the word "binding" instead of "assignment"? If there's an important distinction between the two terms I don't know it. On 11/17/20 1:55 AM, Mark Shannon wrote:
Hi,
I'm wondering why ``` x = "value" try: 1/0 except Exception as x: pass ```
does not restore "value" to x after the `except` block.
There doesn't seem to be an explanation for this behavior in the docs or PEPs, that I can find. Nor does there seem to be any good technical reason for doing it this way.
Anyone know why, or is just one of those unintended consequences like `True == 1`?
Here's an example of restoring the value of the variable after the `except` block:
def f(x): ... try: ... 1/0 ... except Exception as x: ... pass ... return x ... f("hi") 'hi'
Cheers, Mark. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KGYRLITE... Code of Conduct: http://python.org/psf/codeofconduct/
On 2020-11-18 03:36, Larry Hastings wrote:
[snip]
But then we get to generator expressions and list/dict/set comprehensions. I think of those as doing an "assignment", but I have to remember the assignment "doesn't leak":
x = 3 y = list(x**2 for x in range(5)) print(f"{x=}")
This code prints "x=3".
Why this exception to the rule? IIRC from the discussions back when they were added, this one of those "it just seemed best at the time" things. "for" was added to the language long before I got there, but it "leaking" its assignment was judged useful; if you broke out of the loop, you could continue to examine the last value from the loop. But, the thinking went, you'd never want to examine the last value from a list generator, so it was more convenient if it behaved as if it had its own scope.
[snip] I think the reason that generator expressions don't leak is that experience of list comprehensions, which did leak, showed that it was a bad idea. List comprehensions were "fixed" in Python 3.
On 18/11/20 4:36 pm, Larry Hastings wrote:
But, the thinking went, you'd never want to examine the last value from a list generator, so it was more convenient if it behaved as if it had its own scope.
List comprehensions used to leak, but apparently that was considered surprising by enough people that it was changed. Generator expressions are a bit different -- the only sane way to implement them was to make the body implicitly a separate function, and non-leaking behaviour naturally fell out of that. Making them leak would have taken extra work just to get something that nobody really had a good use case for. The desire to make list comprehensions and generator expressions behave consistently may have contributed to the decision to change list comprehensions to be non-leaking. So yes, it's all a bit messy, for reasons that are partly historical and partly pragmatic, but things seem to work out okay in practice most of the time. If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction. -- Greg
On Tue, Nov 17, 2020 at 10:16 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 18/11/20 4:36 pm, Larry Hastings wrote:
But, the thinking went, you'd never want to examine the last value from a list generator, so it was more convenient if it behaved as if it had its own scope.
List comprehensions used to leak, but apparently that was considered surprising by enough people that it was changed.
Generator expressions are a bit different -- the only sane way to implement them was to make the body implicitly a separate function, and non-leaking behaviour naturally fell out of that. Making them leak would have taken extra work just to get something that nobody really had a good use case for.
The desire to make list comprehensions and generator expressions behave consistently may have contributed to the decision to change list comprehensions to be non-leaking.
It did if I remember correctly. For me, generator expressions are small functions like you say (see, people did get the equivalent of multi-line lambdas, just in a very specific format 😉), and all the comprehensions are just passing a generator expression to the appropriate constructor, e.g. list(), set(), and dict(). In that regard the scoping is consistent to me. -Brett
So yes, it's all a bit messy, for reasons that are partly historical and partly pragmatic, but things seem to work out okay in practice most of the time.
If there's anything I would change, it would be to have the for statement create a new binding on each iteration, so that capturing it with a def or lambda inside the loop works as expected. I even came up with a way to do that while still allowing the last-bound value to be seen afterwards, but the idea didn't gain any traction.
-- Greg _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/FC7WYLN7... Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Nov 18, 2020 at 7:45 PM Brett Cannon <brett@python.org> wrote:
On Tue, Nov 17, 2020 at 10:16 PM Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
On 18/11/20 4:36 pm, Larry Hastings wrote:
But, the thinking went, you'd never want to examine the last value from a list generator, so it was more convenient if it behaved as if it had
its
own scope.
List comprehensions used to leak, but apparently that was considered surprising by enough people that it was changed.
Generator expressions are a bit different -- the only sane way to implement them was to make the body implicitly a separate function, and non-leaking behaviour naturally fell out of that. Making them leak would have taken extra work just to get something that nobody really had a good use case for.
The desire to make list comprehensions and generator expressions behave consistently may have contributed to the decision to change list comprehensions to be non-leaking.
It did if I remember correctly.
For me, generator expressions are small functions like you say (see, people did get the equivalent of multi-line lambdas, just in a very specific format 😉), and all the comprehensions are just passing a generator expression to the appropriate constructor, e.g. list(), set(), and dict(). In that regard the scoping is consistent to me.
-Brett
If match were to create a scope, would making names in that scope nonlocal by default work? (Obvs we'd then need a local declaration, but given global's long-standing presence in the language and the more recent addition of nonlocal it seems like a fairly natural extension).
participants (12)
-
Brett Cannon -
Cameron Simpson -
Chris Angelico -
Greg Ewing -
Guido van Rossum -
Larry Hastings -
Mark Shannon -
MRAB -
Richard Damon -
Serhiy Storchaka -
Steve Holden -
Steven D'Aprano