[PoC] Implementing block-scope for the "for" statement variables

Hello, This is continuation of the "Making the for statement work better with nested functions" thread (https://mail.python.org/archives/list/python-ideas@python.org/thread/3ZKYV6W...). I post this as a separate thread to draw more attention to and promote this way of prototyping Python changes. The changes are implemented using Python3 port of Python2's "compiler" module: https://github.com/pfalcon/python-compiler . For more information on background, see the parallel thread: https://mail.python.org/archives/list/python-ideas@python.org/message/W2FREL... python-compiler module requires CPython3.5. So, please be prepared to enjoy again the golden age of Python3 (comparing to iron (perhaps, oil?) age of now). If you don't have that handy, the repository has a script which will build it for you in a couple of minutes (on Linux). The changes are implemented in the following branch: https://github.com/pfalcon/python-compiler/commits/for-block-scope . The top 3 commits are that, shouldn't be too hard to review (and have some additional info in the commit messages). As what's available there is only AST-to-bytecode compiler, not source parser, I couldn't easily implement "for let"/"for const" syntax, so went to switch *all* for statements to the block scope for this prototype. I also prototyped only "block scope" part, not the "constness" part. To describe high-level idea of the changes: The conceptual idea is that we can rewrite each definition of block-scoped variable to have a unique name, and then rewrite usages accordingly in all inner scopes (including inner functions). Direct implementation of that was deemed "too naive", based on the grounds that (again naive) implementation of it would require whole extra pass on AST, and patching it. Note that it's not necessarily that (what? bad), and if it ever comes to implementing in the C compiler, it may be a good idea to actually go that way first and see how actually "bad" it is. It may be possible to combine that pass with another pass (variable scoping AST pass, literally). The benefit of this approach is that it's "obviously correct" and doesn't introduce other overheads. Anyway, my point was to show that even doing it "better" is "not rocket science". I did it half-way still, because there's obvious rising conceptual complexity and rising storage requirements for intermediate info (what made me think that maybe "naive" approach described above isn't that bad). Here's example of it in action: $ cat example_for1.py def fun(): x = 123 for x in range(5): print(x) print("old x:", x) fun() $ ./python3.5-nopeephole -m compiler --dis example_for1.py 1 0 LOAD_CONST 0 (<code object fun at 0x7f7533bb1ed0, file "example_for1.py", line 1>) 3 LOAD_CONST 1 ('fun') 6 MAKE_FUNCTION 0 9 STORE_NAME 0 (fun) 7 12 LOAD_NAME 0 (fun) 15 CALL_FUNCTION 0 (0 positional, 0 keyword pair) 18 POP_TOP 19 LOAD_CONST 2 (None) 22 RETURN_VALUE Disassembly of <code object fun at 0x7f7533bb1ed0, file "example_for1.py", line 1>: 2 0 LOAD_CONST 1 (123) 3 STORE_FAST 0 (x) 3 6 SETUP_LOOP 30 (to 39) 9 LOAD_GLOBAL 0 (range) 12 LOAD_CONST 2 (5) 15 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 18 GET_ITER >> 19 FOR_ITER 16 (to 38) 22 STORE_FAST 1 (x.1) 4 25 LOAD_GLOBAL 1 (print) 28 LOAD_FAST 1 (x.1) 31 CALL_FUNCTION 1 (1 positional, 0 keyword pair) 34 POP_TOP 35 JUMP_ABSOLUTE 19 >> 38 POP_BLOCK 5 >> 39 LOAD_GLOBAL 1 (print) 42 LOAD_CONST 3 ('old x:') 45 LOAD_FAST 0 (x) 48 CALL_FUNCTION 2 (2 positional, 0 keyword pair) 51 POP_TOP 52 LOAD_CONST 0 (None) 55 RETURN_VALUE 0 1 2 3 4 old x: 123 -- Best regards, Paul mailto:pmiscml@gmail.com

29.11.20 18:27, Paul Sokolovsky пише:
Here's example of it in action:
$ cat example_for1.py def fun(): x = 123 for x in range(5): print(x) print("old x:", x)
fun()
I am strong -1. 1. It will break existing code. Including a lot of code written by me. 2. Shadowing local variables considered bad practice in other programming languages, and even forbidden is some of them. So why implement a feature considered harmful?

Hello, On Sun, 29 Nov 2020 22:33:28 +0200 Serhiy Storchaka <storchaka@gmail.com> wrote:
29.11.20 18:27, Paul Sokolovsky пише:
Here's example of it in action:
$ cat example_for1.py def fun(): x = 123 for x in range(5): print(x) print("old x:", x)
fun()
I am strong -1.
1. It will break existing code. Including a lot of code written by me.
I see, you can't really post anything without including 10-20KB of previous discussion history. Linking to it doesn't work either. So let's go over it again: 1. You are not supposed to be using this in "code written by you". 2. This is a demonstration that adding block-level scope to Python is easy enough, nothing else. 3. If this ever be implemented "in production", it will be wrapped in dedicated syntax like "for let" or "for const".
2. Shadowing local variables considered bad practice in other programming languages, and even forbidden is some of them. So why implement a feature considered harmful?
"By whom?" Computational theory doesn't care about superstitions. Neither it really cares about names (humans do). It only cares about where scope for a particular binding starts and ends: def fun1(): x = 1 def fun2(): x = 2 # Horror! x is shadowed! -- Best regards, Paul mailto:pmiscml@gmail.com
participants (2)
-
Paul Sokolovsky
-
Serhiy Storchaka