Visibility scope for "for/while/if" statements
Hello, Don't want to be importunate annoyingly asking the things probably trivial for experienced community, but need to ask it anyway, after spending about two hours trying to find well-camouflaged error caused by it. Why the variables defined inside "for"/"while"/"if" statements (including loop variables for "for") are visible outside this scope? This litters the namespace, makes things more complex and requires even more attention from the programmer than even the uneasy C. Or makes him to watch for using each variable only once (prohibiting from using usual i/j/k variables); otherwise copypasting a piece of code can cause hardly-findable errors. But not even the variables from inside of statements are forgot in the outside namespace; what is most ridiculous is that the inner variables from a list comprehension are kept outside the comprehension! Yes, I am aware of one use case for this... er, feature. It could be useful to remember the last item of the loop after the loop is done... sometimes. But what is it for in other cases, except confusing the programmer? Or maybe can someone hint me whether I can somehow switch the behaviour on source-level to avoid keeping the variables outside the statements? Something like Perlish "import strict"? I couldn't find it myself. While global change of Python to the variables local to statements and list comprehension could definitely affect too much programs, adding it on a per-source level would keep the compatibility while making the life of Python programmers safer. Thanks in advance. Appendix 1: examples. --------cut here-------- #!/usr/bin/env python -t a1 = ['a', 'b', 'c', 'd', 'e'] # Loop variable is alive outside the loop for k in a1: pass print "k: %s" % k # Loop variable is alive even outside the list comprehension! b1 = [l for l in a1] print "l: %s" % l # If there are several loops in a comprehension, # variables from all of them are kept intact c1 = [(m, n) for m in a1 for n in b1] print "m: %s, n: %s" % (m, n) # Loop variable is kept even from nested comprehensions d1 = [o for o in [p for p in a1]] print "o: %s, p: %s" % (o, p) # And the winner is... for r in a1: print "r0: %s, " % r, e1 = [r for r in b1] # Now try to access the "r" variable from the loop! print "r1: %s" % r --------cut here-------- -- With best regards, Alexander mailto:maa_public@sinn.ru
Alexander Myodov <maa_public@sinn.ru> wrote:
In the future you should test your assumptions before writing software with them.
Why the variables defined inside "for"/"while"/"if" statements (including loop variables for "for") are visible outside this scope?
if and while statements don't define variables, so they can't expose them later in the scope. In regards to 'for' loops, they have always done that, there is code that relies on that behavior, and changing behavior would unnecessarily break code. As for list comprehensions, they were literally meant to be a completely equivalent translation of a set of for loops. That is: x = [] for i in foo: if f(i): x.append(i) could be translated into x = [i for i in foo if f(i)] and one would get the exact same side effects, including the 'leaking' of the most recently bound i into the local scope. The leakage was not accidental.
There is a great reason: there generally exists two namespaces in Python, locals and globals (you can get a representation of of locals by using locals() (your changes to the dictionary won't change the local scope), and get a reference to the globals dictionary by globals()...there is also __builtins__, but you shouldn't be playing with that unless you know what you are doing). These namespaces offer you access to other namespaces (class, module, etc.). In most cases (the body of a function or method), the local scope is defined as an array with offset lookups:
This results in the bytecode to be executed being significantly faster than if it were to reference a dictionary (like globals).
Since the LOAD_NAME opcode does a dictionary lookup, it is necessarily slower than an array + offset lookup. Further, you should clarify what you would want this mythical non-leaky for loop to do in various cases. What would happen in the following case? i = None for i in ...: ... print i ... assuming that the loop executed more than once? Would you always get 'None' printed, or would you get the content of the last variable? What about: x = None for i in ...: x = f(i) ... print x Would 'x' be kept after the loop finished executing? I would imagine in your current code you are running something like this: i = #some important value ... #many lines of code for i in ...: ... #you access the 'i' you bound a long time ago In this case, you are making a common new programmer mistake by using the same variable name for two disparate concepts. If the 'i' that was initially bound was important through the lifetime of the scope, you should have named it differently than the 'i' that was merely a loop variable. I will also point out that in general, not leaking/exposing/etc. such variables would necessarily slow down Python. Why? Because it would necessitate nesting the concept of pseudo-namespaces. Now, when a function is compiled, nearly every local name is known, and they can be made fast (in the Python sense). If one were to nest pseudo namespaces, one would necessarily have another namespace lookup for every nested for loop. More specifically, accessing 'foo' in these three different nestings would take different amounts of time. for i in xrange(10): x = foo for i in xrange(10): for j in xrange(10): x = foo for i in xrange(10): for j in xrange(10): for k in xrange(10): x = foo And in the case that you do or don't want 'x' to 'leak' into the surrounding scope, you either take the speed hit again, or break quite a bit of code and be inconsistant.
Or maybe can someone hint me whether I can somehow switch the behaviour on source-level to avoid keeping the variables outside the statements?
No.
Python semantics seem to have been following the rule of "we are all adults here". If your assumptions have caused you bugs, then you should realize that your assumptions should have been tested before they were relied upon. That is what the interactive Python console is for. Generally though, Python follows a common C semantic of variable leakage. C Code: int i; // required in some versions of C for (i=0;i<10;i++) { ... } Equivalent Python: for i in xrange(10): ... As long as those loops don't have 'break' (or goto in the case of C) in them, Python and C will have the same value buond to 'i' after the loop. Again: test your assumptions. If your untested assumptions are wrong, don't complain that the language is broken. Also: python-dev is a mailing list for the development /of/ Python. Being that your questions as of late have been in the realm of "why does or doesn't Python do this?", you should go to python-list (or the equivalent comp.lang.python newsgroup) for answers to questions regarding current Python behavior, and why Python did or didn't do something in its past. - Josiah
Josiah Carlson wrote:
As for list comprehensions, they were literally meant to be a completely equivalent translation of a set of for loops.
I don't think that's quite true. I doubt whether anyone really thought about the issue when LCs were first being discussed. I didn't, but if I had, I wouldn't have considered the variable-leaking as being something that it was necessary to preserve, because the only use case for it is something you can't do with an LC anyway. The reasons for the variable-leaking being preserved are (1) it fell out of the implementation and (2) it makes the documentation slightly simpler, since LCs can be described fully and accurately in terms of translation to for-loops. Whether those are *good* reasons or not is debatable. In Py3k it's possible that this will be resolved by making for-loop variables local to the loop as well. Or maybe not. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg.ewing@canterbury.ac.nz +--------------------------------------+
On 9/22/05, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
It was the cheapest implementation; it was known at the time that it wasn't ideal. I'd like to fix this in 3.0, so that LC's are just a special case of GE's.
In Py3k it's possible that this will be resolved by making for-loop variables local to the loop as well. Or maybe not.
Definitely not. There are lots of uses for the loop control variable after break. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
"Alexander Myodov" <maa_public@sinn.ru> wrote in message news:1477206818.20050922160515@sinn.ru...
Why the variables defined inside "for"/"while"/"if" statements (including loop variables for "for") are visible outside this scope?
Questions about why Python is the way it is belong on comp.lang.python, the general Python mailing list, or gmane.comp.python.general (they are all gatewayed to each other). Moreover, they are likely to have been asked, answered, and discussed previously, with the answers possibly discoverable thru Google search of the c.l.p archives. This one certainly has been.
participants (5)
-
Alexander Myodov -
Greg Ewing -
Guido van Rossum -
Josiah Carlson -
Terry Reedy