Hi. Writing nested scopes support for jython (now it passes test_scope and test_future <wink>), I have come across these further corner cases for nested scopes mixed with global decl, I have tried them with python 2.1b1 and I wonder if the results are consistent with the proposed rule: a free variable is bound according to the nearest outer scope binding (assign-like or global decl), class scopes (for backw-comp) are ignored wrt this. (I) from __future__ import nested_scopes x='top' def ta(): global x def tata(): exec "x=1" in locals() return x # LOAD_NAME return tata print ta()() prints 1, I believed it should print 'top' and a LOAD_GLOBAL should have been produced. In this case the global binding is somehow ignored. Note: putting a global decl in tata xor removing the exec make tata deliver 'top' as I expected (LOAD_GLOBALs are emitted). Is this a bug or I'm missing something? (II) from __future__ import nested_scopes x='top' def ta(): x='ta' class A: global x def tata(self): return x # LOAD_GLOBAL return A print ta()().tata() # -> 'top' should not the global decl in class scope be ignored and so x be bound to x in ta, resulting in 'ta' as output? If one substitutes global x with x='A' that's what happens. Or only local binding in class scope should be ignored but global decl not? regards, Samuele Pedroni
[Samuele Pedroni]
... I have tried them with python 2.1b1 and I wonder if the results are consistent with the proposed rule: a free variable is bound according to the nearest outer scope binding (assign-like or global decl), class scopes (for backw-comp) are ignored wrt this.
"exec" and "import*" always complicate everything, though.
(I) from __future__ import nested_scopes
x='top' def ta(): global x def tata(): exec "x=1" in locals() return x # LOAD_NAME return tata
print ta()() prints 1, I believed it should print 'top' and a LOAD_GLOBAL should have been produced.
I doubt this will change. In the presence of exec, the compiler has no idea what's local anymore, so it deliberately generates LOAD_NAME. When Guido says he intends to "deprecate" exec-without-in, he should also always say "and also deprecate exec in locals()/global() too". But he'll have to think about that and get back to you <wink>. Note that modifications to locals() already have undefined behavior (according to the Ref Man), so exec-in-locals() is undefined too if the exec'ed code tries to (re)bind any names.
In this case the global binding is somehow ignored. Note: putting a global decl in tata xor removing the exec make tata deliver 'top' as I expected (LOAD_GLOBALs are emitted). Is this a bug or I'm missing something?
It's an accident either way (IMO), so it's a bug either way too -- or a feature either way. It's basically senseless! What you're missing is the layers of hackery in support of exec even before 2.1; this "give up on static identification of locals entirely in the presence of exec" goes back many years.
(II) from __future__ import nested_scopes
x='top' def ta(): x='ta' class A: global x def tata(self): return x # LOAD_GLOBAL return A
print ta()().tata() # -> 'top'
should not the global decl in class scope be ignored and so x be bound to x in ta, resulting in 'ta' as output?
Yes, this one is clearly a bug. Good catch!
Hi. [Tim Peters on from __future__ import nested_scopes x='top' def ta(): global x def tata(): exec "x=1" in locals() return x # LOAD_NAME vs LOAD_GLOBAL? return tata print ta()() # 1 vs. 'top' ? ] -- snip --
It's an accident either way (IMO), so it's a bug either way too -- or a feature either way. It's basically senseless! What you're missing is the layers of hackery in support of exec even before 2.1; this "give up on static identification of locals entirely in the presence of exec" goes back many years. (Just a joke) I'm not such a "newbie" that the guess I'm missing something is right with probability > .5. At least I hope so. The same hackery is there in jython codebase and I have taken much care in preserving it <wink>.
The point is simply that 'exec in locals()' is like a bare exec but it has been decided to allow 'exec in' even in presence of nested scopes and we cannot detect the 'locals()' special case (at compile time) because in python 'locals' is the builtin only with high probability. So we face the problem, how to *implement* an undefined behaviour, (the ref says that changing locals is undef,: everybody knows) that historically has never been to seg fault, in the new (nested scopes) context? It also true that what we are doing is "impossible", that's why it has been decided to raise a SyntaxError in the bare exec case <wink>. To be honest, I have just implemented things in jython my/some way, and then discovered that jython CVS version and python 21.b1 (here) behave differently. A posteriori I just tried to solve/explain things using the old problem pattern: I give you a (number) sequence, guess the next term: the sequence is: (over this jython and python agree) from __future__ import nested_scopes def a(): exec "x=1" in locals() return x # LOAD_NAME (jython does the equivalent) def b(): global x exec "x=1" in locals() return x # LOAD_GLOBAL def c(): global x def cc(): return x # LOAD_GLOBAL return cc def d(): x='d' def dd(): exec "x=1" in locals() # without 'in locals()' => SynError return x # LOAD_DEREF (x in d) return dd and then the term to guess: def z(): global x def zz(): exec "x=1" in locals() # without 'in locals()' => SynError return x # ???? python guesses LOAD_NAME, jython the equiv of LOAD_GLOBAL return zz Should python and jython agree here too? Anybody wants to spend some time convincing me that I should change jython meaning of undefined? I will not spend more time to do the converse <wink>. regards, Samuele Pedroni. PS: It is also possible that trying to solve pdb+nested scopes problem we will have to consider the grab the locals problem with more care.
When Guido says he intends to "deprecate" exec-without-in, he should also always say "and also deprecate exec in locals()/global() too". But he'll have to think about that and get back to you <wink>.
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
Note that modifications to locals() already have undefined behavior (according to the Ref Man), so exec-in-locals() is undefined too if the exec'ed code tries to (re)bind any names.
And that's the basis for deprecating it. --Guido van Rossum (home page: http://www.python.org/~guido/)
Hi. [GvR]
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
That's fine for me. Will that deprecation be already active with 2.1, e.g having locals() and param-less vars() raise a warning. I imagine a (new) function that produce a snap-shot of the values in the local,free and cell vars of a scope can do the job required for simple debugging (the copy will not allow to modify back the values), or another approach... regards, Samuele Pedroni
Samuele Pedroni
I imagine a (new) function that produce a snap-shot of the values in the local,free and cell vars of a scope can do the job required for simple debugging (the copy will not allow to modify back the values)
Modifying the values doesn't cause any problem, only adding new names to the scope. So locals() or whatever replaces it could return a mapping object that doesn't allow adding any keys. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
Hi. [GvR]
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
That's fine for me. Will that deprecation be already active with 2.1, e.g having locals() and param-less vars() raise a warning. I imagine a (new) function that produce a snap-shot of the values in the local,free and cell vars of a scope can do the job required for simple debugging (the copy will not allow to modify back the values), or another approach... In the meantime (if there's a meantime) is ok for jython to behave the way I have explained or not? wrt to exec+locals()+global+nested scopes . regards, Samuele Pedroni
[GvR]
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
[Samuele]
That's fine for me. Will that deprecation be already active with 2.1, e.g having locals() and param-less vars() raise a warning.
Hm, I hadn't thought of doing it right now.
I imagine a (new) function that produce a snap-shot of the values in the local,free and cell vars of a scope can do the job required for simple debugging (the copy will not allow to modify back the values), or another approach...
Maybe. I see two solutions: a function that returns a copy, or a function that returns a "lazy mapping". The former could be done as follows given two scopes: def namespace(): d = __builtin__.__dict__.copy() d.update(globals()) d.update(locals()) return d The latter like this: def namespace(): class C: def __init__(self, g, l): self.__g = g self.__l = l def __getitem__(self, key): try: return self.__l[key] except KeyError: try: return self.__g[key] except KeyError: return __builtin__.__dict__[key] return C(globals(), locals()) But of course they would have to work harder to deal with nested scopes and cells etc. I'm not sure if we should add this to 2.1 (if only because it's more work than I'd like to put in this late in the game) and then I'm not sure if we should deprecate locals() yet.
In the meantime (if there's a meantime) is ok for jython to behave the way I have explained or not? wrt to exec+locals()+global+nested scopes .
Sure. You may even document it as one of the known differences. --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido wrote:
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
Samuele Pedroni writes:
That's fine for me. Will that deprecation be already active with 2.1, e.g having locals() and param-less vars() raise a warning. I imagine a (new) function that produce a snap-shot of the values in the local,free and cell vars of a scope can do the job required for simple debugging (the copy will not allow to modify back the values), or another approach...
Nothing has happened on this front yet. Should I add deprecation notes to the docummentation while Guido is on vacation, or wait to ask him when he gets back? Or was this matter resolved when I wasn't paying attention? -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Digital Creations
[Guido]
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
[Fred L. Drake, Jr.]
Nothing has happened on this front yet. Should I add deprecation notes to the docummentation while Guido is on vacation, or wait to ask him when he gets back? Or was this matter resolved when I wasn't paying attention?
I advise continuing to ignore it. Nothing was resolved, and to judge from a trial balloon I floated on c.l.py at the time, it's not a deprecation that will be greeted with enthusiasm. The problems range from people doing def f(...): ... print "..." % locals() to people mutating locals() at module level because they simply don't understand that globals() is the same (but correct) thing to use there. Due to the first example, and as Samuele may <wink> have already suggested, we at least need to implement a mapping object capturing name bindings before we can even think about deprecating locals() for real.
Samuele Pedroni writes:
I imagine a (new) function that produce a snap-shot of the values in the local,free and cell vars of a scope can do the job required for simple debugging
I think there should be methods operating directly on stack frames for debuggers to use. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
[Guido]
Actually, I intend to deprecate locals(). For now, globals() are fine. I also intend to deprecate vars(), at least in the form that is equivalent to locals().
OK by me. Note that we agreed long ago that if nested scopes ever made it in, we would need to supply a way to get a "namespace mapping" object so that stuff like: print "The value of i is %(i)s and j %(j)s" % locals() could be replaced by: print "The value of i is %(i)s and j %(j)s" % namespace_map_object() Also agreed this need not be a dict; fine by me if it's immutable too.
On Sun, 11 Mar 2001, Tim Peters wrote:
OK by me. Note that we agreed long ago that if nested scopes ever made it in, we would need to supply a way to get a "namespace mapping" object so that stuff like:
print "The value of i is %(i)s and j %(j)s" % locals()
could be replaced by:
print "The value of i is %(i)s and j %(j)s" % namespace_map_object()
I remarked to Jeremy at Python 9 that, given that we have new variable lookup rules, there should be an API to perform this lookup. I suggested that a new method on frame objects would be a good idea, and Jeremy & Barry seemed to agree. I was originally thinking of frame.lookup('whatever'), but if that method happens to be tp_getitem, then i suppose print "i is %(i)s and j is %(j)s" % sys.getframe() would work. We could call it something else, but one way or another it's clear to me that this object has to follow lookup rules that are completely consistent with whatever kind of scoping is in effect (i.e. throw out *both* globals() and locals() and provide one function that looks up the whole set of visible names, rather than just one scope's contents). -- ?!ng "Computers are useless. They can only give you answers." -- Pablo Picasso
participants (6)
-
Fred L. Drake, Jr. -
Greg Ewing -
Guido van Rossum -
Ka-Ping Yee -
Samuele Pedroni -
Tim Peters