Is outlawing-nested-import-* only an implementation issue?

Hi all -- i've been reading the enormous thread on nested scopes with some concern, since i would very much like Python to support "proper" lexical scoping, yet i also care about compatibility. There is something missing from my understanding here: - The model is, each environment has a pointer to the enclosing environment, right? - Whenever you can't find what you're looking for, you go up to the next level and keep looking, right? - So what's the issue with not being able to determine which variable binds in which scope? With the model just described, it's perfectly clear. Is all this breakage only caused by the particular optimizations for lookup in the implementation (fast locals, etc.)? Or have i missed something obvious? I could probably go examine the source code of the nested scoping changes to find the answer to my own question, but in case others share this confusion with me, i thought it would be worth asking. * * * Consider for a moment the following simple model of lookup: 1. A scope maps names to objects. 2. Each scope except the topmost also points to a parent scope. 3. To look up a name, first ask the current scope. 4. When lookup fails, go up to the parent scope and keep looking. I believe the above rules are common among many languages and are commonly understood. The only Python-specific parts are then: 5. The current scope is determined by the nearest enclosing 'def'. 6. These statements put a binding into the current scope: assignment (=), def, class, for, except, import And that's all. * * * Given this model, all of the scoping questions that have been raised have completely clear answers: Example I >>> y = 3 >>> def f(): ... print y ... >>> f() 3 Example II >>> y = 3 >>> def f(): ... print y ... y = 1 ... print y ... >>> f() 3 1 >>> y 3 Example III >>> y = 3 >>> def f(): ... exec "y = 2" ... def g(): ... return y ... return g() ... >>> f() 2 Example IV >>> m = open('foo.py', 'w') >>> m.write('x = 1') >>> m.close() >>> def f(): ... x = 3 ... from foo import * ... def g(): ... print x ... g() ... >>> f() 1 In Example II, the model addresses even the current situation that sometimes surprises new users of Python. Examples III and IV are the current issues of contention about nested scopes. * * * It's good to start with a simple model for the user to understand; the implementation can then do funky optimizations under the covers so long as the model is preserved. So for example, if the compiler sees that there is no "import *" or "exec" in a particular scope it can short-circuit the lookup of local variables using fast locals. But the ability of the compiler to make this optimization should only affect performance, not affect the Python language model. The model described above is the approximately the one available in Scheme. It exactly reflects the environment-diagram model of scoping as taught to most Scheme students and i would argue that it is the easiest to explain. Some implementations of Scheme, such as STk, do what is described above. UMB scheme does what Python does now: the use-before-binding of 'y' in Example II would cause an error. I was surprised that these gave different behaviours; it turns out that the Scheme standard actually forbids the use of internal defines not at the beginning of a function body, thus sidestepping the issue. But we can't do this in Python; assignment must be allowed anywhere. Given that internal assignment has to have some meaning, the above meaning makes the most sense to me. -- ?!ng

On Thu, 22 Feb 2001, Ka-Ping Yee wrote:
That was poorly phrased. To clarify, i am making the assumption that the compiler wants each name to be associated with exactly one scope per block in which it appears. 1. Is the assumption true? 2. If so, is this constraint motivated only by lookup optimization? 3. Why enforce this constraint when it would be inconsistent with behaviour that we already have at the top level? If foo.py contains "x = 1", then this works at the top level: >>> if 1: # top level ... x = 3 ... print x ... from foo import * ... def g(): print x ... g() ... 3 1 I am suggesting that it should do exactly the same thing in a function: >>> def f(): # x = 3 inside, no g() ... x = 3 ... print x ... from foo import * ... print x ... >>> f() 3 1 >>> def f(): # x = 3 inside, nested g() ... x = 3 ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 >>> x = 3 >>> def f(): # x = 3 outside, nested g() ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 (Replacing "from foo import *" above with "x = 1" or "exec('x = 1')" should make no difference. So this isn't just about internal-import-* and exec-without-in, even if we do eventually deprecate internal-import-* and exec-without-in -- which i would tend to support.) Here is a summary of the behaviour i observe and propose. 1.5.2 2.1a1 suggested top level from foo import * 3,1 3,1 3,1 exec('x = 1') 3,1 3,1 3,1 x = 1 3,1 3,1 3,1 x = 3 outside, no g() from foo import * 3,1 3,1 3,1 exec('x = 1') 3,1 3,1 3,1 x = 1 x UnboundLocal 3,1 x = 3 inside, no g() from foo import * 3,1 3,1 3,1 exec('x = 1') 3,1 3,1 3,1 x = 1 x UnboundLocal 3,1 x = 3 outside, nested g() from foo import * 3,3 SyntaxError 3,1 exec('x = 1') 3,3 SyntaxError 3,1 x = 1 x UnboundLocal 3,1 x = 3 inside, nested g() from foo import * 3,x SyntaxError 3,1 exec('x = 1') 3,x SyntaxError 3,1 x = 1 3,x 3,1 3,1 (I don't know what the heck is going on in Python 1.5.2 in the cases where it prints 'x'.) My postulates are: 1. "exec('x = 1')" should behave exactly the same as "x = 1" 2. "from foo import *" should do the same as "x = 1" 3. "def g(): print x" should behave the same as "print x" The testing script is attached. -- ?!ng

I think the issue that you didn't address is that lexical scoping is a compile-time issue, and that in most languages that variable names that a program uses are a static property of the code. Off the top of my head, I can't think of another lexically scoped language that allows an exec or eval to create a new variable binding that can later be used via a plain-old reference. One of the reasons I am strongly in favor of making import * and exec errors is that it stymies the efforts of a reader to understand the code. Lexical scoping is fairly clear because you can figure out what binding a reference will use by reading the text. (As opposed to dynamic scoping where you have to think about all the possible call stacks in which the function might end up.) With bare exec and import *, the reader of the code doesn't have any obvious indicator of what names are being bound. This is why I consider it bad form and presumably part of the reason that the language references outlaws it. (But not at the module scope, since practicality beats purity.) If we look at your examples: >>> def f(): # x = 3 inside, no g() ... x = 3 ... print x ... from foo import * ... print x ... >>> f() 3 1 >>> def f(): # x = 3 inside, nested g() ... x = 3 ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 >>> x = 3 >>> def f(): # x = 3 outside, nested g() ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 In these examples, it isn't at all obvious to the reader of the code whether the module foo contains a binding for x or whether the programmer intended to import that name and stomp on the exist definition. Another key difference between Scheme and Python is that in Scheme, each binding operation creates a new scope. The Scheme equivalent of this Python code -- def f(x): y = x + 1 ... y = x + 2 ... -- would presumably be something like this -- (define (f x) (let ((y (+ x 1))) ... (let (y (+ x 2))) ... )) Python is a whole different beast because it supports multiple assignments to a name within a single scope. In Scheme, every binding of a name via lambda introduces a new scope. This is the reason that the example -- x = 3 def f(): print x x = 2 print x -- raises an error rather than printing '3\n2\n'. Jeremy

On Thu, 22 Feb 2001, Jeremy Hylton wrote:
I tried STk Scheme, guile, and elisp, and they all do this.
Yes, i look forward to the day when no one will ever use import-* any more. I can see good reasons to discourage the use of import-* and bare-exec in general anywhere. But as long as they *do* have a meaning, they had better mean the same thing at the top level as internally.
It's perfectly clear -- since we expect the reader to understand what happens when we do exactly the same thing at the top level.
Another key difference between Scheme and Python is that in Scheme, each binding operation creates a new scope.
Scheme separates 'define' and 'set!', while Python only has '='. In Scheme, multiple defines rebind variables: (define a 1) (define a 2) (define a 3) just as in Python, multiple assignments rebind variables: a = 1 a = 2 a = 3 The lack of 'set!' prevents Python from rebinding variables outside of the local scope, but it doesn't prevent Python from being otherwise consistent and having "a = 2" do the same thing inside or outside of a function: it binds a name in the current scope. -- ?!ng "The only `intuitive' interface is the nipple. After that, it's all learned." -- Bruce Ediger, on user interfaces

"KPY" == Ka-Ping Yee <ping@lfw.org> writes:
KPY> On Thu, 22 Feb 2001, Jeremy Hylton wrote:
KPY> I tried STk Scheme, guile, and elisp, and they all do this. I guess I'm just dense then. Can you show me an example? The only way to introduce a new name in Scheme is to use lambda or define which can always be translated into an equivalent letrec. The name binding is then visible only inside the body of the lambda. As a result, I don't see how eval can introduce a new name into a scope. The Python example I was thinking of is: def f(): exec "y=2" return y
f() 2
What would the Scheme equivalent be? The closest analog I can think of is (define (f) (eval "(define y 2)") y) The result here is undefined because y is not bound in the body of f, regardless of the eval. Jeremy

On Fri, 23 Feb 2001, Jeremy Hylton wrote:
I haven't had time in a while to follow up on this thread, but i just wanted to say that i think this is a reasonable and sane course of action. I see the flaws in the model i was advocating, and i'm sorry for consuming all that time in the discussion. -- ?!ng Post Scriptum: On Fri, 23 Feb 2001, Jeremy Hylton wrote:
KPY> I tried STk Scheme, guile, and elisp, and they all do this.
I guess I'm just dense then. Can you show me an example?
The example is pretty much exactly what you wrote: (define (f) (eval '(define y 2)) y) It produced 2. But several sources have confirmed that this is just bad implementation behaviour, so i'm willing to consider that a red herring. Believe it or not, in some Schemes, the following actually happens! STk> (define x 1) x STk> (define (func flag) (if flag (define x 2)) (lambda () (set! x 3))) func STk> ((func #t)) STk> x 1 STk> ((func #f)) STk> x 3 More than one professor that i showed the above to screamed.

"KPY" == Ka-Ping Yee <ping@lfw.org> writes:
Another key difference between Scheme and Python is that in Scheme, each binding operation creates a new scope.
KPY> Scheme separates 'define' and 'set!', while Python only has KPY> '='. In Scheme, multiple defines rebind variables: Really, scheme provides lambda, the let family, define, and set!, where "define" is defined in terms of letrec except at the top level. KPY> (define a 1) KPY> (define a 2) KPY> (define a 3) Scheme distinguishes between top-level definitions and internal defintions. They have different semantics. Since we're talking about what happens inside Python functions, we should only look at what define does for internal definitions. An internal defintion is only allowed at the beginning of a body, so you're example above is equivalent to: (letrec ((a 1) (a 2) (a 3)) ...) But it is an error to have duplicate name bindings in a letrec. At least it is in MzScheme. Not sure what R5RS says about this. KPY> just as in Python, multiple assignments rebind variables: KPY> a = 1 KPY> a = 2 KPY> a = 3 Python's assignment is closer to set!, since it can occur anywhere in a body not just at the beginning. But if we say that = is equivalent to set! we've got a problem, because you can't use set! on an unbound variable. I think that leaves us with two alternatives. As I mentioned in my previous message, one is to think about each assignment in Python introducing a new scope. a = 1 (let ((a 1)) a = 2 (let ((a 2)) a = 3 (let ((a 3)) ....))) or def f(): (define (f) print a (print a) a = 2 (let ((a 2)) ...)) But I don't think it's clear to read a group of equally indented statements as a series of successively nested scopes. The other alternative is to say that = is closer to set! and that the original name binding is implicit. That is: "If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the local namespace." (ref manual, sec. 4) KPY> The lack of 'set!' prevents Python from rebinding variables KPY> outside of the local scope, but it doesn't prevent Python from KPY> being otherwise consistent and having "a = 2" do the same thing KPY> inside or outside of a function: it binds a name in the current KPY> scope. Again, if we look at Scheme as an example and compare = and define, define behaves differently at the top-level than it does inside a lambda. Jeremy

1. "exec('x = 1')" should behave exactly the same as "x = 1"
Sorry, no go. This just isn't a useful feature.
2. "from foo import *" should do the same as "x = 1"
But it is limiting because it hides information from the compiler, and hence it is outlawed when it gets in the way of the compiler.
3. "def g(): print x" should behave the same as "print x"
Huh? again. Defining a function does't call it. Python has always adhered to the principle that the context where a function is defined determines its context, not where it is called. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, 22 Feb 2001, Guido van Rossum wrote:
1. "exec('x = 1')" should behave exactly the same as "x = 1"
Sorry, no go. This just isn't a useful feature.
It's not a "feature" as in "something to be added to the language". It's a consistent definition of "exec" that simplifies understanding. Without it, how do you explain what "exec" does?
Again, consistency simplifies understanding. What it "gets in the way of" is a particular optimization; it doesn't make compilation impossible. The language reference says that import binds a name in the local namespace. That means "import x" has to do the same thing as "x = 1" for some value of 1. "from foo import *" binds several names in the local scope, and so if x is bound in module foo, it should do the same thing as "x = 1" for some value of 1. When "from foo import *" makes it impossible to know at compile-time what bindings will be added to the current scope, we just do normal name lookup for that scope. No big deal. It already works that way at module scope; why should this be any different? With this simplification, there can be a single scope chain: builtins <- module <- function <- nested-function <- ... and all scopes can be treated the same. The implementation could probably be both simpler and faster! Simpler, because we don't have to have separate cases for builtins, local, and global; and faster, because some of the optimizations we currently do for locals could be made to apply at all levels. Imagine "fast globals"! And imagine getting them essentially for free.
3. "def g(): print x" should behave the same as "print x"
Huh? again. Defining a function does't call it.
Duh, obviously i meant 3. "def g(): print x" immediately followed by "g()" should behave the same as "print x" Do you agree with this principle, at least?
Absolutely agreed. I've never intended to contradict this. This is the foundation of lexical scoping. -- ?!ng "Don't worry about people stealing an idea. If it's original, you'll have to jam it down their throats." -- Howard Aiken

[Ka-Ping Yee]
The conceptual model, yes, but the implementation isn't like that.
- Whenever you can't find what you're looking for, you go up to the next level and keep looking, right?
Conceptually, yes. No such looping search occurs at runtime, though.
- So what's the issue with not being able to determine which variable binds in which scope?
That determination is done at compile-time, not runtime. In the presence of "exec" and "import *" in some contexts, compile-time determination is stymied and there is no runtime support for a "slow" lookup. Note that the restrictions are *not* against lexical nesting, they're against particular uses of "exec" and "import *" (the latter of which is so muddy the Ref Man said it was undefined a long, long time ago).
Even locals used to be resolved by dict searches. The entire model there wasn't preserved by the old switch to fast locals either. For example,
IIRC, in the old days that would print 42. Who cares <0.1 wink>? This is nonsense either way. There are tradeoffs here among: conceptual clarity runtime efficiency implementation complexity rate of cyclic garbage creation Your message favors "conceptual clarity" over all else; the implementation doesn't. Python also limits strings to the size of a platform int <0.9 wink>.
But note that eval() didn't make it into the Scheme std: they couldn't agree on its semantics or implementation. eval() is *suggested* in the fifth Revised Report, but there has no access to its lexical environment; instead it acts "as if" its argument had appeared at top level "or in some other implementation-dependent environment" (Dybvig; "The Scheme Programming Language"). Dybvig gives an example of one of the competing Scheme eval() proposals gaining access to a local vrbl via using macros to interpolate the local's value into the argument's body before calling eval(). And that's where refusing to compromise leads. utterly-correct-and-virtually-useless-ly y'rs - tim

On Thu, 22 Feb 2001, Tim Peters wrote:
Would the existence of said runtime support hurt anybody? Don't we already do slow lookup in some situations anyway?
(To want to *take away* the ability to do import-* at all, in order to protect programmers from their own bad habits, is a different argument. I think we all already agree that it's bad form. But the recent clamour has shown that we can't take it away just yet.)
Yes, i do think conceptual clarity is important. The way Python leans towards conceptual simplicity is a big part of its success, i believe. The less there is for someone to fit into their brain, the less time they can spend worrying about how the language will behave and the more they can focus on getting the job done. And i don't think we have to sacrifice much of the others to do it. In fact, often conceptual clarity leads to a simpler implementation, and sometimes even a faster implementation. Now i haven't actually done the implementation so i can't tell you whether it will be faster, but it seems to me that it's likely to be simpler and could stand a chance of being faster. -- ?!ng "The only `intuitive' interface is the nipple. After that, it's all learned." -- Bruce Ediger, on user interfaces

Note that this is moot now -- see my previous post about how we've decided to resolve this using a magical import to enable nested scopes (in 2.1).
Actually, no.
- Whenever you can't find what you're looking for, you go up to the next level and keep looking, right?
That depends. Our model is inspired by the semantics of locals in Python 2.0 and before, and this all happens at compile time. That means that we must be able to know which names are defined in each scope at compile time.
You call it an optimization, and that's how it started. But since it clearly affects the semantics of the language, it's not really an optimization -- it's a particular semantics that lends itself to more and easy compile-time analysis and hence can be implemented more efficiently, but the corner cases are different, and the language semantics define what should happen, optimization or not. In particular: x = 1 def f(): print x x = 2 raises an UnboundLocalError error at the point of the print statement. Likewise, in the official semantics of nested scopes: x = 1 def f(): def g(): print x g() x = 2 also raises an UnboundLocalError at the print statement.
No need to go to the source -- this is all clearly explained in the PEP (http://python.sourceforge.net/peps/pep-0227.html).
Actually, most languages do all this at compile time. Very early Python versions did do all this at run time, but by the time 1.0 was released, the "locals are locals" rule was firmly in place. You may like the purely dynamic version better, but it's been outlawed long ago.
The only Python-specific parts are then:
5. The current scope is determined by the nearest enclosing 'def'.
For most purposes, 'class' also creates a scope.
Sure.
Sure.
You didn't try this, did you? or do you intend to say that it "should" print this? In fact it raises UnboundLocalError: local variable 'y' referenced before assignment. (Before 2.0 it would raise NameError.)
Wrong again. This prints 3, both without and with nested scopes as defined in 2.1a2. However it raises an exception with the current CVS version: SyntaxError: f: exec or 'import *' makes names ambiguous in nested scope.
I didn't try this one, but I'm sure that it prints 3 in 2.1a1 and raises the same SyntaxError as above with the current CVS version.
Too late. The semantics have been bent since 1.0 or before. The flow analysis needed to optimize this in such a way that the user can't tell whether this is optimized or not is too hard for the current compiler. The fully dynamic model also allows the user to play all sorts of stupid tricks. And the unoptimized code is so much slower that it's well worth to hve the optimization.
I don't know Scheme, but isn't it supposed to be a compiled language?
I'm not sure how you can say that Scheme sidesteps the issue when you just quote an example where Scheme implementations differ?
Sorry. Sometimes, reality bites. :-) Note that I want to take more of the dynamicism out of function bodies. The reference manual has for a long time outlawed import * inside functions (but the implementation didn't enforce this). I see no good reason to allow this (it's causing a lot of work to happen each time the function is called), and the needs of being able to clearly define what happens with nested scopes make it necessary to outlaw it. I also want to eventually completely outlaw exec without an 'in' clause inside a class or function, and access to local variables through locals() or vars(). I'm not sure yet about exec without an 'in' clause at the global level, but I'm tempted to think that even there it's not much use. We'll start with warnings for some of these cases in 2.1. I see that Tim posted another rebuttal, explaining better than I do here *why* Ping's "simple" model is not good for Python, so I'll stop now. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, 22 Feb 2001, Guido van Rossum wrote:
Yes, yes. It seems like a good answer for now -- indeed, some sort of mechanism for selecting compilation options has been requested before. But we still need to eventually have a coherent answer. The chart in my other message doesn't look coherent to me -- it would take too long to explain all of the cases to someone. I deserve a smack on the head for my confusion at seeing 'x' printed out -- that happens to be the value of the NameError in 1.5.2. Here is an updated chart (updated test script is attached): 1.5.2 2.1a2 suggested toplevel with print x from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 3 1 3 1 3 1 with g() from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 3 1 3 1 3 1 x = 3 outside f() with print x from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 NameError UnboundLocal 3 1 with g() from foo import * 3 3 SyntaxError 3 1 exec('x = 1') 3 3 SyntaxError 3 1 x = 1 NameError UnboundLocal 3 1 x = 3 inside f() with print x from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 3 1 3 1 3 1 with g() from foo import * NameError SyntaxError 3 1 exec('x = 1') NameError SyntaxError 3 1 x = 1 NameError 3 1 3 1 You can see that the situation in 1.5.2 is pretty messy -- and it's precisely the inconsistent cases that have historically caused confusion. 2.1a2 is better but it still has exceptional cases -- just the cases people seem to be complaining about now.
I'm talking about the model, not the implementation. I'm advocating that we think *first* about what the programmer (the Python user) has to worry about. I think that's a Pythonic perspective, isn't it? Or are you really saying that this isn't even the model that the user should be thinking about?
Well, can we nail down what you mean by "depends"? What reasoning process should the Python programmer go through to predict the behaviour of a given program?
I've been getting the impression that people consider this a language wart (or at least a little unfortunate, as it tends to confuse people). It's a frequently asked question, and when i've had to explain it to people they usually grumble. As others have pointed out, it can be pretty surprising when the assignment happens much later in the body. I think if you asked most people what this would do, they would expect 1. Why? Because they think about programming in terms of some simple invariants, e.g.: - Editing part of a block doesn't affect the behaviour of the block up to the point where you made the change. - When you move some code into a function and then call the function, that code still works the same. This kind of backwards-action-at-a-distance breaks the first invariant. Lexical scoping is good largely because it helps preserve the second invariant (the function carries the context of where it was defined). And so on.
No need to go to the source -- this is all clearly explained in the PEP (http://python.sourceforge.net/peps/pep-0227.html).
It seems not to be that simple, because i was unable to predict what situations would be problematic without understanding how the optimizations are implemented. * * *
5. The current scope is determined by the nearest enclosing 'def'.
For most purposes, 'class' also creates a scope.
Sorry, i should have written: 5. The parent scope is determined by the nearest enclosing 'def'. * * *
I know that. I introduced these examples with "given this model..." to indicate that i'm describing what the "completely clear answers" are. The chart above tries to summarize all of the current behaviour.
I think it's better to try to bend them as little as possible -- and if it's possible to unbend them to make the language easier to understand, all the better. Since we're changing the behaviour now, this is a good opportunity to make sure the model is simple.
That's not the point. There is a scoping model that is straightforward and easy to understand, and regardless of whether the implementation is interpreted or compiled, you can easily predict what a given piece of code is going to do.
I'm not sure how you can say that Scheme sidesteps the issue when you just quote an example where Scheme implementations differ?
That's what i'm saying. The standard sidesteps (i.e. doesn't specify how to handle) the issue, so the implementations differ. I don't think we have the option of avoiding the issue; we should have a clear position on it. (And that position should be as simple to explain as we can make it.)
Let's get a complete specification of the model then. And can i ask you to clarify your position: did you put quotation marks around "simpler" because you disagree that the model i suggest is simpler and easier to understand; or did you agree that it was simpler but felt it was worth compromising that simplicity for other benefits? And if the latter, are the other benefits purely about enabling optimizations in the implementation, or other things as well? Thanks, -- ?!ng

"KPY" == Ka-Ping Yee <ping@lfw.org> writes:
No need to go to the source -- this is all clearly explained in the PEP (http://python.sourceforge.net/peps/pep-0227.html).
KPY> It seems not to be that simple, because i was unable to predict KPY> what situations would be problematic without understanding how KPY> the optimizations are implemented. The problematic cases are exactly those where name bindings are introduced implicitly, i.e. cases where an operation binds a name without the name appearing the program text for that operation. That doesn't sound like an implementation-dependent defintion. [...] KPY> That's not the point. There is a scoping model that is KPY> straightforward and easy to understand, and regardless of KPY> whether the implementation is interpreted or compiled, you can KPY> easily predict what a given piece of code is going to do. [Taking you a little out of context:] This is just what I'm advocating for import * and exec in the presence of nested fucntions. There is no easy way to predict what a piece of code is going to do without (a) knowing what names a module defines or (b) figuring out what values the argument to exec will have. On the subject of easy prediction, what should the following code do according to your model: x = 2 def f(y): ... if y > 3: x = x - 1 ... print x ... x = 3 ... I think the meaning of print x should be statically determined. That is, the programmer should be able to determine the binding environment in which x will be resolved (for print x) by inspection of the code. Jeremy

I hate to be repetitive <snort>, but forget Scheme! Scheme has nothing like "import *" or Python's flavor of eval/exec. The only guidance we'll get there is that the Scheme designers were so put off by mixing lexical scoping with eval that even *referencing* non-toplevel vars inside eval's argument isn't supported. hmm-on-second-thought-let's-pay-a-lot-of-attention-to-scheme<0.6-wink>-ly y'rs - tim

git.tr[0][1:] is ('@test', 8, 'spam', ['def spam(a, b, c, d=3, (e, (f,))=(4, (5,)), *g, **h):\n'], 0) at this point. The test expects it to be ('@test', 9, 'spam', [' eggs(b + d, c + f)\n'], 0) Test passes without -O. This was on Windows. Other platforms?

tim wrote:
the code doesn't take LINENO optimization into account. tentative patch follows: Index: Lib/inspect.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/inspect.py,v retrieving revision 1.2 diff -u -r1.2 inspect.py --- Lib/inspect.py 2001/02/28 08:26:44 1.2 +++ Lib/inspect.py 2001/02/28 22:35:49 @@ -561,19 +561,19 @@ filename = getsourcefile(frame) if context > 0: - start = frame.f_lineno - 1 - context/2 + start = _lineno(frame) - 1 - context/2 try: lines, lnum = findsource(frame) start = max(start, 1) start = min(start, len(lines) - context) lines = lines[start:start+context] - index = frame.f_lineno - 1 - start + index = _lineno(frame) - 1 - start except: lines = index = None else: lines = index = None - return (filename, frame.f_lineno, frame.f_code.co_name, lines, index) + return (filename, _lineno(frame), frame.f_code.co_name, lines, index) def getouterframes(frame, context=1): """Get a list of records for a frame and all higher (calling) frames. @@ -614,3 +614,26 @@ def trace(context=1): """Return a list of records for the stack below the current exception.""" return getinnerframes(sys.exc_traceback, context) + +def _lineno(frame): + # Coded by Marc-Andre Lemburg from the example of PyCode_Addr2Line() + # in compile.c. + # Revised version by Jim Hugunin to work with JPython too. + # Adapted for inspect.py by Fredrik Lundh + + lineno = frame.f_lineno + + c = frame.f_code + if not hasattr(c, 'co_lnotab'): + return tb.tb_lineno + + tab = c.co_lnotab + line = c.co_firstlineno + stopat = frame.f_lasti + addr = 0 + for i in range(0, len(tab), 2): + addr = addr + ord(tab[i]) + if addr > stopat: + break + line = line + ord(tab[i+1]) + return line Cheers /F

On Thu, 22 Feb 2001, Ka-Ping Yee wrote:
That was poorly phrased. To clarify, i am making the assumption that the compiler wants each name to be associated with exactly one scope per block in which it appears. 1. Is the assumption true? 2. If so, is this constraint motivated only by lookup optimization? 3. Why enforce this constraint when it would be inconsistent with behaviour that we already have at the top level? If foo.py contains "x = 1", then this works at the top level: >>> if 1: # top level ... x = 3 ... print x ... from foo import * ... def g(): print x ... g() ... 3 1 I am suggesting that it should do exactly the same thing in a function: >>> def f(): # x = 3 inside, no g() ... x = 3 ... print x ... from foo import * ... print x ... >>> f() 3 1 >>> def f(): # x = 3 inside, nested g() ... x = 3 ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 >>> x = 3 >>> def f(): # x = 3 outside, nested g() ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 (Replacing "from foo import *" above with "x = 1" or "exec('x = 1')" should make no difference. So this isn't just about internal-import-* and exec-without-in, even if we do eventually deprecate internal-import-* and exec-without-in -- which i would tend to support.) Here is a summary of the behaviour i observe and propose. 1.5.2 2.1a1 suggested top level from foo import * 3,1 3,1 3,1 exec('x = 1') 3,1 3,1 3,1 x = 1 3,1 3,1 3,1 x = 3 outside, no g() from foo import * 3,1 3,1 3,1 exec('x = 1') 3,1 3,1 3,1 x = 1 x UnboundLocal 3,1 x = 3 inside, no g() from foo import * 3,1 3,1 3,1 exec('x = 1') 3,1 3,1 3,1 x = 1 x UnboundLocal 3,1 x = 3 outside, nested g() from foo import * 3,3 SyntaxError 3,1 exec('x = 1') 3,3 SyntaxError 3,1 x = 1 x UnboundLocal 3,1 x = 3 inside, nested g() from foo import * 3,x SyntaxError 3,1 exec('x = 1') 3,x SyntaxError 3,1 x = 1 3,x 3,1 3,1 (I don't know what the heck is going on in Python 1.5.2 in the cases where it prints 'x'.) My postulates are: 1. "exec('x = 1')" should behave exactly the same as "x = 1" 2. "from foo import *" should do the same as "x = 1" 3. "def g(): print x" should behave the same as "print x" The testing script is attached. -- ?!ng

I think the issue that you didn't address is that lexical scoping is a compile-time issue, and that in most languages that variable names that a program uses are a static property of the code. Off the top of my head, I can't think of another lexically scoped language that allows an exec or eval to create a new variable binding that can later be used via a plain-old reference. One of the reasons I am strongly in favor of making import * and exec errors is that it stymies the efforts of a reader to understand the code. Lexical scoping is fairly clear because you can figure out what binding a reference will use by reading the text. (As opposed to dynamic scoping where you have to think about all the possible call stacks in which the function might end up.) With bare exec and import *, the reader of the code doesn't have any obvious indicator of what names are being bound. This is why I consider it bad form and presumably part of the reason that the language references outlaws it. (But not at the module scope, since practicality beats purity.) If we look at your examples: >>> def f(): # x = 3 inside, no g() ... x = 3 ... print x ... from foo import * ... print x ... >>> f() 3 1 >>> def f(): # x = 3 inside, nested g() ... x = 3 ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 >>> x = 3 >>> def f(): # x = 3 outside, nested g() ... print x ... from foo import * ... def g(): print x ... g() ... >>> f() 3 1 In these examples, it isn't at all obvious to the reader of the code whether the module foo contains a binding for x or whether the programmer intended to import that name and stomp on the exist definition. Another key difference between Scheme and Python is that in Scheme, each binding operation creates a new scope. The Scheme equivalent of this Python code -- def f(x): y = x + 1 ... y = x + 2 ... -- would presumably be something like this -- (define (f x) (let ((y (+ x 1))) ... (let (y (+ x 2))) ... )) Python is a whole different beast because it supports multiple assignments to a name within a single scope. In Scheme, every binding of a name via lambda introduces a new scope. This is the reason that the example -- x = 3 def f(): print x x = 2 print x -- raises an error rather than printing '3\n2\n'. Jeremy

On Thu, 22 Feb 2001, Jeremy Hylton wrote:
I tried STk Scheme, guile, and elisp, and they all do this.
Yes, i look forward to the day when no one will ever use import-* any more. I can see good reasons to discourage the use of import-* and bare-exec in general anywhere. But as long as they *do* have a meaning, they had better mean the same thing at the top level as internally.
It's perfectly clear -- since we expect the reader to understand what happens when we do exactly the same thing at the top level.
Another key difference between Scheme and Python is that in Scheme, each binding operation creates a new scope.
Scheme separates 'define' and 'set!', while Python only has '='. In Scheme, multiple defines rebind variables: (define a 1) (define a 2) (define a 3) just as in Python, multiple assignments rebind variables: a = 1 a = 2 a = 3 The lack of 'set!' prevents Python from rebinding variables outside of the local scope, but it doesn't prevent Python from being otherwise consistent and having "a = 2" do the same thing inside or outside of a function: it binds a name in the current scope. -- ?!ng "The only `intuitive' interface is the nipple. After that, it's all learned." -- Bruce Ediger, on user interfaces

"KPY" == Ka-Ping Yee <ping@lfw.org> writes:
KPY> On Thu, 22 Feb 2001, Jeremy Hylton wrote:
KPY> I tried STk Scheme, guile, and elisp, and they all do this. I guess I'm just dense then. Can you show me an example? The only way to introduce a new name in Scheme is to use lambda or define which can always be translated into an equivalent letrec. The name binding is then visible only inside the body of the lambda. As a result, I don't see how eval can introduce a new name into a scope. The Python example I was thinking of is: def f(): exec "y=2" return y
f() 2
What would the Scheme equivalent be? The closest analog I can think of is (define (f) (eval "(define y 2)") y) The result here is undefined because y is not bound in the body of f, regardless of the eval. Jeremy

On Fri, 23 Feb 2001, Jeremy Hylton wrote:
I haven't had time in a while to follow up on this thread, but i just wanted to say that i think this is a reasonable and sane course of action. I see the flaws in the model i was advocating, and i'm sorry for consuming all that time in the discussion. -- ?!ng Post Scriptum: On Fri, 23 Feb 2001, Jeremy Hylton wrote:
KPY> I tried STk Scheme, guile, and elisp, and they all do this.
I guess I'm just dense then. Can you show me an example?
The example is pretty much exactly what you wrote: (define (f) (eval '(define y 2)) y) It produced 2. But several sources have confirmed that this is just bad implementation behaviour, so i'm willing to consider that a red herring. Believe it or not, in some Schemes, the following actually happens! STk> (define x 1) x STk> (define (func flag) (if flag (define x 2)) (lambda () (set! x 3))) func STk> ((func #t)) STk> x 1 STk> ((func #f)) STk> x 3 More than one professor that i showed the above to screamed.

"KPY" == Ka-Ping Yee <ping@lfw.org> writes:
Another key difference between Scheme and Python is that in Scheme, each binding operation creates a new scope.
KPY> Scheme separates 'define' and 'set!', while Python only has KPY> '='. In Scheme, multiple defines rebind variables: Really, scheme provides lambda, the let family, define, and set!, where "define" is defined in terms of letrec except at the top level. KPY> (define a 1) KPY> (define a 2) KPY> (define a 3) Scheme distinguishes between top-level definitions and internal defintions. They have different semantics. Since we're talking about what happens inside Python functions, we should only look at what define does for internal definitions. An internal defintion is only allowed at the beginning of a body, so you're example above is equivalent to: (letrec ((a 1) (a 2) (a 3)) ...) But it is an error to have duplicate name bindings in a letrec. At least it is in MzScheme. Not sure what R5RS says about this. KPY> just as in Python, multiple assignments rebind variables: KPY> a = 1 KPY> a = 2 KPY> a = 3 Python's assignment is closer to set!, since it can occur anywhere in a body not just at the beginning. But if we say that = is equivalent to set! we've got a problem, because you can't use set! on an unbound variable. I think that leaves us with two alternatives. As I mentioned in my previous message, one is to think about each assignment in Python introducing a new scope. a = 1 (let ((a 1)) a = 2 (let ((a 2)) a = 3 (let ((a 3)) ....))) or def f(): (define (f) print a (print a) a = 2 (let ((a 2)) ...)) But I don't think it's clear to read a group of equally indented statements as a series of successively nested scopes. The other alternative is to say that = is closer to set! and that the original name binding is implicit. That is: "If a name binding operation occurs anywhere within a code block, all uses of the name within the block are treated as references to the local namespace." (ref manual, sec. 4) KPY> The lack of 'set!' prevents Python from rebinding variables KPY> outside of the local scope, but it doesn't prevent Python from KPY> being otherwise consistent and having "a = 2" do the same thing KPY> inside or outside of a function: it binds a name in the current KPY> scope. Again, if we look at Scheme as an example and compare = and define, define behaves differently at the top-level than it does inside a lambda. Jeremy

1. "exec('x = 1')" should behave exactly the same as "x = 1"
Sorry, no go. This just isn't a useful feature.
2. "from foo import *" should do the same as "x = 1"
But it is limiting because it hides information from the compiler, and hence it is outlawed when it gets in the way of the compiler.
3. "def g(): print x" should behave the same as "print x"
Huh? again. Defining a function does't call it. Python has always adhered to the principle that the context where a function is defined determines its context, not where it is called. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, 22 Feb 2001, Guido van Rossum wrote:
1. "exec('x = 1')" should behave exactly the same as "x = 1"
Sorry, no go. This just isn't a useful feature.
It's not a "feature" as in "something to be added to the language". It's a consistent definition of "exec" that simplifies understanding. Without it, how do you explain what "exec" does?
Again, consistency simplifies understanding. What it "gets in the way of" is a particular optimization; it doesn't make compilation impossible. The language reference says that import binds a name in the local namespace. That means "import x" has to do the same thing as "x = 1" for some value of 1. "from foo import *" binds several names in the local scope, and so if x is bound in module foo, it should do the same thing as "x = 1" for some value of 1. When "from foo import *" makes it impossible to know at compile-time what bindings will be added to the current scope, we just do normal name lookup for that scope. No big deal. It already works that way at module scope; why should this be any different? With this simplification, there can be a single scope chain: builtins <- module <- function <- nested-function <- ... and all scopes can be treated the same. The implementation could probably be both simpler and faster! Simpler, because we don't have to have separate cases for builtins, local, and global; and faster, because some of the optimizations we currently do for locals could be made to apply at all levels. Imagine "fast globals"! And imagine getting them essentially for free.
3. "def g(): print x" should behave the same as "print x"
Huh? again. Defining a function does't call it.
Duh, obviously i meant 3. "def g(): print x" immediately followed by "g()" should behave the same as "print x" Do you agree with this principle, at least?
Absolutely agreed. I've never intended to contradict this. This is the foundation of lexical scoping. -- ?!ng "Don't worry about people stealing an idea. If it's original, you'll have to jam it down their throats." -- Howard Aiken

[Ka-Ping Yee]
The conceptual model, yes, but the implementation isn't like that.
- Whenever you can't find what you're looking for, you go up to the next level and keep looking, right?
Conceptually, yes. No such looping search occurs at runtime, though.
- So what's the issue with not being able to determine which variable binds in which scope?
That determination is done at compile-time, not runtime. In the presence of "exec" and "import *" in some contexts, compile-time determination is stymied and there is no runtime support for a "slow" lookup. Note that the restrictions are *not* against lexical nesting, they're against particular uses of "exec" and "import *" (the latter of which is so muddy the Ref Man said it was undefined a long, long time ago).
Even locals used to be resolved by dict searches. The entire model there wasn't preserved by the old switch to fast locals either. For example,
IIRC, in the old days that would print 42. Who cares <0.1 wink>? This is nonsense either way. There are tradeoffs here among: conceptual clarity runtime efficiency implementation complexity rate of cyclic garbage creation Your message favors "conceptual clarity" over all else; the implementation doesn't. Python also limits strings to the size of a platform int <0.9 wink>.
But note that eval() didn't make it into the Scheme std: they couldn't agree on its semantics or implementation. eval() is *suggested* in the fifth Revised Report, but there has no access to its lexical environment; instead it acts "as if" its argument had appeared at top level "or in some other implementation-dependent environment" (Dybvig; "The Scheme Programming Language"). Dybvig gives an example of one of the competing Scheme eval() proposals gaining access to a local vrbl via using macros to interpolate the local's value into the argument's body before calling eval(). And that's where refusing to compromise leads. utterly-correct-and-virtually-useless-ly y'rs - tim

On Thu, 22 Feb 2001, Tim Peters wrote:
Would the existence of said runtime support hurt anybody? Don't we already do slow lookup in some situations anyway?
(To want to *take away* the ability to do import-* at all, in order to protect programmers from their own bad habits, is a different argument. I think we all already agree that it's bad form. But the recent clamour has shown that we can't take it away just yet.)
Yes, i do think conceptual clarity is important. The way Python leans towards conceptual simplicity is a big part of its success, i believe. The less there is for someone to fit into their brain, the less time they can spend worrying about how the language will behave and the more they can focus on getting the job done. And i don't think we have to sacrifice much of the others to do it. In fact, often conceptual clarity leads to a simpler implementation, and sometimes even a faster implementation. Now i haven't actually done the implementation so i can't tell you whether it will be faster, but it seems to me that it's likely to be simpler and could stand a chance of being faster. -- ?!ng "The only `intuitive' interface is the nipple. After that, it's all learned." -- Bruce Ediger, on user interfaces

Note that this is moot now -- see my previous post about how we've decided to resolve this using a magical import to enable nested scopes (in 2.1).
Actually, no.
- Whenever you can't find what you're looking for, you go up to the next level and keep looking, right?
That depends. Our model is inspired by the semantics of locals in Python 2.0 and before, and this all happens at compile time. That means that we must be able to know which names are defined in each scope at compile time.
You call it an optimization, and that's how it started. But since it clearly affects the semantics of the language, it's not really an optimization -- it's a particular semantics that lends itself to more and easy compile-time analysis and hence can be implemented more efficiently, but the corner cases are different, and the language semantics define what should happen, optimization or not. In particular: x = 1 def f(): print x x = 2 raises an UnboundLocalError error at the point of the print statement. Likewise, in the official semantics of nested scopes: x = 1 def f(): def g(): print x g() x = 2 also raises an UnboundLocalError at the print statement.
No need to go to the source -- this is all clearly explained in the PEP (http://python.sourceforge.net/peps/pep-0227.html).
Actually, most languages do all this at compile time. Very early Python versions did do all this at run time, but by the time 1.0 was released, the "locals are locals" rule was firmly in place. You may like the purely dynamic version better, but it's been outlawed long ago.
The only Python-specific parts are then:
5. The current scope is determined by the nearest enclosing 'def'.
For most purposes, 'class' also creates a scope.
Sure.
Sure.
You didn't try this, did you? or do you intend to say that it "should" print this? In fact it raises UnboundLocalError: local variable 'y' referenced before assignment. (Before 2.0 it would raise NameError.)
Wrong again. This prints 3, both without and with nested scopes as defined in 2.1a2. However it raises an exception with the current CVS version: SyntaxError: f: exec or 'import *' makes names ambiguous in nested scope.
I didn't try this one, but I'm sure that it prints 3 in 2.1a1 and raises the same SyntaxError as above with the current CVS version.
Too late. The semantics have been bent since 1.0 or before. The flow analysis needed to optimize this in such a way that the user can't tell whether this is optimized or not is too hard for the current compiler. The fully dynamic model also allows the user to play all sorts of stupid tricks. And the unoptimized code is so much slower that it's well worth to hve the optimization.
I don't know Scheme, but isn't it supposed to be a compiled language?
I'm not sure how you can say that Scheme sidesteps the issue when you just quote an example where Scheme implementations differ?
Sorry. Sometimes, reality bites. :-) Note that I want to take more of the dynamicism out of function bodies. The reference manual has for a long time outlawed import * inside functions (but the implementation didn't enforce this). I see no good reason to allow this (it's causing a lot of work to happen each time the function is called), and the needs of being able to clearly define what happens with nested scopes make it necessary to outlaw it. I also want to eventually completely outlaw exec without an 'in' clause inside a class or function, and access to local variables through locals() or vars(). I'm not sure yet about exec without an 'in' clause at the global level, but I'm tempted to think that even there it's not much use. We'll start with warnings for some of these cases in 2.1. I see that Tim posted another rebuttal, explaining better than I do here *why* Ping's "simple" model is not good for Python, so I'll stop now. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Thu, 22 Feb 2001, Guido van Rossum wrote:
Yes, yes. It seems like a good answer for now -- indeed, some sort of mechanism for selecting compilation options has been requested before. But we still need to eventually have a coherent answer. The chart in my other message doesn't look coherent to me -- it would take too long to explain all of the cases to someone. I deserve a smack on the head for my confusion at seeing 'x' printed out -- that happens to be the value of the NameError in 1.5.2. Here is an updated chart (updated test script is attached): 1.5.2 2.1a2 suggested toplevel with print x from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 3 1 3 1 3 1 with g() from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 3 1 3 1 3 1 x = 3 outside f() with print x from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 NameError UnboundLocal 3 1 with g() from foo import * 3 3 SyntaxError 3 1 exec('x = 1') 3 3 SyntaxError 3 1 x = 1 NameError UnboundLocal 3 1 x = 3 inside f() with print x from foo import * 3 1 3 1 3 1 exec('x = 1') 3 1 3 1 3 1 x = 1 3 1 3 1 3 1 with g() from foo import * NameError SyntaxError 3 1 exec('x = 1') NameError SyntaxError 3 1 x = 1 NameError 3 1 3 1 You can see that the situation in 1.5.2 is pretty messy -- and it's precisely the inconsistent cases that have historically caused confusion. 2.1a2 is better but it still has exceptional cases -- just the cases people seem to be complaining about now.
I'm talking about the model, not the implementation. I'm advocating that we think *first* about what the programmer (the Python user) has to worry about. I think that's a Pythonic perspective, isn't it? Or are you really saying that this isn't even the model that the user should be thinking about?
Well, can we nail down what you mean by "depends"? What reasoning process should the Python programmer go through to predict the behaviour of a given program?
I've been getting the impression that people consider this a language wart (or at least a little unfortunate, as it tends to confuse people). It's a frequently asked question, and when i've had to explain it to people they usually grumble. As others have pointed out, it can be pretty surprising when the assignment happens much later in the body. I think if you asked most people what this would do, they would expect 1. Why? Because they think about programming in terms of some simple invariants, e.g.: - Editing part of a block doesn't affect the behaviour of the block up to the point where you made the change. - When you move some code into a function and then call the function, that code still works the same. This kind of backwards-action-at-a-distance breaks the first invariant. Lexical scoping is good largely because it helps preserve the second invariant (the function carries the context of where it was defined). And so on.
No need to go to the source -- this is all clearly explained in the PEP (http://python.sourceforge.net/peps/pep-0227.html).
It seems not to be that simple, because i was unable to predict what situations would be problematic without understanding how the optimizations are implemented. * * *
5. The current scope is determined by the nearest enclosing 'def'.
For most purposes, 'class' also creates a scope.
Sorry, i should have written: 5. The parent scope is determined by the nearest enclosing 'def'. * * *
I know that. I introduced these examples with "given this model..." to indicate that i'm describing what the "completely clear answers" are. The chart above tries to summarize all of the current behaviour.
I think it's better to try to bend them as little as possible -- and if it's possible to unbend them to make the language easier to understand, all the better. Since we're changing the behaviour now, this is a good opportunity to make sure the model is simple.
That's not the point. There is a scoping model that is straightforward and easy to understand, and regardless of whether the implementation is interpreted or compiled, you can easily predict what a given piece of code is going to do.
I'm not sure how you can say that Scheme sidesteps the issue when you just quote an example where Scheme implementations differ?
That's what i'm saying. The standard sidesteps (i.e. doesn't specify how to handle) the issue, so the implementations differ. I don't think we have the option of avoiding the issue; we should have a clear position on it. (And that position should be as simple to explain as we can make it.)
Let's get a complete specification of the model then. And can i ask you to clarify your position: did you put quotation marks around "simpler" because you disagree that the model i suggest is simpler and easier to understand; or did you agree that it was simpler but felt it was worth compromising that simplicity for other benefits? And if the latter, are the other benefits purely about enabling optimizations in the implementation, or other things as well? Thanks, -- ?!ng

"KPY" == Ka-Ping Yee <ping@lfw.org> writes:
No need to go to the source -- this is all clearly explained in the PEP (http://python.sourceforge.net/peps/pep-0227.html).
KPY> It seems not to be that simple, because i was unable to predict KPY> what situations would be problematic without understanding how KPY> the optimizations are implemented. The problematic cases are exactly those where name bindings are introduced implicitly, i.e. cases where an operation binds a name without the name appearing the program text for that operation. That doesn't sound like an implementation-dependent defintion. [...] KPY> That's not the point. There is a scoping model that is KPY> straightforward and easy to understand, and regardless of KPY> whether the implementation is interpreted or compiled, you can KPY> easily predict what a given piece of code is going to do. [Taking you a little out of context:] This is just what I'm advocating for import * and exec in the presence of nested fucntions. There is no easy way to predict what a piece of code is going to do without (a) knowing what names a module defines or (b) figuring out what values the argument to exec will have. On the subject of easy prediction, what should the following code do according to your model: x = 2 def f(y): ... if y > 3: x = x - 1 ... print x ... x = 3 ... I think the meaning of print x should be statically determined. That is, the programmer should be able to determine the binding environment in which x will be resolved (for print x) by inspection of the code. Jeremy

I hate to be repetitive <snort>, but forget Scheme! Scheme has nothing like "import *" or Python's flavor of eval/exec. The only guidance we'll get there is that the Scheme designers were so put off by mixing lexical scoping with eval that even *referencing* non-toplevel vars inside eval's argument isn't supported. hmm-on-second-thought-let's-pay-a-lot-of-attention-to-scheme<0.6-wink>-ly y'rs - tim

git.tr[0][1:] is ('@test', 8, 'spam', ['def spam(a, b, c, d=3, (e, (f,))=(4, (5,)), *g, **h):\n'], 0) at this point. The test expects it to be ('@test', 9, 'spam', [' eggs(b + d, c + f)\n'], 0) Test passes without -O. This was on Windows. Other platforms?

tim wrote:
the code doesn't take LINENO optimization into account. tentative patch follows: Index: Lib/inspect.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/inspect.py,v retrieving revision 1.2 diff -u -r1.2 inspect.py --- Lib/inspect.py 2001/02/28 08:26:44 1.2 +++ Lib/inspect.py 2001/02/28 22:35:49 @@ -561,19 +561,19 @@ filename = getsourcefile(frame) if context > 0: - start = frame.f_lineno - 1 - context/2 + start = _lineno(frame) - 1 - context/2 try: lines, lnum = findsource(frame) start = max(start, 1) start = min(start, len(lines) - context) lines = lines[start:start+context] - index = frame.f_lineno - 1 - start + index = _lineno(frame) - 1 - start except: lines = index = None else: lines = index = None - return (filename, frame.f_lineno, frame.f_code.co_name, lines, index) + return (filename, _lineno(frame), frame.f_code.co_name, lines, index) def getouterframes(frame, context=1): """Get a list of records for a frame and all higher (calling) frames. @@ -614,3 +614,26 @@ def trace(context=1): """Return a list of records for the stack below the current exception.""" return getinnerframes(sys.exc_traceback, context) + +def _lineno(frame): + # Coded by Marc-Andre Lemburg from the example of PyCode_Addr2Line() + # in compile.c. + # Revised version by Jim Hugunin to work with JPython too. + # Adapted for inspect.py by Fredrik Lundh + + lineno = frame.f_lineno + + c = frame.f_code + if not hasattr(c, 'co_lnotab'): + return tb.tb_lineno + + tab = c.co_lnotab + line = c.co_firstlineno + stopat = frame.f_lasti + addr = 0 + for i in range(0, len(tab), 2): + addr = addr + ord(tab[i]) + if addr > stopat: + break + line = line + ord(tab[i+1]) + return line Cheers /F
participants (5)
-
Fredrik Lundh
-
Guido van Rossum
-
Jeremy Hylton
-
Ka-Ping Yee
-
Tim Peters