Hi.
[Mark Hammond]
> The point isn't about my suffering as such. The point is more that
> python-dev owns a tiny amount of the code out there, and I don't believe we
> should put Python's users through this.
>
> Sure - I would be happy to "upgrade" all the win32all code, no problem. I
> am also happy to live in the bleeding edge and take some pain that will
> cause.
>
> The issue is simply the user base, and giving Python a reputation of not
> being able to painlessly upgrade even dot revisions.
I agree with all this.
[As I imagined explicit syntax did not catch up and would require
lot of discussions.]
[GvR]
> > Another way is to use special rules
> > (similar to those for class defs), e.g. having
> >
> > <frag>
> > y=3
> > def f():
> > exec "y=2"
> > def g():
> > return y
> > return g()
> >
> > print f()
> > </frag>
> >
> > # print 3.
> >
> > Is that confusing for users? maybe they will more naturally expect 2
> > as outcome (given nested scopes).
>
> This seems the best compromise to me. It will lead to the least
> broken code, because this is the behavior that we had before nested
> scopes! It is also quite easy to implement given the current
> implementation, I believe.
>
> Maybe we could introduce a warning rather than an error for this
> situation though, because even if this behavior is clearly documented,
> it will still be confusing to some, so it is better if we outlaw it in
> some future version.
>
Yes this can be easy to implement but more confusing situations can arise:
<frag>
y=3
def f():
y=9
exec "y=2"
def g():
return y
return y,g()
print f()
</frag>
What should this print? the situation leads not to a canonical solution
as class def scopes.
or
<frag>
def f():
from foo import *
def g():
return y
return g()
print f()
</frag>
[Mark Hammond]
> > This probably won't be a very popular suggestion, but how about pulling
> > nested scopes (I assume they are at the root of the problem)
> > until this can be solved cleanly?
>
> Agreed. While I think nested scopes are kinda cool, I have lived without
> them, and really without missing them, for years. At the moment the cure
> appears worse then the symptoms in at least a few cases. If nothing else,
> it compromises the elegant simplicity of Python that drew me here in the
> first place!
>
> Assuming that people really _do_ want this feature, IMO the bar should be
> raised so there are _zero_ backward compatibility issues.
I don't say anything about pulling nested scopes (I don't think my opinion
can change things in this respect)
but I should insist that without explicit syntax IMO raising the bar
has a too high impl cost (both performance and complexity) or creates
confusion.
[Andrew Kuchling]
> >Assuming that people really _do_ want this feature, IMO the bar should be
> >raised so there are _zero_ backward compatibility issues.
>
> Even at the cost of additional implementation complexity? At the cost
> of having to learn "scopes are nested, unless you do these two things
> in which case they're not"?
>
> Let's not waffle. If nested scopes are worth doing, they're worth
> breaking code. Either leave exec and from..import illegal, or back
> out nested scopes, or think of some better solution, but let's not
> introduce complicated backward compatibility hacks.
IMO breaking code would be ok if we issue warnings today and implement
nested scopes issuing errors tomorrow. But this is simply a statement
about principles and raised impression.
IMO import * in an inner scope should end up being an error,
not sure about 'exec's.
We will need a final BDFL statement.
regards, Samuele Pedroni.
I'm still trying to sort this out. Some concerns and questions:
I don't like the new MatchFilename, because it triggers on *all* platforms
that #define HAVE_DIRENT_H.
Anyone, doesn't that trigger on straight Linux systems too (all I know is
that it's part of the Single UNIX Specification)?
I don't like it because it implements a woefully inefficient algorithm: it
cycles through the entire directory looking for a case-sensitive match. But
there can be hundreds of .py files in a directory, and on average it will
need to look at half of them, while if this triggers on straight Linux
there's no need to look at *any* of them there. I also don't like it because
it apparently triggers on Cygwin too but the code that calls it doesn't cater
to that Cygwin possibly *should* be defining ALTSEP as well as SEP.
Would rather dump MatchFilename and rewrite in terms of the old check_case
(which should run much quicker, and already comes in several appropriate
platform-aware versions -- and I clearly minimize the chance of breakage if I
stick to that time-tested code).
Steven, there is a "#ifdef macintosh" version of check_case already. Will
that or won't that work correctly on your variant of Mac? If not, would you
please supply a version that does (along with the #ifdef'ery needed to
recognize your Mac variant)?
Jason, I *assume* that the existing "#if defined(MS_WIN32) ||
defined(__CYGWIN__)" version of check_case works already for you. Scream if
that's wrong.
Steven and Jack, does getenv() work on both your flavors of Mac? I want to
make PYTHONCASEOK work for you too.
stoopid question: why the heck is xmllib using
"RuntimeError" to flag XML syntax errors?
raise RuntimeError, 'Syntax error at line %d: %s' % (self.lineno, message)
what's wrong with "SyntaxError"?
</F>
Windows.
> python ../lib/test/regrtest.py test_global
test_global
<test code>:2: SyntaxWarning: name 'a' is assigned to before global
declaration
<test code>:2: SyntaxWarning: name 'b' is assigned to before global
declaration
The actual stdout doesn't match the expected stdout.
This much did match (between asterisk lines):
**********************************************************************
test_global
**********************************************************************
Then ...
We expected (repr): 'got SyntaxWarning as e'
But instead we got: 'expected SyntaxWarning'
test test_global failed -- Writing: 'expected SyntaxWarning',
expected: 'got SyntaxWarning as e'
1 test failed: test_global
>
Hi all -- i've been reading the enormous thread on nested scopes
with some concern, since i would very much like Python to support
"proper" lexical scoping, yet i also care about compatibility.
There is something missing from my understanding here:
- The model is, each environment has a pointer to the
enclosing environment, right?
- Whenever you can't find what you're looking for, you
go up to the next level and keep looking, right?
- So what's the issue with not being able to determine
which variable binds in which scope? With the model
just described, it's perfectly clear. Is all this
breakage only caused by the particular optimizations
for lookup in the implementation (fast locals, etc.)?
Or have i missed something obvious?
I could probably go examine the source code of the nested scoping
changes to find the answer to my own question, but in case others
share this confusion with me, i thought it would be worth asking.
* * *
Consider for a moment the following simple model of lookup:
1. A scope maps names to objects.
2. Each scope except the topmost also points to a parent scope.
3. To look up a name, first ask the current scope.
4. When lookup fails, go up to the parent scope and keep looking.
I believe the above rules are common among many languages and are
commonly understood. The only Python-specific parts are then:
5. The current scope is determined by the nearest enclosing 'def'.
6. These statements put a binding into the current scope:
assignment (=), def, class, for, except, import
And that's all.
* * *
Given this model, all of the scoping questions that have been
raised have completely clear answers:
Example I
>>> y = 3
>>> def f():
... print y
...
>>> f()
3
Example II
>>> y = 3
>>> def f():
... print y
... y = 1
... print y
...
>>> f()
3
1
>>> y
3
Example III
>>> y = 3
>>> def f():
... exec "y = 2"
... def g():
... return y
... return g()
...
>>> f()
2
Example IV
>>> m = open('foo.py', 'w')
>>> m.write('x = 1')
>>> m.close()
>>> def f():
... x = 3
... from foo import *
... def g():
... print x
... g()
...
>>> f()
1
In Example II, the model addresses even the current situation
that sometimes surprises new users of Python. Examples III and IV
are the current issues of contention about nested scopes.
* * *
It's good to start with a simple model for the user to understand;
the implementation can then do funky optimizations under the covers
so long as the model is preserved. So for example, if the compiler
sees that there is no "import *" or "exec" in a particular scope it
can short-circuit the lookup of local variables using fast locals.
But the ability of the compiler to make this optimization should only
affect performance, not affect the Python language model.
The model described above is the approximately the one available in
Scheme. It exactly reflects the environment-diagram model of scoping
as taught to most Scheme students and i would argue that it is the
easiest to explain.
Some implementations of Scheme, such as STk, do what is described
above. UMB scheme does what Python does now: the use-before-binding
of 'y' in Example II would cause an error. I was surprised that
these gave different behaviours; it turns out that the Scheme
standard actually forbids the use of internal defines not at the
beginning of a function body, thus sidestepping the issue. But we
can't do this in Python; assignment must be allowed anywhere.
Given that internal assignment has to have some meaning, the above
meaning makes the most sense to me.
-- ?!ng
PEP 236 states that the intention of the proposed feature is to allow
modules "to request that the code in module M use the new syntax or
semantics in the current release C".
It achieves this by introducing a new statement, the
future_statement. This looks like an import statement, but isn't. The
PEP author admits that 'overloading "import" does suck'. I agree (not
surprisingly, since Tim added this QA item after we discussed it in
email).
It also says "But if we introduce a new keyword, that in itself would
break old code". Here I disagree, and I propose patch 404997 as an
alternative
(https://sourceforge.net/tracker/index.php?func=detail&aid=404997&group_id=5…)
<specification section="Alternative Solution">
In essence, with that patch, you would write
directive nested_scopes
instead of
from __future__ import nested_scopes
This looks like as it would add a new keyword directive, and thus
break code that uses "directive" as an identifier, but it doesn't. In
this release, "directive" is only a keyword if it is the first keyword
in a file (i.e. potentially after a doc string, but not after any
other keyword). So
class directive:
def __init__(self, directive):
self.directive = directive
continues to work as it did in previous releases (it does not even
produce a warning, but could if desired). Only when you do
directive nested_scopes
directive braces
class directive:
def __init__(self, directive):
self.directive = directive
you get a syntax error, since "directive" is then a keyword in that
module.
The directive statement has a similar syntax to the C #pragma
"statement", in that each directive has a name and an optional
argument. The choice of the keyword "directive" is somewhat arbitrary;
it was deliberately not "pragma", since that implies an
implementation-defined semantics (which directive does not have).
In terms of backwards compatibility, it behaves similar to "from
__future__ import ...": older releases will give a SyntaxError for the
directive syntax (instead of an ImportError, which a __future__ import
will give). "Unknown" directives will also give a SyntaxError, similar
to the ImportError from the __future__ import.
</specification>
Please let me know what you think. If you think this should be written
down in a PEP, I'd request that the specification above is added into
PEP 236.
Regards,
Martin
Hi. Here are some things i noticed tonight.
1. The error message for UnboundLocalError isn't really accurate.
>>> def f():
... x = 1
... del x
... print x
...
>>> f()
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 4, in f
UnboundLocalError: local variable 'x' referenced before assignment
>>>
It's not a question of the variable being referenced "before
assignment" -- it's just that the variable is undefined. Better
would be a straightforward message such as
UnboundLocalError: local name 'x' is not defined
This message would be consistent with the others:
NameError: name 'x' is not defined
NameError: global name 'x' is not defined
2. Why does imp.find_module('') succeed?
>>> import imp
>>> imp.find_module('')
(None, '/home/ping/python/', ('', '', 5))
I think it should fail with "empty module name" or something similar.
3. Normally when a script is run, it looks like '' gets prepended to
sys.path so that the current directory will be searched. But if
the script being run is a symlink, the symlink is resolved first to
an actual file, and the directory containing that file is prepended
to sys.path. This leads to strange behaviour:
localhost[1004]% cat > spam.py
bacon = 5
localhost[1005]% cat > /tmp/eggs.py
import spam
localhost[1006]% ln -s /tmp/eggs.py .
localhost[1007]% python eggs.py
Traceback (most recent call last):
File "eggs.py", line 1, in ?
import spam
ImportError: No module named spam
localhost[1008]% python
Python 2.1a2 (#23, Feb 11 2001, 16:26:17)
[GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> import spam
>>>
(whereupon the confused programmer says, "Huh? If *i* could
import spam, why couldn't eggs?"). Was this a design decision?
Should it be changed to always prepend ''?
4. As far as i can tell, the curses.wrapper package is inaccessible.
It's obscured by a curses.wrapper() function in the curses package.
>>> import curses.wrapper
>>> curses.wrapper
<function wrapper at 0x80ebe14>
>>> import sys
>>> sys.modules['curses.wrapper']
<module 'curses.wrapper' from '/home/ping/dev/python/dist/src/Lib/curses/wrapper.pyc'>
I don't see any way around this other than renaming curses.wrapper.
-- ?!ng
"If I have not seen as far as others, it is because giants were standing
on my shoulders."
-- Hal Abelson
Fred,
You made a change to the syntax error generation code last August.
I don't understand what the code is doing. It appears that the code
you added is redundant, but it's hard to tell for sure because
responsbility for generating well-formed SyntaxErrors is spread
across several files.
The code you added in pythonrun.c, line 1084, in err_input(), starts
with the test (v != NULL):
w = Py_BuildValue("(sO)", msg, v);
PyErr_SetObject(errtype, w);
Py_XDECREF(w);
if (v != NULL) {
PyObject *exc, *tb;
PyErr_Fetch(&errtype, &exc, &tb);
PyErr_NormalizeException(&errtype, &exc, &tb);
if (PyObject_SetAttrString(exc, "filename",
PyTuple_GET_ITEM(v, 0)))
PyErr_Clear();
if (PyObject_SetAttrString(exc, "lineno",
PyTuple_GET_ITEM(v, 1)))
PyErr_Clear();
if (PyObject_SetAttrString(exc, "offset",
PyTuple_GET_ITEM(v, 2)))
PyErr_Clear();
Py_DECREF(v);
PyErr_Restore(errtype, exc, tb);
}
What's weird about this code is that the __init__ code for a
SyntaxError (all errors will be SyntaxErrors at this point) sets
filename, lineno, and offset. Each of the values is passed to the
constructor as the tuple v; then the new code gets the items out of
the tuple and sets the explicitly.
You also made a bunch of changes to SyntaxError__str__ at the same
time. I wonder if they were sufficient to fix the bug (which has
tracker aid 210628 incidentally).
Can you shed any light?
Jeremy
Hi.
Sorry if everybody is already aware of this.
I have checked the code for pdb in CVS , especially for the p cmd,
maybe I'm wrong but given actual the implementation of things that
gives no access to the value of free or cell variables. Should that
be fixed?
AFAIK pdb as it is works with jython too. So when fixing that, it would
be nice if this would be preserved.
regards, Samuele Pedroni.
