[ python-Bugs-1333982 ] Bugs of the new AST compiler
SourceForge.net
noreply at sourceforge.net
Sun Jul 2 13:29:46 CEST 2006
Bugs item #1333982, was opened at 2005-10-21 10:08
Message generated for change (Comment added) made by arigo
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1333982&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: AST
Status: Open
Resolution: None
Priority: 7
Submitted By: Armin Rigo (arigo)
Assigned to: Jeremy Hylton (jhylton)
Summary: Bugs of the new AST compiler
Initial Comment:
The newly merged AST branch is likely to expose
a number of small problems before it stabilizes,
so here is a tentative bug tracker entry to
collect such small problems.
----------------------------------------------------------------------
>Comment By: Armin Rigo (arigo)
Date: 2006-07-02 11:29
Message:
Logged In: YES
user_id=4771
Attached a patch for the LOAD_CONST POP_TOP optimization
(modified from Georg Brandl on python-dev).
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2006-04-11 09:41
Message:
Logged In: YES
user_id=6656
Good morning Armin!
I've reported that bug already: http://python.org/sf/1441486
There's a patch which purports to fix it: http://python.org/sf/1446922
but I haven't gotten around to testing it.
(this is running the pypy/module/array tests or something, isn't it?)
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2006-04-11 08:45
Message:
Logged In: YES
user_id=4771
Another one: the literal -2147483648 (i.e. the value of
-sys.maxint-1) gives a long in 2.5, but an int in <= 2.4.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2006-04-03 07:30
Message:
Logged In: YES
user_id=33168
The tuple store problem is fixed. The only outstanding item
is the LOAD_CONST/POP_TOP. I will fix that soon.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2006-02-18 06:56
Message:
Logged In: YES
user_id=33168
Jeremy, there's no need to read anything before my last
comment at 2005-12-17 23:10. The last two by Armin,
Michael, then my last comment are the only important ones.
Everything that occurred before my 2005-12-17 comment was
taken care of AFAIK.
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2006-02-12 21:54
Message:
Logged In: YES
user_id=4771
Subscripting is generally a bit sloppy: e.g. the AST model has
no way to distinguish between a single value and a one-element
tuple value! See:
>>> d = {}
>>> d[1,] = 6
>>> d
{1: 6} # !
I suggest we fix the model to turn the 'subs' of the 'Subscript' node
from a list of nodes to a single, mandatory 'sub' node. If tupling is
necessary, it can be explicitly represented with a 'sub' containing a
'Tuple' node.
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2006-02-09 15:02
Message:
Logged In: YES
user_id=6656
We found another one. Something is wrong in the compilation of augmented
assignment to subscriptions containing tuples; running this code:
class C:
def __setitem__(self, i, v):
print i, v
def __getitem__(self, i):
print i
return 0
c = C()
c[4,5] += 1
gives a spurious exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: object does not support item assignment
By contrast, "c[(4,5)] += 1" works fine.
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2005-12-18 07:10
Message:
Logged In: YES
user_id=33168
EXTENDED_ARG problem was fixed a while ago.
The assert/pass problem was fixed with: 41756.
That leaves the LOAD_CONST/POP_TOP optimization that was
lost and one compiler warning: marshal_write_mod() not being
used.
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2005-12-11 00:41
Message:
Logged In: YES
user_id=6656
You have to include those lines in a source file. It still crashes for me.
----------------------------------------------------------------------
Comment By: Brett Cannon (bcannon)
Date: 2005-12-10 23:44
Message:
Logged In: YES
user_id=357491
I just checked Armin's problem code on rev. 41638 and it
worked for me in the interpreter. You still having
problems, Armin?
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-12-04 10:30
Message:
Logged In: YES
user_id=4771
At rev 41584, the following code snippet triggers an assert
if --with-pydebug is enabled:
Python/compile.c:3843: assemble_lnotab: Assertion 'd_lineno >= 0' failed
-----------------------
assert 1, ([s for s in x] +
[s for s in x])
pass
-----------------------
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-23 19:14
Message:
Logged In: YES
user_id=33168
Checkpoint of outstanding issues. I think all the others
mentioned so far have been fixed:
* Raymond's warnings in compile.c (unused locals are fixed)
* EXTENDED_ARG problem
* LOAD_CONST/POP_TOP (note we can fix this in the optimizer
generally which would get rid of other useless code too)
----------------------------------------------------------------------
Comment By: Raymond Hettinger (rhettinger)
Date: 2005-10-23 04:53
Message:
Logged In: YES
user_id=80475
FWIW, here are the warnings issued by my compiler:
Python-ast.c
C:\py25\Python\Python-ast.c(1995) : warning C4101: 'i' :
unreferenced local variable
C:\py25\Python\Python-ast.c(2070) : warning C4101: 'i' :
unreferenced local variable
C:\py25\Python\Python-ast.c(2085) : warning C4101: 'i' :
unreferenced local variable
C:\py25\Python\Python-ast.c(2130) : warning C4101: 'i' :
unreferenced local variable
C:\py25\Python\Python-ast.c(2151) : warning C4101: 'i' :
unreferenced local variable
C:\py25\Python\Python-ast.c(2261) : warning C4101: 'i' :
unreferenced local variable
C:\py25\Python\Python-ast.c(2270) : warning C4101: 'i' :
unreferenced local variable
compile.c
C:\py25\Python\compile.c(3782) : warning C4305: '=' : truncation
from 'const int ' to 'char '
C:\py25\Python\compile.c(3802) : warning C4305: '=' : truncation
from 'const int ' to 'char '
C:\py25\Python\compile.c(3806) : warning C4305: '=' : truncation
from 'const int ' to 'char '
----------------------------------------------------------------------
Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-22 07:28
Message:
Logged In: YES
user_id=33168
I assigned to Jeremy and Neil in the hopes they will see
this message and know about these problems.
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 12:45
Message:
Logged In: YES
user_id=4771
The following (similarly strange-looking) code snippets
compiled successfully before, now they give SyntaxErrors:
--------------------
def f():
class g:
exec "hi"
x
--------------------
def f(x):
class g:
exec "hi"
x
--------------------
def f():
class g:
from a import *
x
--------------------
def f(x):
class g:
from a import *
x
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 12:33
Message:
Logged In: YES
user_id=4771
I would suspect the following one to be due to incorrect
handling of EXTENDED_ARG -- it's from a PyPy test about that:
longexpr = 'x = x or ' + '-x' * 2500
code = '''
def f(x):
%s
%s
%s
%s
%s
%s
%s
%s
%s
%s
while x:
x -= 1
# EXTENDED_ARG/JUMP_ABSOLUTE here
return x
''' % ((longexpr,)*10)
exec code
f(5)
SystemError: unknown opcode
dis.dis() shows that the target of both the SETUP_LOOP and
the JUMP_IF_FALSE at the start of the loop are wrong.
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 12:15
Message:
Logged In: YES
user_id=4771
The Python rules about which names get mangled are a bit
insane. I share mwh's view that mangling should never have
been invented in the first place, but well:
>>> def f():
... __x = 5
... class X:
... def g(self):
... return __x
... return X
...
Fatal Python error: unknown scope for _X__x in X(135832776)
in <stdin>
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 12:01
Message:
Logged In: YES
user_id=4771
For reference, an optimization that got lost:
def f():
'a'
'b'
'a' is the docstring, but the 'b' previously did not show
up anywhere in the code object. Now there is the
LOAD_CONST/POP_TOP pair.
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 11:54
Message:
Logged In: YES
user_id=4771
any reason why lambda functions have a __name__ of
'lambda' now instead of '<lambda>' ?
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 10:22
Message:
Logged In: YES
user_id=4771
import a as b, c
the 'c' part gets completely forgotten and there is
no 'IMPORT_NAME c' in the bytecode.
----------------------------------------------------------------------
Comment By: Armin Rigo (arigo)
Date: 2005-10-21 10:13
Message:
Logged In: YES
user_id=4771
A scoping problem (comparing with the old compiler):
class X:
print hello
The variable 'hello' is incorrectly looked up with
LOAD_GLOBAL instead of LOAD_NAME. It causes a crash
in PyPy in a case where the name 'hello' is stored
into the class implicitely (via locals()). It can
probably be discussed if the bug is in PyPy, but it
is a difference in behavior.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1333982&group_id=5470
More information about the Python-bugs-list
mailing list