[ python-Bugs-1333982 ] Bugs of the new AST compiler

[SourceForge.net]

Bugs item #1333982, was opened at 2005-10-21 11:08
Message generated for change (Comment added) made by mwh
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1333982&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: AST
Status: Open
Resolution: None
Priority: 5
Submitted By: Armin Rigo (arigo)
Assigned to: Nobody/Anonymous (nobody)
Summary: Bugs of the new AST compiler

Initial Comment:
The newly merged AST branch is likely to expose
a number of small problems before it stabilizes,
so here is a tentative bug tracker entry to
collect such small problems.


----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2005-12-11 00:41

Message:
Logged In: YES 
user_id=6656

You have to include those lines in a source file.  It still crashes for me.

----------------------------------------------------------------------

Comment By: Brett Cannon (bcannon)
Date: 2005-12-10 23:44

Message:
Logged In: YES 
user_id=357491

I just checked Armin's problem code on rev. 41638 and it
worked for me in the interpreter.  You still having
problems, Armin?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-12-04 10:30

Message:
Logged In: YES 
user_id=4771

At rev 41584, the following code snippet triggers an assert
if --with-pydebug is enabled:
Python/compile.c:3843: assemble_lnotab: Assertion 'd_lineno >= 0' failed

-----------------------
assert 1, ([s for s in x] +
           [s for s in x])
pass
-----------------------

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-23 20:14

Message:
Logged In: YES 
user_id=33168

Checkpoint of outstanding issues.  I think all the others
mentioned so far have been fixed:

 * Raymond's warnings in compile.c (unused locals are fixed)
 * EXTENDED_ARG problem
 * LOAD_CONST/POP_TOP (note we can fix this in the optimizer
generally which would get rid of other useless code too)


----------------------------------------------------------------------

Comment By: Raymond Hettinger (rhettinger)
Date: 2005-10-23 05:53

Message:
Logged In: YES 
user_id=80475

FWIW, here are the warnings issued by my compiler:

Python-ast.c
C:\py25\Python\Python-ast.c(1995) : warning C4101: 'i' : 
unreferenced local variable
C:\py25\Python\Python-ast.c(2070) : warning C4101: 'i' : 
unreferenced local variable
C:\py25\Python\Python-ast.c(2085) : warning C4101: 'i' : 
unreferenced local variable
C:\py25\Python\Python-ast.c(2130) : warning C4101: 'i' : 
unreferenced local variable
C:\py25\Python\Python-ast.c(2151) : warning C4101: 'i' : 
unreferenced local variable
C:\py25\Python\Python-ast.c(2261) : warning C4101: 'i' : 
unreferenced local variable
C:\py25\Python\Python-ast.c(2270) : warning C4101: 'i' : 
unreferenced local variable


compile.c
C:\py25\Python\compile.c(3782) : warning C4305: '=' : truncation 
from 'const int ' to 'char '
C:\py25\Python\compile.c(3802) : warning C4305: '=' : truncation 
from 'const int ' to 'char '
C:\py25\Python\compile.c(3806) : warning C4305: '=' : truncation 
from 'const int ' to 'char '


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2005-10-22 08:28

Message:
Logged In: YES 
user_id=33168

I assigned to Jeremy and Neil in the hopes they will see
this message and know about these problems.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 13:45

Message:
Logged In: YES 
user_id=4771

The following (similarly strange-looking) code snippets
compiled successfully before, now they give SyntaxErrors:

--------------------
def f():
    class g:
        exec "hi"
        x
--------------------
def f(x):
    class g:
        exec "hi"
        x
--------------------
def f():
    class g:
        from a import *
        x
--------------------
def f(x):
    class g:
        from a import *
        x


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 13:33

Message:
Logged In: YES 
user_id=4771

I would suspect the following one to be due to incorrect
handling of EXTENDED_ARG -- it's from a PyPy test about that:

longexpr = 'x = x or ' + '-x' * 2500
code = '''
def f(x):
    %s
    %s
    %s
    %s
    %s
    %s
    %s
    %s
    %s
    %s
    while x:
        x -= 1
        # EXTENDED_ARG/JUMP_ABSOLUTE here
    return x
''' % ((longexpr,)*10)

exec code
f(5)
SystemError: unknown opcode

dis.dis() shows that the target of both the SETUP_LOOP and
the JUMP_IF_FALSE at the start of the loop are wrong.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 13:15

Message:
Logged In: YES 
user_id=4771

The Python rules about which names get mangled are a bit
insane.  I share mwh's view that mangling should never have
been invented in the first place, but well:

>>> def f():
...   __x = 5
...   class X:
...     def g(self):
...       return __x
...   return X
... 
Fatal Python error: unknown scope for _X__x in X(135832776)
in <stdin>


----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 13:01

Message:
Logged In: YES 
user_id=4771

For reference, an optimization that got lost:

    def f():
        'a'
        'b'

'a' is the docstring, but the 'b' previously did not show
up anywhere in the code object.  Now there is the
LOAD_CONST/POP_TOP pair.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 12:54

Message:
Logged In: YES 
user_id=4771

any reason why lambda functions have a __name__ of
'lambda' now instead of '<lambda>' ?

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 11:22

Message:
Logged In: YES 
user_id=4771

import a as b, c

the 'c' part gets completely forgotten and there is
no 'IMPORT_NAME c' in the bytecode.

----------------------------------------------------------------------

Comment By: Armin Rigo (arigo)
Date: 2005-10-21 11:13

Message:
Logged In: YES 
user_id=4771

A scoping problem (comparing with the old compiler):

class X:
    print hello

The variable 'hello' is incorrectly looked up with
LOAD_GLOBAL instead of LOAD_NAME.  It causes a crash
in PyPy in a case where the name 'hello' is stored
into the class implicitely (via locals()).  It can
probably be discussed if the bug is in PyPy, but it
is a difference in behavior.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1333982&group_id=5470