[Patches] [ python-Patches-979728 ] Implementation for PEP 318
using java-style syntax
SourceForge.net
noreply at sourceforge.net
Sat Jul 31 23:49:13 CEST 2004
Patches item #979728, was opened at 2004-06-25 13:26
Message generated for change (Comment added) made by mark_t_russell
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=979728&group_id=5470
Category: Parser/Compiler
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Mark Russell (mark_t_russell)
Assigned to: Nobody/Anonymous (nobody)
Summary: Implementation for PEP 318 using java-style syntax
Initial Comment:
This implements function decorators using the
Java-style syntax described in
http://mail.python.org/pipermail/python-dev/2004-June/045516.html
As the patch changes the grammar, you'll need to
make sure Include/graminit.h and Python/graminit.c
are writable after applying the patch but before
compiling (after a standard CVS checkout they are
read-only, which stops pgen from working).
Changes:
- adding @ as a token
- changing the grammer for funcdef
- adding com_decorator in compile.c
- adding test_decorator.py (based on Guido's version)
----------------------------------------------------------------------
>Comment By: Mark Russell (mark_t_russell)
Date: 2004-07-31 21:49
Message:
Logged In: YES
user_id=1017234
I've updated the patch again. See my previous
comment here for the procedure to apply the patch.
Main changes:
- Added decorator support to Lib/compiler.
test_decorators.py now passes when compiled
with Lib/compiler. The regression tests
also pass when built with
Tools/compiler/regrtest.py, apart from
test_dis.py which I assume is due to
trivial differences in the bytecode output.
- Added a description of decorators to the
reference manual. Changed the docs for
staticmethod and classmethod in the library
manual to use decorator syntax (with a pointer
to the decorator description in the
reference manual).
- Added more tests to test_decorator.py,
including some possibly useful example
decorators (e.g. memoize)
Other changes (not directly related to decorators):
- I regenerated Lib/compiler/ast.py using
Tools/compiler/astgen.py. I'm not sure
whether astgen.py is still in use -- I had
to tweak it a bit to make the output match
ast.py. I changed astgen.py to emit the
classes in alphabetical rather than dict
scan order to make it easier to diff the
output against ast.py
- test_pyclbr.py had a broken definition of
assertEquals - it always returned True if
called with two arguments. I renamed this
to assertEqualsOrIgnored and fixed the bug.
The working test revealed a test failure in
pyclbr.py:
- Changed pyclbr to use the full (dotted)
module name for pyclbr.Class.module attribute
Issues:
- test_pyclbr.py currently fails because its
is_method() method fails to spot staticmethod-
and classmethod- wrapped methods. See the
comment on the new test test_decorators().
I'll have another look at this next week.
Note that this is a problem with the test,
not the pyclbr.py module. For now
test_decorator could just be deleted.
----------------------------------------------------------------------
Comment By: Mark Russell (mark_t_russell)
Date: 2004-07-28 17:37
Message:
Logged In: YES
user_id=1017234
I've added an updated version of the patch, with the
following changes:
- Applied against CVS head (as of 2004-07-28)
- Non-standard C removed (see Guido's comment)
- Newline before the def is now mandatory
- Grammer for decorator expressions more restricted
- tokenize.py and test_tokenize updated to handle @ tokens
- Lib/compiler/transformer.py minimally updated
Still to do:
- Complete work on Lib/compiler (i.e. get it to the
point where it
produces a correct .pyc for code with decorators).
- Write docs
- Work through all the steps in PEP 306
- Add more regression tests to test_decorator
I can put in some time on this in the next few days if this
patch seems likely
to be applied.
The grammar is now:
decorator: '@' dotted_name [ '(' [arglist] ')' ]
decorators: decorator ([NEWLINE] decorator)* NEWLINE
funcdef: [decorators] 'def' NAME parameters ':' suite
This allows
@foo @bar
def f(): pass
but not:
@foo @bar def f(): paqss
Issues:
- Someone should take a look at the "case decorator:"
code in
symtable_node() in compile.c. All the tests pass, but
this
seems a fragile bit of code to be changing
- Lib/compiler needs some more work - at present
decorators are
ignored there. I'll do some more work on this when I
get time
(probably this weekend).
- As noted in the comments here, the build system does
not cope
automatically with grammar changes. The steps I used
to apply
the patch are:
cd .../dist/src
patch -p0 < /path/to/patch-file
./configure
make
chmod +w Lib/symbol.py Lib/token.py
./python Lib/symbol.py
./python Lib/token.py
make clean
make
make test >& make-test.out
I'm sure this sequence is non-optimal, but it works
for me, and
all the tests pass on my Linux system afterwards.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2004-07-28 03:39
Message:
Logged In: YES
user_id=6380
There's some non-standard C: in compile.c line 5641 there's a
char* declaration that isn't at the start of a block. Easily
fixed but should definitely be fixed before releasing a2.
----------------------------------------------------------------------
Comment By: Guido van Rossum (gvanrossum)
Date: 2004-07-28 03:22
Message:
Logged In: YES
user_id=6380
Before 2.4final I'd like to see some changes to the syntax:
- Instead of @test, it should be somewhat more restricted,
preferably @ followed by a dotted name followed by an
optional argument list. @1+2 or @[1,2,3] just doesn't make
sense IMO.
- I'd like there to always be a newline between the last
decorator and the 'def' keyword. (This in the sake
of "greppability" per python-dev discussion.)
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2004-07-26 14:20
Message:
Logged In: YES
user_id=6656
Well, PEP 306 is the reference for this sort of thing: also
needed (before 2.4 final) are changes to Lib/compiler and
tokenize.py. Mark, are you willing/able to work on these?
----------------------------------------------------------------------
Comment By: Anthony Baxter (anthonybaxter)
Date: 2004-07-26 14:09
Message:
Logged In: YES
user_id=29957
Argh. Our Makefile's ability to Do The Right Thing... is not
good. Rebuilding Lib/symbol.py and Lib/token.py makes this
better. The setup.py really should do this automatically.
So - all tests pass. As far as I can see, things still to be
done for this to be in a2:
- Documentation (can wait until a3, I guess)
- More coverage in test_decorator (ditto)
----------------------------------------------------------------------
Comment By: Anthony Baxter (anthonybaxter)
Date: 2004-07-26 13:58
Message:
Logged In: YES
user_id=29957
I still get a failure in test_transformer after a make
distclean and rebuild and make distclean and rebuild.
FAILED (errors=1)
Traceback (most recent call last):
File "Lib/test/test_transformer.py", line 37, in ?
test_main()
File "Lib/test/test_transformer.py", line 32, in test_main
test_support.run_unittest(
File
"/home/anthony/src/py/pyhead/dist/src/Lib/test/test_support.py",
line 290, in run_unittest
run_suite(suite, testclass)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/test/test_support.py",
line 275, in run_suite
raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
File "Lib/test/test_transformer.py", line 16, in
testMultipleLHS
a = transformer.parse(s)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 50, in parse
return Transformer().parsesuite(buf)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 120, in parsesuite
return self.transform(parser.suite(text))
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 113, in transform
return self.compile_node(tree)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 149, in compile_node
return self.file_input(node[1:])
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 180, in file_input
self.com_append_stmt(stmts, node)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 1025, in com_append_stmt
result = self.com_node(node)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 768, in com_node
return self._dispatch[node[0]](node[1:])
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 252, in stmt
return self.com_stmt(nodelist[0])
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 1018, in com_stmt
result = self.lookup_node(node)(node[1:])
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 312, in expr_stmt
lval = self.com_augassign(nodelist[0])
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 922, in com_augassign
l = self.com_node(node)
File
"/home/anthony/src/py/pyhead/dist/src/Lib/compiler/transformer.py",
line 768, in com_node
return self._dispatch[node[0]](node[1:])
KeyError: 268
----------------------------------------------------------------------
Comment By: Anthony Baxter (anthonybaxter)
Date: 2004-07-26 13:41
Message:
Logged In: YES
user_id=29957
Hm. Pilot error on my part. I had to build once to get
graminit.h and graminit.c rebuilt, then rebuild again (after
nuking all .o files) to get it to work properly. *sigh*
----------------------------------------------------------------------
Comment By: Anthony Baxter (anthonybaxter)
Date: 2004-07-26 13:30
Message:
Logged In: YES
user_id=29957
After applying this patch, any attempts to use the
newly-built python (I deleted graminit.c and graminit.h, to
make sure they were rebuilt) gives the following on any
'import':
*-s*) CC='gcc -pthread' LDSHARED='gcc -pthread -shared'
OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ./python -E
./setup.py -q build;; *) CC='gcc -pthread' LDSHARED='gcc -pthread -shared'
OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ./python -E
./setup.py build;; esac
Traceback (most recent call last):
File "./setup.py", line 6, in ?
import sys, os, getopt, imp, re
SystemError: compile_node: unexpected node type
make: *** [sharedmods] Error 1
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=979728&group_id=5470
More information about the Patches
mailing list