Mailman 3 September 2004 - Python-Dev

Re: [Python-checkins] python/dist/src/Modules threadmodule.c, 2.56, 2.56.8.1
by Guido van Rossum 03 Sep '04

03 Sep '04

On Wed, 01 Sep 2004 15:31:26 -0700, mhammond(a)users.sourceforge.net <mhammond(a)users.sourceforge.net> wrote: > Update of /cvsroot/python/python/dist/src/Modules > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22421/Modules > > Modified Files: > Tag: release23-maint > threadmodule.c > Log Message: > Backport [ 1010677 ] thread Module Breaks PyGILState_Ensure() > to the 2.3 maint branch. As long as we're backporting C APIs to 2.3, can I request that the new datetime API be backported to 2.3? Anthony Tuininga (the cx_Oracle author) would be interested in using this and might be willing to help out with the work. (And yes, I'm encouraging this because I could use this myself.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) Ask me about gmail.

9 23

RE: [Python-Dev] PEP 309 updated slightly
by Moore, Paul 03 Sep '04

03 Sep '04

From: Aahz >On Mon, Aug 30, 2004, Peter Harris wrote: >> >> I have updated PEP309 a little, [...] > >> I hope there will still be time to squeeze it in before 2.4 beta 1, and >> if there is any reason why not (apart from there has to be a cut-off >> point somewhere, which I accept), please someone let me know what work >> is still needed to make it ready to go in. > What's PEP309? PEP 309 is the "Partial Function Application" class. (Used to be referred to as "Currying", but that was agreed not to be an accurate name). Paul __________________________________________________________________________ This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Atos Origin group liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted. __________________________________________________________________________

3 3

[Python-Dev] FW: [Python-checkins] python/nondist/peps pep-0318.txt, 1.25, 1.26
by Raymond Hettinger 03 Sep '04

03 Sep '04

> List some possible reasons why arriving at consensus about > decorators has been so hard (or impossible) to achieve Thanks. I think that was an important contribution to the discussion. At this point, looking at the meta-discussion is likely to best way to help cut through the current morass. +There is no one clear reason why this should be so, but a +few problems seem to be most problematic. A possible social issue is that decorators can be used in a tremendous variety of ways, each of which is only needed or appreciated by small, disjoint groups of users. For instance, applications to ctypes have unique needs that not understood or shared by others. Some users want to use decorators for metaclass magic that scares the socks off of the rest. > + Almost everyone agrees that decorating/transforming a function at > + the end of its definition is suboptimal. One other point of agreement is that no one like having to write the function name three times: def f(): ... f = deco(f) > +* Syntactic constraints. There is an existing (though clumsy) syntax. Most of the proposals are less flexible but more elegant. This trade-off has created much more consternation than if there were no existing ways to apply decorations. +* Overall unfamiliarity with the concept. For people who have a + passing acquaintance with algebra (or even basic arithmetic) or have + used at least one other programming language, much of Python is + intuitive. Very few people will have had any experience with the + decorator concept before encountering it in Python. There's just no + strong preexisting meme that captures the concept. My take on this one is that there are some existing memes from C# and Java and that in the future they will be more widely known. However, there are not good existing, thought-out concepts of what exactly decorators should do in terms of preserving docstrings, attributes, using parameters, etc. Most have only a single application in mind and the full implications (and possible applications) of a given syntax are shrouded in the mists of the future. Like metaclasses, everyone knew they were powerful when they were put in, but no one knew how they would be used or whether they were necessary. In fact, two versions later, we still don't know those answers. Raymond

4 3

Re: [Python-Dev] problem with pymalloc on the BeOS port.
by François Revol 02 Sep '04

02 Sep '04

Tim Peters <tim.peters(a)gmail.com>: > [François Revol] > > I'm updating the BeOS port of python to include the latest version > > in > > Zeta, the next incarnation of BeOS. > > > > I'm having some problem when enabling pymalloc: > > [zapped] > > 0 bytes originally requested > > The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected. > > The 4 pad bytes at tail=0x80010fb8 are not all FORBIDDENBYTE > > (0xfb): > > at tail+0: 0xdb *** OUCH > > at tail+1: 0xdb *** OUCH > > at tail+2: 0xdb *** OUCH > > at tail+3: 0xdb *** OUCH > > The block was made by call #3688627195 to debug malloc/realloc. > > No it wasn't. The four bytes following the 4 trailing 0xfb hold the > call number, and they're apparently corrupt too. Eh... > > > Fatal Python error: bad trailing pad byte > > > > indeed, there seem to be 4 deadblock marks between the forbidden > > ones, > > while the len is supposed to be null: > > That's reliable. If there actually was a request for 0 bytes (that > is, assuming this pointer isn't just pointing at a random memory > address), the debug pymalloc allocates 16 bytes for it, filled with > > 00000000 fbfbfbfb fbfbfbfb serial > > where "serial" is a big-endian 4-byte "serial number" uniquely > identifying which call to malloc (or realloc) this was. The address > of the second block of fb's is returned. Yes that's what I deduced from the code of pymalloc. > > > python:dm 0x80010fb8-8 32 > > 80010fb0 00000000 fbfbfbfb dbdbdbdb fbfbdbdb > > ................. > > > > 80010fc0 0100fbfb 507686ef 04000000 fbfbfbfb > > .......vP........ > > 80010fd0 8013cbc8 fbfbfbfb 44ee0100 ffed0100 > > ............D.... > > > > Any clue ? > > Try gcc without -O. Nobody has reported anything like this before -- > you're in for a lot of fun <wink>. > OK, tried -O0 -g but same result. I suspect it might be a bad interaction with fork(), as it crashes in a child, quite badly, as no images are repported as loaded in the team (= no binary are mapped in the process), however the areas are there (= pages). Now, I don't see why malloc itself would give such a result, it's pyMalloc which places those marks, so the thing malloc does wouldn't place them 4 bytes of each other for no reason, or repport 0 bytes where 4 are allocated. François.

2 2

Decimal for Py2.3 (was: about presicion)
by Batista, Facundo 02 Sep '04

02 Sep '04

[Alex Martelli] #- I think that's an excellent policy -- Python 2.3 will no doubt remain #- widely used for a long time to come. I think it would be nice if #- Decimal was packaged up with its own docs for easy download and install #- into an existing 2.3 installation, then... make life as easy as possible #- for 2.3 users who need to do some decimal arithmetic! Don't know what's commong usage, so I ask. Should I prepare a "decimal package" with the module and docs and whatever, in form of a tgz, rmp, .exe, etc, to let people "install" decimal in their Py2.3? Or it's better to somewhere tell the user that if he/she wants to use Decimal in Py2.3 to follow this simple n steps (and the detail of the steps, of course ;)? Considering that it's only a file, and the docs could be accessed through Py2.4 documentation, I'll go for the latter. . Facundo

2 1

Installing python-dev
by Vishal Kapoor 02 Sep '04

02 Sep '04

Hi, I am trying to install Zope and it requires python2.3-dev. I downloaded Python and installed it, how do i install python-dev ?? Thank you Vishal Kapoor

4 3

Rejecting the J2 decorators proposal
by Guido van Rossum 01 Sep '04

01 Sep '04

Robert and Python-dev, I've read the J2 proposal up and down several times, pondered all the issues, and slept on it for a night, and I still don't like it enough to accept it. The only reason to accept it would be to pacify the supporters of the proposal, and that just isn't a good enough reason in language design. However, it got pretty darn close! I'm impressed with how the community managed to pull together and face the enormous challenge of picking a single alternative (from more than two dozen on the Wiki!) and arguing consistently. I expect to see more proposals like this in the future, and I'm sure that some of them will be good enough to make it into the language. I've also (again) learned a lesson: dramatic changes must be discussed with the community at large. In a large enough group there are no uncontroversial proposals, so this will take time, but it's worth it -- one of the main issues with the @decorator syntax was not technical but socio-political, in the sense that it hadn't been properly discussed outside a *very* small circle. I take the full blame for that, and I don't want to hide behind my current lack of time which, realistically, won't change until either my ESI stock options earn me an early retirement, or the PSF strikes it rich and can pay me full time :-). So let me explain why I'm not choosing J2, and what's next. There are two major issues and one minor that made me decide against J2. Major issue one: the syntactic form of an indented block strongly suggests that its contents should be a sequence of statements, but in fact it is not -- only expressions are allowed, and there is an implicit "collecting" of these expressions going on until they can be applied to the subsequent function definition. To me, this is a more serious problem than the namespace questions brought up in the proposal (unfortunately that particular section of the proposal is its most confused part; but even if the text had been crystal clear, the problem remains). The best counter-argument to this I've heard is "you'll get used to it", which is also what I'm saying of @decorators; and many people have already testified that they indeed got used to it and even liked it. Major issue two: the keyword starting the line that heads a block draws a lot of attention to it. This is true for "if", "while", "for", "try", "def" and "class". But the "using" keyword (or any other keyword in its place) doesn't deserve that attention; the emphasis should be on the decorator or decorators inside the suite, since those are the important modifiers to the function definition that follows. When a function definition carries one or more decorators, the most important information is not the fact that it has decorators, but the specific decorators used. A classmethod or staticmethod decorator adds a completely different flavor than a decorator that provides an external linkage hint for ObjC, or one that adds synchronization, or one that declares deprecation. I expect that at least 80% of the use of decorators will have a single decorator per function, and it's a pain for that decorator to be hiding behind a content-free keyword. (This is *not* a number-of-keystrokes argument. You know I don't care much about that.) Minor issue: "using" is a poor choice of keyword. It resembles C#'s "using" and perhaps Perl's "use", both of which have completely different meanings. But there don't seem to be any better alternatives (the best I could come up with was "transmogrify" :-). So, what's next? In Python 2.4a3 (to be released this Thursday), everything remains as currently in CVS. For 2.4b1, I will consider a change of @ to some other single character, even though I think that @ has the advantage of being the same character used by a similar feature in Java. It's been argued that it's not quite the same, since @ in Java is used for attributes that don't change semantics. But Python's dynamic nature makes that its syntactic elements never mean quite the same thing as similar constructs in other languages, and there is definitely significant overlap. Regarding the impact on 3rd party tools: IPython's author doesn't think there's going to be much impact; Leo's author has said that Leo will survive (although it will cause him and his users some transitional pain). I actually expect that picking a character that's already used elsewhere in Python's syntax might be harder for external tools to adapt to, since parsing will have to be more subtle in that case. But I'm frankly undecided, so there's some wiggle room here. I don't want to consider further syntactic alternatives at this point: the buck has to stop at some point, everyone has had their say, and the show must go on. In the coming years I hope that as a community we'll gain enough experience with decorators to decide whether we need to adopt a different syntax for Python 3000 or not. One of the difficulties with choosing a decorator syntax has definitely been that nobody can predict how they are going to be used predominantly. Different alternatives look better depending on whether there are many or few decorators per function, whether they have long argument lists or not, and perhaps also whether their use is for transformation or for annotation. Despite the novelty of using the @ character, I personally feel that prefix decorators are a huge improvement over the "f = staticmethod(f)" style of decorating. A warning: some people have shown examples of extreme uses of decorators. I've seen decorators proposed for argument and return type annotations, and even one that used a decorator to create an object that did a regular expression substitution. Those uses are cute, but I recommend being conservative when deciding between using a decorator or some other approach, especially in code that will see a large audience (like 3rd party library packages). Using decorators for type annotations in particular looks tedious, and this particular application is so important that I expect Python 3000 will have optional type declarations integrated into the argument list. Thanks to everyone who read until the end of this message! -- --Guido van Rossum (home page: http://www.python.org/~guido/)

2 2

[Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.30, 1.31
by Fred L. Drake, Jr. 01 Sep '04

01 Sep '04

On Wednesday 01 September 2004 11:02 am, anthonybaxter(a)users.sourceforge.net wrote: > (I'm not sure if the "Community Concensus" section should be trimmed > down radically now - it's a lot of words for a rejected form, and the > case for the form is still available on the web and in the mailing > list archives... opinions, anyone?) I'm for leaving the text in; wikis are fragile, and this is a valuable bit of Python history. A reader can ignore it if that's not interesting to them. -Fred -- Fred L. Drake, Jr. <fdrake at acm.org>

2 1

[Python-Dev] Re: [Python-checkins] python/nondist/peps pep-0318.txt, 1.30, 1.31
by Skip Montanaro 01 Sep '04

01 Sep '04

anthony> (I'm not sure if the "Community Concensus" section should be anthony> trimmed down radically now - it's a lot of words for a rejected anthony> form, and the case for the form is still available on the web anthony> and in the mailing list archives... opinions, anyone?) I'd just refer to the wiki and Robert Brewer's J2 proposal. Skip

1 0

(my) revisions to PEP318 finally done.
by Anthony Baxter 01 Sep '04

01 Sep '04

Here's the version of PEP 318 I just checked in. If you have any additions or changes, I prefer patches mailed to me in unified diff format. Comments like "You should cover point X, Y and Z", without the accompanying words, I won't be acting on (others with checkin privs are more than welcome to do so). Sorry about that, but I _cannot_ spend any more time on this document now - this whole decorators thing has consumed far more of my time than I can afford right now. PEP: 318 Title: Decorators for Functions and Methods Version: $Revision: 1.28 $ Last-Modified: $Date: 2004/08/30 13:16:56 $ Author: Kevin D. Smith, Jim Jewett, Skip Montanaro, Anthony Baxter Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 05-Jun-2003 Python-Version: 2.4 Post-History: 09-Jun-2003, 10-Jun-2003, 27-Feb-2004, 23-Mar-2004, 30-Aug-2004 WarningWarningWarning ===================== The final decision on the syntax for 2.4a3 is not yet made. This will be done before 2.4a3, and this document will be updated to match. Note also that this document does not attempt to cover the huge number of potential alternative syntaxes, nor is it an attempt to exhaustively list all the positives and negatives of each form. Abstract ======== The current method for transforming functions and methods (for instance, declaring them as a class or static method) is awkward and can lead to code that is difficult to understand. Ideally, these transformations should be made at the same point in the code where the declaration itself is made. This PEP introduces new syntax for transformations of a function or method declaration. Motivation ========== The current method of applying a transformation to a function or method places the actual translation after the function body. For large functions this separates a key component of the function's behavior from the definition of the rest of the function's external interface. For example:: def foo(self): perform method operation foo = classmethod(foo) This becomes less readable with longer methods. It also seems less than pythonic to name the function three times for what is conceptually a single declaration. A solution to this problem is to move the transformation of the method closer to the method's own declaration. While the new syntax is not yet final, the intent is to replace:: def foo(cls): pass foo = synchronized(lock)(foo) foo = classmethod(foo) with an alternative that places the decoration in the function's declaration:: @classmethod @synchronized(lock) def foo(cls): pass Modifying classes in this fashion is also possible, though the benefits are not as immediately apparent. Almost certainly, anything which could be done with class decorators could be done using metaclasses, but using metaclasses is sufficiently obscure that there is some attraction to having an easier way to make simple modifications to classes. For Python 2.4, only function/method decorators are being added. Why Is This So Hard? -------------------- Two decorators (``classmethod()`` and ``staticmethod()``) have been available in Python since version 2.2. It's been assumed since approximately that time that some syntactic support for them would eventually be added to the language. Given this assumption, one might wonder why it's been so difficult to arrive at a consensus. Discussions have raged off-and-on at times in both comp.lang.python and the python-dev mailing list about how best to implement function decorators. There is no one clear reason why this should be so, but a few problems seem to be most problematic. * Disagreement about where the "declaration of intent" belongs. Almost everyone agrees that decorating/transforming a function at the end of its definition is suboptimal. Beyond that there seems to be no clear consensus where to place this information. * Syntactic constraints. Python is a syntactically simple language with fairly strong constraints on what can and can't be done without "messing things up" (both visually and with regards to the language parser). There's no obvious way to structure this information so that people new to the concept will think, "Oh yeah, I know what you're doing." The best that seems possible is to keep new users from creating a wildly incorrect mental model of what the syntax means. * Overall unfamiliarity with the concept. For people who have a passing acquaintance with algebra (or even basic arithmetic) or have used at least one other programming language, much of Python is intuitive. Very few people will have had any experience with the decorator concept before encountering it in Python. There's just no strong preexisting meme that captures the concept. * Syntax discussions in general appear to cause more contention than almost anything else. Readers are pointed to the ternary operator discussions that were associated with PEP 308 for another example of this. Background ========== There is general agreement that syntactic support is desirable to the current state of affairs. Guido mentioned `syntactic support for decorators`_ in his DevDay keynote presentation at the `10th Python Conference`_, though `he later said`_ it was only one of several extensions he proposed there "semi-jokingly". `Michael Hudson raised the topic`_ on ``python-dev`` shortly after the conference, attributing the initial bracketed syntax to an earlier proposal on ``comp.lang.python`` by `Gareth McCaughan`_. .. _syntactic support for decorators: http://www.python.org/doc/essays/ppt/python10/py10keynote.pdf .. _10th python conference: http://www.python.org/workshops/2002-02/ .. _michael hudson raised the topic: http://mail.python.org/pipermail/python-dev/2002-February/020005.html .. _he later said: http://mail.python.org/pipermail/python-dev/2002-February/020017.html .. _gareth mccaughan: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=slrna40k88… Class decorations seem like an obvious next step because class definition and function definition are syntactically similar. The discussion continued on and off on python-dev from February 2002 through July 2004. Hundreds and hundreds of posts were made, with people proposing many possible syntax variations. Guido took a list of proposals to `EuroPython 2004`_, where a discussion took place. Subsequent to this, he decided that for 2.4a2 we'd have the `Java-style`_ @decorator syntax. Barry Warsaw named this the 'pie-decorator' syntax, in honor of the Pie-thon Parrot shootout which was announced about the same time as the decorator syntax, and because the @ looks a little like a pie. Guido `outlined his case`_ on Python-dev, including `this piece`_ on the various rejected forms. .. _EuroPython 2004: http://www.python.org/doc/essays/ppt/euro2004/euro2004.pdf .. _outlined his case: http://mail.python.org/pipermail/python-dev/2004-August/author.html .. _this piece: http://mail.python.org/pipermail/python-dev/2004-August/046672.html .. _Java-style: http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html On the name 'Decorator' ======================= There's been a number of complaints about the choice of the name 'decorator' for this feature. The major one is that the name is not consistent with its use in the `GoF book`_. The name 'decorator' probably owes more to its use in the compiler area -- a syntax tree is walked and annotated. It's quite possible that a better name may turn up. .. _GoF book: http://patterndigest.com/patterns/Decorator.html Design Goals ============ The new syntax should * work for arbitrary wrappers, including user-defined callables and the existing builtins ``classmethod()`` and ``staticmethod()``. This requirement also means that a decorator syntax must support passing arguments to the wrapper constructor * work with multiple wrappers per definition * make it obvious what is happening; at the very least it should be obvious that new users can safely ignore it when writing their own code * be a syntax "that ... [is] easy to remember once explained" * not make future extensions more difficult * be easy to type; programs that use it are expected to use it very frequently * not make it more difficult to scan through code quickly. It should still be easy to search for all definitions, a particular definition, or the arguments that a function accepts * not needlessly complicate secondary support tools such as language-sensitive editors and other "`toy parser tools out there`_" * allow future compilers to optimize for decorators. With the hope of a JIT compiler for Python coming into existence at some point this tends to require the syntax for decorators to come before the function definition * move from the end of the function, where it's currently hidden, to the front where it is more `in your face`_ Andrew Kuchling has links to a bunch of the discussions about motivations and use cases `in his blog`_. Particularly notable is `Jim Huginin's list of use cases`_. .. _toy parser tools out there: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&selm=mailman.10… .. _in your face: http://mail.python.org/pipermail/python-dev/2004-August/047112.html .. _in his blog: http://www.amk.ca/diary/archives/cat_python.html#003255 .. _Jim Huginin's list of use cases: http://mail.python.org/pipermail/python-dev/2004-April/044132.html Current Syntax ============== The current syntax for function decorators as implemented in Python 2.4a2 is:: @dec2 @dec1 def func(arg1, arg2, ...): pass This is equivalent to:: def func(arg1, arg2, ...): pass func = dec2(dec1(func)) without the intermediate assignment to the variable ``func``. The decorators are near the function declaration. The @ sign makes it clear that something new is going on here. The decorator statement is limited in what it can accept -- arbitrary expressions will not work. Guido preferred this because of a `gut feeling`_. .. _gut feeling: http://mail.python.org/pipermail/python-dev/2004-August/046711.html Syntax Alternatives =================== There have been `a large number`_ of different syntaxes proposed -- rather than attempting to work through these individual syntaxes, it's worthwhile to break the syntax discussion down into a number of areas. Attempting to discuss `each possible syntax`_ individually would be an act of madness, and produce a completely unwieldy PEP. .. _a large number: http://www.python.org/moin/PythonDecorators .. _each possible syntax: http://ucsu.colorado.edu/~bethard/py/decorators-output.py Decorator Location ------------------ The first syntax point is the location of the decorators. For the following examples, we use the @syntax used in 2.4a2. Decorators before the def statement are the first alternative, and the syntax used in 2.4a2:: @classmethod def foo(arg1,arg2): pass @accepts(int,int) @returns(float) def bar(low,high): pass There have been a number of objections raised to this location -- the primary one is that it's the first real Python case where a line of code has a result on a following line. The syntax that will be in 2.4a3 will also require one decorator per line (in a2, multiple decorators can be specified on the same line). People also complained that the syntax got unworldly quickly when multiple decorators were used. The point was made, though, that the chances of a large number of decorators being used on a single function were small and thus this was not a large worry. Some of the advantages of this form are that the decorators live outside the method body -- they are obviously executed at the time the function is defined. Another advantage is that being prefix to the function definition fit the idea of knowing about a change to the semantics of the code before the code itself, thus knowing how to interpret the code's semantics properly without having to go back and change your initial perceptions if the syntax did not come before the function definition. Guido decided `he preferred`_ having the decorators on the line before the 'def', because it was felt that a long argument list would mean that the decorators would be 'hidden' .. _he preferred: http://mail.python.org/pipermail/python-dev/2004-March/043756.html The second form is the decorators between the def and the function name, or the function name and the argument list:: def @classmethod foo(arg1,arg2): pass def @accepts(int,int),@returns(float) bar(low,high): pass def foo @classmethod (arg1,arg2): pass def bar @accepts(int,int),@returns(float) (low,high): pass There are a couple of objections to this form. The first is that it breaks easily 'greppability' of the source -- you can no longer search for 'def foo(' and find the definition of the function. The second, more serious, objection is that in the case of multiple decorators, the syntax would be extremely unwieldy. The next form, which has had a number of strong proponents, is to have the decorators between the argument list and the trailing ``:`` in the 'def' line:: def foo(arg1,arg2) @classmethod: pass def bar(low,high) @accepts(int,int),@returns(float): pass Guido `summarized the arguments`_ against this form (many of which also apply to the previous form) as: - it hides crucial information (e.g. that it is a static method) after the signature, where it is easily missed - it's easy to miss the transition between a long argument list and a long decorator list - it's cumbersome to cut and paste a decorator list for reuse, because it starts and ends in the middle of a line .. _summarized the arguments: http://mail.python.org/pipermail/python-dev/2004-August/047112.html The next form is that the decorator syntax go inside the method body at the start, in the same place that docstrings currently live: def foo(arg1,arg2): @classmethod pass def bar(low,high): @accepts(int,int) @returns(float) pass The primary objection to this form is that it requires "peeking inside" the method body to determine the decorators. In addition, even though the code is inside the method body, it is not executed when the method is run. Guido felt that docstrings were not a good counter-example, and that it was quite possible that a 'docstring' decorator could help move the docstring to outside the function body. The final form is a new block that encloses the method's code. For this example, we'll use a 'decorate' keyword, as it makes no sense with the @syntax. :: decorate: classmethod def foo(arg1,arg2): pass decorate: accepts(int,int) returns(float) def bar(low,high): pass This form would result in inconsistent indentation for decorated and undecorated methods. In addition, a decorated method's body would start three indent levels in. Syntax forms ------------ * ``@decorator``:: @classmethod def foo(arg1,arg2): pass @accepts(int,int) @returns(float) def bar(low,high): pass The major objections against this syntax are that the @ symbol is not currently used in Python (and is used in both IPython and Leo), and that the @ symbol is not meaningful. Another objection is that this "wastes" a currently unused character (from a limited set) on something that is not perceived as a major use. * ``|decorator``:: |classmethod def foo(arg1,arg2): pass |accepts(int,int) |returns(float) def bar(low,high): pass This is a variant on the @decorator syntax -- it has the advantage that it does not break IPython and Leo. Its major disadvantage compared to the @syntax is that the | symbol looks like both a capital I and a lowercase l. * list syntax:: [classmethod] def foo(arg1,arg2): pass [accepts(int,int), returns(float)] def bar(low,high): pass The major objection to the list syntax is that it's currently meaningful (when used in the form before the method). It's also lacking any indication that the expression is a decorator. * list syntax using other brackets (``<...>``, ``[[...]]``, ...):: <classmethod> def foo(arg1,arg2): pass <accepts(int,int), returns(float)> def bar(low,high): pass None of these alternatives gained much traction. The alternatives which involve square brackets only serve to make it obvious that the decorator construct is not a list. They do nothing to make parsing any easier. The '<...>' alternative presents parsing problems because '<' and '>' already parse as un-paired. They present a further parsing ambiguity because a right angle bracket might be a greater than symbol instead of a closer for the decorators. * ``decorate()`` The ``decorate()`` proposal was that no new syntax be implemented -- instead a magic function that used introspection to manipulate the following function. Both Jp Calderone and Philip Eby produced implementations of functions that did this. Guido was pretty firmly against this -- with no new syntax, the magicness of a function like this is extremely high: Using functions with "action-at-a-distance" through sys.settraceback may be okay for an obscure feature that can't be had any other way yet doesn't merit changes to the language, but that's not the situation for decorators. The widely held view here is that decorators need to be added as a syntactic feature to avoid the problems with the postfix notation used in 2.2 and 2.3. Decorators are slated to be an important new language feature and their design needs to be forward-looking, not constrained by what can be implemented in 2.3. * new keyword (and block) This idea was the consensus alternate from comp.lang.python. Robert Brewer wrote up a detailed `J2 proposal`_ document outlining the arguments in favor of this. The issues with this form are that it requires a new keyword (and therefore a ``from __future__ import decorators`` statement), it seems like there is no obvious choice for the keyword (``using`` is the choice of the proposal and implementation), and that the form produces something that looks like a normal code block, but isn't. Attempts to use statements in this block will cause a syntax error. It's also going to potentially confuse users who don't expect a block to contain bare expressions. .. _J2 proposal: http://www.aminus.org/rbre/python/pydec.html There are plenty of other variants and proposals on `the wiki page`_. .. _the wiki page: http://www.python.org/moin/PythonDecorators Why @? ------ There is some history in Java using @ initially as a marker in `Javadoc comments`_ and later in Java 1.5 for `annotations`_, which are similar to Python decorators. The fact that @ was previously unused as a token in Python also means it's clear there is no possibility of such code being parsed by an earlier version of Python, leading to possibly subtle semantic bugs. It also means that ambiguity of what is a decorator and what isn't is removed. of That said, @ is still a fairly arbitrary choice. Some have suggested using | instead. For syntax options which use a list-like syntax (no matter where it appears) to specify the decorators a few alternatives were proposed: ``[|...|]``, ``*[...]*``, and ``<...>``. .. _Javadoc comments: http://java.sun.com/j2se/javadoc/writingdoccomments/ .. _annotations: http://java.sun.com/j2se/1.5.0/docs/guide/language/annotations.html Current Implementation, History =============================== Guido asked for a volunteer to implement his preferred syntax, and Mark Russell stepped up and posted a `patch`_ to SF. The syntax accepted for 2.4a2 is:: @dec2 @dec1 def func(arg1, arg2, ...): pass This is equivalent to:: def func(arg1, arg2, ...): pass func = dec2(dec1(func)) though without the intermediate creation of a variable named ``func``. A `previous patch`_ from Michael Hudson which implements the list-after-def syntax is also still kicking around. .. _patch: http://www.python.org/sf/979728 .. _previous patch: http://starship.python.net/crew/mwh/hacks/meth-syntax-sugar-3.diff After 2.4a2 was released, in response to community reaction, Guido stated that he'd re-examine a community proposal, if the community could come up with a community consensus, a decent proposal, and an implementation. After an amazing number of posts, collecting a vast number of alternatives in the `Python wiki`_, the proposed J2 syntax (the new keyword ``using`` in a block before the def). Robert Brewer wrote a `detailed proposal`_ for this form, and Michael Sparks produced `a patch`_. As at time of writing, we're waiting for Guido's decision. .. _Python wiki: http://www.python.org/moin/PythonDecorators .. _detailed proposal: http://www.aminus.org/rbre/python/pydec.html .. _a patch: http://www.python.org/sf/1013835 Examples ======== Much of the discussion on ``comp.lang.python`` and the ``python-dev`` mailing list focuses on the use of decorators as a cleaner way to use the ``staticmethod()`` and ``classmethod()`` builtins. This capability is much more powerful than that. This section presents some examples of use. 1. Define a function to be executed at exit. Note that the function isn't actually "wrapped" in the usual sense. :: def onexit(f): import atexit atexit.register(f) return f @onexit def func(): ... Note that this example is probably not suitable for real usage, but is for example purposes only. 2. Define a class with a singleton instance. Note that once the class disappears enterprising programmers would have to be more creative to create more instances. (From Shane Hathaway on ``python-dev``.) :: def singleton(cls): instances = {} def getinstance(): if cls not in instances: instances[cls] = cls() return instances[cls] return getinstance @singleton class MyClass: ... 3. Add attributes to a function. (Based on an example posted by Anders Munch on ``python-dev``.) :: def attrs(**kwds): def decorate(f): for k in kwds: setattr(f, k, kwds[k]) return f return decorate @attrs(versionadded="2.2", author="Guido van Rossum") def mymethod(f): ... 4. Enforce function argument and return types. Note that this copies the func_name attribute from the old to the new function. func_name was made writable in Python 2.4a3:: def accepts(*types): def check_accepts(f): assert len(types) == f.func_code.co_argcount def new_f(*args, **kwds): for (a, t) in zip(args, types): assert isinstance(a, t), \ "arg %r does not match %s" % (a,t) return f(*args, **kwds) new_f.func_name = f.func_name return new_f return check_accepts def returns(rtype): def check_returns(f): def new_f(*args, **kwds): result = f(*args, **kwds) assert isinstance(result, rtype), \ "return value %r does not match %s" % (result,rtype) return result new_f.func_name = f.func_name return new_f return check_returns @accepts(int, (int,float)) @returns((int,float)) def func(arg1, arg2): return arg1 * arg2 5. Declare that a class implements a particular (set of) interface(s). This is from a posting by Bob Ippolito on ``python-dev`` based on experience with `PyProtocols`_. :: def provides(*interfaces): """ An actual, working, implementation of provides for the current implementation of PyProtocols. Not particularly important for the PEP text. """ def provides(typ): declareImplementation(typ, instancesProvide=interfaces) return typ return provides class IBar(Interface): """Declare something about IBar here""" @provides(IBar) class Foo(object): """Implement something here...""" .. _PyProtocols: http://peak.telecommunity.com/PyProtocols.html Of course, all these examples are possible today, though without syntactic support. Open Issues =========== 1. It's not yet certain that class decorators will be incorporated into the language at this point. Guido expressed skepticism about the concept, but various people have made some `strong arguments`_ (search for ``PEP 318 -- posting draft``) on their behalf in ``python-dev``. .. _strong arguments: http://mail.python.org/pipermail/python-dev/2004-March/thread.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End:

3 5