Mailman 3 August 2004 - Python-Dev

Decorator syntax J2 (decorate..def), with implementation
by Michael Sparks Aug. 15, 2004

Aug. 15, 2004

Hi, I suspect people are getting sick of the decorator discussion by now, so I'll try to be brief, and useful. Syntax option J2 from the wiki seems to have provoked an interesting discussion[1] at least by a small subset of c.l.p, so I wondered how difficult it would be to implement - this turned out to be relatively simple [2]. [1] http://mail.python.org/pipermail/python-list/2004-August/233413.html [2] http://mail.python.org/pipermail/python-list/2004-August/233591.html (implementation) Example of the syntax... (I'm sure it's familiar...) #Syntax J2 from http://www.python.org/moin/PythonDecorators #Some random comment #More random comment decorate: staticmethod grammarrule('statement : expression') versioninfo("Added in 2.4") deprecated typeinfo(None) def p_statement_expr(self, p): print p[1] For comparison, this is intended as equivalent to: #Current syntax in 2.4a2 #Some random comment #More random comment @staticmethod @grammarrule('statement : expression') @versioninfo("Added in 2.4") @deprecated @typeinfo(None) def p_statement_expr(self, p): print p[1] With the exact semantics and restrictions remaining the same - this is just syntactic sugar. This was my first time looking at the python source to see how easy it would be to implement - so it's not perfect, and probably far from ideal. My reasons I looking and giving it a go though were: * To me it looks much clearer - you can instantly see where the method (and decorator block) starts. * Lack of repetition of the decoration hint * It hints to a non-python programmer very clearly that this is something different. If they're aware of the decorator patterm, they will realise (perhaps) what it means, and if they don't they have something to google for. I have read most of the arguments against this syntax, and if the decision ends up to stay with @ or take a different syntax, I'm sure it'll work out fine :) (This isn't my favourite syntax, my favourite has a keyword inside the function suite, which I'm well aware is dead in the water!) The post I made to c.l.p with a first attempt at an implementation (has minor issues regarding scopes, but does work) can be found here: * http://mail.python.org/pipermail/python-list/2004-August/233591.html If there is interest in this syntax, then I suspect/hope making it suitable for a "proper" patch would be relatively simple. In case people are wondering what the grammar change I made for this is, I'll include here: -decorator: '@' dotted_name [ '(' [arglist] ')' ] +decorator: dotted_name [ '(' [arglist] ')' ] decorators: decorator ([NEWLINE] decorator)* NEWLINE -funcdef: [decorators] 'def' NAME parameters ':' suite +funcdef: ['decorate' ':' NEWLINE INDENT decorators DEDENT ] 'def' NAME parameters ':' suite parameters: '(' [varargslist] ')' Downsides here: * Looks like a suite, implying it allows arbitrary statements, but isn't * Introduces a new keyword * Some people have expressed reservations about the syntax, some of which also apply to the @pie syntax, and some which don't. * The patch above has scope issues, which means currently it requires some cruft to force functions into scope. Biggest upside from a personal perspective though is this: (aside from others) * Forces things like the common cases of classmethod and staticmethod to jump out at a maintainer. As the wiki mentions, this two line syntax is relatively heavyweight for these cases, and I agree. I disagree about this being bad - static methods and class methods are rare. Having a BIG indication they exist when you have (say) 20 methods in a class strikes me as a very positive thing, and is surely 90% of the reason for moving from the existing mechanism of: foo = staticmethod(foo) Example tested: class Foo: staticmethod # This is cruft to force staticmethod into scope decorate: staticmethod def hello(who): print "woo?", who Anyway, hope this is considered useful, rather than extending the headache! (However, I am for one very happy that decorators are now available in something clearer than we had in pre(2.4) and it certainly made implementing the syntax/semantics above relatively trivial :) Regards, Michael. -- "The 'nice' thing about decorator syntaxes is there are so many to choose from."

6 6

block-based decorators and other block issues
by Martin Zarate Aug. 15, 2004

Aug. 15, 2004

Our discussion has taken an interesting turn, leading to this concept of "super-decorators" that are more versatile than normal decorators, because they can accept multiple objects, and because they can play with namespaces. First of all, it is important not to get too far ahead of ourselves - decorators should not be full "custom blocks" like having a "threadsafe" decorator block and a "fork function" block wherein the decoratotor is changing the fundamental behaviour of a block of code. Concepts that we see in Ruby (I highly, highly recommend that people interested in this subject read over the Ruby tutorial, it is very insightful on the concept of versatile block constructs) could be brought to Python, but it would have to be done properly through a good general-purpose mechanism. Decorators are simply a filtering mechanism - they take an object defined with the decorator, and then either a) return a substitute to the parent scope or, possibly b) throw it out. They are not blocking concepts. The reason is simple - speed. A decorator shouldn't have to perform fasttolocals and vice versa. It doesn't need to be its own variable namespace - it can sit within the external namespace. To do otherwise is suboptimal. The decorator should not have its own scope. Instead, it should provide a simple filter. Now, it would be nice if Python had another versatile block structure usable for custom stuff that does involve having its own full namespace (hell, just a "block" keyword would do) but I'll get to that later. First, I need a keyword. I will use "turn", just to keep things different, but anything will do - has has been remarked elsewhere, a naked tuple with a ":" suffix would be enough - it would differentiate the conversion block from standard comma-delimited lists, while not looking like any other block. At any rate, people advocating the "decorator block" concept that I suggested keep assuming that the descriptor functions are functions that take a whole namespace in and put a whole other namespace out, which is then imported into the parent namespace - horribly inefficient. Use that later for custom blocks. The call signature of a descriptor could simply be descriptor(object, name, holder, doc, args) where object is the object being described, name is the string with its name (immutable, as always) and holder is the current contents of the holder. The return value of the descriptor function is assigned into the holder at the end of the operation - if no change is desired and the nested objects are to be thrown out, then simply return the holder parameter. The name parameter exists for the sake of naming objects. With the holder parameter, inspection of the existing object is allowed, so that the "property_get" descriptor (which actually alters an existing property or creates a new one if none exists) can work. The reason for the longer function signature is so that more complicated syntax is available to the decorator function - such as a docstring that can be prepended to the nested docstrings. This seems silly for functions, but would be appropriate when decorating non-function objects (such as data members). Creating a new scope block for decorators would be too slow, but still would be very useful for more bizarre custom linguistic constructs. A general purpose "scope" command with access to external and internal namespaces would be extremely useful. Imagine the general purpose "scope" statement: scope handler1(args), handler2(args), handler3(args): where the contained code within the block would be run, then its local namespace harvested, as well as the external namespace, both passed in to handlerN(inner, outer, args) this would pretty much let you do whatever you want within a random scope: block, allowing for some bizzarro macroesque stuff that would make descriptors look simple by comparison. Personally I think such a structure would be silly and dangerous, but its what many coders sound like they want. A better, more general-purpose structure would be an "object" block like object name handler(args): wherein the contained code is run on the spot and the local namespace is converted into a dictionary, which is then called with the signature handler(namespace, name, args) where namespace is the namespace is the local namespace, name is the string of the name, and args is the argument tuple. You could use this to code your own analogs to the class structure of Python. Say you want a forced single- inheritence-tree class system? You could have one with such custom blocks, or any other alternate-class-structure. Hell, with such a system the class itself wouldn't need to be a true keyword as it is now, but instead could behave as simply just one more "object" expression handler - as I understand it, all it does is take that namespace and the argument tuple and do some fun stuff with it and then plonk that info (plus the name of the class) into the "type" constructor. That doesn't need to be interpreter level. So instead of class foo(bar baz): we get object foo class(bar baz): which could be substituted with any wierd thing you want instead of "class" - like a quick module object creator with none of the overhead of a class, for example. Or you could do the much coveted "instance" block that many people have clamored for, in which you subclass an instance instead of subclassing a class. Or lets say you're making a game engine and you need to make a separate class system for engine objects (the situation I was in, actually)and you want to be able to work with them as if they were classes, but also want it to inherit some invisible engine settings not apparent to the python interpreter from other engine objects, as well as instead of making some kinds of functions visible to the interpreter they're loaded into the event handler automatically (this is personal experience here - I wanted these features very badly). Or you could just make a quick dict maker def dictblock(namespace, name, args): return namespace; Like sort of the reverse of a "with" block. Take note that this system would be by far the best way to handle the property object, rather than mucking about with descriptors. Works as such... object foo propertyblock: def fget(self): return "bar" def fset(self): pass def fdel(self): del self.foo doc = "baz" (alternately, docstrings could be implemented like functions, filling the __doc__ entry of the dictionary) and implemented like... propertyblock(space, name, args): return property(space["fget"], space["fset"], space["fdel"], space["doc"]) You could do a nice shortcut involving passing the namespace dict into the keywords dict while calling "property", but that's the more legible way. The point is that this would be distinct from the descriptor block concept, as descriptors don't need to play with the namespaces beyond one object and one name at a time. But none of this explains why I like using a block for descriptors - I'm just trying to explain to those who thought of the possibilites of a block where you could play with the local and surrounding namespace how yes, that stuff would be useful, but would have a lot more overhead than simply a descriptor block. There are places a multi-line descriptor block is ugly, but there are also places where it is goregeous. Public and Private are good examples that C++ devs would love, rather than saying public and private for each function its turn private: foo = 1 bar = 2 baz = 3 Hell, if you really don't like the keyword, we could get even wierder and play with a new syntax: use the @ with a bracket and nest the block within Guido's syntax... @[static, private, functor]: def foo: pass Wow, I covered way, way too much ground with that post. Seriously people - read the Ruby tutorial "Programming Ruby" http://www.rubycentral.com/book/ to see the gorgeous way that Ruby handles the whole concept of blocks. Unfortunately the rest of the language is thoroughly illegible, but the block concept is cool.

2 1

Decorators are function/method subclassing technique?
by Roman Suzi Aug. 15, 2004

Aug. 15, 2004

After considering the docstring problem of decorators, it occured to me that decorators could be seen as an inheritance mechanism of functions. class docsafe(function): ... @ docsafe def f(x, y, z): """ To be saved """ ... print f <__main__.docsafe object at 0x401e074c> Sincerely yours, Roman Suzi -- rnd(a)onego.ru =\= My AI powered by GNU/Linux RedHat 7.3

1 0

Decorators and docstrings
by Hans Nowak Aug. 15, 2004

Aug. 15, 2004

Hi, When playing with the decorators in 2.4a2, I saw the following behavior: # wraps an object with the function, and prints this object # when the function is called. >>> def wrapwith(obj): ... def decorator(f): ... def _wrapper(*args, **kwargs): ... print "##", obj ... return f(*args, **kwargs) ... return _wrapper ... return decorator ... >>> @wrapwith("hello") ... def foo(x, y): ... """ foo! """ ... return 2*x + y ... >>> foo(4, 5) ## hello 13 >>> foo.__doc__ >>> I.e. if you wrap the function foo with the decorator, it loses the docstring. foo.__doc__ is empty. Is this behavior intended? Should the decorator in question (in this case, 'wrapwith') take care of copying of the docstring itself? Or should the decorator mechanism do this automatically (if possible)? Thanks, -- Hans Nowak (hans(a)zephyrfalcon.org) http://zephyrfalcon.org/

3 2

Question about win32com in MS-Excel
by Jiang Yu Aug. 14, 2004

Aug. 14, 2004

Hello, My name is Jiangyu, a Chinese man. You can call me Joe. I learn Python for only one year. I think it's a simple tools in my job some time. But in my country, Python is a new tools for many people. So If i have some problems on it, I found it's difficult to look for the answers。So I add the this mail list. Now, I have a question about use win32com moudle for MS-Excel. Question: I want to insert a file into a Excel sheet. But a exception occured in my code. Using VBA, I insert a file OK. And the processing in Python is equivalent in VBA. My environment: Python2.2.3 + win32com + Win2000 Server + Excel(Office XP) My Python code: >>> from win32com.client import Dispatch >>> excel = Dispatch("Excel.Application") >>> excel.Visible = 1 >>> excel.WorkBooks.Add() >>> cursheet = excel.ActiveWorkBook.ActiveSheet >>> oleobjs = cursheet.OLEObjects() >>> oleobjs.Add(Filename = r"c:\temp.txt") My VBA code: ActiveWorkbook.ActiveSheet.OLEObjects.Add Filename:="c:\temp.txt", Link:=False, DisplayAsIcon:=False So, If you know how to do it. please answer me. Thanks! joe 2004/8/13

2 1

RE: [Python-Dev] Multi-line import implementation (was: 2.4a2, and @decorators)
by Michael Chermside Aug. 14, 2004

Aug. 14, 2004

My patch (attached) accepts > > import (os, sys) > from sys import (stdin, stdout, stderr) > import (os) > from sys import (*) > > but rejects > > from (sys) import stdin > import (os), (sys) > import (os,) > > Should any of those be allowed? Anything that I missed? Others have already mentioned that "from sys import xxx, xxx, xxx" is the case that badly needs line wrapping. I would, however, like to suggest that (if it's easy to do) you allow a trailing comma when parenthesees are used. Of course this looks silly: from mymodule import (xxx, xxx, xxx, xxx,) But in the most extreme case, like this: from myBigModule import ( xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, xxxxxxxx, ) it's awefully nice on cut-and-paste to allow the trailing comma. But the most important reason for doing so is that Python already allows (and disregards) an extra trailing comma in similar lists: >>> [1,2,3,] [1, 2, 3] >>> {1:1,2:2,3:3,} {1: 1, 2: 2, 3: 3} >>> def func(a,b,c,): ... print a, b, c ... >>> func(1,2,3) 1 2 3 >>> (1,2,3,) (1, 2, 3) -- Michael Chermside PS: The length-one-tuple is a special case (and a bit of a wart) in that it MANDATES the use of the trailing comma. But that's got nothing to do with the basic principle.

3 2

def ... decorate
by Skip Montanaro Aug. 14, 2004

Aug. 14, 2004

(posting to python-dev separately from c.l.py so the thread can live or die on its own in both places) I responded on c.l.py to a couple proposals: Steven> decorate: Steven> grammarrule('statement : expression') Steven> versioninfo("Added in 2.4") Steven> deprecated Steven> typeinfo(None) Steven> def p_statement_expr(self, p): Steven> print p[1] Nick> as: Nick> staticmethod Nick> grammarrule('statement : expression') Nick> version("Added in 2.4") Nick> deprecatedmethod Nick> type_(None) Nick> def p_statement_expr(self, p): Nick> print p[1] with def p_statement_expr: staticmethod grammarrule('statement : expression') version("Added in 2.4") deprecatedmethod type_(None) decorate (self, p): """docstring here""" print p[1] It seems different enough from other solutions that I thought it worth tossing out. I didn't see it mentioned in the PythonDecorators moin page. Read it something like "define a function named p_statement_expr using a bunch of functions to decorate the basic function". It solves a couple problems: 1. "def" introduces the function definition instead of an arbitrary number of @-expressions. 2. There is no extra indentation of the main body. 3. The name of the function is known early on. 4. "def"/"decorate" pair up visually much the same as "try"/"except" or "if"/"then", though they don't represent alternative blocks of code to be executed. On the minus side it introduces a vertical separation between the function name and parameter list and introduces a new keyword, "decorate". >From a parsing standpoint I think it will work. You'll see either a colon or a left paren after the function name to distinguish between the two types of function definition. I'm not sure if a token needs to be used to separate the various decorator functions or if requiring a newline and indentation is sufficient. Skip

12 20

New Developer
by Raymond Hettinger Aug. 13, 2004

Aug. 13, 2004

[Johannes Gijsbers] > > Consider this a request for checkin privileges. [amk] > Seconded. Johannes, your permissions are enabled. Welcome. Raymond

1 0

Third Bug Day outcome
by A.M. Kuchling Aug. 13, 2004

Aug. 13, 2004

The bug day on Saturday went well, as usual. 19 bugs and 12 patches were closed. That's not as good as the first bug day (30 bugs), but is competitive with the second (18 bugs, 21 patches). Lots of the easy bugs have been cleaned out, I expect, so each remaining bug takes more time to fix. The composition of the turnout was the surprising thing. My plaintive bleating on python-dev resulted in much higher participation by people with CVS committers, but there weren't many non-committer people around (Seo Sanghyeon and Mike Coleman were the two non-developers I noticed). This is a pity, because bug days seem like a good way to gently introduce more people to hacking on the source. During the day, Armin Rigo wrote a nifty IRC bot that takes an SF bug/patch ID and returns the title; taking an example from the transcript: 20:53:56 <jlgijsbers> Looking at #777659. 20:53:57 <sf_number> * sf_number bug 777659 - Uninitialized variable used in Tools/faqwiz/faqwiz.py I won't be running a bug day in September; if someone else wants to arrange it, please feel free. Otherwise, they'll resume in October. --amk

6 10

Re: def ... decorate
by Gareth McCaughan Aug. 13, 2004

Aug. 13, 2004

Josiah Carlson wrote, in reply to me: > You seem to have missed the point of function decorations entirely. We > already have what you offer in current Python syntax. The point was to > move decorations to near/next to the function signature. No, that was only part of the point. That it's part of the point is why I still prefer the list-before-colon syntax, which seems to me to go furthest towards keeping all information about the interface of a function together. And yes, I know that Guido finds that syntax unacceptable. > Read the PEP: > http://www.python.org/peps/pep-0318.html I have read the PEP, thanks. Heck, I'm even named in it. (I think I was the first person to propose a generalized decoration facility for Python, rather than just special syntax for a few special cases such as static methods. And, for what it's worth, the list-before-colon syntax was my proposal. Not that it matters much who thought of what, but I do think it's a bit off to tell me that I've "completely missed the point" of something I *invented*.) >> I actually quite like this. It seems more Pythonic >> than the @foo proposal. Its obvious problem is that >> it involves something that looks at first glance like >> an ordinary suite of statements or expressions, but >> whose interpretation is substantially different. At >> least the @foo proposal avoids that. > > What you like is what has existed with Python since the beginning. No, it is not. What has existed with Python since the beginning involves redundancy: def f(...): ... f = decorate_somehow(f) requires three instances of the name of f. If you have N decorators it requires 2N+1 instances of the name of f. Not a big deal when the name is "f", but not so good if it's "build_interface_subgraph" or something. Not even great when the name is "f": what if you decide to rename it "g"? *And* it's ugly. These are reasons to want something else, even without the issue of putting all the "interface" information together. -- g

1 0