
A while ago there was a proposal floating around to add an optional part to function/method definitions, that would replace the current clumsy classmethod etc. notation, and could be used for other purposes too. I think the final proposal looked like this: def name(arg, ...) [expr, ...]: ...body... Does anyone remember or know where to find the thread where this proposal was discussed? It ought to be turned into a PEP. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
I don't think there was much discussion. The suggested semantics was that this is equivalent to def name(arg, ...): ...body... name=expr(name) ... I *think* there was discussion as to the namespace in which expr is evaluated, or whether certain identifiers have keyword or predefined meaning (so you can write 'static' instead of 'staticmethod'). I don't think there was ever a complete analysis whether this syntax meets all requirements, i.e. whether you could use it for all newstyle features. In particular, I don't recall what the proposal was how properties should be spelled. Regards, Martin

Martin v. Löwis wrote:
In particular: def name(arg, ...) [expr1, expr2, expr3]: ...body... would be equivalent to (some variation on): def name(arg, ...): ...body... name=expr1(expr2(expr3(name))) I wonder if the same mechanism could be used in class statements. If I had this, I might say what interface a class implements with: class foo(spam, eggs) [implements(IFoo, IBar)]: body of class or (wo parens): class foo [implements(IFoo, IBar)]: body of class Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Jim Fulton wrote:
With Michael's patch (which indeed still works) it's actually name = expr3(expr2(expr1(name)))
I don't know how Zope interfaces work, but I can imagine the following could be made to work as well: class foo(spam, eggs) [IFoo, IBar]: body of class I think this would be wonderful. Would someone be interested in extending Michael's patch to also cover the class statement? Just

Guido van Rossum wrote:
Some other interesting related posts: - http://mail.python.org/pipermail/python-list/2001-July/056224.html - http://mail.python.org/pipermail/python-list/2001-July/056416.html - http://mail.python.org/pipermail/python-dev/2001-July/016287.html
It ought to be turned into a PEP.
John Williams sent a very rough candidate PEP in October that was interesting (below). I sent back some suggestions (below the PEP), but I haven't received anything back yet. Perhaps a joint PEP with syntax alternatives? <pep> PEP: XXX Title: Multiword Method Names Version: $Revision:$ Last-Modified: $Date: 2002/10/09 21:11:59 $ Author: John Williams <jrw@pobox.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 09-Oct-2002 Post-History: Python-Version: 2.3 Abstract ======== This PEP proposes allowing the space character in method names. Motivation ========== With the addition of static and class methods to Python, there are now three distinct kinds of methods. All are declared using the same syntax, and static and class methods are indicated by the use of the built-in "wrapper" types staticmethod and classmethod. This technique is unfortunate because it requires two statements to define a single method and requires the method name to appear three times. These problems could be solved by allowing the space character in method names and adding metaclass support for methods with certain names, in effect allowing arbitrary pseudo-keywords to be added to method declarations. Specification ============= The core of this proposal is to allow an arbitrary sequence of identifiers and keywords to appear between the keyword "def" and the opening parenthesis of the function declaration. The result would be identical to existing "def" statements, except that the name of the new function would would consist of the words joined together with space characters. Although no syntax exists for working with variables whose names contain spaces, the name would be accessible through explicit use of the underlying variable dictionaries. Using the new syntax along with special metaclass support, properties and static and class methods could be declared like this:: class MyClass(object): def class myClassMethod(cls): . . . def static myStaticMethod(): . . . def get myProperty(self): . . . def set myProperty(self, value): . . . The declaration above would be equivalent to: class MyClass(object): def myClassMethod(cls): . . . myClassMethod = classmethod(myClassMethod) def myStaticMethod(): . . . myStaticMethod = staticmethod(myStaticMethod) def __getMyProperty(self): . . . def __setMyProperty(self, value): . . . myProperty = property(__getMyProperty, __setMyProperty) Copyright ========= This document has been placed in the public domain. </pep> Here's my reply to John: <reply> This PEP proposal is interesting, although the idea has come up before (not a bad thing; see references below). The first thing I notice is that the title is misleading. The title and abstract ("This PEP proposes allowing the space character in method names") make me think you're proposing that this kind of declaration would be legal and somehow callable:: def a multi word method name(): pass This would be too large a syntax change IMO. I think the "multiword" aspect and "allowing spaces" are merely side-effects of what the PEP is proposing. They're implementation details. The real issue is the lack of a unifying syntax for descriptors. The text of the PEP is looking at the issue from the wrong direction IMO: from the bottom up (implementation) instead of from the top down (concept). I haven't wrapped my head around descriptors yet, and I'm not familiar with Python's internals. I have no idea if what you're proposing is even feasible. But here are some ideas and suggestions for the PEP: * I think the PEP should be called "Syntax for Descriptors". (I also thought of "Pseudo-Keywords in Method Declarations", but it's not as good a title.) * Rather than "pseudo-keywords", what about calling these tokens "modifiers"? * Perhaps metaclasses could grow a mechanism to add new pseudo-keywords of their own? Maybe not right away. When someone comes up with a novel use for descriptors, it would be nice to be able to implement it with syntax. * Expand the examples to include the "delete" property method. (I guess the modifier can't be "del", since that's already a bona-fide keyword. But then again...) * It may not be feasible to use "class" as a modifier, since it's already a keyword. Perhaps "classmethod", as suggested in one of the threads below. * Add some blanks to the examples to make them easier to read. The same idea was brought up back in July 2001. See the following threads: - http://mail.python.org/pipermail/python-list/2001-July/056224.html (A reply from Guido: http://mail.python.org/pipermail/python-list/2001-July/056416.html) - http://mail.python.org/pipermail/python-dev/2001-July/016287.html There may have been other similar discussions. Guido's reply is discouraging, but he may have changed his mind since. Having a PEP clearly proposing the syntax change would be useful even if it's rejected. I would recommend you revise the PEP and send it to Python-Dev to gather feedback before resubmitting it for a PEP number. </reply> -- David Goodger <http://starship.python.net/~goodger> Python Enhancement Proposal (PEP) Editor <http://www.python.org/peps/> (Please cc: all PEP correspondence to <peps@python.org>.)

This syntax is clear, explicit, and attractive.
def get myProperty(self): . . . def set myProperty(self, value): . . .
This ibe doesn't extend as cleanly: def del myProperty(self, value): "Oops, del is a keyword" def doc myProperty ... ? what goes here Also, can two properties share a getter or setter as they can now? x = property(notifyOneWay, setx) y = propetry(notifyOneWay, sety) Raymond Hettinger ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# #################################################################

David Goodger wrote:
Thanks for bringing this up. I've been following python-dev but I've mostly gotten sidetracked from Python stuff since then. I'm also having second thoughts about the whole idea of my proposal, since I basically wrote it in a fit of excitement over possibilities of metaclasses. Compared to the other proposal going around (which I'll call Guido's, since he brought it up), the really big advantage of my proposal is that you can use it to do something like adding a property to a class implicitly by defining its getter and setter methods: class A(object): def get foo(self): "Getter for property 'foo'." return self.__foo def set foo(self, foo): "Setter for property 'foo'." self.__foo = foo It's critical here that neither of these declarations actually defines the name "foo"; they define names like "get foo" and "set foo", and it's up to the metaclass to expose these methods in a sane way (i.e. by creating a property named "foo" in this case). I agree with the criticisms that others have made about this proposal, but the real killer (for me, at least) is that using method name modifiers would require case-by-case support in the metaclass to achieve the desired effects. Guido's proposal (adding expressions after the argument list) is easy to extend and generally much less magical, since the modifier expressions don't have to have their meaning "interpreted" by a metaclass. At this stage I'd much rather see Guido's proposal implemented, unless someone comes up with a truly ingenious way to combine the advantages of both.

John Williams <jrw@pobox.com> wrote in news:3E300A13.6020303@pobox.com:
<snip>
How about this: class A(object): def foo(self, foo) [property.set]: "Setter for property 'foo'." self.__foo = foo def foo(self) [property.get]: "Getter for property 'foo'." return self.__foo Then add static methods to property that look something like this: def set(fn): if isinstance(fn, property): return property(fn.fget, fn, fn.fdel, fn.__doc__) else: return property(fset=fn) def get(fn): ... def delete(fn): ... -- Duncan Booth duncan@rcp.co.uk int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3" "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

Duncan Booth wrote:
This is beautiful, but it does not work: when defining the getter, you need both the old property, and the new function object. Looking at your code def set(fn): if isinstance(fn, property): return property(fn.fget, fn, fn.fdel, fn.__doc__) you first assume fn is the property object, and then assume it is the setter function. Of course, there is no reason why the namespace-under-construction couldn't be passed to the annotation, but that would be an extension to the protocol. Martin

"Martin v. Löwis" <martin@v.loewis.de> wrote in news:3E312900.90100@v.loewis.de:
Gah!, I must be asleep today. Something like this might work (although it is getting a bit messy): def set(fn): get, delete, doc = None, None, fn.__doc__ namespace = inspect.getcurrentframe().f_back.f_locals oldfn = namespace.get(fn.__name__) if isinstance(oldfn, property): get, delete, doc = oldfn.fget, oldfn.fdel, oldfn.__doc__ return property(get, fn, delete, doc) -- Duncan Booth duncan@rcp.co.uk int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3" "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

Duncan Booth <duncan@rcp.co.uk>:
I think there's something that needs to be decided before going any further with this: Do we want to be able to define and/or override get/set/del methods individually? If so, it would be better to re-design the property mechanism to make it easier, instead of coming up with kludges to fit it on top of the existing mechanism. Whatever the mechanism, here's my current thoughts on a property syntax: def foo.get(self): ... def foo.set(self, x): ... def foo.del(self): ... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:
Do we want to be able to define and/or override get/set/del methods individually?
As opposed to? The rationale for introducing the extended function syntax is that extended functions should be introduced by means of a definition, not of an assignment. For the same reason, I think properties should not be introduced by means of an assignment. I can't picture how *not* to define them individually, unless you are thinking of something like property foo: def get(self): ... def set(self, value): ...
So would you also be in favour of defining static methods through def static foo(): ... ? Regards, Martin

Something like that, yes. The point is that, with the current implementation of properties, any kind of syntactic sugar which *doesn't* group the three methods together somehow goes against the grain.
I'm all in favour of that, but as things stand, the extended function syntax only lends itself well to function filters that take a single function as argument. The attempts I've seen so far to extend it to handle more than one function at once all seem prohibitively ugly to me. This means that either (a) we shouldn't try to use the extended function syntax for properties, or (b) properties need to be re-designed so that they fit the extended function syntax better.
I wouldn't object to it. I also wouldn't object to using the extended function syntax for static and class methods. I just don't want to see some horrible kludge stretching the extended function syntax to places it doesn't naturally want to go. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Greg Ewing wrote:
Is that a dislike towards the notation, or towards the implementation strategy. I agree that an implementation using getframe is ugly. However, I do think that the proposed notation is natural, and that there is a clean implementation for it, too (just provide the filter with a reference to the namespace-under-construction). Regards, Martin

Martin Loewis:
Is that a dislike towards the notation, or towards the implementation strategy.
The notation is the important thing to get right, I suppose, since the implementation can be changed if need be.
there is a clean implementation for it, too (just provide the filter with a reference to the namespace-under-construction).
Yes, I can see that now (I was thinking that the property mechanism itself would need changing, but it wouldn't). But even so, I don't think it really works all that well for properties. There would be something distinctly odd about writing this sort of thing: def foo(self) [get_property]: ... def foo(self, x) [set_property]: ... because it looks like you're defining two things both called "foo". There would be too much hidden magic going on there for my taste. I've just had another thought: This would also make it hard to do a text-editor search to answer questions such as "where is the get-function for the foo property defined". With my proposal you could search for "def foo.get". Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

"GE" == Greg Ewing <greg@cosc.canterbury.ac.nz> writes:
GE> Whatever the mechanism, here's my current thoughts GE> on a property syntax: | def foo.get(self): | ... | def foo.set(self, x): | ... | def foo.del(self): | ... Interesting. I've always wanted to be able to write... class Foo: ... def Foo.baz(self, a, b, c): ... -Barry

Yes, that would be nice. It would be incompatible with that idea, unfortunately. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

The itertools module is ready for comment and review. It implements ten high speed, memory efficient looping constructs inspired by Haskell and SML. It is ready-to-run and packaged with setup.py, a news item, docs, and unittests. The files are in the sandbox at python/nondist/sandbox/itertools. If you see a typo or clear error, feel free to edit the files directly. If you don't feel like reading the C code, the docs list all known issues and include pure python equivalent code for each function. Let me know if I omitted your favorite function (tabulate, partition, etc). Raymond Hettinger

Raymond> If you see a typo or clear error, feel free to edit the files Raymond> directly. If you don't feel like reading the C code, the docs Raymond> list all known issues and include pure python equivalent code Raymond> for each function. Let me know if I omitted your favorite Raymond> function (tabulate, partition, etc). (Note, I've never used Haskell or SML, so have no direct experience with any of these iterators.) I fixed a couple typos, but have a few (more subjective) comments: * islice() - The description seems a bit confusing to me - perhaps a simple example would be useful. * takewhile()/dropwhile() - I assume these only return a prefix of their iterable arguments. Dropwhile()'s help suggests that, but takewhile()'s description is more vague about the notion. * imap() - It's not clear to me why it differs from map() other than the fact that it's an iterator. Can you motivate why it stops when the shortest iterable is exhausted and doesn't accept Non for its func arg? * loopzip() - It's not clear why its next() method should return a list instead of a tuple (again, a seemingly needless distiction with its builtin counterpart, zip()). * starmap() - How does it differ from imap() and map()? * times() - Why not declare times to take an optional argument to be returned? (In general, it seems like count(), repeat() and times() overlap heavily. Examples of their usage might be helpful in understanding when to use them, or at least when they are commonly used in their Haskell/SML roots environments.) Skip

I fixed a couple typos, but have a few (more subjective) comments:
Thanks for the rapid review.
* islice() - The description seems a bit confusing to me - perhaps a simple example would be useful.
I'll clarify the docs and add simple examples for each function. islice is one of the more powerful functions: for line in islice(afile, 10, 20, 2): print line # Starting with line 10, prints every other line # upto but not including line 20. nth = lambda iterable, n: islice(iterable,n,n+1).next() # get the nth item
I'll clarify the docs and add clear examples.
* imap() - It's not clear to me why it differs from map() other than the fact that it's an iterator.
The other differences are that it stops with the shortest iterable and doesn't accept None for a func argument.
Because one or more useful inputs are potentially infinite, filling in Nones is less useful than stopping with the shortest iterator. The function doesn't accept None for a function argument for 1) simplicity 2) we have zip() for that purpose OTOH, if it is important to someone, I can easily re-embed that functionality.
I've wrestled with the one. The short answer is that zip() already does a pretty good job and that the only use for loopzip() is super high speed looping. To that end, reusing a single list instead of allocating and building tuples is *much* faster.
* starmap() - How does it differ from imap() and map()?
for computing a operator.pow, if your data looks like this: b=[2,3,5] p=[3,5,7], then use imap(operator.pow, a, b) OTOH, if your data looks like this: c =[(2,3), (3,5), (5,7)], then use starmap(operator.pow, c) Essentially, it's the difference between f(a,b) and f(*c).
* times() - Why not declare times to take an optional argument to be returned?
The use case is for looping when you don't care about the value: for i in itertools.times(3): print "hello"
Yes. I opted for using the atomic combinable building blocks rather than constructing the more elaborate things like tabulate(f) which can easily be made from the basic pieces: imap(f,count()) I'll add more examples so that the usage becomes more obvious. Thanks again for the review. Raymond ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# #################################################################

>> * imap() - It's not clear to me why it differs from map() other than >> the fact that it's an iterator. Raymond> The other differences are that it stops with the shortest Raymond> iterable and doesn't accept None for a func argument. I understand that. I was questioning why with a name like "imap" you chose to make it differ from map() in ways other than its iterator-ness. The other semantic differences make it more difficult to replace map() with itertools.imap() than it might be. Raymond> Because one or more useful inputs are potentially infinite, Raymond> filling in Nones is less useful than stopping with the shortest Raymond> iterator. Yes, but it still seems a gratuitous change from map() to me. >> * loopzip() - It's not clear why its next() method should return a >> list instead of a tuple (again, a seemingly needless distiction >> with its builtin counterpart, zip()). Raymond> I've wrestled with the one. The short answer is that zip() Raymond> already does a pretty good job and that the only use for Raymond> loopzip() is super high speed looping. To that end, reusing a Raymond> single list instead of allocating and building tuples is *much* Raymond> faster. How do you know the caller doesn't squirrel away the list you returned on the n-th iteration? I don't see how you can safely reuse the same list. Skip

Okay, it's no problem to put back in the function=None behavior. I had thought it an outdated hack that could be left behind, but there is no loss from including it. And, you're right, it may help someone transition their code a little more easily.
I understand; however, for me, replicating quirks of map ranks less in importance than creating a cohesive set of tools that work well together. The SML/Haskell tools have a number of infinite iterators as basic building blocks; using them requires that other functions know when to shut off. I would like the package to be unified by the idea that the iterators all terminate with shortest input (assuming they have one and some don't). Also, I'm a little biased because that map feature has never been helpful to me and more than once has gotten in the way. The implementations in Haskell and SML also did not include a None fillin feature.
If needed, I can add in an izip() function that returns tuples just like zip() does. I would like to keep loopzip(). It is very effective and efficient for the use case that zip was meant to solve, namely lockstep iteration: for i, j in loopzip(ivector, jvector): # results are immediately unpacked process(i,j) This use case is even more prevalent with this package where loopzip can combine algebraicly with other itertools or functionals: takewhile(binarypredicate, loopzip(ivec, jvec) It's a terrible waste to constantly allocate tuples, build them, pass them, unpack them, and throw them away on every pass. Reuse is an optimization that is already built into the existing implementations of filter() and map().
Skip
Thanks again for the useful comments. I'll add the map(None, s1, s2, ...) behavior and write an izip() function which can be used with full safety for non-looping use cases. Raymond Hettinger

From: "Skip Montanaro" <skip@pobox.com>
After more thought, I think loopzip() is too unsavory and taints an otherwise clean package, so I'll take it out unless someone wants to stand-up for it. Too bad, it was an exceptionally fast solution to the lock-step iteration problem. Raymond

On Mon, Jan 27, 2003 at 06:32:22PM +1300, Greg Ewing wrote:
The statement "def foo(...):" is just a shortcut for the assignment "foo=new.function(CODE, GLOBS, 'foo')". So why not make "def foo.bar(...):" a shortcut for "foo.bar=new.function(CODE, GLOBS, 'foo.bar')" ? If the attributes of a property object are made assignable the following code will work: prop = property() def prop.fget(self): ... def prop.fset(self, value): ... This will also allow creating classes the way Barry suggested or creating new kinds of namespaces. "Namespaces are one honking great idea -- let's do more of those!"-ly yours, Oren

"OT" == Oren Tirosh <oren-py-d@hishome.net> writes:
OT> The statement "def foo(...):" is just a shortcut for the OT> assignment "foo=new.function(CODE, GLOBS, 'foo')". So why not OT> make "def foo.bar(...):" a shortcut for OT> "foo.bar=new.function(CODE, GLOBS, 'foo.bar')" ? OT> If the attributes of a property object are made assignable the OT> following code will work: +1 -Barry

Oren Tirosh <oren-py-d@hishome.net> writes:
So we're around to suggesting def <arbitrary-rvalue>(params): ... ? Hey, here's a way of writing switches: dispatch = {} def dispatch['a'](a): print 'it's a' ... dispatch[var](param) Not at all sure if I like this or not. It's quite a change. Cheers, M. -- "An infinite number of monkeys at an infinite number of keyboards could produce something like Usenet." "They could do a better job of it." -- the corollaries to Gene Spafford's Axiom #2 of Usenet

Oren Tirosh <oren-py-d@hishome.net> writes:
Well, not entirely. I was more trying to tease out of you what you intended.
Just in case you are not: a fully-qualified name consists of identifiers separated by dots.
I hadn't seen the phrase "fully-qualified name" used in this thread; it wasn't obvious to me that any such limitation was implied. I may have missed something, of course.
It's not an arbitrary expression. At least that's how it works in the Python module namespace.
But as a generalisation of "foo=new.function(CODE, GLOBS, 'foo')", what I mentioned would be valid too. I'm not sure it's *sensible*, but that's a different kettle of fish... Cheers, M. -- ... Windows proponents tell you that it will solve things that your Unix system people keep telling you are hard. The Unix people are right: they are hard, and Windows does not solve them, ... -- Tim Bradshaw, comp.lang.lisp

Oren Tirosh <oren-py-d@hishome.net>:
I'm not sure I like the need for the prop = property() part. It exposes too much of the implementation for my taste. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Greg Ewing wrote:
What happened to Guido's proposal? class Foo(object): property myprop: """A computed property on Foo objects.""" def __get__(self): return ... def __set__(self): ... def __delete__(self): ... (see http://mail.python.org/pipermail/python-dev/2002-February/020081.html) Bye, Walter Dörwald

Just van Rossum <just@letterror.com> writes:
I'm pretty sure you can do this with a metaclass, but I've just spent half an hour thoroughly confusing myself trying to do so. I'm not sure I like using 'class' to introduce a propery, FWIW. Cheers, M. -- First of all, email me your AOL password as a security measure. You may find that won't be able to connect to the 'net for a while. This is normal. The next thing to do is turn your computer upside down and shake it to reboot it. -- Darren Tucker, asr

Just van Rossum <just@letterror.com>:
-0, too clever. (I wouldn't mind if it was *equivalent* to that, but I wouldn't like to have to write it out like that all the time.) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Seems to me like the following should work for this in 2.2 and beyond:: #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #~ In some common file... class Property(object): def __init__(prop, __doc__ = ''): prop.__doc__ = __doc__ def __get__(prop, self, klass): if self is None: return klass else: return prop.Get(self) def __set__(prop, self, value): prop.Set(self, value) def __delete__(prop, self): prop.Del(self) #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #~ To use... class Parrot(object): class count(Property): def Get(prop, self): return self._count def Set(prop, self, value): self._count = value def Del(prop, self): self._count = 0 count = count('Current parrot count') _count = 0 As for me, I like the simplistic syntax of method assignments. What I do not like are the polymorphism implications. For instance:: class A(object): def method(self): return "A.method" value = property(method) alias = method class B(A): def method(self): return "B.method" obj = B() obj.method() # "B.method" obj.value # "A.method" obj.alias() # "A.method" Note that "obj.alias()" is not the same as "obj.method()", but rather is equivalent to calling "A.method(obj)". So, my question is, now that I've defined some property methods, how do I override them in a simple and straightforward manner? The crux seems to be the awkwardness of properties and aliases, and their relationships to methods. Well, at least to me, it is. ;) -Shane Holloway

I'm not sure that this has much to recommend it over the current approach. The unused 'prop' argument smells like a remnant of the implementation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
OTOH, if the class were merely used as a container (and never instantiated), it looks fairly decent (and indeed doesn't need a [filter] modifier, as MWH already suggested): class Parrot(object): _count = 0 class count(Property): """The count property.""" def __get__(self): return self._count def __set__(self, value): self._count = value def __del__(self): self._count = 0 Still, a lot of magic... This is the Property class going with the above: class Property(object): class __metaclass__(type): def __new__(cls, name, bases, methods): if not bases or bases[0] == object: # this is to make the Property class itself... return type.__new__(cls, name, bases, methods) return property(methods.get("__get__"), methods.get("__set__"), methods.get("__del__"), methods.get("__doc__")) Just

These all abuse the class keyword for something that's definitely not a class. That's a fatal flaw. --Guido van Rossum (home page: http://www.python.org/~guido/)

Hello, On Tue, Jan 28, 2003 at 05:44:26PM -0500, Guido van Rossum wrote:
Too bad Python's metaclasses are powerful enough for doing all the kind of things that a macro system could do without actually manipulating the syntax, if it is to hear that actually doing so is fatally flawed :-( Armin

"AR" == Armin Rigo <arigo@tunes.org> writes:
AR> On Tue, Jan 28, 2003 at 05:44:26PM -0500, Guido van Rossum wrote:
These all abuse the class keyword for something that's definitely not a class. That's a fatal flaw.
AR> Too bad Python's metaclasses are powerful enough for doing all AR> the kind of things that a macro system could do without actually AR> manipulating the syntax, if it is to hear that actually doing so AR> is fatally flawed :-( Zope is full of hacks like this :-). In Zope3, I see many new language features added through metaclasses, special objects created with class statements, elaborate protocols to declare interfaces and lookup adapters for objects by interface. I think many or all of these features are necessary in such a large system intended to accomodate extension at so many levels. Many of the Zope3 features are begging for syntactic support. That is, the use of the features would be clearer if they could be spelled in a straightforward way. Python provides plenty of features that allow you to extend the language. It's unfortunate that you can't extend the syntax, too. Jeremy

Jeremy Hylton <jeremy@alum.mit.edu> writes:
It's unfortunate that you can't extend the syntax, too.
You often don't need to extend the syntax, as long as the existing syntax and semantics do the right thing. My favourite example of this is the C++ idiom of "resource acquisition is initialisation", which combines the semantics of deterministic destructors and the syntax of being able to define a new variable scope (via {...}) whenever you need to, to give a very readable and general solution to all sorts of acquire/release problems. In my view, Guido's thunk proposal gives the right semantic flexibility to let people define idioms like this within the fixed syntax. Even if the generality of it scares him :-) Paul. -- This signature intentionally left blank

Walter Dörwald:
I'd be satisfied with that, I think. If Guido likes it too, that's great, and I'll happily give it a +1. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

But the problem is that it makes proprty a keyword, which is a Big Change. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido:
But the problem is that it makes proprty a keyword, which is a Big Change.
The more kludgy workarounds I see proposed here for *not* having a new keyword, the more I feel that it would be worth whatever pain it takes to add one, to get a decent, clean, magic-free syntax for properties. It's a bit unfortunate that you've already decided to use "property" as a type name. Is that meant to be official, or is it still considered an experimental detail? If you don't want to change it, maybe the keyword could be something else, like defproperty or __property__. Hmmm, lets try that: class Foo(object): __property__ myprop: def __get__(self): ... def __set__(self, x): ... Doesn't look too bad, once you get used to the idea that an __xxx___ name can be a keyword... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Agreed. It will take longer, but it's The Right Thing. Which is why I'm shooting down all ugly half-solutions.
Does that matter at this point? It's been in Python 2.2 for over a year, and we'll can't break that in 2.3.
If you don't want to change it, maybe the keyword could be something else, like defproperty or __property__.
Both are ugly. (See above. :-)
__xxx__ as a keyword is ugly. (See above. :-) I'd like the new keyword to be more generally useful than just for defining property. --Guido van Rossum (home page: http://www.python.org/~guido/)

On woensdag, jan 29, 2003, at 22:42 Europe/Amsterdam, Guido van Rossum wrote:
Can't we do something along the lines of "import as"? I.e. allow either one or two identifiers after a def, and if there's two then the first one changes what happens? Then we could say def property foo: ... Hmm, the __name__'s of various builtin types are not exposed in any namespace, are they? Otherwise we could say that "def foo():" is really a shorthand for "def xxx.function foo():", and maybe even "class Bar:" for "def xxx.classobj Bar:" -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

I was just thinking something like that, too. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

On Wed, Jan 29, 2003, Jack Jansen wrote:
That sounds good to me, but what about the syntax for the actual property functions? Are you suggesting: def property foo: def __get__(self): ... def __set__(self, value): ...
Do you perhaps mean "def function foo" and "def class Bar"? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach

From: "Jack Jansen" <Jack.Jansen@oratrix.com>
function have formal parameters which are not a tuple of values computed at definition time like superclasses for a class, so it seems that both shorthand cannot be made work at the same time. reusing 'def' is a possibility for the KEYW-TO-BE in my post, but it's clear-cut that's the least confusing choice. Unless we go for a generalization of 'def' (function) instead of 'class' but it would make def property more odd because property would likely have to receive a code object instead of a populated dict.

"Samuele Pedroni" <pedronis@bluewin.ch> writes:
This sentences took some time to parse, but you're right.
reusing 'def' is a possibility for the KEYW-TO-BE in my post, but it's clear-cut that's the least confusing choice.
There's a "not" missing here, I think.
There's something odd about writing def property foo: ... Both of defprop foo: ... or define property foo: ... seem more natural. def foo [property]: ... anyone? Probably relying on the difference between [ and ( kills it. Cheers, M. -- Just put the user directories on a 486 with deadrat7.1 and turn the Octane into the afforementioned beer fridge and keep it in your office. The lusers won't notice the difference, except that you're more cheery during office hours. -- Pim van Riezen, asr

Yes. How about foo = property: ... --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum <guido@python.org> writes:
Well on a purely instinctive level, I think it's horrible. What does it do? I seem to have missed a concise explanation, and a few minutes of thread trawling hasn't found one. In general I agree with Samuele: aiming for a one size fits all sysntactic extension is fruitless unless it's essentially macros and I thought consesnus was we didn't want them. Cheers, M. -- languages shape the way we think, or don't. -- Erik Naggum, comp.lang.lisp

Guido van Rossum wrote:
IMO none of the new syntax ideas really have much appeal. One way to define lightweight properties today is something like foo = property( lambda s: s.__dict__.get('_foo_', 42), lambda s,x: s.__dict__.__setitem__('_foo_', x), lambda s: s.__dict__.__delitem__('_foo_'), "foo property" ) I don't claim this is a great solution but it is rather efficient syntaxwise. If properties-accessor/modifiers have considerably more body-code then the current overhead of def _getfoo(self, ...): ... def _setfoo(self, ...): ... def _delfoo(self, ...): ... foo = property(_getfoo, _setfoo, _delfoo, "docstring") isn't too bad, either. So I think that any new syntax needs to satisfy more needs than just curing some property-uglyness. Neverthelss, Michael's original patch is in a different class. Maybe property-aspects should be left out of its dicussion. just my 2c, holger

holger krekel <pyth@devel.trillke.net> writes:
IMO none of the new syntax ideas really have much appeal.
I have the same feeling.
and this is something I actually have done, for readonly computed properties (without docstring), where this is reduced to foo = property(lambda self: self._bar) All in all, I'm not so unhappy with the current way. Thomas

On Thu, Jan 30, 2003 at 08:00:59AM -0500, Guido van Rossum wrote:
So far, it seems there are 3 constructs we have talked about improving: * classmethod, staticmethod * property * thunks on arbitrary blocks which could help locking, e.g. I'm not sure any one (proposed) solution can solve all of these. I don't really like any of the proposals so far. The reason is that none obviously demonstrate the semantics. Currently, this obviously defines a function. def foo(): # ... mwh/sameule's proposals for: def foo() [staticmethod]: # ... is fairly clear. I suppose it could be extended for threading/locks at the function level: def foo() [staticmethod, synchronized]: # ... One could also define pre- and post- conditions on a method basis with user-defined callables: def foo() [precond, postcond]: # ... This doesn't seem optimal, but it's also the best I've seen so far. I'm not sure how useful it would be to extend to classes though: class foo(base) [singleton]: is one case I can think of. However, it doesn't seem worth extending the syntax to save one line from time to time. Are there many other potential uses? I don't like Guido's proposal for properties because the intent is not clear: foo = property: ... What is the code block? What is the block required to do? Writing the above seems awkward and error-prone. But perhaps I'm just being too conservative. I'm not even sure if I like this version better: def foo property: ... However, using def is more obvious to me what's going on. The last issue could provide many benefits, but also seems to be the hardest. One could do: synchronize(lockvar): ... Where lockvar has certain attributes acquiring/releasing a lock. But if we wanted to extend it to a file: synchronize(somefile): ... doesn't seem very obvious. The "synchronize" keyword doesn't make sense. "with" is more general, but I don't think it's very clear to say: with(somefile): ... or with(lockvar): ... But even if this was decent, how would one operate on the variable? Special methods __prewith__ and __postwith__? I'm just adding some fuel to the fire. I don't particularly like any of these suggestions either. :-) Neal

Guido:
That looks quite nice to me, if it can be parsed... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

From: "Raymond Hettinger" <python@rcn.com> To: "Greg Ewing" <greg@cosc.canterbury.ac.nz>; <python-dev@python.org> Sent: Thursday, January 30, 2003 11:34 PM Subject: Re: [Python-Dev] Extended Function syntax
you mean the syntax? (because the sematics is quite up in the air). do you see it like foo = lambda: ... because I could sympathize with tha line of reasoning? still one is a statement and the other is an expression and that could be confusing.

That's the first bit of positive feedback I got on this proposal all day. :-) Fortunately the second bit came soon (although Raymond left in an attribution to Greg Ewing that seemed to imply it was his idea :-). And as another opening remark, I didn't mean this to replace the [prop, prop, ...] syntax on function/method defs -- that seems a useful addition in its own right. I'm not so sure that it also applies to classes -- there we already have metaclasses. The form above can be parsed easily, if we say that "colon followed by suite" is allowed only at the end of a simple assignment or expression statement. Then, v = e: S would be equivalent to v = e(T) where T is a thunk created from S. (More about thunks below.) Similarly, e: S woul be equivalent to e(T) Note that 'property' and 'synchronized' used as examples are not new keywords! They are just built-in objects that have the desired semantics and know about thunks. Syntactically, this would be valid too: x = 42: print "Hello world" but when executing it, you'd get a TypeError when calling the number 42 with a thunk as argument. Other small stuff: v, w = e: S is of course equivalent to v, w = e(T) and similarly v = w = e: S is equivalent to v = w = e(T) Another exercise to see if this construct can be used flexibly might be to see if we can use it for creating an interface syntax inspired by Zope interfaces. Let's see... I1 = interface(I2, I3): def m1(a, b, c): "docs for method m1" def m2(a, b, c): "docs for method m2" This doesn't look very attractive, certainly not much better than Zope's abuse of the class keyword. BTW, I'd be interested in entertaining different syntactic proposals that would make it less likely that a typo would accidentally turn something into a thunk call. It worries me that leaving out 'def' in a simple function definition could turn it into code that gives a TypeError when executed rather than a SyntaxError. (Forgetting 'def' is a mistake I frequently make.) Samuele rightly questioned the semantics of the thunk with regard to namespaces though. In the example of a locking section: synchronized(aLock): BLOCK you'd want the scope of BLOCK to be the same as that of the surrounding code, just as with an if or try statement. Note that synchronized() must be a function of one argument that *returns a function*. Something like this: def synchronized(lock): def f(thunk): lock.acquire() try: thunk() finally: lock.release() return f On the other hand, when using the same syntax to define a property: foo = property: "docs" def get(self): return self.__foo def set(self, value): self.__foo = value we would like the thunk to be executed in a new scope. In this case, the property constructor must (1) detect that it is being called with a thunk argument, and (2) in that case, *call* the thunk (so the def statements inside the thunk are executed) and somehow extract them from the thunk's namespace. I don't mind if a function that takes a thunk has to work a little harder (although there should be a simple case that's roughly equivalent to Ruby blocks). I also don't mind making a thunk an object with various pieces of metadata and perhaps different methods so that synchronized and property can pick it apart and call it in different ways. If we didn't have local variable optimizations or nested scopes, and all namespaces were simply dictionaries, then it might be sufficient if a thunk were a code object. synchronized() could use this to execute its thunk in its caller's scope: exec thunk in sys._getframe(1).f_locals and property could execute it in a freshly created namespace: d = {} exec thunk in d but alas, life isn't that simple any more. A thunk may: - use variables defined in the containing scope (but which may have no value when the thunk is executed) - set variables that are defined in the containing scope - set variables that don't appear in the containing scope - and more (e.g. think about globals, outer scopes, and so on) Worse, you may have multiple thunks, and they may set variables with the same name (e.g. all properties will tend to define get and set). Also, if a global is used in the scope containing a thunk and set in the thunk, does that make it a local variable shared between the containing scope and the thunk? Or does it make it a local variable in the thunk that shadows the global only in the thunk's scope? The compiler could generate code for thunks that leave the decision on all these issues up to the code (e.g. the implementation of property or synchronized) that calls the thunk. Maybe the cells used to implement nested scopes can help. But it would be pretty hairy! --Guido van Rossum (home page: http://www.python.org/~guido/)

--On Thursday, January 30, 2003 21:02:23 -0500 Guido van Rossum <guido@python.org> wrote:
I think it's quite cool too.
But class Foo(FooBase) is synchronised: pass or class Foo(FooBase) [synchronised]: pass as you prefer make nice syntax for a metabase application.
Can we allow it as an argument to a function? Something like: do_stuff(a=e: S # Hmm, this comma is odd... maybe it needs to be implicit ,b=e: S1 )
Which is neat.
How about using the containing namespace if we return a thunk rather than a function (using two arguments so we can pass in the body): def synchronized(block, lock): f = thunk: lock.acquire() try: block() finally: lock.release() return f or even: def synchronized(block, lock): return thunk: lock.acquire() try: block() finally: lock.release() This would allow: def defproperty(block): block() return property(get, set, del, block.__doc__) so we can make the new namespace that the property exists in be our locals. Or am I making no sense at all? Andrew

I'm going off on a tangent (again, as usual:-), but would it be an idea to add a "meta-reserved-word" to Python, so it would become easier to add new constructions to the language? What I envision is something (let's call it "keyword" for now, even though that's probably a bad choice) that we could use to make an identifier a keyword. So, if we're discussing a lock .....: <block> statement at some point this would be implemented in 2.N as keyword(lock) ....: <block> with lock not a reserved word (but it would generate future warnings). In 2.N+1 lock would become a reserved word. But: keyword(lock) would also continue to work for all eternity. (And, as a corollary, you could use keyword(class) and any other construct too). I don't have intimate knowledge of the workings of the parser, but I can't imagine that this would be difficult on that account. -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

Hello, On Thu, Jan 30, 2003 at 09:02:23PM -0500, Guido van Rossum wrote:
Just to throw more oil on the fire, note that this looks quite a lot like for v in e: S For example, it is quite messy but you can already define 'newproperty' to let you do the following in 2.2: class X: for count in newproperty: def get(self): return self._count def set(self, value): self._count = value Similarily you can "almost" already write the following for locks: for _ in acquired(lock): ... i.e. you can define acquired() so that the loop is done exactly once, and the lock released at the end. It doesn't really work because you there is no place in acquired() you can put the try:...finally:. Just-drawing-parallels'ly yours, Armin.

[Armin]
Huh? Not to my eyes. Unless you basically ignore all keywords, in which case there's no difference between an if and a while block either. That doesn't seem to be a useful POV.
Messy is the word. I don't think this existence proof will convince anyone that we don't need a new feature then.
I'm confused. You & Samuele both sometimes have a rhetorical style that leaves me with no idea of the point you're trying to make. Please say it outright. --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Guido van Rossum" <guido@python.org>
is this the reason why you have not given much feedback on my "rants"? for example do you agree or disagree that it bad to have scoping rules for some piece of code that depend on some user code somewhere else and that can sometime be quite ambiguous at first glance? Thus the 'class'-like scoping rules case should be syntactically distinguishable from the inline-suite-like case?

No, this time your rants were quite clear (at least the second time around). I need to have time to read all feedback again and think about it.
I haven't made up my mind on that yet. It appears that the scope of a block in Ruby is quite ambiguous, yet it seems to work there. What are the scope rules for blocks passed into method calls in Smalltalk? I hope that if this feature is used in a fairly stylized way, where usually the function used to lead into the block is a well-known function, the varying scope rules may not be a problem. Many users have only a vague notion of the difference between a syntactic construct and a function (witness repeated suggestions here and elsewhere that confuse these matters), yet nobody has a problem with understanding the difference in scope rules between 'def' and 'if' statements, for example. So I hope that learning that some functions treat a thunk's scope different than others is no huge burden. In some Lisps, I believe a function can determine whether its arguments are evaluated before the call (== normal Python semantics) or whether they are somehow (I don't know the details!) passed to the function as parse trees so the function can evaluate them at will. This does not seem to confuse Lisp users. Or does it? I honestly don't know if this is popular or considered a wart. I do worry that if my thunks are to be maximally useful, they may end up having all variables bound through "cells" (as used for nested scopes) which may give them a reputation of being slow compared to in-line code, which would limit their popularity for control flow statements like synchronized(). --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Guido van Rossum" <guido@python.org>
they are not ambiguous in the sense we are referring here: def f(): x=3 foo: x=4 # new x or old x?
evolutive ancestors of macros, now they have been substituted with macros, CL and Scheme do not have them.
yes, that worries me to, it seems at least a likely source of recurring comp.lang.python threads

From: "Samuele Pedroni" <pedronis@bluewin.ch>
'with' syntax does not share the problem, because it is just syntax-sugar for a try-finally, not involving thunks. Repeating myself, unless one considers the most extreme proposals that border macros, thunks [smalltalk-ish anonymous blocks] are quite a non-orthogonal addition for a language already with functions and closures, generators and 'for'. [I leave aside generalizing 'class', which is something useful but not about control-flow statements and for me a separate problem] Much of what: do beh(args): (x,...): suite can do, can be done already with for x,.. in behgen(args): # behgen is a generator that yields where beh would invoke the thunk suite What is not covered is: 1) the case where the thunk passes back some result on invocation by beh using the proposed 'value' direct-to-caller-return statement. 2) the case where beh returns a useful value do e = beh(args): (x,...): suite Both could be adressed with the following similar syntax-sugar but that defines a usual function instead of a thunk: fdo [e=] beh(args): [(x,...):] suite # can contain return would be sugar for (or something similar): def <anonymous>(x,...): suite [e = ] beh(args)(<anonymous>) In this case we don't need to devise new rules for what 'break','continue','return' do in the suite. The suite is just the body of an anonymous function, 'return' has the usual meaning for a function in there. No new concept and rules would be added to the language. Obviously the suite cannot rebind locals in the surrounding function. Given the functional flavor of the construct especially if there's an assigmnent e = or beh uses values produced by the suite, this can be considered a feature. The addition of 'with' and 'for' plus generators would cover the other cases! OTOH what 'return' does in the suite and the fact that the suite is not an inline suite can be ambigous at first glance. It is for sure a kind of wart. But this would counterbalance the problem of defining/implementing thunks, and what 'break','continue','return' would do exactly in them, and the necessity to add 'value' to get full expressivity for thunks.

[Guido]
[Samuele]
yes, that worries me to, it seems at least a likely source of recurring comp.lang.python threads
[Samuele again]
'with' syntax does not share the problem, because it is just syntax-sugar for a try-finally, not involving thunks.
Right. Of course, many programmers (the majority? :-) will expect 'with' to do something similar to 'with' in Pascal, VB and JavaScript. So maybe we'll have to pick a different keyword. But let's focus on the semantics of your 'with'.
Yes, but Python isn't particularly orthogonal anyway. *If* (and that's a big if) we can make it fit on all the edges, I don't mind it being unorthogonal, *if* it also solves a large (better be very large) class of issues where users would like to extend the language but currently have to write a lot of extra code, and *if* most of those issues can't be solved almost as well by something a lot less powerful (like your 'with'). I think so far I've been thinking mostly about fitting my proposal on all edges (that's where my discussion of the thunk's scope comes from). Maybe I should try to focus more on collecting real-world examples that are asking for syntactic support, and then try to look for the simplest way to support most of them.
And 3) the case where the thunk is saved for later, e.g. when it is used as a callback or as some other extension of function definition. I haven't shown examples of this, but anonymous blocks in Smalltalk are used for this all the time. I can't find good docs about Ruby's iterators that describe whether this is possible or not. I did find out that Ruby has two iterator syntaxes (do...end and {...}) that differ in whether they introduce a new scope or not!
Yes, that's one of the cases that I had wanted to cover.
It just isn't a good way to implement things like synchronized(), which really need clean interaction with the environment (through break, continue, return, and local variable assignment). (I'm giving up on 'yield' for now.)
The addition of 'with' and 'for' plus generators would cover the other cases!
Not case (3).
To be continued... --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Guido van Rossum" <guido@python.org>
yes, real-world examples would be good.
there would be 'with' ('using' in C#) for that.
I was assuming that fdo (do) woud be also added, that would cover case 3, no non-local return and manipulation of surrounding locals, but e.g. for long-lived callbacks those would be warty.

From: "Samuele Pedroni" <pedronis@bluewin.ch>
In particular real-world examples, not covered by 'for' and generators or 'with'*, that need rebinding the surrounding function locals and/or non-local return. [Just for fun, on the hackery/contrived side of things, we would finally get a version of Paul Graham's accumulator generator http://www.paulgraham.com/accgen.html with mostly lisp-y length: def accmaker(n): do acc = (lambda x:x): (i): n += i value n # ! not 'return', that would be a non-local return return acc ] * with <expr>: <suite> would be possibly sugar for: _x = <expr> _x.__enter__() try: <suite> finally: _x.__exit__() OR: _x = <expr> _x.__enter__() try: try: <suite> except getattr(_x,'__excepts__',()),_e: _x.__except__(_e) finally: _x.__exit__()

On Sunday 02 February 2003 08:21 pm, Samuele Pedroni wrote: ...
wouldn't a version: with <identifier> = <expr>: <suite> be helpful too? Meaning to use local variable <identifier> in lieu of the abstract _x -- for example in order to enable: with myfile = auto_closing_file('blah.txt', 'rb'): xx = myfile.read(23) # rest of suite snipped where auto_closing_file is a subclass of file defining useful __enter__ (empty -- might be nice to have it optional...) and __exit__ = close synonyms (or file itself might grow __exit__ as a synonym for close). How would I do this elegantly without the assignment...? Apart from this I cannot, offhand, think of any case that your 'with' (perhaps with some tweaks such as making __enter__ optional, as above mentioned) could not handle elegantly and effectively. Alex

From: "Samuele Pedroni" <pedronis@bluewin.ch>
you're right, you would also get this idiom for iterators that support also
the
With Guido's 'do', you could define an iterclose(): do iterclose(open('blah.txt','rb')): (line): ... Btw, the two snippets illustrate quite well the different evolutive directions' on the table.

On Sunday 02 February 2003 09:39 pm, Samuele Pedroni wrote: ...
with myfile = auto_closing_file('blah.txt', 'rb'): for line in myfile:
[where here we could also have e.g. xx = myfile.read(23) # rest of suite snipped ]
If the Btw does in fact hold, then (speaking as a teacher of Python and as a writer) I hope the "with" line prevails -- I would find that one trivially easy to explain (even to newbies), while I think I would have a harder time teaching the second form (it may have other advantages -- I haven't followed the thread well enough to say -- but it seems it would be harder for newbies to learn and use). Alex

From: "Alex Martelli" <aleax@aleax.it>
to be fair I don't think that Guido suggested to add a iterclose builtin to the language, OTOH nobody would be stopped to write and use it in its code, basically short of community style guides and pressure both: do iterclose(open('blah.txt','rb')): (line): ... do myfile = autoclose(open('blah.txt','rb')): for line in myfile: ... would be potential idioms.

[Samuele Pedroni]
Could we possibly flesh out both of these examples to what would amount to working code (including code that would manipulate what they create by searching for "Python" in each line) in a single email? I would like to make sure that I am comprehending how this will all work in the end. So if someone (read: Guido and Samuele =) could implement a file that auto-closes once the iterator on the file is exhausted with each suggestion I would *really* appreciate it. And if there could even be code given that used what these things returned by searching each returned line for the word "Python" would be great as well. That way we not only see how the new syntax would work, but we also see how code is going to interface with what the new code generates. -Brett

On Sunday 02 February 2003 10:41 pm, Brett Cannon wrote: ...
As I understood Samuele's "with" proposal (enhanced to allow assignment in the with, and to make __enter__ optional): class auto_closing_file(file): __exit__ = file.close with myfile = auto_closing_file('blah.txt'): for line in myfile: if line.find('Python')>=0: print line, If __enter__ were not optional, auto_closing_file would have to add one boilerplate line defining it: class auto_closing_file(file): __exit__ = file.close def __enter__(self): pass Actually, it would seem sensible, if we added this 'with', to extent built-in file by this tiny bit -- let it have __exit__ as a synonym of close (and if needed __enter__ as a no-operation method) so it can be used directly in what would then become a very common idiom: with myfile = open('blah.txt'): for line in myfile: if line.find('Python')>=0: print line, I don't understand the 'do' concepts well enough to code to them, as I have not followed this thread closely enough for this. Alex

With Guido's 'do': [notice that count is rebindable in the thunk] class iterclose: def __init__(self,iterwclose): self.iter = iterwclose def __call__(self,thunk): try: for x in self.iter: thunk(x) finally: self.iter.close() count = 0 do iterclose(open('blah.txt')): (line): if line.find('Python') >=0: count += 1 print line, ---- class autoclose1: def __init__(self,file): self.file = file def __call__(self,thunk): try: thunk(self.file) finally: self.file.close() count = 0 do autoclose1(open('blah.txt')): (myfile): for line in myfile: if line.find('Python') >=0: count += 1 print line, ---- class autoclose2: def __init__(self,file): self.file = file def __call__(self,thunk): try: thunk() finally: self.file.close() count = 0 myfile = open('blah.txt') do autoclose2(open('blah.txt')): for line in myfile: if line.find('Python') >=0: count += 1 print line,

On Sunday 02 February 2003 11:04 pm, Samuele Pedroni wrote:
OK -- I *don't* get that " (line):" part, and how calling thunk() in iterclose.__init__ binds/rebinds the local (?) variable line of the thunk. Looks like black magic to me. Guess I must just be a bit thick tonight -- sorry.
shouldn't that be "do autoclose2(myfile):" ? or is there some even-blacker magic in this "do" business that I don't get...? Alex

From: "Alex Martelli" <aleax@aleax.it>
a thunk is like a Smalltalk block, a closure that can rebind the locals and can get arguments ( (line): above) ), and maybe non-local returns ( ^ in Smalltalk): in pseudo-python the above is: count = 0 defClosureAllowingRebinding _thunk(line): if line.find('Python') >=0: count += 1 print line, iterclose(open('blah.txt'))(_thunk)
yes, you're right.

Samuele Pedroni wrote:
If _rebinding_ is what we're after, then why not go all the way an introduce a rebind operator? Say ':='. Augmented assignment would also become rebinding then (hm, I hear the sound of code breaking in the distance ;-) Add that to MWH's function/class filter patch and you can write: count = 0 def _thunk(line) [iterclose(open('blah.txt'))]: if line.find('Python') >=0: count += 1 # or count := count + 1 print line, Not nice that _thunk needs to be named, but perhaps the function name can become optional: count = 0 def (line) [iterclose(open('blah.txt'))]: if line.find('Python') >=0: count += 1 print line, How's that for a change of direction? ;-) I'm just throwing in an idea, I'm not saying I like this (I don't know yet). Just

[Alex Martelli]
After staring at this for a minute I figured out what ``(line):`` does. So what you have to realize is that the block (from ``if line...`` to ``print line,`` is passed into ``__call__()`` and bound to ``thunk()``. Now the object bound to ``thunk`` accepts a single argument, named ``line`` in ``thunk``. In ``__call__()`` the variable ``x`` is what is being passed into the thunk code and being bound to ``line``. Phew. =) So, to make sure I am not confuing myself, ``do`` does the following: Instantiates ``iterclose(open('blah.txt'))``. Compiles the block of code that makes up the thunk. It ends up accepting a single argument which bound to ``line``. Calls ``iterclose().__call__(<thunk>)`` and executes it. We all let our heads stop spinning from all that and have a good, stiff drink. =) -Brett

[Alex Martelli]
... or Alex. =)
Now if ``__exit__()`` would be executed regardless of exceptions this would probably eliminate a decent chunk of code that uses ``finally:`` such as:: FILE = file('blah.txt', 'rb') try: for line in FILE: if line.find('Python')>=0: print line, finally: FILE.close() Wouldn't this also allow one to code in a contract programming style ala Icon (assuming I remember correctly how contract programming works)? I mean you have control over the enter and exit of the thunk (would this still be called a thunk, or just syntactic sugar for calling ``__exit__()`` and ``__enter__()``; more like a thunk-ish protocol?) which is what I thought was the focus of contract programming. Either way this seems rather nice. And if you can pass in arguments (as I think ``FILE = file('blah.txt', 'rb'): (some_argument):`` is supposed to implement) this would be really nice. =) Now we just need the ``do`` syntax example. -Brett

Hello, On Sun, Feb 02, 2003 at 11:14:41PM +0100, Samuele Pedroni wrote:
If no first-class thunks are wanted, then maybe the most direct path from today to auto-closing files is to extend the iterator protocol with an optional __leave__() method. It would be called exactly once when the iterator is considered to be exhausted. The semantics of 'for' would be modified to always call it at the end of the loop, even when an exception or a return occurs within the loop. It would allow users to write f = auto_closing_file('data.txt', 'r') for line in f: ... or for lock in synchronized(lock): ... This second one looks a bit unexpected, but it needs no new syntax and no brand new protocol. (The 'lock' variable between 'for' and 'in' is not really needed in this example.) Armin

Hello everybody, Sorry to reply to myself, just wanted to mention that I found a real need for iterators to have an explicit "finally" action. For my program I will have to invent some convoluted way of doing it :-( On Mon, Feb 03, 2003 at 05:10:42PM +0100, Armin Rigo wrote:
This seems (to me) natural because in generators you could then put a 'yield' inside of a 'try-finally' pair. The semantics of def g(): try: yield 5 finally: <so-some-cleanup> are quite clear if it is guaranteed that for x in g(): ... always cause the 'finally' part of the generator to be called at the end of the loop. That's even something that is straightforward to implement, given that loops already set up a block stack entry, just like try-finally clauses. It just adds an extra function to the iterator protocol (we could even reasonably re-use tp_clear, for example). Armin

Note that this was discussed endlessly when generators were first invented. IIRC, it appeared impossible to implement this in a reasonable manner.
But since not all generators are used in for-loops, you can't really make this the for-loop's responsibility. And the for-loop might break out and the iterator might be used more later. The only place where you could possibly do this to cover all cases would be in the generator's destructor, which isn't guaranteed to be called at a specific time in Jython. Not covering all cases seems worse to me than the current explicit rule that you can't do this. --Guido van Rossum (home page: http://www.python.org/~guido/)

On Sunday 02 February 2003 11:07 pm, Brett Cannon wrote: ...
Now if ``__exit__()`` would be executed regardless of exceptions this would probably eliminate a decent chunk of code that uses ``finally:``
Hmmm, yes, I _was_ sort of assuming that __exit__ runs much like 'finally' would -- otherwise cleanup/release code would more often than not have to be duplicated.
I think THAT is the complicated part of the "do" syntax -- that trailing "(whatever):" that I don't understand. What would its pluses be? Where would that 'whatever' be directed to...? Could you please provide a use case for with + "(some_argument):" ...? Alex

Brett Cannon <bac@OCF.Berkeley.EDU> writes:
That was the major point of the proposal. Holger has a Friendly Competing Proposal that I think allows the "monitor" (i.e. the thing with the __enter__ and __exit__ methods) to distinguish between normal and exceptional return. I thought that was a bit overcomplicated. Cheers, M. -- [1] If you're lost in the woods, just bury some fibre in the ground carrying data. Fairly soon a JCB will be along to cut it for you - follow the JCB back to civilsation/hitch a lift. -- Simon Burr, cam.misc

Samuele Pedroni <pedronis@bluewin.ch>:
I must say that, for this particular usage at least, I find the first one a darn sight easier to follow than the second! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

I think 'do' is being misrepresented, and I also doubt that this is a very practical example. But I may have to wait until the next weekend before I can continue in this thread -- I can't work on this on workdays. --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Guido van Rossum" <guido@python.org>
OK, to be even more fair, obviously do can be used to implement with class with: def __init__(self,wobj): self.wobj = wobj def __call__(self,thunk): if hasattr(wobj,'__enter__'): wobj.__enter__() try: thunk(wobj) finally: wobj.__exit__() do with(auto_closing_file('blah.txt', 'rb')): (myfile): # bad that is not meaningful: do myfile=with(...): for line in myfile: ... FURTHER 'do' is more expressive than generators because it allows to capture and abstract 'try' patterns very generally, more than 'with'.
and I also doubt that this is a very practical example.
Yes and no. What would be typical is people abstracting over recurrent pattern of 'for' and 'try' in their program and stuff them in a 'do' behavior, I'm inferring this from what people do with CL macros. So if one does what iterclose does a bunch of times in program, she could be tempted to write iterclose. It's succinctness daemon. General mileage can vary.
OK

On Monday 03 February 2003 01:05 pm, Samuele Pedroni wrote: ...
does what iterclose does a bunch of times in program, she could be tempted to write iterclose. It's succinctness daemon. General mileage can vary.
I think "succintness" is quite a secondary motivation; what generally drives me to "collect" repetitively coded behavior into one class, function, iterator, metaclass, template, macro, or other such "collecting" construct that a language may make available to me, is the desire to remove duplication of code -- both (primarily) duplication of meaningful code, and (secondarily) boilerplate code too. I.e., it's fighting the "copy and paste" daemon, with all of its minuses. Having to code empty special methods for common cases is an example of boilerplate I'd much rather do without, in a somewhat different context than that of "collecting" constructs (though such constructs may at times be usable to alleviate the problem). Consider the only way in which I find classic classes preferable to new-style ones, now that I have some substantial amount of experience using and teaching each kind...: class Ic: "whatever contents, but no __init__" ok = Ic() # fine notok = Ic(23) # raises an exception, just like it should __metaclass__ = type class Nu: "whatever contents, but no __init__" ok = Nu() # fine notok = Nu(23) # "fine" -- *eep!!!* -- masks an error...! To ensure runtime catching of what's typically an error, passing parameters to the class that will be ignored, I have to do nothing special in a classic class, but I do have to have some def __init__(self): pass for a new-style class. I can define a custom metaclass (a subclass of 'type') or baseclass (a subclass of 'object') to relegate this tiny wart (if wart it be) appropriately, of course: class noinit(object): def __init__(self): pass and then subclass noinit rather than subclassing object, or use the (more complicated to code, but perhaps more transparent in use) metaclass equivalent, e.g., something like: class noinittype(type): def __new__(cls, classname, bases, classdict): if not bases and not classdict.has_key('__init__'): bases = noinit, return type.__new__(cls, classname, bases, classdict) with a __metaclass__=noinittype in global scope. Generally, I think that when the language offers ways to reduce code duplication (including boilerplate), those ways WILL eventually be used -- at least by some coders -- because code duplication is such a blight (and many experienced coders have been taught by bitter experience to perceive it as such). That's part of why I'd much rather see macros *NOT* added to Python: I feel that, if they are there, they WILL eventually become widely used -- appropriately in some cases, sure, but (I fear) quite obfuscatingly in many other cases. Similar, even though probably weaker, qualms can be raised about other wide-use mechanisms (not quite as wide and powerful as macros might be) that can reduce code duplication -- they WILL be widely used. The richer and potentially more complicated the mechanism is, the more such wide use can be seen as a rather mixed blessing. Alex

Alex Martelli <aleax@aleax.it> writes:
Just as you do with if: myfile = auto_closing_file('blah.txt', 'rb') with myfile: xx = myfile.read(23) # rest of suite snipped Maybe not quite as elegant as allowing assignment in with but very similar to the normal try...final version: myfile = auto_closing_file('blah.txt', 'rb') try: xx = myfile.read(23) # rest of suite snipped finally: myfile.close() Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/

Bernhard Herzog <bh@intevation.de> writes:
Good point. "with" testlist ':' NEWLINE it is, if it's me that gets to write the PEP. Cheers, M. -- Or here's an even simpler indicator of how much C++ sucks: Print out the C++ Public Review Document. Have someone hold it about three feet above your head and then drop it. Thus you will be enlightened. -- Thant Tessman

[Samuele]
yes, real-world examples would be good.
synchronized() is definitely a real-world example -- writing try/finally for locks is a pain for all threaded code. Another, related example would be something that blocks all signals (or at least SIGINT) during a certain code block. I imagine that there are quite a few examples that basically wrap a code block in something that does some preparation and some cleanup. These require that the cleanup action is done even if an exception occurred in the block, and this can't be done with the "for loop" hack proposed by Armin (unless you rely on __del__, which would be a no-no for Jython). 'do' or 'with' should be up to this task. I'm now wondering if there aren't really only two interesting cases, none of which require saving the thunk and calling it later: - synchronized-like, where the block should connect to its environment - property-like, where the block should introduce a new scope, and the caller should be able to inspect or control that scope's namespace All cases where something is saved to be called later can be done by using the extended 'def' syntax (def f(...) [something]: ...). I think that for synchronized-like cases, even if it is known at compile time that it is to be part of the surrounding scope, the code generated will still need to use nested-scope-cells, since it will be invoked on a different stack frame: the "thunk processor" pushes its own frame on the stack, and I think you can't have something that executes in a frame that's not the topmost frame. (Or can you? I don't know exactly what generators can get away with.) --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Guido van Rossum" <guido@python.org>
with 'do' you mean your 'do' whose thunk can rebind locals?
How can you avoid that someone stores away the thunk, you would need a special keyword to invoke it (Ruby uses "yield" which here is already taken), then we still have return/break/continue semantics to get right, whether we want 'value' or not'... If I understand correctly you are still proposing to have: a 'do' and thunks that can mess with locals with their headaches <.5 wink>, and new concepts added to the language. On my part, I'm still wondering whether Michael Hudson's 'with' and my 'fdo' (maybe still to name just 'do'), that uses an anonymous function as-we-know-it, not a thunk will give most of the functionality without enlarging the language (they are basically sugar, useful sugar).

Guido van Rossum <guido@python.org> writes:
I certainly haven't seen any use cases other than these two. In fact, pretty much all of the cases I am aware of are synchronised-like, which I've been referring to as acquire/release. The only property-like cases I'm aware of are properties themselves. Personally, I don't have much problem with the current property approach, and the comments I've seen have been generally 50-50. I'd like to see some focused discussion on property syntax, but maybe it could be spun off onto a separate thread?
All cases where something is saved to be called later can be done by using the extended 'def' syntax (def f(...) [something]: ...).
This sounds interesting, but I can't quite see what you are saying. Do you have a real use case in mind? As far as the acquire/release case is concerned, I think that there is definitely a strong case for syntactic support for this. The equivalent idiom in C++ is "resource acquisition is initialisation" which, while not having syntax support, does rely on the existence of deterministic destructors, and on the ability to introduce a new scope at will. Python has neither of these, and the try...finally construct is verbose enough to make something better worth having. On the other hand, the simple "with" construct proposed by Samuele
[...]
extended as suggested by Alex to allow __enter__ to be optional and to allow for an assignment in the with statement, seems to cover this excellently. (I'm not sure the exception handling needs to be quite that complex, but that's a minor point). Holger Kregel's suggestion is pretty much identical (although he allows for multiple objects in the same statement).
So, to offer a unified proposal, I'd suggest the following syntax: with [ var1 [, var2, ... ] = ] expr1 [, expr2 , ... ]: suite This is equivalent to var1, var2, ... = expr1, expr2, ... if hasattr(var1, "__enter__"): var1.__enter__() if hasattr(var2, "__enter__"): var2.__enter__() ... try: try: suite except: # Handwave here for now finally: ... var2.__exit__() var1.__exit__() In this, any varN can be omitted and a dummy hidden variable is used. The handwaving around the except clause needs to be clarified - what I want is to call varN.__except__() ... var1.__except__() allowing each in turn to catch the exception. If none of them do then the exception should be propogated. The problem is that I'm not sure what is the best way for an __except__ method to signal that it caught the exception. The __except__ attribute should be optional, as well. I can't see any reason why it would ever be useful for __exit__ to be optional, but for generality it might be worth allowing that as well. I like "with" as the keyword here. One obvious question - if we decide to go with syntax like this, would we be looking at "pre-reserving" the keyword in Python 2.3 somehow, so that it could become a full keyword in 2.4? Or would the new syntax need a future statement in Python 2.4, to become finalised in 2.5? Paul. PS Just to summarise things once more: as I see it, we now have a number of threads lurking: - The def f(arg) [...] syntax that started all this off - With blocks (which give us acquire/release) for which I propose a unified solution above - Properties, which don't have a specific solution proposed yet (AFAICT) - Thunks in various forms, most notably Guido's hyper-general version (the one that lets us implement switch) It's not at all clear to me that the thunk based solutions have killer use cases that they solve better than anything else (assuming that switch doesn't count :-)) but they may be aiming at the property case. If so, I'd appreciate a concrete "this is how we'd write a property" explanation, as I'm losing the plot on that... On the other hand, thunks seem too heavyweight to me if they don't offer some other major advantage as well as properties (I actually prefer with for the acquire/release case). I'll stop now before I feel a need to summarise the summary of my summary :-) -- This signature intentionally left blank

On Sunday 02 February 2003 10:49 pm, Paul Moore wrote: ...
which I've been referring to as acquire/release. The only property-like cases I'm aware of are properties themselves.
Wouldn't classmethods and staticmethods also be able to exploit (in a simpler way) the syntax/semantics used for properties? That's the impression I had. I do like your "unified proposal" for the "with" case (I have no idea if "with" is the pluperfectly right sugar for this, btw -- no biggie anyway, i think).
Maybe the return value of __except__ should be evaluated in a boolean context encoding this? False -> exception not caught, True -> caught ... So, an __except__ without return statements (implicitly returning None, thus false) would be a clean-up thingy that does no catching (but e.g. just rollbacks some DB transactions or whatever), for example -- a "return True" would explicitly signal the exception was caught -- etc. Alex

Alex Martelli <aleax@aleax.it> writes:
__except__ could just reraise the exception if it doesn't handle it. This behavior would be similar to that of a normal except clause. Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/

Paul Moore wrote:
A use case I can think of is some kind of XML DOM API (Ruby does something like that with it's cgi module, where node content is created inside a block). With the current Python you can have something like that: node = html.table( html.tr( html.th("foo"), html.td("bar") ), html.tr( html.th("spam"), html.td("eggs") ), border=0, cellpadding=3 ) i.e. element classes that use * and ** arguments in their constructor to collect content and attribute nodes. The problem with this approach is, that the attributes are maximally separated from their element and that control structures (like if) are not possible inside the constructor. With blocks ifs and fors could look like this: node = html.table(border=0, cellpadding=0): html.tr(): html.th(): "foo" html.td(): "bar" if cond: html.tr(): html.th(): "spam" html.td(): "eggs" for i in xrange(10): html.tr(): html.th(): i html.td(): i*i Bye, Walter Dörwald

From: "Walter Dörwald" <walter@livinglogic.de>
this assumes that there's a way to evaluate a thunk as an expression or such that it has the value of the last expression that appears in it. Otherwise, you would have to write things using 'value' direct-return: html.th(): value i regards.

Walter D?rwald wrote:
you probably missed my other posting where i actually went one step further and implemented this via <...> syntax. Actually i tried *exactly* the syntax you propose here first but i think it's *hopelessly* ambigous with the Python grammar. You simple have no way to know which construct you are in until you read the ':'. Until then you could be in a normal expression. The 'class', 'def', 'if', 'while' etc. keywords disambiguate this right at the beginning. Without a much deeper parser look-ahead this can't be done, i am afraid. holger

holger krekel wrote:
I saw it. I actually took a look at your IEXEC patch, where this XML stuff is one of your use cases. Compared to the patch, which uses sys._getframe(1).f_locals['__icontent__'] to get at the content nodes I like the __catch__ idea much better. But I find your syntax to be too unpythonic.
You're right, but my point is that in this XML case the new block feature is used so often that an additional keyword might be too much typing overhead. Bye, Walter Dörwald

Paul Moore <lists@morpheus.demon.co.uk> writes:
IMO if you want multiple controllers for a with statement you should just nest the withs: with var1 = expr1: with var2 = expr2: suite This makes the order in which the methods of the varN will be called obvious.
In this, any varN can be omitted and a dummy hidden variable is used.
Meaning something like with var1, , var3 = expr1, expr2, expr3: suite I guess. Urgs. You can't normally omit assignment targets on the left hand side of a sequence unpacking in python. I don't see why this should be allowed for with. Nested withs handle this in a cleaner way as well, IMO. Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ MapIt! http://www.mapit.de/

Paul Moore <lists@morpheus.demon.co.uk> writes:
I like this.
Making the methods optional seems reasonable but writing do-nothing methods isn't much of a strain. I don't care either way.
(I'm not sure the exception handling needs to be quite that complex, but that's a minor point).
I'm not at all sure. In fact, on thinking about it I feel it may be a bad idea. Examples of when __except__ might be different from __exit__ would be appreciated. [....]
I think this might be unnecessarily hard to compile. class multimonitor(object): def __init__(self, monitors): self.monitors = monitors def __enter__(self): for monitor in self.monitors: monitor.__enter__() def __exit__(self): for monitor in self.monitors: monitor.__exit__() It should probably be stated that an __exit__ method should never raise an exception.
In this, any varN can be omitted and a dummy hidden variable is used.
Persuading the parser to distinguish with foo(): and with var = foo(): is going to be tricky. I think we'll have to settle for one or the other.
This sort of problem is an argument for throwing away the except: clause.
Hmm. Maybe. Not fond of generality for the sake of it.
I think nothing should change in 2.3.
I'll write PEPs for these two in due course if noone beats me to it.
Me too. Guido can do this PEP :-) [...]
I'll stop now before I feel a need to summarise the summary of my summary :-)
OK, I cheated with the sig :-) Cheers, M. -- To summarise the summary of the summary:- people are a problem. -- The Hitch-Hikers Guide to the Galaxy, Episode 12

Hello, I'll try to take the discussion about a "unified approach" towards 'code block' or thunk handling into a fresh thread and present what Michael Hudson truthfully calls "a friendly competing approach". Preliminaries ------------- We have some use cases for a construct that allows encapsulating try/except/finally hooks into an object. Especially for timely finalization (of files, locks or other resources) we don't want to scatter try-finally constructs all over the place. Please understand that the following proposal has nothing to do with "function-level" thunk-manipulation ala classmethod but only with "inline" thunks (which don't and shouldn't have their own scope). Syntax and semantic proposal ---------------------------- I think we can may get away with only a "weak" keyword and allow the aforementioned encapsulation of execution events into an object like this: exec expr [with params]: suite where the expression is evaluated to return a "thunk" handler with these optional "execution" hooks: def __enter__(self): "before suite start" def __except__(self, type, value, tb): "swallow given exception, reraise if neccessary" def __leave__(self): """upon suite finish (not called if __except__ exists and an exception happened) """ The above "with" parameters (of the form name=expr, comma-separated) are bound in local (or global/nested) *and* handler instance namespace. The 'suite' is what we call "thunk". The above logic allows clean timely finalization for *multiple* ressources: exec autoclose() with f1=open(name1), f2=open(name2, 'w'): for line in f1: ... f2.write(...) which would execute as follows a) autoclose() instance is created and stored as the "thunk"-handler b) f1/f2 are stored as attributes on the autoclose instance c) f1/f2 are put into the local/global namespace (and nested ones if rebinding is allowed) d) thunk executes (for line ...) e) autoclose 'leave' hook is called (with or without exception) and is implemented like this: def __leave__(self): for obj in self.__dict__.values(): obj.close() f) thunk handler is removed Because computing 'f1' may succeed but 'f2' can subsequently fail the assignments *have to* execute within "autoclose" control. Now on to the usage of the except hook. Nice use cases might be exec retry(maxretry=3, on=IOError): # do network io-stuff or exec skip_on(AttributeError, TypeError): some_object.notify_hook() but i am sure there are more. Exception handling is often ugly when inlined with the code. I think that stating 'exception behaviour' up-front allows to write nice readable constructs. __exit__ versus __leave__ --------------------------- One remark (mainly to Michael as he does that other patch) about the hook-name __leave__ versus __exit__. we may want to eventually allow 'yield' within the thunk and then '__exit__' would be misleading. Here is the use case: exec self.mylock: # would lock/unlock on entering/leaving # the generator ... for whatever in something: yield whatever ... Or do you think that this (future) use case warrants yet another hook? If there is interest i can probably modify my patch to allow all of the proposed syntax so that you could play around with it. the next additional idea is not essential for my so-far proposal (but hopefully interesting, nonetheless). regards and interested in comments, holger ------------- Catching values ala Quixote ------------------- With my former patch there was another "execution" event: the catching of "dropped" values ala Quixote. For values that are not assigned to a name and would be discarded otherwise the execution handler can implement another hook: def __catch__(self, value): "catch dropped value from thunk" this allows anonymous (un-named) objects to be passed *within the thunk* to the execution handler. some example use-cases: exec debug: if condition: "something questionable happenend" so you can quite completly encapsulate debug-handling into one object. Note that this exec could be special-cased to do nothing if "debug" is None. Another one (ala the Quixote-framework): exec html_stream: "<h1>title</h1>" "<p>%s</p>" % html_quote(para) or even: exec html.tr(): exec html.td(): "column" which comes close to what Walter wants. And maybe you understand now why I choose this strange xml-syntax with my former patch :-) Actually i didn't have the idea of reusing 'exec' which makes a lot more sense to me, now. again with regards and intersted in any remarks, holger

On Mon, Feb 03, 2003, holger krekel wrote:
No, __leave__/__exit__ should always be called regardless of whether an exception hits. That's the whole point of this proposal, IMO.
I'm thinking that this is *way* too complex. If you want something like that, code it like this: for whatever in something: exec self.mylock: new_whatever = process(whatever) yield new_whatever -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "Argue for your limitations, and sure enough they're yours." --Richard Bach

holger krekel <pyth@devel.trillke.net> writes:
I should point out that I stole that phrase from Guido (in PEP 283).
Gut reaction: ugh!
That looks messy.
We need a name for these. I've been using "monitor" for a while, but I'm not sure it's that apposite.
"Too much magic!"
I am *still* not convinced that an __except__ hook is worth the pain. Can you implement those for me?
I guess. But enter/exit is just such a canonical pairing. To me, leave is paired with arrive and arrive/leave doesn't make much sense.
yields already can't go in blocks with finally statements, right? Would you propose calling the __enter__ method each time the generator resumed?
Or do you think that this (future) use case warrants yet another hook?
Guess :)
I think all these proposals are in danger of bending the language into a place it should not go. Still, we should write some PEPs, even if they wind up rejected. Cheers, M. -- Those who have deviant punctuation desires should take care of their own perverted needs. -- Erik Naggum, comp.lang.lisp

Michael Hudson wrote:
then i guess you rather prefer new keywords. I thought that there is a general reluctance to introduce new keywords *and* many people dislike 'exec' for its existence. So reusing it (and we are dealing with execution aspects IMO) makes some sense to me.
as compared to? I think i have seen messier ideas with all kinds of brackets and double-colons :-)
So far i'd liked "execution handler" because 'enter/exit/except' are IMHO execution events which you can hook into. Do you agree with calling them 'execution events' or how would you call them?
Hmmm. That seems a bit unfair to me as many other proposals didn't care to elaborate the exact sequence. You probably are refering to the "namespace interactions" as the other stuff is probably the same with any other proposal so far.
Would the following be the correct way to achieve this with your patch? f1=open(inputfn) with autoclose(f1): f2 = open(outputfn, 'w') with autoclose(f2): for line in f1: ... f2.write(line) I think there should be a better solution for multiple ressources.
Maybe, i haven't looked at your implementation yet and there are some other projects pending :-)
right.
Would you propose calling the __enter__ method each time the generator resumed?
yes, neccessary for locking (as the above example tried to indicate).
Then i guess it doesn't. That way my point, anyway :-)
sure. cheers, holger

On Tuesday 04 February 2003 12:54 pm, holger krekel wrote: ...
I agree this is slight too cumbersome, and deeply wish it was feasible to go with: with f1=autoclose(inputfn): with f2=autoclose(outputfn, 'w'): for line in f1: ... f2.write(line) i.e., nested with's would be just fine, IF binding the with'd expression to a name was allowed as part of the with statement, by whatever sugar. I'd rather have it mandatory and use "with junk = autolock(xx):", rather than not having the binding in the "with" -- just as I have only modest problems HAVING to bind a name in "for i in range(5):" even when I don't care about i at all in the loop's body. But if such binding is unfeasible, then maybe it's not worth having the "with" after all -- or else we need something richer and more complicated (and perhaps ugly) as in your proposal. Alex

On Tue, Feb 04, 2003 at 01:03:24PM +0100, Alex Martelli wrote:
This made me realize something. What happens if some sort of exception is raised between 'f1 = open()' and the start of the block 'with autoclose(f1):'? KeyboardInterrupt could be raised between the statements, for instance. autoclose(f1).__exit__() will never be called now, since autoclose(f1) is never even computed. Even if assignment in the statement is supported ( 'with f1 = autoclose(outputfn, "w")') I don't immediately see what bytecode would be written to make sure that an exception didn't arrive at just the wrong moment, leaving 'with' block incompletely entered but with the file open. Doesn't this concern apply to the common idiom l.acquire() try: pass finally: l.release() too? (most of the 'with' proposals are suggested to be equivalent to a Python program of this structure) The exception could be delivered just before the POP_TOP of the acquire statement, or the SET_LINENO of the try: statement, and things would go south from there. Please let me know what simple fact I'm missing. Jeff

Jeff Epler wrote:
With CPython the file object will be refcount-finalized sooner or later. With JPython there should be a leak. With self.acquire() # KeyBoardInterrupt here try: ... finally: self.release() you can get a deadlock (as you stated). It's just so unlikely that it hardly ever causes problems. You can mask out asynchronous signals if this is really critical. And the unmaskable SIG9 will eventually take care anyway :-) holger

holger krekel <pyth@devel.trillke.net> writes:
There is. But __future__ makes a sensible transition possible, at least.
It makes some sense, sure. But not enough, in *my* opinion. We don't want to go down the C route -- consider poor "static"[1]. [...]
As compared to the dirt simple scheme Paul just posted about.
Sure enter/exit/except are execution events, but execution handler rubs me the wrong way because that imples the (e.g.) autoclose object is somehow responsible for executing the suite, which it isn't: it;s more an observer of the execution. And it's a suite goddamit, not a thunk (in this proposal).
True. However, I'm currently in favour of a very simple proposal.
You probably are refering to the "namespace interactions" as the other stuff is probably the same with any other proposal so far.
I'd very much hope that some of the sillier proposal are dead in the water. Sure, your proposal is hardly the most complex out there, but *I* still think it's too complex. Actually, it's mainly the b) and c) bits I object to.
Yup.
I think there should be a better solution for multiple ressources.
Fair enough. I don't. [...]
I meant, can you implement retry and skip_on? Sorry, that was unclear. [...]
Hmm, that could work. Tricky to implement as things currently stand but managable for sure. Good grief, I should do something useful today. Cheers, M. [1] Strange small worldy fact: the phrase "consider poor static" has stuck with from my earliest usenet experiences, so I google for it, and it turns out it was John W. Baxter, comp.lang.python regular. -- : Giant screaming pieces of excrement, they are. I have a feeling that some of the people in here have a MUCH more exciting time relieving themselves than I do. -- Mike Sphar & Dave Brown, asr

On Tuesday, Feb 4, 2003, at 12:54 Europe/Amsterdam, holger krekel wrote:
I agree with Michael's "Ugh!". And actually I extend the "Ugh!" to Guido's proposal of "expr [= expr] [lambda]: suite". While I think that the reluctance to introduce new keywords is a good thing in principle it shouldn't go at the expense of the clarity of the language. In other words, if there is a clear construct that doesn't require a new keyword (cf. import as) that is a good thing, but if the clarity of Python would be compromised then I think we should bite the bullet and add a new keyword. We should think of people who come fresh to Python in three years time. What "if" does and what the parameters are is immediately clear. Same for "for". Same for "class", "def", assignments and what-have-you. And then suddenly they come across foo(): ... or exec foo(): ... The only construct I can think of that would give the user an idea of what was happening is "with expr [= expr]". And even that will become ugly once you want to support both thunks with a new scope and parameters, and thunks in the existing scope. Last week I made a suggestion that was completely ignored, so let me try again (i.e. if everyone thinks it's silly please shoot it down in stead of ignoring it:-): how about adding a meta-keyword that would be used to turn an identifier into a keyword. For sake of the example let's use "keyword" as the meta-keyword, although that's a bad choice, probably. Here's how it would be used: if we decide to go with "with" as the thunk-keyword what happens is that in Python 2.4 you would have to type it as keyword(with) expr: suite or, alternatively, do something like "from future import with_keyword". In 2.4, using "with" as an identifier would give a FutureWarning. In Python 2.5 "with" would become a reserved word, but keyword(with) would still continue to work. Whether the latter is for one release (with accompanying FutureWarning) or for all eternity is open to discussion. -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

Jack Jansen wrote:
I am a bit wary about all those syntax details. Actually i try to focus on which events an 'execution monitor' should handle and how it interacts. The problem is that talking about new semantics immeditately brings up the question of syntax and then everybody discusses syntax.
Yes, but names are important and exec autoclose with f1=open(inputfn): ... is not that hard to read and understand. I'd even dare to say it's easier than understanding metaclasses :-) Also today 'exec' is hardly ever needed and so reinterpreting it in the long run might be viable. "Oh yes, i forgot to tell, you can also pass it a string or a code object ..."
Isn't that a road to macros? :-) I don't know but it sure reduces readability and it looks like cheating. we don't want to introduce new keywords, anyway. To me it is not only a matter of 'syntax' problems or beeing backward-incompatible but the real problem is the number of concepts and constructs you have to remember, to understand, to use and to teach. holger

holger krekel wrote:
Why not simply have one method def __execute__(self, thunk): which gets passed the thunk. This method can do anything it likes before executing the thunk, it can do anything after executing the thunk, and it can implement exception handling: class twice: def __execute__(self, thunk): print "start" try: thunk() print "once" thunk() print "twice" except Exception, exc: print "exception", ext else: print "OK" Then you can do: exec twice(): for i in xrange(10): print i Bye, Walter Dörwald

[Guido]
"resumable function" captures almost all salient aspects of the Python generator implementation. Generator resumption acts like a method call to the resumer (generator_iterator.next(), whether explicit or implicit), and yielding acts *almost* like a function return to the generator (the generator's stack frame is popped just as for a function return; the primary difference is just that the frame's refcount isn't decremented). In short, the implementation is a way to get the same frame on the top of the stack more than once; there's nothing there now to support execution in a non-top frame. If such were added, the biggest mystery would be what to do when a non-top frame tries to exit.

Tim Peters wrote:
What do you mean with "generator's stack frame is popped"? I interpreted this code in ceval.c if (why != WHY_YIELD) { /* Pop remaining stack entries -- but when yielding */ while (!EMPTY()) { v = POP(); Py_XDECREF(v); } } to mean the frame is suspended with a preserved stack. f_stacktop/last_i later are used to resume. holger

Given the length of this thread, could someone please distill this into one or more PEPs ?! Thanks, -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/

[holger krekel]
What do you mean with "generator's stack frame is popped"?
At the end of eval_frame(): /* pop frame */ --tstate->recursion_depth; tstate->frame = f->f_back; return retval; The current frame is always popped from the thread state's frame stack, regardless of the reason for leaving eval_frame() (and generator yield is identical to function return in this respect).
Like that's news to me <wink>. Until it is resumed, the frame doesn't appear anywhere on the thread state's stack -- it would be lost entirely except that a generator-iterator object still holds a reference to it

--On Saturday, February 01, 2003 15:02:51 -0500 Guido van Rossum <guido@python.org> wrote:
I think it's not a burden, and in fact the ability to make such things would be very useful. I've come up with a few examples below.
It's fundamental. Lisp syntax is almost exactly a representation of the parser output; lisps basically don't have a parser, up to syntactic sugar. So a macro definition can specify in its definition which arguments are passed evaluated or unevaluated (in some lisps, this is done by explicity evaluating the appropriate arguments; elisp is one such). The difference between macros and functions comes down to if arguments are evaluated or not. Lots of ordinary, standard lisp idiom is implemented this way, even such things as 'let', which is the local variable binder and even sometimes 'defun', which is the function definer. For example, in xemacs elisp syntax for simplicity, meaning all macro arguments are passed unevaluated, defun is approximately defined as (up to the docstring handling): (defmacro defun (name arglist &optional docstring &rest body) (quote (setf name (function (lambda arglist . body)))) ; return code to set the docstring here ) which has an effect essentially identical to python's 'def' keyword. The return value of the macro (and what is actually compiled) is (setf name (function (lambda arglist . body))) (substituting the passed in arguments) which when executed binds name to the compiled version of the lambda. So if we then do: (defun inc (a) (add 1 a)) it compiles as if it were: (setf 'inc (function (lambda (a) . ((add 1 a)))))) For a really huge demonstration of what lisp macros can do, see advice.el from an emacs distribution. It's a good example because it's extremely well commented, and some of the comments show the downsides of macros as well.
Well, they could be treated like lisp macros, that is as source code (or, potentially, parse tree) transformation functions, and compiled just like any other code. This would probably be the simplest to implement, and would allow such things to work with no slowdown at all. For example, presuming python had a defmacro that took all arguments unevaluated and can return a block, that @block introduces a block argument, and that interpolate(block) compiles the block instead of itself, we could have: defmacro synchronized(lck, @block): return: lck.acquire() try: interpolate(block) finally: lck.release() synchronized(lock): # text of block here pass which compiles identically to: lck.acquire() try: # text of block here pass finally: lck.release() Then: defmacro property(name, docstring, @get, @set, @delete): def getter interpolate(get.arglist): interpolate(get.block) def setter interpolate(set.arglist): interpolate(set.block) def deleter interpolate(delete.arglist): interpolate(delete.block) return: name = property(docstring, getter, setter, deleter) class Circle(object): property circumference, """Circumference of the circle""": get (self): return self.radius*6.283 set (self, value): self.radius = value/6.283 delete (self): del self.radius would do the obvious thing using block keyword arguments (I can't think how to express the anonymous scope implied here in regular Python; the intention is that getter and friends are defined in the local namespace of the macro invocation). Of course the proposed syntax is hypothetical, but I think it's quite clean. Andrew

--On Sunday, February 02, 2003 10:56:29 +1300 Andrew McGregor <andrew@indranet.co.nz> wrote:
Hmmm, maybe that should have been: defmacro property(name, docstring, @block): block() return: name = property(docstring, get, set, delete) class Circle(object): property(circumference, """Circumference of the circle"""): def get(self): return self.radius*6.283 def set(self, value): self.radius = value/6.283 def delete(self): del self.radius I still think block keyword arguments would be nice, but this can be done without them. Andrew

code_thunk = ``` if Guido.let_it_fly(): print "This is my syntax for inline code block" ``` Now I understood the summary of this thread for myself. It's all about making extended lambdas in Python so Python could manipulate code as it does with other objects! Now I am convinced this is great idea and quite Pythonic too. Also if I understood correctly, the main problem is namespace interaction. My thought here is that it shouldnot be different to lambdas: environment need to be visible in the inside. Another problem is with exceptions. But is it really a problem? How different it is when we exec something? So, the problem (IMHO) has this subproblems: 1. How to express an arbitrary codeblock object anywhere in the expression? 2. What protocol to use to communicate with this codeblock? Let's see what we already have in Python: I. case: user-defined function 1. def-statement 2. arguments and one value after return statement. Environment of place of definition available. II. case: lambda function 1. lambda-operator 2. arguments and a single return value. Environment available. III. case: compiled code block 1. any string and then compile() or directly exec(), etc 2. globals(), locals() Now: IV. case: inline code block aka extended lambda ;-) 1. special (probably ':'-powered) syntax, allowing to insert code snippet into any place in the expression 2. this is a big question. I propose to use some functions around codeblock which will add needed namespaces, exception handlers and other things needed for code snippet to work. BTW, it's my understanding that is such construct (inline code block) will be added to Python, then almost we will be able to model ANY Python control-flow operator with it! Isn't it really cool? And of course inline code snippet object will be extensible objects, allowing for any kinds of modifications, adding aspects (for those who uses aspect programming), pre/post conditions, currying, locks, security control, etc, etc. It will be a new era of Python development. It will nearer Python to LISP, where code and data are same. I am sure there is no problem defining +, *, % operations for them (obvious, isn't it?): + - make a sequence * - make a for _ in range(N) cycle, % - provide parameters (with a dict or whatever), locals() for example. (Am i dreaming?) As for optimisations, drop them, Guido, for now: inline code blocks are good target for JIT-optimisations. Now syntax. ('lambda' is an example keyword, just for illustration!!!) For loop: codesnippet = FOR_LOOP_FUNCTION(lst, lambda i: do whatever with index i . . .. . .... : # for closing block explicitely ) Function: codesnippet = DEF_FUNCTION(param_names, lambda pms: ..... .... return k : , defaults = dict1) Now if we want to provide codesnippet with parameters: codesnippet = lambda what, filename: print >> open(filename, "w"), what : codesnippet2 = codesnippet % {'filename': 'ex1', } what = "123" codesnippet.exec(vars()) Of course, parameter getting/setting needs some formal protocol. If grammar will not suffer too much, one-line version: codesn = lambda: print 2*2: + lambda: print 2*2: codesn.exec() will give 4 4 on the output. (Where 'lambda' is an example keyword, just for illustration) I even think that we can use $ as new type of blocking, needed for this. It is fundamental addition and needs extraordinary syntax: codesn = $ print 2*2 $ + $ print 2*2 $ Or maybe some asymettric: codesnippet = <? print 2*2 ?> + <? print 2*2 ?> But Guido will never let this fly, even if parser will be happy ;-) What about ```print 2*2``` + ```print 2*2``` - very near to the sense of what we want to achieve! Sincerely yours, Roman Suzi -- rnd@onego.ru =\= My AI powered by Linux RedHat 7.3

Hello Guido, On Sat, Feb 01, 2003 at 01:29:12PM -0500, Guido van Rossum wrote:
Sorry, I forgot one step. I was first thinking about a new 'with' keyword inspired by an old proposal seen in this list: with v in e: S and then realized that the magic necessary for such a 'with' statement (and thus for "v=e: S") was close to the magic of 'for', in the sense that it invokes magic methods of 'e' and bind the result to the variable 'v'. I was saying that it might be possible to extend the semantics of 'for' to cover that case too. Well, in the meantime I've seen Greg's suggestion "def ... as expr:", which I find much nicer in general. I am all for it. The only risk I see is the temptation to abuse its power for control flow, as it would allow def _ as run_synchronized(lock): ... This temptation looks similar to the abuse of the 'class' keyword to use the metaclass mecanism with concepts unrelated to classes. In other words, let's be careful before adding powerful features with associated syntax... If I may attempt a summary, given all the use cases that have been expressed in this thread, we could have a need for constructions whose associated code block is either (1) run immediately or (2) passed unexecuted to some constructor; and whose variables are (a) in their own namespace or (b) in the parent namespace. Currently we have 'class' (1a) and, althought currently a bit limited in post-processing facilities, 'def' (2a). I've shown contrived examples of (2b) using 'for'. The question is whether we want something generic enough for all four possible combinations or not, and whether they should still be statically distinguishable or not. If they should not, then the 'thunk' approach is fine, given a way to execute a thunk in its original namespace. This is a well-known approach in block-oriented languages, but I agree with Samuele that this can be much confusing and for some reason I dislike this programming style. Maybe the reason is that Python has a pseudo-declarative look & feel that is incompatible with direct manipulation of control flow. Such a feature would make the life impossible for source-processing utilities like code checkers. But if we want the four cases to be statically distinguisable, then we might be bound to add various unrelated syntax extensions at several places, all of which might come with an associated original use case that makes the syntax look like a hack for radically different uses (like Just's nice metaclass-based definition of properties). A bientôt, Armin.

But a mechanism that only allows (2) can trivially be used to provide (1) as well -- the function to which the thunk is passed simply calls the thunk. The real issue is about (a) vs. (b). I think that it is technically possible to make the function control this as well, but it may not execute as fast (unless you compile two versions of the bytecode, or something similarly wacky). It is also possible to make the function control what to do about return, break, and continue (using pseudo-exceptions), but it definitely feels like a stretch.
Actually, I don't think 'for' can really be considered a case of of (2). A true version of (2) allows you to save the thunk off in a global somewhere and call it at a later time. With 'for' your only choice is to decide whether to execute it immediately or not; and you don't get to sniff the contents of the block before executing it. So I don't think your contrived examples are worth much.
That's a good summary.
I don't think so. Code checkers make many assumptions that a compiler isn't allowed to make, and it seems totally reasonable to build knowledge about popular thunk consumers into the checker. Just like GCC has an extension to typecheck printf().
It doesn't have to be that bad. If, as I argue above, we really only need to distinguish two cases, we could probably come up with two fairly neutral syntaxes that cover both ends of the spectrum. A Pythonic way would be to introduce each with a new keyword: # for control flow; <block> is part of surrounding scope foo <expr>: <block> # for definitions; <block> defines a new scope # maybe <block> needs formal parameters too bar <expr> = <expr>: <block> (I'm not at all sure that only one of these needs the "<expr> =" part.) For 'foo', 'do' seems a reasonable keyword; at least, do synchronized(mylock): BLOCK looks reasonable enough. I'm not so comfortable with the leading contender for 'bar', which seems to be 'def'. --Guido van Rossum (home page: http://www.python.org/~guido/)

Ahhh. Sorry to say that. I'd like to contribute, but python-dev is hitting my limits completely. This thread is absolutely incomprehensible. Message frequency is unbearable. Relationship indentation hits by browser's capabilities. How can I follow this while working for some bread. Poor guy who has to write the summary. sigh - chris

Too bad, Chris. This thread is not for the benefit of posterity, but for the benefit of those participating in it. There are enough of us who *are* still following it that it makes sense to keep it on python-dev. If you can't follow it, just ignore it -- that's what I do with most other threads on python-dev, knowing that there are others who do follow it. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

From: "Guido van Rossum" <guido@python.org>
I'd like the new keyword to be more generally useful than just for defining property.
the first candidate would be a generalization of 'class' (although that make it redundant with 'class' and meta-classes) so that KEYW-TO-BE kind name [ '(' expr,... ')' ] [ maybe [] extended syntax ]: suite would be equivalent to name = kind(name-as-string,(expr,...),dict-populated-executing-suite,__doc__=doc-strin g-in-suite) so we would have: class C: KEYW-TO-BE property foo: ... the remaining problem would be to pick a suitable KEYW-TO-BE

On Thu, Jan 30, 2003 at 12:06:10AM +0100, Samuele Pedroni wrote:
KEYW-TO-BE kind name [ '(' expr,... ')' ] [ maybe [] extended syntax ]: suite
+1 assuming we can pick a kw that makes sense. []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo@laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/

Guido van Rossum <guido@python.org> writes:
I don't think there was much discussion. The suggested semantics was that this is equivalent to def name(arg, ...): ...body... name=expr(name) ... I *think* there was discussion as to the namespace in which expr is evaluated, or whether certain identifiers have keyword or predefined meaning (so you can write 'static' instead of 'staticmethod'). I don't think there was ever a complete analysis whether this syntax meets all requirements, i.e. whether you could use it for all newstyle features. In particular, I don't recall what the proposal was how properties should be spelled. Regards, Martin

Martin v. Löwis wrote:
In particular: def name(arg, ...) [expr1, expr2, expr3]: ...body... would be equivalent to (some variation on): def name(arg, ...): ...body... name=expr1(expr2(expr3(name))) I wonder if the same mechanism could be used in class statements. If I had this, I might say what interface a class implements with: class foo(spam, eggs) [implements(IFoo, IBar)]: body of class or (wo parens): class foo [implements(IFoo, IBar)]: body of class Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (888) 344-4332 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

Jim Fulton wrote:
With Michael's patch (which indeed still works) it's actually name = expr3(expr2(expr1(name)))
I don't know how Zope interfaces work, but I can imagine the following could be made to work as well: class foo(spam, eggs) [IFoo, IBar]: body of class I think this would be wonderful. Would someone be interested in extending Michael's patch to also cover the class statement? Just

Guido van Rossum wrote:
Some other interesting related posts: - http://mail.python.org/pipermail/python-list/2001-July/056224.html - http://mail.python.org/pipermail/python-list/2001-July/056416.html - http://mail.python.org/pipermail/python-dev/2001-July/016287.html
It ought to be turned into a PEP.
John Williams sent a very rough candidate PEP in October that was interesting (below). I sent back some suggestions (below the PEP), but I haven't received anything back yet. Perhaps a joint PEP with syntax alternatives? <pep> PEP: XXX Title: Multiword Method Names Version: $Revision:$ Last-Modified: $Date: 2002/10/09 21:11:59 $ Author: John Williams <jrw@pobox.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 09-Oct-2002 Post-History: Python-Version: 2.3 Abstract ======== This PEP proposes allowing the space character in method names. Motivation ========== With the addition of static and class methods to Python, there are now three distinct kinds of methods. All are declared using the same syntax, and static and class methods are indicated by the use of the built-in "wrapper" types staticmethod and classmethod. This technique is unfortunate because it requires two statements to define a single method and requires the method name to appear three times. These problems could be solved by allowing the space character in method names and adding metaclass support for methods with certain names, in effect allowing arbitrary pseudo-keywords to be added to method declarations. Specification ============= The core of this proposal is to allow an arbitrary sequence of identifiers and keywords to appear between the keyword "def" and the opening parenthesis of the function declaration. The result would be identical to existing "def" statements, except that the name of the new function would would consist of the words joined together with space characters. Although no syntax exists for working with variables whose names contain spaces, the name would be accessible through explicit use of the underlying variable dictionaries. Using the new syntax along with special metaclass support, properties and static and class methods could be declared like this:: class MyClass(object): def class myClassMethod(cls): . . . def static myStaticMethod(): . . . def get myProperty(self): . . . def set myProperty(self, value): . . . The declaration above would be equivalent to: class MyClass(object): def myClassMethod(cls): . . . myClassMethod = classmethod(myClassMethod) def myStaticMethod(): . . . myStaticMethod = staticmethod(myStaticMethod) def __getMyProperty(self): . . . def __setMyProperty(self, value): . . . myProperty = property(__getMyProperty, __setMyProperty) Copyright ========= This document has been placed in the public domain. </pep> Here's my reply to John: <reply> This PEP proposal is interesting, although the idea has come up before (not a bad thing; see references below). The first thing I notice is that the title is misleading. The title and abstract ("This PEP proposes allowing the space character in method names") make me think you're proposing that this kind of declaration would be legal and somehow callable:: def a multi word method name(): pass This would be too large a syntax change IMO. I think the "multiword" aspect and "allowing spaces" are merely side-effects of what the PEP is proposing. They're implementation details. The real issue is the lack of a unifying syntax for descriptors. The text of the PEP is looking at the issue from the wrong direction IMO: from the bottom up (implementation) instead of from the top down (concept). I haven't wrapped my head around descriptors yet, and I'm not familiar with Python's internals. I have no idea if what you're proposing is even feasible. But here are some ideas and suggestions for the PEP: * I think the PEP should be called "Syntax for Descriptors". (I also thought of "Pseudo-Keywords in Method Declarations", but it's not as good a title.) * Rather than "pseudo-keywords", what about calling these tokens "modifiers"? * Perhaps metaclasses could grow a mechanism to add new pseudo-keywords of their own? Maybe not right away. When someone comes up with a novel use for descriptors, it would be nice to be able to implement it with syntax. * Expand the examples to include the "delete" property method. (I guess the modifier can't be "del", since that's already a bona-fide keyword. But then again...) * It may not be feasible to use "class" as a modifier, since it's already a keyword. Perhaps "classmethod", as suggested in one of the threads below. * Add some blanks to the examples to make them easier to read. The same idea was brought up back in July 2001. See the following threads: - http://mail.python.org/pipermail/python-list/2001-July/056224.html (A reply from Guido: http://mail.python.org/pipermail/python-list/2001-July/056416.html) - http://mail.python.org/pipermail/python-dev/2001-July/016287.html There may have been other similar discussions. Guido's reply is discouraging, but he may have changed his mind since. Having a PEP clearly proposing the syntax change would be useful even if it's rejected. I would recommend you revise the PEP and send it to Python-Dev to gather feedback before resubmitting it for a PEP number. </reply> -- David Goodger <http://starship.python.net/~goodger> Python Enhancement Proposal (PEP) Editor <http://www.python.org/peps/> (Please cc: all PEP correspondence to <peps@python.org>.)

This syntax is clear, explicit, and attractive.
def get myProperty(self): . . . def set myProperty(self, value): . . .
This ibe doesn't extend as cleanly: def del myProperty(self, value): "Oops, del is a keyword" def doc myProperty ... ? what goes here Also, can two properties share a getter or setter as they can now? x = property(notifyOneWay, setx) y = propetry(notifyOneWay, sety) Raymond Hettinger ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# #################################################################

David Goodger wrote:
Thanks for bringing this up. I've been following python-dev but I've mostly gotten sidetracked from Python stuff since then. I'm also having second thoughts about the whole idea of my proposal, since I basically wrote it in a fit of excitement over possibilities of metaclasses. Compared to the other proposal going around (which I'll call Guido's, since he brought it up), the really big advantage of my proposal is that you can use it to do something like adding a property to a class implicitly by defining its getter and setter methods: class A(object): def get foo(self): "Getter for property 'foo'." return self.__foo def set foo(self, foo): "Setter for property 'foo'." self.__foo = foo It's critical here that neither of these declarations actually defines the name "foo"; they define names like "get foo" and "set foo", and it's up to the metaclass to expose these methods in a sane way (i.e. by creating a property named "foo" in this case). I agree with the criticisms that others have made about this proposal, but the real killer (for me, at least) is that using method name modifiers would require case-by-case support in the metaclass to achieve the desired effects. Guido's proposal (adding expressions after the argument list) is easy to extend and generally much less magical, since the modifier expressions don't have to have their meaning "interpreted" by a metaclass. At this stage I'd much rather see Guido's proposal implemented, unless someone comes up with a truly ingenious way to combine the advantages of both.

John Williams <jrw@pobox.com> wrote in news:3E300A13.6020303@pobox.com:
<snip>
How about this: class A(object): def foo(self, foo) [property.set]: "Setter for property 'foo'." self.__foo = foo def foo(self) [property.get]: "Getter for property 'foo'." return self.__foo Then add static methods to property that look something like this: def set(fn): if isinstance(fn, property): return property(fn.fget, fn, fn.fdel, fn.__doc__) else: return property(fset=fn) def get(fn): ... def delete(fn): ... -- Duncan Booth duncan@rcp.co.uk int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3" "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

Duncan Booth wrote:
This is beautiful, but it does not work: when defining the getter, you need both the old property, and the new function object. Looking at your code def set(fn): if isinstance(fn, property): return property(fn.fget, fn, fn.fdel, fn.__doc__) you first assume fn is the property object, and then assume it is the setter function. Of course, there is no reason why the namespace-under-construction couldn't be passed to the annotation, but that would be an extension to the protocol. Martin

"Martin v. Löwis" <martin@v.loewis.de> wrote in news:3E312900.90100@v.loewis.de:
Gah!, I must be asleep today. Something like this might work (although it is getting a bit messy): def set(fn): get, delete, doc = None, None, fn.__doc__ namespace = inspect.getcurrentframe().f_back.f_locals oldfn = namespace.get(fn.__name__) if isinstance(oldfn, property): get, delete, doc = oldfn.fget, oldfn.fdel, oldfn.__doc__ return property(get, fn, delete, doc) -- Duncan Booth duncan@rcp.co.uk int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3" "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

Duncan Booth <duncan@rcp.co.uk>:
I think there's something that needs to be decided before going any further with this: Do we want to be able to define and/or override get/set/del methods individually? If so, it would be better to re-design the property mechanism to make it easier, instead of coming up with kludges to fit it on top of the existing mechanism. Whatever the mechanism, here's my current thoughts on a property syntax: def foo.get(self): ... def foo.set(self, x): ... def foo.del(self): ... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:
Do we want to be able to define and/or override get/set/del methods individually?
As opposed to? The rationale for introducing the extended function syntax is that extended functions should be introduced by means of a definition, not of an assignment. For the same reason, I think properties should not be introduced by means of an assignment. I can't picture how *not* to define them individually, unless you are thinking of something like property foo: def get(self): ... def set(self, value): ...
So would you also be in favour of defining static methods through def static foo(): ... ? Regards, Martin

Something like that, yes. The point is that, with the current implementation of properties, any kind of syntactic sugar which *doesn't* group the three methods together somehow goes against the grain.
I'm all in favour of that, but as things stand, the extended function syntax only lends itself well to function filters that take a single function as argument. The attempts I've seen so far to extend it to handle more than one function at once all seem prohibitively ugly to me. This means that either (a) we shouldn't try to use the extended function syntax for properties, or (b) properties need to be re-designed so that they fit the extended function syntax better.
I wouldn't object to it. I also wouldn't object to using the extended function syntax for static and class methods. I just don't want to see some horrible kludge stretching the extended function syntax to places it doesn't naturally want to go. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Greg Ewing wrote:
Is that a dislike towards the notation, or towards the implementation strategy. I agree that an implementation using getframe is ugly. However, I do think that the proposed notation is natural, and that there is a clean implementation for it, too (just provide the filter with a reference to the namespace-under-construction). Regards, Martin

Martin Loewis:
Is that a dislike towards the notation, or towards the implementation strategy.
The notation is the important thing to get right, I suppose, since the implementation can be changed if need be.
there is a clean implementation for it, too (just provide the filter with a reference to the namespace-under-construction).
Yes, I can see that now (I was thinking that the property mechanism itself would need changing, but it wouldn't). But even so, I don't think it really works all that well for properties. There would be something distinctly odd about writing this sort of thing: def foo(self) [get_property]: ... def foo(self, x) [set_property]: ... because it looks like you're defining two things both called "foo". There would be too much hidden magic going on there for my taste. I've just had another thought: This would also make it hard to do a text-editor search to answer questions such as "where is the get-function for the foo property defined". With my proposal you could search for "def foo.get". Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

"GE" == Greg Ewing <greg@cosc.canterbury.ac.nz> writes:
GE> Whatever the mechanism, here's my current thoughts GE> on a property syntax: | def foo.get(self): | ... | def foo.set(self, x): | ... | def foo.del(self): | ... Interesting. I've always wanted to be able to write... class Foo: ... def Foo.baz(self, a, b, c): ... -Barry

Yes, that would be nice. It would be incompatible with that idea, unfortunately. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

The itertools module is ready for comment and review. It implements ten high speed, memory efficient looping constructs inspired by Haskell and SML. It is ready-to-run and packaged with setup.py, a news item, docs, and unittests. The files are in the sandbox at python/nondist/sandbox/itertools. If you see a typo or clear error, feel free to edit the files directly. If you don't feel like reading the C code, the docs list all known issues and include pure python equivalent code for each function. Let me know if I omitted your favorite function (tabulate, partition, etc). Raymond Hettinger

Raymond> If you see a typo or clear error, feel free to edit the files Raymond> directly. If you don't feel like reading the C code, the docs Raymond> list all known issues and include pure python equivalent code Raymond> for each function. Let me know if I omitted your favorite Raymond> function (tabulate, partition, etc). (Note, I've never used Haskell or SML, so have no direct experience with any of these iterators.) I fixed a couple typos, but have a few (more subjective) comments: * islice() - The description seems a bit confusing to me - perhaps a simple example would be useful. * takewhile()/dropwhile() - I assume these only return a prefix of their iterable arguments. Dropwhile()'s help suggests that, but takewhile()'s description is more vague about the notion. * imap() - It's not clear to me why it differs from map() other than the fact that it's an iterator. Can you motivate why it stops when the shortest iterable is exhausted and doesn't accept Non for its func arg? * loopzip() - It's not clear why its next() method should return a list instead of a tuple (again, a seemingly needless distiction with its builtin counterpart, zip()). * starmap() - How does it differ from imap() and map()? * times() - Why not declare times to take an optional argument to be returned? (In general, it seems like count(), repeat() and times() overlap heavily. Examples of their usage might be helpful in understanding when to use them, or at least when they are commonly used in their Haskell/SML roots environments.) Skip

I fixed a couple typos, but have a few (more subjective) comments:
Thanks for the rapid review.
* islice() - The description seems a bit confusing to me - perhaps a simple example would be useful.
I'll clarify the docs and add simple examples for each function. islice is one of the more powerful functions: for line in islice(afile, 10, 20, 2): print line # Starting with line 10, prints every other line # upto but not including line 20. nth = lambda iterable, n: islice(iterable,n,n+1).next() # get the nth item
I'll clarify the docs and add clear examples.
* imap() - It's not clear to me why it differs from map() other than the fact that it's an iterator.
The other differences are that it stops with the shortest iterable and doesn't accept None for a func argument.
Because one or more useful inputs are potentially infinite, filling in Nones is less useful than stopping with the shortest iterator. The function doesn't accept None for a function argument for 1) simplicity 2) we have zip() for that purpose OTOH, if it is important to someone, I can easily re-embed that functionality.
I've wrestled with the one. The short answer is that zip() already does a pretty good job and that the only use for loopzip() is super high speed looping. To that end, reusing a single list instead of allocating and building tuples is *much* faster.
* starmap() - How does it differ from imap() and map()?
for computing a operator.pow, if your data looks like this: b=[2,3,5] p=[3,5,7], then use imap(operator.pow, a, b) OTOH, if your data looks like this: c =[(2,3), (3,5), (5,7)], then use starmap(operator.pow, c) Essentially, it's the difference between f(a,b) and f(*c).
* times() - Why not declare times to take an optional argument to be returned?
The use case is for looping when you don't care about the value: for i in itertools.times(3): print "hello"
Yes. I opted for using the atomic combinable building blocks rather than constructing the more elaborate things like tabulate(f) which can easily be made from the basic pieces: imap(f,count()) I'll add more examples so that the usage becomes more obvious. Thanks again for the review. Raymond ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# #################################################################

>> * imap() - It's not clear to me why it differs from map() other than >> the fact that it's an iterator. Raymond> The other differences are that it stops with the shortest Raymond> iterable and doesn't accept None for a func argument. I understand that. I was questioning why with a name like "imap" you chose to make it differ from map() in ways other than its iterator-ness. The other semantic differences make it more difficult to replace map() with itertools.imap() than it might be. Raymond> Because one or more useful inputs are potentially infinite, Raymond> filling in Nones is less useful than stopping with the shortest Raymond> iterator. Yes, but it still seems a gratuitous change from map() to me. >> * loopzip() - It's not clear why its next() method should return a >> list instead of a tuple (again, a seemingly needless distiction >> with its builtin counterpart, zip()). Raymond> I've wrestled with the one. The short answer is that zip() Raymond> already does a pretty good job and that the only use for Raymond> loopzip() is super high speed looping. To that end, reusing a Raymond> single list instead of allocating and building tuples is *much* Raymond> faster. How do you know the caller doesn't squirrel away the list you returned on the n-th iteration? I don't see how you can safely reuse the same list. Skip

Okay, it's no problem to put back in the function=None behavior. I had thought it an outdated hack that could be left behind, but there is no loss from including it. And, you're right, it may help someone transition their code a little more easily.
I understand; however, for me, replicating quirks of map ranks less in importance than creating a cohesive set of tools that work well together. The SML/Haskell tools have a number of infinite iterators as basic building blocks; using them requires that other functions know when to shut off. I would like the package to be unified by the idea that the iterators all terminate with shortest input (assuming they have one and some don't). Also, I'm a little biased because that map feature has never been helpful to me and more than once has gotten in the way. The implementations in Haskell and SML also did not include a None fillin feature.
If needed, I can add in an izip() function that returns tuples just like zip() does. I would like to keep loopzip(). It is very effective and efficient for the use case that zip was meant to solve, namely lockstep iteration: for i, j in loopzip(ivector, jvector): # results are immediately unpacked process(i,j) This use case is even more prevalent with this package where loopzip can combine algebraicly with other itertools or functionals: takewhile(binarypredicate, loopzip(ivec, jvec) It's a terrible waste to constantly allocate tuples, build them, pass them, unpack them, and throw them away on every pass. Reuse is an optimization that is already built into the existing implementations of filter() and map().
Skip
Thanks again for the useful comments. I'll add the map(None, s1, s2, ...) behavior and write an izip() function which can be used with full safety for non-looping use cases. Raymond Hettinger

From: "Skip Montanaro" <skip@pobox.com>
After more thought, I think loopzip() is too unsavory and taints an otherwise clean package, so I'll take it out unless someone wants to stand-up for it. Too bad, it was an exceptionally fast solution to the lock-step iteration problem. Raymond

On Mon, Jan 27, 2003 at 06:32:22PM +1300, Greg Ewing wrote:
The statement "def foo(...):" is just a shortcut for the assignment "foo=new.function(CODE, GLOBS, 'foo')". So why not make "def foo.bar(...):" a shortcut for "foo.bar=new.function(CODE, GLOBS, 'foo.bar')" ? If the attributes of a property object are made assignable the following code will work: prop = property() def prop.fget(self): ... def prop.fset(self, value): ... This will also allow creating classes the way Barry suggested or creating new kinds of namespaces. "Namespaces are one honking great idea -- let's do more of those!"-ly yours, Oren

"OT" == Oren Tirosh <oren-py-d@hishome.net> writes:
OT> The statement "def foo(...):" is just a shortcut for the OT> assignment "foo=new.function(CODE, GLOBS, 'foo')". So why not OT> make "def foo.bar(...):" a shortcut for OT> "foo.bar=new.function(CODE, GLOBS, 'foo.bar')" ? OT> If the attributes of a property object are made assignable the OT> following code will work: +1 -Barry

Oren Tirosh <oren-py-d@hishome.net> writes:
So we're around to suggesting def <arbitrary-rvalue>(params): ... ? Hey, here's a way of writing switches: dispatch = {} def dispatch['a'](a): print 'it's a' ... dispatch[var](param) Not at all sure if I like this or not. It's quite a change. Cheers, M. -- "An infinite number of monkeys at an infinite number of keyboards could produce something like Usenet." "They could do a better job of it." -- the corollaries to Gene Spafford's Axiom #2 of Usenet

Oren Tirosh <oren-py-d@hishome.net> writes:
Well, not entirely. I was more trying to tease out of you what you intended.
Just in case you are not: a fully-qualified name consists of identifiers separated by dots.
I hadn't seen the phrase "fully-qualified name" used in this thread; it wasn't obvious to me that any such limitation was implied. I may have missed something, of course.
It's not an arbitrary expression. At least that's how it works in the Python module namespace.
But as a generalisation of "foo=new.function(CODE, GLOBS, 'foo')", what I mentioned would be valid too. I'm not sure it's *sensible*, but that's a different kettle of fish... Cheers, M. -- ... Windows proponents tell you that it will solve things that your Unix system people keep telling you are hard. The Unix people are right: they are hard, and Windows does not solve them, ... -- Tim Bradshaw, comp.lang.lisp

Oren Tirosh <oren-py-d@hishome.net>:
I'm not sure I like the need for the prop = property() part. It exposes too much of the implementation for my taste. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Greg Ewing wrote:
What happened to Guido's proposal? class Foo(object): property myprop: """A computed property on Foo objects.""" def __get__(self): return ... def __set__(self): ... def __delete__(self): ... (see http://mail.python.org/pipermail/python-dev/2002-February/020081.html) Bye, Walter Dörwald

Just van Rossum <just@letterror.com> writes:
I'm pretty sure you can do this with a metaclass, but I've just spent half an hour thoroughly confusing myself trying to do so. I'm not sure I like using 'class' to introduce a propery, FWIW. Cheers, M. -- First of all, email me your AOL password as a security measure. You may find that won't be able to connect to the 'net for a while. This is normal. The next thing to do is turn your computer upside down and shake it to reboot it. -- Darren Tucker, asr

Just van Rossum <just@letterror.com>:
-0, too clever. (I wouldn't mind if it was *equivalent* to that, but I wouldn't like to have to write it out like that all the time.) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Seems to me like the following should work for this in 2.2 and beyond:: #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #~ In some common file... class Property(object): def __init__(prop, __doc__ = ''): prop.__doc__ = __doc__ def __get__(prop, self, klass): if self is None: return klass else: return prop.Get(self) def __set__(prop, self, value): prop.Set(self, value) def __delete__(prop, self): prop.Del(self) #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #~ To use... class Parrot(object): class count(Property): def Get(prop, self): return self._count def Set(prop, self, value): self._count = value def Del(prop, self): self._count = 0 count = count('Current parrot count') _count = 0 As for me, I like the simplistic syntax of method assignments. What I do not like are the polymorphism implications. For instance:: class A(object): def method(self): return "A.method" value = property(method) alias = method class B(A): def method(self): return "B.method" obj = B() obj.method() # "B.method" obj.value # "A.method" obj.alias() # "A.method" Note that "obj.alias()" is not the same as "obj.method()", but rather is equivalent to calling "A.method(obj)". So, my question is, now that I've defined some property methods, how do I override them in a simple and straightforward manner? The crux seems to be the awkwardness of properties and aliases, and their relationships to methods. Well, at least to me, it is. ;) -Shane Holloway

I'm not sure that this has much to recommend it over the current approach. The unused 'prop' argument smells like a remnant of the implementation. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido van Rossum wrote:
OTOH, if the class were merely used as a container (and never instantiated), it looks fairly decent (and indeed doesn't need a [filter] modifier, as MWH already suggested): class Parrot(object): _count = 0 class count(Property): """The count property.""" def __get__(self): return self._count def __set__(self, value): self._count = value def __del__(self): self._count = 0 Still, a lot of magic... This is the Property class going with the above: class Property(object): class __metaclass__(type): def __new__(cls, name, bases, methods): if not bases or bases[0] == object: # this is to make the Property class itself... return type.__new__(cls, name, bases, methods) return property(methods.get("__get__"), methods.get("__set__"), methods.get("__del__"), methods.get("__doc__")) Just

These all abuse the class keyword for something that's definitely not a class. That's a fatal flaw. --Guido van Rossum (home page: http://www.python.org/~guido/)

Hello, On Tue, Jan 28, 2003 at 05:44:26PM -0500, Guido van Rossum wrote:
Too bad Python's metaclasses are powerful enough for doing all the kind of things that a macro system could do without actually manipulating the syntax, if it is to hear that actually doing so is fatally flawed :-( Armin

"AR" == Armin Rigo <arigo@tunes.org> writes:
AR> On Tue, Jan 28, 2003 at 05:44:26PM -0500, Guido van Rossum wrote:
These all abuse the class keyword for something that's definitely not a class. That's a fatal flaw.
AR> Too bad Python's metaclasses are powerful enough for doing all AR> the kind of things that a macro system could do without actually AR> manipulating the syntax, if it is to hear that actually doing so AR> is fatally flawed :-( Zope is full of hacks like this :-). In Zope3, I see many new language features added through metaclasses, special objects created with class statements, elaborate protocols to declare interfaces and lookup adapters for objects by interface. I think many or all of these features are necessary in such a large system intended to accomodate extension at so many levels. Many of the Zope3 features are begging for syntactic support. That is, the use of the features would be clearer if they could be spelled in a straightforward way. Python provides plenty of features that allow you to extend the language. It's unfortunate that you can't extend the syntax, too. Jeremy

Jeremy Hylton <jeremy@alum.mit.edu> writes:
It's unfortunate that you can't extend the syntax, too.
You often don't need to extend the syntax, as long as the existing syntax and semantics do the right thing. My favourite example of this is the C++ idiom of "resource acquisition is initialisation", which combines the semantics of deterministic destructors and the syntax of being able to define a new variable scope (via {...}) whenever you need to, to give a very readable and general solution to all sorts of acquire/release problems. In my view, Guido's thunk proposal gives the right semantic flexibility to let people define idioms like this within the fixed syntax. Even if the generality of it scares him :-) Paul. -- This signature intentionally left blank

Walter Dörwald:
I'd be satisfied with that, I think. If Guido likes it too, that's great, and I'll happily give it a +1. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

But the problem is that it makes proprty a keyword, which is a Big Change. --Guido van Rossum (home page: http://www.python.org/~guido/)

Guido:
But the problem is that it makes proprty a keyword, which is a Big Change.
The more kludgy workarounds I see proposed here for *not* having a new keyword, the more I feel that it would be worth whatever pain it takes to add one, to get a decent, clean, magic-free syntax for properties. It's a bit unfortunate that you've already decided to use "property" as a type name. Is that meant to be official, or is it still considered an experimental detail? If you don't want to change it, maybe the keyword could be something else, like defproperty or __property__. Hmmm, lets try that: class Foo(object): __property__ myprop: def __get__(self): ... def __set__(self, x): ... Doesn't look too bad, once you get used to the idea that an __xxx___ name can be a keyword... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

Agreed. It will take longer, but it's The Right Thing. Which is why I'm shooting down all ugly half-solutions.
Does that matter at this point? It's been in Python 2.2 for over a year, and we'll can't break that in 2.3.
If you don't want to change it, maybe the keyword could be something else, like defproperty or __property__.
Both are ugly. (See above. :-)
__xxx__ as a keyword is ugly. (See above. :-) I'd like the new keyword to be more generally useful than just for defining property. --Guido van Rossum (home page: http://www.python.org/~guido/)

On woensdag, jan 29, 2003, at 22:42 Europe/Amsterdam, Guido van Rossum wrote:
Can't we do something along the lines of "import as"? I.e. allow either one or two identifiers after a def, and if there's two then the first one changes what happens? Then we could say def property foo: ... Hmm, the __name__'s of various builtin types are not exposed in any namespace, are they? Otherwise we could say that "def foo():" is really a shorthand for "def xxx.function foo():", and maybe even "class Bar:" for "def xxx.classobj Bar:" -- - Jack Jansen <Jack.Jansen@oratrix.com> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman -

I was just thinking something like that, too. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
participants (38)
-
"Martin v. Löwis"
-
Aahz
-
Alex Martelli
-
Andrew McGregor
-
Armin Rigo
-
Arne Koewing
-
barry@python.org
-
Bernhard Herzog
-
Brett Cannon
-
Christian Tismer
-
David Goodger
-
Duncan Booth
-
Greg Ewing
-
Guido van Rossum
-
holger krekel
-
Jack Jansen
-
Jack Jansen
-
Jeff Epler
-
Jeremy Hylton
-
Jim Fulton
-
John Williams
-
Just van Rossum
-
Lalo Martins
-
M.-A. Lemburg
-
martin@v.loewis.de
-
Michael Hudson
-
Neal Norwitz
-
Neil Schemenauer
-
Oren Tirosh
-
Paul Moore
-
Raymond Hettinger
-
Roman Suzi
-
Samuele Pedroni
-
Shane Holloway (IEEE)
-
Skip Montanaro
-
Thomas Heller
-
Tim Peters
-
Walter Dörwald