New syntax for 'dynamic' attribute access
Hi, A few days ago I posted to python-ideas with a suggestion for some new Python syntax, which would allow easier access to object attributes where the attribute name is known only at run-time. For example: setattr(self, method_name, getattr(self.metadata, method_name)) from Lib/distutils/dist.py could be rewritten self.(method_name) = self.metadata.(method_name) The new syntax would also be usable in augmented assignments, as in obj.(attr_name) += 1 There was some discussion on python-ideas, which I've linked to in the draft PEP below. In particular, Guido van Rossum wrote:
I've thought of the same syntax. I think you should submit this to the PEP editor and argue on Python-dev for its inclusion in Python 2.6 -- there's no benefit that I see of waiting until 3.0.
so here I am. Does anybody have any opinions/suggestions, particularly on the "open questions" referred to in the draft PEP? To summarise these open questions: * The draft currently allows a two-argument form, to supply a default value if the object has no attribute of that name. This mimics the behaviour of the three-argument form of getattr, but looks a bit wrong: s = obj.(attr_name, 'default string') I agree that it looks odd, but perhaps the extra expressive power gained might be worth the oddness. * The draft implementation (a sourceforge patch, linked to in the draft PEP) may have a general performance penalty of around 1%, although my initial measurements were quite noisy. Josiah Carlson thought this would not be too substantial, but he did suggest a couple of other implementation routes which could be explored. The general performance hit is offset by a speed gain of around 40% for attribute access using the new syntax over getattr etc. Is 1% "too much" for this feature? Ben. - - - - 8< - - - - PEP: 363 [PROVISIONAL NUMBER] Title: Syntax For Dynamic Attribute Access Version: $Revision$ Last-Modified: $Date$ Author: Ben North <ben at redfrontdoor.org> Status: Draft Type: Standards Track Content-Type: text/plain Created: 29-Jan-2007 Post-History: Abstract Dynamic attribute access is currently possible using the "getattr" and "setattr" builtins. The present PEP suggests a new syntax to make such access easier, allowing the coder for example to write x.('foo_%d' % n) += 1 z = y.('foo_%d' % n).('bar_%s' % s) instead of attr_name = 'foo_%d' % n setattr(x, attr_name, getattr(x, attr_name) + 1) z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s) Note (I wrote this patch mostly to advance my own understanding of and experiment with the python language, but I've written it up in the style of a PEP in case it might be a useful idea.) Rationale Dictionary access and indexing both have a friendly invocation syntax: instead of x.__getitem__(12) the coder can write x[12]. This also allows the use of subscripted elements in an augmented assignment, as in "x[12] += 1". The present proposal brings this ease-of-use to dynamic attribute access too. Attribute access is currently possible in two ways: * When the attribute name is known at code-writing time, the ".NAME" trailer can be used, as in x.foo = 42 y.bar += 100 * When the attribute name is computed dynamically at run-time, the "getattr" and "setattr" builtins must be used: x = getattr(y, 'foo_%d' % n) setattr(z, 'bar_%s' % s, 99) The "getattr" builtin also allows the coder to specify a default value to be returned in the event that the object does not have an attribute of the given name: x = getattr(y, 'foo_%d' % n, 0) This PEP describes a new syntax for dynamic attribute access --- "x.(expr)" --- with examples given in the Abstract above. The new syntax also allows the provision of a default value in the "get" case, as in: x = y.('foo_%d' % n, None) This 2-argument form of dynamic attribute access is not permitted as the target of an (augmented or normal) assignment. Also, this part of the new syntax was not as well received [6] in initial discussions on python-ideas, and I agree that it does not fit so cleanly. I'm happy to prepare a revised PEP/patch without the 2-argument form if the consensus is that this would be preferred. Finally, the new syntax can be used with the "del" statement, as in del x.(attr_name) Impact On Existing Code The proposed new syntax is not currently valid, so no existing well-formed programs have their meaning altered by this proposal. Across all "*.py" files in the 2.5 distribution, there are around 600 uses of "getattr", "setattr" or "delattr". They break down as follows (figures have some room for error because they were arrived at by partially-manual inspection): c.300 uses of plain "getattr(x, attr_name)", which could be replaced with the new syntax; c.150 uses of the 3-argument form, i.e., with the default value; these could be replaced with the 2-argument form of the new syntax (the cases break down into c.125 cases where the attribute name is a literal string, and c.25 where it's only known at run-time); c.5 uses of the 2-argument form with a literal string attribute name, which I think could be replaced with the standard "x.attribute" syntax; c.120 uses of setattr, of which 15 use getattr to find the new value; all could be replaced with the new syntax, the 15 where getattr is also involved would show a particular increase in clarity; c.5 uses which would have to stay as "getattr" because they are calls of a variable named "getattr" whose default value is the builtin "getattr"; c.5 uses of the 2-argument form, inside a try/except block which catches AttributeError and uses a default value instead; these could use 2-argument form of the new syntax; c.10 uses of "delattr", which could use the new syntax. As examples, the line setattr(self, attr, change_root(self.root, getattr(self, attr))) from Lib/distutils/command/install.py could be rewritten self.(attr) = change_root(self.root, self.(attr)) and the line setattr(self, method_name, getattr(self.metadata, method_name)) from Lib/distutils/dist.py could be rewritten self.(method_name) = self.metadata.(method_name) Performance Impact Initial pystone measurements are inconclusive, but suggest there may be a performance penalty of around 1% in the pystones score with the patched version. One suggestion is that this is because the longer main loop in ceval.c hurts the cache behaviour, but this has not been confirmed. (Maybe a tool like valgrind [2] could help here?) On the other hand, measurements suggest a speed-up of around 40--45% for dynamic attribute access. Discussion To Date Initial posting of this PEP in draft form was to python-ideas on 20070209 [4], and the response was generally positive: I've thought of the same syntax. I think you should submit this to the PEP editor and argue on Python-dev for its inclusion in Python 2.6 -- there's no benefit that I see of waiting until 3.0. --- Guido van Rossum [5] Wow! I have to say this is a compelling idea. The syntax is a bit foreign looking, but [...] I feel like I could learn to like it anyway. --- Greg Falcon [6] I look forward to seeing this in Python 2.6. --- Josiah Carlson, further down the thread [8] with Greg Falcon expressing the reservations about the 2-argument form already noted above, and Josiah Carlson raising a query about performance: My only concern with your propsed change is your draft implementation. [...] Specifically, your changes [...] may negatively affect general Python performance. --- Josiah Carlson [7] Some initial measurements (see above) suggest the performance penalty is small, and Josiah Carlson said of such cost that it "isn't really substantial". [8] Questions To Be Resolved * Whether to allow the 2-argument form for default arguments. * Whether the performance impact is real; whether it is acceptable; whether alternative implementations might improve this aspect. Alternative Syntax For The New Feature Other syntaxes could be used, for example braces are currently invalid in a "trailer", so could be used here, giving x{'foo_%d' % n} += 1 My personal preference is for the x.('foo_%d' % n) += 1 syntax though: the presence of the dot shows there is attribute access going on; the parentheses have an analogous meaning to the mathematical "work this out first" meaning. This is also the syntax used in the language Matlab [1] for dynamic "field" access (where "field" is the Matlab term analogous to Python's "attribute"). Discussions on python-ideas (see above) made no comment on the brace alternative, and the .() notation was well-enough received, so the brace alternative should be considered rejected, I think. Error Cases Only strings are permitted as attribute names, so for instance the following error is produced: >>> x.(99) = 8 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: attribute name must be string, not 'int' This is handled by the existing PyObject_GetAttr function. Draft Implementation A draft implementation adds a new alternative to the "trailer" clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in Python.asdl, with accompanying changes to symtable.c, ast.c, and compile.c, and three new opcodes (load/store/del) with accompanying changes to opcode.h and ceval.c. The patch consists of c.180 additional lines in the core code, and c.100 additional lines of tests. It is available as sourceforge patch #1657573 [3]. References [1] Using Dynamic Field Names :: Data Types (MATLAB Programming) http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/f2-41859.h... [2] Valgrind: "suite of tools for debugging and profiling Linux programs" http://www.valgrind.org/ [3] Sourceforge patch #1657573 http://sourceforge.net/tracker/index.php?func=detail&aid=1657573&group_id=5470&atid=305470 [4] http://mail.python.org/pipermail/python-ideas/2007-February/000210.html [5] http://mail.python.org/pipermail/python-ideas/2007-February/000211.html [6] http://mail.python.org/pipermail/python-ideas/2007-February/000212.html [7] http://mail.python.org/pipermail/python-ideas/2007-February/000213.html [8] http://mail.python.org/pipermail/python-ideas/2007-February/000227.html Copyright This document has been placed in the public domain.
Ben North wrote:
Hi,
A few days ago I posted to python-ideas with a suggestion for some new Python syntax, which would allow easier access to object attributes where the attribute name is known only at run-time. For example:
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
I like it.
The new syntax would also be usable in augmented assignments, as in
obj.(attr_name) += 1
Even nicer; much, much better than the current spelling.
*snip*
* The draft currently allows a two-argument form, to supply a default value if the object has no attribute of that name. This mimics the behaviour of the three-argument form of getattr, but looks a bit wrong:
s = obj.(attr_name, 'default string')
I agree that it looks odd, but perhaps the extra expressive power gained might be worth the oddness.
It's not just odd, but because you can't use the result of that syntax as an assignment target (according to the PEP), it smells special-casey. Neil
"Ben North" <ben@redfrontdoor.org> wrote in message news:45CFAA81.5040906@redfrontdoor.org... | so here I am. Does anybody have any opinions/suggestions, particularly | on the "open questions" referred to in the draft PEP? To summarise | these open questions: Need: Runtime attributes are a fairly frequent 'How?' question on c.l.p. Syntax: I initially did not like it and was surprised at its current invalidity (I expected the 'surperfluous' parens to be ignored) but could not think of anything better, and after 10 min. it looks better for a couple of reasons. First, I vaguely remember () being used for a similar purpose of indicating indirection in some assembly languages. Second, parens are already special cased as the call operator when following an object expression (without an intervening operator). So I can swallow a new special case when they follow '.'. The 'two-arg' form: this analogizes .() as being like a obj() call, but the sematics are different enough that I think it confuses. So I suggest drop it at least for now. Terry Jan Reedy
Terry Reedy wrote:
Need: Runtime attributes are a fairly frequent 'How?' question on c.l.p.
That's true, but how many of those are due to an actual need for runtime attributes, as opposed to someone trying to transplant idioms from another language that would be better served by a dict? In my experience, the amount of code which truly needs to use getattr is extremely small. -- Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | Carpe post meridiem! | Christchurch, New Zealand | (I'm not a morning person.) | greg.ewing@canterbury.ac.nz +--------------------------------------+
On 2/11/07, Ben North <ben@redfrontdoor.org> wrote: [SNIP]
* The draft currently allows a two-argument form, to supply a default value if the object has no attribute of that name. This mimics the behaviour of the three-argument form of getattr, but looks a bit wrong:
s = obj.(attr_name, 'default string')
I agree that it looks odd, but perhaps the extra expressive power gained might be worth the oddness.
I don't think it is. getattr can just be kept around if need be in order to support this use case. It just makes it look like too much of a function call at that point to me, especially if someone ends up writing some expression that is so long that they put in a newline between the two values: s = obj.(attr_name, 'default_string') So -1 on the two-item version, +0 on the one-item version. -Brett
I have to say that I'm not that impressed by either the 1-arg or 2-arg versions. Someone coming across this syntax for the first time will not have any hints as to what it means - and worse, it looks like a syntax error to me. -1 from me.
Anthony Baxter wrote:
I have to say that I'm not that impressed by either the 1-arg or 2-arg versions. Someone coming across this syntax for the first time will not have any hints as to what it means - and worse, it looks like a syntax error to me. -1 from me.
I'm not sure the "looks like a syntax error" argument holds much weight, because any new syntax is likely to be a syntax error until the syntax is changed. :) "No hints" is a decent argument against it, though. Parenthesis are already used for tuples, function calls, precedence, and generator comprehensions. The more something gets overloaded, the more ambiguous it looks. How about obj.*str_expression instead? "*" is pretty common in the C family of languages as a dereferencing operator. Neil
On Monday 12 February 2007 18:38, Neil Toronto wrote:
Anthony Baxter wrote:
I have to say that I'm not that impressed by either the 1-arg or 2-arg versions. Someone coming across this syntax for the first time will not have any hints as to what it means - and worse, it looks like a syntax error to me. -1 from me.
I'm not sure the "looks like a syntax error" argument holds much weight, because any new syntax is likely to be a syntax error until the syntax is changed. :)
Yes and no. My point is that it's extremely similar to existing syntax. (Worse yet, it looks the same but for what's possibly the smallest and hardest-to-see character in any character set) "foo(baz)" vs "foo.(baz)" is... not good.
"No hints" is a decent argument against it, though. Parenthesis are already used for tuples, function calls, precedence, and generator comprehensions. The more something gets overloaded, the more ambiguous it looks. How about
obj.*str_expression
instead? "*" is pretty common in the C family of languages as a dereferencing operator.
I don't know. This all still smacks of magic line noise to me :-) Although I should also add that after thinking about it a bit, I'm still not convinced this is something that needs shorthand syntax - particularly not one that's so line-noisy... Hey, we could always use backticks in Python 3.0! <wink> Anthony
On Mon, 12 Feb 2007 18:50:35 +1100, you wrote:
Yes and no. My point is that it's extremely similar to existing syntax. (Worse yet, it looks the same but for what's possibly the smallest and hardest-to-see character in any character set)
"foo(baz)" vs "foo.(baz)" is... not good.
To me (as a newbee to the language) I only see possible confusion if one gives 'static' looking examples like: x.(foo) += 1 which does indeed look a bit like a function call. However when giving more 'dynamic' looking examples like: x.('foo_%d' % n) += 1 I don't get confused at all and intuitively recognize the intention immediately. In this example I consider the parenthesis 'grouping operators' (which would not be the case when square brackets would have been used) So, +1 for the idea from me, since the main intention is to use it in a 'dynamic' context, and there it would improve readability. All IMHO as a newbee of course :) Ton.
Hi, On Mon, Feb 12, 2007 at 12:38:27AM -0700, Neil Toronto wrote:
obj.*str_expression
x = *('variable%d' % n) f(a, b, *('keyword%d' % n) = c) class *('33strangename'): pass def *(funcname)(x, y, *(argname), *args, **kwds): pass import *modname as mymodule Sorry for the noise, Armin
Armin Rigo wrote:
Hi,
On Mon, Feb 12, 2007 at 12:38:27AM -0700, Neil Toronto wrote:
obj.*str_expression
x = *('variable%d' % n)
f(a, b, *('keyword%d' % n) = c)
class *('33strangename'): pass
def *(funcname)(x, y, *(argname), *args, **kwds): pass
import *modname as mymodule
Are you for these uses or mocking them ? Some of them look interesting... FWIW, at Resolver (75k lines of IronPython code currently) we use getattr / setattr almost as many as 8 times. (I was very surprised at how low that was.) That said, when I explained the proposal to two of my colleagues both of them were *very* happy with the obj.[name] suggestion. Michael Foord
Sorry for the noise,
Armin _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
Hi Michael, On Tue, Feb 13, 2007 at 11:55:46PM +0000, Michael Foord wrote:
x = *('variable%d' % n) f(a, b, *('keyword%d' % n) = c) class *('33strangename'): pass def *(funcname)(x, y, *(argname), *args, **kwds): pass import *modname as mymodule
Are you for these uses or mocking them ? Some of them look interesting...
I don't actually have any strong opinion, I was just exploring all the places in the grammar that say NAME... Some of the above are definitely just absurd. In general I am failing to see much the point of syntax extensions, so I'll step out of this discussion again. Armin
I like the general idea, but the syntax looks like dirt on my monitor. The period is too easy to lose visually and without it, there's nothing to distinguish this from a function call. Also, like Anthony Baxter said, someone coming across this for the first time will think it's a syntax error, allusions to MATLAB and assembly indirection syntax not withstanding. Ignoring the syntax, I'm -1 on the 2-argument form, especially since it can only be used in an expression context; getattr() can be kept around for this. I'm +0 on the idea, -1 on the means. Collin Winter
Collin Winter schrieb:
I like the general idea, but the syntax looks like dirt on my monitor. The period is too easy to lose visually and without it, there's nothing to distinguish this from a function call. Also, like Anthony Baxter said, someone coming across this for the first time will think it's a syntax error, allusions to MATLAB and assembly indirection syntax not withstanding.
Ignoring the syntax, I'm -1 on the 2-argument form, especially since it can only be used in an expression context; getattr() can be kept around for this.
I'm +0 on the idea, -1 on the means.
-1 here too. I fear that this gets too indistinguishable from normal calling syntax, leading to confusion. (Of course, one could propose other syntax, such as obj->(attr) or whatnot, but I doubt an ideal and Pythonic syntax can be found here...) To me, dynamic attribute access is something "special", and it justifies a different way of doing it, namely getattr and setattr. For the speed argument -- there were quite a few proposals to take builtins as constants under certain conditions, in which case getattr() usage could be optimized just as well as new syntax. Georg
Georg Brandl wrote:
For the speed argument -- there were quite a few proposals to take builtins as constants under certain conditions, in which case getattr() usage could be optimized just as well as new syntax.
Even more aggressively, the compiler could recognise it and make a direct call to the __getattr__ method, or maybe even have a new opcode for doing that. In other words, "special syntax" doesn't necessarily have to look like special syntax. :-) -- Greg
On 12-feb-2007, at 0:45, Ben North wrote:
self.(method_name) = self.metadata.(method_name)
I like the functionality, but I don't like the syntax, to me it looks too much like a method call. To me self.[method_name] = self.metadata.[method_name] looks better: what we're doing here is more like dictionary lookup than calling functions. -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman
[Jack Jansen]
I like the functionality, but I don't like the syntax, to me it looks too much like a method call.
To me self.[method_name] = self.metadata.[method_name] looks better: what we're doing here is more like dictionary lookup than calling functions.
I also like the functionality. Rather than munge existing syntaxes, an altogether new one would be more clear: self->name = self.metadata->name I like the arrow syntax because is the lookup process can be more involved than a simple dictionary lookup (perhaps traveling up to base classes). IOW, getattr(a,n) is not always the same as a.__dict__[n]. The a.__getattribute__(n) process can be more complex than that and a bracketed dictionary-like syntax would misleadingly mask the lookup process. Raymond
On 2/12/07, Raymond Hettinger <python@rcn.com> wrote:
[Jack Jansen]
I like the functionality, but I don't like the syntax, to me it looks too much like a method call.
To me self.[method_name] = self.metadata.[method_name] looks better: what we're doing here is more like dictionary lookup than calling functions.
I also like the functionality.
Rather than munge existing syntaxes, an altogether new one would be more clear:
self->name = self.metadata->name
I like the arrow syntax because is the lookup process can be more involved than a simple dictionary lookup (perhaps traveling up to base classes). IOW, getattr(a,n) is not always the same as a.__dict__[n]. The a.__getattribute__(n) process can be more complex than that and a bracketed dictionary-like syntax would misleadingly mask the lookup process.
I actually kind of like that. The connection to pointer indirection meshes well with the idea of indirectly figuring out what attribute to access at runtime. -Brett
On 2/12/07, Brett Cannon <brett@python.org> wrote:
On 2/12/07, Raymond Hettinger <python@rcn.com> wrote:
[Jack Jansen]
I like the functionality, but I don't like the syntax, to me it looks too much like a method call.
To me self.[method_name] = self.metadata.[method_name] looks better: what we're doing here is more like dictionary lookup than calling functions.
I also like the functionality.
Rather than munge existing syntaxes, an altogether new one would be more clear:
self->name = self.metadata->name
I like the arrow syntax because is the lookup process can be more involved than a simple dictionary lookup (perhaps traveling up to base classes). IOW, getattr(a,n) is not always the same as a.__dict__[n]. The a.__getattribute__(n) process can be more complex than that and a bracketed dictionary-like syntax would misleadingly mask the lookup process.
I actually kind of like that. The connection to pointer indirection meshes well with the idea of indirectly figuring out what attribute to access at runtime.
There's a connection, but I'd say it's the wrong one. In C, "x->y" dereferences x, while in Python, "x->y" would dereference y. That's just begging for trouble. Collin Winter
Collin Winter wrote:
On 2/12/07, Brett Cannon <brett@python.org> wrote:
I actually kind of like that. The connection to pointer indirection meshes well with the idea of indirectly figuring out what attribute to access at runtime.
There's a connection, but I'd say it's the wrong one. In C, "x->y" dereferences x, while in Python, "x->y" would dereference y. That's just begging for trouble.
Then the syntax should obviously be "x<-y". [insert in-Soviet-Russia-variables-dereference-you joke here] -- Benji York
2007/2/12, Benji York <benji@benjiyork.com>:
Collin Winter wrote:
There's a connection, but I'd say it's the wrong one. In C, "x->y" dereferences x, while in Python, "x->y" would dereference y. That's just begging for trouble.
Then the syntax should obviously be "x<-y".
<delurk> Someone with OCaml background could confuse that with an assignment <evil_grin/> </delurk> -- Marek Baczyński
Raymond Hettinger wrote:
Rather than munge existing syntaxes, an altogether new one would be more clear:
self->name = self.metadata->name
My problem with this is that it isn't a "name". It should grammatically be a test (i.e. it can take on the expression any function argument could take). How do you spell "getattr(self, 'req_' + state)" with that arrow? You need some kind of grouping delimiters to make this operator be a syntax benefit otherwise you didn't shorten anything. -Scott -- Scott Dial scott@scottdial.com scodial@cs.indiana.edu
* Ben North <ben@redfrontdoor.org> [2007-02-11 23:45:05 +0000]:
Dynamic attribute access is currently possible using the "getattr" and "setattr" builtins. The present PEP suggests a new syntax to make such access easier, allowing the coder for example to write
x.('foo_%d' % n) += 1
z = y.('foo_%d' % n).('bar_%s' % s)
instead of
attr_name = 'foo_%d' % n setattr(x, attr_name, getattr(x, attr_name) + 1)
z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)
Clipper (and other members of the "xBase" family) have something vaguely similar, and from working with that syntax, my feeling is that this is too "subtle" given the mixing of levels going on here (a "literal" name vs. an expression evaluating to a name). That is, it's too easy to not properly notice the presence / absence of the () and become confused between foo.bar and foo.(bar) or similar. (For the curious, the Clipper syntax is generally used in "commands" which are a kind of statement that is macro-evaluated; so, for example, something like this: USE clients will be transformed into something like the following expression: dbUseArea("clients") during the preprocessor pass, care of a "builtin" definition of USE; this function has the effect of opening a database file on disk for use. Now, in order to use the command syntax with a database name stored in a variable, one does something like the following: USE (dbName) which is transformed into something like: dbUseArea(dbname) with similar effects as before.) -- mithrandi, i Ainil en-Balandor, a faer Ambar
On 2/11/07, Ben North <ben@redfrontdoor.org> wrote:
Hi,
A few days ago I posted to python-ideas with a suggestion for some new Python syntax, which would allow easier access to object attributes where the attribute name is known only at run-time. For example:
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
The new syntax would also be usable in augmented assignments, as in
obj.(attr_name) += 1
-1 from me. It does not solve a common problem, therefore it does not deserve a special syntax. Moreover, the existing syntax may be longer to type but is far more readable. -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic."
Ben North wrote:
Hi,
A few days ago I posted to python-ideas with a suggestion for some new Python syntax, which would allow easier access to object attributes where the attribute name is known only at run-time. For example:
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
The new syntax would also be usable in augmented assignments, as in
obj.(attr_name) += 1
There was some discussion on python-ideas, which I've linked to in the draft PEP below. In particular, Guido van Rossum wrote:
I've thought of the same syntax. I think you should submit this to the PEP editor and argue on Python-dev for its inclusion in Python 2.6 -- there's no benefit that I see of waiting until 3.0.
I don't like this. Just because an enhancement is syntactically permissible doesn't mean it's a good idea. This seems to me to take Python further away from the "Computer Programming for Everyone" arena and closer to the "Systems Programming for Clever Individuals" camp.
so here I am. Does anybody have any opinions/suggestions, particularly on the "open questions" referred to in the draft PEP? To summarise these open questions:
* The draft currently allows a two-argument form, to supply a default value if the object has no attribute of that name. This mimics the behaviour of the three-argument form of getattr, but looks a bit wrong:
s = obj.(attr_name, 'default string')
I agree that it looks odd, but perhaps the extra expressive power gained might be worth the oddness.
It looks extremely odd. Since the opening parenthesis takes us into new syntactic territory the unimaginative use of the (already heavily overloaded) comma would sound the idea's death-knell. If we *have* to have this (and I agree with Greg that many uses cases for dynamic attribute access are invalid) then why not s = obj.(attr_name: 'default-string') I presume that the default value would in fact be an arbitrary expression?
* The draft implementation (a sourceforge patch, linked to in the draft PEP) may have a general performance penalty of around 1%, although my initial measurements were quite noisy. Josiah Carlson thought this would not be too substantial, but he did suggest a couple of other implementation routes which could be explored. The general performance hit is offset by a speed gain of around 40% for attribute access using the new syntax over getattr etc. Is 1% "too much" for this feature?
Yes. I believe it would decrease the sum total of Python's efficiency for a very marginal return in performance on a very marginal feature. It threatens to reduce Python's readability substantially, and readability is more important than efficiency. "Expressive power" is all very well as long as the expressiveness is comprehensible. This proposal makes the left parenthesis carry a larger burden than the introduction of generator expressions did, and it makes Python a more difficult language to understand. [provisional PEP snipped] If it's added in 2.6 I propose it should be deprecated in 2.7 and removed from 3.0 ... regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Blog of Note: http://holdenweb.blogspot.com See you at PyCon? http://us.pycon.org/TX2007
On Tue, Feb 13, 2007 at 10:10:37AM +0000, Steve Holden wrote:
Python further away from the "Computer Programming for Everyone" arena and closer to the "Systems Programming for Clever Individuals" camp.
That's because Python is being developed by "Clever Individuals" and not by "Computer Programming for Everyone Committee". Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
On Tue, Feb 13, 2007 at 10:10:37AM +0000, Steve Holden wrote:
Python further away from the "Computer Programming for Everyone" arena and closer to the "Systems Programming for Clever Individuals" camp.
That's because Python is being developed by "Clever Individuals" and not by "Computer Programming for Everyone Committee".
I agree that the developers are Clever Individuals. So clever, in fact, that they realise Python needs to be as approachable as possible, not a language for them alone. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Blog of Note: http://holdenweb.blogspot.com See you at PyCon? http://us.pycon.org/TX2007
On Wed, Feb 14, 2007 at 03:24:30PM +0000, Steve Holden wrote:
Oleg Broytmann wrote:
On Tue, Feb 13, 2007 at 10:10:37AM +0000, Steve Holden wrote:
Python further away from the "Computer Programming for Everyone" arena and closer to the "Systems Programming for Clever Individuals" camp.
That's because Python is being developed by "Clever Individuals" and not by "Computer Programming for Everyone Committee".
I agree that the developers are Clever Individuals. So clever, in fact, that they realise Python needs to be as approachable as possible, not a language for them alone.
Anyway I don't believe it's possible to create a CP4E language without a "CP4E Steering Committee", and I think such committee it Python land is... dormant at present... Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
On Wed, Feb 14, 2007 at 03:24:30PM +0000, Steve Holden wrote:
Oleg Broytmann wrote:
On Tue, Feb 13, 2007 at 10:10:37AM +0000, Steve Holden wrote:
Python further away from the "Computer Programming for Everyone" arena and closer to the "Systems Programming for Clever Individuals" camp. That's because Python is being developed by "Clever Individuals" and not by "Computer Programming for Everyone Committee".
I agree that the developers are Clever Individuals. So clever, in fact, that they realise Python needs to be as approachable as possible, not a language for them alone.
Anyway I don't believe it's possible to create a CP4E language without a "CP4E Steering Committee", and I think such committee it Python land is... dormant at present...
Given that they say "a camel is a horse designed by a committee" I think that language design by committee is very unlikely to produce something well suited to the needs of unsophisticated users. That would require a single individual with good language design skills and the ability to veto features that were out of line with the design requirements. A lot like a BDFL, really. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Blog of Note: http://holdenweb.blogspot.com See you at PyCon? http://us.pycon.org/TX2007
On Thu, Feb 15, 2007 at 12:48:48AM +0000, Steve Holden wrote:
Given that they say "a camel is a horse designed by a committee"
Metaphors can go that far but not farther. And, BTW, camels are very suited for their environments... I am not afraid of committees for large tasks. Well, that has to be a small committee ruling by a cleverest ruler.
require a single individual with good language design skills and the ability to veto features that were out of line with the design requirements. A lot like a BDFL, really.
Of course, but I don't know if "CP4E" idea is still on his agenda and with what priority. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Oleg Broytmann wrote:
Given that they say "a camel is a horse designed by a committee"
BTW, camels are very suited for their environments...
The quote is actually "a camel is a *racehorse* designed by a committee". Camels are very good at surviving in the desert, but not so good at winning a horse race (not camel race). Which is the point of the saying. -- Greg
Greg Ewing wrote:
Oleg Broytmann wrote:
Given that they say "a camel is a horse designed by a committee" BTW, camels are very suited for their environments...
The quote is actually "a camel is a *racehorse* designed by a committee". Camels are very good at surviving in the desert, but not so good at winning a horse race (not camel race). Which is the point of the saying.
As far as I know Sir Alec Issigonis, the inventor of the Mini (the car, not the Mac Mini) said this, and he used "horse", not "racehorse". The point of the saying is that a camel has properties that are completely unnecessary in a horse, such as the ability to travel many days without water. He was saying that committees tend to over-specify and add redundant features rather than designing strictly for purpose. A bit like Python 3.0 ;-) regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Blog of Note: http://holdenweb.blogspot.com See you at PyCon? http://us.pycon.org/TX2007
On Fri, Feb 16, 2007 at 12:40:54PM +1300, Greg Ewing wrote:
The quote is actually "a camel is a *racehorse* designed by a committee". Camels are very good at surviving in the desert, but not so good at winning a horse race (not camel race). Which is the point of the saying.
That changes the meaning, but... have you ever tried to ride a horse designed by a group of Clever Individuals loosely connected by email? ;) I am afraid of even thinking of its ugliness and speed. (-: I think a committee is better than nothing, and I believe CP4E has been dropped from the agenda. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
I think a committee is better than nothing, and I believe CP4E has been dropped from the agenda.
The general CP4E idea is part of the "General Pythonic Ideal", whatever it may be :P -- EduardoOPadoan (eopadoan->altavix::com) Bookmarks: http://del.icio.us/edcrypt
Oleg Broytmann schrieb:
That changes the meaning, but... have you ever tried to ride a horse designed by a group of Clever Individuals loosely connected by email? ;) I am afraid of even thinking of its ugliness and speed. (-: I think a committee is better than nothing, and I believe CP4E has been dropped from the agenda.
Ah, this passive voice again, and again the assumption that there is an agenda of python-dev. Regards, Martin
On Fri, Feb 16, 2007 at 01:42:54PM +0100, "Martin v. L?wis" wrote:
Ah, this passive voice again, and again the assumption that there is an agenda of python-dev.
Exactly opposite. There is no agenda, and thus there is no pojnt in trying to prevent new features in Python language based on the idea the features are not "4E". Python, IMHO, has gone far beyond "4E" already. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Taking a step back a bit... the basic issue is that we have an attribute namespace (compile time key determination) that we want to access in a dictionary style (runtime key determination). This is currently done by switching from syntactic access to the getattr/setattr/delattr builtin functions. Elsewhere in the thread, Calvin made the suggestion that, rather than introducing new syntax, this could instead be achieved with a wrapper class that automatically converted dict-style access into the appropriate calls to getattr/setattr/delattr. I've tried this out on Brett's urllib & urllib2 examples below. (calling the new builtin attrview() to emphasise the fact that it retains a reference to the original instance). I don't consider it any uglier than the proposed syntax changes, and it provides a few other benefits: - the two-argument form is naturally available as the .get() method on the resulting dict-like object (e.g. "attrview(obj).get(some_attr, None)") - hasattr() is naturally replaced by containment testing (e.g. "some_attr in attrview(obj)") - keywords/builtins are easier to look up in the documentation than symbolic syntax With this approach, performance would be attained by arranging to create the view objects once, and then performing multiple dynamic attribute accesses using those view objects. First urllib.py example:: name = 'open_' + urltype self.type = urltype name = name.replace('-', '_') self_d = attrview(self) if name in self_d: if proxy: return self.open_unknown_proxy(proxy, fullurl, data) else: return self.open_unknown(fullurl, data) try: if data is None: return self_d[name](url) else: return self_d[name](url, data) except socket.error, msg: raise IOError, ('socket error', msg), sys.exc_info()[2] Second urllib.py example:: name = 'http_error_%d' % errcode self_d = attrview(self) if name in self_d: method = self_d[name] if data is None: result = method(url, fp, errcode, errmsg, headers) else: result = method(url, fp, errcode, errmsg, headers, data) if result: return result return self.http_error_default(url, fp, errcode, errmsg, headers) First urllib.py example:: if attr[:12] == '_Request__r_': name = attr[12:] get_name = 'get_' + name if get_name in attrview(Request): self_d = attrview(self) self_d[get_name]() return self_d[attr] raise AttributeError, attr Second urllib2.py example:: handlers = chain.get(kind, ()) for handler in handlers: func = attrview(handler)[meth_name] result = func(*args) if result is not None: return result Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org
Nick Coghlan wrote:
I've tried this out on Brett's urllib & urllib2 examples below. (calling the new builtin attrview() to emphasise the fact that it retains a reference to the original instance).
Ooh, ooh! I wanna change my vote! +1 on attrview(), +0.25 on ".[]". Maybe I haven't written enough Python, but I don't think you need this specialized form of accessing attributes very often. So I've shifted to the "new syntax seems like overkill" camp. Besides, once you've got the attrview() you can use all the existing dict syntax and it looks totally clean. The original example: setattr(self, method_name, getattr(self.metadata, method_name) which became in the new syntax: self.[method_name] = self.metadata.[method_name] would be: attrview(self)[method_name] = attrview(self.metadata)[method_name] And an attrview lets you use get() and the in operator. Plus, if you were performing a lot of operations on attributes all at one go, you'd cache it in a local and then it'd look even better. Perhaps object.__attrs__() returns a view on the object's attributes, and attrview() is simply a convenience function. /larry/
On 2/13/07, Nick Coghlan <ncoghlan@gmail.com> wrote:
Taking a step back a bit... the basic issue is that we have an attribute namespace (compile time key determination) that we want to access in a dictionary style (runtime key determination).
This is currently done by switching from syntactic access to the getattr/setattr/delattr builtin functions.
Elsewhere in the thread, Calvin made the suggestion that, rather than introducing new syntax, this could instead be achieved with a wrapper class that automatically converted dict-style access into the appropriate calls to getattr/setattr/delattr.
In other words, an object view analagous to what Py3K is doing with keys()/values()/items().
I've tried this out on Brett's urllib & urllib2 examples below. (calling the new builtin attrview() to emphasise the fact that it retains a reference to the original instance). I don't consider it any uglier than the proposed syntax changes, and it provides a few other benefits:
- the two-argument form is naturally available as the .get() method on the resulting dict-like object (e.g. "attrview(obj).get(some_attr, None)")
- hasattr() is naturally replaced by containment testing (e.g. "some_attr in attrview(obj)")
- keywords/builtins are easier to look up in the documentation than symbolic syntax
Yeah, the generalization is really nice. It allows use to ditch getattr/setattr/hasattr all without losing the expressiveness of those built-ins.
With this approach, performance would be attained by arranging to create the view objects once, and then performing multiple dynamic attribute accesses using those view objects.
First urllib.py example::
name = 'open_' + urltype self.type = urltype name = name.replace('-', '_') self_d = attrview(self) if name in self_d: if proxy: return self.open_unknown_proxy(proxy, fullurl, data) else: return self.open_unknown(fullurl, data) try: if data is None: return self_d[name](url) else: return self_d[name](url, data) except socket.error, msg: raise IOError, ('socket error', msg), sys.exc_info()[2]
Second urllib.py example::
name = 'http_error_%d' % errcode self_d = attrview(self) if name in self_d: method = self_d[name] if data is None: result = method(url, fp, errcode, errmsg, headers) else: result = method(url, fp, errcode, errmsg, headers, data) if result: return result return self.http_error_default(url, fp, errcode, errmsg, headers)
First urllib.py example::
if attr[:12] == '_Request__r_': name = attr[12:] get_name = 'get_' + name if get_name in attrview(Request): self_d = attrview(self) self_d[get_name]() return self_d[attr] raise AttributeError, attr
Second urllib2.py example::
handlers = chain.get(kind, ()) for handler in handlers: func = attrview(handler)[meth_name] result = func(*args) if result is not None: return result
I also think it is just as clean as doing it any of the proposed ways:: getattr(self, name) self.[name] attrview(self)[name] So my vote is for Nick's object attribute view; +1. If we are going to do the view thing, let's go all the way! =) -Brett
On 2/13/07, Nick Coghlan <ncoghlan@gmail.com> wrote: [snip]
I've tried this out on Brett's urllib & urllib2 examples below. (calling the new builtin attrview() to emphasise the fact that it retains a reference to the original instance). I don't consider it any uglier than the proposed syntax changes, and it provides a few other benefits:
- the two-argument form is naturally available as the .get() method on the resulting dict-like object (e.g. "attrview(obj).get(some_attr, None)")
- hasattr() is naturally replaced by containment testing (e.g. "some_attr in attrview(obj)")
- keywords/builtins are easier to look up in the documentation than symbolic syntax
With this approach, performance would be attained by arranging to create the view objects once, and then performing multiple dynamic attribute accesses using those view objects.
This changes my vote: +1 on including attrview(), -1 on the syntax proposal. Collin Winter
On 2/13/07, Nick Coghlan <ncoghlan@gmail.com> wrote: [snip]
I've tried this out on Brett's urllib & urllib2 examples below. (calling the new builtin attrview() to emphasise the fact that it retains a reference to the original instance). I don't consider it any uglier than the proposed syntax changes, and it provides a few other benefits:
- the two-argument form is naturally available as the .get() method on the resulting dict-like object (e.g. "attrview(obj).get(some_attr, None)")
- hasattr() is naturally replaced by containment testing (e.g. "some_attr in attrview(obj)")
- keywords/builtins are easier to look up in the documentation than symbolic syntax
With this approach, performance would be attained by arranging to create the view objects once, and then performing multiple dynamic attribute accesses using those view objects.
On 2/13/07, Collin Winter <collinw@gmail.com> wrote:
This changes my vote: +1 on including attrview(), -1 on the syntax proposal.
Me too! I imagine that the addition of the proposed "obj.[name]" syntax to Python 2.6 would lead to more software that will not run on 2.5 and earlier versions. I don't think the benefits outweigh this backward incompatibility of new code. Therefore I'm -1 on including "obj.[name]" in Python 2.6. For Python 3000, I think it would be alright. With "attrview(obj)[name]" it would be possible to provide a fallback implementation in the same module for pre-2.6 users, so +1 for that. Johann C. Rocholl
participants (25)
-
"Martin v. Löwis" -
Anthony Baxter -
Armin Rigo -
Ben North -
Benji York -
Brett Cannon -
Collin Winter -
Eduardo "EdCrypt" O. Padoan -
Georg Brandl -
Greg Ewing -
Gustavo Carneiro -
Jack Jansen -
Johann C. Rocholl -
Larry Hastings -
Marek \"Baczek\" Baczyński -
Michael Foord -
Neil Toronto -
Nick Coghlan -
Oleg Broytmann -
Raymond Hettinger -
Scott Dial -
Steve Holden -
Terry Reedy -
Ton van Vliet -
Tristan Seligmann