[Python-ideas] New syntax for 'dynamic' attribute access
Ben North
ben at redfrontdoor.org
Fri Feb 9 09:31:22 CET 2007
Hi,
I'd like to describe an addition I made to Python syntax which allows
easier access to attributes where the attribute name is only known at
run-time. For example:
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
As noted in the PEP-style description below, I mostly did this for
fun, but I thought it might be worth bringing to the attention of
python-ideas. A quick search through prior postings and Google for
this idea didn't come up with anything.
Ben.
- - - - 8< - - - -
PEP: XXX
Title: Syntax For Dynamic Attribute Access
Version: $Revision$
Last-Modified: $Date$
Author: Ben North <ben at redfrontdoor.org>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 29-Jan-2007
Post-History:
Abstract
Dynamic attribute access is currently possible using the "getattr"
and "setattr" builtins. The present PEP suggests a new syntax to
make such access easier, allowing the coder for example to write
x.('foo_%d' % n) += 1
z = y.('foo_%d' % n).('bar_%s' % s)
instead of
attr_name = 'foo_%d' % n
setattr(x, attr_name, getattr(x, attr_name) + 1)
z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)
Note
(I wrote this patch mostly to advance my own understanding of and
experiment with the python language, but I've written it up in the
style of a PEP in case it might be a useful idea.)
Rationale
Dictionary access and indexing both have a friendly invocation
syntax: instead of x.__getitem__(12) the coder can write x[12].
This also allows the use of subscripted elements in an augmented
assignment, as in "x[12] += 1". The present proposal brings this
ease-of-use to dynamic attribute access too.
Attribute access is currently possible in two ways:
* When the attribute name is known at code-writing time, the
".NAME" trailer can be used, as in
x.foo = 42
y.bar += 100
* When the attribute name is computed dynamically at run-time, the
"getattr" and "setattr" builtins must be used:
x = getattr(y, 'foo_%d' % n)
setattr(z, 'bar_%s' % s, 99)
The "getattr" builtin also allows the coder to specify a default
value to be returned in the event that the object does not have
an attribute of the given name:
x = getattr(y, 'foo_%d' % n, 0)
This PEP describes a new syntax for dynamic attribute access ---
"x.(expr)" --- with examples given in the Abstract above. The new
syntax also allows the provision of a default value in the "get"
case, as in:
x = y.('foo_%d' % n, None)
This 2-argument form of dynamic attribute access is not permitted
as the target of an (augmented or normal) assignment. Finally,
the new syntax can be used with the "del" statement, as in
del x.(attr_name)
Impact On Existing Code
The proposed new syntax is not currently valid, so no existing
well-formed programs have their meaning altered by this proposal.
Across all "*.py" files in the 2.5 distribution, there are around
600 uses of "getattr", "setattr" or "delattr". They break down as
follows (figures have some room for error because they were
arrived at by partially-manual inspection):
c.300 uses of plain "getattr(x, attr_name)", which could be
replaced with the new syntax;
c.150 uses of the 3-argument form, i.e., with the default
value; these could be replaced with the 2-argument form
of the new syntax (the cases break down into c.125 cases
where the attribute name is a literal string, and c.25
where it's only known at run-time);
c.5 uses of the 2-argument form with a literal string
attribute name, which I think could be replaced with the
standard "x.attribute" syntax;
c.120 uses of setattr, of which 15 use getattr to find the
new value; all could be replaced with the new syntax,
the 15 where getattr is also involved would show a
particular increase in clarity;
c.5 uses which would have to stay as "getattr" because they
are calls of a variable named "getattr" whose default
value is the builtin "getattr";
c.5 uses of the 2-argument form, inside a try/except block
which catches AttributeError and uses a default value
instead; these could use 2-argument form of the new
syntax;
c.10 uses of "delattr", which could use the new syntax.
As examples, the line
setattr(self, attr, change_root(self.root, getattr(self, attr)))
from Lib/distutils/command/install.py could be rewritten
self.(attr) = change_root(self.root, self.(attr))
and the line
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
Alternative Syntax For The New Feature
Other syntaxes could be used, for example braces are currently
invalid in a "trailer", so could be used here, giving
x{'foo_%d' % n} += 1
My personal preference is for the
x.('foo_%d' % n) += 1
syntax though: the presence of the dot shows there is attribute
access going on; the parentheses have an analogous meaning to the
mathematical "work this out first" meaning. This is also the
syntax used in the language Matlab [1] for dynamic "field" access
(where "field" is the Matlab term analogous to Python's
"attribute").
Error Cases
Only strings are permitted as attribute names, so for instance the
following error is produced:
>>> x.(99) = 8
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: attribute name must be string, not 'int'
This is handled by the existing PyObject_GetAttr function.
Draft Implementation
A draft implementation adds a new alternative to the "trailer"
clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in
Python.asdl, with accompanying changes to symtable.c, ast.c, and
compile.c, and three new opcodes (load/store/del) with
accompanying changes to opcode.h and ceval.c. The patch consists
of c.180 additional lines in the core code, and c.100 additional
lines of tests.
References
[1] Using Dynamic Field Names :: Data Types (MATLAB Programming)
http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/f2-41859.html
Copyright
This document has been placed in the public domain.
[PAGE-BREAK GOES HERE BUT REMOVED FOR EMAIL]
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
- - - - 8< - - - -
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Grammar/Grammar Python-2.5/Grammar/Grammar
--- ORIG__Python-2.5/Grammar/Grammar 2006-05-25 12:25:51.000000000 +0100
+++ Python-2.5/Grammar/Grammar 2007-02-01 18:07:04.133160000 +0000
@@ -119,7 +119,7 @@
listmaker: test ( list_for | (',' test)* [','] )
testlist_gexp: test ( gen_for | (',' test)* [','] )
lambdef: 'lambda' [varargslist] ':' test
-trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
+trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | '.' '(' test [',' test] ')'
subscriptlist: subscript (',' subscript)* [',']
subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
sliceop: ':' [test]
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Include/opcode.h Python-2.5/Include/opcode.h
--- ORIG__Python-2.5/Include/opcode.h 2006-02-27 22:32:47.000000000 +0000
+++ Python-2.5/Include/opcode.h 2007-02-02 13:36:58.542848000 +0000
@@ -39,6 +39,10 @@
#define SLICE 30
/* Also uses 31-33 */
+/* LOAD_DYNAMIC_ATTR is below because it takes an argument. */
+#define STORE_DYNAMIC_ATTR 36
+#define DELETE_DYNAMIC_ATTR 37
+
#define STORE_SLICE 40
/* Also uses 41-43 */
@@ -89,6 +93,9 @@
#define UNPACK_SEQUENCE 92 /* Number of sequence items */
#define FOR_ITER 93
+#define LOAD_DYNAMIC_ATTR 94 /* Whether default given; 0 = no, 1 = yes */
+/* STORE_DYNAMIC_ATTR, DELETE_DYNAMIC_ATTR are above; they take no argument. */
+
#define STORE_ATTR 95 /* Index in name list */
#define DELETE_ATTR 96 /* "" */
#define STORE_GLOBAL 97 /* "" */
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Lib/test/test_dynattr.py Python-2.5/Lib/test/test_dynattr.py
--- ORIG__Python-2.5/Lib/test/test_dynattr.py 1970-01-01 01:00:00.000000000 +0100
+++ Python-2.5/Lib/test/test_dynattr.py 2007-02-02 17:37:37.982390000 +0000
@@ -0,0 +1,91 @@
+import unittest
+import warnings
+import sys
+from test import test_support
+
+class AttrHolder:
+ pass
+
+class TestDynAttr(unittest.TestCase):
+
+ def test_simple_get(self):
+ a = AttrHolder()
+ a.foo = 100
+ a.attr_42 = 1
+ self.assertEqual(a.('foo'), 100)
+ self.assertEqual(a.('fo' + 'o'), 100)
+ self.assertEqual(a.('f' + 'o' + [('o' if True else 'obar')][0]), 100)
+ self.assertEqual(a.({'FOO': 'fo'}['FOO'] + 'o'), 100)
+ self.assertEqual(a.('fo%s' % 'o'), 100)
+ self.assertEqual(a.('attr_42'), 1)
+ self.assertEqual(a.('attr_%d' % 42), 1)
+ self.assertEqual(a.('foo' if True else 'attr_42'), 100)
+
+ def test_nested_get(self):
+ a = AttrHolder()
+ a.b = AttrHolder()
+ a.b.c = 1
+ attr_name_b = 'b'
+ attr_name_c = 'c'
+ self.assertEqual(a.(attr_name_b).(attr_name_c), 1)
+
+ def test_defaulting_get(self):
+ a = AttrHolder()
+ a.foo = 100
+ self.assertEqual(a.('foo', 99), 100)
+ self.assertEqual(a.('bar', 99), 99)
+ self.assertEqual(a.('baz', 99), 99)
+ self.assertEqual(a.('foo' if True else 'attr_42', 99), 100)
+ self.assertEqual(a.('foo' if False else 'attr_42', 99), 99)
+
+ @staticmethod
+ def attempt_non_string_use(attr_name):
+ a = AttrHolder()
+ return a.(attr_name)
+
+ def test_only_strings_allowed(self):
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, 99)
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, None)
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, 1.0)
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, AttrHolder)
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, sys)
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, ())
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, [])
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, {})
+ self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, (1, 2))
+
+ def test_augassign(self):
+ a = AttrHolder()
+ a.foo = 100
+ a.('foo') += 10
+ self.assertEqual(a.foo, 110)
+ self.assertEqual(a.('fo' + 'o'), 110)
+ a.('f' + 'o' + 'o') *= 10
+ self.assertEqual(a.foo, 1100)
+ self.assertEqual(a.('fo' + 'o'), 1100)
+ a.('foobar'[:3]) /= 5
+ self.assertEqual(a.foo, 220)
+ self.assertEqual(a.('fo' + 'o'), 220)
+ a.(['foo', 'bar', 'baz'][0]) -= 40
+ self.assertEqual(a.foo, 180)
+ self.assertEqual(a.('fo' + 'o'), 180)
+
+ def test_setattr(self):
+ a = AttrHolder()
+ a.('foo') = 99
+ self.assertEqual(a.foo, 99)
+ a.('bar' + '_baz') = 100
+ self.assertEqual(a.bar_baz, 100)
+
+ def test_delattr(self):
+ a = AttrHolder()
+ a.foo = 99
+ del a.('foo')
+ self.assertEqual(hasattr(a, 'foo'), False)
+
+
+def test_main():
+ test_support.run_unittest(TestDynAttr)
+
+if __name__ == "__main__":
+ test_main()
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Lib/test/test_syntax.py Python-2.5/Lib/test/test_syntax.py
--- ORIG__Python-2.5/Lib/test/test_syntax.py 2006-05-19 07:43:50.000000000 +0100
+++ Python-2.5/Lib/test/test_syntax.py 2007-02-02 17:21:04.991119000 +0000
@@ -235,6 +235,18 @@
>>> f() += 1
Traceback (most recent call last):
SyntaxError: illegal expression for augmented assignment (<doctest test.test_syntax[33]>, line 1)
+
+Outlawed uses of dynamic attributes:
+
+>>> x.('foo', 0) += 1
+Traceback (most recent call last):
+SyntaxError: augmented assignment to 2-argument dynamic-attribute expression not possible (<doctest test.test_syntax[34]>, line 1)
+>>> x.('foo', 0) = 1
+Traceback (most recent call last):
+SyntaxError: can't assign to 2-argument form of dynamic-attribute expression (<doctest test.test_syntax[35]>, line 1)
+>>> del x.('foo', 0)
+Traceback (most recent call last):
+SyntaxError: can't delete 2-argument form of dynamic-attribute expression (<doctest test.test_syntax[36]>, line 1)
"""
import re
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Parser/Python.asdl Python-2.5/Parser/Python.asdl
--- ORIG__Python-2.5/Parser/Python.asdl 2006-04-04 05:00:23.000000000 +0100
+++ Python-2.5/Parser/Python.asdl 2007-02-02 12:39:36.665151000 +0000
@@ -72,6 +72,7 @@
-- the following expression can appear in assignment context
| Attribute(expr value, identifier attr, expr_context ctx)
+ | DynamicAttribute(expr value, expr attr, expr? dflt, expr_context ctx)
| Subscript(expr value, slice slice, expr_context ctx)
| Name(identifier id, expr_context ctx)
| List(expr* elts, expr_context ctx)
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/ast.c Python-2.5/Python/ast.c
--- ORIG__Python-2.5/Python/ast.c 2006-09-05 04:56:01.000000000 +0100
+++ Python-2.5/Python/ast.c 2007-02-02 13:54:51.388446000 +0000
@@ -353,6 +353,13 @@
}
e->v.Attribute.ctx = ctx;
break;
+ case DynamicAttribute_kind:
+ if ((ctx == Store || ctx == Del)
+ && e->v.DynamicAttribute.dflt)
+ expr_name = "2-argument form of dynamic-attribute expression";
+ else
+ e->v.DynamicAttribute.ctx = ctx;
+ break;
case Subscript_kind:
e->v.Subscript.ctx = ctx;
break;
@@ -1427,7 +1434,7 @@
static expr_ty
ast_for_trailer(struct compiling *c, const node *n, expr_ty left_expr)
{
- /* trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
+ /* trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | '.' '(' test [',' test] ')'
subscriptlist: subscript (',' subscript)* [',']
subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
*/
@@ -1440,8 +1447,30 @@
return ast_for_call(c, CHILD(n, 1), left_expr);
}
else if (TYPE(CHILD(n, 0)) == DOT ) {
+ if (TYPE(CHILD(n, 1)) == NAME)
return Attribute(left_expr, NEW_IDENTIFIER(CHILD(n, 1)), Load,
LINENO(n), n->n_col_offset, c->c_arena);
+ else {
+ expr_ty e_val, e_dflt;
+ REQ(CHILD(n, 1), LPAR);
+ if (!(e_val = ast_for_expr(c, CHILD(n, 2))))
+ return NULL;
+
+ if (TYPE(CHILD(n, 3)) == RPAR) {
+ assert(NCH(n) == 3);
+ e_dflt = NULL;
+ } else {
+ assert(NCH(n) == 5);
+ REQ(CHILD(n, 3), COMMA);
+ REQ(CHILD(n, 5), RPAR);
+ if (!(e_dflt = ast_for_expr(c, CHILD(n, 4))))
+ return NULL;
+ }
+
+ return DynamicAttribute(left_expr, e_val, e_dflt, Load,
+ LINENO(n), n->n_col_offset,
+ c->c_arena);
+ }
}
else {
REQ(CHILD(n, 0), LSQB);
@@ -1964,6 +1993,15 @@
case Attribute_kind:
case Subscript_kind:
break;
+ case DynamicAttribute_kind:
+ if (expr1->v.DynamicAttribute.dflt) {
+ ast_error(ch, "augmented assignment to "
+ "2-argument dynamic-attribute "
+ "expression not possible");
+ return NULL;
+ }
+ break;
+
default:
ast_error(ch, "illegal expression for augmented "
"assignment");
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/ceval.c Python-2.5/Python/ceval.c
--- ORIG__Python-2.5/Python/ceval.c 2006-08-13 19:10:10.000000000 +0100
+++ Python-2.5/Python/ceval.c 2007-02-08 12:56:39.637479000 +0000
@@ -1787,6 +1787,17 @@
if (err == 0) continue;
break;
+ case STORE_DYNAMIC_ATTR:
+ w = POP();
+ v = POP();
+ u = POP();
+ err = PyObject_SetAttr(v, w, u); /* v.w = u */
+ Py_DECREF(w);
+ Py_DECREF(v);
+ Py_DECREF(u);
+ if (err == 0) continue;
+ break;
+
case DELETE_ATTR:
w = GETITEM(names, oparg);
v = POP();
@@ -1795,6 +1806,14 @@
Py_DECREF(v);
break;
+ case DELETE_DYNAMIC_ATTR:
+ w = POP();
+ v = POP();
+ err = PyObject_SetAttr(v, w, (PyObject *)NULL);
+ Py_DECREF(w);
+ Py_DECREF(v);
+ break;
+
case STORE_GLOBAL:
w = GETITEM(names, oparg);
v = POP();
@@ -1994,6 +2013,28 @@
if (x != NULL) continue;
break;
+ case LOAD_DYNAMIC_ATTR:
+ if (oparg)
+ u = POP();
+ else
+ u = NULL;
+ w = POP();
+ v = TOP();
+ x = PyObject_GetAttr(v, w);
+ if (x == NULL && u != NULL
+ && PyErr_ExceptionMatches(PyExc_AttributeError))
+ {
+ PyErr_Clear();
+ Py_INCREF(u);
+ x = u;
+ }
+ Py_DECREF(v);
+ Py_DECREF(w);
+ Py_XDECREF(u); /* This one may be NULL (if no default) */
+ SET_TOP(x);
+ if (x != NULL) continue;
+ break;
+
case COMPARE_OP:
w = POP();
v = TOP();
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/compile.c Python-2.5/Python/compile.c
--- ORIG__Python-2.5/Python/compile.c 2006-08-12 02:45:47.000000000 +0100
+++ Python-2.5/Python/compile.c 2007-02-08 12:55:51.958809000 +0000
@@ -1357,6 +1357,13 @@
case SLICE+3:
return -1;
+ case LOAD_DYNAMIC_ATTR:
+ return -1 - oparg;
+ case STORE_DYNAMIC_ATTR:
+ return -3;
+ case DELETE_DYNAMIC_ATTR:
+ return -2;
+
case STORE_SLICE+0:
return -2;
case STORE_SLICE+1:
@@ -3641,6 +3648,41 @@
return 0;
}
break;
+ case DynamicAttribute_kind:
+ {
+ int has_default_p = (e->v.DynamicAttribute.dflt != NULL);
+
+ if (e->v.DynamicAttribute.ctx != AugStore) {
+ VISIT(c, expr, e->v.DynamicAttribute.value);
+ VISIT(c, expr, e->v.DynamicAttribute.attr);
+ if (has_default_p)
+ VISIT(c, expr, e->v.DynamicAttribute.dflt);
+ }
+ switch (e->v.DynamicAttribute.ctx) {
+ case AugLoad:
+ assert(!has_default_p);
+ ADDOP_I(c, DUP_TOPX, 2);
+ /* Fall through to Load */
+ case Load:
+ ADDOP_I(c, LOAD_DYNAMIC_ATTR, has_default_p);
+ break;
+ case AugStore:
+ ADDOP(c, ROT_THREE);
+ /* Fall through to Store */
+ case Store:
+ assert(!has_default_p);
+ ADDOP(c, STORE_DYNAMIC_ATTR);
+ break;
+ case Del:
+ ADDOP(c, DELETE_DYNAMIC_ATTR);
+ break;
+ default:
+ PyErr_SetString(PyExc_SystemError,
+ "invalid context in dynamic-attribute expression");
+ return 0;
+ }
+ break;
+ }
case Subscript_kind:
switch (e->v.Subscript.ctx) {
case AugLoad:
@@ -3700,6 +3742,18 @@
auge->v.Attribute.ctx = AugStore;
VISIT(c, expr, auge);
break;
+ case DynamicAttribute_kind:
+ assert(e->v.DynamicAttribute.dflt == NULL);
+ auge = DynamicAttribute(e->v.DynamicAttribute.value, e->v.DynamicAttribute.attr, NULL,
+ AugLoad, e->lineno, e->col_offset, c->c_arena);
+ if (auge == NULL)
+ return 0;
+ VISIT(c, expr, auge);
+ VISIT(c, expr, s->v.AugAssign.value);
+ ADDOP(c, inplace_binop(c, s->v.AugAssign.op));
+ auge->v.DynamicAttribute.ctx = AugStore;
+ VISIT(c, expr, auge);
+ break;
case Subscript_kind:
auge = Subscript(e->v.Subscript.value, e->v.Subscript.slice,
AugLoad, e->lineno, e->col_offset, c->c_arena);
diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/symtable.c Python-2.5/Python/symtable.c
--- ORIG__Python-2.5/Python/symtable.c 2006-08-12 02:43:40.000000000 +0100
+++ Python-2.5/Python/symtable.c 2007-02-08 12:53:36.279608000 +0000
@@ -1192,6 +1192,12 @@
case Attribute_kind:
VISIT(st, expr, e->v.Attribute.value);
break;
+ case DynamicAttribute_kind:
+ VISIT(st, expr, e->v.DynamicAttribute.value);
+ VISIT(st, expr, e->v.DynamicAttribute.attr);
+ if (e->v.DynamicAttribute.dflt)
+ VISIT(st, expr, e->v.DynamicAttribute.dflt);
+ break;
case Subscript_kind:
VISIT(st, expr, e->v.Subscript.value);
VISIT(st, slice, e->v.Subscript.slice);
More information about the Python-ideas
mailing list