Re: [Python-ideas] New syntax for 'dynamic' attribute access

I've thought of the same syntax. I think you should submit this to the PEP editor and argue on Python-dev for its inclusion in Python 2.6 -- there's no benefit that I see of waiting until 3.0. --Guido On 2/9/07, Ben North wrote:

Hi,

I'd like to describe an addition I made to Python syntax which allows easier access to attributes where the attribute name is only known at run-time. For example:

setattr(self, method_name, getattr(self.metadata, method_name))

from Lib/distutils/dist.py could be rewritten

self.(method_name) = self.metadata.(method_name)

As noted in the PEP-style description below, I mostly did this for fun, but I thought it might be worth bringing to the attention of python-ideas. A quick search through prior postings and Google for this idea didn't come up with anything.

Ben.

- - - - 8< - - - -

PEP: XXX Title: Syntax For Dynamic Attribute Access Version: $Revision$ Last-Modified: $Date$ Author: Ben North Status: Draft Type: Standards Track Content-Type: text/plain Created: 29-Jan-2007 Post-History:

Abstract

Dynamic attribute access is currently possible using the "getattr" and "setattr" builtins. The present PEP suggests a new syntax to make such access easier, allowing the coder for example to write

x.('foo_%d' % n) += 1

z = y.('foo_%d' % n).('bar_%s' % s)

instead of

attr_name = 'foo_%d' % n setattr(x, attr_name, getattr(x, attr_name) + 1)

z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)

Note

(I wrote this patch mostly to advance my own understanding of and experiment with the python language, but I've written it up in the style of a PEP in case it might be a useful idea.)

Rationale

Dictionary access and indexing both have a friendly invocation syntax: instead of x.__getitem__(12) the coder can write x[12]. This also allows the use of subscripted elements in an augmented assignment, as in "x[12] += 1". The present proposal brings this ease-of-use to dynamic attribute access too.

Attribute access is currently possible in two ways:

* When the attribute name is known at code-writing time, the ".NAME" trailer can be used, as in

x.foo = 42 y.bar += 100

* When the attribute name is computed dynamically at run-time, the "getattr" and "setattr" builtins must be used:

x = getattr(y, 'foo_%d' % n) setattr(z, 'bar_%s' % s, 99)

The "getattr" builtin also allows the coder to specify a default value to be returned in the event that the object does not have an attribute of the given name:

x = getattr(y, 'foo_%d' % n, 0)

This PEP describes a new syntax for dynamic attribute access --- "x.(expr)" --- with examples given in the Abstract above. The new syntax also allows the provision of a default value in the "get" case, as in:

x = y.('foo_%d' % n, None)

This 2-argument form of dynamic attribute access is not permitted as the target of an (augmented or normal) assignment. Finally, the new syntax can be used with the "del" statement, as in

del x.(attr_name)

Impact On Existing Code

The proposed new syntax is not currently valid, so no existing well-formed programs have their meaning altered by this proposal.

Across all "*.py" files in the 2.5 distribution, there are around 600 uses of "getattr", "setattr" or "delattr". They break down as follows (figures have some room for error because they were arrived at by partially-manual inspection):

c.300 uses of plain "getattr(x, attr_name)", which could be replaced with the new syntax;

c.150 uses of the 3-argument form, i.e., with the default value; these could be replaced with the 2-argument form of the new syntax (the cases break down into c.125 cases where the attribute name is a literal string, and c.25 where it's only known at run-time);

c.5 uses of the 2-argument form with a literal string attribute name, which I think could be replaced with the standard "x.attribute" syntax;

c.120 uses of setattr, of which 15 use getattr to find the new value; all could be replaced with the new syntax, the 15 where getattr is also involved would show a particular increase in clarity;

c.5 uses which would have to stay as "getattr" because they are calls of a variable named "getattr" whose default value is the builtin "getattr";

c.5 uses of the 2-argument form, inside a try/except block which catches AttributeError and uses a default value instead; these could use 2-argument form of the new syntax;

c.10 uses of "delattr", which could use the new syntax.

As examples, the line

setattr(self, attr, change_root(self.root, getattr(self, attr)))

from Lib/distutils/command/install.py could be rewritten

self.(attr) = change_root(self.root, self.(attr))

and the line

setattr(self, method_name, getattr(self.metadata, method_name))

from Lib/distutils/dist.py could be rewritten

self.(method_name) = self.metadata.(method_name)

Alternative Syntax For The New Feature

Other syntaxes could be used, for example braces are currently invalid in a "trailer", so could be used here, giving

x{'foo_%d' % n} += 1

My personal preference is for the

x.('foo_%d' % n) += 1

syntax though: the presence of the dot shows there is attribute access going on; the parentheses have an analogous meaning to the mathematical "work this out first" meaning. This is also the syntax used in the language Matlab [1] for dynamic "field" access (where "field" is the Matlab term analogous to Python's "attribute").

Error Cases

Only strings are permitted as attribute names, so for instance the following error is produced:

>>> x.(99) = 8 Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: attribute name must be string, not 'int'

This is handled by the existing PyObject_GetAttr function.

Draft Implementation

A draft implementation adds a new alternative to the "trailer" clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in Python.asdl, with accompanying changes to symtable.c, ast.c, and compile.c, and three new opcodes (load/store/del) with accompanying changes to opcode.h and ceval.c. The patch consists of c.180 additional lines in the core code, and c.100 additional lines of tests.

References

[1] Using Dynamic Field Names :: Data Types (MATLAB Programming) http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/f2-41859.h...

Copyright

This document has been placed in the public domain.

[PAGE-BREAK GOES HERE BUT REMOVED FOR EMAIL] Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

- - - - 8< - - - - diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Grammar/Grammar Python-2.5/Grammar/Grammar --- ORIG__Python-2.5/Grammar/Grammar 2006-05-25 12:25:51.000000000 +0100 +++ Python-2.5/Grammar/Grammar 2007-02-01 18:07:04.133160000 +0000 @@ -119,7 +119,7 @@ listmaker: test ( list_for | (',' test)* [','] ) testlist_gexp: test ( gen_for | (',' test)* [','] ) lambdef: 'lambda' [varargslist] ':' test -trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME +trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | '.' '(' test [',' test] ')' subscriptlist: subscript (',' subscript)* [','] subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop] sliceop: ':' [test] diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Include/opcode.h Python-2.5/Include/opcode.h --- ORIG__Python-2.5/Include/opcode.h 2006-02-27 22:32:47.000000000 +0000 +++ Python-2.5/Include/opcode.h 2007-02-02 13:36:58.542848000 +0000 @@ -39,6 +39,10 @@ #define SLICE 30 /* Also uses 31-33 */

+/* LOAD_DYNAMIC_ATTR is below because it takes an argument. */ +#define STORE_DYNAMIC_ATTR 36 +#define DELETE_DYNAMIC_ATTR 37 + #define STORE_SLICE 40 /* Also uses 41-43 */

@@ -89,6 +93,9 @@ #define UNPACK_SEQUENCE 92 /* Number of sequence items */ #define FOR_ITER 93

...

...

...

f() += 1 Traceback (most recent call last): SyntaxError: illegal expression for augmented assignment (, line 1)

+#define LOAD_DYNAMIC_ATTR 94 /* Whether default given; 0 = no, 1 = yes */ +/* STORE_DYNAMIC_ATTR, DELETE_DYNAMIC_ATTR are above; they take no argument. */ + #define STORE_ATTR 95 /* Index in name list */ #define DELETE_ATTR 96 /* "" */ #define STORE_GLOBAL 97 /* "" */ diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Lib/test/test_dynattr.py Python-2.5/Lib/test/test_dynattr.py --- ORIG__Python-2.5/Lib/test/test_dynattr.py 1970-01-01 01:00:00.000000000 +0100 +++ Python-2.5/Lib/test/test_dynattr.py 2007-02-02 17:37:37.982390000 +0000 @@ -0,0 +1,91 @@ +import unittest +import warnings +import sys +from test import test_support + +class AttrHolder: + pass + +class TestDynAttr(unittest.TestCase): + + def test_simple_get(self): + a = AttrHolder() + a.foo = 100 + a.attr_42 = 1 + self.assertEqual(a.('foo'), 100) + self.assertEqual(a.('fo' + 'o'), 100) + self.assertEqual(a.('f' + 'o' + [('o' if True else 'obar')][0]), 100) + self.assertEqual(a.({'FOO': 'fo'}['FOO'] + 'o'), 100) + self.assertEqual(a.('fo%s' % 'o'), 100) + self.assertEqual(a.('attr_42'), 1) + self.assertEqual(a.('attr_%d' % 42), 1) + self.assertEqual(a.('foo' if True else 'attr_42'), 100) + + def test_nested_get(self): + a = AttrHolder() + a.b = AttrHolder() + a.b.c = 1 + attr_name_b = 'b' + attr_name_c = 'c' + self.assertEqual(a.(attr_name_b).(attr_name_c), 1) + + def test_defaulting_get(self): + a = AttrHolder() + a.foo = 100 + self.assertEqual(a.('foo', 99), 100) + self.assertEqual(a.('bar', 99), 99) + self.assertEqual(a.('baz', 99), 99) + self.assertEqual(a.('foo' if True else 'attr_42', 99), 100) + self.assertEqual(a.('foo' if False else 'attr_42', 99), 99) + + @staticmethod + def attempt_non_string_use(attr_name): + a = AttrHolder() + return a.(attr_name) + + def test_only_strings_allowed(self): + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, 99) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, None) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, 1.0) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, AttrHolder) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, sys) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, ()) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, []) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, {}) + self.assertRaises(TypeError, TestDynAttr.attempt_non_string_use, (1, 2)) + + def test_augassign(self): + a = AttrHolder() + a.foo = 100 + a.('foo') += 10 + self.assertEqual(a.foo, 110) + self.assertEqual(a.('fo' + 'o'), 110) + a.('f' + 'o' + 'o') *= 10 + self.assertEqual(a.foo, 1100) + self.assertEqual(a.('fo' + 'o'), 1100) + a.('foobar'[:3]) /= 5 + self.assertEqual(a.foo, 220) + self.assertEqual(a.('fo' + 'o'), 220) + a.(['foo', 'bar', 'baz'][0]) -= 40 + self.assertEqual(a.foo, 180) + self.assertEqual(a.('fo' + 'o'), 180) + + def test_setattr(self): + a = AttrHolder() + a.('foo') = 99 + self.assertEqual(a.foo, 99) + a.('bar' + '_baz') = 100 + self.assertEqual(a.bar_baz, 100) + + def test_delattr(self): + a = AttrHolder() + a.foo = 99 + del a.('foo') + self.assertEqual(hasattr(a, 'foo'), False) + + +def test_main(): + test_support.run_unittest(TestDynAttr) + +if __name__ == "__main__": + test_main() diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Lib/test/test_syntax.py Python-2.5/Lib/test/test_syntax.py --- ORIG__Python-2.5/Lib/test/test_syntax.py 2006-05-19 07:43:50.000000000 +0100 +++ Python-2.5/Lib/test/test_syntax.py 2007-02-02 17:21:04.991119000 +0000 @@ -235,6 +235,18 @@ + +Outlawed uses of dynamic attributes: + +>>> x.('foo', 0) += 1 +Traceback (most recent call last): +SyntaxError: augmented assignment to 2-argument dynamic-attribute expression not possible (, line 1) +>>> x.('foo', 0) = 1 +Traceback (most recent call last): +SyntaxError: can't assign to 2-argument form of dynamic-attribute expression (, line 1) +>>> del x.('foo', 0) +Traceback (most recent call last): +SyntaxError: can't delete 2-argument form of dynamic-attribute expression (, line 1) """

import re diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Parser/Python.asdl Python-2.5/Parser/Python.asdl --- ORIG__Python-2.5/Parser/Python.asdl 2006-04-04 05:00:23.000000000 +0100 +++ Python-2.5/Parser/Python.asdl 2007-02-02 12:39:36.665151000 +0000 @@ -72,6 +72,7 @@

-- the following expression can appear in assignment context | Attribute(expr value, identifier attr, expr_context ctx) + | DynamicAttribute(expr value, expr attr, expr? dflt, expr_context ctx) | Subscript(expr value, slice slice, expr_context ctx) | Name(identifier id, expr_context ctx) | List(expr* elts, expr_context ctx) diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/ast.c Python-2.5/Python/ast.c --- ORIG__Python-2.5/Python/ast.c 2006-09-05 04:56:01.000000000 +0100 +++ Python-2.5/Python/ast.c 2007-02-02 13:54:51.388446000 +0000 @@ -353,6 +353,13 @@ } e->v.Attribute.ctx = ctx; break; + case DynamicAttribute_kind: + if ((ctx == Store || ctx == Del) + && e->v.DynamicAttribute.dflt) + expr_name = "2-argument form of dynamic-attribute expression"; + else + e->v.DynamicAttribute.ctx = ctx; + break; case Subscript_kind: e->v.Subscript.ctx = ctx; break; @@ -1427,7 +1434,7 @@ static expr_ty ast_for_trailer(struct compiling *c, const node *n, expr_ty left_expr) { - /* trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME + /* trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME | '.' '(' test [',' test] ')' subscriptlist: subscript (',' subscript)* [','] subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop] */ @@ -1440,8 +1447,30 @@ return ast_for_call(c, CHILD(n, 1), left_expr); } else if (TYPE(CHILD(n, 0)) == DOT ) { + if (TYPE(CHILD(n, 1)) == NAME) return Attribute(left_expr, NEW_IDENTIFIER(CHILD(n, 1)), Load, LINENO(n), n->n_col_offset, c->c_arena); + else { + expr_ty e_val, e_dflt; + REQ(CHILD(n, 1), LPAR); + if (!(e_val = ast_for_expr(c, CHILD(n, 2)))) + return NULL; + + if (TYPE(CHILD(n, 3)) == RPAR) { + assert(NCH(n) == 3); + e_dflt = NULL; + } else { + assert(NCH(n) == 5); + REQ(CHILD(n, 3), COMMA); + REQ(CHILD(n, 5), RPAR); + if (!(e_dflt = ast_for_expr(c, CHILD(n, 4)))) + return NULL; + } + + return DynamicAttribute(left_expr, e_val, e_dflt, Load, + LINENO(n), n->n_col_offset, + c->c_arena); + } } else { REQ(CHILD(n, 0), LSQB); @@ -1964,6 +1993,15 @@ case Attribute_kind: case Subscript_kind: break; + case DynamicAttribute_kind: + if (expr1->v.DynamicAttribute.dflt) { + ast_error(ch, "augmented assignment to " + "2-argument dynamic-attribute " + "expression not possible"); + return NULL; + } + break; + default: ast_error(ch, "illegal expression for augmented " "assignment"); diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/ceval.c Python-2.5/Python/ceval.c --- ORIG__Python-2.5/Python/ceval.c 2006-08-13 19:10:10.000000000 +0100 +++ Python-2.5/Python/ceval.c 2007-02-08 12:56:39.637479000 +0000 @@ -1787,6 +1787,17 @@ if (err == 0) continue; break;

+ case STORE_DYNAMIC_ATTR: + w = POP(); + v = POP(); + u = POP(); + err = PyObject_SetAttr(v, w, u); /* v.w = u */ + Py_DECREF(w); + Py_DECREF(v); + Py_DECREF(u); + if (err == 0) continue; + break; + case DELETE_ATTR: w = GETITEM(names, oparg); v = POP(); @@ -1795,6 +1806,14 @@ Py_DECREF(v); break;

+ case DELETE_DYNAMIC_ATTR: + w = POP(); + v = POP(); + err = PyObject_SetAttr(v, w, (PyObject *)NULL); + Py_DECREF(w); + Py_DECREF(v); + break; + case STORE_GLOBAL: w = GETITEM(names, oparg); v = POP(); @@ -1994,6 +2013,28 @@ if (x != NULL) continue; break;

+ case LOAD_DYNAMIC_ATTR: + if (oparg) + u = POP(); + else + u = NULL; + w = POP(); + v = TOP(); + x = PyObject_GetAttr(v, w); + if (x == NULL && u != NULL + && PyErr_ExceptionMatches(PyExc_AttributeError)) + { + PyErr_Clear(); + Py_INCREF(u); + x = u; + } + Py_DECREF(v); + Py_DECREF(w); + Py_XDECREF(u); /* This one may be NULL (if no default) */ + SET_TOP(x); + if (x != NULL) continue; + break; + case COMPARE_OP: w = POP(); v = TOP(); diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/compile.c Python-2.5/Python/compile.c --- ORIG__Python-2.5/Python/compile.c 2006-08-12 02:45:47.000000000 +0100 +++ Python-2.5/Python/compile.c 2007-02-08 12:55:51.958809000 +0000 @@ -1357,6 +1357,13 @@ case SLICE+3: return -1;

+ case LOAD_DYNAMIC_ATTR: + return -1 - oparg; + case STORE_DYNAMIC_ATTR: + return -3; + case DELETE_DYNAMIC_ATTR: + return -2; + case STORE_SLICE+0: return -2; case STORE_SLICE+1: @@ -3641,6 +3648,41 @@ return 0; } break; + case DynamicAttribute_kind: + { + int has_default_p = (e->v.DynamicAttribute.dflt != NULL); + + if (e->v.DynamicAttribute.ctx != AugStore) { + VISIT(c, expr, e->v.DynamicAttribute.value); + VISIT(c, expr, e->v.DynamicAttribute.attr); + if (has_default_p) + VISIT(c, expr, e->v.DynamicAttribute.dflt); + } + switch (e->v.DynamicAttribute.ctx) { + case AugLoad: + assert(!has_default_p); + ADDOP_I(c, DUP_TOPX, 2); + /* Fall through to Load */ + case Load: + ADDOP_I(c, LOAD_DYNAMIC_ATTR, has_default_p); + break; + case AugStore: + ADDOP(c, ROT_THREE); + /* Fall through to Store */ + case Store: + assert(!has_default_p); + ADDOP(c, STORE_DYNAMIC_ATTR); + break; + case Del: + ADDOP(c, DELETE_DYNAMIC_ATTR); + break; + default: + PyErr_SetString(PyExc_SystemError, + "invalid context in dynamic-attribute expression"); + return 0; + } + break; + } case Subscript_kind: switch (e->v.Subscript.ctx) { case AugLoad: @@ -3700,6 +3742,18 @@ auge->v.Attribute.ctx = AugStore; VISIT(c, expr, auge); break; + case DynamicAttribute_kind: + assert(e->v.DynamicAttribute.dflt == NULL); + auge = DynamicAttribute(e->v.DynamicAttribute.value, e->v.DynamicAttribute.attr, NULL, + AugLoad, e->lineno, e->col_offset, c->c_arena); + if (auge == NULL) + return 0; + VISIT(c, expr, auge); + VISIT(c, expr, s->v.AugAssign.value); + ADDOP(c, inplace_binop(c, s->v.AugAssign.op)); + auge->v.DynamicAttribute.ctx = AugStore; + VISIT(c, expr, auge); + break; case Subscript_kind: auge = Subscript(e->v.Subscript.value, e->v.Subscript.slice, AugLoad, e->lineno, e->col_offset, c->c_arena); diff --exclude=graminit.c --exclude='Python-ast.[ch]' -Nwuar ORIG__Python-2.5/Python/symtable.c Python-2.5/Python/symtable.c --- ORIG__Python-2.5/Python/symtable.c 2006-08-12 02:43:40.000000000 +0100 +++ Python-2.5/Python/symtable.c 2007-02-08 12:53:36.279608000 +0000 @@ -1192,6 +1192,12 @@ case Attribute_kind: VISIT(st, expr, e->v.Attribute.value); break; + case DynamicAttribute_kind: + VISIT(st, expr, e->v.DynamicAttribute.value); + VISIT(st, expr, e->v.DynamicAttribute.attr); + if (e->v.DynamicAttribute.dflt) + VISIT(st, expr, e->v.DynamicAttribute.dflt); + break; case Subscript_kind: VISIT(st, expr, e->v.Subscript.value); VISIT(st, slice, e->v.Subscript.slice);

_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

-- --Guido van Rossum (home page: http://www.python.org/~guido/)