[Python-Dev] New syntax for 'dynamic' attribute access

Ben North ben at redfrontdoor.org
Mon Feb 12 00:45:05 CET 2007


A few days ago I posted to python-ideas with a suggestion for some new
Python syntax, which would allow easier access to object attributes
where the attribute name is known only at run-time.  For example:

   setattr(self, method_name, getattr(self.metadata, method_name))

from Lib/distutils/dist.py could be rewritten

   self.(method_name) = self.metadata.(method_name)

The new syntax would also be usable in augmented assignments, as in

   obj.(attr_name) += 1

There was some discussion on python-ideas, which I've linked to in the
draft PEP below.  In particular, Guido van Rossum wrote:

 > I've thought of the same syntax. I think you should submit this to the
 > PEP editor and argue on Python-dev for its inclusion in Python 2.6 --
 > there's no benefit that I see of waiting until 3.0.

so here I am.  Does anybody have any opinions/suggestions, particularly
on the "open questions" referred to in the draft PEP?  To summarise
these open questions:

* The draft currently allows a two-argument form, to supply a default
  value if the object has no attribute of that name.  This mimics the
  behaviour of the three-argument form of getattr, but looks a bit wrong:

     s = obj.(attr_name, 'default string')

  I agree that it looks odd, but perhaps the extra expressive power
  gained might be worth the oddness.

* The draft implementation (a sourceforge patch, linked to in the draft
  PEP) may have a general performance penalty of around 1%, although my
  initial measurements were quite noisy.  Josiah Carlson thought this
  would not be too substantial, but he did suggest a couple of other
  implementation routes which could be explored.  The general
  performance hit is offset by a speed gain of around 40% for attribute
  access using the new syntax over getattr etc.  Is 1% "too much" for
  this feature?


- - - - 8< - - - -
Title: Syntax For Dynamic Attribute Access
Version: $Revision$
Last-Modified: $Date$
Author: Ben North <ben at redfrontdoor.org>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 29-Jan-2007


    Dynamic attribute access is currently possible using the "getattr"
    and "setattr" builtins.  The present PEP suggests a new syntax to
    make such access easier, allowing the coder for example to write

        x.('foo_%d' % n) += 1

        z = y.('foo_%d' % n).('bar_%s' % s)

    instead of

        attr_name = 'foo_%d' % n
        setattr(x, attr_name, getattr(x, attr_name) + 1)

        z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)


    (I wrote this patch mostly to advance my own understanding of and
    experiment with the python language, but I've written it up in the
    style of a PEP in case it might be a useful idea.)


    Dictionary access and indexing both have a friendly invocation
    syntax: instead of x.__getitem__(12) the coder can write x[12].
    This also allows the use of subscripted elements in an augmented
    assignment, as in "x[12] += 1".  The present proposal brings this
    ease-of-use to dynamic attribute access too.

    Attribute access is currently possible in two ways:

    * When the attribute name is known at code-writing time, the
      ".NAME" trailer can be used, as in

          x.foo = 42
          y.bar += 100

    * When the attribute name is computed dynamically at run-time, the
      "getattr" and "setattr" builtins must be used:

          x = getattr(y, 'foo_%d' % n)
          setattr(z, 'bar_%s' % s, 99)

      The "getattr" builtin also allows the coder to specify a default
      value to be returned in the event that the object does not have
      an attribute of the given name:

          x = getattr(y, 'foo_%d' % n, 0)

    This PEP describes a new syntax for dynamic attribute access ---
    "x.(expr)" --- with examples given in the Abstract above.  The new
    syntax also allows the provision of a default value in the "get"
    case, as in:

        x = y.('foo_%d' % n, None)

    This 2-argument form of dynamic attribute access is not permitted as
    the target of an (augmented or normal) assignment.  Also, this part
    of the new syntax was not as well received [6] in initial
    discussions on python-ideas, and I agree that it does not fit so
    cleanly.  I'm happy to prepare a revised PEP/patch without the
    2-argument form if the consensus is that this would be preferred.

    Finally, the new syntax can be used with the "del" statement, as in

        del x.(attr_name)

Impact On Existing Code

    The proposed new syntax is not currently valid, so no existing
    well-formed programs have their meaning altered by this proposal.

    Across all "*.py" files in the 2.5 distribution, there are around
    600 uses of "getattr", "setattr" or "delattr".  They break down as
    follows (figures have some room for error because they were
    arrived at by partially-manual inspection):

        c.300 uses of plain "getattr(x, attr_name)", which could be
              replaced with the new syntax;

        c.150 uses of the 3-argument form, i.e., with the default
              value; these could be replaced with the 2-argument form
              of the new syntax (the cases break down into c.125 cases
              where the attribute name is a literal string, and c.25
              where it's only known at run-time);

        c.5   uses of the 2-argument form with a literal string
              attribute name, which I think could be replaced with the
              standard "x.attribute" syntax;

        c.120 uses of setattr, of which 15 use getattr to find the
              new value; all could be replaced with the new syntax,
              the 15 where getattr is also involved would show a
              particular increase in clarity;

        c.5   uses which would have to stay as "getattr" because they
              are calls of a variable named "getattr" whose default
              value is the builtin "getattr";

        c.5   uses of the 2-argument form, inside a try/except block
              which catches AttributeError and uses a default value
              instead; these could use 2-argument form of the new

        c.10  uses of "delattr", which could use the new syntax.

    As examples, the line

        setattr(self, attr, change_root(self.root, getattr(self, attr)))

    from Lib/distutils/command/install.py could be rewritten

        self.(attr) = change_root(self.root, self.(attr))

    and the line

        setattr(self, method_name, getattr(self.metadata, method_name))

    from Lib/distutils/dist.py could be rewritten

        self.(method_name) = self.metadata.(method_name)

Performance Impact

    Initial pystone measurements are inconclusive, but suggest there may
    be a performance penalty of around 1% in the pystones score with the
    patched version.  One suggestion is that this is because the longer
    main loop in ceval.c hurts the cache behaviour, but this has not
    been confirmed.  (Maybe a tool like valgrind [2] could help here?)

    On the other hand, measurements suggest a speed-up of around 40--45%
    for dynamic attribute access.

Discussion To Date

    Initial posting of this PEP in draft form was to python-ideas on
    20070209 [4], and the response was generally positive:

        I've thought of the same syntax. I think you should submit this
        to the PEP editor and argue on Python-dev for its inclusion in
        Python 2.6 -- there's no benefit that I see of waiting until
        3.0.  --- Guido van Rossum [5]

        Wow!  I have to say this is a compelling idea.  The syntax is a
        bit foreign looking, but [...] I feel like I could learn to like
        it anyway. --- Greg Falcon [6]

        I look forward to seeing this in Python 2.6.  --- Josiah
        Carlson, further down the thread [8]

    with Greg Falcon expressing the reservations about the 2-argument
    form already noted above, and Josiah Carlson raising a query about

        My only concern with your propsed change is your draft
        implementation. [...]  Specifically, your changes [...] may
        negatively affect general Python performance.  --- Josiah
        Carlson [7]

    Some initial measurements (see above) suggest the performance
    penalty is small, and Josiah Carlson said of such cost that it
    "isn't really substantial". [8]

Questions To Be Resolved

    * Whether to allow the 2-argument form for default arguments.

    * Whether the performance impact is real; whether it is acceptable;
      whether alternative implementations might improve this aspect.

Alternative Syntax For The New Feature

    Other syntaxes could be used, for example braces are currently
    invalid in a "trailer", so could be used here, giving

        x{'foo_%d' % n} += 1

    My personal preference is for the

        x.('foo_%d' % n) += 1

    syntax though: the presence of the dot shows there is attribute
    access going on; the parentheses have an analogous meaning to the
    mathematical "work this out first" meaning.  This is also the
    syntax used in the language Matlab [1] for dynamic "field" access
    (where "field" is the Matlab term analogous to Python's

    Discussions on python-ideas (see above) made no comment on the brace
    alternative, and the .() notation was well-enough received, so the
    brace alternative should be considered rejected, I think.

Error Cases

    Only strings are permitted as attribute names, so for instance the
    following error is produced:

      >>> x.(99) = 8
        Traceback (most recent call last):
          File "<stdin>", line 1, in <module>
        TypeError: attribute name must be string, not 'int'

    This is handled by the existing PyObject_GetAttr function.

Draft Implementation

    A draft implementation adds a new alternative to the "trailer"
    clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in
    Python.asdl, with accompanying changes to symtable.c, ast.c, and
    compile.c, and three new opcodes (load/store/del) with
    accompanying changes to opcode.h and ceval.c.  The patch consists
    of c.180 additional lines in the core code, and c.100 additional
    lines of tests.  It is available as sourceforge patch #1657573 [3].


    [1] Using Dynamic Field Names :: Data Types (MATLAB Programming)

    [2] Valgrind: "suite of tools for debugging and profiling Linux 

    [3] Sourceforge patch #1657573







    This document has been placed in the public domain.

More information about the Python-Dev mailing list