[Python-Dev] Meta-reflections

[Kevin Jacobs]

On Fri, 22 Feb 2002, Samuele Pedroni wrote:
> I was thinking along the line of the C equiv of this:
[...code snipped...]

[
  An updated version of a comment to SF issue:
  http://sourceforge.net/tracker/?func=detail&atid=105470&aid=520644&group_id=5470
]

Samuele's sltattr.py is an interesting approach, though I am not entirely
sure it is sufficient to address all of the problems with slots.  Here is a
mostly complete list of smaller changes that are somewhat orthogonal to how
we address accesses to __dict__:

  1) Flatten slot lists:  Change obj.__class__.__slots__ to return an
     immutable list of all slot descriptors in the object (including all
     those of base classes).  The motivation for this is similar in spirit
     to storing a flattened __mro__.

     The advantages of this change are:

     a) allows for fast and explicit object reflection that correctly finds
        all dict attributes, all slot attributes.

     b) allows reflection implementations (like vars(object) and pickle) to
        treat dict and slot attrs differently if we choose not to proxy
        __dict__.  This has several advantages, as explained in change #2.
        Also importantly, this way it is not possible to "lose" descriptors
        permanently by deleting them from obj.__class__.__dict__.

  2) Update reflection API even if we do not choose to proxy __dict__: Alter
     vars(object) to return a dictionary of all attributes, including both
     the contents of the non-proxied __dict__ and the valid attributes that
     result from iterating over __slots__ and evaluating the descriptors.
     The details of how this is best implemented depend on how we wish to
     define the behavior of modifying the resulting dictionary.  It could be
     either:

           a) explicitly immutable, which involves creating proxy objects
           b) mutable, which involves copying
           c) undefined, which means implicitly immutable

     Aside from the questions over the nature of the return type, this
     implementation (coupled with #1) has distinct advantages.  Specifically
     the native object.__dict__ has a very natural internal representation
     that pairs attribute names directly with values.  In contrast, a fair
     amount of additional work is needed to extract the slots that store
     values and create a dictionary of their names and values.  Other
     implementations will require a great deal more work since they would
     have to traverse though base classes to collecting slot descriptors.

  3) Flatten slot inheritance:  Update the new-style object inheritance
     mechanism to re-use slots of the same name, rather than creating a new
     slot and hiding the old.  This makes the inheritance semantics of slots
     equivalent to those of normal instance attributes and avoids
     introducing an ad-hoc and obscure method of data hiding.

  4) Update standard library to use new reflection API (and make them robust
     to properies at the same time) if we choose not to proxy __dict__.
     Virtually all of the changes are simple and involve updating these
     constructs:

           a) obj.__dict__
           b) obj.__dict__[blah]
           c) obj.__dict__[blah] = x

     (What these will become depends on other factors, including the context
      and semantics of vars(obj).)

     Here is a fairly complete list of Python 2.2 modules that will need to
     be updated:
       copy, copy_reg, inspect, pickle, pydoc, cPickle, Bastion, codeop,
       dis, doctest, gettext, ihooks, imputil, knee, pdb, profile, rexec,
       rlcompleter, tempfile, unittest, xmllib, xmlrpclib

    5) (NB: potentially controversial and not required) We could alter the
       descriptor protocol to make slots (and properties) more transparent
       when the values they reference do not exist.  Here is an example to
       illustrate this:

           class A(object):
             foo = 1

           class B(A):
             __slots__ = ('foo',)

           b = B()
           print b.foo
           > 1 or AttributeError?

        Currently an AttributeError is raised.  However, it is a fairly easy
        change to make AttributeErrors signal that attribute resolution is
        to continue until either a valid descriptor is evaluated, an
        instance-attribute is found, or until the resolution fails after
        search the meta-type, the type, and the instance dictionary.

I am prepared to submit patches to address each of these issues.  However, I
do want feedback beforehand, so that I do not waste time implementing
something that will never be accepted.

Regards,
-Kevin

--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com