[Python-Dev] Meta-reflections

[Kevin Jacobs]

On Wed, 20 Feb 2002, Moore, Paul wrote:
> Oops. Sorry, I got that completely wrong. OK, I side with silent re-use.
> That's what attributes do. [Meta-meta comment: the rule should be to work
> "just like" attributes.] So why did you feel the need to separate this point
> from your point (3)? It gives the impression that this point differs from
> attributes, in contrast to the things mentioned in (3).

Having a flat namespace (i.e., no hidden slots), and having all 'reachable'
slots in a single list are really two separate issues.  Right now, we have
this situation:

   class Foo(object):
     __slots__ = ('a','b')

   class Bar(Foo):
     __slots__ = ('c','d')

   bar=Bar()
   print bar.__slots__
   > ('c', 'd')
   print bar.__class__.__base__.__slots__
   > ('a', 'b')

   I content that bar.__slots__ should be:
   > ('a', 'b', 'c', 'd')

I think somewhere along the line I may have mixed up which 'flatness' I was
talking about.

> The official, and supported, way is to use dir().

I agree, but that "official" support has clear limitations.  Right now,
there are several examples in the Python standard library where we use
obj.__dict__.keys() --  most significantly in pickle and cPickle.  There is
also the vars(obj), which may be the better reflection method, though it
currently doesn't know about slots.

> (Whether this advice was included into the formal documentation, I
> couldn't confirm, but it was written down - arguing "if it's not in the
> documentation, it's not official", is a little naive, given the new and
> relatively experimental status of the whole area...)

Naive, maybe, but saying "undocumented" is equivalent to "unsupported
implementation detail" saves us from having to maintain backward
compatibility and following the official Python deprecation process.

> Do you have any reason why you would need to get a list of only slots, or
> only dict-based attributes?

Yes.  Dict-based attributes always have values, while slot-based attributes
can be unset and raise AttributeErrors when trying to access them.  e.g.,
here is how I would handle pickling (excerpt from pickle.py):

        try:
            getstate = object.__getstate__
        except AttributeError:
            stuff = object.__dict__

            # added to support slots
            if hasattr(object.__slots__):
              for slot in object.__slots__:
                if hasattr(object, slot):
                  stuff[slot] = getattr(object, slot)

        else:
            stuff = getstate()
            _keep_alive(stuff, memo)
        save(stuff)
        write(BUILD)

> Given that I'm arguing that the two should work
> exactly the same (apart from storage and efficiency details), it seems
> unreasonable to want to make the distinction (unless you're doing something
> incestuous and low-level, when you're on your own...)

I'm not suggesting anything more incestuous and low-level than what is
already done in many, many, many places.  A larger, more-encompassing
proposal is definitely welcome.

> > EXACTLY! I want to use slots (or slotattrs, or whatever
> > you call them) to be solely an allocation declaration.
> > For small objects that you expect to create many, many
> > instance of, they make a huge difference.
>
> In which case, you are giving the impression of wanting large changes, when
> you really want things to stay as they are (modulo some relatively
> non-controversial (IMHO) bugs). If you read the descrintro document on
> slots, you will see that it presents an identical viewpoint. OK, there are
> some technical restrictions, which will always apply because of the nature
> of the optimisation, but the intention is clear. And it matches yours...

Well, I've not found resounding agreement on the first two of my three basic
issues/bugs I've raised so far:

  1) Flat slot namespaces: Objects should not hiding slots when inherited by
     classes implementing the same slot name.

  2) Flat slot descriptions:  object.__slots__ being an immutable flat tuple
     of all slot names (including inherited ones), as opposed to being a
     potentially mutable sequence of only the slots defined by the most
     derived class.

  3) First class status for slot reflection: making slots picklable by
     default, returned by vars(object), and made part of other relevant
     reflection APIs and standard implementations.

The good news is that once Guido and others have spoken, I can have patches
that accomplish all of this fairly quickly.  I just don't want to do a lot
of unnecessary work if it won't be accepted.

Thanks,
-Kevin


--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (216) 986-0714              WWW:    http://www.theopalgroup.com