[Python-Dev] PEP 253: Subtyping Built-in Types
Sun, 22 Jul 2001 18:28:54 +0200
Guido van Rossum wrote:
> > A few people keep asking me for new features on those types, so
> > I guess enabling this for Python 2.2 would be a real advantage for
> > them.
> > I still haven't found out how to solve the construction problem
> > though (the base type is hard coded into various factory functions
> > and methods)... the factory methods could use self.__class__
> > to solve this, but the factory functions would need some different
> > tweaking.
> Using the new "classmethod" feature you can make the factory functions
> class methods.
Hmm, I don't like these class methods, but it would probably
help with the problem...
from mx.DateTime import DateTime
dt1 = DateTime(2001,1,16)
dt2 = DateTime.From("16. Januar 2001")
Still looks silly to me... (I don't like these class methods).
> > > Yes, I've worked out a scheme to make this work, but I don't think
> > > I've written it down anywhere yet. If your tp_new calls tp_alloc, and
> > > your tp_dealloc calls tp_free, then a subtype can override tp_alloc
> > > *and* tp_free and the right thing will happen. A subtype can also
> > > *extend* tp_new and tp_dealloc. (tp_new and tp_dealloc are sort-of
> > > each other's companions, and ditto for tp_alloc and tp_free.)
> > So I will have to implement tp_free as well ?! Currently I have
> > tp_new (which calls tp_alloc), tp_alloc, tp_init for the creation
> > procedure and tp_dealloc (which does not call tp_free) for the
> > finalization.
> Yes, if your tp_new calls tp_alloc, your tp_dealloc should call
> tp_free. Otherwise the user can override tp_alloc to use a different
> heap, and tp_dealloc would mess up.
> > I wonder whether it'd be a good idea to have a tp_del in there
> > as well (the __del__ at C level) which is then called instead
> > of tp_dealloc if set and which must call tp_dealloc if the
> > instance is going to be deleted for good.
> I've been thinking about this. I don't think that's quite the right
> protocol; I don't want to complicate the DECREF macro any more. I
> think that tp_dealloc must call tp_del and then decide whether to
> proceed depending on the refcount.
Have you tried to move the decref action into a separate function
(which is only called in case the refcount reaches 0) ? I think
that this could in fact enhance the overall performance since
the compiler can then decide whether or not to inline the relevant
I wonder what the impact would be...
> > > > 2. In which order are the allocation/deallocation methods
> > > > of subclass and base class called (if at all) and how
> > > > does this depend on whether they are implemented or inherited ?
> > >
> > > Here's the scheme. A subtype's tp_new should call the base type's
> > > tp_new, passing the subtype. The base class will call tp_alloc, which
> > > is the subtype's version. Similar for deallocation: the subtype's
> > > tp_dealloc calls the base type's tp_dealloc which calls tp_free which
> > > is the subtype's version.
> > Like this... ?
> > subtype basetype
> > ----------------------------------------------------
> > Creation
> > tp_new(subtype)
> > -> tp_new(subtype) # calls tp_alloc & tp_init
> > tp_alloc(subtype) <-
> > -> tp_alloc(subtype)
> Typically, the derved type's tp_alloc shouldn't call the base type's
> tp_alloc -- tp_alloc is supposed to allocate memory for the actual
> type, zero it, set the type pointer and reference count, and register
> it with GC. Any other initializations that can't be left to tp_init
> (which is optional) are tp_new's responsibility.
Good, so overriding the tp_alloc/free slots is generally not
a wise thing to do, I guess.
> > tp_init(instance) <-
> > -> tp_init(instance)
> > Finalization
> > (
> > tp_delete(instance)
> > -> tp_delete(instance) # calls tp_dealloc if
> > # the instance should
> > # be deleted
> > )
> > tp_dealloc(instance)
> > -> tp_dealloc(instance) # calls tp_free
> > tp_free(instance) <-
> > -> tp_free(instance)
> Likewise, tp_free needn't call the base tp_free.
> > > > 3. How can I make attributes visible in subclassed types ?
> > > >
> > > > Even though I found out that I need to use the generic APIs
> > > > PyObject_GenericGet|SetAttr() for the tp_get|setattro to
> > > > make methods visible, attributes cannot be accessed (and this
> > > > even though dir(instance) displays them).
> > >
> > > Strange. This should work. Probably something's subtly wrong in your
> > > setup. Compare your code to xxsubtype.c.
> > The xxsubtype doesn't define any attributes and neither do lists
> > or dictionaries so there seems to be no precedent.
> > In mxDateTime under Python 2.1, the tp_gettattr slot takes care of
> > processing attribute lookup. Now to enable the dynamic goodies in
> > Python 2.2, I have to provide the tp_getattro slot (and set it to
> > the generic APIs mentioned above).
> > Since tp_getattro override the tp_getattr slots, I have to rely
> > on the generic APIs calling back to the tp_getattr slots to process
> > the attributes which are not dynamically set by the user or a
> > subclass. However, the new generic lookup APIs do not call the
> > tp_getattr slot at all and thus the attributes which were "defined"
> > by the tp_getattr in Python 2.1 are no longer visible.
> > - How do I have to implement attribute lookup in Python 2.2
> > for TP_BASETYPEs (methods are now magically handled by the tp_methods
> > slot, there doesn't seem to be a corresponding feature for attributes
> > though) ?
> Ah, now I see the question. There's a tp_members slot, similar to the
> tp_methods slot. The tp_members slot is a pointer to a
> NULL-terminated array of the same form that you would pass to
> PyMember_Get(). If your attributes require custom computation,
> there's also a tp_getset slot which points to a NULL-terminated array
> of 'struct getsetlist' items, which specify a name, a getter C
> function, a setter C function, and a context void *. This means you
> have to write a pair of (very simple) functions for each writable
> attribute, or a single function per read-only attribute. (The context
> pointer gives you a chance to share function implementations, but
> I haven't found the need for this yet.)
> Examples of all of these can be found in typeobject.c, look for
> type_getsets and type_members.
Thanks. I'll take a look at the implementation ...
> > - Could the generic APIs perhaps fall back to tp_getattr to make
> > the transition from classic types to base types a little easier ?
> I'd rather not: that would prevent discovery of attributes supported
> by the classic tp_getattr. The beauty of the new scheme is that *all*
> attributes (methods and data) are listed in the type's __dict__.
Uhm, I think you misunderstood me: tp_getattr is not used anymore
once the Python interpreter finds a tp_getattro slot
implementation, so there's nothing to prevent ;-):
PyObject_GetAttr() does not use tp_getattr if tp_getattro is
defined, while PyObject_GetAttrString() prefers tp_getattr over
tp_getattro -- something is not symmertic here !
As a result, dir() finds the __members__ attribute which lists
the attributes (it uses PyObject_GetAttrString(), but
instance.attribute does not work because it uses PyObject_GetAttr().
CEO eGenix.com Software GmbH
Consulting & Company: http://www.egenix.com/
Python Software: http://www.lemburg.com/python/