These are some half-baked ideas about getting classes and types to look more similar. I would like to know whether they are workable or not and so I present them to the people best equipped to tell me. Many extension types have a __getattr__ that looks like this: static PyObject * Xxo_getattr(XxoObject *self, char *name) { // try to do some work with known attribute names, else: return Py_FindMethod(Xxo_methods, (PyObject *)self, name); } Py_FindMethod can (despite its name) return any Python object, including ordinary (non-function) attributes. It also has complete access to the object's state and type through the self parameter. Here's what we do today for __doc__: if (strcmp(name, "__doc__") == 0) { char *doc = self->ob_type->tp_doc; if (doc != NULL) return PyString_FromString(doc); } Why can't we do this for all magic methods? * __class__ would return for the type object * __add__,__len__, __call__, ... would return a method wrapper around the appropriate slot, * __init__ might map to a no-op I think that Py_FindMethod could even implement inheritance between types if we wanted. We already do this magic for __methods__ and __doc__. Why not for all of the magic methods? Many other types implement no getattr at all (the slot is NULL). In that case, I think that we have carte blanche to define their getattr behavior as instance-like as possible. Finally there are the types with getattrs that do not dispatch to Py_FindMethod. we can just change those over manually. Extension authors will do the same when they realize that their types are not inheriting the features that the other types are. Benefits: * objects based on extension types would "look more like" classes to Python programmers so there is less confusion about how they are different * users could stop using the type() function to get concrete types and instead use __class__. After a version or two, type() could be formally deprecated in favor of isinstance and __class__. * we will have started some momentum towards type/class unification which we could continue on into __setattr__ and subclassing. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook
[Paul Prescod]
These are some half-baked ideas about getting classes and types to look more similar. I would like to know whether they are workable or not and so I present them to the people best equipped to tell me.
[expand Py_FindMethod's actions]
* __class__ would return for the type object * __add__,__len__, __call__, ... would return a method wrapper around the appropriate slot, * __init__ might map to a no-op
I think that Py_FindMethod could even implement inheritance between types if we wanted.
We already do this magic for __methods__ and __doc__. Why not for all of the magic methods?
Those are introspective; typically read in the interactive interpreter. I can't do anything with them except read them. If you wrap, eg, __len__, what can I do with it except call it? I can already do that with len().
Benefits:
* objects based on extension types would "look more like" classes to Python programmers so there is less confusion about how they are different
I think it would probably enhance confusion to have the "look more like" without "being more like".
* users could stop using the type() function to get concrete types and instead use __class__. After a version or two, type() could be formally deprecated in favor of isinstance and __class__.
__class__ is a callable object. It has a __name__. From the Python side, a type isn't much more than an address. Until Python's object model is redone, there are certain objects for which type(o) and o.__class__ return quite different things.
* we will have started some momentum towards type/class unification which we could continue on into __setattr__ and subclassing.
The major lesson I draw from ExtensionClass and friends is that achieving this behavior in today's Python is horrendously complex and fragile. Until we can do it right, I'd rather keep it simple (and keep the warts on the surface). - Gordon
Gordon McMillan wrote:
...
Those are introspective; typically read in the interactive interpreter. I can't do anything with them except read them.
If you wrap, eg, __len__, what can I do with it except call it?
You can store away a reference to it and then call it later. I
can already do that with len().
Benefits:
* objects based on extension types would "look more like" classes to Python programmers so there is less confusion about how they are different
I think it would probably enhance confusion to have the "look more like" without "being more like".
Looking more like is the same as being more like. In other words, there are a finite list of differences in behavior between types and classes and I think we should chip away at them one by one with each release of Python. Do you think that there is a particular difference (perhaps relating to subclassing) that is the "real" difference and the rest are just cosmetic?
* users could stop using the type() function to get concrete types and instead use __class__. After a version or two, type() could be formally deprecated in favor of isinstance and __class__.
__class__ is a callable object. It has a __name__. From the Python side, a type isn't much more than an address.
Type objects also have names. They are not (yet) callable but I cannot think of a circumstance in which that would matter. It would require code like this: cls = getattr(foo, "__class__", None) if cls: cls(...) I don't know where the arglist for cls would come from. In general, I can't imagine what the goal of this code would be. I can see code like this in a "closed world" situation where I know all of the classes involved, but I can't imagine a case where this kind of code will work with any old class. Anyhow, I think that type objects should be callable just like classes...but I'm trying to pick off low-hanging fruit first. I think that the less "superficial" differences there are between types and classes, the easier it becomes to tackle the deep differences because more code out there will be naturally polymorphic instead of using: if type(obj) is InstanceType: do_onething() else: do_anotherthing() That is an evil pattern if we are going to merge types and classes.
Until Python's object model is redone, there are certain objects for which type(o) and o.__class__ return quite different things.
I am very nervous about waiting for a big-bang re-model of the object model.
... The major lesson I draw from ExtensionClass and friends is that achieving this behavior in today's Python is horrendously complex and fragile. Until we can do it right, I'd rather keep it simple (and keep the warts on the surface).
I'm trying to find an incremental way forward because nobody seems to have time or energy for a big bang. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook
[Gordon]
I think it would probably enhance confusion to have the "look more like" without "being more like". [Paul] Looking more like is the same as being more like. In other words, there are a finite list of differences in behavior between types and classes and I think we should chip away at them one by one with each release of Python.
There's only one difference that matters: subclassing. I don't think there's an incremental path to that that leaves Python "easily extended". [Gordon]
__class__ is a callable object. It has a __name__. From the Python side, a type isn't much more than an address.
Type objects also have names.
But not a __name__.
They are not (yet) callable but I cannot think of a circumstance in which that would matter.
Take a look at copy.py.
Anyhow, I think that type objects should be callable just like classes...but I'm trying to pick off low-hanging fruit first. I think that the less "superficial" differences there are between types and classes, the easier it becomes to tackle the deep differences because more code out there will be naturally polymorphic instead of using:
if type(obj) is InstanceType: do_onething() else: do_anotherthing()
That is an evil pattern if we are going to merge types and classes.
And it would likely become: if callable(obj.__class__): .... Explicit is better than implicit for warts, too. - Gordon
Gordon McMillan wrote:
..
There's only one difference that matters: subclassing. I don't think there's an incremental path to that that leaves Python "easily extended".
All of the differences matter! Inconsistency is a problem in and of itself.
But not a __name__.
They really do have __name__s. Try it. type("").__name__
They are not (yet) callable but I cannot think of a circumstance in which that would matter.
Take a look at copy.py.
copy.py only expects the type object to be callable WHEN there is a getinitargs method. Types won't have this method so it won't use the class callably. Plus, the whole section only gets run for objects of type InstanceType. The important point is that it is not useful to know that __class__ is callable without knowing the arguments it takes. __class__ is much more often used as a unique identifier for pointer equality and/or for the __name__. In looking through the standard library, I can only see places that the code would improve if __class__ were available for extension objects. -- Take a recipe. Leave a recipe. Python Cookbook! http://www.activestate.com/pythoncookbook
These are some half-baked ideas about getting classes and types to look more similar.
Somewhat different thoughts: Which limitations are there in python as a consequence of the type/class split? In python code itself, it is not too bad. Instead of deriving from builtin types, you can always delegate to them. In c-code, the situation is worse, on the other hand, ExtensionClass comes to rescue. Write the base class in C, subclass in python. The worst limitation is in the most useful builtin object: the dictionary. One can use or derive from UserDict, but one cannot pass UserDict instances or other homegrown dict lookalikes to exec, you cannot use them as class or instance dictionaries. If this limitation would be removed, you could implement your own rules in namespaces: readonly, case insentitive, whatever. One could even implement a mapping object in a C extension, and use it in the aforementioned ways. IMO this limitation would be easy to remove: The current PyDict_* API could be implemented in terms of the PyMapping_ and PyObject_ protocol, using different code depending on the outcome of an additional PyDict_Check() test. The performance hit would be rather small. Thomas
participants (3)
-
Gordon McMillan -
Paul Prescod -
Thomas Heller