A small step to removing the type/class split
I've run into a problem with ExtensionClass which I believe can only be fixed by modifying Python. I will try to explain. I have an ExtensionClass which defines __cmp__. Depending on the objects being compared, the __cmp__ method may never be called. This is due to code in object.c that looks like this: if (PyInstance_Check(v) || PyInstance_Check(w)) { try to use use __cmp__ method } else { try number coerce and fall back on type name comparison } Extension classes can never pass the PyInstance_Check predicate. I've searched for all occurances of PyInstance_Check in the 2.0 source. In many places that PyInstance_Check occurs a more general "is this an instance-like type" check would seem to be more suitable. Here is my proposal: * Define a new type flag Py_TPFLAGS_INSTANCE. * Create a new predicate Py_IsInstance which checks for this flag. * Set this flag on PyInstance_Type. * Replace most occurances of PyInstance_Check with Py_IsInstance. Extension types (like ExtensionClass) can then define the type flag Py_TPLAGS_INSTANCE and be treated as an instance type by the Python interpreter. This should make it quite a bit easier to make extension types behave like "real" classes. Comments? Neil
Neil Schemenauer wrote:
Sounds fine, except that the "isinstance()" behaviour would have to be defined somewhere / somehow... a mapping of slots to __special_methods__ would be a good start. BTW, mxProxy does some of this mapping already... you may want to have a look. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
"NS" == Neil Schemenauer <nas@arctrix.com> writes:
NS> I've run into a problem with ExtensionClass which I believe NS> can only be fixed by modifying Python. I will try to explain. NS> I have an ExtensionClass which defines __cmp__. Depending on NS> the objects being compared, the __cmp__ method may never be NS> called. This is due to code in object.c that looks like this: | if (PyInstance_Check(v) || PyInstance_Check(w)) { | try to use use __cmp__ method | } | else { | try number coerce and fall back on type name comparison | } I hit a similar wall when I tried (a long time ago) to take a boolean class I'd written in Python and make it a built-in type. The problem was that in Python I could compare a Boolean instance against any other object for truth equivalence, but because of this hack, I couldn't do the same with the built-in type. | * Define a new type flag Py_TPFLAGS_INSTANCE. | * Create a new predicate Py_IsInstance which checks for this | flag. | * Set this flag on PyInstance_Type. | * Replace most occurances of PyInstance_Check with | Py_IsInstance. NS> Extension types (like ExtensionClass) can then define the type NS> flag Py_TPLAGS_INSTANCE and be treated as an instance type by NS> the Python interpreter. This should make it quite a bit NS> easier to make extension types behave like "real" classes. I'm not sure how well this addresses what I ran into. Would PyBooleanObject then have to have it's Py_TPFLAGS_INSTANCE flag set? Does that actually make sense? How does this interact with the rich comparisons idea? Seems like this is a hack that doesn't quite get at the heart of the matter, worthwhile as it may be given the current implementation. -Barry
On Tue, Oct 24, 2000 at 04:06:23PM -0400, Barry A. Warsaw wrote: [on Py_TPLAGS_INSTANCE]
I think it would address your problem but I don't know if the name of the flag makes sense. Code around PyInstance_Check() usually looks like this: if (PyInstance_Check(obj)) { PyObject *foo = PyObject_GetAttrString(obj, "__foo__"); ... } else { ... } There is not a lot of code that assumes (obj->ob_type == PyInstance_Type) after a PyInstance_Check(). That's encouraging.
How does this interact with the rich comparisons idea?
I don't know. I think I am trying to a address a deeper problem here.
Yes, I have that feeling as well. More deep thinking is probably required. :) Neil
Good analysis of a problem -- but I disagree on the solution. Rather than formalizing the exceptions made for instances, we should work on eradicating the differences between classes and types. I believe that the plans for rich comparisons would or could also take care of this. I hope I'll have more time soon to explore this. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, Oct 24, 2000 at 04:50:32PM -0500, Guido van Rossum wrote:
Yes, I see this now. Those PyInstance_Check() calls should really go away. Instead, the type should be responsible for providing different behavior.
I believe that the plans for rich comparisons would or could also take care of this.
I've just briefly looked at the proposal. It consists of two parts. One is splitting __cmp__ into __gt__, __lt__, etc. That is a separate issue to the one we are dicussing. The other part is directly related. In order for types to define their own comparison behavior, it proposes that types have an optional pointer to a rich comparison function in the type structure. I think this is the right idea but it doesn't go far enough. Every operation should be implementable by the type. Code like: if (PyString_Check(obj)) { something } else if (PyInstance_Check(obj)) { something else } ... should disappear unless it is need for performance reasons. I think we are close to achieving this goal. I'll try to come up with a proposal. Neil
Neil Schemenauer <nas@arctrix.com>:
Those PyInstance_Check() calls should really go away. Instead, the type should be responsible for providing different behavior.
Yes, that's the crux of the problem, I think. Another place where this is a nuisance is where it assumes that if x has any tp_as_sequence slots and y has any tp_as_number slots, then x*y is a sequence replication operation. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Oct 25, 2000 at 02:14:52PM +1300, Greg Ewing wrote:
Neil Schemenauer <nas@arctrix.com>:
Those PyInstance_Check() calls should really go away. Instead, the type should be responsible for providing different behavior.
Yes, that's the crux of the problem, I think.
This is screwed in more than one way. The logic of which 'type' operation has precedence isn't even internally consistent. The case of 'x*y' and 'x+y' check the tp_as_number and tp_as_sequence slots in a *different order*, so 'x*y' can end up a number where 'x+y' ends up a sequence. (Or the other way 'round, I forget. I'm also struggling to catch up on my mail since Apachecon had little to no connectivity, so I don't feel like checking :) Brainwashed-into-Apache-2.0-forever-ly y'rs, -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
Guido van Rossum wrote:
(snip)
Rather than formalizing the exceptions made for instances, we should work on eradicating the differences between classes and types.
Yee ha! I couldn't agree more. :)
I believe that the plans for rich comparisons would or could also take care of this.
I assume that rich comparison's have new slots with signatures that don't assume objects of the same type.
I hope I'll have more time soon to explore this.
Me too. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Neil Schemenauer wrote:
(snip) I think you got some good answers later in the thread. I'll point out that a work around is to make your ExtensionClass a numeric type. Numeric types have different comparison semantics that allow you to get around this limitation. I did this a few months back for acquisition wrappers. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Neil Schemenauer wrote:
Sounds fine, except that the "isinstance()" behaviour would have to be defined somewhere / somehow... a mapping of slots to __special_methods__ would be a good start. BTW, mxProxy does some of this mapping already... you may want to have a look. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
"NS" == Neil Schemenauer <nas@arctrix.com> writes:
NS> I've run into a problem with ExtensionClass which I believe NS> can only be fixed by modifying Python. I will try to explain. NS> I have an ExtensionClass which defines __cmp__. Depending on NS> the objects being compared, the __cmp__ method may never be NS> called. This is due to code in object.c that looks like this: | if (PyInstance_Check(v) || PyInstance_Check(w)) { | try to use use __cmp__ method | } | else { | try number coerce and fall back on type name comparison | } I hit a similar wall when I tried (a long time ago) to take a boolean class I'd written in Python and make it a built-in type. The problem was that in Python I could compare a Boolean instance against any other object for truth equivalence, but because of this hack, I couldn't do the same with the built-in type. | * Define a new type flag Py_TPFLAGS_INSTANCE. | * Create a new predicate Py_IsInstance which checks for this | flag. | * Set this flag on PyInstance_Type. | * Replace most occurances of PyInstance_Check with | Py_IsInstance. NS> Extension types (like ExtensionClass) can then define the type NS> flag Py_TPLAGS_INSTANCE and be treated as an instance type by NS> the Python interpreter. This should make it quite a bit NS> easier to make extension types behave like "real" classes. I'm not sure how well this addresses what I ran into. Would PyBooleanObject then have to have it's Py_TPFLAGS_INSTANCE flag set? Does that actually make sense? How does this interact with the rich comparisons idea? Seems like this is a hack that doesn't quite get at the heart of the matter, worthwhile as it may be given the current implementation. -Barry
On Tue, Oct 24, 2000 at 04:06:23PM -0400, Barry A. Warsaw wrote: [on Py_TPLAGS_INSTANCE]
I think it would address your problem but I don't know if the name of the flag makes sense. Code around PyInstance_Check() usually looks like this: if (PyInstance_Check(obj)) { PyObject *foo = PyObject_GetAttrString(obj, "__foo__"); ... } else { ... } There is not a lot of code that assumes (obj->ob_type == PyInstance_Type) after a PyInstance_Check(). That's encouraging.
How does this interact with the rich comparisons idea?
I don't know. I think I am trying to a address a deeper problem here.
Yes, I have that feeling as well. More deep thinking is probably required. :) Neil
Good analysis of a problem -- but I disagree on the solution. Rather than formalizing the exceptions made for instances, we should work on eradicating the differences between classes and types. I believe that the plans for rich comparisons would or could also take care of this. I hope I'll have more time soon to explore this. --Guido van Rossum (home page: http://www.python.org/~guido/)
On Tue, Oct 24, 2000 at 04:50:32PM -0500, Guido van Rossum wrote:
Yes, I see this now. Those PyInstance_Check() calls should really go away. Instead, the type should be responsible for providing different behavior.
I believe that the plans for rich comparisons would or could also take care of this.
I've just briefly looked at the proposal. It consists of two parts. One is splitting __cmp__ into __gt__, __lt__, etc. That is a separate issue to the one we are dicussing. The other part is directly related. In order for types to define their own comparison behavior, it proposes that types have an optional pointer to a rich comparison function in the type structure. I think this is the right idea but it doesn't go far enough. Every operation should be implementable by the type. Code like: if (PyString_Check(obj)) { something } else if (PyInstance_Check(obj)) { something else } ... should disappear unless it is need for performance reasons. I think we are close to achieving this goal. I'll try to come up with a proposal. Neil
Neil Schemenauer <nas@arctrix.com>:
Those PyInstance_Check() calls should really go away. Instead, the type should be responsible for providing different behavior.
Yes, that's the crux of the problem, I think. Another place where this is a nuisance is where it assumes that if x has any tp_as_sequence slots and y has any tp_as_number slots, then x*y is a sequence replication operation. Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
On Wed, Oct 25, 2000 at 02:14:52PM +1300, Greg Ewing wrote:
Neil Schemenauer <nas@arctrix.com>:
Those PyInstance_Check() calls should really go away. Instead, the type should be responsible for providing different behavior.
Yes, that's the crux of the problem, I think.
This is screwed in more than one way. The logic of which 'type' operation has precedence isn't even internally consistent. The case of 'x*y' and 'x+y' check the tp_as_number and tp_as_sequence slots in a *different order*, so 'x*y' can end up a number where 'x+y' ends up a sequence. (Or the other way 'round, I forget. I'm also struggling to catch up on my mail since Apachecon had little to no connectivity, so I don't feel like checking :) Brainwashed-into-Apache-2.0-forever-ly y'rs, -- Thomas Wouters <thomas@xs4all.net> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
Guido van Rossum wrote:
(snip)
Rather than formalizing the exceptions made for instances, we should work on eradicating the differences between classes and types.
Yee ha! I couldn't agree more. :)
I believe that the plans for rich comparisons would or could also take care of this.
I assume that rich comparison's have new slots with signatures that don't assume objects of the same type.
I hope I'll have more time soon to explore this.
Me too. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
Neil Schemenauer wrote:
(snip) I think you got some good answers later in the thread. I'll point out that a work around is to make your ExtensionClass a numeric type. Numeric types have different comparison semantics that allow you to get around this limitation. I did this a few months back for acquisition wrappers. Jim -- Jim Fulton mailto:jim@digicool.com Python Powered! Technical Director (888) 344-4332 http://www.python.org Digital Creations http://www.digicool.com http://www.zope.org Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email address may not be added to any commercial mail list with out my permission. Violation of my privacy with advertising or SPAM will result in a suit for a MINIMUM of $500 damages/incident, $1500 for repeats.
participants (7)
-
barry＠wooz.org -
Greg Ewing -
Guido van Rossum -
Jim Fulton -
M.-A. Lemburg -
Neil Schemenauer -
Thomas Wouters