I've run into another issue in my attempt to transition over to numarray: it's less friendly to user defined types than Numeric. I think this is mainly accidental friednliness on Numeric's part, but it's handy nonetheless. The attached file illustrates the issue. Given some object, in this case *zero*, that a numarray array does not now how to handle, it ends up raising an exception instead of giving the object a chance to try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) + zero`` fails, since in the first case Zero.__add__ is called first, while in the second NumArray.__add__ is called first. This should probably be fixed, but it's not clear what the best way is. My first thought was to always give the other object a chance to use __rop__ first. This is simple and easy to explain, but fails miserably in the common case of multiplying a list and an array. Just to be concrete, __mul__ would be replaced by:: def __mul__(self, operand): try: return operand.__rmul__(self) except: return ufunc.multiply(self, operand) Next thought is to catch exceptions when they occur in numarray and then give the other operand a chance:: def __mul__(self, operand): try: return ufunc.multiply(self, operand) except: return operand.__rmul__(self) This appears like it would fix my particular problem, but still is not ideal. Since numarray is the base of libraries that it will know nothing about, it should defer to classes it doesn't know about whenever possible. Otherewise it's not possible (or maybe just hard) to create new classes that try to be clever, but still interact with numarray in a reasonable way. In my case I'm thinking of proxy objects that don't do some computations till they are actually required. So my current thinking is that __mul__ and friends would be best implemented as:: def __mul__(self, operand): if not isinstance(operand, knownTypes): try: return ufunc.multiply(self, operand) except: pass return operand.__rmul__(self) Where knownTypes is something like (int, long, float, complex, tuple, list). Anyway, that's my excessively long two cents on this. -tim import numarray as na import Numeric as np class Zero: def __add__(self, other): return other __radd__ = __add__ zero = Zero() #~ print zero + np.arange(5) #~ print np.arange(5) + zero #~ print zero + na.arange(5) #~ print na.arange(5) + zero a = na.arange(5) import copy copy.deepcopy(a)
On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
Sorry for the delay; I was pretty much shut down by hurricane Isabelle from the end of last week. I hit and bypassed this issue trying to port MA.
I've run into another issue in my attempt to transition over to numarray: it's less friendly to user defined types than Numeric. I think this is mainly accidental friednliness on Numeric's part, but it's handy nonetheless. The attached file illustrates the issue. Given some object, in this case *zero*, that a numarray array does not now how to handle, it ends up raising an exception instead of giving the object a chance to try. Thus ``zero + numarray.arange(5)`` works while ``numarray.arange(5) + zero`` fails, since in the first case Zero.__add__ is called first, while in the second NumArray.__add__ is called first.
This should probably be fixed, but it's not clear what the best way is. My first thought was to always give the other object a chance to use __rop__ first. This is simple and easy to explain, but fails miserably in the common case of multiplying a list and an array. Just to be concrete, __mul__ would be replaced by::
def __mul__(self, operand): try: return operand.__rmul__(self) except: return ufunc.multiply(self, operand)
Next thought is to catch exceptions when they occur in numarray and then give the other operand a chance::
def __mul__(self, operand): try: return ufunc.multiply(self, operand) except: return operand.__rmul__(self)
This appears like it would fix my particular problem, but still is not ideal. Since numarray is the base of libraries that it will know nothing about, it should defer to classes it doesn't know about whenever possible.
That does sound like the right heuristic and is echoed (somewhat) in the core language.
Otherewise it's not possible (or maybe just hard) to create new classes that try to be clever, but still interact with numarray in a reasonable way.
Looking in Python's Objects/abstract.c, the function binop1 shows one way to squirm around this issue: subclass from NumArray. It would be interesting to know how Numeric does it.
In my case I'm thinking of proxy objects that don't do some computations till they are actually required. So my current thinking is that __mul__ and friends would be best implemented as::
def __mul__(self, operand): if not isinstance(operand, knownTypes):
If the "not" belongs there, I'm lost.
try: return ufunc.multiply(self, operand) except: pass return operand.__rmul__(self)
Where knownTypes is something like (int, long, float, complex, tuple, list).
Anyway, that's my excessively long two cents on this.
I agree with your heuristic, but I am leery of putting an unqualified try/except in front of the numarray ufunc code. On the other hand, qualifying it seems too complicated. What about adding a tuple of types to be deferred to? def __mul__(self, operand): if isinstance(operand, _numarray_deferred_types): operand.__rmul__(self) else: self.__mul__(operand) Then other libraries could register classes with something like: numarray.defer_to(my_class) which is slightly painful, but avoids masking the ufunc exceptions and only needs to be done once for each library base class. How does that sound? Todd
-tim
----
import numarray as na import Numeric as np
class Zero: def __add__(self, other): return other __radd__ = __add__ zero = Zero()
#~ print zero + np.arange(5) #~ print np.arange(5) + zero #~ print zero + na.arange(5) #~ print na.arange(5) + zero
a = na.arange(5)
import copy copy.deepcopy(a) -- Todd Miller jmiller@stsci.edu STSCI / ESS / SSB
Hi Todd, Glad to have you back. I hope you suffered no permanant damage from the hurricane. Or the worm for that matter. Todd Miller wrote:
On Thu, 2003-09-18 at 19:18, Tim Hochberg wrote:
Sorry for the delay; I was pretty much shut down by hurricane Isabelle from the end of last week.
I hit and bypassed this issue trying to port MA.
[SNIP]
Looking in Python's Objects/abstract.c, the function binop1 shows one way to squirm around this issue: subclass from NumArray. It would be interesting to know how Numeric does it.
I think that since Numeric arrays are C classes, they always defer to any Python class although I'm not sure of the details. In any event, I'm pretty sure it's an accident.
In my case I'm thinking of proxy objects that don't do some computations till they are actually required. So my current thinking is that __mul__ and friends would be best implemented as::
def __mul__(self, operand): if not isinstance(operand, knownTypes):
If the "not" belongs there, I'm lost.
It does belong there. KnownTypes (perhaps _numarray_nondeferred_types) would be a better term) are types that numarray knows that it should handle itself. Let me reverse the logic -- maybe it'll be clearer. def __mul__(self, operand): if isinstance(operand, _numarray_nondeferred_types): return operand.__rmul__(self) else: try: return ufunc.multiply(self, operand) except: return operand.__rmul__(self)
I agree with your heuristic, but I am leery of putting an unqualified try/except in front of the numarray ufunc code. On the other hand, qualifying it seems too complicated.
I agree on all counts.
What about adding a tuple of types to be deferred to?
def __mul__(self, operand): if isinstance(operand, _numarray_deferred_types): operand.__rmul__(self) else: self.__mul__(operand)
Then other libraries could register classes with something like:
numarray.defer_to(my_class)
which is slightly painful, but avoids masking the ufunc exceptions and only needs to be done once for each library base class. How does that sound?
I agree that using try/except is probably not worth the danger. However, I think I'd prefer the inverse of what you propose. That is: def __mul__(self, operand): if isinstance(operand, _numarray_nondeferred_types): self.__mul__(operand) else: operand.__rmul__(self) and of course a dont_defer_to registration function. Hopefully with a better name. The choice comes down to whether we defer by default or handle by default. I'm marginally on the side of deferring, but don't feel too strongly either way. Either one would be a big improvement. Perhaps someone else out there has some profound thoughts. -tim
Todd
-tim
----
import numarray as na import Numeric as np
class Zero: def __add__(self, other): return other __radd__ = __add__ zero = Zero()
#~ print zero + np.arange(5) #~ print np.arange(5) + zero #~ print zero + na.arange(5) #~ print na.arange(5) + zero
a = na.arange(5)
import copy copy.deepcopy(a)
Tim Hochberg wrote:
I agree that using try/except is probably not worth the danger. However, I think I'd prefer the inverse of what you propose. That is:
def __mul__(self, operand): if isinstance(operand, _numarray_nondeferred_types): self.__mul__(operand) else: operand.__rmul__(self)
and of course a dont_defer_to registration function. Hopefully with a better name. The choice comes down to whether we defer by default or handle by default. I'm marginally on the side of deferring, but don't feel too strongly either way. Either one would be a big improvement.
Perhaps someone else out there has some profound thoughts.
Well, certainly not profound, but here's my proverbial $0.02. I tend to really dislike separate registration functions, since they reek too much of 'programming by side-effect'. Why not put the necessary information into a keyword argument to the constructor? Once a sensible default is chosen, then it can be overridden with 'defers=True' (if the default was False) at __init__ time, for example. Since this is a class-wide issue, perhaps it may make more sense to handle it via a meta-class mechanism. But I'm not an expert on metaclasses, so I'll shut up there :) I know that the above is rather vague, partly because I haven't followed the whole thread in detail. But I hope that my intent, as a design idea, is clear (and perhaps even useful :) While the actual implementation details (constructor keyword, metaclass, class-wide global like self.__defers, etc.) will have to be settled by the experts, I hope that an approach which doesn't rely on separate registration functions can be achieved. Best regards, Fernando.
Fernando Perez wrote:
Tim Hochberg wrote:
I agree that using try/except is probably not worth the danger. However, I think I'd prefer the inverse of what you propose. That is:
def __mul__(self, operand): if isinstance(operand, _numarray_nondeferred_types): self.__mul__(operand) else: operand.__rmul__(self)
and of course a dont_defer_to registration function. Hopefully with a better name. The choice comes down to whether we defer by default or handle by default. I'm marginally on the side of deferring, but don't feel too strongly either way. Either one would be a big improvement.
Perhaps someone else out there has some profound thoughts.
Well, certainly not profound, but here's my proverbial $0.02. I tend to really dislike separate registration functions, since they reek too much of 'programming by side-effect'. Why not put the necessary information into a keyword argument to the constructor? Once a sensible default is chosen, then it can be overridden with 'defers=True' (if the default was False) at __init__ time, for example. Since this is a class-wide issue, perhaps it may make more sense to handle it via a meta-class mechanism. But I'm not an expert on metaclasses, so I'll shut up there :)
I actually have no idea how you plan to make keyword arguments work here, perhaps you could explain that in more detail. Metaclasses are overkill, but a mixin, marker class could be used. That is, when designing a class for use with numarray, one would derive a class from a marker class in numarray:: class MyArrayLikeClass(numarray.DeferToMe): .... Hmmm. That's not too bad. Todd, what do you think about using this logic:: def __mul__(self, operand): if isinstance(operand, DeferToMe): operand.__rmul__(self) else: self.__mul__(operand) The only case where I see a potential problem is an old-style C-extenstion that can't be subclassed. I think that might be a case of YAGNI though. Avoiding registration is appealing. -tim
Tim Hochberg wrote:
I actually have no idea how you plan to make keyword arguments work here, perhaps you could explain that in more detail. Metaclasses are overkill, but a mixin, marker class could be used. That is, when designing a class for use with numarray, one would derive a class from a marker class in numarray::
Well, the joys of being vague :) As I said, I haven't followed in enough detail to be too specific, so perhaps my idea plain doesn't work. But what I had in mind was something along the lines of: class ArrayLike(numarray): def __init__(self,...): ... numarray.__init__(defers=1) This is ultimately equivalent to a registration process, but since it is done in the constructor, it feels less like a separate dangling side-effect. On the other hand, it makes your new constructor more expensive, and it feels ugly to be doing at the instance level things which truly belong at the class level. Hence my comment about this perhaps being better done via metaclasses. As I said, I knew I was being vague. But hopefully these comments at least suggest a way out of a separate registration step. Regards, Fernando.
On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:
I actually have no idea how you plan to make keyword arguments work here, perhaps you could explain that in more detail. Metaclasses are overkill, but a mixin, marker class could be used. That is, when designing a class for use with numarray, one would derive a class from a marker class in numarray::
class MyArrayLikeClass(numarray.DeferToMe): ....
Hmmm. That's not too bad. Todd, what do you think about using this logic::
def __mul__(self, operand): if isinstance(operand, DeferToMe): operand.__rmul__(self) else: self.__mul__(operand)
I like the core idea a lot. My only doubt is whether forcing the use of inheritance is appropriate / a good thing. We might also consider spelling it like: class MyArrayLikeClass: _numarray_defer_to_me = True class NumArray: def __mul__(self, operand): if hasattr(operand, "_numarray_defer_to_me"): return operand.__rmul__(self) else: return ufunc.multiply(self, operand)
The only case where I see a potential problem is an old-style C-extenstion that can't be subclassed. I think that might be a case of YAGNI though.
Sounds YAGNI to me.
Avoiding registration is appealing.
Good to have this seconded.
-tim
All in all, great ideas! Todd -- Todd Miller <jmiller@stsci.edu>
On Mon, 2003-09-22 at 19:41, Fernando Perez wrote:
Tim Hochberg wrote:
I actually have no idea how you plan to make keyword arguments work here, perhaps you could explain that in more detail. Metaclasses are overkill, but a mixin, marker class could be used. That is, when designing a class for use with numarray, one would derive a class from a marker class in numarray::
Well, the joys of being vague :) As I said, I haven't followed in enough detail to be too specific, so perhaps my idea plain doesn't work. But what I had in mind was something along the lines of:
class ArrayLike(numarray): def __init__(self,...): ... numarray.__init__(defers=1)
I think this behavior is already granted by Python at the abstract object level; that is, if you subclass, the r-method of the subclass is given preference over the l-method of the superclass. That's my impression anyway. Thus, I think we already get "deference" for free with subclasses, but that is too weak in general because often (like MA for instance) people won't want to subclass from NumArray.
This is ultimately equivalent to a registration process, but since it is done in the constructor, it feels less like a separate dangling side-effect. On the other hand, it makes your new constructor more expensive, and it feels ugly to be doing at the instance level things which truly belong at the class level. Hence my comment about this perhaps being better done via metaclasses.
As I said, I knew I was being vague. But hopefully these comments at least suggest a way out of a separate registration step.
They certainly appear to have done so. Thanks for commenting. Todd -- Todd Miller <jmiller@stsci.edu>
Hi Todd, There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate): 1. numarray.defer_to(my_class) 2. class ArrayLike(numarray.DeferTo): # ... 3. class ArrayLike: _numarray_defer_to = True # ... I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can. The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though. The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3:: import A class Klass2(a.Klass, numarray.DeferTo): #2 #... class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ... A.Klass._numarray_defer_to = True #3 the evil way. Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution. regards, -tim Todd Miller wrote:
On Mon, 2003-09-22 at 19:27, Tim Hochberg wrote:
I actually have no idea how you plan to make keyword arguments work here, perhaps you could explain that in more detail. Metaclasses are overkill, but a mixin, marker class could be used. That is, when designing a class for use with numarray, one would derive a class from a marker class in numarray::
class MyArrayLikeClass(numarray.DeferToMe): ....
Hmmm. That's not too bad. Todd, what do you think about using this logic::
def __mul__(self, operand): if isinstance(operand, DeferToMe): operand.__rmul__(self) else: self.__mul__(operand)
I like the core idea a lot. My only doubt is whether forcing the use of inheritance is appropriate / a good thing. We might also consider spelling it like:
class MyArrayLikeClass: _numarray_defer_to_me = True
class NumArray: def __mul__(self, operand): if hasattr(operand, "_numarray_defer_to_me"): return operand.__rmul__(self) else: return ufunc.multiply(self, operand)
The only case where I see a potential problem is an old-style C-extenstion that can't be subclassed. I think that might be a case of YAGNI though.
Sounds YAGNI to me.
Avoiding registration is appealing.
Good to have this seconded.
-tim
All in all, great ideas!
Todd
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
regards,
-tim
Thanks again, Todd
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray) For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy. -tim [not that this matters much....]
regards,
-tim
Thanks again, Todd
On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
For something as esoteric as this, I think wordy is probably best. Barring a Perrenial edict, DeferredToByNumArray it is. I'm psyched. The MA port should inter-operate with numarray... today. Thanks for all the lucid, constructive suggestions, Todd -- Todd Miller jmiller@stsci.edu STSCI / ESS / SSB
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay! Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay! Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer: 1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order. 2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression. I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later. Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
-tim
[not that this matters much....]
regards,
-tim
Thanks again, Todd
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller@stsci.edu STSCI / ESS / SSB
Todd Miller wrote:
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay!
Excellent.
Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay!
Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer:
Not so excellent.
1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order.
I think that this is YAGNI right now, but may well be an issue is "proper" subclassing get's implemented. So we may as well think about it a bit. I call it YAGNI now because these subclasses either use the default __op__ methods, which presently have the same behaviour regardless of the subclasses (they always return NumArrays) or they define there own __op__ methods in which case they can worry about it ;> If NumArray starts to become more friendly to subclassing, this will probably change though.
2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression.
I'd make the same comment for this case as for the above case except that all else being equal, subclasses should prevail over superclasses.
I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later.
Here's some quick thoughts. In order for this to work out sensibly you want the order between classes to be transistive and you probably also always want to be able to insert a class into the order between any two classes This is equivalent to deciding the op priority based on a comparison based on some real number associated with each class. This leads to something like: class NumArrayFamily: op_priority = 0.0 # This makes it compatible with the current proposal , but isn't strictly necessary. class DeferredToByNumarray(NumArrayFamily): op_priority = 1000.0 class NumArrary(..., NumArrayFamily): # ... def __add__(self, object): if isinstance(object, NumArrayFamily): if ((object.op_priority > self.object.op_priority) or (object.op_priority == self.object.op_priority) and issubclass(object, self.__class__)): return other.__radd__(self) return ufunc.add(self, other) class MySubclassOfNumarray(NumArray): op_priority = 10.0 # deffered to by NumArray, but defers to DeferredToByNumarray classes. Of course the user still has to be careful to make sure the priority order makes sense and you could have big problems when combining packages with disprate ideas about priority levels, but I'm not sure how you can get away from that. -tim
Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
-tim
[not that this matters much....]
regards,
-tim
Thanks again, Todd
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
Todd,
This scheme seems rather complex. The purpose isn't clear to me.
What is the problem which this arrangement is intended to solve?
Bottom line, currently: numarray + another_class --> exception It would be nice if: numarray + another_class --> another_class.__radd__(numarray) Also, look backward at Tim's original post on this subject.
Could I suggest that some scheme along the lines of the Python PEP's be used to set out the Why's and Wherefore's?
Sounds good for next time.
It might also be useful to copy this discussion to comp.lang.python as that newsgroup has people experienced with classes.
OK, I'll stop being a chicken. Everyone, get ready for the sound of a pin dropping on c.l.py.
The problem appears to be Python related rather than a problem of numerical analysis.
Yes indeed.
Colin W.
Todd Miller wrote:
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay!
Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay!
Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer:
1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order.
2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression.
I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later.
Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
-tim
[not that this matters much....]
regards,
-tim
Thanks again, Todd
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- Todd Miller jmiller@stsci.edu STSCI / ESS / SSB
Todd, This scheme seems rather complex. The purpose isn't clear to me. What is the problem which this arrangement is intended to solve? Could I suggest that some scheme along the lines of the Python PEP's be used to set out the Why's and Wherefore's? It might also be useful to copy this discussion to comp.lang.python as that newsgroup has people experienced with classes. The problem appears to be Python related rather than a problem of numerical analysis. Colin W. Todd Miller wrote:
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay!
Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay!
Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer:
1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order.
2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression.
I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later.
Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
-tim
[not that this matters much....]
regards,
-tim
Thanks again, Todd
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Tim, Todd refers to your original post. The only thing I've found is a reference to a pre and post addition of zero, the latter was said not to work. Both work for me, see below:
zero= 0 zero + _num.arange(5) array([0, 1, 2, 3, 4]) _num.arange(5) + zero array([0, 1, 2, 3, 4])
If I'm barking up the wrong tree, could you point me to your original posting in the archive please? Colin W.
Todd Miller wrote:
On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
Todd,
This scheme seems rather complex. The purpose isn't clear to me.
What is the problem which this arrangement is intended to solve?
Bottom line, currently:
numarray + another_class --> exception
It would be nice if:
numarray + another_class --> another_class.__radd__(numarray)
Also, look backward at Tim's original post on this subject.
Could I suggest that some scheme along the lines of the Python PEP's be used to set out the Why's and Wherefore's?
Sounds good for next time.
It might also be useful to copy this discussion to comp.lang.python as that newsgroup has people experienced with classes.
OK, I'll stop being a chicken. Everyone, get ready for the sound of a pin dropping on c.l.py.
The problem appears to be Python related rather than a problem of numerical analysis.
Yes indeed.
Colin W.
Todd Miller wrote:
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay!
Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay!
Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer:
1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order.
2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression.
I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later.
Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
Hi Todd,
There are three ways to spell "defer to me" on the table (the precise details of each spelling are, of course, still open for debate):
1. numarray.defer_to(my_class)
2. class ArrayLike(numarray.DeferTo): # ...
3. class ArrayLike: _numarray_defer_to = True # ...
I'd prefer a non-registration solution since those are both aesthetically displeasing and leave you open to the situation where a class in module A gets registered by module B, but module C expects it not to be registered and everything breaks. Not all that likely, I admit, but let's avoid the registration version if we can.
I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
The other two solutions are almost equivalent. The one case where 3 has an edge over 2 is if I have an object (not a class), I could potentially set a _numarray_defer_to on the object before passing it to numarray without having to mess with the class of the object at all. YAGNI, though.
I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
The advantage of 2 in my view is that it *does* force you to subclass. With 3, there will be the temptation to poke into some other module and set _numarray_defer_to on some poor unsuspecting class. This has the same disadvantage as 1, that it could confuse some other poor unsuspecting module. The correct way to do get a deferred class from a third party module is to import and subclass. This works with either 2 or 3::
import A
class Klass2(a.Klass, numarray.DeferTo): #2 #...
class Klass3(a.Klass): #3 the good way _numarray_defer_to = True # ...
A.Klass._numarray_defer_to = True #3 the evil way.
Version 2 is cleaner and encourages you to do the right thing, so I'd prefer that solution.
Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
-tim
[not that this matters much....]
regards,
-tim
Thanks again, Todd
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Sat, 2003-09-27 at 15:53, Colin J. Williams wrote:
Tim,
Todd refers to your original post. The only thing I've found is a reference to a pre and post addition of zero, the latter was said not to work. Both work for me, see below:
zero= 0 zero + _num.arange(5) array([0, 1, 2, 3, 4]) _num.arange(5) + zero array([0, 1, 2, 3, 4])
If I'm barking up the wrong tree, could you point me to your original posting in the archive please?
It's not the wrong tree, but there's more than one case: objects which look like numeric sequence work but always return numarrays, objects which do not look like a numeric sequence raise an exception. Here's a link to the numpy-discussion-list archives on Source Forge: http://sourceforge.net/mailarchive/forum.php?thread_id=3154046&forum_id=4890
Colin W.
Todd Miller wrote:
On Fri, 2003-09-26 at 13:31, Colin J. Williams wrote:
Todd,
This scheme seems rather complex. The purpose isn't clear to me.
What is the problem which this arrangement is intended to solve?
Bottom line, currently:
numarray + another_class --> exception
It would be nice if:
numarray + another_class --> another_class.__radd__(numarray)
Also, look backward at Tim's original post on this subject.
Could I suggest that some scheme along the lines of the Python PEP's be used to set out the Why's and Wherefore's?
Sounds good for next time.
It might also be useful to copy this discussion to comp.lang.python as that newsgroup has people experienced with classes.
OK, I'll stop being a chicken. Everyone, get ready for the sound of a pin dropping on c.l.py.
The problem appears to be Python related rather than a problem of numerical analysis.
Yes indeed.
Colin W.
Todd Miller wrote:
I tried out DeferredToByNumArray yesterday morning and was able to get it to work as planned with numarray and MA. So yay!
Yesterday afternoon, I stopped by Perry's office for a final check, and not surprisingly, there are some aspects of this solution that we've overlooked. So not yay!
Perry pointed out two ways that the DeferredToByNumArray scheme may fall apart as our inheritance hierarchy becomes richer:
1) Imagine not one, but two independent subclasses of NumArray, each of which want NumArray to defer to them. Now imagine them trying to inter-operate with each other. In this case, the meaning of the expression is determined by the order of the operands, so A+B and B+A will either return an A or a B depending on the expression order.
2) Imagine again two subclasses of numarray, but this time imagine B as a subclass of A. Here again, both might want to defer to numarray, and again, A+B and B+A return different types driven by the type order of the expression.
I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later.
Todd On Wed, 2003-09-24 at 19:55, Tim Hochberg wrote:
Todd Miller wrote:
On Wed, 2003-09-24 at 18:27, Tim Hochberg wrote:
>Hi Todd, > >There are three ways to spell "defer to me" on the table (the precise >details of >each spelling are, of course, still open for debate): > > 1. numarray.defer_to(my_class) > > 2. class ArrayLike(numarray.DeferTo): > # ... > > 3. class ArrayLike: > _numarray_defer_to = True > # ... > >I'd prefer a non-registration solution since those are both >aesthetically displeasing and leave you open to the situation where a >class in module A gets registered by module B, but module C expects it >not to be registered and everything breaks. Not all that likely, I >admit, but let's avoid the registration version if we can. > > > > > > I was picturing this as module A registering it's own classes only. Nevertheless, inverting the problem and distributing the registration as you suggested is better.
The case you describe probably will describe the majority of actual use cases, and in fact describes mine. I'm trying to think ahead a bit to cases may encounter as start using NumArray more extensively. Let's hope this solution still looks good in six months!
>The other two solutions are almost equivalent. The one case where 3 has >an edge over 2 is if I have an object (not a class), I could potentially >set a _numarray_defer_to on the object before passing it to numarray >without having to mess with the class >of the object at all. YAGNI, though. > > > > > > I was more concerned about the potential impact of lots of multiple inheritance, but that's probably just my own personal blend of FUD.
>The advantage of 2 in my view is that it *does* force you to subclass. >With 3, there will be the temptation to poke into some other module and >set _numarray_defer_to on some poor unsuspecting class. This has the >same disadvantage as 1, that it could confuse some other poor >unsuspecting module. The correct way to do get a deferred class from a >third party module is to import and subclass. This works with either 2 >or 3:: > >import A > >class Klass2(a.Klass, numarray.DeferTo): #2 > #... > >class Klass3(a.Klass): #3 the good way > _numarray_defer_to = True > # ... > > A.Klass._numarray_defer_to = True #3 the evil way. > >Version 2 is cleaner and encourages you to do the right thing, so I'd >prefer that solution. > > > > > > > Good enough for me. If no one else has any comments, then numarray.DeferTo is where I'll start implementing. Tomorrow.
One more minor thing. I'm not sure tha DeferTo is ideal as the mix-in class name. It was perfect for the registration function name, but I'm not sure it's so clear whether the class or numarray is being deferred to when you say numarray.DeferTo. DeferToMe is more descriptive, but seems sort of slangy. DeferredTo is better than DeferTo, but still not as descriptive as DeferToMe. numarray.DefersTo reads perfect as long as numarray is included but is a disaster if you read it on your own. Below I've put down all the ideas I could come up with
class CustomArray(numarray.DeferTo) class CustomArray(numarray.DefersTo) class CustomArray(numarray.DeferredTo) class CustomArray(numarray.DeferToMe) class CustomArray(numarray.DeferredToByNumarray) class CustomArray(DeferTo) class CustomArray(DefersTo) class CustomArray(DeferredTo) class CustomArray(DeferToMe) class CustomArray(DeferredToByNumarray)
For me it's a toss up between DefferedTo, DeferToMe and DeferredToByNumarray. The first is a little lacking in descriptive power, the second is slangy and the third is wordy.
-tim
[not that this matters much....]
>regards, > >-tim > > > > > > Thanks again, Todd
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- Todd Miller jmiller@stsci.edu STSCI / ESS / SSB
On Fri, 2003-09-26 at 12:20, Tim Hochberg wrote:
Todd Miller wrote: I'd make the same comment for this case as for the above case except that all else being equal, subclasses should prevail over superclasses.
I don't have a solution yet, but am hopeful that more free candy will fall from the sky... or YAGNI. Perry pointed out that similar problems exist for *any* Python class hierarchy, so we're not alone, and perhaps should forget about this until it matters. If someone sees an easy fix, now would be better than later.
Here's some quick thoughts. In order for this to work out sensibly you want the order between classes to be transistive and you probably also always want to be able to insert a class into the order between any two classes This is equivalent to deciding the op priority based on a comparison based on some real number associated with each class. This leads to something like:
Once again I think your ideas solve the problem, so thanks. I have a few things I wanted to bounce off you.
class NumArrayFamily: op_priority = 0.0
# This makes it compatible with the current proposal , but isn't strictly necessary. class DeferredToByNumarray(NumArrayFamily): op_priority = 1000.0
class NumArrary(..., NumArrayFamily): # ... def __add__(self, object): if isinstance(object, NumArrayFamily): if ((object.op_priority > self.object.op_priority) or (object.op_priority == self.object.op_priority) and issubclass(object, self.__class__)): return other.__radd__(self) return ufunc.add(self, other)
class MySubclassOfNumarray(NumArray): op_priority = 10.0 # deffered to by NumArray, but defers to DeferredToByNumarray classes.
As I understand it, 2.2 new style object subclass r-operators automatically get priority (subclass.__radd__ is called in preference to superclass.__add__). I think this has two implications: 1) the ==/subclass term won't work, because __add__ will never get called. 2) we have to redefine __radd__ as well to implement the priority scheme. I'm not sure the isinstance is necessary for __radd__, but I like the symmetry. How does this look: class DeferredToByNumArray: # name seems a little off now, but whatever pass class NumArray(..., DeferredToByNumArray): op_priority = 0 def __add__(self, object): if (isinstance(object, DeferredToByNumArray) and object.op_priority > self.op_priority): return object.__radd__(self) return ufunc.add(self, object) def __radd__(self, object): if (isinstance(object, DeferredToByNumArray) and object.op_priority > self.op_priority): return object.__add__(self) return ufunc.add(object, self) class MyForeignClass(DeferredToNumArray): op_priority = 1.0 class MySubclassOfNumArray(NumArray): op_priority = -1.0
Of course the user still has to be careful to make sure the priority order makes sense and you could have big problems when combining packages with disprate ideas about priority levels, but I'm not sure how you can get away from that.
As long as there's a scheme in place to resolve potential conflicts with a little discussion, I think your solution is already ahead of the demand. I told Colin last week I'd raise this issue on c.l.py. I have decided not to (although someone else could obviously do it if they are still concerned about it), because (a) the problem is solved, (b) explaining the problem well enough to get usable input is difficult and I've got other stuff to do. Thanks again for these ideas, Todd -- Todd Miller <jmiller@stsci.edu>
participants (4)
-
Colin J. Williams -
Fernando Perez -
Tim Hochberg -
Todd Miller