I just came across a problem with the intention to specify unset boundaries given to numpy.clip() or array.clip():
a.clip(1e-10, None) a.clip(None, -1e-10)
When doing this, the returned array is dtype=numpy.object, seemingly None gets converted to a numpy.asarray(None, dtype=numpy.object), and this is then used for the comparison. Since (*any number* > None) is true, and (*any number* < None) is false, it works or not, depending on the usecase, despite one is still left with a dtype=numpy.object array. Precisely, using None as the lower boundary semi-works, while using None as the upper boundary doesn't match expectations at all.
One can work around this, sure, but in my opinion it would be handy anyway to have clip() interpret None as argument as \pm \infty. Maybe I'm just missing the appropriate function, though.
Friedrich
Ah, no need to answer, I do this myself:
Friedrich, would you please use numpy.inf and -numpy.inf.
Thanks, and sorry for the noise, Friedrich
On Tue, Sep 7, 2010 at 15:12, Friedrich Romstedt friedrichromstedt@gmail.com wrote:
Ah, no need to answer, I do this myself:
Friedrich, would you please use numpy.inf and -numpy.inf.
But if you have an integer array, you will run into the same problem. The result will be upcast to float. I think we would accept a patch that interprets None to be the appropriate extreme bound given the input datatype. This will be tricky, though, since it must be done in C (PyArray_Clip in numpy/core/src/multiarray/calculation.c).
Robert Kern <robert.kern <at> gmail.com> writes:
On Tue, Sep 7, 2010 at 15:12, Friedrich Romstedt <friedrichromstedt <at> gmail.com> wrote:
Ah, no need to answer, I do this myself:
Friedrich, would you please use numpy.inf and -numpy.inf.
But if you have an integer array, you will run into the same problem. The result will be upcast to float. I think we would accept a patch that interprets None to be the appropriate extreme bound given the input datatype. This will be tricky, though, since it must be done in C (PyArray_Clip in numpy/core/src/multiarray/calculation.c).
I've been a bit confused about what numpy's clip is supposed to support. The documentation (e.g. http://docs.scipy.org/doc/numpy/reference/generated/numpy.clip.html) doesn't make any mention of supporting only one of min or max, but I remember a message some time back from Travis Oliphant that seemed to suggest it does (http://thread.gmane.org/gmane.comp.python.numeric.general/17844/focus=17877).
Right now, I'm surprised by two aspect's of clip's behavior, both demonstrated below...
$ python Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy numpy.__version__
'2.0.0.dev8698'
a = numpy.array([1,2,3,4,5]) a.clip(2,None)
array([2, 2, 2, 2, 2], dtype=object)
I'm not sure why the returned array has a dtype of object (although that can be avoided by using clip's "out" argument), and also why the values are not like this:
numpy.maximum(a,2)
array([2, 2, 3, 4, 5])
I've been using minimum() and maximum() instead of clip() to have "clip upper" and "clip lower" functions, so it's not really a problem for me, I'm just curious (and thought maybe this could be useful for others searching in the future).
Thanks, Chris
On Wed, Sep 8, 2010 at 14:42, Chris Ball ceball@gmail.com wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
On Tue, Sep 7, 2010 at 15:12, Friedrich Romstedt <friedrichromstedt <at> gmail.com> wrote:
Ah, no need to answer, I do this myself:
Friedrich, would you please use numpy.inf and -numpy.inf.
But if you have an integer array, you will run into the same problem. The result will be upcast to float. I think we would accept a patch that interprets None to be the appropriate extreme bound given the input datatype. This will be tricky, though, since it must be done in C (PyArray_Clip in numpy/core/src/multiarray/calculation.c).
I've been a bit confused about what numpy's clip is supposed to support. The documentation (e.g. http://docs.scipy.org/doc/numpy/reference/generated/numpy.clip.html) doesn't make any mention of supporting only one of min or max, but I remember a message some time back from Travis Oliphant that seemed to suggest it does (http://thread.gmane.org/gmane.comp.python.numeric.general/17844/focus=17877).
Right now, I'm surprised by two aspect's of clip's behavior, both demonstrated below...
$ python Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import numpy numpy.__version__
'2.0.0.dev8698'
a = numpy.array([1,2,3,4,5]) a.clip(2,None)
array([2, 2, 2, 2, 2], dtype=object)
I'm not sure why the returned array has a dtype of object
The usual type-matching semantics. a.clip() tries to find a common type that fits `a`, 2 and None. Since None only fits in an object dtype, that's what you get.
(although that can be avoided by using clip's "out" argument), and also why the values are not like this:
numpy.maximum(a,2)
array([2, 2, 3, 4, 5])
clip(a, low, high) boils down to this in the C code itself (or at least the code path that it goes down when the input array needs to be upcast):
maximum(minimum(a, high), low)
Since all integer objects compare > None based on the default Python 2.x implementation of comparison across types, you would just get the low value out of that.
Robert Kern <robert.kern <at> gmail.com> writes:
On Wed, Sep 8, 2010 at 14:42, Chris Ball <ceball <at> gmail.com> wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
...
a = numpy.array([1,2,3,4,5]) a.clip(2,None)
array([2, 2, 2, 2, 2], dtype=object)
I'm not sure why the returned array has a dtype of object
The usual type-matching semantics. a.clip() tries to find a common type that fits `a`, 2 and None. Since None only fits in an object dtype, that's what you get.
...
and also why the values are not like this:
numpy.maximum(a,2)
array([2, 2, 3, 4, 5])
clip(a, low, high) boils down to this in the C code itself (or at least the code path that it goes down when the input array needs to be upcast):
maximum(minimum(a, high), low)
Since all integer objects compare > None based on the default Python 2.x implementation of comparison across types, you would just get the low value out of that.
Thanks for explaining that. It all makes sense if clip() requires both bounds to be passed, which is what the documentation indicates, so there is not really a problem. I was just confused because of the post and associated commit from Travis a while back implying that he'd altered clip() to support only one bound ("Allow clip method to have either min or max passed in", 4566).
Chris
On Thu, Sep 9, 2010 at 05:05, Chris Ball ceball@gmail.com wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
On Wed, Sep 8, 2010 at 14:42, Chris Ball <ceball <at> gmail.com> wrote:
Robert Kern <robert.kern <at> gmail.com> writes:
...
a = numpy.array([1,2,3,4,5]) a.clip(2,None)
array([2, 2, 2, 2, 2], dtype=object)
I'm not sure why the returned array has a dtype of object
The usual type-matching semantics. a.clip() tries to find a common type that fits `a`, 2 and None. Since None only fits in an object dtype, that's what you get.
...
and also why the values are not like this:
numpy.maximum(a,2)
array([2, 2, 3, 4, 5])
clip(a, low, high) boils down to this in the C code itself (or at least the code path that it goes down when the input array needs to be upcast):
maximum(minimum(a, high), low)
Since all integer objects compare > None based on the default Python 2.x implementation of comparison across types, you would just get the low value out of that.
Thanks for explaining that. It all makes sense if clip() requires both bounds to be passed, which is what the documentation indicates, so there is not really a problem. I was just confused because of the post and associated commit from Travis a while back implying that he'd altered clip() to support only one bound ("Allow clip method to have either min or max passed in", 4566).
I believe that the clip function was overhauled sometime after that. The enhancement probably got lost then.
-
Chris Ball -
Friedrich Romstedt -
Robert Kern