how to do this efficiently?
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like: last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1 There's probably a better way to do it, without the arange, though...
Zachary Pincus wrote:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like:
last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1
There's probably a better way to do it, without the arange, though...
I'm trying to find the first point in a power spectrum such that all subsequent points are below some level. I've started with: db is my power spectrum in dB, It is already reversed. mag = np.maximum.accumulate (db) - db[-1] Now all I need is to find the first point such that mag < -50. How to do this efficiently?
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like:
last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1
There's probably a better way to do it, without the arange, though...
I'm trying to find the first point in a power spectrum such that all subsequent points are below some level. I've started with:
db is my power spectrum in dB, It is already reversed.
mag = np.maximum.accumulate (db) - db[-1]
Now all I need is to find the first point such that mag < -50. How to do this efficiently?
Right -- that's what I showed above. Find the last point in mag that is >= -50, and by definition the next point is the first point such that the remainder of mag is < -50.
Zachary Pincus wrote:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like:
last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1
There's probably a better way to do it, without the arange, though...
I'm trying to find the first point in a power spectrum such that all subsequent points are below some level. I've started with:
db is my power spectrum in dB, It is already reversed.
mag = np.maximum.accumulate (db) - db[-1]
Now all I need is to find the first point such that mag < -50. How to do this efficiently?
Right -- that's what I showed above. Find the last point in mag that is >= -50, and by definition the next point is the first point such that the remainder of mag is < -50.
But where is numpy's 'find_first' function? I can't seem to find it so I had to make my own in C++.
On Feb 9, 2011, at 10:58 AM, Neal Becker wrote:
Zachary Pincus wrote:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like:
last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1
There's probably a better way to do it, without the arange, though...
I'm trying to find the first point in a power spectrum such that all subsequent points are below some level. I've started with:
db is my power spectrum in dB, It is already reversed.
mag = np.maximum.accumulate (db) - db[-1]
Now all I need is to find the first point such that mag < -50. How to do this efficiently?
Right -- that's what I showed above. Find the last point in mag that is >= -50, and by definition the next point is the first point such that the remainder of mag is < -50.
But where is numpy's 'find_first' function? I can't seem to find it so I had to make my own in C++.
As before, the line below does what you said you need, though not maximally efficiently. (Try it in an interpreter...) There may be another way in numpy that doesn't rely on constructing the index array, but this is the first thing that came to mind. last_greater = numpy.arange(arr.shape)[arr >= T][-1] Let's unpack that dense line a bit: mask = arr >= T indices = numpy.arange(arr.shape) above_threshold_indices = indices[mask] last_above_threshold_index = above_threshold_indices[-1] Does this make sense?
On Wed, Feb 9, 2011 at 11:04 AM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
On Feb 9, 2011, at 10:58 AM, Neal Becker wrote:
Zachary Pincus wrote:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like:
last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1
There's probably a better way to do it, without the arange, though...
I'm trying to find the first point in a power spectrum such that all subsequent points are below some level. I've started with:
db is my power spectrum in dB, It is already reversed.
mag = np.maximum.accumulate (db) - db[-1]
Now all I need is to find the first point such that mag < -50. How to do this efficiently?
Right -- that's what I showed above. Find the last point in mag that is >= -50, and by definition the next point is the first point such that the remainder of mag is < -50.
But where is numpy's 'find_first' function? I can't seem to find it so I had to make my own in C++.
As before, the line below does what you said you need, though not maximally efficiently. (Try it in an interpreter...) There may be another way in numpy that doesn't rely on constructing the index array, but this is the first thing that came to mind.
last_greater = numpy.arange(arr.shape)[arr >= T][-1]
Let's unpack that dense line a bit:
mask = arr >= T indices = numpy.arange(arr.shape) above_threshold_indices = indices[mask] last_above_threshold_index = above_threshold_indices[-1]
Does this make sense?
This assumes monotonicity. Is that allowed? Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
As before, the line below does what you said you need, though not maximally efficiently. (Try it in an interpreter...) There may be another way in numpy that doesn't rely on constructing the index array, but this is the first thing that came to mind.
last_greater = numpy.arange(arr.shape)[arr >= T][-1]
Let's unpack that dense line a bit:
mask = arr >= T indices = numpy.arange(arr.shape) above_threshold_indices = indices[mask] last_above_threshold_index = above_threshold_indices[-1]
Does this make sense?
This assumes monotonicity. Is that allowed?
The twice-stated problem was:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
So that should do the trick... Though Alan's argmax solution is definitely a better one than indexing an indices array. Same logic and result though, just more compact.
This assumes monotonicity. Is that allowed?
The twice-stated problem was:
[Note to avert email-miscommunications] BTW, I wasn't trying to snipe at you with that comment, Josef... I just meant to say that this solution solves the problem as Neal posed it, though that might not be the exact problem he has.
On Wed, Feb 9, 2011 at 11:25 AM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
This assumes monotonicity. Is that allowed?
The twice-stated problem was:
[Note to avert email-miscommunications] BTW, I wasn't trying to snipe at you with that comment, Josef...
I just meant to say that this solution solves the problem as Neal posed it, though that might not be the exact problem he has.
In a 1-d array, find the first point where all subsequent points have values less than a threshold.
This doesn't imply monotonicity. Suppose with have a sin curve, and I want to find the last trough. Or a business cycle and I want to find the last recession. Unless my english deteriorated recently. :) Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
In a 1-d array, find the first point where all subsequent points have values less than a threshold.
This doesn't imply monotonicity. Suppose with have a sin curve, and I want to find the last trough. Or a business cycle and I want to find the last recession.
Unless my english deteriorated recently.
Not sure that I follow? I read the statement as: find the smallest non-negative index i such that array[j] < t for all i <= j < len(array) for which the various solutions proposed work, except for the corner case where array.min() >= t. I'm not sure where monotonicity comes into play, unless one interprets the "all subsequent points" clause as "some number of subsequent points" or something... For those sort of problems, I usually wind up smoothing the array [optional], and then using scipy.ndimage.maximum to calculate peak values within a given window, and then find the points that equal the peak values -- this works pretty robustly for peak/trough detection in arbitrary dimension.
On Wed, Feb 9, 2011 at 1:18 PM, Zachary Pincus <zachary.pincus@yale.edu> wrote:
In a 1-d array, find the first point where all subsequent points have values less than a threshold.
This doesn't imply monotonicity. Suppose with have a sin curve, and I want to find the last trough. Or a business cycle and I want to find the last recession.
Unless my english deteriorated recently.
Not sure that I follow? I read the statement as: find the smallest non-negative index i such that array[j] < t for all i <= j < len(array)
for which the various solutions proposed work, except for the corner case where array.min() >= t.
I'm not sure where monotonicity comes into play, unless one interprets the "all subsequent points" clause as "some number of subsequent points" or something...
For those sort of problems, I usually wind up smoothing the array [optional], and then using scipy.ndimage.maximum to calculate peak values within a given window, and then find the points that equal the peak values -- this works pretty robustly for peak/trough detection in arbitrary dimension.
(Sorry, I should reply when I'm not able to actually pay attention) your code works fine with a hump, I didn't read the example carefully enough
x = np.arange(10,0,-1) x2 = 25-(x-5)**2 x2 array([ 0, 9, 16, 21, 24, 25, 24, 21, 16, 9]) T=20 np.argmax(x2<T) 0 np.arange(x2.shape[0])[x2 >= T][-1]+1 8 len(x)-np.argmax(x2[::]>T) + 1 8
Josef
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 02/09/2011 10:17 AM, Zachary Pincus wrote:
As before, the line below does what you said you need, though not maximally efficiently. (Try it in an interpreter...) There may be another way in numpy that doesn't rely on constructing the index array, but this is the first thing that came to mind.
last_greater = numpy.arange(arr.shape)[arr>= T][-1]
Let's unpack that dense line a bit:
mask = arr>= T indices = numpy.arange(arr.shape) above_threshold_indices = indices[mask] last_above_threshold_index = above_threshold_indices[-1]
Does this make sense? This assumes monotonicity. Is that allowed? The twice-stated problem was:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T. So that should do the trick... Though Alan's argmax solution is definitely a better one than indexing an indices array. Same logic and result though, just more compact.
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion This is similar to Zachary's as it just uses 'np.where'.
x=np.array([5,4,3,6,7,3,2,1]) x array([5, 4, 3, 6, 7, 3, 2, 1]) np.argmax(x>5) # doesn't appear to be correct 3 np.where(x>5)[0][-1]#since np.where gives a tuple and you need the last element 4
This should give the index where all subsequent points are less than some threshold. However, this and Zachary's version fail when if all points are lower than the threshold (eg T=10 in this array). Bruce
On 2/9/2011 11:39 AM, Bruce Southey wrote:
np.argmax(x>5) # doesn't appear to be correct
It was an answer to the particular question of how to do find_first, which it does (at the cost of a boolean array): it finds the first element greater than 5. >>> x array([5, 4, 3, 6, 7, 3, 2, 1]) >>> np.argmax(x>5) 3 >>> x[3] 6 You can also find the first element less than 4, etc. >>> np.argmax(x<4) 2 The condition should be chosen for the problem. As Josef pointed out, the usefulness of this and other approaches can depend on the monotonicity of the series. fwiw, Alan
Zachary Pincus wrote:
On Feb 9, 2011, at 10:58 AM, Neal Becker wrote:
Zachary Pincus wrote:
In a 1-d array, find the first point where all subsequent points have values less than a threshold, T.
Maybe something like:
last_greater = numpy.arange(arr.shape)[arr >= T][-1] first_lower = last_greater + 1
There's probably a better way to do it, without the arange, though...
I'm trying to find the first point in a power spectrum such that all subsequent points are below some level. I've started with:
db is my power spectrum in dB, It is already reversed.
mag = np.maximum.accumulate (db) - db[-1]
Now all I need is to find the first point such that mag < -50. How to do this efficiently?
Right -- that's what I showed above. Find the last point in mag that is >= -50, and by definition the next point is the first point such that the remainder of mag is < -50.
But where is numpy's 'find_first' function? I can't seem to find it so I had to make my own in C++.
As before, the line below does what you said you need, though not maximally efficiently. (Try it in an interpreter...) There may be another way in numpy that doesn't rely on constructing the index array, but this is the first thing that came to mind.
last_greater = numpy.arange(arr.shape)[arr >= T][-1]
Let's unpack that dense line a bit:
mask = arr >= T indices = numpy.arange(arr.shape) above_threshold_indices = indices[mask] last_above_threshold_index = above_threshold_indices[-1]
Does this make sense?
Yes, thanks for the explanation. Still, I wonder if numpy would benefit from a 'find_first' function.
On Wed, Feb 9, 2011 at 10:48, Neal Becker <ndbecker2@gmail.com> wrote:
Still, I wonder if numpy would benefit from a 'find_first' function.
I certainly could have used one many times. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (6)
-
Alan G Isaac -
Bruce Southey -
josef.pktd＠gmail.com -
Neal Becker -
Robert Kern -
Zachary Pincus