[Numpy-discussion] Quick array value assignment based on common values

[PHobson at Geosyntec.com]

I've deleted the code b/c it was absurdly slow. It was pretty brute-force.
-Looped through each row (r) of y
-check to see where y[r,0] - x[:,0] < eps (call that row r_hit)
-set y[r,1] = x[r_hit,1]

There was kind of a short fuse on this, and I was already reading the data from a text file. So I just wrote all of the continuous dates to a file, threw it in a spreadsheet, brought in the other data and did a lookup function that about destroyed my machine. Probably would have been faster to let the looping run [hangs head in shame].

In the future, I'll definitely try the solutions you've outlined.

Thanks again!
-paul


From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-bounces at scipy.org] On Behalf Of John Salvatier
Sent: Wednesday, August 04, 2010 6:23 PM
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Quick array value assignment based on common values

Perhaps try the following:

1) sort x by x[:,0]
2) sort y by y[:,0]
3) loop through both at the same time building an array of indexes A that tells you the index of y[i,0] in x or just building a new array z with the value if you don't need them in order
4) if you do need them in order, unsort A by the sorting used to sort y and then index into x using the unsorted A.

use http://docs.scipy.org/doc/numpy/reference/generated/numpy.argsort.html#numpy.argsort
On Wed, Aug 4, 2010 at 6:09 PM, John Salvatier <jsalvati at u.washington.edu<mailto:jsalvati at u.washington.edu>> wrote:
How exactly are you looping? That sounds absurdly slow.

What you need is a fast dictionary.
On Wed, Aug 4, 2010 at 6:00 PM, Gökhan Sever <gokhansever at gmail.com<mailto:gokhansever at gmail.com>> wrote:

On Wed, Aug 4, 2010 at 6:59 PM, <PHobson at geosyntec.com<mailto:PHobson at geosyntec.com>> wrote:
Hey folks,

I've one array, x, that you could define as follows:
[[1, 2.25],
 [2, 2.50],
 [3, 2.25],
 [4, 0.00],
 [8, 0.00],
 [9, 2.75]]

Then my second array, y, is:
[[1, 0.00],
 [2, 0.00],
 [3, 0.00],
 [4, 0.00],
 [5, 0.00],
 [6, 0.00],
 [7, 0.00],
 [8, 0.00],
 [9, 0.00],
 [10,0.00]]

Is there a concise, Numpythonic way to copy the values of x[:,1] over to y[:,1] where x[:,0] = y[:,0]? Resulting in, z:
[[1, 2.25],
 [2, 2.50],
 [3, 2.25],
 [4, 0.00],
 [5, 0.00],
 [6, 0.00],
 [7, 0.00],
 [8, 0.00],
 [9, 2.75],
 [10,0.00]]

My current task has len(x) = 25000 and len(y) = 350000 and looping through is quite slow unfortunately.

Many thanks,
-paul


_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion

My simplest approach would be:

y[x[:0]-1] = x

# Providing the arrays are nicely ordered and 1st column x is all integer.

--
Gökhan

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org<mailto:NumPy-Discussion at scipy.org>
http://mail.scipy.org/mailman/listinfo/numpy-discussion


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100804/b99ff56f/attachment.html>