Hello, I have had encountered some problem while I was trying to create the following code which finds number of the same values and indexes in an array or list. Here is the code: y = [ 1, 12, 3, 3, 5, 1, 1, 34, 0, 0, 1, 5] OR y = array( [ 1, 12, 3, 3, 5, 1, 1, 34, 0, 0, 1, 5 ] ) b = [ [ item for item in range( len(y) ) if y[ item ] == y[ j ] ] for j in range(0, len( y ) ) ] answer: [ [ 0, 5, 6, 10], [1], [2, 3], [2, 3], [4, 11], [0, 5, 6, 10], [0, 5, 6, 10], [7], [8, 9], [8, 9], [0, 5, 6, 10], [4, 11] ] The result I want to get is ,not that shown above as an answer, that I want to calculate the number of the same values and their indexes in not repeated way as well. For example, '1' - 4, index: '0, 5, 6, 10' '12' - 1, index: '1' '3' - 2, index: '2, 3' '5' - 2, index: '4, 11' '34' - 1, index: '7' '0' - 2, index: '8, 9' Any answer would be appreciated.. --
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind) for ind in range(len(x))] x will give you the list of unique values, and f will give you the indices of each corresponding value in x. So f[0] is the indices of x[0] in y. To explain, unique in this form gives two outputs, a sorted, non-repeating list of values (x), and an array of the same shape as y that gives you the indices of x of each corresponding value of y (i, that is x[i] is the same as y) The second goes through each index of x and finds where that index occurs in i. On Wed, Apr 17, 2013 at 9:44 AM, Happyman <bahtiyor_zohidov@mail.ru> wrote:
Hello,
I have had encountered some problem while I was trying to create the following code which finds number of the same values and indexes in an array or list. Here is the code:
y = [ 1, 12, 3, 3, 5, 1, 1, 34, 0, 0, 1, 5] OR y = array( [ 1, 12, 3, 3, 5, 1, 1, 34, 0, 0, 1, 5 ] )
b = [ [ item for item in range( len(y) ) if y[ item ] == y[ j ] ] for j in range(0, len( y ) ) ]
answer: [ [ 0, 5, 6, 10], [1], [2, 3], [2, 3], [4, 11], [0, 5, 6, 10], [0, 5, 6, 10], [7], [8, 9], [8, 9], [0, 5, 6, 10], [4, 11] ]
The result I want to get is ,not that shown above as an answer, that I want to calculate the number of the same values and their indexes in not repeated way as well. For example, '1' - 4, index: '0, 5, 6, 10' '12' - 1, index: '1' '3' - 2, index: '2, 3' '5' - 2, index: '4, 11' '34' - 1, index: '7' '0' - 2, index: '8, 9'
Any answer would be appreciated..
--
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Apr 17, 2013 at 10:46 AM, Todd <toddrjen@gmail.com> wrote:
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind) for ind in range(len(x))]
A better version would be (np.where returns tuples, but we don't want tuples): x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))] You can also do it this way, but it is much harder to read IMO: x=numpy.unique(y) f=numpy.split(numpy.argsort(y), numpy.nonzero(numpy.diff(numpy.sort(y)))[0]+1) This version figures out the indexes needed to put the values of y in sorted order (the same order x uses), then splits it into sub-arrays based on value. The principle is simpler but the implementation looks like clear to me. Note that these are only guaranteed to work on 1D arrays, I have not tested them on multidimensional arrays
Hi Todd, Greaaat thanks for your help.. By the way, the first one (I think) is much simpler.. I tested it and ,of course, it is 1D, but it is also a good idea to consider it for Ndimensional. I prefer the first one! Do you you think first version is okay to use? Среда, 17 апреля 2013, 11:02 +02:00 от Todd <toddrjen@gmail.com>:
On Wed, Apr 17, 2013 at 10:46 AM, Todd < toddrjen@gmail.com > wrote:
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind) for ind in range(len(x))]
A better version would be (np.where returns tuples, but we don't want tuples):
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))]
You can also do it this way, but it is much harder to read IMO:
x=numpy.unique(y) f=numpy.split(numpy.argsort(y), numpy.nonzero(numpy.diff(numpy.sort(y)))[0]+1)
This version figures out the indexes needed to put the values of y in sorted order (the same order x uses), then splits it into sub-arrays based on value. The principle is simpler but the implementation looks like clear to me.
Note that these are only guaranteed to work on 1D arrays, I have not tested them on multidimensional arrays
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, 2013-04-17 at 13:32 +0400, Happyman wrote:
Hi Todd, Greaaat thanks for your help.. By the way, the first one (I think) is much simpler.. I tested it and ,of course, it is 1D, but it is also a good idea to consider it for Ndimensional. I prefer the first one! Do you you think first version is okay to use?
If you are only interested in the count, using np.bincount should be much faster then the list comprehension with "==". Of course that gives you a count of zero for all indexes that do not exist. But even then I very much expect that filtering those out afterwards will be faster unless your "indexes" can be arbitrary large. Of course bincount loses the order information, so if you need that, you can only replace the second step with it. - Sebastian
Среда, 17 апреля 2013, 11:02 +02:00 от Todd <toddrjen@gmail.com>: On Wed, Apr 17, 2013 at 10:46 AM, Todd <toddrjen@gmail.com> wrote: x,i=numpy.unique(y, return_inverse=True)
f=[numpy.where(i==ind) for ind in range(len(x))]
A better version would be (np.where returns tuples, but we don't want tuples):
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))]
You can also do it this way, but it is much harder to read IMO:
x=numpy.unique(y) f=numpy.split(numpy.argsort(y), numpy.nonzero(numpy.diff(numpy.sort(y)))[0]+1)
This version figures out the indexes needed to put the values of y in sorted order (the same order x uses), then splits it into sub-arrays based on value. The principle is simpler but the implementation looks like clear to me.
Note that these are only guaranteed to work on 1D arrays, I have not tested them on multidimensional arrays
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Okay Todd, In both results, I got the proper values.. for me indices are important also counting. From your code: let's say function--> sort_out(data): x , f = sort_out( data ) The data type: x in ndarray and x[ i ]--> int64 type(f) --> ' list ' type( f[ 0 ] ) --> ' tuple ' type( f[ 0][0] ) --> 'ndarray' type( f[ 0 ][ 0 ][ 0] ) --> 'int64' How do you think to avoid diversity if data type in this example? I think it is not necessary to get diverse dtype as well as more than 1D array.. ?? Среда, 17 апреля 2013, 11:53 +02:00 от Sebastian Berg <sebastian@sipsolutions.net>:
On Wed, 2013-04-17 at 13:32 +0400, Happyman wrote:
Hi Todd, Greaaat thanks for your help.. By the way, the first one (I think) is much simpler.. I tested it and ,of course, it is 1D, but it is also a good idea to consider it for Ndimensional. I prefer the first one! Do you you think first version is okay to use?
If you are only interested in the count, using np.bincount should be much faster then the list comprehension with "==". Of course that gives you a count of zero for all indexes that do not exist. But even then I very much expect that filtering those out afterwards will be faster unless your "indexes" can be arbitrary large. Of course bincount loses the order information, so if you need that, you can only replace the second step with it.
- Sebastian
Среда, 17 апреля 2013, 11:02 +02:00 от Todd < toddrjen@gmail.com >: On Wed, Apr 17, 2013 at 10:46 AM, Todd < toddrjen@gmail.com > wrote: x,i=numpy.unique(y, return_inverse=True)
f=[numpy.where(i==ind) for ind in range(len(x))]
A better version would be (np.where returns tuples, but we don't want tuples):
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))]
You can also do it this way, but it is much harder to read IMO:
x=numpy.unique(y) f=numpy.split(numpy.argsort(y), numpy.nonzero(numpy.diff(numpy.sort(y)))[0]+1)
This version figures out the indexes needed to put the values of y in sorted order (the same order x uses), then splits it into sub-arrays based on value. The principle is simpler but the implementation looks like clear to me.
Note that these are only guaranteed to work on 1D arrays, I have not tested them on multidimensional arrays
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The data type: x in ndarray and x[ i ]--> int64 type(f) --> ' list ' type( f[ 0 ] ) --> ' tuple ' type( f[ 0][0] ) --> 'ndarray' type( f[ 0 ][ 0 ][ 0] ) --> 'int64'
How do you think to avoid diversity if data type in this example? I think it is not necessary to get diverse dtype as well as more than 1D array..
That is why I suggested this approach was better ( note the that this is where()[0] instead of just where() as it was in my first example): x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))] type(f) --> list type(f[0]) --> ndarray type(f[0][0]) is meaningless since it is just a single element in an array. It must be an int type of some sort of since indices have to be int types. x will be the same dtype as your input array. You could conceivably change the type of f[0] to a list, but why would you want to? One of the big advantages of python is that usually it doesn't matter what the type is. In this case, a numpy ndarray will work the same as a list in most cases where you would want to use these sorts of indices. It is possibly to change the ndarray to a list, but unless there is a specific reason you need to use lists so then it is better not to. You cannot change the list to an ndarray because the elements of the list are different lengths. ndarray doesn't support that.
At bit OT, but I am new to numpy. The help for np.where says: Returns ------- out : ndarray or tuple of ndarrays If both `x` and `y` are specified, the output array contains elements of `x` where `condition` is True, and elements from `y` elsewhere. If only `condition` is given, return the tuple ``condition.nonzero()``, the indices where `condition` is True. However, I don't see any case that it returns an ndarray (it always seems to return a tuple of ndarrys). It seems to me for the case where only 'condition' is given it should return just the ndarry, eg (using this case discussed above): In [44]: np.where(i==0) Out[44]: (array([8, 9]),) This should just return the ndarray and not the tuple of ndarrays. In what case does it only return the ndarray? Thanks, Bob On Wed, Apr 17, 2013 at 4:34 AM, Todd <toddrjen@gmail.com> wrote:
The data type:
x in ndarray and x[ i ]--> int64 type(f) --> ' list ' type( f[ 0 ] ) --> ' tuple ' type( f[ 0][0] ) --> 'ndarray' type( f[ 0 ][ 0 ][ 0] ) --> 'int64'
How do you think to avoid diversity if data type in this example? I think it is not necessary to get diverse dtype as well as more than 1D array..
That is why I suggested this approach was better ( note the that this is where()[0] instead of just where() as it was in my first example):
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))]
type(f) --> list type(f[0]) --> ndarray
type(f[0][0]) is meaningless since it is just a single element in an array. It must be an int type of some sort of since indices have to be int types. x will be the same dtype as your input array.
You could conceivably change the type of f[0] to a list, but why would you want to? One of the big advantages of python is that usually it doesn't matter what the type is. In this case, a numpy ndarray will work the same as a list in most cases where you would want to use these sorts of indices. It is possibly to change the ndarray to a list, but unless there is a specific reason you need to use lists so then it is better not to.
You cannot change the list to an ndarray because the elements of the list are different lengths. ndarray doesn't support that.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
okay Todd, I got it. There are some reasons why I preferred asking that question. Let me explain: I am using Excel data which contains 3 columns and say 12 rows to process some simple data. What I want to do with the code you provided is that In the first column A has data that indicates the same ID where 2nd and 3rd has the same value. In other words I will put the following sample data to explain better: A_column = [ 22, 92, 64, 64, 77, 77, 64, 64, 22, 92, 99, 200 ] # The same length 12 B_column = [ 8, 8, 8, 8, 8, 0, 0, 0, 0, 0, 12, 0] # The same length 12 C_column = [ 0, 0, 0, 0, 0, 8, 8, 8, 8, 8, 4, 13] # The same length 12 The main reason for the question is as we discussed we already processed data in A_column. B_column has five "8" numbers in C_column as well but not in the same index!!! I want to get the following result which really confused me in terms of 'dtype': IF A_column the same THEN The value of B_column and C_columns in the ID (index, we got) is the same THEN get B and C value otherwise NO... I hope you would understand my problem Среда, 17 апреля 2013, 12:34 +02:00 от Todd < toddrjen@gmail.com >:
The data type: x in ndarray and x[ i ]--> int64 type(f) --> ' list ' type( f[ 0 ] ) --> ' tuple ' type( f[ 0][0] ) --> 'ndarray' type( f[ 0 ][ 0 ][ 0] ) --> 'int64'
How do you think to avoid diversity if data type in this example? I think it is not necessary to get diverse dtype as well as more than 1D array.. That is why I suggested this approach was better ( note the that this is where()[0] instead of just where() as it was in my first example):
x,i=numpy.unique(y, return_inverse=True) f=[numpy.where(i==ind)[0] for ind in range(len(x))]
type(f) --> list type(f[0]) --> ndarray
type(f[0][0]) is meaningless since it is just a single element in an array. It must be an int type of some sort of since indices have to be int types. x will be the same dtype as your input array.
You could conceivably change the type of f[0] to a list, but why would you want to? One of the big advantages of python is that usually it doesn't matter what the type is. In this case, a numpy ndarray will work the same as a list in most cases where you would want to use these sorts of indices. It is possibly to change the ndarray to a list, but unless there is a specific reason you need to use lists so then it is better not to.
You cannot change the list to an ndarray because the elements of the list are different lengths. ndarray doesn't support that. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
Bob Nnamtrop -
Happyman -
Sebastian Berg -
Todd