I have picked up a strange limit to np.in1d(): ---------- b Out[100]: array(['2007-01-01T02:30:00+0200', '2007-01-01T03:00:00+0200', '2007-01-01T03:30:00+0200', ..., '2008-01-01T01:00:00+0200', '2008-01-01T01:30:00+0200', '2008-01-01T02:00:00+0200'], dtype='datetime64[s]') b.shape Out[101]: (17520,) a = b[0:42] np.in1d(b,a) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/grove/<ipython-input-103-a7d9d12b1353> in <module>() ----> 1 np.in1d(b,a) /usr/local/lib/python2.6/dist-packages/numpy/lib/arraysetops.pyc in in1d(ar1, ar2, assume_unique) 338 # here. The values from the first array should always come before 339 # the values from the second array. --> 340 order = ar.argsort(kind='mergesort') 341 sar = ar[order] 342 equal_adj = (sar[1:] == sar[:-1]) TypeError: requested sort not available for type But this works: a = b[0:41] np.in1d(b,a) Out[105]: array([ True, True, True, ..., False, False, False], dtype=bool) --------- In other words the limit seems to be 41 elements for a. Is this a bug or am I getting something wrong? Grové
26.10.2011 09:09, Grové kirjoitti:
I have picked up a strange limit to np.in1d(): ----------
b Out[100]: array(['2007-01-01T02:30:00+0200', '2007-01-01T03:00:00+0200', '2007-01-01T03:30:00+0200', ..., '2008-01-01T01:00:00+0200', '2008-01-01T01:30:00+0200', '2008-01-01T02:00:00+0200'], dtype='datetime64[s]')
b.shape Out[101]: (17520,)
a = b[0:42]
np.in1d(b,a) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) [clip] In other words the limit seems to be 41 elements for a. Is this a bug or am I getting something wrong?
The problem here seems to be that argsort (or only the mergesort?) for datetime datatypes is not implemented. There's a faster code path that is triggered for small selection arrays, and that does not require argsort, and that's why the error occurs in only some of the cases. -- Pauli Virtanen
Pauli Virtanen <pav <at> iki.fi> writes:
The problem here seems to be that argsort (or only the mergesort?) for datetime datatypes is not implemented.
There's a faster code path that is triggered for small selection arrays, and that does not require argsort, and that's why the error occurs in only some of the cases.
Yes, that makes sense. Anyway, I have just implemented a workaround for now. Thanks. Grové
participants (2)
-
Grové -
Pauli Virtanen