[Numpy-discussion] segfaulting numpy with dot and datetime

[josef.pktd at gmail.com]

On Mon, Dec 10, 2012 at 9:46 PM,  <josef.pktd at gmail.com> wrote:
> On Mon, Dec 10, 2012 at 8:54 PM,  <josef.pktd at gmail.com> wrote:
>> On Mon, Dec 10, 2012 at 8:26 PM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>>>
>>> On Mon, Dec 10, 2012 at 5:39 PM, <josef.pktd at gmail.com> wrote:
>>>>
>>>> >>> np.__version__
>>>> '1.6.2'
>>>> >>> aa
>>>> array([1970-01-13 96:00:00, 1970-01-13 120:00:00, 1970-01-13 144:00:00,
>>>>        1970-01-13 168:00:00, 1970-01-13 192:00:00], dtype=datetime64[ns])
>>>> >>> np.dot(aa, [1])
>>>> <segfault>
>>>>
>>>
>>> Hmm, I can't even get that array using current master, what with illegal
>>> hours and all. Datetime in 1.6.x wasn't quite up to snuff, so things might
>>> have been fixed in 1.7.0. Is there any way you can check that.
>>
>> I didn't know the dates are illegal, they were created with pandas.
>
> here's the minimal numpy version
>
>>>> np.ones(5, 'datetime64[ns]')
> array([1970-01-01 00:00:00, 1970-01-01 00:00:00, 1970-01-01 00:00:00,
>        1970-01-01 00:00:00, 1970-01-01 00:00:00], dtype=datetime64[ns])
>>>> b = np.ones(5, 'datetime64[ns]')
>>>> np.dot(b, [1])
> <segfault>
>
> virtualenv is next

much better

(py27d) E:\Josef\testing\tox\py27d\Scripts>python
Python 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__version__
'1.7.0b2'
>>> b = np.ones(5, 'datetime64[ns]')
>>> b
array(['1969-12-31T19:00:00.000000001-0500',
       '1969-12-31T19:00:00.000000001-0500',
       '1969-12-31T19:00:00.000000001-0500',
       '1969-12-31T19:00:00.000000001-0500',
       '1969-12-31T19:00:00.000000001-0500'], dtype='datetime64[ns]')
>>> np.dot(b, [1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Cannot cast array data from dtype('<M8[ns]') to
dtype('int32') according to the rule 'safe'


I get the same casting exception instead of the segfault when I run
the original script with pandas and statsmodels.

All clear with 1.7.0b2,
we just need to find a way to avoid user errors that cause segfaults
with numpy 1.6.2


---------
other issue:

Does using linalg with datetime64 make sense?
Implicit casting to float without warning

Isn't there an inconsistency between dot and linalg ?

(I don't know since I didn't pay much attention to the details of datetime.)

>>> b = np.arange(10).astype('datetime64[ns]').reshape(2,5)
>>> b
array([['1969-12-31T19:00:00.000000000-0500',
        '1969-12-31T19:00:00.000000001-0500',
        '1969-12-31T19:00:00.000000002-0500',
        '1969-12-31T19:00:00.000000003-0500',
        '1969-12-31T19:00:00.000000004-0500'],
       ['1969-12-31T19:00:00.000000005-0500',
        '1969-12-31T19:00:00.000000006-0500',
        '1969-12-31T19:00:00.000000007-0500',
        '1969-12-31T19:00:00.000000008-0500',
        '1969-12-31T19:00:00.000000009-0500']], dtype='datetime64[ns]')

>>> np.linalg.pinv(b)
array([[ -3.20000000e-01,   1.20000000e-01],
       [ -1.80000000e-01,   8.00000000e-02],
       [ -4.00000000e-02,   4.00000000e-02],
       [  1.00000000e-01,  -1.73472348e-17],
       [  2.40000000e-01,  -4.00000000e-02]])

>>> np.linalg.qr(b)
(array([[ 0., -1.],
       [-1.,  0.]]), array([[-5., -6., -7., -8., -9.],
       [ 0., -1., -2., -3., -4.]]))

>>> c
array([['2003-09-28T20:00:00.000000000-0400',
        '2003-09-29T20:00:00.000000000-0400',
        '2003-09-30T20:00:00.000000000-0400',
        '2003-10-01T20:00:00.000000000-0400',
        '2003-10-02T20:00:00.000000000-0400'],
       ['2003-10-05T20:00:00.000000000-0400',
        '2003-10-06T20:00:00.000000000-0400',
        '2003-10-07T20:00:00.000000000-0400',
        '2003-10-08T20:00:00.000000000-0400',
        '2003-10-09T20:00:00.000000000-0400']], dtype='datetime64[ns]')
>>> np.linalg.pinv(c)
array([[ -4.07870371e-12,   4.07638889e-12],
       [ -2.03951719e-12,   2.03835978e-12],
       [ -3.30684814e-16,   3.30684816e-16],
       [  2.03885582e-12,  -2.03769841e-12],
       [  4.07804232e-12,  -4.07572751e-12]])

Josef

>
> Josef
>
>>
>> Skipper said he couldn't replicate the segfault so it might be gone
>> with a more recent numpy.
>> I still have to setup a virtualenv for a 1.7.0 beta so I can start to test it.
>> (I rely on binaries for numpy and scipy now.)
>>
>> Thanks,
>>
>> Josef
>>
>>>
>>> Chuck
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>