[Numpy-discussion] datetime dtype possible regression

[Wes McKinney]

On Sat, Apr 28, 2012 at 11:18 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Sat, Apr 28, 2012 at 9:13 AM, Wes McKinney <wesmckinn at gmail.com> wrote:
>>
>> On Fri, Apr 27, 2012 at 4:57 PM, Robert Kern <robert.kern at gmail.com>
>> wrote:
>> > On Fri, Apr 27, 2012 at 21:52, Travis Vaught <travis at vaught.net> wrote:
>> >> With NumPy 1.6.1 (from EPD 7.2-2) I get this behavior:
>> >>
>> >>
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >> In [1]: import numpy as np
>> >>
>> >> In [2]: schema = np.dtype({'names':['symbol', 'date', 'open', 'high',
>> >> 'low',
>> >>    ...:                        'close', 'volume', 'adjclose'],
>> >>    ...:                    'formats':['S8', 'M8', float, float, float,
>> >> float,
>> >>    ...:                        float, float]})
>> >>
>> >> In [3]: data = [("AAPL", "2012-04-12", 600.0, 605.0, 598.0, 602.0,
>> >> 50000000,
>> >> 602.0),]
>> >>
>> >> In [4]: recdata = np.array(data, dtype=schema)
>> >>
>> >> In [5]: recdata
>> >> Out[5]:
>> >> array([ ('AAPL', datetime.datetime(2012, 4, 12, 0, 0), 600.0, 605.0,
>> >> 598.0,
>> >> 602.0, 50000000.0, 602.0)],
>> >>       dtype=[('symbol', '|S8'), ('date', ('<M8[us]', {})), ('open',
>> >> '<f8'),
>> >> ('high', '<f8'), ('low', '<f8'), ('close', '<f8'), ('volume', '<f8'),
>> >> ('adjclose', '<f8')])
>> >>
>> >>
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >>
>> >> With numpy-1.7.0.dev_3cb783e I get this:
>> >>
>> >>>>> import numpy as np
>> >>
>> >>>>> schema =
>> >>>>>
>> >>>>> np.dtype({'names':['symbol','data','open','high','low','close','volume','adjclose'],
>> >>>>> 'formats':['S8','M8',float,float,float,float,float,float]})
>> >>
>> >>>>> data =  [("AAPL", "2012-04-12", 600.0, 605.0, 598.0, 602.0,
>> >>>>> 50000000,
>> >>>>> 602.0),]
>> >>
>> >>>>> recdata = np.array(data, dtype=schema)
>> >> Traceback (most recent call last):
>> >>   File "<stdin>", line 1, in <module>
>> >> ValueError: Cannot create a NumPy datetime other than NaT with generic
>> >> units
>> >>
>> >>>>> np.version.version
>> >> '1.7.0.dev-3cb783e'
>> >>
>> >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> >>
>> >> Any hints about a regression I can check for? Or perhaps I missed an
>> >> api
>> >> change for specifying datetime dtypes?
>> >
>> > Judging from the error message, it looks like an intentional API change.
>> >
>> > --
>> > Robert Kern
>> > _______________________________________________
>> > NumPy-Discussion mailing list
>> > NumPy-Discussion at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> Maybe this should be raised as a bug (where do we report NumPy bugs
>> these days, still Trac?). As I'm moving to datetime64 in pandas if
>> NumPy 1.6.1 data has unpickling issues on NumPy 1.7+ it's going to be
>> very problematic.
>
>
> I was wondering what datetime you were using since the version in 1.6 had
> issues. Have you tested with both?
>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

Could you define issues? I haven't had a chance to make the library
compatible with both 1.6.1 and 1.7.0 yet (like Travis I'm using NumPy
1.6.1 from EPD); it's important though as pandas will be the first
widely used library I know of that will make heavy use of datetime64.

- Wes