[Numpy-discussion] numpy scalars and savez -- bug?

[Chris Barker - NOAA Federal]

On Fri, Apr 19, 2013 at 10:21 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On Fri, Apr 19, 2013 at 8:45 PM, Chris Barker - NOAA Federal
> <chris.barker at noaa.gov> wrote:

>> Given that numpy scalars do exist, and have their uses -- I found this
>> wiki page to remind me:
>>
>> http://projects.scipy.org/numpy/wiki/ZeroRankArray
>>
>> It would be nice if the .npy format could support them. Would that be
>> a major change? I'm trying to decide if this bugs me enough to work on
>> that.
>
> I think that is significant scope creep for the .npy format, and I
> would like to avoid it.

hmm -- maybe it's more work that we want, but it seems to me that
numpy scalars are part and parcel of numpy -- so it makes sense for
.npy to save them.

> A case might be made for letting np.savez()
> simply pickle non-arrays that it is given. I have a vague recollection
> that that was discussed when savez() was designed and rejected as a
> moral hazard, but I could be wrong.

That could be a nice solution -- I'm not _so_ worried about moral hazards!

> The .npy and .npz formats are
> intentionally limited by design. As soon as you feel constrained by
> those limitations, you should start using more full-fledged and
> standard file formats.

well, maybe -- in this case, I'm using it to cache a bunch  of data on
disk. The data are all in a dict of numpy arrays, so it was really
natural and easy (and I presume fast) to use npz. All I want to is
dump it to disk, and get back the same way. It worked great.

Then I needed a datetime stored with it all -- so I figured a
datetime64 scalar would be perfect. It's not a huge deal to use a
rank-zero array instead, but it would have been nicer to be able to
store a scalar (I suppose one trick may be that there are numpy
scalars, and there are regular old pyton scalars...)

Anyway -- going to HDF, or netcdf, or role-your-own really seems like
overkill for this. I just need something fast and simple and it
doesn't need to interchange with anything else.

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov