Hi Alex, 2017-02-22 12:45 GMT+01:00 Alex Rogozhnikov <alex.rogozhnikov@yandex.ru>:
Hi Nathaniel,
pandas
yup, the idea was to have minimal pandas.DataFrame-like storage (which I was using for a long time), but without irritating problems with its row indexing and some other problems like interaction with matplotlib.
A dict of arrays?
that's what I've started from and implemented, but at some point I decided that I'm reinventing the wheel and numpy has something already. In principle, I can ignore this 'column-oriented' storage requirement, but potentially it may turn out to be quite slow-ish if dtype's size is large.
Suggestions are welcome.
You may want to try bcolz: https://github.com/Blosc/bcolz bcolz is a columnar storage, basically as you require, but data is compressed by default even when stored in-memory (although you can disable compression if you want to).
Another strange question: in general, it is considered that once numpy.array is created, it's shape not changed. But if i want to keep the same recarray and change it's dtype and/or shape, is there a way to do this?
You can change shapes of numpy arrays, but that usually involves copies of the whole container. With bcolz you can change length and add/del columns without copies. If your containers are large, it is better to inform bcolz on its final estimated size. See: http://bcolz.blosc.org/en/latest/opt-tips.html Francesc
Thanks, Alex.
22 февр. 2017 г., в 3:53, Nathaniel Smith <njs@pobox.com> написал(а):
On Feb 21, 2017 3:24 PM, "Alex Rogozhnikov" <alex.rogozhnikov@yandex.ru> wrote:
Ah, got it. Thanks, Chris! I thought recarray can be only one-dimensional (like tables with named columns).
Maybe it's better to ask directly what I was looking for: something that works like a table with named columns (but no labelling for rows), and keeps data (of different dtypes) in a column-by-column way (and this is numpy, not pandas).
Is there such a magic thing?
Well, that's what pandas is for...
A dict of arrays?
-n _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
-- Francesc Alted