I sure like the idea of this. I agree that this should be external to numpy. At least until it becomes a standard in a sense that json itself is. And that n-dimensional array should ideally be extended to indexed structures with named dimensions and attributes. To accommodate a superset of: xarray & scipp. Regards, DG
On 28 Feb 2024, at 08:34, fangqq--- via NumPy-Discussion <numpy-discussion@python.org> wrote:
aside from the previously mentioned ticket https://github.com/numpy/numpy/issues/12481, I also made a similar proposal, posted in 2021
https://github.com/numpy/numpy/issues/20461 https://mail.python.org/archives/list/numpy-discussion@python.org/message/EV...
lightweight JSON annotations for various data structures (trees, tables, graphs), especially ND-arrays, are defined in the JData spec
https://github.com/NeuroJSON/jdata/blob/master/JData_specification.md#data-a...
JSON/binary JSON annotation encoders/decoders have been implemented for Python (https://pypi.org/project/jdata/), MATLAB/Octave (https://github.com/fangq/jsonlab), JavaScript/NodeJS (https://www.npmjs.com/package/jda), as well as C++ (JSON for Modern C++, https://json.nlohmann.me/features/binary_formats/bjdata/)
I have been extensively used this annotation in JSON/binary JSON in my neuroimaging data portal, https://neurojson.io/, for example, for 3D data
https://neurojson.org/db/fieldtrip(atlas)/FieldTrip--Brainnetome--Brainnetom... https://neurojson.org/db/fieldtrip(atlas)/FieldTrip--Brainnetome--Brainnetom...
for mesh data
https://neurojson.org/db/brainmeshlibrary/BrainWeb--Subject04--gm--surf https://neurojson.org/db/brainmeshlibrary/BrainWeb--Subject04--gm--surf#prev...
the ND array supports binary data with loss-less compression. I've also implemented
in a renewed thread posted in 2022, I also tested the blosc2 (https://www.blosc.org/) compression codecs and got excellent read/write speed
https://mail.python.org/archives/list/numpy-discussion@python.org/thread/JIT... https://mail.python.org/archives/list/numpy-discussion@python.org/message/TU...
the blosc2 compression codes are supported in my python and matlab/C parsers.
Qianqian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-leave@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: dom.grigonis@gmail.com