On Sun, Feb 25, 2024 at 12:34 AM <philippe@loco-labs.io> wrote:
Perhaps, like the Pandas package, it should live outside NumPy for a while until some wider consensus could emerge.
Regarding this initial remark, this is indeed a possible option but it depends on the answer to the question:
- does Numpy want to have a neutral JSON exchange format to exchange data with other frameworks (tabular, multidimensional or other)?
I'd say it's unlikely. There are a lot of data storage formats; NumPy has support for almost none of them, and for the few that we do have support for (e.g. CSV) the reason for having that inside of NumPy is mostly historical. There are packages like Zarr, h5py, PyTables, scipy.io that implement support for reading and writing NumPy arrays in a large number of I/O formats. Typically there is no reason for such code to live inside NumPy. I'd expect the same to be true for JSON. That isn't to say that a new JSON-based storage format wouldn't be of interest to NumPy users - they may very well need it. We do have docs that mention popular I/O formats, and if yours gets popular we may want to add it to those docs: https://numpy.org/devdocs/user/how-to-io.html#write-or-read-large-arrays (that could use more detail too). Cheers, Ralf