multiple JSON documents in one file, change proposal

Akkana Peck akkana at shallowsky.com
Sat Dec 1 21:09:33 EST 2018


Grant Edwards <grant.b.edwards at gmail.com> writes:
> This is what "archive" file formats are for.  Just use tar, zip, ar,
> cpio, or some other file format designed to store multiple "files" of
> arbitrary data -- there are plenty of "standard" formats to choose
> from and plenty of libraries to deal with them.

Then the data files wouldn't be human readable, making debugging
a lot more hassle.

Cameron Simpson writes:
> There's a common format called Newline Delimited JSON (NDJSON) for just this
> need.
> 
> Just format the outbound records as JSON with no newlines (i.e. make the
> separator a space or the empty string), put a newline at the end of each.
> 
> On ingest, read lines of text, and JSON parse each line separately.

I'll second that. It's very easy to deal with. A shorter name for it
is JSONL -- I use .jsonl for filenames.
https://en.wikipedia.org/wiki/JSON_streaming#Line-delimited_JSON
discusses that as well as several other solutions.

        ...Akkana



More information about the Python-list mailing list