Help with Streaming and Chunk Processing for Large JSON Data (60 GB) from Kenna API

[Grant Edwards]

On 2024-09-30, Left Right via Python-list <python-list at python.org> wrote:
> Whether and to what degree you can stream JSON depends on JSON
> structure. In general, however, JSON cannot be streamed (but commonly
> it can be).
>
> Imagine a pathological case of this shape: 1... <60GB of digits>. This
> is still a valid JSON (it doesn't have any limits on how many digits a
> number can have). And you cannot parse this number in a streaming way
> because in order to do that, you need to start with the least
> significant digit.

Which is how arabic numbers were originally parsed, but when
westerners adopted them from a R->L written language, thet didn't flip
them around to match the L->R written language into which they were
being adopted.

So now long numbers can't be parsed as a stream in software. They
should have anticipated this problem back in the 13th century and
flipped the numbers around.