[Python-ideas] iterator/stream-based JSON API

[Daniel Holth]

On Mon, Jun 3, 2013 at 11:43 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 3 Jun 2013 09:01:43 -0400
> Daniel Holth <dholth at gmail.com> wrote:
>>
>> The ijson API yields a stream of events containing the full path to
>> each item in the parsed JSON, an event name like "start_map",
>> "end_map", "start_array", ...
>>
>> list(ijson.parse(StringIO.StringIO("""{ "a": { "b": "c" } }""")))
>>
>> [('', 'start_map', None),
>>  ('', 'map_key', 'a'),
>>  ('a', 'start_map', None),
>>  ('a', 'map_key', 'b'),
>>  ('a.b', 'string', u'c'),
>>  ('a', 'end_map', None),
>>  ('', 'end_map', None)]
>>
>> It also has a higher-level API yielding only the objects under a
>> certain prefix. Pass "a.b" and you would get only "c".
>>
>> Besides memory this kind of thing makes it much easier to know which
>> level of the JSON structure you are in compared to the existing
>> object_pairs hook.
>
> But did you encounter a use case where the existing API didn't fit the
> bill?

Sometimes it's nice to have a stream-based API; when your memory is
very small, your JSON is very large, or your JSON may be very large,
or the JSON is being streamed to you little by little and you want to
parse part of it, or just don't want to wait for that closing }. It's
just a different way to parse than the current all-at-once option.