JSON-encoding very long iterators
alfred at 54.org
alfred at 54.org
Mon Sep 29 21:19:53 EDT 2014
I would like to add the ability to JSONEncode large iterators. Right now there is no way to do this without modifying the code.
The JSONEncoder.default() doc string suggests to do this:
For example, to support arbitrary iterators, you could
implement default like this::
def default(self, o):
try:
iterable = iter(o)
except TypeError:
pass
else:
return list(iterable)
# Let the base class default method raise the TypeError
return JSONEncoder.default(self, o)
but this method requires the whole serialized object to fit in memory and it's a good chance that your iterator is an iterator to save on memory in the first place.
By changing the code to accept iterators it is then possible to stream json as I did here:
http://stackoverflow.com/a/26094558/289240
This would ideal if it were included in the standard library. Is there any reason why it shouldn't be?
More information about the Python-list
mailing list