[pypy-dev] Memory leak in 7.3.4

[Brad Kish]

Thanks Carl. Appreciate the feedback and suggestions. We will try this out.

Brad.

Hi Brad,

the new json decoder caches values, but not across json.load calls. ie
if you don't process a single gigantic message but many tiny messages,
the caching of (string) values cannot be the problem.

what *could* be the problem is the key caching though, in theory. The
key cache is partially persisted across calls. Are there arbitrarily
many different keys in your messages? The key cache shouldn't grow
without bounds, however.

what do you set your max heap size to?

To debug further, I would first try to see whether it is indeed the json
module or something else. Maybe you could try another json decoder and
see whether the problem persists with that? Eg ujson works on PyPy
(slowly) or you could even use the pure python builtin one that you get
by importing json.decoder.JSONDecoder.

Cheers,

Carl Friedrich



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pypy-dev/attachments/20210531/7b6e1773/attachment.html>