Re: [pypy-dev] Memory leak in 7.3.4
Thanks Carl. Appreciate the feedback and suggestions. We will try this out. Brad. Hi Brad, the new json decoder caches values, but not across json.load calls. ie if you don't process a single gigantic message but many tiny messages, the caching of (string) values cannot be the problem. what *could* be the problem is the key caching though, in theory. The key cache is partially persisted across calls. Are there arbitrarily many different keys in your messages? The key cache shouldn't grow without bounds, however. what do you set your max heap size to? To debug further, I would first try to see whether it is indeed the json module or something else. Maybe you could try another json decoder and see whether the problem persists with that? Eg ujson works on PyPy (slowly) or you could even use the pure python builtin one that you get by importing json.decoder.JSONDecoder. Cheers, Carl Friedrich
participants (1)
-
Brad Kish