Hi Rob, I only looked briefly at your example so far but it seems to be dominated by one big string. Therefore you should definitely try PyPy 7.1. since that improves the size of strings dramatically. Cheers, Carl Friedrich On August 22, 2019 1:34:11 PM GMT+02:00, Robert Whitcher <robert.whitcher@rubrik.com> wrote:

Shared test file with Carl (to avoid posting to everyones inbox). PyPy version is currently 6.0 (I don't have ability to affect a change here unless I can prove something)

On Thu, Aug 22, 2019 at 1:00 AM Carl Friedrich Bolz-Tereick <cfbolz@gmx.de> wrote:

...

Hi Rob,

Which version of PyPy are you running this with? I have a long running branch that I really should merge someday that is supposed to help with memory consumption of json deserialization. Is there a chance you could share a (anonymized) version of your test file?

Alternatively, you could try a nightly build from this branch yourself:

http://buildbot.pypy.org/nightly/json-decoder-maps-py3.6/

Carl Friedrich

On August 22, 2019 1:02:42 AM GMT+02:00, Robert Whitcher < robert.whitcher@rubrik.com> wrote:

...

Hi, I am running a very simple test case (as we are hitting OOM on our

larger

...

PyPy deployments) and I'd love some help understanding what is happening here.... We have a lot of processes that send messages to each other. These can be large JSON serializations of objects. But the memory being consumed seems out of order and hard to manage across processes.

I have this loop running:

import time import json

def main(): with open("/tmp/test12334.1234", "r") as f: json_msg = f.read()

while True: j = json.loads(json_msg) time.sleep(10)

if __name__ == "__main__": main()

I have tried 3 separate general runs across both pypy and cpython. The first does nothing but the sleep (it doesn't load or json the message) The second just loaded the json_str from a file The third is the full loop.

If I run this in cpython I get (80MB, 92MB and 136MB) respectively This makes sense as the file is 11MB json serialization of a dictionary and json.loads takes up some memory

However if I run this in pypy I get 120MB, 153MB and between 360-405MB when it settles out. I get the JIT and startup memory being higher, spending a little more loading the string but WOW does json loading the string chew up a bunch.

Multiplying that memory across processes is eating a bunch.

What easy things am I missing?

Thanks, Rob

-- [image: photo] Robert Whitcher Member of Technical Staff at Rubrik M 512-633-1771 <512-633-1771> E robert.whitcher@rubrik.com <robert.whitcher@rubrik.com> W www.rubrik.com <http://www.rubrik.com?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>

Hi Rob, I only looked briefly at your example so far but it seems to be dominated by one big string. Therefore you should definitely try PyPy 7.1. since that improves the size of strings dramatically. Cheers, Carl Friedrich On August 22, 2019 1:34:11 PM GMT+02:00, Robert Whitcher <robert.whitcher@rubrik.com> wrote:

Shared test file with Carl (to avoid posting to everyones inbox). PyPy version is currently 6.0 (I don't have ability to affect a change here unless I can prove something)

On Thu, Aug 22, 2019 at 1:00 AM Carl Friedrich Bolz-Tereick <cfbolz@gmx.de> wrote:

...

Hi Rob,

Which version of PyPy are you running this with? I have a long running branch that I really should merge someday that is supposed to help with memory consumption of json deserialization. Is there a chance you could share a (anonymized) version of your test file?

Alternatively, you could try a nightly build from this branch yourself:

http://buildbot.pypy.org/nightly/json-decoder-maps-py3.6/

Carl Friedrich

On August 22, 2019 1:02:42 AM GMT+02:00, Robert Whitcher < robert.whitcher@rubrik.com> wrote:

...

Hi, I am running a very simple test case (as we are hitting OOM on our

larger

...

PyPy deployments) and I'd love some help understanding what is happening here.... We have a lot of processes that send messages to each other. These can be large JSON serializations of objects. But the memory being consumed seems out of order and hard to manage across processes.

I have this loop running:

import time import json

def main(): with open("/tmp/test12334.1234", "r") as f: json_msg = f.read()

while True: j = json.loads(json_msg) time.sleep(10)

if __name__ == "__main__": main()

I have tried 3 separate general runs across both pypy and cpython. The first does nothing but the sleep (it doesn't load or json the message) The second just loaded the json_str from a file The third is the full loop.

If I run this in cpython I get (80MB, 92MB and 136MB) respectively This makes sense as the file is 11MB json serialization of a dictionary and json.loads takes up some memory

However if I run this in pypy I get 120MB, 153MB and between 360-405MB when it settles out. I get the JIT and startup memory being higher, spending a little more loading the string but WOW does json loading the string chew up a bunch.

Multiplying that memory across processes is eating a bunch.

What easy things am I missing?

Thanks, Rob

-- [image: photo] Robert Whitcher Member of Technical Staff at Rubrik M 512-633-1771 <512-633-1771> E robert.whitcher@rubrik.com <robert.whitcher@rubrik.com> W www.rubrik.com <http://www.rubrik.com?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>