On Fri, Apr 17, 2020 at 1:39 PM Victor Stinner <vstinner@python.org> wrote:
Sharing directly singletons like None can become a performance kill once subinterpreters will run in parallel: https://bugs.python.org/issue39511
Mark Shannon summarized: "Having two CPUs write to the same cache line is a well known performance problem. There's nothing special about CPython here. The proper name for it seems to be "cache line ping-pong", but a search for "false sharing" might be more informative."
The problem is that PyObject.ob_refcnt should be protected by atomic operations or locks if these objects are shared directly.
I agree. Sharing objects between subinterpreters is a recipe for pain. That's why the PEP says we will not. :)
I proposed to have singletons per interpreter, some people would prefer immortal singletons.
Yeah, we definitely have to sort that out before we make the GIL per-interpreter.
For bytes or buffer objects, I understand that you propose to share the exact same PyObject objects between interpreters, at least in the first implementation.
It may be better to have one proxy object in each interpreter which would control which interpreters can read and which interpreters can write into the object. It's a similar but more complex issue than singletons.
It isn't the same Python object. It is the buffer that gets shared (for objects that support the buffer protocol). However, I suppose that it will have a similar problem to what you described above, since it is the same (read-only) memory in both interpreters. I'm not too worried about that at the moment though, particularly if we make the module provisional. -eric