[Python-Dev] Const access to CPython objects outside of GIL?
vstinner at redhat.com
Tue Jul 17 07:34:27 EDT 2018
2018-07-17 6:18 GMT+02:00 Radim Řehůřek <radim at rare-technologies.com>:
> one of our Python projects calls for pretty heavy, low-level optimizations.
> We went down the rabbit hole and determined that having access to
> PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict, unicode)
> on Python objects **outside of GIL** might be a good-enough solution. The
> Python objects in question are guaranteed to live and not be mutated
> externally in any way. They're "frozen" and read-only.
IMHO it's a great path to introduce very tricky race conditions in
multithreaded applications! :-)
At the C level, even immutable Python objects are mutable: str, tuple, etc.
IMHO you need a different approach to implement optimizations. For
example, use your objects which don't rely on the GIL to be
consistent. Sadly, I have no experience with that and so cannot
provide any example.
Python releases the GIL *often* and pass pointers to buffers to C
functions. For example, os.stat() writes into a memory block allocated
by Python. But it's a raw memory block, it's much simpler than a
Python object like a tuple.
More information about the Python-Dev