[Python-Dev] Const access to CPython objects outside of GIL?
vano at mail.mipt.ru
Tue Jul 17 11:22:00 EDT 2018
On 17.07.2018 7:18, Radim Řehůřek wrote:
> Hi all,
> one of our Python projects calls for pretty heavy, low-level
> We went down the rabbit hole and determined that having access to
> PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict,
> unicode) on Python objects **outside of GIL** might be a good-enough
> solution. The Python objects in question are guaranteed to live and
> not be mutated externally in any way. They're "frozen" and read-only.
The standard practice if you need to access something outside of GIL is
to get a private C object from it.
For immutable types, you can get a pointer to the underlying data if the
internal representation is compatible with some C type (e.g. char*
|PyBytes_AsString|(PyObject/ *o/) and FILE*
|PyFile_AsFile|(PyObject/ *p/) (Py2 only -- in Py3, PyFile no longer
wraps stdio FILE) ); otherwise, the C API can produce a copy (e.g.
wchar_t* |PyUnicode_AsWideCharString|(PyObject/ *unicode/,
Py_ssize_t/ *size/) ).
Though you can call whatever you want outside of GIL (it's not like we
can prevent you), anything that's not officially guaranteed you're doing
on your own risk. Even if it happens to work now, it can break in
unpredictable ways at any point in the future.
> Under what conditions is it OK to call these 3 functions on such objects?
> More generally, what is the CPython 2.7/3.5 contract regarding (lack
> of) object mutation, and the need for reference counting and
> synchronization via GIL?
> Which C API functions are safe to call on "const" objects?
> Obviously releasing GIL and then calling C API is hacky, but from
> initial experiments, it seems to work (see
> But I'm wondering if there's a more formal contract around this behaviour.
> Python-Dev mailing list
> Python-Dev at python.org
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev