Nikolaus Rath, 24.10.2010 18:09:
On 10/18/2010 02:53 AM, Stefan Behnel wrote:
Nikolaus Rath, 18.10.2010 04:02:
I a thread does not hold the GIL, is it totally forbidden to call any Py* API functions at all, or are there some exceptions?
More concretely, am I allowed to create fresh Python objects with PyBytes_FromString et. al. without holding the GIL?
No. Anything that requires Python memory management and especially reference counting must be protected by the GIL. You can read the C pointer and length of a Python string without the GIL (using the C-API macros) and you can acquire and release thread locks, but you can't create new objects.
I'd like to follow up with another question directly: if I call PyBytes_AsString() on a Python object (and keep a reference to that object), can I rely on the char* to stay valid when I am releasing the GIL?
First of all, you shouldn't call PyBytes_AsString() without holding the GIL because it may raise an exception. You can call PyBytes_AS_STRING(), though. As I said, check the implementation.
I just learned that in OCAML, the interpreter may happily move data around so that pointers are not guaranteed to stay valid if the GIL is released.
Is this a problem with Python as well, or can I rely on the string staying at the same memory location all the time?
CPython doesn't do that. Strings are immutable, so they never need to be reallocated by Python runtime operations. And CPython is tuned for making it easy to interact with C code through its C-API. One feature is that the memory management doesn't move stuff around that may be referenced by some C code somewhere.
Stefan