[capi-sig] What can I do without GIL?

[Stefan Behnel]

Nikolaus Rath, 24.10.2010 18:09:
> On 10/18/2010 02:53 AM, Stefan Behnel wrote:
>> Nikolaus Rath, 18.10.2010 04:02:
>>> I a thread does not hold the GIL, is it totally forbidden to call any
>>> Py* API functions at all, or are there some exceptions?
>>>
>>> More concretely, am I allowed to create fresh Python objects with
>>> PyBytes_FromString et. al. without holding the GIL?
>>
>> No. Anything that requires Python memory management and especially
>> reference counting must be protected by the GIL. You can read the C
>> pointer and length of a Python string without the GIL (using the C-API
>> macros) and you can acquire and release thread locks, but you can't
>> create new objects.
>
> I'd like to follow up with another question directly: if I call
> PyBytes_AsString() on a Python object (and keep a reference to that
> object), can I rely on the char* to stay valid when I am releasing the GIL?

First of all, you shouldn't call PyBytes_AsString() without holding the GIL 
because it may raise an exception. You can call PyBytes_AS_STRING(), 
though. As I said, check the implementation.


> I just learned that in OCAML, the interpreter may happily move data
> around so that pointers are not guaranteed to stay valid if the GIL is
> released.
>
> Is this a problem with Python as well, or can I rely on the string
> staying at the same memory location all the time?

CPython doesn't do that. Strings are immutable, so they never need to be 
reallocated by Python runtime operations. And CPython is tuned for making 
it easy to interact with C code through its C-API. One feature is that the 
memory management doesn't move stuff around that may be referenced by some 
C code somewhere.

Stefan