[capi-sig] What can I do without GIL?

[Nikolaus Rath]

On 10/24/2010 12:57 PM, Stefan Behnel wrote:
> Nikolaus Rath, 24.10.2010 18:09:
>> On 10/18/2010 02:53 AM, Stefan Behnel wrote:
>>> Nikolaus Rath, 18.10.2010 04:02:
>>>> I a thread does not hold the GIL, is it totally forbidden to call any
>>>> Py* API functions at all, or are there some exceptions?
>>>>
>>>> More concretely, am I allowed to create fresh Python objects with
>>>> PyBytes_FromString et. al. without holding the GIL?
>>>
>>> No. Anything that requires Python memory management and especially
>>> reference counting must be protected by the GIL. You can read the C
>>> pointer and length of a Python string without the GIL (using the C-API
>>> macros) and you can acquire and release thread locks, but you can't
>>> create new objects.
>>
>> I'd like to follow up with another question directly: if I call
>> PyBytes_AsString() on a Python object (and keep a reference to that
>> object), can I rely on the char* to stay valid when I am releasing the
>> GIL?
> 
> First of all, you shouldn't call PyBytes_AsString() without holding the
> GIL because it may raise an exception. You can call PyBytes_AS_STRING(),
> though. As I said, check the implementation.

That's what I meant. I want to call PyBytes_* and *afterwards* release
the GIL.

>> I just learned that in OCAML, the interpreter may happily move data
>> around so that pointers are not guaranteed to stay valid if the GIL is
>> released.
>>
>> Is this a problem with Python as well, or can I rely on the string
>> staying at the same memory location all the time?
> 
> CPython doesn't do that. Strings are immutable, so they never need to be
> reallocated by Python runtime operations. And CPython is tuned for
> making it easy to interact with C code through its C-API. One feature is
> that the memory management doesn't move stuff around that may be
> referenced by some C code somewhere.

Great, thanks again!



   -Nikolaus

-- 
 »Time flies like an arrow, fruit flies like a Banana.«

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C