[Python-Dev] PyWeakref_GetObject() borrows its reference from... whom?

[MRAB]

On 2016-10-10 20:36, Chris Angelico wrote:
> On Tue, Oct 11, 2016 at 5:24 AM, Random832 <random832 at fastmail.com> wrote:
>> On Mon, Oct 10, 2016, at 14:04, MRAB wrote:
>>> Instead of locking the object, could we keep the GIL, but have it
>>> normally released?
>>>
>>> A thread could then still call a function such as PyWeakref_GetObject()
>>> that returns a borrowed reference, but only if it's holding the GIL. It
>>> would be able to INCREF the reference before releasing the GIL again.
>>
>> So, what stops the other thread which never asks for the GIL from
>> blowing away the reference? Or is this a special kind of lock that you
>> can "assert isn't locked" without locking it for yourself, and
>> INCREF/DECREF does so?
>
> "assert isn't locked" is pretty cheap (race conditions wouldn't be
> applicable here, as you would have to completely obtain the GIL before
> even attempting a dangerous operation), but what would INCREF/DECREF
> do if the GIL is locked by another thread?
>
> Hmm. Here's a naughty, and maybe dangerous, theory. Obtain a "memory
> deallocation lock". While it is held (by any thread - it's a guard,
> more than a lock), Py_DECREF will not actually deallocate memory -
> objects can fall to zero references without being wiped. Once the
> lock/guard is freed/cleared, anything that had fallen to zero is now
> deallocated. This probably would mean stuffing them onto a list of
> "doomed objects", and upon release of the guard, any doomed objects
> that still have no refs would get deallocated.
>
> That would, by definition, make *all* borrowed refs legal; you can
> either use them and finish, or INCREF them before releasing the
> deallocation lock, and either way, you have a guarantee that (a)
> you're not messing with junk memory, and (b) it really is the object
> you think it is. There should be no risk of race conditions, because
> you would completely claim the deallocation lock before calling
> something that gives you a borrowed ref, meaning that the whole time
> the ref is borrowed, you must have that lock.
>
> But I'm guessing other people have thought far more about this than I have.
>
I have to admit that the GIL thing was just a vague feeling I had, and I 
hadn't thought too deeply about it, but I think that you've crystallised 
it. :-)