[Python-Dev] pymalloc killer

[Tim Peters]

[Guido]
> How about if the PyMem_Free guy saved the address of the vector before
> using it, and checked that it was still the same afterwards, *and* if
> the PyMem_Malloc guy didn't use realloc to resize the vector but
> copied it to a newly malloc'ed vector, stored the new vector's
> address, and then freed the old vector?

It already does the latter (it deliberately avoided realloc all along -- an
even stronger reason to avoid realloc is that if the realloc failed, we'd
lose the original vector of base addresses entirely and so would have to
kill the program; as is, it can continue gracefully).

> (I guess this is the same as what Martin suggested.

He suggested letting the old vector *leak*, which is the same idea God gave
me in my sleep <wink>.

> His point that the global holding the vector address should be declared
> volatile is a good one.)

I see no point to that.  If someone can explain *why* it's a good idea,
sure, but I don't see it.

Let's simplify things enormously:  p is a pointer to "something".  Thread A
contains

    *p

Thread B contains

    free(p)

Without a lock, they're screwed period.  Thread A has to first read up p,
and then dereference it, and all the volatiles in the universe can't stop
free(p) from happening *between* thread A's two machine instructions.  It
doesn't matter either if Thread B does

    oldp = p;
    p = newp;
    free(oldp);

because thread A can still pick up "p" before the "p = newp" line.  Indeed,
the current obmalloc code is exactly like that, and it's not safe (although
I bet if I left it alone, the hole wouldn't be hit in my lifetime!).

Neither would it do any real good for Thread A to do

    if (p == p && p == p && p == p && wink)
        ... *p ...

any number of times, volatile or not.  Nothing short of a pair of critical
sections is a genuine cure for a genuine race.

Letting the old vector leak is fine; in most programs, it won't lose more
than about 1KB of memory total.  Note that there are obscure reasons for why
it's OK to reference a stale vector in the cases where a race is possible
(this is already explained in obmalloc.c comments, so I won't repeat that
here).