[Python-Dev] Activating pymalloc

[Tim Peters]

[Tim]
> Martin, you were concerned earlier about the possibilities for tricking
> pymalloc into crashing the system, given that its flavor of free() takes
> a careful but still probabilistic guess about who (pymalloc or "the
> system") owns the memory being returned.  Are you no longer concerned
> about that?

[Martin v. Loewis]
> You convinced me last time that this won't be an issue. I think you
> even offered some gift in case somebody runs into the problem in real
> life, although I forgot what that gift was :-)

Na, I passed along *Vladimir's* offer to buy us lunch.  It's very curious
that he vanished from Python Life soon after making that offer <wink>.

I agree with him that the odds of "an accident" are vanishingly small,
provided that pymalloc continues never returning areans to the system (if it
starts to return them, it's trickier).  The practical question is more
whether a knowledgeable hostile user could *provoke* pymalloc into erring
here; but competing with hostile users is outside my personal definition of
"practical", so I haven't thought about that since raising the question.

> I think some action needs to be taken: either the code needs to be
> removed from Python for that reason, or it should be activated. I
> doubt that the problem (if there is a problem) will get addressed from
> the code just sitting in the CVS.

This is demonstrably true.

> Trusting that the problem is *not* real, I favour activating the code.
> It still remains an option to turn it off should somebody be worried.

Sounds good to me.  There are other things I'd like to do then more than I'd
like to play Security Geek:

+ Boost the limit for the max request pymalloc handles on its own.

+ Add debug-build checks similar to the Microsoft debug malloc/free
  checks.

+ Add realloc variants that specifically ask to, or ask not to,
  copy to a smaller block if the resize is a shrinking one (one size
  can't fit all here, and Python uses realloc a lot).