[Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite

[Tim Peters]

[Jeremy Hylton]
>>> I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
>>> fixed the problem.

[Tim
>> We don't know why, though, nor do we have a small test case, right?

[Jeremy]
> Not a small test case, no, but I have boiled it down a bit.  If you
> run the tests from a Zope checkout on the Zope-2_7-branch, you can
> provoke the crash with "test.py Template."

Sorry, I cannot.  I tried a fresh Zope-2_7-branch, on Windows, with

    released Python 2.3, release build
    current CVS Python, release build
    current CVS Python, debug build

"test.py Template" gave the same result in all cases:  77 tests run, 1
failure (AssertionError: HTML Output Changed) in checkStringExpressions.

But note that Py_UNICODE is unsigned short on this box, and reading a trash
one of those as an index isn't nearly as likely to reach into unmapped
memory as on a box where you get 4 bytes of trash.

>> ...
>> On second thought, the test could be correct even if it does read up
>> heap trash:  the
>>
>>     unicode_latin1[unicode->str[0]] == unicode
>>
>> part is a very strong condition.  If str[0] does contain heap trash,
>> it can't pass, but it could cause Purify (etc) to whine about reading
>> uninitialized memory.  If we don't care about that,

> It could also cause a segmentation violation depending on the value of
> the stack trash it reads.  At least, that's what appears to be going
> wrong.

Right, I meant the condition is very strong *if* the unicode->str[0] part is
known to lie in range(256).  unicode_latin1 is statically declared to
contain 256 elements, so if the trash index is in range(256) then the index
resolves to legitimate memory.

>>      unicode->length == 1 &&
>>       (unsigned int)unicode->str[0] < 256U &&
>>       unicode_latin1[unicode->str[0]] == unicode

>> would be a safe and correct guard.

> That sounds better.

To worm around something that shouldn't happen at all, sure <wink>.

>> Still, it doesn't look to me like the code *expected* that str could
>> contain uninitialized heap trash at this point, so I'd like someone
>> who thinks they understand this code to think about how that could
>> happen (it apparently is happening, so "beats me" isn't good enough
>> <wink>).

> I certainly don't understand the code yet.

Me neither, but offhand didn't see anything glaringly wrong.  If some
internal Unicode operation decided to allocate a short Unicode string, but
freed it before filling in any of the string bytes, I suppose the keepalive
optimization would retain a Unicode object with uninitialized str space on
the unicode free list.  A subsequent _PyUnicode_New could grab that and try
to boost its ->str size.  That could explain it.