[issue7358] cStringIO not 64-bit safe

[Antoine Pitrou]

Antoine Pitrou <pitrou at free.fr> added the comment:

The issue looks quite clear: cStringIO.write() asserts that the required storage size is less than INT_MAX. Therefore, in all likelihood, the pickle dump is simply larger than 2GB.

Now, the cStringIO structures seem 64-bit safe, so the comparison to INT_MAX seems useless. Also, in non-debug builds, the assert is skipped but the Py_ssize_t values are first cast to int before being saved into a Py_ssize_t struct member! Which would lead to data corruption and/or crashes.

The INT_MAX asserts were added in r42382 which also made cStringIO internals 64-bit safe. Martin, do you remember why you did this?

The asserts don't even protect against the following crash:

$ ./python -c "from cStringIO import StringIO; b=b'x'*(2**31+1); s=StringIO(); s.write(b)"
Erreur de segmentation

gdb shows the following stack excerpt:

#0  0x00007ffff724e8c2 in memcpy () from /lib64/libc.so.6
#1  0x00007ffff6b47a46 in O_cwrite (self=<unknown at remote 0xa3f790>, c=0x7fff76b44054 'x' <repeats 200 times>..., 
    l=-2147483647) at /home/antoine/cpython/27/Modules/cStringIO.c:415


Now, onto the problem of reproducing, here's another interesting thing: while some internal structures of cStringIO are 64-bit safe, the Python-facing API isn't:

>>> from cStringIO import StringIO
>>> b = b"x" * (2**32+1)
>>> s = StringIO()
>>> s.write(b)
>>> s.tell()
1

(the module doesn't use Py_SSIZE_T_CLEAN)

----------
nosy: +loewis, pitrou
title: cPickle crash on failed assertion -> cStringIO not 64-bit safe
versions: +Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7358>
_______________________________________