[issue9602] PyObject_AsCharBuffer() should only accept read-only objects

[Marc-Andre Lemburg]

Marc-Andre Lemburg <mal at egenix.com> added the comment:

STINNER Victor wrote:
> 
> New submission from STINNER Victor <victor.stinner at haypocalc.com>:
> 
> mmap, buffer, bytearray, string and unicode objects set the char buffer callback (bf_getcharbuffer). The bytearray object sets also the release buffer callback (bf_releasebuffer).
> 
> In Python 2.7, PyObject_AsCharBuffer() accepts bytearray objects, whereas the "t#" format of PyArg_Parse functions rejects byte bytearray objects (expect an "pinned buffer object").
> 
> In Python 3.2, PyObject_AsCharBuffer() releases the buffer.
> 
> PyObject_AsCharBuffer() documentation (in 2.7 and 3.2) says that the function only accepts read-only objects.
> 
> Something is wrong here. If the caller doesn't hold a kind of lock, the object cannot be protected against futher modifications. The caller has to ensure that the object is not modifiable until it finishs to use the char* pointer.
> 
> I think that it would be safer to respect the documentation: PyObject_AsCharBuffer() should only accept read-only objects. The most important change is that functions using PyObject_AsCharBuffer() will not accept bytearray objects anymore.
> 
> Attached patch (for Python 2.7) changes PyObject_AsCharBuffer() to reject modifiable objects. It removes also the character buffer callback from the bytearray type. To avoid breaking compatibility too much, I patched int(), long() and float() to still support bytearray objects.
> 
> Examples of functions rejecting bytearray with the patch:
>  - int(), long(), float(), complex()
>  - many str methods: split, partition, rpartition, rsplit, index, find, count, translate, replace, startswith, endswith
>  - writelines() of file objects (eg. sys.stdout.writelines)
>  - writelines() method of a bz2 file
> 
> --
> 
> My patch breaks backward compatibility, and I don't know that it is acceptable in Python 2.7.

Simple answer: no.

Long answer: The caller is responsible for making sure that the object
is not modified while in use.

> I will write a similar patch for Python 3.2.

Restricting the API to read-only buffers would seriously limit
it's functionality. I'm -1 on doing that.

Instead, it's better to clarify the documentation and mention the
fact that the used object may not change during use.

Note that the buffer interface release API is meant to protect
against such modifications, so I don't see why rejecting objects
that do implement this API should be rejected. It's object that
don't implement the release buffer slot which you'd have to worry
about. Then again, this has never really been an issue in practice
during the 10 years of the 2.x branch, so I wouldn't call this
a serious issue.

See PEP 3118... "All that is specifically required by the exporter, however, is to ensure that any
memory shared through the bufferinfo structure remains valid until releasebuffer is called on the
bufferinfo structure exporting that memory."

----------
nosy: +lemburg

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9602>
_______________________________________