[Python-Dev] pre-PEP: The Safe Buffer Interface
Thomas Heller
thomas.heller@ion-tof.com
Fri, 26 Jul 2002 16:28:50 +0200
Here is the draft PEP for the ideas posted here.
Regards,
Thomas
--------
PEP: xxx
Title: The Safe Buffer Interface
Version: $Revision: $
Last-Modified: $Date: 2002/07/26 14:19:38 $
Author: theller@python.net (Thomas Heller)
Status: Draft
Type: Standards Track
Created: 26-Jul-2002
Python-Version: 2.3
Post-History: 26-Jul-2002
Abstract
This PEP proposes an extension to the buffer interface called the
'safe buffer interface'.
The safe buffer interface fixes the flaws of the 'old' buffer
interface as defined in Python versions up to and including 2.2:
The lifetime of the retrieved pointer is clearly defined.
The buffer size is returned as a 'size_t' data type, which allows
access to 'large' buffers on platforms where sizeof(int) !=
sizeof(void *).
Specification
The 'safe' buffer interface exposes new functions which return the
size and the pointer to the internal memory block of any python
object which chooses to implement this interface.
The size and pointer returned must be valid as long as the object
is alive (has a positive reference count). So, only objects which
never reallocate or resize the memory block are allowed to
implement this interface.
The safe buffer interface ommits the memory segment model which is
present in the old buffer interface - only a single memory block
can be exposed.
Implementation
Define a new flag in Include/object.h:
#define Py_TPFLAGS_HAVE_GETSAFEBUFFER
/* PyBufferProcs contains bf_getsafereadbuffer
and bf_getsafewritebuffer */
#define Py_TPFLAGS_HAVE_GETSAFEBUFFER (1L<<15)
This flag would be included in Py_TPFLAGS_DEFAULT:
#define Py_TPFLAGS_DEFAULT ( \
....
Py_TPFLAGS_HAVE_GETCHARBUFFER | \
....
0)
Extend the PyBufferProcs structure by new fields in
Include/object.h:
typedef size_t (*getlargereadbufferproc)(PyObject *, void **);
typedef size_t (*getlargewritebufferproc)(PyObject *, void **);
typedef struct {
getreadbufferproc bf_getreadbuffer;
getwritebufferproc bf_getwritebuffer;
getsegcountproc bf_getsegcount;
getcharbufferproc bf_getcharbuffer;
/* safe buffer interface functions */
getsafereadbufferproc bf_getsafereadbufferproc;
getsafewritebufferproc bf_getsafewritebufferproc;
} PyBufferProcs;
The new fields are present if the Py_TPFLAGS_HAVE_GETLARGEBUFFER
flag is set in the object's type.
XXX Py_TPFLAGS_HAVE_GETLARGEBUFFER implies the
Py_TPFLAGS_HAVE_GETCHARBUFFER flag.
The getsafereadbufferproc and getsafewritebufferproc functions
return the size in bytes of the memory block on success, and fill
in the passed void * pointer on success. If these functions fail
- either because an error occurs or no memory block is exposed -
they must set the void * pointer to NULL and raise an exception.
The return value is undefined in these cases and should not be
used.
Backward Compatibility
There are no backward compatibility problems.
Reference Implementation
Will be uploaded to the sourceforge patch manager by the author.
Additional Notes/Comments
It may be a good idea to expose the following convenience functions:
int PyObject_AsSafeReadBuffer(PyObject *obj,
void **buffer,
size_t *buffer_len);
int PyObject_AsSafeWriteBuffer(PyObject *obj,
void **buffer,
size_t *buffer_len);
These functions return 0 on success, set buffer to the memory
location and buffer_len to the length of the memory block in
bytes. On failure, they return -1 and set an exception.
Python strings, unicode strings, mmap objects, and maybe other
types would expose the safe buffer interface, but the array type
would *not*, because it's memory block may be reallocated during
it's lifetime.
References
[1] The buffer interface
http://mail.python.org/pipermail/python-dev/2000-October/009974.html
[2] The Buffer Problem
http://www.python.org/peps/pep-0296.html
Copyright
This document has been placed in the public domain.
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End: