[Python-Dev] PEP 296 - The Buffer Problem

[Thomas Heller]

> PEP: 296
> Title: The Buffer Problem

IMO should better be 'The bytes Object'

>     6. In C/C++ extensions, the bytes object can be created from a supplied
>     pointer and destructor function to free the memory when the
>     reference count goes to zero.
> 
>     The special implementation of slicing for the bytes object allows
>     multiple bytes objects to refer to the same pointer/destructor.
>     As such, a refcount will be kept on the actual
>     pointer/destructor.  This refcount is separate from the refcount
>     typically associated with Python objects.
> 

Why is this? Wouldn't it be sufficient if views keep references
to the 'viewed' byte object?

>     8. The bytes object keeps track of the length of its data with a Python
>     LONG_LONG type.  Even though the current definition for PyBufferProcs
>     restricts the length to be the size of an int, this PEP does not propose
>     to make any changes there.  Instead, extensions can work around this limit
>     by making an explicit PyBytes_Check(...) call, and if that succeeds they
>     can make a PyBytes_GetReadBuffer(...) or PyBytes_GetWriteBuffer call to
>     get the pointer and full length of the object as a LONG_LONG.
> 
>     The bytes object will raise an exception if the standard PyBufferProcs
>     mechanism is used and the size of the bytes object is greater than can be
>     represented by an integer.
> 
>     From Python scripting, the bytes object will be subscriptable with longs
>     so the 32 bit int limit can be avoided.
> 
>     There is still a problem with the len() function as it is PyObject_Size()
>     and this returns an int as well.  As a workaround, the bytes object will
>     provide a .length() method that will return a long.
> 
Is this worth the trouble?
(Hm, 64-bit platforms with 32-bit integers remind my of the broken
DOS/Windows 3.1 platforms with near/far/huge pointers).

>     9. The bytes object can be constructed at the Python scripting level by
>     passing an int/long to the bytes constructor with the number of bytes to
>     allocate.  For example:
> 
>        b = bytes(100000) # alloc 100K bytes
> 
>     The constructor can also take another bytes object.  This will be useful
>     for the implementation of unpickling, and in converting a read-write bytes
>     object into a read-only one.  An optional second argument will be used to
>     designate creation of a readonly bytes object.

> 
>     10. From the C API, the bytes object can be allocated using any of the
>     following signatures:
> 
>        PyObject* PyBytes_FromLength(LONG_LONG len, int readonly);
>        PyObject* PyBytes_FromPointer(void* ptr, LONG_LONG len, int readonly
>                 void (*dest)(void *ptr, void *user), void* user);
>     
>     In the PyBytes_FromPointer(...) function, if the dest function pointer is
>     passed in as NULL, it will not be called.  This should only be used for
>     creating bytes objects from statically allocated space.
>     
>     The user pointer has been called a closure in other places.  It is a
>     pointer that the user can use for whatever purposes.  It will be passed to
>     the destructor function on cleanup and can be useful for a number of
>     things.  If the user pointer is not needed, NULL should be passed instead.

Shouldn't there be constructors to create a view of a bytes/view object,
or are we supposed to create them by slicing?

>     11. The bytes type will be a new style class as that seems to be where all
>     standard Python types are headed.
 
Good.

Thanks,

Thomas