[Python-Dev] PEP 296 - The Buffer Problem

[Scott Gilbert]

--- Thomas Heller <thomas.heller@ion-tof.com> wrote:
> Let me ask some questions and about platforms with 32-bit
> integers and 64-bit longs:
> 
> >     2. Alignment of the allocated byte array is whatever is promised by
> >     the platform implementation of malloc.
> 
> On these platforms, does malloc() accept an unsigned long argument
> for the requested size?
> 

At the moment, the only 64 bit platform that I have easy access to is
Tru64/Alpha.  That version of malloc takes a size_t which is a 64 bit
quantity.

I believe most semi-sane platforms will use a size_t as argument for
malloc, and I believe most semi-sane platforms will have a size_t that is
the same number of bits as a pointer for that platform.


> > [...]
> >     8. The bytes object keeps track of the length of its data with a
> >     Python LONG_LONG type.
> > [...]
> >     From Python scripting, the bytes object will be subscriptable with
> >     longs so the 32 bit int limit can be avoided.
> 
> How is indexing done in C?> 

Indexing is done by grabbing the pointer and length via a call like:

    int PyObject_AsLargeReadBuffer(PyObject* bo, unsigned char** ptr,
                                   LONG_LONG* len);

Note that the name could be different depending on whether it ends up in
abstract.h or bytesobject.h.

> Can you index these byte arrays by longs?

You could index it via a long, but using a LONG_LONG is safer.  My
understanding is that on Win64 a long will only be 32 bits even though
void* is 64 bits.  So for that platform, LONG_LONG will be a typedef for
__int64 which is 64 bits.

None of this matters for 32 bit platforms.  All 32 bit platforms that I
know of have sizeof(int) == sizeof(long) == sizeof(void*) == 4.  So even if
you wanted to subscript with a long or LONG_LONG, the pointer could only
point to something about 2 Gigs (31 bits) in size.


> > 
> >     10. From the C API, the bytes object can be allocated using any of
> >     the following signatures:
> > 
> >        PyObject* PyBytes_FromLength(LONG_LONG len, int readonly);
> >        PyObject* PyBytes_FromPointer(void* ptr, LONG_LONG len,
> >                int readonly void (*dest)(void *ptr, void *user),
> >                void* user);
> 
> Hm, if 'bytes' is a new style class, these functions should
> require a 'PyObject *type' parameter as well. OTOH, new style
> classes are usually created by calling their *type*, so you
> should describe the signature of the byte type's tp_call.
> (It may be possible to supply variations of the above functions
> for convenience as well.)
> 

I consider these to be the minimum convenience functions that are necessary
for the functionality I'd like to see.  I'll follow the conventions for
creating a new style class for PyBytesObject to the letter, and any other
variations of the above convenience functions can be added as needed. 
(It's easier to add stuff than take it away...)


Cheers,
    -Scott


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com