[Python-Dev] PEP 3118: Extended buffer protocol (new version)
pythondev at aerojockey.com
Thu Apr 19 08:53:50 CEST 2007
Travis Oliphant wrote:
> Carl Banks wrote:
>> Ok, I've thought quite a bit about this, and I have an idea that I
>> think will be ok with you, and I'll be able to drop my main
>> objection. It's not a big change, either. The key is to explicitly
>> say whether the flag allows or requires. But I made a few other
>> changes as well.
> I'm good with using an identifier to differentiate between an "allowed"
> flag and a "require" flag. I'm not a big fan of
> VERY_LONG_IDENTIFIER_NAMES though. Just enough to understand what it
> means but not so much that it takes forever to type and uses up
> horizontal real-estate.
That's fine with me. I'm not very particular about spellings, as long
as they're not misleading.
>> Now, here is a key point: for these functions to work (indeed, for
>> PyObject_GetBuffer to work at all), you need enough information in
>> bufinfo to figure it out. The bufferinfo struct should be
>> self-contained; you should not need to know what flags were passed to
>> PyObject_GetBuffer in order to know exactly what data you're looking at.
>> Therefore, format must always be supplied by getbuffer. You cannot
>> tell if an array is contiguous without the format string. (But see
> No, I don't think this is quite true. You don't need to know what
> "kind" of data you are looking at if you don't get strides. If you use
> the SIMPLE interface, then both consumer and exporter know the object is
> looking at "bytes" which always has an itemsize of 1.
But doesn't this violate the above maxim? Suppose these are the
contents of bufinfo:
ndim = 1
len = 20
shape = (10,)
strides = (2,)
format = NULL
How does it know whether it's looking at contiguous array of 10 two-byte
objects, or a discontiguous array of 10 one-byte objects, without having
at least an item size? Since item size is now in the mix, it's moot, of
The idea that Py_BUF_SIMPLE implies bytes is news to me. What if you
want a contiguous, one-dimensional array of an arbitrary type? I was
thinking this would be acceptable with Py_BUF_SIMPLE. It seems you want
to require Py_BUF_FORMAT for that, which would suggest to me that
Py_BUF_ALLOW_ND amd Py_BUF_ALLOW_NONCONTIGUOUS, etc., would imply
Py_BUF_FORMAT? IOW, pretty much anything that's not SIMPLE implies FORMAT?
If that's the case, then most of the issues I brought up about item size
don't apply. Also, if that's the case, you're right that Py_BUF_FORMAT
makes more sense than Py_BUF_DONT_NEED_FORAMT.
But it now it seems even more unnecessary than it did before. Wouldn't
any consumer that just wants to look at a chunk of bytes always use
Py_BUF_FORMAT, especially if there's danger of a presumptuous exporter
raising an exception?
> I'll update the PEP with my adaptation of your suggestions in a little
Ok. Thanks for taking the lead, and for putting up with my verbiose
More information about the Python-Dev