Thanks for a swift reply: you're just the person I hoped would do so.
On 04/08/2012 10:11, Stefan Krah wrote:
You are right that the PEP does not explicitly state that rule for strides. However, NULL always has an implied meaning:
format=NULL -> treat the buffer as unsigned bytes.
shape=NULL -> one-dimensional AND treat the buffer as unsigned bytes.
strides=NULL -> C-contiguous
I think relaxing the NULL rule for strides would complicate things, since it would introduce yet another special case.
... Ok, I think I see that how the absence of certain arrays is used to deduce structural simplicity, over and above their straightforward use in navigating the data. So although no shape array is (sort of) equivalent to ndim==1, shape==len, it also means I can call simpler code instead of using the arrays for navigation.
I still don't see why, if the consumer says "I'm assuming 1-D unsigned bytes", and that's what the data is, memoryview_getbuf could not provide a shape and strides that agree with the data. Is the catch perhaps that there is code (in abstract.c etc.) that does not know what the consumer promised not to use/look at? Would it actually break, e.g. not treat it as bytes, or just be inefficient?
Because of all the implied meanings of NULL, I think the safest way is to implement memoryview_getbuf() for Jython. After all the PEP describes a protocol, so everyone should really be doing the same thing.
I'll look carefully at what you've written (snipped here) because it is these "consumer expectations" that are most important. The Jython buffer API is necessarily a lot different from the C one: some things are not possible in Java (pointer arithmetic) and some are just un-Javan activities (allocate a struct and have the library fill it in). I'm only going for a logical conformance to the PEP: the same navigational and other attributes, that mean the same things for the consumer.
When you say such-and-such is disallowed, but the PEP or the data structures seem to provide for it, you mean memoryview_getbuf() disallows it, since you've concluded it is not sensible?
I think the protocol would benefit from changing the getbuffer rules to:
a) The buffer gets a 'flags' field that can store properties like PyBUF_SIMPLE, PyBUF_C_CONTIGUOUS etc. b) The exporter must *always* provide full information. c) If a buffer can be exported as unsigned bytes but has a different layout, the exporter must perform a full cast so that the above mentioned invariants are kept.
Just like PyManagedBuffer mbuf and its sister view in memoryview? I've thought the same things, but the tricky part is to do it compatibly.
a) I think I can achieve this. As I have interfaces and polymorphism on my side, and a commitment only to logical equivalence to CPython, I can have the preserved flags stashed away inside to affect behaviour. But it's not as simple as saving the consumer's request, and I'm still trying to work it out what to do, e.g. when the consumer didn't ask for C-contiguity, but in this case it happens to be true.
In the same way, functions you have in abstract.c etc. can be methods that, rather than work out by inspection of a struct how to navigate the data on this call, already know what kind of buffer they are in. So SimpleBuffer.isContiguous(char order) can simply return true.
b) What I'm hoping can work, but maybe not.
c) Java will not of course give you raw memory it thinks is one thing, to treat as another, so this aspect is immature in my thinking. I got as far as accommodating multi-byte items, but have no use for them as yet.
Thanks again for the chance to test my ideas. Jeff Allen