- Summary:

The PEP, or sometimes just the documentation, definitely requires that features not requested shall be NULL.

The API would benefit from:
a. stored flags that tell you the actual structural features.
b. requiring exporters to provide full information (e.g. strides = {1}, format = "B") even when trivial.
It could and possibly should work this way in Python 4.0.

Nick thinks we could *allow* exporters to behave this way (PEP change) in Python 3.x. Stefan thinks not, because "Perhaps there is code that tests for shape==NULL to determine C-contiguity."

Jython exporters should return full information unconditionally from the start:  "any implementation that doesn't use the Py_buffer struct directly in a C-API should just always return a full buffer" (Stefan); "I think that's the way Jython should go: *require* that those fields be populated appropriately" (Nick).

- But what I now think is:

If the only problem really is "code that tests for shape==NULL to determine C-contiguity", or makes similar deductions, I agree that providing unasked-for information is safe. I think the stipulation in PEP/documentation has some efficiency value: on finding shape!=NULL the code has to do a more complicated test, as inPyBuffer_IsContiguous(). I have the option to provide an isContiguous that has the answer written down already, so the risk is only from/to ported code. If it is only a risk to the efficiency of ported code, I'm relaxed: I hesitate only to check that there's no circumstance that logically requires nullity for correctness. Whether it was safe that was the key question.

In the hypothetical Python 4.0 buffer API (and in Jython) where feature flags are provided, the efficiency is still useful, but complicated deductive logic in the consumer should be deprecated in favour of (functions for) interrogating the flags.

An example illustrating the semantics would then be:
1. consumer requests a buffer, saying "I can cope with a strided arrays" (PyBUF_STRIDED);
2. exporter provides a strides array, but in the feature flags STRIDED=0, meaning "you don't need the strides array";
3. exporter (optionally) uses efficient, non-strided access.

I do not think that full provision by the exporter has to be mandatory, as the discussion has gone on to suggest. I know your experience is that you have often had to regenerate the missing information to write generic code, but I think this does not continue once you have the feature flags. An example would be:
1. consumer requests a buffer, saying "I can cope with a N-dimensional but not strided arrays" (PyBUF_ND);
2. exporter sets strides=NULL, and the feature flag STRIDED=0;
3. exporter accesses the data, without reference to the strides array, as it planned;
4. new generic code that respects the feature flag STRIDED=0, does not reference the strides array;
5. old generic code, ignorant of the feature flags, finds the strides=NULL and so does not dereference strides.
Insofar as it is not necessary, there is some efficiency in not providing it. There would only be a problem with broken code that both ignores the feature flag and uses the strides array unchecked. But this code was always broken.

Really useful discussion this.
Jeff