[Numpy-discussion] Cython issues w/ 1.4.0

[Pauli Virtanen]

ti, 2009-12-08 kello 12:14 -0600, Robert Kern kirjoitti:
> On Tue, Dec 8, 2009 at 12:08, Pauli Virtanen <pav at iki.fi> wrote:
> > ke, 2009-12-09 kello 02:47 +0900, David Cournapeau kirjoitti:
> > [clip]
> >> Of course, this does not prevent from applying your suggested change -
> >> I don't understand why you want to add it to 1.4.0, though. 1.4.0 does
> >> not break the ABI compared to 1.3.0. Or is it "just" to avoid the
> >> cython issue to reappear for 1.5.0 ?
> >
> > Yes, it's to avoid having to deal with the Cython issue again in 1.5.0.
> 
> Do we have any features on deck that would add a struct member? I
> think it's pretty rare for us to do so, as it should be.

If we want to support PEP 3118 on Py2.6, then new fields would be
useful:

- Python2.6 currently has issues with PyArg_ParseTuple("s#", ...):
  defining a bf_releasebuffer breaks that particular feature.

  Consequently, if we want backwards compatibility, we cannot keep track
  of allocated memory using the Py_buffer structure, so something else
  is needed.

  We can probably get this fixed in future Python 2.6/2.7 releases,
  making it prefer the old buffer interface.  The issue also most likely
  unfixable on Py3, since "s#" has semantics that are not really
  compatible with the new buffer interface.

- We need to cache the buffer protocol format string somewhere,
  if we do not want to regenerate it on each buffer acquisition.

  The natural place for this would be in the ArrayDescr.

The alternative at the moment is to #ifdef the PEP 3118 implementation
out on Py2, at least until the issues on 2.6 are fixed.

This would be actually a cleaner alternative -- working around bugs like
this is icky.

> > Although it's not strictly speaking an ABI break, it seems this is a bit
> > of a nuisance for some people, so if we can work around it cheaply, I
> > think we should do it.
> 
> Breaking compatibility via a major reorganization of our structs is not cheap!

We already break the compatibility in 1.4.0 because of the datetime
"metadata" field added in the ArrayDescr. Adding an additional field
reserved for future use at the same time should not cause additional
breakage. They could come useful later on, even if it turns out we don't
immediately need them.

(Or maybe you commented on David's suggestion...)

-- 
Pauli Virtanen