[Numpy-discussion] Removing datetime support for 1.4.x series ?

[Pauli Virtanen]

to, 2010-02-04 kello 12:09 -0700, Charles R Harris kirjoitti:
> Realistically, I don't think Py3k will be ready by April-May. Fall is
> probably doable and maybe there will be some things for a SOC person
> to work on this summer.

Well, we have many components of Py3 support in place in the SVN
trunk. 1470 of 1983 unit tests currently pass. (Except that some recent
commit introduced some Py3-breaking Python code.)

Main things missing:

- String arrays, and other worms out of the str/unicode/bytes can.
  I've audited many parts of the C code -- there are 66 remaining
  unchecked points where PyString -> PyBytes is assumed without much
  thinking (grep PyString).

  The Python side (= mainly the tests), however, needs more love.
  Several tests break at the moment since array(['a']).dtype == 'U'
  on Py3.

- There's also the decision about whether 'S' == bytes or str.
  If the former, the tests need fixing, if the latter, the code needs
  fixing.

- Consuming PEP 3118 buffer arrays so that shape and strides are
  correctly acquired. This could in the end allow ufuncs to
  operate on PEP 3118 arrays.

  The buffer provider support is already done, also for Py2.6.

- Rewriting fromfile to use the low-level I/O. Py3 does not support
  getting FILE* pointers out, and the handle from fdopen is currently
  left dangling.

That's certainly a manageable amount of work. For me, however, I believe
real life(TM) will still intrude a bit on my working on Numpy before the
summer, so I can't promise that by April-May all of the test suite
passes on Py3. With some luck it might be possible, though, and
certainly by fall.

I hoped to have Py3 support the only bigger change in the next release,
so that the number of moving pieces of code would be kept down. It does
probably not require an ABI break.

But if an ABI break is in the pipeline, it might make sense to put it
out before "officially" supporting Py3.

    ***

So how to proceed?

I would prefer if new code introduced in the rewrite would compile and
work correctly both on Py2 and Py3. I wouldn't expect this to be a high
overhead, and it would remove the need for a separate Py3 branch.

Most C code that works on Py2 works also on Py3. Py3 mainly means not
using PyString, but choosing between Unicode + Bytes + UString (=Bytes
for Py2 & Unicode for Py3). Also, it may be necessary to avoid FILE*
pointers in the API (on Py3 those are no longer as easily obtained), and
be wary when working with buffers.


I assume the rewrite will be worked on a separate SVN branch? Also, is
there a plan yet on what needs changing to make Numpy's ABI more
resistant?

Cheers,
Pauli