[New-bugs-announce] [issue15540] Python 3.3 and numpy

[Dave Malcolm]

New submission from Dave Malcolm:

I've been trying to get numpy working with Python 3.3, and to so I had to make some changes to CPython - hence I'm posting this to the Python bug tracker.

numpy pokes at the insides of PyUnicodeObject in a few places and is thus affected by the PEP 393 changes in Python 3.3

I'm attaching my latest work-in-progress patch for numpy, which mostly works (it has 3 remaining errors).

AIUI, the "numpy.str_" type subclasses PyUnicodeObject but with its own custom allocator, which takes a size (this is called in PyArray_Scalar when type_num == NPY_UNICODE).  unicode_new_subtype calls tp_alloc but passes in 0 for the size, so we can't use that.  So I had to reimplement parts of unicode creation in-place within numpy's PyArray_Scalar, copying macros from out of cpython's unicodeobject.c

The other wart is that, AIUI, numpy supports byte-swapping the values within an array, and when this is done for a unicode array, it byte-swaps the 4-byte UCS4 values.  At that point, it's unlikely that the resulting 4-byte values are below 0x10ffff, leading to various failures from inside CPython's unicode handling.

So I hacked those test from out of CPython :)   I'm attaching the diff I've got against cpython (clearly just a hack at this stage).

I may of course be misunderstanding the insides of numpy.

With these changes, the numpy test suite runs, with just these remaining errors:

======================================================================
ERROR: Ticket #16
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/core/tests/test_regression.py", line 41, in test_pickle_transposed
    b = pickle.load(f)
EOFError

======================================================================
ERROR: test_power_zero (test_umath.TestPower)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/core/tests/test_umath.py", line 139, in test_power_zero
    assert_complex_equal(np.power(zero, 0+1j), cnan)
RuntimeWarning: invalid value encountered in power

======================================================================
ERROR: Failure: ValueError (can't handle version 187 of numpy.ndarray pickle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/nose-1.1.2-py3.3.egg/nose/failure.py", line 37, in runTest
    raise self.exc_class(self.exc_val).with_traceback(self.tb)
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/nose-1.1.2-py3.3.egg/nose/loader.py", line 232, in generate
    for test in g():
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/lib/tests/test_format.py", line 429, in test_roundtrip
    arr2 = roundtrip(arr)
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/lib/tests/test_format.py", line 420, in roundtrip
    arr2 = format.read_array(f2)
  File "/home/david/coding/python3.3/local-install/lib/python3.3/site-packages/numpy/lib/format.py", line 449, in read_array
    array = pickle.load(fp)
ValueError: can't handle version 187 of numpy.ndarray pickle

----------------------------------------------------------------------
Ran 4776 tests in 68.189s

FAILED (KNOWNFAIL=6, SKIP=1, errors=3)

----------
components: Interpreter Core
files: get-numpy-working-with-python-3.3.patch
keywords: patch
messages: 167256
nosy: dmalcolm
priority: normal
severity: normal
status: open
title: Python 3.3 and numpy
versions: Python 3.3
Added file: http://bugs.python.org/file26668/get-numpy-working-with-python-3.3.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15540>
_______________________________________