[Numpy-discussion] Status of NumPy and Python 3.3

[Ondřej Čertík]

On Mon, Jul 30, 2012 at 5:00 PM, Ronan Lamy <ronan.lamy at gmail.com> wrote:
> Le lundi 30 juillet 2012 à 11:07 -0700, Ondřej Čertík a écrit :
>> On Mon, Jul 30, 2012 at 10:04 AM, Ronan Lamy <ronan.lamy at gmail.com> wrote:
>> > Le lundi 30 juillet 2012 à 17:10 +0100, Ronan Lamy a écrit :
>> >> Le lundi 30 juillet 2012 à 04:57 +0100, Ronan Lamy a écrit :
>> >> > Le lundi 30 juillet 2012 à 02:00 +0100, Ronan Lamy a écrit :
>> >> >
>> >> > >
>> >> > > Anyway, I managed to compile (by blanking
>> >> > > numpy/distutils/command/__init__.py) and to run the tests. I only see
>> >> > > the 2 pickle errors from your latest gist. So that's all good!
>> >> >
>> >> > And the cause of these errors is that running the test suite somehow
>> >> > corrupts Python's internal cache of bytes objects, causing the
>> >> > following:
>> >> > >>> b'\x01XXX'[0:1]
>> >> > b'\xbb'
>> >>
>> >> The culprit is test_pickle_string_overwrite() in test_regression.py. The
>> >> test actually tries to check for that kind of problem, but on Python 3,
>> >> it only manages to trigger it without detecting it. Here's a simple way
>> >> to reproduce the issue:
>> >>
>> >> >>> a = numpy.array([1], 'b')
>> >> >>> b = pickle.loads(pickle.dumps(a))
>> >> >>> b[0] = 77
>> >> >>> b'\x01  '[0:1]
>> >> b'M'
>> >>
>> >> Actually, this problem is probably quite old: I can see it in 1.6.1 w/
>> >> Python 3.2.3. 3.3 only makes it more visible.
>> >>
>> >> I'll open an issue on GitHub ASAP.
>> >>
>> > https://github.com/numpy/numpy/issues/370
>>
>> Thanks Ronan, nice work!
>>
>> Since you looked into this -- do you know a way to fix this? (Both
>> NumPy and the test.)
>
> Pauli found out how to fix the code, so I'll try to send a PR tonight.


So this PR is now in and the issue is fixed.

As far as swapping the unicode issues, I finally understand what is
going on and I posted my current understanding into the Python tracker
issue (http://bugs.python.org/issue15540) which was recently created
for this same issue:

http://bugs.python.org/msg167280

but it was determined that it is not a bug in Python so it is closed
now. Finally, I have submitted a reworked version of my patch here:

https://github.com/numpy/numpy/pull/372

It implements things in a clean way.

Ondrej