[pypy-dev] pickle numpy array from pypy to cpython?

[Eli Stevens (Gmail)]

I was thinking about doing it on import of the micronumpy module
(pypy/module/micronumpy/app_numpy.py).

Right now, when I try and import pickle during the tests:

$ cat pypy/module/micronumpy/test/test_pickling_app.py
import sys
import py

from pypy.module.micronumpy.test.test_base import BaseNumpyAppTest
from pypy.conftest import option

class AppTestPicklingNumpy(BaseNumpyAppTest):
    def setup_class(cls):
        if option.runappdirect and '__pypy__' not in sys.builtin_module_names:
            py.test.skip("pypy only test")
        BaseNumpyAppTest.setup_class.im_func(cls)

    def test_pickle_module(self):
        import pickle
        ... # more code


 I get this error:

>   import struct

lib-python/2.7/pickle.py:34:
_ _ _ _ _ _

>   from _struct import *
E   (application-level) ImportError: No module named _struct

lib-python/2.7/struct.py:1: ImportError


But everything seems fine with struct:

$ ./pytest.py pypy/module/struct/test/test_struct.py
==== test session starts ====
platform linux2 -- Python 2.7.11 -- py-1.4.20 -- pytest-2.5.2
pytest-2.5.2 from /home/elis/edit/play/pypy/pytest.pyc
collected 30 items

pypy/module/struct/test/test_struct.py ..............................

==== 30 passed in 11.95 seconds ====

Any idea what's going on here?

Thanks,
Eli


On Fri, Jun 24, 2016 at 9:19 PM, matti picus <matti.picus at gmail.com> wrote:
> Sounds reasonable. You might want to generalize it a bit by trying to import
> _numpypy /numpy, and setting up the replacement by whichever fails to
> import.
> Matti
>
> On Saturday, 25 June 2016, Eli Stevens (Gmail) <wickedgrey at gmail.com> wrote:
>>
>> Heh, interestingly, if I add the following to the local dir and files
>> when trying to unpickle under cpython, it works (note that cpython to
>> pypy actually works out of the box, which I hadn't realized):
>>
>> $ cat _numpypy/__init__.py
>> from numpy.core import *
>>
>> $ cat _numpypy/multiarray.py
>> from numpy.core.multiarray import *
>> import numpy.core.multiarray as _ncm
>> _reconstruct = _ncm._reconstruct
>>
>> This is obviously a total hack, and not one I'm comfortable with
>> (since I need to use this codebase from both cpython and pypy), but it
>> demonstrates that it's just bookkeeping that needs to change to get
>> things to work.
>>
>> My first approach would be to add a wrapper around save_global here
>>
>> https://bitbucket.org/pypy/pypy/src/a0105e0d00dbd0f73d06fc704db704868a6c6ed2/lib-python/2.7/pickle.py?at=default&fileviewer=file-view-default#pickle.py-814
>> that special-cases the global '_numpypy.multiarray' to instead be
>> 'numpy.core.multiarray'. That seem like a reasonable thing to do?
>>
>> Cheers,
>> Eli
>>
>