[pypy-dev] pickle numpy array from pypy to cpython?

[Matti Picus]

You need to add the modules to those that the class-local space is built 
with using a spaceconfig, so something like

class AppTestPicklingNumpy(BaseNumpyAppTest):
     spaceconfig = dict(usemodules=["micronumpy", "struct", "binascii"])

     def setup_class(cls):
         if option.runappdirect and '__pypy__' not in 
sys.builtin_module_names:
             py.test.skip("pypy only test")
         BaseNumpyAppTest.setup_class.im_func(cls)


     def test_pickle_module(self):
         import pickle




On 25/06/16 09:01, Eli Stevens (Gmail) wrote:
> I was thinking about doing it on import of the micronumpy module
> (pypy/module/micronumpy/app_numpy.py).
>
> Right now, when I try and import pickle during the tests:
>
> $ cat pypy/module/micronumpy/test/test_pickling_app.py
> import sys
> import py
>
> from pypy.module.micronumpy.test.test_base import BaseNumpyAppTest
> from pypy.conftest import option
>
> class AppTestPicklingNumpy(BaseNumpyAppTest):
>      def setup_class(cls):
>          if option.runappdirect and '__pypy__' not in sys.builtin_module_names:
>              py.test.skip("pypy only test")
>          BaseNumpyAppTest.setup_class.im_func(cls)
>
>      def test_pickle_module(self):
>          import pickle
>          ... # more code
>
>
>   I get this error:
>
>>    import struct
> lib-python/2.7/pickle.py:34:
> _ _ _ _ _ _
>
>>    from _struct import *
> E   (application-level) ImportError: No module named _struct
>
> lib-python/2.7/struct.py:1: ImportError
>
>
> But everything seems fine with struct:
>
> $ ./pytest.py pypy/module/struct/test/test_struct.py
> ==== test session starts ====
> platform linux2 -- Python 2.7.11 -- py-1.4.20 -- pytest-2.5.2
> pytest-2.5.2 from /home/elis/edit/play/pypy/pytest.pyc
> collected 30 items
>
> pypy/module/struct/test/test_struct.py ..............................
>
> ==== 30 passed in 11.95 seconds ====
>
> Any idea what's going on here?
>
> Thanks,
> Eli
>
>
> On Fri, Jun 24, 2016 at 9:19 PM, matti picus <matti.picus at gmail.com> wrote:
>> Sounds reasonable. You might want to generalize it a bit by trying to import
>> _numpypy /numpy, and setting up the replacement by whichever fails to
>> import.
>> Matti
>>
>> On Saturday, 25 June 2016, Eli Stevens (Gmail) <wickedgrey at gmail.com> wrote:
>>> Heh, interestingly, if I add the following to the local dir and files
>>> when trying to unpickle under cpython, it works (note that cpython to
>>> pypy actually works out of the box, which I hadn't realized):
>>>
>>> $ cat _numpypy/__init__.py
>>> from numpy.core import *
>>>
>>> $ cat _numpypy/multiarray.py
>>> from numpy.core.multiarray import *
>>> import numpy.core.multiarray as _ncm
>>> _reconstruct = _ncm._reconstruct
>>>
>>> This is obviously a total hack, and not one I'm comfortable with
>>> (since I need to use this codebase from both cpython and pypy), but it
>>> demonstrates that it's just bookkeeping that needs to change to get
>>> things to work.
>>>
>>> My first approach would be to add a wrapper around save_global here
>>>
>>> https://bitbucket.org/pypy/pypy/src/a0105e0d00dbd0f73d06fc704db704868a6c6ed2/lib-python/2.7/pickle.py?at=default&fileviewer=file-view-default#pickle.py-814
>>> that special-cases the global '_numpypy.multiarray' to instead be
>>> 'numpy.core.multiarray'. That seem like a reasonable thing to do?
>>>
>>> Cheers,
>>> Eli
>>>