Importing error in pypy
Hi Dears, I am currently using pypy(pypy-7.0.0-linux_x86_64-portable) and I am facing some issues using it. I installed pip, cython and numpy using commands: $ ./pypy-xxx/bin/pypy -m ensurepip $ ./pypy-xxx/bin/pip install cython numpy After that, I installed one package *"pysubnettree" *thorough pypy's pip. While importing SubnetTree class, I get segmentation fault. It works completely file with python2.7. *test.py:* import SubnetTree Here are some logs that I collected using pypy's falthandler tool. ======================================================================================== Fatal Python error: Segmentation fault Stack (most recent call first, approximate line numbers): File "/root/pypy_7/pypy-7.0.0-linux_x86_64-portable/site-packages/SubnetTree.py", line 13 in swig_import_helper File "/root/pypy_7/pypy-7.0.0-linux_x86_64-portable/site-packages/SubnetTree.py", line 11 in <module> File "test.py", line 1 in <module> File "<builtin>/app_main.py", line 767 in run_it File "<builtin>/app_main.py", line 78 in run_toplevel File "<builtin>/app_main.py", line 568 in run_command_line File "<builtin>/app_main.py", line 848 in entry_point Segmentation fault ========================================================================================= I tried to debug it further, it seems like segfault is observed while loading *(.so)* file using python's inbuild module *imp.load_module('_SubnetTree', fp, pathname, description).* It would be a great help if you can throw some light on this. Thanks and Regards, Nabil
Editing the subject. On Wed, Jul 24, 2019 at 11:39 AM Nabil Memon <nabilmemon.ec@gmail.com> wrote:
Hi Dears,
I am currently using pypy(pypy-7.0.0-linux_x86_64-portable) and I am facing some issues using it. I installed pip, cython and numpy using commands: $ ./pypy-xxx/bin/pypy -m ensurepip $ ./pypy-xxx/bin/pip install cython numpy After that, I installed one package *"pysubnettree" *thorough pypy's pip. While importing SubnetTree class, I get segmentation fault. It works completely file with python2.7.
*test.py:* import SubnetTree
Here are some logs that I collected using pypy's falthandler tool.
======================================================================================== Fatal Python error: Segmentation fault Stack (most recent call first, approximate line numbers): File "/root/pypy_7/pypy-7.0.0-linux_x86_64-portable/site-packages/SubnetTree.py", line 13 in swig_import_helper File "/root/pypy_7/pypy-7.0.0-linux_x86_64-portable/site-packages/SubnetTree.py", line 11 in <module> File "test.py", line 1 in <module> File "<builtin>/app_main.py", line 767 in run_it File "<builtin>/app_main.py", line 78 in run_toplevel File "<builtin>/app_main.py", line 568 in run_command_line File "<builtin>/app_main.py", line 848 in entry_point Segmentation fault
=========================================================================================
I tried to debug it further, it seems like segfault is observed while loading *(.so)* file using python's inbuild module *imp.load_module('_SubnetTree', fp, pathname, description).* It would be a great help if you can throw some light on this.
Thanks and Regards, Nabil
Hi Nabil, On Wed, 24 Jul 2019 at 09:02, Nabil Memon <nabilmemon.ec@gmail.com> wrote:
I am currently using pypy(pypy-7.0.0-linux_x86_64-portable) and I am facing some issues using it. I installed pip, cython and numpy using commands: $ ./pypy-xxx/bin/pypy -m ensurepip $ ./pypy-xxx/bin/pip install cython numpy After that, I installed one package "pysubnettree" thorough pypy's pip. While importing SubnetTree class, I get segmentation fault.
This is because the module does something that is unexpected from us, and probably in the gray area of "correctness". The C++ module SubnetTree.cc contains this global variable declaration (it couldn't occur in C): static PyObject* dummy = Py_BuildValue("s", "<dummy string>"); This is in the gray area because there is no particular guarantee that, even on CPython, this is called while the GIL is held. In the case of PyPy, that crashes if it's the first cpyext module you import. A quick workaround is to say "import numpy" before; this should fix that particular problem. We may figure out an acceptable workaround, but this is really a "don't do that". Ideally you should fix it in SubnetTree.cc and submit a pull request. A bientôt, Armin.
I did figure out the workaround by not importing this cpyext module at first. I didn't dig deep into the CPP module, thanks a lot for pointing out the exact gray area. You saved me a lot of time. Au revoir. Regards, Nabil On Thu, 25 Jul, 2019, 7:52 PM Armin Rigo, <armin.rigo@gmail.com> wrote:
Hi Nabil,
On Wed, 24 Jul 2019 at 09:02, Nabil Memon <nabilmemon.ec@gmail.com> wrote:
I am currently using pypy(pypy-7.0.0-linux_x86_64-portable) and I am facing some issues using it. I installed pip, cython and numpy using commands: $ ./pypy-xxx/bin/pypy -m ensurepip $ ./pypy-xxx/bin/pip install cython numpy After that, I installed one package "pysubnettree" thorough pypy's pip. While importing SubnetTree class, I get segmentation fault.
This is because the module does something that is unexpected from us, and probably in the gray area of "correctness". The C++ module SubnetTree.cc contains this global variable declaration (it couldn't occur in C):
static PyObject* dummy = Py_BuildValue("s", "<dummy string>");
This is in the gray area because there is no particular guarantee that, even on CPython, this is called while the GIL is held. In the case of PyPy, that crashes if it's the first cpyext module you import. A quick workaround is to say "import numpy" before; this should fix that particular problem.
We may figure out an acceptable workaround, but this is really a "don't do that". Ideally you should fix it in SubnetTree.cc and submit a pull request.
A bientôt,
Armin.
Hi again, On Thu, 25 Jul 2019 at 16:53, Nabil Memon <nabilmemon.ec@gmail.com> wrote:
I did figure out the workaround by not importing this cpyext module at first.
Cool. In the meantime I managed to sneak in a workaround for the problem that shouldn't have a performance impact, in c89821342184. A bientôt, Armin.
Awesome, thanks! On Thu, Jul 25, 2019 at 8:52 PM Armin Rigo <armin.rigo@gmail.com> wrote:
Hi again,
On Thu, 25 Jul 2019 at 16:53, Nabil Memon <nabilmemon.ec@gmail.com> wrote:
I did figure out the workaround by not importing this cpyext module at first.
Cool. In the meantime I managed to sneak in a workaround for the problem that shouldn't have a performance impact, in c89821342184.
A bientôt,
Armin.
participants (2)
-
Armin Rigo -
Nabil Memon