Cython inserting unqualified module name into sys.module on python 3?
Hi all, Can anyone shed any light on this? https://github.com/numpy/numpy/issues/5680 -n -- Nathaniel J. Smith -- http://vorpus.org
That is strange, looks like it was an attempt to support relative imports? https://github.com/cython/cython/blob/384cc660f5c7958524b8839ba24099fdbc6eaf... On Sat, Mar 14, 2015 at 1:17 AM, Nathaniel Smith <njs@vorpus.org> wrote:
Hi all,
Can anyone shed any light on this?
https://github.com/numpy/numpy/issues/5680
-n
-- Nathaniel J. Smith -- http://vorpus.org _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
On Mar 14, 2015 2:03 PM, "Robert Bradshaw" <robertwb@gmail.com> wrote:
That is strange, looks like it was an attempt to support relative imports?
https://github.com/cython/cython/blob/384cc660f5c7958524b8839ba24099fdbc6eaf...
Ah, I see. I don't see how this could affect relative imports, because if foo.bar does 'import .baz', this doesn't actually trigger any access to sys.modules["foo.bar"]. (Exception: if you have foo/__init__.pyx. Is that actually supported?) The critical thing for relative imports is having correct __name__ and/or __package__ module-level global variables, and AFAICT cython is not currently doing anything to set these up. But it probably should, because relative imports are a thing. OTOH, putting the module into sys.modules *is* crucial to handle recursive imports, i.e. where foo.pyx's module init function imports bar.py, and bar.py imports foo. For regular python modules or for python 2 extension modules, this works because even while foo is still initializing, you can already get its (incomplete) module object from sys.modules; for python 3 extension modules this won't work unless we do it by hand. So the code that Cython is generating seems to be correct and necessary, it just has the wrong idea about what the fully-qualified module name is, and this breaks things. So I'm convinced that Cython has to know the fully-qualified module name for correct operation, and if it's wrong then weird real bugs will happen. Next question: how am I supposed to make this work? Maybe I'm just missing it, but I can't find anything in the docs about how I should tell cython that mtrand.pyx is really numpy.random.mtrand...? -n
On Sat, Mar 14, 2015 at 1:17 AM, Nathaniel Smith <njs@vorpus.org> wrote:
Hi all,
Can anyone shed any light on this?
https://github.com/numpy/numpy/issues/5680
-n
-- Nathaniel J. Smith -- http://vorpus.org _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
Is this why I can only make scipy.spatial.cKDTree support pickle on Python 3? Since scipy.spatial.cKDTree comes from a relative import of .ckdtree.cKDTree into scipy.spatial, it somehow works on Python 3 but fails on Python 2. Sturla Nathaniel Smith <njs@vorpus.org> wrote:
On Mar 14, 2015 2:03 PM, "Robert Bradshaw" <robertwb@gmail.com> wrote:
That is strange, looks like it was an attempt to support relative imports?
https://github.com/cython/cython/blob/384cc660f5c7958524b8839ba24099fdbc6eaf...
Ah, I see.
I don't see how this could affect relative imports, because if foo.bar does 'import .baz', this doesn't actually trigger any access to sys.modules["foo.bar"]. (Exception: if you have foo/__init__.pyx. Is that actually supported?) The critical thing for relative imports is having correct __name__ and/or __package__ module-level global variables, and AFAICT cython is not currently doing anything to set these up. But it probably should, because relative imports are a thing.
OTOH, putting the module into sys.modules *is* crucial to handle recursive imports, i.e. where foo.pyx's module init function imports bar.py, and bar.py imports foo. For regular python modules or for python 2 extension modules, this works because even while foo is still initializing, you can already get its (incomplete) module object from sys.modules; for python 3 extension modules this won't work unless we do it by hand. So the code that Cython is generating seems to be correct and necessary, it just has the wrong idea about what the fully-qualified module name is, and this breaks things.
So I'm convinced that Cython has to know the fully-qualified module name for correct operation, and if it's wrong then weird real bugs will happen. Next question: how am I supposed to make this work? Maybe I'm just missing it, but I can't find anything in the docs about how I should tell cython that mtrand.pyx is really numpy.random.mtrand...?
-n
On Sat, Mar 14, 2015 at 1:17 AM, Nathaniel Smith <njs@vorpus.org> wrote:
Hi all,
Can anyone shed any light on this?
https://github.com/numpy/numpy/issues/5680
-n
-- Nathaniel J. Smith -- http://vorpus.org _______________________________________________ cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
cython-devel mailing list cython-devel@python.org https://mail.python.org/mailman/listinfo/cython-devel
participants (3)
-
Nathaniel Smith -
Robert Bradshaw -
Sturla Molden