Trouble combining namespace packages and package_dir option in setup
Hello, I'm trying to create a package that has a namespace package inside a directory src below the root directory of the project. I was trying to follow the instructions laid out in the documentation here <https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages>, but could not get the example described there to work using setuptools 41.0.0: [adamson@home] {build-test-37} ~/test-pkg$ tree --charset unicode ..|-- setup.py`-- src `-- namespace `-- mypackage |-- __init__.py `-- mod1.py 3 directories, 3 files [adamson@home] {build-test-37} ~/test-pkg$ cat setup.py from setuptools import setup, find_namespace_packages setup(name="namespace.mypackage", version="0.1", package_dir={'': 'src'}, packages=find_namespace_packages(where='src')) [adamson@home] {build-test-37} ~/test-pkg$ conda list -e# This file may be used to create an environment using:# $ conda create --name <env> --file <this file># platform: linux-64 ca-certificates=2019.1.23=0 certifi=2019.3.9=py37_0 libedit=3.1.20181209=hc058e9b_0 libffi=3.2.1=hd88cf55_4 libgcc-ng=8.2.0=hdf63c60_1 libstdcxx-ng=8.2.0=hdf63c60_1 ncurses=6.1=he6710b0_1 openssl=1.0.2r=h7b6447c_0 pip=19.1=py37_0 python=3.7.0=h6e4f718_3 readline=7.0=h7b6447c_5 setuptools=41.0.0=py37_0 sqlite=3.28.0=h7b6447c_0 tk=8.6.8=hbc83047_0 wheel=0.33.1=py37_0 xz=5.2.4=h14c3975_4 zlib=1.2.11=h7b6447c_3 [adamson@home] {build-test-37} ~/test-pkg$ python setup.py install --record files.txt running install running bdist_egg running egg_info creating src/namespace.mypackage.egg-info writing src/namespace.mypackage.egg-info/PKG-INFO writing dependency_links to src/namespace.mypackage.egg-info/dependency_links.txt writing top-level names to src/namespace.mypackage.egg-info/top_level.txt writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' package init file 'src/namespace/__init__.py' not found (or not a regular file) reading manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/namespace creating build/lib/namespace/mypackage copying src/namespace/mypackage/mod1.py -> build/lib/namespace/mypackage copying src/namespace/mypackage/__init__.py -> build/lib/namespace/mypackage creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/namespace creating build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/mod1.py -> build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/__init__.py -> build/bdist.linux-x86_64/egg/namespace/mypackage byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/mod1.py to mod1.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/__init__.py to __init__.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/namespace.mypackage-0.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing namespace.mypackage-0.1-py3.7.egg Copying namespace.mypackage-0.1-py3.7.egg to /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages Adding namespace.mypackage 0.1 to easy-install.pth file Installed /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages/namespace.mypackage-0.1-py3.7.egg Processing dependencies for namespace.mypackage==0.1 Finished processing dependencies for namespace.mypackage==0.1 writing list of installed files to 'files.txt' [adamson@home] {build-test-37} ~/test-pkg$ python -c 'import namespace.mypackage' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'namespace' Is there a known issue preventing package_dir to work with namespace packages? Is there a known good version of setuptools that I could use to package/install a project using a namespace package and package_dir? Thank you! - Alex
I've replied on this setuptools issue <https://github.com/pypa/setuptools/issues/1754#issuecomment-488300730>, you should be able to fix this by switching to pip install. The core issue is that setup.py install is installing a `.egg` called "namespace.foo" *directly into site-packages* $ ls venv/lib/python3.7/site-packages/ easy-install.pth namespace.foo-0.0.1-py3.7.egg pip-19.1.dist-info __pycache__ setuptools-41.0.1.dist-info wheel-0.33.1.dist-info easy_install.py pip pkg_resources setuptools wheel While pip correctly installs the "namespace" folder with a "foo" module under it: $ ls venv/lib/python3.7/site-packages/ easy_install.py namespace.foo-0.0.1.dist-info pip-19.1.dist-info __pycache__ setuptools-41.0.1.dist-info wheel-0.33.1.dist-info namespace pip pkg_resources setuptools wheel Best, Paul On 5/1/19 10:24 AM, Alex Adamson wrote:
Hello,
I'm trying to create a package that has a namespace package inside a directory |src| below the root directory of the project. I was trying to follow the instructions laid out in the documentation here <https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages>, but could not get the example described there to work using setuptools 41.0.0:
[adamson@home] {build-test-37} ~/test-pkg$ tree --charset unicode . . |-- setup.py `-- src `-- namespace `-- mypackage |-- __init__.py `-- mod1.py
3 directories, 3 files
[adamson@home] {build-test-37} ~/test-pkg$ cat setup.py from setuptools import setup, find_namespace_packages
setup(name="namespace.mypackage", version="0.1", package_dir={'': 'src'}, packages=find_namespace_packages(where='src'))
[adamson@home] {build-test-37} ~/test-pkg$ conda list -e # This file may be used to create an environment using: # $ conda create --name <env> --file <this file> # platform: linux-64 ca-certificates=2019.1.23=0 certifi=2019.3.9=py37_0 libedit=3.1.20181209=hc058e9b_0 libffi=3.2.1=hd88cf55_4 libgcc-ng=8.2.0=hdf63c60_1 libstdcxx-ng=8.2.0=hdf63c60_1 ncurses=6.1=he6710b0_1 openssl=1.0.2r=h7b6447c_0 pip=19.1=py37_0 python=3.7.0=h6e4f718_3 readline=7.0=h7b6447c_5 setuptools=41.0.0=py37_0 sqlite=3.28.0=h7b6447c_0 tk=8.6.8=hbc83047_0 wheel=0.33.1=py37_0 xz=5.2.4=h14c3975_4 zlib=1.2.11=h7b6447c_3
[adamson@home] {build-test-37} ~/test-pkg$ python setup.py install --record files.txt running install running bdist_egg running egg_info creating src/namespace.mypackage.egg-info writing src/namespace.mypackage.egg-info/PKG-INFO writing dependency_links to src/namespace.mypackage.egg-info/dependency_links.txt writing top-level names to src/namespace.mypackage.egg-info/top_level.txt writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' package init file 'src/namespace/__init__.py' not found (or not a regular file) reading manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/namespace creating build/lib/namespace/mypackage copying src/namespace/mypackage/mod1.py -> build/lib/namespace/mypackage copying src/namespace/mypackage/__init__.py -> build/lib/namespace/mypackage creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/namespace creating build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/mod1.py -> build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/__init__.py -> build/bdist.linux-x86_64/egg/namespace/mypackage byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/mod1.py to mod1.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/__init__.py to __init__.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/namespace.mypackage-0.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing namespace.mypackage-0.1-py3.7.egg Copying namespace.mypackage-0.1-py3.7.egg to /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages Adding namespace.mypackage 0.1 to easy-install.pth file
Installed /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages/namespace.mypackage-0.1-py3.7.egg Processing dependencies for namespace.mypackage==0.1 Finished processing dependencies for namespace.mypackage==0.1 writing list of installed files to 'files.txt'
[adamson@home] {build-test-37} ~/test-pkg$ python -c 'import namespace.mypackage' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'namespace'
Is there a known issue preventing |package_dir| to work with namespace packages? Is there a known good version of setuptools that I could use to package/install a project using a namespace package and |package_dir|? Thank you!
- Alex
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/G4MHK...
Thanks, Paul! I noticed the misnamed egg, but I didn’t realize using pip install would fix it. That did indeed fix it. On Wed, May 1, 2019 at 10:50 AM Paul Ganssle <paul@ganssle.io> wrote:
I've replied on this setuptools issue <https://github.com/pypa/setuptools/issues/1754#issuecomment-488300730>, you should be able to fix this by switching to pip install.
The core issue is that setup.py install is installing a `.egg` called "namespace.foo" *directly into site-packages*
$ ls venv/lib/python3.7/site-packages/ easy-install.pth namespace.foo-0.0.1-py3.7.egg pip-19.1.dist-info __pycache__ setuptools-41.0.1.dist-info wheel-0.33.1.dist-info easy_install.py pip pkg_resources setuptools wheel
While pip correctly installs the "namespace" folder with a "foo" module under it:
$ ls venv/lib/python3.7/site-packages/ easy_install.py namespace.foo-0.0.1.dist-info pip-19.1.dist-info __pycache__ setuptools-41.0.1.dist-info wheel-0.33.1.dist-info namespace pip pkg_resources setuptools wheel
Best, Paul On 5/1/19 10:24 AM, Alex Adamson wrote:
Hello,
I'm trying to create a package that has a namespace package inside a directory src below the root directory of the project. I was trying to follow the instructions laid out in the documentation here <https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages>, but could not get the example described there to work using setuptools 41.0.0:
[adamson@home] {build-test-37} ~/test-pkg$ tree --charset unicode ..|-- setup.py`-- src `-- namespace `-- mypackage |-- __init__.py `-- mod1.py
3 directories, 3 files
[adamson@home] {build-test-37} ~/test-pkg$ cat setup.py from setuptools import setup, find_namespace_packages
setup(name="namespace.mypackage", version="0.1", package_dir={'': 'src'}, packages=find_namespace_packages(where='src'))
[adamson@home] {build-test-37} ~/test-pkg$ conda list -e# This file may be used to create an environment using:# $ conda create --name <env> --file <this file># platform: linux-64 ca-certificates=2019.1.23=0 certifi=2019.3.9=py37_0 libedit=3.1.20181209=hc058e9b_0 libffi=3.2.1=hd88cf55_4 libgcc-ng=8.2.0=hdf63c60_1 libstdcxx-ng=8.2.0=hdf63c60_1 ncurses=6.1=he6710b0_1 openssl=1.0.2r=h7b6447c_0 pip=19.1=py37_0 python=3.7.0=h6e4f718_3 readline=7.0=h7b6447c_5 setuptools=41.0.0=py37_0 sqlite=3.28.0=h7b6447c_0 tk=8.6.8=hbc83047_0 wheel=0.33.1=py37_0 xz=5.2.4=h14c3975_4 zlib=1.2.11=h7b6447c_3
[adamson@home] {build-test-37} ~/test-pkg$ python setup.py install --record files.txt running install running bdist_egg running egg_info creating src/namespace.mypackage.egg-info writing src/namespace.mypackage.egg-info/PKG-INFO writing dependency_links to src/namespace.mypackage.egg-info/dependency_links.txt writing top-level names to src/namespace.mypackage.egg-info/top_level.txt writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' package init file 'src/namespace/__init__.py' not found (or not a regular file) reading manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/namespace creating build/lib/namespace/mypackage copying src/namespace/mypackage/mod1.py -> build/lib/namespace/mypackage copying src/namespace/mypackage/__init__.py -> build/lib/namespace/mypackage creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/namespace creating build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/mod1.py -> build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/__init__.py -> build/bdist.linux-x86_64/egg/namespace/mypackage byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/mod1.py to mod1.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/__init__.py to __init__.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/namespace.mypackage-0.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing namespace.mypackage-0.1-py3.7.egg Copying namespace.mypackage-0.1-py3.7.egg to /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages Adding namespace.mypackage 0.1 to easy-install.pth file
Installed /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages/namespace.mypackage-0.1-py3.7.egg Processing dependencies for namespace.mypackage==0.1 Finished processing dependencies for namespace.mypackage==0.1 writing list of installed files to 'files.txt'
[adamson@home] {build-test-37} ~/test-pkg$ python -c 'import namespace.mypackage' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'namespace'
Is there a known issue preventing package_dir to work with namespace packages? Is there a known good version of setuptools that I could use to package/install a project using a namespace package and package_dir? Thank you!
- Alex
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.orghttps://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/G4MHK...
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/3M7SW...
participants (2)
-
Alex Adamson -
Paul Ganssle