setup.py develop un-egg-spected result
I have a problem with "setup.py develop" installation. I have the following folder tree: pubsub \- setup.py \- extern \- module0.py \- core \- __init__.py \- module1.py \- module2.py \- ... I want my setup.py to install modules from pubsub/extern (such as module0.py) into site-packages, and to install the 'core' package in site-packages as pubsubcore: setup( ... py_modules = ['module0'], packages = ['pubsubcore'], package_dir = {'': 'extern', 'pubsubcore':'core'}, scripts = [], ... ) For sdist, bdist, bdist_egg etc, it works: the output shows module0 is in the "root" of the distribution, together with a package named "pubsubcore" which contains what is in 'pubsub/core'. HOWEVER, and here is the problem: when I do "setup.py develop" (this is on WinXP with Python 2.4.4 and the latest stable setuptools), only the contents of 'extern' get 'installed': C:\Documents and Settings\schoenb\My Documents\pubsub>setup.py develop running develop running egg_info writing external\PyPubSub.egg-info\PKG-INFO writing top-level names to external\PyPubSub.egg-info\top_level.txt writing dependency_links to external\PyPubSub.egg-info\dependency_links.txt writing manifest file 'external\PyPubSub.egg-info\SOURCES.txt' running build_ext Creating c:\python24\lib\site-packages\PyPubSub.egg-link (link to external) Adding PyPubSub 3.0a6 to easy-install.pth file Installed c:\documents and settings\schoenb\my documents\pubsub\extern Processing dependencies for PyPubSub==3.0a6 The above allows me, as the developer, to do "import module0" but not "import pubsubcore". Indeed, looking at \python24\lib\site-packages\easy-install.pth, there is only one new line added, pointing to c:\documents and settings\schoenb\my documents\pubsub\extern. There should be a second line added for the package, ie c:\documents and settings\schoenb\my documents\pubsub so that I (developer) can do "import pubsubcore", just as any user would be able to if I had done a full install. When I add the above line manually to the .pth file the "developer" installation works as expected. What am I doing wrong? Thanks, Oliver
At 10:39 AM 3/25/2008 -0400, Oliver Schoenborn wrote:
I have a problem with "setup.py develop" installation.
I have the following folder tree:
pubsub \- setup.py \- extern \- module0.py \- core \- __init__.py \- module1.py \- module2.py \- ...
I want my setup.py to install modules from pubsub/extern (such as module0.py) into site-packages, and to install the 'core' package in site-packages as pubsubcore:
setup( ... py_modules = ['module0'], packages = ['pubsubcore'], package_dir = {'': 'extern', 'pubsubcore':'core'}, scripts = [], ... )
This use of package_dir is not compatible with 'develop'. For 'develop' to work on a project, you may not have any non-empty keys in package_dir. If you want to use develop with this project, you will have to alter your directory layout so that 'core' is under 'extern', or else move the 'extern' modules out of 'extern' and up to the top-level directory, and update (or get rid of) package_dir accordingly.
Quoting "Phillip J. Eby" <pje@telecommunity.com>:
At 10:39 AM 3/25/2008 -0400, Oliver Schoenborn wrote:
I have a problem with "setup.py develop" installation.
...
This use of package_dir is not compatible with 'develop'. For 'develop' to work on a project, you may not have any non-empty keys in package_dir. If you want to use develop with this project, you will have to alter your directory layout so that 'core' is under 'extern', or else move the 'extern' modules out of 'extern' and up to the top-level directory, and update (or get rid of) package_dir accordingly.
Thanks for your quick reply. That's a pitty, would it be difficult to patch? E.g. from a list of all 'values' in the package_dir then add a line to easy-install.pth: # pseudo code: ... dirs = [os.path.join(os.getcwd(), dd) for key, dd in package_dir.iteritems() if key.find('.') < 0] for dd in dirs: easyinst_pth.write(dd+'\n') ...
At 12:07 PM 3/25/2008 -0400, Oliver Schoenborn wrote:
Quoting "Phillip J. Eby" <pje@telecommunity.com>:
At 10:39 AM 3/25/2008 -0400, Oliver Schoenborn wrote:
I have a problem with "setup.py develop" installation.
...
This use of package_dir is not compatible with 'develop'. For 'develop' to work on a project, you may not have any non-empty keys in package_dir. If you want to use develop with this project, you will have to alter your directory layout so that 'core' is under 'extern', or else move the 'extern' modules out of 'extern' and up to the top-level directory, and update (or get rid of) package_dir accordingly.
Thanks for your quick reply. That's a pitty, would it be difficult to patch? E.g. from a list of all 'values' in the package_dir then add a line to easy-install.pth:
# pseudo code: ... dirs = [os.path.join(os.getcwd(), dd) for key, dd in package_dir.iteritems() if key.find('.') < 0] for dd in dirs: easyinst_pth.write(dd+'\n') ...
That won't work correctly with respect to linking the package's .egg-info to its import path, and of course, uninstall still has to be handled. Implementing uninstall would be further complicated by the possibility of package_dir having been changed between the install and subsequent uninstall, or any intervening re-install. In short, it'd be a serious pain, and I haven't see any use cases for doing anything with package_dir besides defining a root location for packages (e.g. a src/ or lib/ directory). I have no idea why the ability to do that was added to distutils in the first place, and if I ever did a full distutils replacement, it'd likely be one of the very first things lined up against the wall and shot. ;-)
Quoting "Phillip J. Eby" <pje@telecommunity.com>:
At 12:07 PM 3/25/2008 -0400, Oliver Schoenborn wrote:
Quoting "Phillip J. Eby" <pje@telecommunity.com>:
At 10:39 AM 3/25/2008 -0400, Oliver Schoenborn wrote:
I have a problem with "setup.py develop" installation.
...
This use of package_dir is not compatible with 'develop'. For 'develop' to work on a project, you may not have any non-empty keys in package_dir. If you want to use develop with this project, you will have to alter your directory layout so that 'core' is under 'extern', or else move the 'extern' modules out of 'extern' and up to the top-level directory, and update (or get rid of) package_dir accordingly.
Thanks for your quick reply. That's a pitty, would it be difficult to patch? E.g. from a list of all 'values' in the package_dir then add a line to easy-install.pth:
# pseudo code: ... dirs = [os.path.join(os.getcwd(), dd) for key, dd in package_dir.iteritems() if key.find('.') < 0] for dd in dirs: easyinst_pth.write(dd+'\n') ...
...
In short, it'd be a serious pain, and I haven't see any use cases for doing anything with package_dir besides defining a root location for packages (e.g. a src/ or lib/ directory). I have no idea why the ability to do that was added to distutils in the first place, ...
Well, I ended up restructuring the source to reflect final installation, ie moved what was in extern into a new src folder, and move pubsubcore folder into src too. However, I still need to use package_dir: setup(... packages = [''], package_dir = {'': 'src'}, ... ) How would you do it without package_dir? Oliver
At 12:52 PM 3/25/2008 -0400, Oliver Schoenborn wrote:
I still need to use package_dir:
setup(... packages = [''], package_dir = {'': 'src'}, ... )
That's fine - that layout is fully supported by the 'develop' command.
How would you do it without package_dir?
If we were replacing the distutils, I'd simply have a "source root" option, rather than a dictionary with a magic key.
On Tue, Mar 25, 2008 at 01:22:28PM -0400, Phillip J. Eby wrote:
If we were replacing the distutils, I'd simply have a "source root" option, rather than a dictionary with a magic key.
FWIW, +1 on the "source root" option. package_dir has always seemed rather confusing to me. I used it for a source root at one time, but I just use packages now with a physical layout. It's easier to understand what's going on(not just for my sake, but others as well).
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote:
At 12:07 PM 3/25/2008 -0400, Oliver Schoenborn wrote:
Quoting "Phillip J. Eby" <pje@telecommunity.com>:
I have a problem with "setup.py develop" installation.
... This use of package_dir is not compatible with 'develop'. For 'develop' to work on a project, you may not have any non-empty keys in
At 10:39 AM 3/25/2008 -0400, Oliver Schoenborn wrote: package_dir. If you want to use develop with this project, you will have to alter your directory layout so that 'core' is under 'extern', or else move the 'extern' modules out of 'extern' and up to the top-level directory, and update (or get rid of) package_dir accordingly. Thanks for your quick reply. That's a pitty, would it be difficult to patch? E.g. from a list of all 'values' in the package_dir then add a line to easy-install.pth:
# pseudo code: ... dirs = [os.path.join(os.getcwd(), dd) for key, dd in package_dir.iteritems() if key.find('.') < 0] for dd in dirs: easyinst_pth.write(dd+'\n') ...
That won't work correctly with respect to linking the package's .egg-info to its import path, and of course, uninstall still has to be handled. Implementing uninstall would be further complicated by the possibility of package_dir having been changed between the install and subsequent uninstall, or any intervening re-install.
In short, it'd be a serious pain, and I haven't see any use cases for doing anything with package_dir besides defining a root location for packages (e.g. a src/ or lib/ directory). I have no idea why the ability to do that was added to distutils in the first place, and if I ever did a full distutils replacement, it'd likely be one of the very first things lined up against the wall and shot. ;-)
/me hums G & S chorus[1] in the background: He's got 'em on the list -- he's got 'em on the list; And they'll none of 'em be missed -- they'll none of 'em be missed. [1] http://www.cs.rice.edu/~ssiyer/minstrels/poems/135.html Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFH6S6j+gerLs4ltQ4RAhZLAKCDSxEmKnG4a0jZxH5ru0XXWqX96wCeKYYR AYIMBgAVfvME63eRIPmeBKg= =Ivo1 -----END PGP SIGNATURE-----
participants (4)
-
James William Pye -
Oliver Schoenborn -
Phillip J. Eby -
Tres Seaver