I dont know if this is the right place to ask but I know i'v read a couple of mails on the list about setuptools so I will give it a try. I seem to have a problem with setuptools. I install all packages with --home=~ (using .pydistutils.cfg) . this means I need to use require to import a egg, which is fine. But I cant import pkg_resources.require since it is installed in ~ and needs to be required itself. cheers elvelind grandin
At 09:07 PM 9/13/2005 +0200, Elvelind Grandin wrote:
I seem to have a problem with setuptools. I install all packages with --home=~ (using .pydistutils.cfg) . this means I need to use require to import a egg, which is fine. But I cant import pkg_resources.require since it is installed in ~ and needs to be required itself.
See: http://cvs.sourceforge.net/viewcvs.py/python/python/nondist/sandbox/setuptools/EasyInstall.txt?rev=HEAD&view=auto under "Non-Root Installation" for details of the best way to do this; it may require a slight adjustment to your layout, but it will then allow you to use .pth files in your home-based setup. You will have to clear out your --home setup from .pydistutils.cfg, though. (Also, you may need the latest snapshot of setuptools (0.6a1c2) for the described installation procedure to work, as older versions may try to install to the "real" Python installation instead of your alternate home, when following the procedure given on the page above.)
Phillip J. Eby wrote:
See:
under "Non-Root Installation" for details of the best way to do this; it may require a slight adjustment to your layout, but it will then allow you to use .pth files in your home-based setup. You will have to clear out your --home setup from .pydistutils.cfg, though.
Do you think it might be possible for ez_setup.py and easy_install.py to interpret the --home and --prefix options sensibly? Two people I've tried to convert to setuptools work exclusively on Linux with non-root installations. --prefix was the first thing they tried because it's the standard way for distutils to do this kind of thing. I suspect many other people will do the same. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
At 04:44 PM 9/13/2005 -0700, Robert Kern wrote:
Phillip J. Eby wrote:
See:
under "Non-Root Installation" for details of the best way to do this; it may require a slight adjustment to your layout, but it will then allow you to use .pth files in your home-based setup. You will have to clear out your --home setup from .pydistutils.cfg, though.
Do you think it might be possible for ez_setup.py and easy_install.py to interpret the --home and --prefix options sensibly?
They already respect these options. The problem is that Python's 'site' module will *only* recognize .pth files in a very limited number of places, which means that installing setuptools itself won't work with those options.
Two people I've tried to convert to setuptools work exclusively on Linux with non-root installations. --prefix was the first thing they tried because it's the standard way for distutils to do this kind of thing. I suspect many other people will do the same.
--prefix only makes sense if you've set things up the way shown on the page above, and if you set it up that way, you won't need --prefix. I realize the situation sucks, but unfortunately I'm not the one who made it suck, and am not in a position to fix it. Guido previously vetoed Bob Ippolito's proposal for more sane .pth processing on platforms other than Mac.
Phillip J. Eby wrote:
At 04:44 PM 9/13/2005 -0700, Robert Kern wrote:
Phillip J. Eby wrote:
See:
under "Non-Root Installation" for details of the best way to do this; it may require a slight adjustment to your layout, but it will then allow you to use .pth files in your home-based setup. You will have to clear out your --home setup from .pydistutils.cfg, though.
Do you think it might be possible for ez_setup.py and easy_install.py to interpret the --home and --prefix options sensibly?
They already respect these options. The problem is that Python's 'site' module will *only* recognize .pth files in a very limited number of places, which means that installing setuptools itself won't work with those options.
I wasn't suggesting that it try to modify the .pth-enabled directories itself, but that easy_install.py and ez_setup.py (especially) should be able to calculate --site-dirs from --prefix. And that ez_setup.py should actually take the --prefix option on the command line gracefully. There are many systems which use other prefixes and have already set up their sitecustomize.py (or site.py) to enable .pth handling for them. Lots of packages use .pth's. [From a CVS checkout of setuptools on a Debian unstable system:] """ $ sudo python2.4 ez_setup.py --prefix=/usr/local . usage: ez_setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: ez_setup.py --help [cmd1 cmd2 ...] or: ez_setup.py --help-commands or: ez_setup.py cmd --help error: option --prefix not recognized """ When the prefix is set in ~/.pydistutils.cfg, the egg gets installed, but not the .pth's: """ $ sudo ./ez_setup.py . Processing . Running setup.py -q bdist_egg --dist-dir /usr/local/kern_src/python/nondist/sandbox/setuptools/egg-dist-tmp-85Xeyn Installing easy_install.py script to /usr/local/bin Installed /usr/local/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg Because this distribution was installed --multi-version or --install-dir, before you can import modules from this package in an application, you will need to 'import pkg_resources' and then use a 'require()' call similar to one of these examples, in order to select the desired version: pkg_resources.require("setuptools") # latest installed version pkg_resources.require("setuptools==0.6a1c2") # this exact version pkg_resources.require("setuptools>=0.6a1c2") # this version or higher Processing dependencies for setuptools==0.6a1c2 """ But if --site-dirs is also provided, everything works: """ $ sudo python2.4 ez_setup.py --site-dirs=/usr/local/lib/python2.4/site-packages . Processing . Running setup.py -q bdist_egg --dist-dir /usr/local/kern_src/python/nondist/sandbox/setuptools/egg-dist-tmp--GRqoe Adding setuptools 0.6a1c2 to easy-install.pth file Installing easy_install.py script to /usr/local/bin Installed /usr/local/lib/python2.4/site-packages/setuptools-0.6a1c2-py2.4.egg Processing dependencies for setuptools==0.6a1c2 """ My request is that if --prefix is provided, then the value of --site-dirs be calculated from it. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
At 06:43 PM 9/13/2005 -0700, Robert Kern wrote:
My request is that if --prefix is provided, then the value of --site-dirs be calculated from it.
I'd suggest using PYTHONHOME instead, since that will actually make Python run from that directory. However, my main concern is not making these path calculations any more screwy than they already have to be, as they are already quite complex. So, if you can suggest a patch to implement your desired behavior that doesn't break anything else and doesn't make my head explode upon reviewing it, I'll be happy to check it in. :)
Phillip J. Eby wrote:
At 06:43 PM 9/13/2005 -0700, Robert Kern wrote:
My request is that if --prefix is provided, then the value of --site-dirs be calculated from it.
I'd suggest using PYTHONHOME instead, since that will actually make Python run from that directory.
That's not what these users need, however. Python is installed to /usr and the standard library and all of the third-party packages that are part of Debian's packaging system go under /usr/lib/python2.x/ . Third-party packages that are compiled by the user go under /usr/local/lib/python2.x/site-packages/ . The way everybody accomplishes this is by setting --prefix=/usr/local for distutils.
However, my main concern is not making these path calculations any more screwy than they already have to be, as they are already quite complex. So, if you can suggest a patch to implement your desired behavior that doesn't break anything else and doesn't make my head explode upon reviewing it, I'll be happy to check it in. :)
The path part isn't hard. If --prefix or --home is provided, the install command will calculate the path for you and place it in install.install_lib . The hard part seems to be figuring out whether the user explicitly provided --prefix or --install-lib or neither. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
At 09:42 PM 9/13/2005 -0700, Robert Kern wrote:
Phillip J. Eby wrote:
At 06:43 PM 9/13/2005 -0700, Robert Kern wrote:
My request is that if --prefix is provided, then the value of --site-dirs be calculated from it.
I'd suggest using PYTHONHOME instead, since that will actually make Python run from that directory.
That's not what these users need, however. Python is installed to /usr and the standard library and all of the third-party packages that are part of Debian's packaging system go under /usr/lib/python2.x/ . Third-party packages that are compiled by the user go under /usr/local/lib/python2.x/site-packages/ . The way everybody accomplishes this is by setting --prefix=/usr/local for distutils.
Well, as long as it ends up on sys.path *and* processes .pth files, it should be fine. Question: are these /usr/local/* versions on sys.path *before* the /usr versions? Or after?
However, my main concern is not making these path calculations any more screwy than they already have to be, as they are already quite complex. So, if you can suggest a patch to implement your desired behavior that doesn't break anything else and doesn't make my head explode upon reviewing it, I'll be happy to check it in. :)
The path part isn't hard. If --prefix or --home is provided, the install command will calculate the path for you and place it in install.install_lib . The hard part seems to be figuring out whether the user explicitly provided --prefix or --install-lib or neither.
I'll take your word for it. :) Again, I'll be happy to look at a patch, but I don't have the bandwidth to figure out this particular setup. One possible hack: assume that any directory on sys.path whose basename is "site-packages" is a valid --site-dir. The worst that can do is fail. :)
Phillip J. Eby wrote:
At 09:42 PM 9/13/2005 -0700, Robert Kern wrote:
Phillip J. Eby wrote:
At 06:43 PM 9/13/2005 -0700, Robert Kern wrote:
My request is that if --prefix is provided, then the value of --site-dirs be calculated from it.
I'd suggest using PYTHONHOME instead, since that will actually make Python run from that directory.
That's not what these users need, however. Python is installed to /usr and the standard library and all of the third-party packages that are part of Debian's packaging system go under /usr/lib/python2.x/ . Third-party packages that are compiled by the user go under /usr/local/lib/python2.x/site-packages/ . The way everybody accomplishes this is by setting --prefix=/usr/local for distutils.
Well, as long as it ends up on sys.path *and* processes .pth files, it should be fine.
Not for installing setuptools itself with ez_setup.py . It doesn't recognize /usr/local/... as a site-dir so it doesn't install the .pth files. It assumes that it must be a multi-version install.
Question: are these /usr/local/* versions on sys.path *before* the /usr versions? Or after?
After. Does it matter? /usr/local/* doesn't contain the stdlib; it doesn't override anything.
However, my main concern is not making these path calculations any more screwy than they already have to be, as they are already quite complex. So, if you can suggest a patch to implement your desired behavior that doesn't break anything else and doesn't make my head explode upon reviewing it, I'll be happy to check it in. :)
The path part isn't hard. If --prefix or --home is provided, the install command will calculate the path for you and place it in install.install_lib . The hard part seems to be figuring out whether the user explicitly provided --prefix or --install-lib or neither.
I'll take your word for it. :) Again, I'll be happy to look at a patch, but I don't have the bandwidth to figure out this particular setup.
I've attached my original attempt. As I mentioned it doesn't actually differentiate between the cases where --prefix is provided and --install-dir is.
One possible hack: assume that any directory on sys.path whose basename is "site-packages" is a valid --site-dir. The worst that can do is fail. :)
That's probably a decent rule. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter Index: setuptools/command/easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v retrieving revision 1.28 diff -u -r1.28 easy_install.py --- setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 +++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ -141,6 +141,10 @@ self.set_undefined_options('install', ('record', 'record')) normpath = map(normalize_path, sys.path) self.all_site_dirs = get_site_dirs() + install_cmd = self.distribution.get_command_obj('install') + if install_cmd.prefix is not None: + self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) + if self.site_dirs is not None: site_dirs = [ os.path.expanduser(s.strip()) for s in self.site_dirs.split(',')
At 11:27 PM 9/13/2005 -0700, Robert Kern wrote:
Index: setuptools/command/easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v retrieving revision 1.28 diff -u -r1.28 easy_install.py --- setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 +++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ -141,6 +141,10 @@ self.set_undefined_options('install', ('record', 'record')) normpath = map(normalize_path, sys.path) self.all_site_dirs = get_site_dirs() + install_cmd = self.distribution.get_command_obj('install') + if install_cmd.prefix is not None: + self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) + if self.site_dirs is not None: site_dirs = [ os.path.expanduser(s.strip()) for s in self.site_dirs.split(','_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Your patch came through horribly mangled, as you can see. You probably need to make it an attachment rather than pasting it into your email.
Phillip J. Eby wrote:
At 11:27 PM 9/13/2005 -0700, Robert Kern wrote:
Index: setuptools/command/easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v retrieving revision 1.28 diff -u -r1.28 easy_install.py --- setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 +++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ -141,6 +141,10 @@ self.set_undefined_options('install', ('record', 'record')) normpath = map(normalize_path, sys.path) self.all_site_dirs = get_site_dirs() + install_cmd = self.distribution.get_command_obj('install') + if install_cmd.prefix is not None: + self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) + if self.site_dirs is not None: site_dirs = [ os.path.expanduser(s.strip()) for s in self.site_dirs.split(','_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Your patch came through horribly mangled, as you can see. You probably need to make it an attachment rather than pasting it into your email.
Odd. It *was* sent as an attachment. Thunderbird on OS X sees it fine through GMane. Maybe GMane mangled it somewhere along the line. Or perhaps line-endings are getting confused. I'm cc'ing this to you with an attached patch that uses CRLF line endings. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter Index: setuptools/command/easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v retrieving revision 1.28 diff -u -r1.28 easy_install.py --- setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 +++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ -141,6 +141,10 @@ self.set_undefined_options('install', ('record', 'record')) normpath = map(normalize_path, sys.path) self.all_site_dirs = get_site_dirs() + install_cmd = self.distribution.get_command_obj('install') + if install_cmd.prefix is not None: + self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) + if self.site_dirs is not None: site_dirs = [ os.path.expanduser(s.strip()) for s in self.site_dirs.split(',')
At 06:08 PM 9/14/2005 -0700, Robert Kern wrote:
Index: setuptools/command/easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/easy_install.py,v retrieving revision 1.28 diff -u -r1.28 easy_install.py --- setuptools/command/easy_install.py 3 Sep 2005 04:50:05 -0000 1.28 +++ setuptools/command/easy_install.py 14 Sep 2005 06:17:28 -0000 @@ -141,6 +141,10 @@ self.set_undefined_options('install', ('record', 'record')) normpath = map(normalize_path, sys.path) self.all_site_dirs = get_site_dirs() + install_cmd = self.distribution.get_command_obj('install') + if install_cmd.prefix is not None: + self.all_site_dirs.append(normalize_path(install_cmd.install_lib)) + if self.site_dirs is not None: site_dirs = [ os.path.expanduser(s.strip()) for s in self.site_dirs.split(',')
This code will break on other people's installations, in that it will not be able to tell you're installing to a non-"site" directory unless you explicitly provide an --install-dir on the command line. It will thus create a non-working easy-install.pth, and will fail to warn the user that 'require()' is needed.
Phillip J. Eby wrote:
This code will break on other people's installations, in that it will not be able to tell you're installing to a non-"site" directory unless you explicitly provide an --install-dir on the command line. It will thus create a non-working easy-install.pth, and will fail to warn the user that 'require()' is needed.
I know. I said it was not satisfactory and that the real tricky part wasn't calculating the path but determining whether --prefix was explicitly provided or not. Ooh! But if install_cmd.prefix != sys.prefix, then --prefix must have been explicitly set by the user! Eureka! Now I just have to figure out a way to find out if --install-dir wasn't set.... -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
At 09:39 PM 9/14/2005 -0700, Robert Kern wrote:
Phillip J. Eby wrote:
This code will break on other people's installations, in that it will not be able to tell you're installing to a non-"site" directory unless you explicitly provide an --install-dir on the command line. It will thus create a non-working easy-install.pth, and will fail to warn the user that 'require()' is needed.
I know. I said it was not satisfactory and that the real tricky part wasn't calculating the path but determining whether --prefix was explicitly provided or not.
Ooh! But if install_cmd.prefix != sys.prefix, then --prefix must have been explicitly set by the user! Eureka! Now I just have to figure out a way to find out if --install-dir wasn't set....
Maybe this is a dumb question, but if this is a Debian-specific issue, can't you just create a Debian package that combines the setuptools .egg and .pth files with a '/usr/lib/python2.4/distutils/distutils.cfg' file that contains the right options? Then, --prefix and --site-dirs would both be unnecessary at the CLI level.
Phillip J. Eby wrote:
Maybe this is a dumb question, but if this is a Debian-specific issue, can't you just create a Debian package that combines the setuptools .egg and .pth files with a '/usr/lib/python2.4/distutils/distutils.cfg' file that contains the right options? Then, --prefix and --site-dirs would both be unnecessary at the CLI level.
I'm not about to request for a change in Debian's Python policy. But now that I've had a chance to review the new documentation (much improved! Thanks!), I think that the easiest route is to tell people to set their prefix and site-dirs in their ~/.pydistutils.cfg . And no, it's not just Debian. One of my complainants uses Fedora Core, and the other uses Debian, but maintains his self-built Python packages using Stow. I just use Debian as an example because I'm familiar with it. This is mostly an issue for people on various Unix-type setups with package managers. For example, Ian Bicking has the same issue with FreeBSD and comes to the same distutils.cfg solution: http://blog.ianbicking.org/alternate-python-install-dir.html -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter
participants (3)
-
Elvelind Grandin -
Phillip J. Eby -
Robert Kern