easy_install adds bad interpreter shebang to installed scripts
Hi *, I'm using Python 2.5 on Fedora Linux 9. If the path to the interpreter contains spaces and I install some eggs with scripts using easy_install, all installed scripts will have a non-functional shebang line like this: #!"/home/foo bar/bin/python" On Linux (and I think this is true for all Un*x flavors) you can not quote the shebang path and there is no way around this [1]. To make it more clear where the problem is I attached some kind of 'patch'. My idea was that quoting shebang paths seem to work on Windows (at least this was my impression from reading the changelogs) but on Linux there is no way to generate a working shebang line with an absolute path. Therefore it would help me if /usr/bin/env is queried. Probably this will break for some people but currently the behavior is broken for everyone. With virtualenv you have to call activate (instead of calling the installed script directly) but then everything will work fine. Do you consider this a kind of bug you might be inclined to fix? fs [1] http://groups.google.com/group/gnu.bash.bug/browse_thread/thread/861a98e53be...
At 06:15 PM 12/17/2008 +0100, Felix Schwarz wrote:
Index: setuptools/command/easy_install.py =================================================================== --- setuptools/command/easy_install.py (Revision 67824) +++ setuptools/command/easy_install.py (Arbeitskopie) @@ -1418,8 +1418,12 @@ if options: options = ' '+options if wininst: executable = "python.exe" + elif sys.platform=='win32': + executable = nt_quote_arg(executable) + elif ' ' in executable: + executable = '/usr/bin/env python' else: - executable = nt_quote_arg(executable) + executable = executable hdr = "#!%(executable)s%(options)s\n" % locals() if unicode(hdr,'ascii','ignore').encode('ascii') != hdr: # Non-ascii path to sys.executable, use -x to prevent warnings
I'm okay with the change in principle, but in practice, just dropping to "/usr/bin/env python" isn't acceptable. It should point env to the specific sys.executable, just like the "fix_jython_executable()" function does.
Phillip J. Eby wrote:
At 06:15 PM 12/17/2008 +0100, Felix Schwarz wrote:
Index: setuptools/command/easy_install.py =================================================================== --- setuptools/command/easy_install.py (Revision 67824) +++ setuptools/command/easy_install.py (Arbeitskopie) @@ -1418,8 +1418,12 @@ if options: options = ' '+options if wininst: executable = "python.exe" + elif sys.platform=='win32': + executable = nt_quote_arg(executable) + elif ' ' in executable: + executable = '/usr/bin/env python' else: - executable = nt_quote_arg(executable) + executable = executable hdr = "#!%(executable)s%(options)s\n" % locals() if unicode(hdr,'ascii','ignore').encode('ascii') != hdr: # Non-ascii path to sys.executable, use -x to prevent warnings
I'm okay with the change in principle, but in practice, just dropping to "/usr/bin/env python" isn't acceptable. It should point env to the specific sys.executable, just like the "fix_jython_executable()" function does.
Does this work? #!/usr/bin/env "/path/to/weird path/python" -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
Ian Bicking schrieb:
Does this work?
#!/usr/bin/env "/path/to/weird path/python"
No (at least not for me). As far as I understand this, no kind of quoting or escaping will work on whole shebang line because the kernel just splits the string and passes them as arguments to the application. No shell is involved there. fs
At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote:
Ian Bicking schrieb:
Does this work? #!/usr/bin/env "/path/to/weird path/python"
No (at least not for me).
Did you actually try that, as opposed to the version you showed before? There *is* a difference.
As far as I understand this, no kind of quoting or escaping will work on whole shebang line because the kernel just splits the string and passes them as arguments to the application.
You understand incorrectly; shebang parsing is platform dependent. See: http://www.in-ulm.de/~mascheck/various/shebang/ In particular, I just tested Linux (CentOS 4.3, to be precise) and found that this works: #!/bin/env /path/witha/s pace/init However, if I understand correctly, this would not work correctly on certain *BSD-derived operating systems, which would pass "/path/witha/s" and "pace/init" to env as separate arguments, instead of a single combined argument, thereby keeping it from working. It seems that OS X would be a good place to test also, especially since IIRC it is *BSD-derived.
Phillip J. Eby schrieb:
At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote:
Ian Bicking schrieb:
Does this work? #!/usr/bin/env "/path/to/weird path/python"
No (at least not for me).
Did you actually try that, as opposed to the version you showed before? There *is* a difference.
Of course I tried. :-) I just tried again and it still does not work. However I noticed that it works without any quoting for me too. Which I thought I tried before... So for me (using Fedora Linux with Python 2.5, nothing unusual) the solution would be to add /usr/bin/env <unquoted path> Probably this is the way to go forward. Let's see if the behavior is different for BSDs (I have neither a Mac nor any BSD-based system available right now). fs
On Thu, 18 Dec 2008 16:20:28 +0100, Felix Schwarz <felix.schwarz@web.de> wrote:
Phillip J. Eby schrieb:
At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote:
Ian Bicking schrieb:
Does this work? #!/usr/bin/env "/path/to/weird path/python"
No (at least not for me).
Did you actually try that, as opposed to the version you showed before? There *is* a difference.
Of course I tried. :-) I just tried again and it still does not work. However I noticed that it works without any quoting for me too. Which I thought I tried before...
So for me (using Fedora Linux with Python 2.5, nothing unusual) the solution would be to add /usr/bin/env <unquoted path>
Probably this is the way to go forward. Let's see if the behavior is different for BSDs (I have neither a Mac nor any BSD-based system available right now).
OS X results (note, ~/foo exists and is a directory): neutron:~ exarkun$ cat > foobar.sh #!/Users/exarkun/foo bar/python print 'yes' neutron:~ exarkun$ chmod u+x foobar.sh neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: /Users/exarkun/foo: bad interpreter: Permission denied neutron:~ exarkun$ cat > foobar.sh #!/Users/exarkun/foo\ bar/python print 'yes' neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: /Users/exarkun/foo\: bad interpreter: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!"/Users/exarkun/foo bar/python" print "yes" neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: "/Users/exarkun/foo: bad interpreter: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!'/Users/exarkun/foo bar/python' print 'yes' neutron:~ exarkun$ ./foobar.sh -bash: ./foobar.sh: '/Users/exarkun/foo: bad interpreter: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env /Users/exarkun/foo bar/python print 'yes' neutron:~ exarkun$ ./foobar.sh env: /Users/exarkun/foo: Permission denied neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env /Users/exarkun/foo\ bar/python print 'yes' neutron:~ exarkun$ ./foobar.sh env: /Users/exarkun/foo\: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env "/Users/exarkun/foo bar/python" print 'yes' neutron:~ exarkun$ ./foobar.sh env: "/Users/exarkun/foo: No such file or directory neutron:~ exarkun$ cat > foobar.sh #!/usr/bin/env '/Users/exarkun/foo bar/python' print 'yes' neutron:~ exarkun$ ./foobar.sh env: '/Users/exarkun/foo: No such file or directory neutron:~ exarkun$ Jean-Paul
At 10:40 AM 12/18/2008 -0500, Jean-Paul Calderone wrote:
On Thu, 18 Dec 2008 16:20:28 +0100, Felix Schwarz <felix.schwarz@web.de> wrote:
Phillip J. Eby schrieb:
At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote:
Ian Bicking schrieb:
Does this work? #!/usr/bin/env "/path/to/weird path/python"
No (at least not for me).
Did you actually try that, as opposed to the version you showed before? There *is* a difference.
Of course I tried. :-) I just tried again and it still does not work. However I noticed that it works without any quoting for me too. Which I thought I tried before...
So for me (using Fedora Linux with Python 2.5, nothing unusual) the solution would be to add /usr/bin/env <unquoted path>
Probably this is the way to go forward. Let's see if the behavior is different for BSDs (I have neither a Mac nor any BSD-based system available right now).
OS X results (note, ~/foo exists and is a directory):
Hm. So no matter what, it's not going to work on OS X; I guess that means we could just go with the way that works on Linux. ;-) It appears that OS X is doing what I'd expect a *BSD to - it's splitting the #! line on spaces and feeding each one to env as a separate argument -- quotes and all. The only other way to do this that I can think of would be to write the file with a bilingual shell script header, such that it does an exec of the appropriate Python on $0. That would work on all platforms, at the cost of incurring an extra exec -- but then, /bin/env does that too.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote:
At 10:40 AM 12/18/2008 -0500, Jean-Paul Calderone wrote:
On Thu, 18 Dec 2008 16:20:28 +0100, Felix Schwarz <felix.schwarz@web.de> wrote:
At 10:17 PM 12/17/2008 +0100, Felix Schwarz wrote:
Ian Bicking schrieb:
Does this work? #!/usr/bin/env "/path/to/weird path/python" No (at least not for me). Did you actually try that, as opposed to the version you showed before? There *is* a difference. Of course I tried. :-) I just tried again and it still does not work. However I noticed
Phillip J. Eby schrieb: that it works without any quoting for me too. Which I thought I tried before...
So for me (using Fedora Linux with Python 2.5, nothing unusual) the solution would be to add /usr/bin/env <unquoted path>
Probably this is the way to go forward. Let's see if the behavior is different for BSDs (I have neither a Mac nor any BSD-based system available right now). OS X results (note, ~/foo exists and is a directory):
Hm. So no matter what, it's not going to work on OS X; I guess that means we could just go with the way that works on Linux. ;-)
It appears that OS X is doing what I'd expect a *BSD to - it's splitting the #! line on spaces and feeding each one to env as a separate argument -- quotes and all.
The only other way to do this that I can think of would be to write the file with a bilingual shell script header, such that it does an exec of the appropriate Python on $0. That would work on all platforms, at the cost of incurring an extra exec -- but then, /bin/env does that too.
I would way rather see that kind of solution than using 'env': scripts installed by easy_install should *not* use whatever python happens to be found at the moment on PATH. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJSxvn+gerLs4ltQ4RAoGpAJ41Xur84yvoRyVjmOxI041xqp1lgQCeK+qG ODdMvlZ2S2SUWIvMUOlag00= =ZgH0 -----END PGP SIGNATURE-----
Tres Seaver wrote:
I would way rather see that kind of solution than using 'env': scripts installed by easy_install should *not* use whatever python happens to be found at the moment on PATH.
I agree that's necessary, but I don't think anyone has been proposing that (well, except the initial proposal). I'm guessing the script in this case could look like: #!/bin/sh exec "path/to/python" -c "everything that would normally be in the body of the script" -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
At 01:33 AM 12/19/2008 -0600, Ian Bicking wrote:
Tres Seaver wrote:
I would way rather see that kind of solution than using 'env': scripts installed by easy_install should *not* use whatever python happens to be found at the moment on PATH.
I agree that's necessary, but I don't think anyone has been proposing that (well, except the initial proposal). I'm guessing the script in this case could look like:
#!/bin/sh exec "path/to/python" -c "everything that would normally be in the body of the script"
Actually, I was thinking more like: #!/bin/sh # standard easy_install comment lines here """:" exec '/path/to/python' "$0" "$@" """ # rest of the code here The idea being to preserve the inspectable info that easy_install puts in the first few lines of script comments. And of course, to properly pass through sys.argv, etc. The path to sys.executable also needs to be properly escaped by the code writing the script.
At 03:00 PM 12/17/2008 -0600, Ian Bicking wrote:
Does this work?
#!/usr/bin/env "/path/to/weird path/python"
Depends on the Unix, unfortunately. Some would need it without the quotes to work, and some probably won't work with or without it. My point was more that it should at least try for something specific, i.e. python2.5, rather than just 'python', and perhaps verify that searching PATH for that string yields sys.executable. If not, then you already know it's not going to work right, and you might as well issue a warning right away.
Phillip J. Eby schrieb:
I'm okay with the change in principle, but in practice, just dropping to "/usr/bin/env python" isn't acceptable. It should point env to the specific sys.executable, just like the "fix_jython_executable()" function does.
Maybe I misunderstood you but I think this is not possible because the executable path may contain spaces. The shebang line will be split along spaces so arguments must not contain spaces (they will be considered as two separate arguments). fs
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Felix Schwarz wrote:
I'm using Python 2.5 on Fedora Linux 9. If the path to the interpreter contains spaces and I install some eggs with scripts using easy_install, all installed scripts will have a non-functional shebang line like this: #!"/home/foo bar/bin/python"
On Linux (and I think this is true for all Un*x flavors) you can not quote the shebang path and there is no way around this [1].
To make it more clear where the problem is I attached some kind of 'patch'. My idea was that quoting shebang paths seem to work on Windows (at least this was my impression from reading the changelogs) but on Linux there is no way to generate a working shebang line with an absolute path. Therefore it would help me if /usr/bin/env is queried.
Probably this will break for some people but currently the behavior is broken for everyone. With virtualenv you have to call activate (instead of calling the installed script directly) but then everything will work fine.
Do you consider this a kind of bug you might be inclined to fix?
I *never* want to see /usr/bin/env in a generated script's shebang: I want the script to use the python which was used to install it, and no other./ - From my POV, I don't understand why on a Linux box you would choose to ape Windows and use broken paths to important system components: "just say no" to paths with spaces works for me. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJSY/o+gerLs4ltQ4RAotoAJ9AQQPiWbu4B4wE8/206AAa+inCUQCfaJ1i vN1giPJfE6M2vCBlYtZANTQ= =/Y9F -----END PGP SIGNATURE-----
participants (6)
-
Felix Schwarz -
Felix Schwarz -
Ian Bicking -
Jean-Paul Calderone -
Phillip J. Eby -
Tres Seaver