[Distutils] How to handle launcher script importability?

[Oscar Benjamin]

On 13 August 2013 20:58, Paul Moore <p.f.moore at gmail.com> wrote:
>
> On 13 August 2013 18:08, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>>
>> On 13 August 2013 17:33, Paul Moore <p.f.moore at gmail.com> wrote:
>> >
>> > On another point you mention, Cygwin Python should be using Unix-style shell
>> > script wrappers, not Windows-style exes, surely? The whole point of Cygwin
>> > is that it emulates Unix, after all... So I don't see that as an argument
>> > either way.
>>
>> So say I have a ~/bin directory where I put my scripts that I want to
>> be generally available. I install something with
>>     python setup.py install --install-scripts=~/bin
>> so that the scripts/script-wrappers go in there because I want to be
>> able to always access that program under that name. Don't be fooled by
>> the unixy tilde: I'm running ordinary Windows Python in that command
>> in git-bash, not Cygwin. Now if that folder is on PATH while I am in
>> Cygwin I can run the program with the same name if an .exe wrapper was
>> added. I can't run it with the same name if it's a .py/,bat file
>> because Cygwin doesn't have the implicit strip-the-extension PATHEXT
>> feature and can't run .bat files.
>
> Ah, OK, thanks for the clarification.
>
> In that case I can see why you'd prefer exe wrappers (or maybe cygwin bash shell wrappers, or shell aliases...). Maybe an option to still use exe wrappers is worth it - but honestly, I'd say that in that context you probably have enough expertise to understand the issue and make your own solution relatively easily.

Yes, but I'd like it if pip install some_cmd would "just work".

> What about having in your .bashrc:
>
> for prog in ls ~/bin/*.py; do
>     alias $(basename $prog .py)=$prog
> done
>
> (Excuse me if I got the precise details wrong there). OK, you need to rerun .bashrc if you add new scripts. It's not perfect. But it's not a showstopper either.

There are ways to make it work for every different environment where I
would type the command. Really though it's a pain to have to set these
things up everywhere.

Also this still doesn't work with subprocess(..., shell=False). There
are a huge range of programs that can invoke subprocesses of a given
name and I want them all to work with commands that I install from
pypi. There are good reasons to use shell=False: the subprocess
documentation contains no less than 5 warning boxes about shell=True!
This is not peculiar to Python's subprocess module: it is the
underlying Windows API calls regardless of which language the parent
process is implemented in. Here's a demo of what happens with Robert
Kern's kernprof.py script that doesn't have an .exe wrapper (on my
system; it's possible that I didn't install it with pip).

$ python
Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.call(['kernprof.py'], shell=True)  # Uses file-association
Usage: kernprof.py [-s setupfile] [-o output_file_path] scriptfile [arg] ...

2
>>> import os
>>> os.environ['PATHEXT']
'.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.PY;.PYC;.PSC1;.RB;.RBW'
>>> subprocess.call(['kernprof'], shell=True)  # Uses PATHEXT
Usage: kernprof.py [-s setupfile] [-o output_file_path] scriptfile [arg] ...

2
>>> subprocess.call(['kernprof'], shell=False)  # Needs an .exe wrapper!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "q:\tools\Python27\lib\subprocess.py", line 524, in call
    return Popen(*popenargs, **kwargs).wait()
  File "q:\tools\Python27\lib\subprocess.py", line 711, in __init__
    errread, errwrite)
  File "q:\tools\Python27\lib\subprocess.py", line 948, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
>>> subprocess.call(['kernprof.py'], shell=False)  # Needs an .exe wrapper!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "q:\tools\Python27\lib\subprocess.py", line 524, in call
    return Popen(*popenargs, **kwargs).wait()
  File "q:\tools\Python27\lib\subprocess.py", line 711, in __init__
    errread, errwrite)
  File "q:\tools\Python27\lib\subprocess.py", line 948, in _execute_child
    startupinfo)
WindowsError: [Error 193] %1 is not a valid Win32 application

Here's what happens if I put kernprof.bat next to kernprof.py (the
.bat file just @echos "running kernprof"):

>>> import subprocess
>>> subprocess.call(['kernprof'], shell=True)  # PATHEXT in action
running kernprof
0
>>> subprocess.call(['kernprof'], shell=False)  # No PATHEXT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "q:\tools\Python27\lib\subprocess.py", line 524, in call
    return Popen(*popenargs, **kwargs).wait()
  File "q:\tools\Python27\lib\subprocess.py", line 711, in __init__
    errread, errwrite)
  File "q:\tools\Python27\lib\subprocess.py", line 948, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
>>> subprocess.call(['kernprof.bat'], shell=False)  # Works but not what I want
running kernprof

>
> I do think, as I said before, that this needs some sort of policy-type PEP on the standard approach for wrapping scripts, with all the pros and cons of the various approaches documented and reviewed.

There have been so many emails on this list that I can't immediately
find it but somewhere Steve Dower of Microsoft said something like:
'''
    Don't worry. Exe wrappers are *definitely* the best solution for this.
'''
(quote is approximate and emphasis added by me).

My experience has lead me to the same conclusion as Steve. It may be
worth documenting the reasons why but make no mistake about it: .exe
wrappers of some form are the way to go.


Oscar