[Python-Dev] Draft PEP and reference implementation of a Python launcher for Windows

[Mark Hammond]

Thanks for the feedback!

On 19/03/2011 7:44 PM, Glenn Linderman wrote:
> Not all of the ideas below are complementary to each other, some are
> either or, to allow different thoughts to be inspired or different
> directions to be taken.
>
> Thanks for starting a PEP.
>
> On 3/18/2011 11:02 PM, Mark Hammond wrote:
>
>>      The launcher should be as simple as possible (but no simpler.)
>
> The launcher could be simpler if it isn't used for launching interactive
> interpreters as well as script references via Windows associations (more
> about that after the next quote).
>
> The launcher could be simpler if it just read through the file of its
> first parameter until it finds a line starting with #@ (process as a
> Windows version of Unix #!) or starting without a # (error case). This
> avoids the need parse such lines.

I don't agree with that at all.  The rules for the shebang parsing are 
very simple - see the man-page I reference in the pep - the file must 
start with the characters '#!' and the entire command must be < 127 
chars.  The only real "parsing" needed is to check if the specified 
command starts with one of 2 fixed strings.  I believe this is simpler 
than parsing multiple lines of the file.

 > Remember, the typical
> Windows user is not likely to place a #! line in their scripts in the
> first place, so teaching them what a Unix #! line is, and how the
> parameter after it should be something that Windows doesn't even use,
> and the launcher has to work hard to interpret, is not as simple as
> possible.

I disagree with various aspects of that - the "typical Windows user" is 
not going to add a shebang or variation of to their Python source files, 
period.  IMO, the kind of user who would is already somewhat likely to 
know of the shebang convention, so it would not be foreign.  Those 
remaining who are not familiar with it can simply be pointed at existing 
docs etc for the shebang line and their new knowledge now works on more 
than Windows (and more than Python on non-Windows platforms)

> The launcher could be simpler if the Python installer placed versioned
> Python executables on the PATH. Unfortunately, historically it hasn't.
> If it did, would, or the launcher installer would place them there for
> pre-existing versions of Python, then the launcher could work by
> launching the appropriate version of Python, expecting Windows to find
> it on the PATH. The PEP doesn't address the level of internal complexity
> of the launcher necessary to find where Python has been installed,
> neither for CPython, nor for the alternate implementations to be supported.

The PEP intentionally doesn't, but the implementation does - it already 
does exactly that for CPython.  Other implementation may need a 
different strategy, but we can cross that bridge when we come to it.

> The launcher could be simpler if a directory \usr\bin were created under
> Windows Program Files, placed on the PATH, and %ProgramFiles% prepended
> to the Unix #! line, with the Python/Jython/Cython installers placing
> appropriately versioned executables in that directory. Could even start
> a trend for programs ported from Unix. One could even place an "env"
> program there, for more simplicity.

Again, I disagree - I think in practice the code would be more complex, 
and having Python assert it "owns" such directories and executables is a 
can of worms we should avoid.

>
>>      * When used to launch an interactive Python interpreter, the launcher
>>        will support the first command-line argument optionally be a
>>        version specifier in the form "-n[.n]" (where n is a single
>>        integer) to nominate a specific version be used.  For example,
>>        while "py.exe" may locate and launch the latest Python 2.x
>>        implementation installed, a command-line such as "py.exe -3" could
>>        specify the latest Python 3.x implementation be launched, while
>>        "py.exe -2.6" could specify Python 2.6 be located and launched.
>>        If a Python 2.x implementation is desired to be launched with the
>>        -3 flag, the command-line would need to be similar to "py.exe -2
>>        -3" (or the specific version of Python could obviously be
>>        launched manually without use of this launcher.)
>
> I think that a python launcher that is "on the PATH" that could be used
> to launch an interactive Python, should be different than one that is
> used to launch XXXX.py[w] scripts.

I believe that if you know the Python you want is already on the PATH, 
you should just use 'python.exe' instead of 'py.exe'.  I don't see any 
reason to use this launcher for interactive Python sessions where this 
is the case.

> 1) python should be invoked interactively by typing "python" or
> "pythonX[.Y]" at the CMD prompt, not "py". This can be done without a
> launcher, if appropriate versioned pythons are placed on the PATH. The
> launcher is really and only needed for handling XXXX.py[w] scripts,
> which, in the Windows way of thinking, can only be associated with one
> specific, system-wide configured version of Python (presently, the
> latest one wins). The script itself is not examined to modify such an
> association. The Unix !# line provides such modification on Unix.

OK, I think we found something we can agree on :)  If the python.exe you 
want is on your path, just ignore this launcher for anything other than 
file associations.

> 2) If the launcher provides command line options for the "benefit" of
> launching interactive Python interpreters, those command line options
> can have data puns with script names, or can conflict with actual Python
> options. I believe Python 2 already has a -3 option, for example. And
> Windows users are not trained that "-" introduces an option syntax, but
> rather "/". Most _programmer_ users would probably be aware of "-" as an
> option syntax, but Python is used as a language for non-programmers in
> some circles, and few Windows non-programmers understand "/" much less
> "-" and not even command lines very well. So not using a launcher for
> launching interactive Python sidesteps all that: Python itself is
> introduced and taught, and no need to teach about (or even have)
> launcher options that could possibly conflict and confuse, in addition
> to Python options that may conflict with script names already. (I have
> seen lots of Windows users use leading punctuation characters on
> filenames to affect sort order and grouping of files in My Documents,
> not knowing they can create subdirectories/subfolders, or not wanting to
> bother with them, since all their applications default to opening things
> from My Documents.)

I'm not 100% sure of the points you are trying to make above, but the 
gist of it seems to be the same as (1) - use python.exe directly if you 
prefer - in which case I agree.  Obviously if people choose to use the 
new launcher interactively they are doing so because they see some 
benefit and would be willing to understand how it works.  So I don't see 
any problems here as I'm not advocating it would replace "python.exe" in 
interactive scenarios except where the user actively chooses to.

> 3) Unix !# lines can have embedded options after the program name on the
> line. Such options would be another source of potential conflict with
> launcher options, if the launcher has options for use with launching
> interactive interpreters.
>
> Item 3 is also an issue for the PEP even apart from its use as an
> interactive Python launcher; since options may exist on the Unix #!
> line, a discussion of how and if they are handled by the launcher should
> be included in the PEP.

I believe a reference to the execve man-page and the note in the PEP 
that we will support the description there, including limitations, is 
enough - but I'm happy to change this if people agree it is 
underspecified or confusing.

>
>>      * Environment varialbes will be used to override the semantics for
>>        determining exactly what version of Python will be used.  For
>>        example, while a shebang line of "/usr/bin/python2" will
>>        automatically locate a Python 2.x implementation, an environment
>>        variable can override exactly which Python 2.x implementation will
>>        be chosen.  Similarly for "/usr/bin/python" etc.
>
> Clarify if environment variables can be used to override semantics for
> shebang lines of the form "/usr/bin/python2.x".

On re-reading the PEP, I notice I deleted something which is important 
on the mistaken belief the execve man page would make it clear.  Only 
the first command-line argument will be checked for a shebang line.  If 
the first argument is an option (ie, starts with a '-'), nothing is 
examined for a shebang line.  This is what the reference implementation 
does and I'll be sure to update the PEP to reflect this.

The PEP does state the optional -N.N argument must be the first 
argument.  Therefore, the presence of the -N.N argument will avoid any 
shebang processing at all so can't impact the version selected via the 
shebang line.  This makes sense to me as someone explicitly executing 
"py.exe -3 foo.py" is explicitly overriding the version they want to run 
the script with.

> If alternate implementations are to be supported, additional virtual
> commands will be required, not just these two. Each one adds complexity
> to the launcher.

This is true - each implementation would need custom code to sniff out 
the requested implementation version.  I don't think this is a burden 
though and can be worked around by using fully-qualified executable 
names in the shebang line.

>>      Non-virtual shebang lines should be discouraged as they make the
>>      script specific to a specific Windows installation. However, they
>>      are supported for maximum flexibility.
>
> This is a false statement.

I believe it is a true statement - needing words like "likely" and 
"most" to dispute it just demonstrates that IMO.  Note that I don't 
dispute your statements, but instead believe that "most" simply isn't 
good enough - I want "always".

Unfortunately, it is clear we disagree on some fundamental points and I 
can't see a way to change my PEP that would keep both you and I happy. 
Therefore, the only reasonable resolution would be for you to draft a 
competing PEP and reference implementation which python-dev can 
informally vote on.

Cheers,

Mark