[Distutils] How to handle launcher script importability?

[Paul Moore]

On 12 August 2013 19:14, Jason R. Coombs <jaraco at jaraco.com> wrote:

> My preference is to reject the idea of the side-by-side executable
> launcher.
> There are several downsides that I'm trying to avoid by moving away from
> the
> executable:
>

Using a dedicated filetype and associated systemwide launcher is the ideal
solution in many ways, but the behaviour is subtle, shell-dependent, and
under-documented. I have been adding .PY to PATHEXT for ages now, and
running .py scripts as commands with no issues - but people have raised
objections to the proposal that we do this for exe wrappers, and I don't
have convincing answers in all cases (mostly because "I've never seen that
case" is often my response...). I can't recall details, but look back over
recent threads on distutils-sig and python-dev that I've been involved in,
for details.

Also, a dedicated filetype won't be registered for users of older Pythons -
unless they install the latest launcher. Whether that is an issue to be
concerned about needs to be agreed. (Backward compatibility vs making
progress, I guess...)


> If it's '.py*', I don't see why it's not reasonable to allow omission of
> the
> shebang, and assume the default python. After encountering and now
> understanding the subtle import semantics, I'm hoping that this new
> extension
> can also be used in my personal 'scripts' collection to serve the same
> purpose
> it does for setuptools console entry points. I guess one could require
> #!/usr/bin/python in each, but that seems superfluous on Windows. I don't
> feel
> at all strongly on this point.
>

IIRC, the shebang can be omitted, and in that case you get the launcher's
configured default Python (from either py.ini, or built in). I tend to add
/usr/bin/python{2,3} (or /usr/bin/env python if I want to respect PATH) for
Unix compatibility even though I never expect the scripts to be used on
Unix.

Also, in my mind, this approach is most directly addressing the fundamental
> challenge (distinguishing a (executable) script from a module) in much the
> way
> Unix has previously enjoyed.
>

Agreed - and I think that's a more important point than the merely
technical issue of whether it manipulates sys.path the way we want.


> I'm warming up to this idea a bit, especially how it supports the most
> elegant
> approach but degrades gracefully. Some questions that arise:
>
> Where would Setuptools expect to find these launchers? Would it expect
> them to
> be present on the system? Would it symlink or hardlink them or simply
> copy? Is
> py.exe subject to the 'looks like installer' behavior, such that it would
> need
> a manifest?
>
> I still feel like this approach would require substantial special-casing,
> but
> since it provides a transition to the simple, elegant approach, I'm not
> opposed.
>

You ask important questions (and I don't know the answers :-)). I suggested
(in the issue where I implemented the wrapper functionality) that we add a
stdlib module that "knows" where the wrapper is and can wrap a script for
you (something like distlib's "distlib.script" module). But that felt a bit
over-engineered and the idea was abandoned almost immediately.

In practical terms, of course, there are many, many ways of writing
wrappers, depending on what you want them to do, and how manageable you
want to make the job of manipulating them. The current setuptools wrappers
make manipulating them easy (it's just an exe plus a script, with related
names) at the cost of needing multiple files. You could append the script
to the exe, which is a one-file solution but harder to edit the script. You
could append a zipfile to the exe, which has the advantage that you can
prepend arbitrary content to a zipfile, so there are no "mixed content"
issues to worry about, in theory. The exe would be coded slightly
differently in each case, of course, so there's a maintenance cost...

Also, of course, as was mentioned elsewhere in the thread, you need
separate wrapper exes for every architecture/platform you intend to
support. And unless the wrappers are added at install time, wrapped scripts
change a platform-neutral script into a platform-specific exe.

I would still like to see the standard be registered .pye (I'm happy with a
bikeshed of this colour) and .pwe extensions which are added to PATHEXT and
associated with the launcher. But someone would need to collect and
document the issues which have been raised with this approach, confirm
which ones are real and which are merely potential, offer solutions for the
real ones and confirm that the potential ones don't actually happen (or
they do!). I won't have the time to do this in the near future - but I
could consider it in a month or two if someone reminds me. It should
probably be a PEP, although I'm not 100% sure what that PEP would propose
(what changes in Python and wording in a PEP would be needed for you to
change setuptools to abandon wrappers, for instance? And why would you need
a PEP?)

Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130812/d45ad862/attachment-0001.html>