[Distutils] How to handle launcher script importability?

[Paul Moore]

On 13 August 2013 23:30, Chris Barker - NOAA Federal
<chris.barker at noaa.gov>wrote:

> On Tue, Aug 13, 2013 at 2:27 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>
> >> 3) I'd rather not have to mess with PATHEXT, and I particularly don't
> >> want to have to tell my students to do it -- environment variables are
> >> a pain, and somehow PATHEXT has been fragile for me (and I don't use
> >> Cygwin)
> >
> > Nobody is suggesting that end users mess with PATHEXT. The proposal is
> that
> > the Python installer does this
>
> ouch! I don't think I'd want PATHEXT set for *.py files -- I'd rather
> they get opened by an editor by default than run...or is point+click
> behavior different than command line -- shows you how well I know
> Windows.
>

It's OK, PATHEXT has nothing to do with double clicks - that's the file
associations (and "run the script" is the default action set by the
installer, and has been for many, many versions).

>
> > (indeed, that's been done for Python 3.4, I
> > don't know if the installer for the standalone launcher has been updated
> in
> > the same way yet).
>
> There are those of us still in the 2.7 world -- and I suspect for a good
> while.
>

I understand this. It's the biggest issue here, that any changes have to
not forget users of older Pythons. Personally, I tend to err towards a view
that we improve things for current Pythons (3.3+) and make sure we don't
ruin things for older versions, possibly by providing workarounds the user
needs to apply. I view "install the standalone launcher" to be in this
category of workaround. There are others who are more conservative, so
don't worry your views are well represented!

> I'm suggesting that we collect specifics on any "fragility" (can you
> provide
> > details of what has gone wrong for you?)
>
> well, for PATHEXT, the env variable has to be set right, and Windows
> kind of hides all that from you -- it's really a pain to edit them by
> hand, so if the installer doesn't do it, or someone re-builds their
> registry or profile, or what have you, then it'll break. Oh and the
> cygwin (and who know what other shell alternatives) issue. At least we
> can pretty much count on an exe running if the shell can find it..
>

Thanks. I'd class most of that as relatively non-specific or fixed. Users
shouldn't have to set PATHEXT if the installer does it, and in any case I'd
like to clearly understand what sort of users are writing command line
Python scripts that they want to run transparently as if they were
executables, who still wouldn't know how to set an environment variable (or
be able to understand whatever documentation we produce on how to do it).

The cygwin issue has been mentioned/addressed elsewhere, but thanks for
that. It's certainly something that needs to be weighed in the balance.

Fragiity for the exe approach -- all I know is that the setuptools
> binary was proken a while back -- but setuptools itself was in a bit
> of a void of not-quite-sure-if-its maintained, and
> not-sure-even-how-to-report-a-bug state. It seems that setuptools (or
> whatever this will be part of) has now been adopted by the core Python
> community, so we can expect good support -- let's hope so. Anyway,
> whether the exe approach was more or less fragile than anyting else,
> is sure was harder to debug/fix for anyone that doesn't know Windows
> and non-python development well.
>

My personal experience with the exes:

1. If the shebang line in the script gets corrupted, the resulting error is
misleading and extremely hard to debug.
2. The existence of the exe in an installer makes it platform-specific,
which is a problem for otherwise portable scripts/packages.
3. To use the exes, you really need to create them via setuptools, so they
are not practical for standalone scripts - and having 2 distinct ways of
setting scripts up to be runnable is less than ideal.
4. If you are upgrading/reinstalling the script, you cannot do so without
workarounds if the exe is currently running (mostly a nice case for pip
upgrading itself).

Also, not directly related to the exes, but definitely to setuptools
wrappers (see point 3 above):

1. The wrappers need pkg_resources installed *at runtime* not just at build
time.

>> I cant help thinking a more elegant solution exists, but maybe not.
> >
> > Personally, I believe that executable .py scripts (or maybe a dedicated
> > .pys/.pye/.pya/whatever extension) *is* a more elegant solution - but
> > equally, I concede, "maybe not"...
>
> well, I guess that's the way Windows does things -- *nix has the
> executable bit, Windows uses extensions, so I suppose that is "the
> way" to get an executable script -- but it really SHOULDN'T be *.py!
>

Agreed, we need two concepts "Python module" (.py) and "Python script" (the
new extension).


> I've lost track of what needs to be tried -- can't we just associate
> an extension with py and give it a go? (got to get that Windos VM
> working....)
>

Yes, essentially. I can confidently report that the solution is flawless,
and should be adopted at once :-)

More seriously, we need *more* people to try the approach and report *real*
issues. At the moment, I feel that we have a few people saying it works
(mostly me!) and some people saying that they suspect there may be issues
but without providing much that's reproducible. (I worry that I'm
exhibiting extreme confirmation bias here, though, which is why I want
someone to collate a proper list).


> > Also, you mention develop mode. I don't use develop mode, most of my
> > standalone scripts are single-file scripts, not anything that I'd bother
> > packaging up with a setup.py, etc.
>
> Good point -- it seem most of my scripts are part of a larger package,
> either a set of scripts, or a couple scripts that all rely on a
> particular package, so the whole setup.py thing works well.
>
> but for a single stand-alone script, the PATHEXT and py launcher
> approach seems really natural.
>
> > And many of the issues I've had with the exe wrappers are particular to
> > built installers (wheels, wininsts) and *not* develop mode.
>
> in theory, wheels and develop mode should do the same thing -- not
> sure about wininsts -- previously, the launcher thing was setuptools,
> not plain distutils, not sure how that was handled.
>

Yes, I think that consensus is that wheels including the exe wrappers is
essentially a bug - they should contain details on how to *create* the
scripts/wrappers at install time. But it's a messy bug to address, because
of history and lack of standardisation in this area. That's why I thing a
PEP saying "how to do scripts" the official way would be useful. It was
debated many years ago (when distutils was first added to the stdlib!) but
things have got much better since then and it should be easier to come up
with something acceptable.

but it would be best if they all did it the same way.
>

Precisely. Then we'd only have "legacy" non-standard approaches to address,
and we are better able to do the best we can and acknowledge it's not
perfect, if we can say "follow the PEP and it works".

> PS I still think that long-term a policy PEP on the recommended way of
> > making "executable scripts" is worthwhile.
>
> Probably, yes:  "There should be one-- and preferably only one
> --obvious way to do it."
>
> I'm leaning toward the PATHEXT approach -- it seems more si milar to
> the *nix way, and perhaps easeir for lay folks to debug and fix. But
> I'm saying tha tthe exe method worked fine for many of us, too.


The exe method has many, many advantages, and I think it's important that
anything new is measured against it. But I do think that a carefully judged
approach of considering whether to abandon certain specific benefits
because that allows us to fix some of the disadvantages is a good exercise.

Thanks for the feedback!
Paul
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130814/5cfc16b5/attachment.html>