[Distutils] How to handle launcher script importability?

Jason R. Coombs jaraco at jaraco.com
Mon Aug 12 20:14:34 CEST 2013

> -----Original Message-----
> From: Distutils-SIG [mailto:distutils-sig-
> bounces+jaraco=jaraco.com at python.org] On Behalf Of PJ Eby
> Sent: Monday, 12 August, 2013 11:22
> On Mon, Aug 12, 2013 at 10:32 AM, Paul Moore <p.f.moore at gmail.com>
> wrote:
> > On 12 August 2013 14:01, PJ Eby <pje at telecommunity.com> wrote:
> >
> > As far as zipped Python applications are concerned (pyz), these can be
> > created by just using a pys file containing a #! line prepended to the
> > zip file. Certainly, it's a binary file with a filename that would
> > normally indicate a text file format, but is that any less true on
> > Unix when users create these files? I don't know what the user
> > experience with zipped Python applications on Unix is like - I doubt
> > it's *that* much better than on Windows. Probably the reality is that
> > nobody uses zipped applications anyway, so the problems haven't been
> > identified yet. Maybe the pyz PEP would bet better rewritten to
> > propose providing tools to create and manage zipped Python
> > applications, but *not* to require new extensions, merely to reuse
> > existing ones (pys on Windows, no extension on Unix) with binary (zipped)
> content.
> Seems reasonable...  but then somebody will need to write another PEP for
> the file extension(s) issue.

My preference is to reject the idea of the side-by-side executable launcher. 
There are several downsides that I'm trying to avoid by moving away from the 

1. Disparity with Unix. Better parity means cleaner code, easier 
documentation, and less confusion moving from platform to platform.
2. Executables that look like installers. If a launcher executable is used and 
Windows detects that it "looks like" an installer and it's a 32-bit executable 
and it doesn't have a manifest to disable the functionality, Windows will 
execute the file in a separate UAC context (often in a separate Window).
3. Additional files are needed. In particular, due to (2), a manifest must be 
provided for 32-bit executables.
4. Word size accounting. It's not clear to me what word size is needed. 32-bit 
may be sufficient, though 64-bit seem to have some advantages: a manifest is 
not needed, and it can match the word size of the installed Python executable 
(for consistency). Setuptools currently provides both (and installs the one 
that matches the Python executable).
5. Platform support. We're fortunate that Windows is one of the most stable 
binary platforms out there. Nevertheless, Setuptools recently got support for 
AMD binaries in the launcher. By relying on an external launcher, the launcher 
becomes responsible for platform support.
6. Two to three files to do the job of one. In fact, the "job" isn't much more 
than to invoke code elsewhere, so it seems ugly to require as many as three 
files to do the job. Then multiply that by the Python-specific version and you 
have up to six files for a single script.
7. Obfuscation of purpose. A single script pretty directly communicates its 
purpose. When there are multiple files, it's not obvious why they exist or 
what their purpose is. Indeed, I went years without realizing we had an open 
issue in Distribute due to a missing manifest (which was fixed in Setuptools), 
all because I used the 64-bit executable. While it may take some time for the 
community to learn what a '.pyl' is, it's easily documented and simple to 
grasp, unlike the subtle and sometimes implicit nuances (and fragility) of a 
side-by-side executable.
8. Unwanted content. Some Unix users have complained about finding Windows 
executables in their Linux packages, so now Setuptools has special handling to 
omit the launchers when installed on Unix systems. This is far from beautiful.

> I think the issue of "too many extensions" vs. "source/binary confusion" is
> going to boil down to a BDFL judgment call, whether it's by Nick, Guido, or
> some more Windows-specific BDFL For One PEP.
> If we go with One Extension To Rule Them All, I would actually suggest 
> '.pyl'
> (for PyLauncher), since really all that extension does is say, "hey, run 
> this as a
> console app via PyLauncher", not that it's a "script" (which would be
> assumed to be text).  And that all you can be sure of is that a .pyl files 
> will
> start with a #! line, and launch whatever other program is specified there, 
> on
> the contents of the file
> -- which may actually be a zipfile.

If it's '.py*', I don't see why it's not reasonable to allow omission of the 
shebang, and assume the default python. After encountering and now 
understanding the subtle import semantics, I'm hoping that this new extension 
can also be used in my personal 'scripts' collection to serve the same purpose 
it does for setuptools console entry points. I guess one could require 
#!/usr/bin/python in each, but that seems superfluous on Windows. I don't feel 
at all strongly on this point.

> > PS Either the ref file marker approach, or a new Python command line
> > argument with appropriate behaviour, could avoid the need for even the
> > pys/pws extension, if people prefer to reduce the number of extensions
> > claimed still further.
> But those would only be available for future Python versions.  A file
> extension would solve the problem upon installing PyLauncher and PATHEXT,
> at least for those OSes and shells that recognize PATHEXT.

Also, in my mind, this approach is most directly addressing the fundamental 
challenge (distinguishing a (executable) script from a module) in much the way 
Unix has previously enjoyed.

> Hm, here's a side thought: what if PyLauncher added the ability to serve as 
> a
> script wrapper, just like setuptools' existing wrappers?
> Then setuptools could just copy py.exe or pyw.exe alongside a .pyl or .pyw,
> and presto!  No PATHEXT compatibility needed, but users could still opt out
> of using the .exe wrappers if they're sure their shell works right without 
> it.
> (The wrapper facility would be implemented by simply checking for an
> adjacent file of matching filename and extension (.pyl for py.exe, .pyw for
> pyw.exe), and if found, insert that filename as argv[1] before proceeding
> with the normal launch process.  For efficiency, the file check could be
> skipped if the executable has its original name, at the minor cost of it not
> being possible to name a console script 'py' or a windows app 'pyw'.  But
> that's an optional tweak.)

I'm warming up to this idea a bit, especially how it supports the most elegant 
approach but degrades gracefully. Some questions that arise:

Where would Setuptools expect to find these launchers? Would it expect them to 
be present on the system? Would it symlink or hardlink them or simply copy? Is 
py.exe subject to the 'looks like installer' behavior, such that it would need 
a manifest?

I still feel like this approach would require substantial special-casing, but 
since it provides a transition to the simple, elegant approach, I'm not 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6572 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130812/414664f5/attachment-0001.bin>

More information about the Distutils-SIG mailing list