<div dir="ltr"><div><div><div>Accepted!<br><br></div>Thanks for your patience, Paul, and thanks everyone for their feedback.<br><br></div>I know there are still a few small edits to the PEP, but those don't affect my acceptance. Congrats!<br><br></div>--Guido<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 26, 2015 at 9:05 AM, Paul Moore <span dir="ltr"><<a href="mailto:p.f.moore@gmail.com" target="_blank">p.f.moore@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 24 February 2015 at 18:24, Guido van Rossum <<a href="mailto:guido@python.org">guido@python.org</a>> wrote:<br>
> Here's my review. I really like where this is going but I have a few<br>
> questions and suggestions (I can't help myself :-).<br>
<br>
</span>OK, I've updated both the PEP and the patch based on follow-up<br>
discussions. I think (again!) it is ready to go.<br>
<br>
I've included the updated PEP inline below, it'll be available at<br>
<a href="http://peps.python.org" target="_blank">peps.python.org</a> as soon as someone has a chance to upload it.<br>
<br>
Thanks to everyone for the various comments. If I've missed anything<br>
that someone thinks I'd said I would change, please let me know.<br>
<div><div class="h5"><br>
Paul<br>
<br>
PEP: 441<br>
Title: Improving Python ZIP Application Support<br>
Version: $Revision$<br>
Last-Modified: $Date$<br>
Author: Daniel Holth <<a href="mailto:dholth@gmail.com">dholth@gmail.com</a>>,<br>
Paul Moore <<a href="mailto:p.f.moore@gmail.com">p.f.moore@gmail.com</a>><br>
Discussions-To:<br>
<a href="https://mail.python.org/pipermail/python-dev/2015-February/138277.html" target="_blank">https://mail.python.org/pipermail/python-dev/2015-February/138277.html</a><br>
Status: Draft<br>
Type: Standards Track<br>
Content-Type: text/x-rst<br>
Created: 30 March 2013<br>
Post-History: 30 March 2013, 1 April 2013, 16 February 2015<br>
<br>
Improving Python ZIP Application Support<br>
========================================<br>
<br>
Python has had the ability to execute directories or ZIP-format<br>
archives as scripts since version 2.6 [1]_. When invoked with a zip<br>
file or directory as its first argument the interpreter adds that<br>
directory to sys.path and executes the ``__main__`` module. These<br>
archives provide a great way to publish software that needs to be<br>
distributed as a single file script but is complex enough to need to<br>
be written as a collection of modules.<br>
<br>
This feature is not as popular as it should be mainly because it was<br>
not promoted as part of Python 2.6 [2]_, so that it is relatively<br>
unknown, but also because the Windows installer does not register a<br>
file extension (other than ``.py``) for this format of file, to associate<br>
with the launcher.<br>
<br>
This PEP proposes to fix these problems by re-publicising the feature,<br>
defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications"<br>
and "Windowed Python ZIP Applications", and providing some simple<br>
tooling to manage the format.<br>
<br>
A New Python ZIP Application Extension<br>
======================================<br>
<br>
The terminology "Python Zip Application" will be the formal term used<br>
for a zip-format archive that contains Python code in a form that can<br>
be directly executed by Python (specifically, it must have a<br>
``__main__.py`` file in the root directory of the archive). The<br>
extension ``.pyz`` will be formally associated with such files.<br>
<br>
The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python<br>
Zip Applications" with the platform launcher so they can be executed.<br>
A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a<br>
windowed application, indicating whether the console should appear<br>
when running the app.<br>
<br>
On Unix, it would be ideal if the ``.pyz`` extension and the name<br>
"Python Zip Application" were registered (in the mime types database?).<br>
However, such an association is out of scope for this PEP.<br>
<br>
Python Zip applications can be prefixed with a ``#!`` line<br>
pointing to the correct Python interpreter and an optional<br>
explanation::<br>
<br>
#!/usr/bin/env python3<br>
# Python application packed with zipapp module<br>
(binary contents of archive)<br>
<br>
On Unix, this allows the OS to run the file with the correct<br>
interpreter, via the standard "shebang" support. On Windows, the<br>
Python launcher implements shebang support.<br>
<br>
However, it is always possible to execute a ``.pyz`` application by<br>
supplying the filename to the Python interpreter directly.<br>
<br>
As background, ZIP archives are defined with a footer containing<br>
relative offsets from the end of the file. They remain valid when<br>
concatenated to the end of any other file. This feature is completely<br>
standard and is how self-extracting ZIP archives and the bdist_wininst<br>
installer format work.<br>
<br>
<br>
Minimal Tooling: The zipapp Module<br>
==================================<br>
<br>
This PEP also proposes including a module for working with these<br>
archives. The module will contain functions for working with Python<br>
zip application archives, and a command line interface (via ``python<br>
-m zipapp``) for their creation and manipulation.<br>
<br>
More complete tools for managing Python Zip Applications are<br>
</div></div>encouraged as 3rd party applications on PyPI. Currently, pyzzer [5]_<br>
and pex [6]_ are two such tools.<br>
<span class=""><br>
Module Interface<br>
----------------<br>
<br>
The zipapp module will provide the following functions:<br>
<br>
</span>``create_archive(source, target=None, interpreter=None, main=None)``<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
<span class=""><br>
Create an application archive from *source*. The source can be any<br>
of the following:<br>
<br>
</span><span class="">* The name of a directory, in which case a new application archive<br>
will be created from the content of that directory.<br>
</span><span class="">* The name of an existing application archive file, in which case the<br>
</span> file is copied to the target. The file name should include the<br>
``.pyz`` extension, if required.<br>
<span class="">* A file object open for reading in bytes mode. The content of the<br>
file should be an application archive, and the file object is<br>
assumed to be positioned at the start of the archive.<br>
<br>
</span><span class="">The *target* argument determines where the resulting archive will be<br>
written:<br>
<br>
* If it is the name of a file, the archive will be written to that<br>
file.<br>
</span><span class="">* If it is an open file object, the archive will be written to that<br>
file object, which must be open for writing in bytes mode.<br>
* If the target is omitted (or None), the source must be a directory<br>
and the target will be a file with the same name as the source, with<br>
a ``.pyz`` extension added.<br>
<br>
</span><span class="">The *interpreter* argument specifies the name of the Python<br>
</span>interpreter with which the archive will be executed. It is written as<br>
a "shebang" line at the start of the archive. On Unix, this will be<br>
<span class="">interpreted by the OS, and on Windows it will be handled by the Python<br>
</span>launcher. Omitting the *interpreter* results in no shebang line being<br>
written. If an interpreter is specified, and the target is a<br>
<span class="">filename, the executable bit of the target file will be set.<br>
<br>
</span><span class="">The *main* argument specifies the name of a callable which will be<br>
used as the main program for the archive. It can only be specified if<br>
the source is a directory, and the source does not already contain a<br>
``__main__.py`` file. The *main* argument should take the form<br>
"pkg.module:callable" and the archive will be run by importing<br>
</span>"pkg.module" and executing the given callable with no arguments. It<br>
<span class="">is an error to omit *main* if the source is a directory and does not<br>
contain a ``__main__.py`` file, as otherwise the resulting archive<br>
would not be executable.<br>
<br>
If a file object is specified for *source* or *target*, it is the<br>
caller's responsibility to close it after calling create_archive.<br>
<br>
When copying an existing archive, file objects supplied only need<br>
``read`` and ``readline``, or ``write`` methods. When creating an<br>
archive from a directory, if the target is a file object it will be<br>
passed to the ``zipfile.ZipFile`` class, and must supply the methods<br>
needed by that class.<br>
<br>
</span><span class="">``get_interpreter(archive)``<br>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^<br>
<br>
Returns the interpreter specified in the shebang line of the<br>
*archive*. If there is no shebang, the function returns ``None``.<br>
</span>The *archive* argument can be a filename or a file-like object open<br>
<span class="">for reading in bytes mode.<br>
<br>
<br>
</span><span class="">Command Line Usage<br>
------------------<br>
<br>
The zipapp module can be run with the python ``-m`` flag. The command<br>
line interface is as follows::<br>
<br>
</span> python -m zipapp directory [options]<br>
<br>
Create an archive from the given directory. An archive will<br>
be created from the contents of that directory. The archive<br>
will have the same name as the source directory with a .pyz<br>
<span class=""> extension.<br>
<br>
The following options can be specified:<br>
<br>
-o archive / --output archive<br>
<br>
</span> The destination archive will have the specified name. The<br>
given name will be used as written, so should include the<br>
".pyz" extension.<br>
<span class=""><br>
-p interpreter / --python interpreter<br>
<br>
The given interpreter will be written to the shebang line<br>
of the archive. If this option is not given, the archive<br>
will have no shebang line.<br>
<br>
-m pkg.mod:fn / --main pkg.mod:fn<br>
<br>
The source directory must not have a __main__.py file. The<br>
archiver will write a __main__.py file into the target<br>
which calls fn from the module pkg.mod.<br>
<br>
The behaviour of the command line interface matches that of<br>
</span>``zipapp.create_archive()``.<br>
<br>
In addition, it is possible to use the command line interface to work<br>
with an existing archive::<br>
<br>
python -m zipapp app.pyz --show<br>
<br>
Displays the shebang line of an archive. Output is of the<br>
form<br>
<br>
Interpreter: /usr/bin/env<br>
or<br>
Interpreter: <none><br>
<br>
and is intended for diagnostic use, not for scripts.<br>
<br>
python -m zipapp app.pyz -o newapp.pyz [-p interpreter]<br>
<br>
Copy app.pyz to newapp.pyz, modifying the shebang line based<br>
on the -p option (as for creating an archive, no -p option<br>
means remove the shebang line). Specifying a destination is<br>
mandatory.<br>
<br>
In-place modification of an archive is *not* supported, as the<br>
risk of damaging archives is too great for a simple tool.<br>
<span class=""><br>
As noted, the archives are standard zip files, and so can be unpacked<br>
</span>using any standard ZIP utility or Python's zipfile module. For this<br>
reason, no interfaces to list the contents of an archive, or unpack<br>
them, are provided or needed.<br>
<div><div class="h5"><br>
FAQ<br>
---<br>
<br>
Are you sure a standard ZIP utility can handle ``#!`` at the beginning?<br>
Absolutely. The zipfile specification allows for arbitrary data to<br>
be prepended to a zipfile. This feature is commonly used by<br>
"self-extracting zip" programs. If your archive program can't<br>
handle this, it is a bug in your archive program.<br>
<br>
Isn't zipapp just a very thin wrapper over the zipfile module?<br>
Yes. If you prefer to build your own Python zip application<br>
archives using other tools, they will work just as well. The<br>
zipapp module is a convenience, nothing more.<br>
<br>
Why not use just use a ``.zip`` or ``.py`` extension?<br>
Users expect a ``.zip`` file to be opened with an archive tool, and<br>
expect a ``.py`` file to contain readable text. Both would be<br>
confusing for this use case.<br>
<br>
How does this compete with existing package formats?<br>
The sdist, bdist and wheel formats are designed for packaging of<br>
modules to be installed into an existing Python installation.<br>
They are not intended to be used without installing. The<br>
executable zip format is specifically designed for standalone use,<br>
without needing to be installed. They are in effect a multi-file<br>
version of a standalone Python script.<br>
<br>
Rejected Proposals<br>
==================<br>
<br>
Convenience Values for Shebang Lines<br>
------------------------------------<br>
<br>
Is it worth having "convenience" forms for any of the common<br>
interpreter values? For example, ``-p 3`` meaning the same as ``-p<br>
"/usr/bin/env python3"``. It would save a lot of typing for the<br>
common cases, as well as giving cross-platform options for people who<br>
don't want or need to understand the intricacies of shebang handling<br>
on "other" platforms.<br>
<br>
Downsides are that it's not obvious how to translate the<br>
abbreviations. For example, should "3" mean "/usr/bin/env python3",<br>
"/usr/bin/python3", "python3", or something else? Also, there is no<br>
obvious short form for the key case of "/usr/bin/env python" (any<br>
available version of Python), which could easily result in scripts<br>
being written with overly-restrictive shebang lines.<br>
<br>
Overall, this seems like there are more problems than benefits, and as<br>
a result has been dropped from consideration.<br>
<br>
Registering ``.pyz`` as a Media Type<br>
</div></div>------------------------------------<br>
<div class="HOEnZb"><div class="h5"><br>
It was suggested [3]_ that the ``.pyz`` extension should be registered<br>
in the Unix database of extensions. While it makes sense to do this<br>
as an equivalent of the Windows installer registering the extension,<br>
the ``.py`` extension is not listed in the media types database [4]_.<br>
It doesn't seem reasonable to register ``.pyz`` without ``.py``, so<br>
this idea has been omitted from this PEP. An interested party could<br>
arrange for *both* ``.py`` and ``.pyz`` to be registered at a future<br>
date.<br>
<br>
Default Interpreter<br>
-------------------<br>
<br>
The initial draft of this PEP proposed using ``/usr/bin/env python``<br>
as the default interpreter. Unix users have problems with this<br>
behaviour, as the default for the python command on many distributions<br>
is Python 2, and it is felt that this PEP should prefer Python 3 by<br>
default. However, using a command of ``python3`` can result in<br>
unexpected behaviour for Windows users, where the default behaviour of<br>
the launcher for the command ``python`` is commonly customised by users,<br>
but the behaviour of ``python3`` may not be modified to match.<br>
<br>
As a result, the principle "in the face of ambiguity, refuse to guess"<br>
has been invoked, and archives have no shebang line unless explicitly<br>
requested. On Windows, the archives will still be run (with the<br>
default Python) by the launcher, and on Unix, the archives can be run<br>
by explicitly invoking the desired Python interpreter.<br>
<br>
Command Line Tool to Manage Shebang Lines<br>
-----------------------------------------<br>
<br>
It is conceivable that users would want to modify the shebang line for<br>
an existing archive, or even just display the current shebang line.<br>
This is tricky to do so with existing tools (zip programs typically<br>
ignore prepended data totally, and text editors can have trouble<br>
editing files containing binary data).<br>
<br>
The zipapp module provides functions to handle the shebang line, but<br>
does not include a command line interface to that functionality. This<br>
is because it is not clear how to provide one without the resulting<br>
interface being over-complex and potentially confusing. Changing the<br>
shebang line is expected to be an uncommon requirement.<br>
<br>
Reference Implementation<br>
========================<br>
<br>
A reference implementation is at <a href="http://bugs.python.org/issue23491" target="_blank">http://bugs.python.org/issue23491</a>.<br>
<br>
References<br>
==========<br>
<br>
.. [1] Allow interpreter to execute a zip file<br>
(<a href="http://bugs.python.org/issue1739468" target="_blank">http://bugs.python.org/issue1739468</a>)<br>
<br>
.. [2] Feature is not documented<br>
(<a href="http://bugs.python.org/issue17359" target="_blank">http://bugs.python.org/issue17359</a>)<br>
<br>
.. [3] Discussion of adding a .pyz mime type on python-dev<br>
(<a href="https://mail.python.org/pipermail/python-dev/2015-February/138338.html" target="_blank">https://mail.python.org/pipermail/python-dev/2015-February/138338.html</a>)<br>
<br>
.. [4] Register of media types<br>
(<a href="http://www.iana.org/assignments/media-types/media-types.xhtml" target="_blank">http://www.iana.org/assignments/media-types/media-types.xhtml</a>)<br>
<br>
.. [5] pyzzer - A tool for creating Python-executable archives<br>
(<a href="https://pypi.python.org/pypi/pyzzer" target="_blank">https://pypi.python.org/pypi/pyzzer</a>)<br>
<br>
.. [6] pex - The PEX packaging toolchain<br>
(<a href="https://pypi.python.org/pypi/pex" target="_blank">https://pypi.python.org/pypi/pex</a>)<br>
<br>
The discussion of this PEP took place on the python-dev mailing list,<br>
in the thread starting at<br>
<a href="https://mail.python.org/pipermail/python-dev/2015-February/138277.html" target="_blank">https://mail.python.org/pipermail/python-dev/2015-February/138277.html</a><br>
<br>
Copyright<br>
=========<br>
<br>
This document has been placed into the public domain.<br>
<br>
<br>
..<br>
Local Variables:<br>
mode: indented-text<br>
indent-tabs-mode: nil<br>
sentence-end-double-space: t<br>
fill-column: 70<br>
coding: utf-8<br>
End:<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)</div>
</div>