[Distutils] PEP: Improving Python ZIP Application Support

[Daniel Holth]

On Sat, Mar 30, 2013 at 7:07 PM, Brett Cannon <brett at yvrsfo.ca> wrote:
>
> On Mar 30, 2013 3:23 PM, "Daniel Holth" <dholth at gmail.com> wrote:
>>
>> Python ZIP Application Support -
>>
>> https://docs.google.com/document/d/1MKXgPzhWD5wIUpoSQX7dxmqgTZVO6l9iZZis8dnri78/edit?usp=sharing
>>
>>
>> PEP: 4XX
>>
>> Title: Improving Python ZIP Application Support
>>
>> Author: Daniel Holth <dholth at gmail.com>
>>
>> Status: Draft
>>
>> Type: Standards Track
>>
>> Python-Version: 3.4
>>
>> Created: 30 March 2013
>>
>> Post-History: 30 March 2013
>>
>>
>> Improving Python ZIP Application Support
>>
>>
>> Python has had the ability to execute directories or ZIP-format
>> archives as scripts since version 2.6. When invoked with a zip file or
>> directory as its first argument the interpreter adds that directory to
>> sys.path and executes the __main__ module. These archives provide a
>> great way to publish software that needs to be distributed as a single
>> file script but is complex enough to need to be written as a
>> collection of modules.
>>
>>
>> This feature is not as popular as it should be for two reasons: a)
>> users haven’t heard of it because it wasn’t mentioned in earlier
>> versions of Python 2.6 “What’s New” document, and b) Windows users
>> don’t have a file extension (other than .py) to associate with the
>> launcher.
>>
>>
>> This PEP proposes to fix these problems by re-publicising the feature,
>> defining the .pyz and .pyzw extensions as “Python ZIP applications”
>> and “Windowed Python Zip Applications”, and providing some simple
>> tooling to manage the format.
>>
>> A New Python ZIP Application Extension
>>
>>
>> The Python 3.4 installer will associate .pyz and .pyzw “Python ZIP
>> Applications” with itself so they can be executed by the Windows
>> launcher. A .pyz archive is a console application and a .pyzw archive
>> is a windowed application. This indicates whether the console should
>> appear when running the app.
>>
>>
>> Why not use .zip or .py? Users expect a .zip file would be opened with
>> an archive tool, and users expect .py to be opened with a text editor.
>> Both would be confusing for this use case.
>>
>>
>> For UNIX users, .pyz applications should be prefixed with a #! line
>> pointing to the correct Python interpreter and an optional
>> explanation.
>>
>>
>> #!/usr/bin/env python
>>
>> # This is a Python application stored in a ZIP archive.
>
> ... built using pyzaa.
>
>>
>> (binary contents of archive)
>>
>>
>> As background, ZIP archives are defined with a footer containing
>> relative offsets from the end of the file. They remain valid when
>> concatenated to the end of any other file. This feature is completely
>> standard and is how self-extracting ZIP archives and the bdist_wininst
>> installer format work.
>>
>> Minimal Tooling: The pyzaa Module
>>
>> This PEP also proposes including a simple application for working with
>> Python ZIP Archives: The Python Zip Application Archiver “pyzaa”
>> (rhymes with “huzzah” or “pizza”). “pyzaa” can archive or extract
>> these files, compile bytecode, and can write the __main__ module if it
>> is not present.
>>
>> Usage
>>
>> python -m pyzaa (pack | unpack | compile)
>>
>>
>> python -m pyzaa pack [-o path/name] [-m module.submodule:callable]
>> [-c] [-w] [-p interpreter] directory:
>>
>>     ZIP the contents of directory as directory.pyz or [-w]
>> directory.pyzw. Adds the executable flag to the archive.
>>
>>     -c compile .pyc files and add them to the archive
>>
>>     -p interpreter include #!interpreter as the first line of the archive
>
> Would `/usr/bin/env python` (or python3 depending on interpreter used to
> compile) be set otherwise? Or how about the specific python version to
> prevent possible future-compatibility issues (e.g. specifying python3.3)?
>
>>
>>     -o path/name archive is written to path/name.pyz[w] instead of
>> dirname. The extension is added if not specified.
>>
>>     -m module.submodule:callable __main__.py is written as “import
>> module.submodule; module.submodule.callable()”
>>
>>
>>     pyzaa pack will warn if the directory contains C extensions or if
>> it doesn’t contain __main__.py.
>>
>>
>> python -m pyzaa unpack arcname.pyz[w]
>>
>>     The archive is unpacked into a directory [arcname]
>
> Is this truly necessary? If it's a zip file any archiving tool can unzip it.
> Heck, we can add a CLI to the ziofile module if it doesn't already have one.

It is a convenience so that the contents don't wind up in $PWD, I like
the idea of dropping it though.

I'll be sure to emphasize that these are completely standard ZIP
archives with a new extension.

I'm pretty sure zipfile has a CLI.

The pack command is a convenience on top of zip & cat. Hope we don't
need ignore / MANIFEST.in type features...

>>
>>
>> python -m pyzaa compile arcname.pyz[w]
>>
>>     The Python files in arcname.pyz[w] are compiled and appended to
>> the ZIP file.
>
> I would suggest allowing multiple versions for compilation (when specified).
> There should also be a check that people don't specify multiple versions
> that can't exist on the same directory (e.g 2.6 and 2.7). Otherwise the
> compileall module's CLI handles this and people can call it with different
> interpreters as necessary.

If that's easy with compileall then great.

> IOW I'm advocating KISS for what the tool does. Since making the zip file is
> only really tricky bit it should only handle that case. Heck you can make it
> part of the zipfile module if there is resistance to adding yet another
> script that python installs (obviously will need a Cheeseshop package for
> older versions).
>
> Otherwise I like everything else.

Thanks!

> -Brett
>
>>
>> References
>>
>> [1] http://bugs.python.org/issue1739468 “Allow interpreter to execute
>> a zip file”
>>
>> Copyright
>>
>> This document has been placed into the public domain.
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> http://mail.python.org/mailman/listinfo/distutils-sig