[Python-Dev] PEP 441 - Improving Python ZIP Application Support

[Nick Coghlan]

On 15 February 2015 at 00:44, Paul Moore <p.f.moore at gmail.com> wrote:
> I'm looking at putting together a patch for CPython to implement PEP
> 441. In doing so, there are a few issues that I'd like to raise with
> the PEP. These are all to do with the supporting app "pyzaa" (IIRC,
> Nick proposed renaming this to "pyzapp", which I like, but it's not a
> big deal either way).
>
> 1. The PEP says that the application will warn if you try to add C
> extensions to the archive. In practice, this will be quite hard to do
> - extensions vary across platforms, and it's not only C extensions
> that won't work from an archive (consider a DLL with a pure-python
> ctypes wrapper). We could check the filenames for "known" extensions
> (maybe using importlib.machinery.EXTENSION_SUFFIXES, but that varies
> by platform) but I think that's going to be fragile. IMO, the costs
> aren't worth the benefits, and I'd like to remove this proposal and
> simply document that applications packed up with pyzaa need to be
> tested to ensure they work from a zipfile.
>
> 2. The option to compile Python source seems to me to be of limited
> use. It ties the archive to a particular Python version (more
> precisely, only one version gets the benefits of precompiled
> bytecode). It's also tricky to implement, particularly the option to
> compile an existing archive, because there standard library py_compile
> and compileall modules work on the filesystem, and for compiling files
> in an archive it would be better to build the bytecode in memory (no
> need to use tempfiles). I'd prefer to say that if users want to
> include bytecode, they can precompile on disk (using compileall) for
> as many versions as they want.
>
> FWIW, Daniel's standalone pyzaa program doesn't currently include
> either of the above features.

"Do the simplest thing that could possibly work" applies, so I'd be OK
with dropping both of those features.

> As well as removing things, I'd like to add a programmable API for the
> utility. At the moment I only have a rough idea of how that would
> work, but I'm thinking of a PyZApp class, which can be created from a
> filename or an open stream (like a ZipFile object) and which has
> methods to set the shebang line, setting the main entry point, and
> adding files or directories. Does that need to go into the PEP or is
> it OK to review the API as part of the patch? I don't mind either way,
> but my normal coding process tends to be to fiddle with the API as I
> code it up and use it in the main app, so I'd normally leave
> documenting the final API till it's written...

I'd suggest including the API in the PEP, which can co-evolve with the
reference implementation.

The following suggestions are ones I came up with last time I looked
closely at the draft PEP:

* use "zipapp" for the support module name, distinct from the command
line tool name (think venv vs pyvenv)

* expose a zipapp.pack programmatic interface that takes all the input
pieces and builds a suitable archive

I'd definitely prefer a simple procedural API like that over offering
a public stateful object-oriented API, as the latter significantly
increases testing complexity without offering a sufficiently
compelling gain in expressiveness to justify the additional upfront
test development cost and the ongoing maintenance and support burden.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia