[Python-Dev] PEP 441 - Improving Python ZIP Application Support

[Thomas Wouters]

FWIW, we have a similar tool to 'pyzaa' at Google, although a lot older (it
has a pure-python implementation of zipimport, because it had to work with
Python 2.2 back in the day) that *does* support extension modules and other
shared libraries (by extracting them to disk on program startup.) We're
also working to replace that by loading the extension modules directly from
the archive (using dlopen_with_offset as described by ppluzhnikov in
https://sourceware.org/bugzilla/show_bug.cgi?id=11767, although it should
also be possible using dlopen_ehdr/dlopen_phdr), which requires that
extension modules are stored uncompressed (simple) and page-aligned
(harder, as the zipfile.ZipFile class doesn't directly support
page-aligning anything, but it turns out it's easy to hack around by
overriding _writecheck :P) And yes, we really are in a position to modify
glibc to make our life distributing Python applications easier, internally;
the old code involves shell and Python bootstrap code that tries very hard
to avoid race conditions and security problems, which is a pain to maintain.

It might be nice to consider those use-cases in pyzapp as well, especially
once the glibc feature is released. It requires some fairly big changes to
zipimport (I ended up rewriting the whole thing) but we can easily
opensource the code, if there's any interest at all. I'd be happy to
discuss this in more detail at PyCon, at which time we should be deploying
code internally using all of this.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150215/f8a5d2b7/attachment.html>