[Python-Dev] Safely importing zip files with C extensions

[PJ Eby]

On Wed, Mar 27, 2013 at 5:19 PM, Bradley M. Froehle
<brad.froehle at gmail.com> wrote:
> I implemented just such a path hook ---- zipimporter plus the magic required
> for C extensions --- as a challenge to myself to learn more about the Python
> import mechanisms.
>
> See https://github.com/bfroehle/pydzipimport.

FYI, there appears to be a bug for Windows with packages: you're using
'/__init__' in a couple places that should actually be
os.sep+'__init__'.

This does seem like a good way to address the issue, for those rare
situations where this would be a good idea.

The zipped .egg approach was originally intended for user-managed
plugin directories for certain types of extensible platforms, where
"download a file and stick it in the plugins directory" is a
low-effort way to install plugins, without having to build a lot of
specialized install capability.

As Jim has pointed out, though, this doesn't generalize well to a
full-blown packaging system.

Technically, you can blame Bob Ippolito for this, since he's the one
who talked me into using eggs to install Python libraries in general,
not just as a plugin packaging mechanism.  ;-)

That being said, *unpacked* egg, er, wheels, are still a great way to
meet all of the "different apps needing different versions" use cases
(without needing one venv per app), and nowadays the existence of
automated installer tools means that using one to install a plugin for
a low-tech plugin system is not a big deal, as long as that tool
supports the simple unpacked wheel scenario.  So I wholeheartedly
support some kind of mount/unmount or "require"-type mechanism for
finding plugins.  pkg_resources even has an API for handling simple
dynamic plugin dependency resolution scenarios:

  http://peak.telecommunity.com/DevCenter/PkgResources#locating-plugins

It'd be a good idea if distlib provides a similar feature, or at least
the APIs upon which apps or frameworks can implement such features.

(Historical note for those who weren't around back then: easy_install
wasn't even an *idea* until well after eggs were created; the original
idea was just that people would build plugins and libraries as eggs
and manually drop them in directories, where a plugin support library
would discover them and add them to sys.path as needed.  And Bob and I
also considered a sort of "update site" mechanism ala Eclipse, with a
library to let apps fetch plugins.  But as soon as eggs existed and
PyPI allowed uploads, it was kind of an obvious follow-up to make an
installation tool as a kind of "technology demonstration"....  which
promptly became a monster.   The full story with all its twists and
turns can also be found here:
http://mail.python.org/pipermail/python-dev/2006-April/064145.html )