[Distutils] Using Wheel with zipimport

[Nick Coghlan]

On 29 January 2014 20:36, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>
> You don't want it to be there, even if it might be useful to others,
> just because it isn't useful to you? It's not as if distlib is forcing
> that functionality on anyone.

I believe Paul's concern is with anything that suggests that arbitrary
*third party* code can be run from wheel files, when the reality is
that it is fairly easy to accidentally write code that assumes it is
installed on the filesystem in a way that isn't easy for a quick scan
of the files in the zip archive to detect (especially since the PEP
376 installation database PEP doesn't include any support for
arbitrary metapath importers).

By contrast PEP 441 is a *distribution* utility - the creator of the
application is expected to ensure that doing so actually works
correctly before publishing their app that way, just as we would
expect py2exe, py2app and cx-freeze users to do.

With the "reference implementation" position that distlib is likely to
occupy in a post-PEP-426/440/459 world, though, there's an additional
legitimate concern about allowing end users to easily distinguish
between "this API is fully supported by the PyPA as part of the
reference implementation for metadata 2.0" and "this is an
experimental packaging related API that may or may not be useful in
general, and some members of the PyPA may still have grave
reservations about it".

At the moment, distlib contains both kinds of API, and it confuses
*us*, let alone anyone else that isn't closely following along on
distutils-sig. As long as distlib is serving the dual role of
providing both "the reference implementation for metadata 2.0" and
"some experimental packaging related APIs", we're going to get
concerns like this one arising. If there was a clear way to
distinguish them (ideally with a separate project for either the
reference implementation or the experimental stuff, but even a
distinct namespace within the distlib project would help a great
deal), I suspect there would be less concern.

In the specific case of distlib.mount, if it's eventually combined
with a metadata extension like "distlib.mount" which packages must
export in order for the command to allow them to be automatically used
that way, then I don't see anything wrong with it *in general* - it's
a natural extension of the setuptools "zip_safe" flag, but with the
ability to include additional details (like whether or not there are C
extensions that need to be automatically extracted). Note that this
goes further than the current EXTENSIONS approach - this proposal
would be akin to *requiring* an empty EXTENSIONS file, and/or the
setuptools zip_safe flag in order to allow mounting of even the pure
Python wheel. Such a conservative approach is also the antithesis of
the setuptools "attempt to guess": if the package publisher doesn't
explicitly opt in to zip support, then distlib.mount would assume that
it is *not* supported (but may provide an API for the caller to
override that, like "assume_zip_safe=True" or "force=True").

However, like Paul, I have some concerns about a still experimental
API like that being in the metadata 2.0 reference implementation,
since that will likely end up having to deal with stdlib-like levels
of backwards compatibility requirements, and removing experimental
APIs that we later decided we weren't happy with could prove
problematic.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia