[Distutils] wheels on sys.path clarification (reboot)

[Donald Stufft]

On Jan 29, 2014, at 5:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> I have clearly done a bad job so far of explaining the clarification in PEP 427, so here's a new attempt that relies solely on the PEP text and the way the import system works, rather than the fact that the discussions around the PEP show that the import system compatibility was a deliberate feature that I took into account when accepting the PEP.
> 
> Here is the key quote from PEP 427:
> 
> """Although a specialized installer is recommended, a wheel file may be installed by simply unpacking into site-packages with the standard 'unzip' tool while preserving enough information to spread its contents out onto their final paths at any later time."""
> 
> That feature (which *is* explicitly documented in the accepted PEP), ensures that there will always be a subset of wheel files which can be used without needing a customised installer - just unzip them into a directory on sys.path and go. Not all wheels will work that way, and there are some downsides when you do it (e.g.  launch scripts won't be created and bytecode files won't be compiled with elevated permissions if that is needed). However, it's explicitly covered in the PEP and is an apparently non-controversial design goal for the format.
> 
> Once you have that "can be installed just by unzipping the archive" feature, then it is *an inherent property of how zipimport works* that there will also be an even smaller subset of wheels that will work correctly without even unpacking them first - you can just add the archive directly to sys.path and let zipimport do its thing.
> 
> Aside from deliberately choosing a zipimport incompatible archive format (which we didn't do), you cannot design a format that meets the first requirement without inevitably also supporting the latter behaviour. However, this fact isn't obvious (as has become eminently clear to me since reading Armin Ronacher's recent wheel article), as not everyone is intimately with the subtleties of the import system. That is why I added the new text to the PEP - to make this capability clear without needing to make that connection independently.
> 
> I do now plan to reword the new FAQ entry to make it clear that the import compatibility is mainly a side effect of the "no specialised installer needed" feature, but hopefully the above helps make it clear why this compatibility is already an inherent feature of the accepted PEP rather than something that can be discussed and accepted independently in a new version - it is pointing out a non-obvious consequence of an existing capability, not actually adding anything new.
> 
> Regards,
> Nick.
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig


I understand what you're saying. My point is there is a difference between
supported because we didn't go out of our way to break it, and supported
because it's an API contract we intend to honor. The first one is fine, because
it's an implementation detail and anyone relying on it cannot expect it to have
backwards compatibility concerns. It's the second one that concerned me. By
including this in the PEP we've moved the fact that Wheel's are directly
importable by adding them to the sys.path from an implementation detail to
a documented feature (maybe documented is a better word than supported here)
which means we've made some level of promise that this is going to continue
working in the future.

It's that second part that I have a problem with. It means that any change to
Wheels in the future must be weighed against the fact that we can't break
compatibility with zip import without incrementing to a new major rev of the
Wheel format. It means that we're promising that this feature will work. It's
essentially the difference between naming a function "_foo" instead of "foo".
Yes you can use a function named "_foo" just fine, but there's no promise that
it won't break in the future if you rely on that.

Again let me stress that this is separate from the issue of *if* we should
officially support zip importing Wheels as a feature. I have reservations about
that but I would not be angry if that made it into the spec *after* discussion.
What has me upset is that this "API" promise is being added without discussion
with the rationale of "We didn't actively attempt to break it so now it's an
officially supported feature”.

I'm upset that the ability to discuss and home and perhaps exclude a certain
API promise was kept from the population at large because of the fact that
the format didn't go out it's way to prevent itself from being used that way.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140129/3e6a61ff/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140129/3e6a61ff/attachment-0001.sig>