[Distutils] wheels on sys.path clarification (reboot)

[Donald Stufft]

On Jan 30, 2014, at 9:30 AM, Donald Stufft <donald at stufft.io> wrote:

> 
> On Jan 30, 2014, at 6:43 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
>> On 30 Jan 2014 15:27, "Donald Stufft" <donald at stufft.io> wrote:
>> > I can't believe folks are unable to differentiate between the difference of
>> > "It's possible to do so because we didn't actively attempt to prevent it" and
>> > "This is a documented goal of the format and thus must be considered as part of
>> > the backward compatibility contract that the format has whenever changes are
>> > made to the format".
>> 
>> Donald, there are multiple design decisions in the PEP which share a common theme of zipimport compatibility, as well as an explicit statement from the PEP author that the zipimport compatibility was intentional (https://mail.python.org/pipermail/distutils-sig/2012-September/018960.html). The flaw was that we never explicitly noted that rationale in the PEP, and so people got the wrong idea both that this capability wasn't available and also that it might be something that didn't need to be taken into account in the future evolution of the format.
>> 
>> This has resulted in two conflicting messages being presented to the community in the time since the wheel format was approved: if someone asked if using wheels like eggs was supported, the answer would have depended on who was giving it.
>> 
>> I've certainly been telling people it was deliberately designed to offer a superset of egg functionality *because it was*. Daniel wrote it that way, the capability was noted several times during the design discussions, I approved it that way and *not once* did anyone suggest declaring that advanced users should not take advantage of the zipimport compatibility that is a natural consequence of the design.
>> 
>> For example, at the PyCon US 2014 packaging panel last year, I state explicitly that you can import things from wheel files without installing them, and not one of the other panelists or attendees at the session raised any objection (neither at the time, nor privately to me after the session): https://www.youtube.com/watch?v=ePFWp3oSfyU#t=15m40
>> 
>> This particular genie got out of the bottle a long time ago, so my new FAQ in the PEP was just bringing it up to speed with promises that have already been made.
>> 
>> > The first option is, as far as I can tell, what the PEP read as and the
>> > discussion, that occurred in public at least, read as. However since the change
>> > Nick made he's shifted it from the first to the second type of "supports".
>> >
>> > At this point I can only imagine people are being willfully ignorant because
>> > they want this particular feature and don't want to have it available to be
>> > discussed as per the PEP process and are actively attempting to sidestep the
>> > process. I'm very explicitly trying not to argue for or against this feature,
>> > although I believe it a bad idea, but instead that before we commit to
>> > promising that Wheels will be zip importable by simply adding them to sys.path
>> > that we *need* to discuss it and figure out what the contract we're providing
>> > for that is.
>> 
>> The problem with believing that we still have a choice about this topic is twofold:
>> 
>> - firstly, depending on who they have spoken to users may *already* have been told it is supported (which includes everyone at the packaging panel last year and those who watched that video since, as well as everyone that directly asked either me or Daniel about it)
>> - secondly, when a particular behaviour is an inevitable consequence of other design decisions, then users are justified in assuming that the resulting use case is supported unless it is *explicitly* disclaimed as unsupported (the comparison with eggs in PEP 427 would be a very thin reed indeed to hang a backwards compatibility break on)
>> You're free to tell us (and the users we have communicated our intent to directly) that you would *prefer* if this was not a supported usage model, but there's a significant asymmetry here:
>> 
>> - those of us who have always interpreted the wheel format as specifying a functional superset of the egg format (which includes both Daniel as the PEP author and me as the BDFL-Delegate that accepted it), want to ensure that this feature is properly taken into account in any future evolution of the format (including wheel 1.1)
>> - you would like to retain the option of *not* honouring that promise, solely because we left out that detail from the PEP itself, even though we always intended it to be that way, made multiple references to the capability in discussions prior to acceptance and have told multiple people (including the attendees at the packaging panel at PyCon US 2013) that we intended it to be that way
>> I can apologise for not reviewing PEP 427 sufficiently and failing to realise that this design assumption was not correctly captured, and so people that would have preferred to explicitly disclaim support for this feature didn't realise they needed to. However, I can also point out that all that raising such objections would have done is to ensure that a clause along these lines was added to the PEP in February 2013, rather than in January 2014, as both Daniel and I consider this a supported feature of the wheel format.
>> 
>> That said, I will also note that the wheel *format* supporting being used this way is a very different question from *pip* supporting using them this way. pip is an installer - it takes wheels and sdists and adds them to a Python installation or virtual environment with full PEP 376 metadata. It's entirely appropriate to declare "uninstalled wheels" to be out of pip's scope, and I'd fully back any decision by the pip team to take that stance. wheel is just a file format - not every tool that supports a format for one use case needs to support *all* the use cases of that format. If people want to write tools to make it easier to run directly from a wheelhouse, they can do that - that's a very different use case from installation, one that gets back closer to the original easy_install model and one that would be better addressed by tools dedicated specifically to the task.
>> 
>> Regards,
>> Nick.
> 
> Yea I've tried to respond to this about 5 times without being a dick and I
> can't. So whatever I give up. Now I need to go figure out how to make my
> projects bomb out under zip import since Wheel has managed to be worse than
> Egg and not even provide a way to disclaim support for it.
> 
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

For those following along at home who have no interest in supporting zipped
Wheels who want to disclaim support for it, here’s how you do zip_safe=False
for Wheels: https://gist.github.com/dstufft/8710270.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140130/06ecd710/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140130/06ecd710/attachment-0001.sig>