[Distutils] wheels on sys.path clarification (reboot)

[Nick Coghlan]

On 30 Jan 2014 15:27, "Donald Stufft" <donald at stufft.io> wrote:
> I can't believe folks are unable to differentiate between the difference
of
> "It's possible to do so because we didn't actively attempt to prevent it"
and
> "This is a documented goal of the format and thus must be considered as
part of
> the backward compatibility contract that the format has whenever changes
are
> made to the format".

Donald, there are multiple design decisions in the PEP which share a common
theme of zipimport compatibility, as well as an explicit statement from the
PEP author that the zipimport compatibility was intentional (
https://mail.python.org/pipermail/distutils-sig/2012-September/018960.html).
The flaw was that we never explicitly noted that rationale in the PEP, and
so people got the wrong idea both that this capability wasn't available and
also that it might be something that didn't need to be taken into account
in the future evolution of the format.

This has resulted in two conflicting messages being presented to the
community in the time since the wheel format was approved: if someone asked
if using wheels like eggs was supported, the answer would have depended on
who was giving it.

I've certainly been telling people it was deliberately designed to offer a
superset of egg functionality *because it was*. Daniel wrote it that way,
the capability was noted several times during the design discussions, I
approved it that way and *not once* did anyone suggest declaring that
advanced users should not take advantage of the zipimport compatibility
that is a natural consequence of the design.

For example, at the PyCon US 2014 packaging panel last year, I state
explicitly that you can import things from wheel files without installing
them, and not one of the other panelists or attendees at the session raised
any objection (neither at the time, nor privately to me after the session):
https://www.youtube.com/watch?v=ePFWp3oSfyU#t=15m40

This particular genie got out of the bottle a long time ago, so my new FAQ
in the PEP was just bringing it up to speed with promises that have already
been made.

> The first option is, as far as I can tell, what the PEP read as and the
> discussion, that occurred in public at least, read as. However since the
change
> Nick made he's shifted it from the first to the second type of "supports".
>
> At this point I can only imagine people are being willfully ignorant
because
> they want this particular feature and don't want to have it available to
be
> discussed as per the PEP process and are actively attempting to sidestep
the
> process. I'm very explicitly trying not to argue for or against this
feature,
> although I believe it a bad idea, but instead that before we commit to
> promising that Wheels will be zip importable by simply adding them to
sys.path
> that we *need* to discuss it and figure out what the contract we're
providing
> for that is.

The problem with believing that we still have a choice about this topic is
twofold:

- firstly, depending on who they have spoken to users may *already* have
been told it is supported (which includes everyone at the packaging panel
last year and those who watched that video since, as well as everyone that
directly asked either me or Daniel about it)
- secondly, when a particular behaviour is an inevitable consequence of
other design decisions, then users are justified in assuming that the
resulting use case is supported unless it is *explicitly* disclaimed as
unsupported (the comparison with eggs in PEP 427 would be a very thin reed
indeed to hang a backwards compatibility break on)

You're free to tell us (and the users we have communicated our intent to
directly) that you would *prefer* if this was not a supported usage model,
but there's a significant asymmetry here:

- those of us who have always interpreted the wheel format as specifying a
functional superset of the egg format (which includes both Daniel as the
PEP author and me as the BDFL-Delegate that accepted it), want to ensure
that this feature is properly taken into account in any future evolution of
the format (including wheel 1.1)

- you would like to retain the option of *not* honouring that promise,
solely because we left out that detail from the PEP itself, even though we
always intended it to be that way, made multiple references to the
capability in discussions prior to acceptance and have told multiple people
(including the attendees at the packaging panel at PyCon US 2013) that we
intended it to be that way
I can apologise for not reviewing PEP 427 sufficiently and failing to
realise that this design assumption was not correctly captured, and so
people that would have preferred to explicitly disclaim support for this
feature didn't realise they needed to. However, I can also point out that
all that raising such objections would have done is to ensure that a clause
along these lines was added to the PEP in February 2013, rather than in
January 2014, as both Daniel and I consider this a supported feature of the
wheel format.

That said, I will also note that the wheel *format* supporting being used
this way is a very different question from *pip* supporting using them this
way. pip is an installer - it takes wheels and sdists and adds them to a
Python installation or virtual environment with full PEP 376 metadata. It's
entirely appropriate to declare "uninstalled wheels" to be out of pip's
scope, and I'd fully back any decision by the pip team to take that stance.
wheel is just a file format - not every tool that supports a format for one
use case needs to support *all* the use cases of that format. If people
want to write tools to make it easier to run directly from a wheelhouse,
they can do that - that's a very different use case from installation, one
that gets back closer to the original easy_install model and one that would
be better addressed by tools dedicated specifically to the task.

Regards,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140130/5e9289c4/attachment.html>