[Distutils] PEP470, backward compat is a ...

[Donald Stufft]

On May 16, 2014, at 11:38 AM, Carl Meyer <carl at oddbird.net> wrote:

> Hi Donald and Holger,
> 
> Let me try to summarize the core points here to make sure I'm
> understanding correctly:
> 
> 1. A transition to allowing only pypi-explicit links (deprecating and
> removing pypi-*-crawl), as already envisioned in PEP 438, would solve
> the worst problem that PEP 470 is trying to solve - the user confusion
> around the multiple levels of --allow-* flags in pip. (I am not claiming
> it would bring every benefit of PEP 470, just that particular benefit).
> 
> 2. To make even just that transition requires either a) breaking
> installation of externally-hosted packages on PyPI without active
> maintainers (let's call these "legacy packages" for short), or b)
> automatically scraping their external links and turning them into
> "verified" links (even though they are not actually verified at all).
> 
> Is this an accurate summary?

It’s accurate as far as I can tell.

> 
> If so, I think I agree with Donald that 2b is just not acceptable, which
> means that some form of 2a is inevitable; it's just a matter of finding
> the smoothest and simplest deprecation path to get there. Holger seems
> to be proposing a sort of deprecation path for these packages (or the
> beginnings of one) involving a new "stale" flag.
> 
> It seems to me that it would be simpler to just start a deprecation path
> for pip's --allow-unverified flag, and allow that deprecation path to
> run its course (with the deprecation message recommending replacing
> --allow-unverified with the appropriate --find-links). By the time
> --allow-unverified is removed from pip at the end of this deprecation
> period, only users of old pip versions might still be relying on legacy
> packages unawares.
> 
> At that point, we'd have two choices. We could just leave those
> unverified links in the simple API for some longer time, choosing not to
> break legacy installers, and knowing that any modern installer will
> totally ignore them anyway. Or we could bite the bullet and remove the
> links, potentially breaking some legacy deploys using legacy installers
> to install legacy packages. I'm not going to venture an opinion on this
> choice right now - I think it could be punted to that later date.
> 
> Getting back to PEP 470 (which I basically support as the direction we
> should be heading), I'd suggest these changes to the PEP text:
> 
> 1. A clearer separation of the various problems the PEP is aiming to
> fix, and acknowledgment that just removing pypi-*-crawl (and leaving
> pypi-explicit) _would_ address at least the user-confusion issue around
> pip's flags (because there would only be --allow-external, whose meaning
> is clear), and might be a reasonable first step along the path towards
> PEP 470's goals.

I can try to make that clearer.

> 
> 2. Add a deprecation path for --allow-unverified; can describe it in
> general terms as "the PEP 438 installer flag allowing installation of
> unverified external packages" if you don't want to be pip-specific.
> Currently PEP 470 has no mention of this, but I think letting a
> deprecation of --allow-unverified fully run its course _before_ making
> breaking changes on the PyPI side is a critical part of making this
> transition in a user-friendlier way.

Perhaps I wasn't being as obvious as I thought! My goal was to nail down what
the final destination looked like, and then once we figured out an end goal
deprecate everything that doesn't match that end goal (and ultimately remove
it).

One thing I don't want to do is add another intermediary fix with it's own
combination of flags that will do the thing that the user wants to do. We
already have:

1) No additional flags
2) --allow-external + --allow-insecure
3) --allow-external + --allow-unverified
4) --allow-unverified

Depending on what version of pip you're using[1]. I really really want the 5th
incantation of this to be the final incantation. That was one of the reasons
why I went the way I did in PEP 470. I don't believe it's possible to move
away from these without a break in compatibility for <=7% of projects,
*however* a key benefit of PEP 470 is that the mechanisms for allowing
additional locations has existed in pip for a long time. We can have a singular
clear message that says "If you want to do X then use these flags" and it
doesn't matter what version you're on. I vastly prefer that to the current
situation (and the "just let the deprecation run it's course" proposal) where
you have to pick the right combination of flags based on pip version.


[1] Technically --allow-external + --allow-insecure works on them all, but it's
    not the preferred or documented way.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140516/2c9bd80d/attachment.sig>