[Distutils] Expectations on how pip needs to change for Python 3.4

[Nick Coghlan]

On 14 July 2013 00:54, Paul Moore <p.f.moore at gmail.com> wrote:

> This issue has been skirted round for some time now, and I think it needs
> explicit discussion, as I am not at all sure everyone has the same
> expectations.
>
> We're talking about Python 3.4 installations having pip as the default
> package manager - whether by bundling, having a bootstrap process or
> whatever. Regardless of the means, pip will be *the* installer for Python
> 3.4+. And yet, I don't think pip 1.4 currently does what people want "the
> Python 3.4 pip" to do in some ways - and we need to make sure that any work
> on the pip side is understood, agreed to, and planned to match the Python
> 3.4 timescales.
>

Good point. We also need to start to articulate the relevant questions for
the core development side of the fence - Richard, it would be good if PEP
439 could be the vehicle for this, even though it does mean I'm shifting
the goal posts on you fairly substantially.

Then we can turn it into a set of tracker issues for pip and CPython. As a
reminder, here's the current deadlines as per PEP 429 (the 3.4 release
schedule):

Changes to CPython: November 23, 2013 (3.4 beta 1)
Changes to pip: January 18, 2014 (3.4 rc 1)

(allowing changes to bundled applications up until the first RC is what I
think we *should* do, but that will require agreement from Larry Hastings
as release manager)

So, here's my initial list of things that I think people might be expecting
> to happen. This is just my impressions, and I don't necessarily have a view
> on the individual items. And if anyone else can think of other things to
> add to the list, please do so!
>
> 1. Install to user-packages by default.
>

I made this suggestion at one point, but Marcus and others convinced me it
was a bad idea. Issue for better error message filed as
https://github.com/pypa/pip/issues/1048 (I don't have the power to set
milestones)


> 2. Not depend on setuptools (??? - Nick's "inversion" idea)
>

I think Donald is right that bundling a vendor'ed copy of setuptools is the
most sensible near-term option - there's too much risk of upgrade/downgrade
issues when allowing arbitrary combinations of pip with setuptools on
target systems for source based installs.


> 3. Possibly change the wrapper command name from pip to pip3 on Unix.
>

I think there's a bit more to it than that. Really, what we want to try to
ensure is that the following commands are available across Windows, Mac OS
X and *nix (ignoring, for the moment, the behaviour of vendor provided
installations for Mac OS X and Linux):

* python, python3, python3.4
* pip, pip3, pip3.4

That is, the version qualifier on the pip executable would relate to the
*default Python version* associated with that executable/script, rather
than the version of pip. We would take whatever steps were needed in our
Windows and Mac OS X installers to ensure all these wrappers were provided.

The reason why I think we still want to offer "python -m getpip" is because
I think that sends a clearer message to repackagers that we *do* consider
pip a part of Python now, but we keep the source control and issue
management for the two projects separate for pragmatic reasons (notably,
the different update lifecycles). It's still going to map to two separate
source tarballs (and hence SRPMs) for

While I acknowledge you *can* invoke the Python launcher directly on
Windows, I think it is better to leave that in the background as a tool for
advanced users, as well the engine that lets us base shebang line
processing on Windows file associations. If/when we start offering a "py"
style launcher on POSIX systems as well, then we can revisit that question.


> 4. Ensure that pip upgrading itself in-place is sufficiently robust and
> reliable that users don't get "stuck" on the Python-supplied version.
>

As Vinay noted, we also need to ensure downgrades work. However, must of
these have been related to depending on an external setuptools, so
eliminating that should help a lot.


> I'm sure I've seen people say other things that have made me think "are
> you expecting the pip maintainers to make that change?" in the various
> threads, so I doubt this list is definitive.
>

The other big one is the one you noted about pip *not* offering a stable
API, *but* exposing an apparently stable API to introspection.
Introspection currently tells me that pip exports *at least* 32 public
names (and this is without checking for public submodules that aren't
implicitly imported by pip/__init__.py):

>>> import pip; public = set(k for k, v in pip.__dict__.items() if not
k.startswith('_') and (not hasattr(v, "__name__") or hasattr(v,
"__module__") or v.__name__.startswith("pip."))); print(len(public))
32

If pip really has no stable public API, then it should properly indicate
this under introspection (if it already uses relative imports correctly,
then the easiest ways to achieve that are to just shove everything under a
"pip._impl" subpackage or shuffle it sideways into a "_pip" package).


> Comments anyone? Is this discussion premature? The pip maintainers team is
> not huge, so we'll need time (or assistance!) to plan in and make changes
> like this, if they are needed...
>

Agreed, I think refocusing the discussion on "What do we need to do in
pip?" and "What do we need to do in CPython?" is a very necessary step at
this point.


> At a minimum, can we get the key items logged on the pip issue tracker
> with a milestone of Python 3.4?
>

The existing 1.5 milestone is probably usable - I expect 1.5 is the version
that would be bundled with 3.4.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130714/85fbe705/attachment.html>