[Distutils] PEP439 and backward compat / easy_install / distlib

[holger krekel]

On Sun, Mar 24, 2013 at 09:04 -0700, Nick Coghlan wrote:
> On Sun, Mar 24, 2013 at 2:48 AM, holger krekel <holger at merlinux.eu> wrote:
> > Hi Richard, all,
> >
> > two first notes on PEP439.
> 
> PEP 439 is just one small piece of a much larger puzzle, and the
> entire puzzle won't be explained in this PEP. I realise this makes it
> hard to evaluate in isolation, but the beta freeze for 3.4 is still
> several months away and all the other pieces will be in place well
> before then. Please be patient with us as we get all the pieces
> documented and published over the coming weeks.

Looking forward to more details.  It's indeed a bit hard to get
a grasp on what's going on currently.  Even though i've even just written 
the related PEP438 myself.  In any case, each PEP should make enough
sense on its own or refer to other PEPs or documents for details.

> However, I do have some comments on your specific questions:
> 
> - while easy_install has provided good service for many years, it is
> not a viable choice as the officially supported installer. It's
> default behaviour is completely broken from a system administrator's
> point of view, as using it to install things has side effects on every
> Python application run on that system, and the lack of uninstall
> support is not acceptable either. By contrast, pip has tried from the
> beginning to accommodate the interests of system administrators *as
> well as* developers, making it much easier to justify its being
> blessed as the official installer. The approach taken to this over the
> coming months will be to identify the reasons that people still use
> easy_install for some tasks and add support for them to pip.

I understand your high level motivation and views here.  Can you
point to a more technical detailed comparison of easy_install and pip?

FWIW I've heart from many people that they have to use easy_install
because of egg support and the many packages in that format out there.
This is why i asked about a "backward compatibility" strategy 
specifically. 

> - eggs are too closely associated with easy_install to easily
> rehabilitate their image with system administrators and platform
> packagers, and also lack some of the necessary metadata to
> interoperate correctly with platform packaging tools. The new wheel
> format builds on a combination of eggs and the sysconfig installation
> path concept to create a format that can be more readily mapped to FHS
> compliant platform specific packages. Wheel also introduces the
> enhanced "compatibility tag" format for filenames, which covers more
> details of the Python and platform dependencies of the built
> distribution.

I also have the impression that wheels are a very good development.
My last mail didn't question the merits of wheels over eggs, though.

> - We could potentially provide server side support for automatically
> generating wheels from eggs uploaded to PyPI, but that would be a
> question for catalog-sig (since it is purely about PyPI's feature set
> and behaviour, and independent of the packaging and distribution
> standards themselves)

Maybe. In any case, I see this the issue of automated egg->wheel
conversion is on topic for PEP439.  It should be part of a focus
on an evolutionary strategy (rather than a pure replacement one) 
which helps to get as many people on board and benefiting from the PEP
as possible.
 
> - metadata 2.0 specifically includes the "Setup-Requires-Dist" field
> so that projects that require additional dependencies when building
> from source will be correctly supported by pip and other metadata 2.0
> compliant installers. This will be supported transparently for users,
> so long as they update to metadata 2.0 compliant versions of their
> build tools (setuptools/distribute, distutils2, d2to1, hashdist, etc).

Sounds good.  The present-day packages which will not be updated 
(soon or ever), may be a problem, though.  

> Python 3.4 will also continue to provide distutils, so that metadata
> 1.0 and 1.1 packages generated with older versions of distutils will
> continue to work correctly.
> 
> - once we can bootstrap pip, then bootstrapping easy_install if it
> still needed for some edge cases will be as easy as installing
> anything else that is either pure Python or publishes an appropriate
> wheel for the platform: "pip install setuptools"

Well, one of the stated goals is to make it easier for package maintainers
to explain things.  If you have to explain pip-bootstrapping, then 
install setuptools, then the actual package, it's not much of an 
improvement anymore.  Especially if you need to explain things
depending on Python version.  To avoid the latter the PEP439
should include offering installer-bootstraps from python.org
for older Python versions, particularly Python2.  This way
we could see the benefits of PEP439 and related developments
much much earlier than if we all need to wait until everyone
uses Python3.4 :)

FWIW I am open to do a hangout over all this some time.

best,
holger

> Cheers,
> Nick.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>