[Python-Dev] updating ensurepip to include wheel

[Nick Coghlan]

On 6 August 2015 at 19:04, Robert Collins <robertc at robertcollins.net> wrote:
> On 6 August 2015 at 15:04, Nick Coghlan <ncoghlan at gmail.com> wrote:
> When I consider the harm to a production pipeline that using
> setuptools can cause (in that it triggers easy_install, and
> easy_install has AFAIK none of the security improvements pip has added
> over the last couple years....), I find the acceptance of setuptools,
> but non-acceptance of wheel flummoxing.

When ensurepip was implemented, pip couldn't install from wheel files
without setuptools yet, and the level of adoption of wheel files in
general was lower than it is today.

>> The wheel package itself is only needed in order to support the
>> setuptools "bdist_wheel" command, which then allows pip to implicitly
>> cache wheel files when installing from an sdist.
>>
>> Installing from sdist in production is a *fundamentally bad idea*,
>> because it means you have to have a build toolchain on your production
>> servers. One of the benefits of the wheel format and projects like
>> devpi is that it makes it easier to discourage people from doing that.
>> Even without getting into Linux containers and tools like pyp2rpm,
>> it's also possible to create an entire virtualenv on a build server,
>> bundle that up as an RPM or DEB file, and use the system package
>> manager to do the production deployment.
>
> Yes: but the logic chain from 'its a bad idea' to 'we don't include
> wheel but we do include setuptools' is the bit I'm having a hard time
> with.

Just an accident of history due to the relative timing of ensurepip's
introduction, pip gaining the ability to install wheel files without
setuptools, and high levels of adoption of the wheel format on PyPI.

If PEP 453 was redone today, it's entirely possible setuptools
wouldn't have been bundled, but it wasn't a viable option at the time.
Accepting the bundling was a nice piece of technical debt that bought
several additional months of feature availability :)

>> Hence my request for a PEP - I can see why adding wheel to the
>> ensurepip bundle would be a good idea for upstream, but I can also see
>> why it's a near certainty downstream Linux distros (including Fedora)
>> would take it out again in at least some situations to better meet the
>
> Does Fedora also take out setuptools? If not, why not?

Not at the moment - while I'd like to see the dependency go away
eventually, there are plenty of other things in the world that bother
me more, especially since it comes back the moment someone has an
"import pkg_resources" anywhere in their application.

>> needs of *our* user base. (Since RPM has weak dependency support now,
>> we'd likely make python-wheel a "Recommends:" dependency, rather than
>> a "Requires:" dependency - still installed by default, but easy to
>> omit if not wanted or needed)
>
> So, a new PEP?

Yeah. I don't think it needs to be too fancy, just provide a way to
indicate whether or not ensurepip should install the wheel package,
and make it clear that if folks want to ensure pip can build wheels,
they should install it explicitly (at the command line or as a
dependency), rather than assuming it will always be there by default.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia