[Distutils] build system abstraction PEP, take #2

[Robert Collins]

On 10 December 2015 at 08:59, Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>
> On Wed, Dec 9, 2015 at 2:55 AM, Robert Collins <robertc at robertcollins.net>
> wrote:
>>
>> Updated - tl;dr:
>>
>> The thing I'm least happy about is that implementing install support
>> will require recursively calling back into pip, that or reimplementing
>> the installation of wheels logic from within pip - because
>> sufficiently old pip's won't call wheel at all.
>
>
> You're specifying a new interface here, and updating pip itself is quite
> easy. So why would you do things you're not happy about to support
> "sufficiently old pip's"?

Because the goal is to make the barrier to entry for new build systems
low. Right now, for instance, flit doesn't support building an sdist
(From its pypi page : "

Flit only creates packages in the new ‘wheel’ format. People using
older versions of pip (<1.5) or easy_install will not be able to
install them. People may also want a traditional sdist for other
reasons, such as Linux distro packaging. I hope that these problems
will diminsh over time.

")

The current defacto contract is:
 - egg-info
 - install
 - wheel
 - develop

The issues with this are:
 - install is a responsibility for pip/conda etc, not build systems
 - ditto develop
 - there's no way to bootstrap the build system
 - there's no way to determine build requirements

Of those four things, two are very low hanging fruit (the last two),
one is nontrivial - develop mode - and one is inbetween - since we're
requiring 'wheel', "install" is something pip can do itself with a
trivial change... but it shuffles the complexity onto the setuptools
shim, which we likely want to enable rapid adoption.

So we can choose:
 - delay the ability for pip/conda etc to own the install stage [by
putting install in the contract]
 - centralise that complexity by having setuptools_shim wear it

So - so far the second is better I think, I just don't like it :)

>> And even modern pips
>> can be told *not to call wheel*.
>
>
> Isn't that something you can ignore? If the plan for pip anyway is to always
> go sdist-wheel-install, why support this flag for a new build interface?

Well, there's still debate about that. I think its waste and will piss
developers off (heck, even in tox OpenStack folk find sdist too long
and disable it routinely - we've added CI checks that sdist doesn't
error to allow keeping the local developer workflow smooth).

-Rob



-- 
Robert Collins <rbtcollins at hpe.com>
Distinguished Technologist
HP Converged Cloud