[Distutils] moving things forward (was: wheel including files it shouldn't)

[Daniel Holth]

Here's the kind of thing that you should expect. Someone will write

setup.cfg:

[bootstrap_requires]
pbr

pip installs pbr in a directory that is added to PYTHONPATH for that build.

The package builds.

And there was much rejoicing.

The big gain from this simple feature is that people will be able to re-use
code and have actual abstractions in setup.py. People have been desperate
for this feature for years.

Imagine that someone forgets to declare their build dependency. Do enough
hand-holding to build a locally cached wheel of the package and submit a
patch to the author. Easy.

In exchange for this potential inconvenience you will get dramatically
easier to write setup.py.

Here is my own implementation of the feature in 34 lines to be prepended to
setup.py: https://bitbucket.org/dholth/setup-requires/src/tip/setup.py , I
see Nick linked to it already, it only executes pip in a subprocess if the
requirements are not already installed but your regularly scheduled
setup.py can just go at the bottom of the file and runs in the same process.

Pedantic note

setup_requires is a setuptools parameter used to install packages after
setup() is called. Even though very many people expect or want those
packages to be installed before setup.py executes. I think it is reasonable
to call the new feature setup_requires but some prefer to eliminate
uncertainty by calling it bootstrap_requires.

On Thu, May 5, 2016 at 6:42 AM Paul Moore <p.f.moore at gmail.com> wrote:

> On 5 May 2016 at 10:10, Nathaniel Smith <njs at pobox.com> wrote:
> > ...The main thing I want to point out though, is that all of these
> > problems you're raising are complications caused entirely by wanting
> > to avoid build isolation in the name of simplicity. If each package
> > gets its own isolated build environment, then it can depend on
> > whatever it wants without any danger of collision with the ambient
> > environment.
>
> Understood. But that doesn't mean they can *only* be solved by build
> isolation. One relatively naive approach might be:
>
> 1. Collect build requirements. If they are satisfied, fine, move on to
> build.
> 2. If the only unsatisfied requirements are new installs, install them
> and move on to build.
> 3. If there are unsatisfied requirements needing an upgrade, do the
> upgrades (possibly require a flag for the pip invocation to allow
> silent upgrades), and move on to build.
> 4. Anything more complicated (downgrades, conflicts) report the issue
> and stop to let the user resolve it manually.
>
> Sure, build isolation might be better (probably is, IMO, but that's
> clearly a topic for some serious debate) but the whole point here is
> to do something incremental, and "good enough to make things better",
> not to stall on a completely perfect solution.
>
> > Premise 1: Without build isolation enabled by default, then in
> > practice everyone will putter along putting up with broken builds all
> > the time. It's *incredibly* easy to forget to declare a build
> > dependency, it's the kind of mistake that every new user makes, and
> > experienced users too.
>
> That's certainly possible, but "some missing dependencies" is still
> better than "no way to specify dependencies at all". And all it needs
> is an end user to hit the problem, raise an issue, and the dependency
> gets added and we're closer to perfection. Why is that worse than the
> current status quo?
>
> > Premise 2: We can either enable build isolation together with the new
> > static bootstrap requirements, or we can never enable build isolation
> > at all, ever.
>
> I have no idea why you feel that's the case. Why can't we add build
> isolation later? You clearly have more experience in this area than I
> do, so I'm genuinely trying to see what I'm missing here. You mention
> "we can't add it later without breaking everything". But the same
> applies now surely? And why can't we introduce the feature gradually -
> initially as opt-in via an "--isolated-build" flag, then in a later
> release change the default to "--isolated-build" with
> "--no-isolated-build" as a fallback for people who haven't fixed their
> builds yet, and ultimately removing "--no-isolated-build" and making
> isolated builds the only option? That's the standard approach we use,
> and I see no reason it wouldn't work here for introducing isolated
> builds at *any* time.
>
> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160505/372ecbc5/attachment-0001.html>