[Distutils] Ensuring source availability for PyPI entries / PEP: Build system abstraction for pip/conda etc

[Oscar Benjamin]

On 10 February 2016 at 12:21, M.-A. Lemburg <mal at egenix.com> wrote:
>> So "easy to achieve" still needs someone to take the time to deal with
>> these sorts of issue. It's the usual process of the people willing to
>> put in the effort get to choose the direction (which is also why I
>> just provide feedback, and don't tend to offer my own proposals,
>> because I'm not able to commit that sort of time).
>
> Wait. You are missing the point that the setup.py interface
> already does work, so no extra effort is needed. All that's
> needed is some documentation of what's currently being used,
> so that other tools can support the interface going forward.

You can see an example of a minimal setup.py file here:

https://github.com/oscarbenjamin/setuppytest/blob/master/setuppytest/setup.py

I wrote that some time ago and don't know if it still works (that's
the problem with just having a de facto standard).

> At the moment, pip this interface is only defined by
> "what pip uses" and that's a moving target.

The setup.py interface is a terrible interface for tools like pip to
use and for tools like flit to emulate. Currently what pip does is to
invoke

    $ python setup.py egg_info --egg-base $TEMPDIR

to get the metadata. It is not possible to get the metadata without
executing the setup.py which is problematic for many applications.
Providing a static pypa.json file is much better: tools can read a
static file to get the metadata.

To install a distribution pip runs:

    $ python setup.py install --record $RECORD_FILE         \
                        --single-version-externally-managed

So the setup.py is entirely responsible not just for building but also
for installing everything. This makes it very difficult to develop a
system where different installer tools and different build tools can
cooperate to allow end users to specify installation options. It also
means that the installer has no direct control over where any of the
files are installed.

If you were designing this from scratch then there are some obvious
things that you would want to do differently here. The setup.py
interface also has so much legacy usage that it's difficult for
setuptools and pip to evolve. The idea with this proposal is to
decouple things by introducing a new interface with well defined and
sensible behaviour.

--
Oscar