[Distutils] abstract build system PEP update

[Robert Collins]

On 18 February 2016 at 01:44, Donald Stufft <donald at stufft.io> wrote:
>
>> On Feb 17, 2016, at 6:20 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>> Currently, however, people using distutils can create sdists using
>> "setup.py sdist". For other tools, this PEP does not specify how a
>> sdist should be created, but it does imply that it is sufficient to
>> make an archive of a source directory, including a "pypa.json" file,
>> and pip will be able to consume that as a sdist. Whether tools provide
>> a command to do this is out of scope for this PEP.
>
>
> I'm writing my own alternative to this problem (was hoping to have it ready
> yesterday, but I ended up with a migraine and spent most of the day MIA
> instead) and it finally clicked to me why I felt that sdist needed to be a part
> of this, and part of why I felt mildly uncomfortable by the PEP.
>
> Right now, distutils/setuptools provides a number of functions:
>
> * Installation from Sdist
> * Building a Wheel
> * Fetching Metadata
> * Building a sdist
> * Uploading
>
> It's fair to say that I think we're trying to get away from the first of those
> and it's a good thing that neither of these proposals include it. In addition
> both proposals have fetching metadata and building wheels built into them.
>
> Where they fall short is that there isn't a way to build a sdist or upload said
> sdist to PyPI. The assumption is that people will just package up their local
> directories and then publish them to PyPI. However I don't think that
> assumption is reasonable.

I don't understand why that should be part of a PEP about *consuming*
sdists. They're entirely different use cases.

> You could say that using twine to handle the uploading is a thing people should
> do (and I agree!) but that currently relies on having static metadata inside of
> the sdist that twine can parse, static metadata that isn't going to exist if
> you just simply tarball up a directory on disk.

So thats a good reason to have an sdist format newer than PEP314, one
that includes all the data we want to have (and that will hopefully be
usually-static). But the existing sdist PEP's are sufficient to cover
twine's needs today.

> So I think just tarballing up a directory on disk is *not* a reasonable way to
> create a sdist, however you can't reasonably (IMO) create an independent tool
> to handle this task with either of these build proposals either.

Sure you can - PEP 314 already documents PKG-INFO; setup.py has been
defacto - and thats been the blocker for flit- AIUI it at least.

> For instance,
> building a sdist requires getting the version of the project, that is going to
> require some logic which is (in both of these PEPs) build system specific. So
> taking flit for example, in order to build a sdist of a flit using package, a
> tool is going to have to implement the same version detection logic that flit
> itself is using (and the same goes for other metadata like name, and
> long_description). As far as I can tell, any tool to create a sdist is going to
> have to be specific to a particular build tool in order to have the same
> metadata that you would get building a wheel.
>
> The closest that either of these PEPs make it to being able to interrogate a
> directory for the metadata is by asking it for the metadata that would occur
> if we created a wheel file right now. While that is theortically enough
> information, it's also going to contain things which do not belong in a sdist
> at all.
>
> In the end, I think *both* of the current proposals, as written, will cause
> people to not uploads sdists because neither of them really address the fact
> that their intent is to remove the standard interface for doing that, without
> including a replacement to that interface.

I disagree that thats the intent though.


The interface for a developer has the things you listed. The interface
for an installer has been defined in an adhoc fashion as a subset of
'whatever setup.py does' - and we're fixing *that*, but - and see the
preamble for my PEP - the whole point is to decouple the rich do
whatever developers need from the very narrow thing installers like
pip need.

-Rob



-- 
Robert Collins <rbtcollins at hpe.com>
Distinguished Technologist
HP Converged Cloud