[Distutils] moving things forward (was: wheel including files it shouldn't)

[Donald Stufft]

> On May 4, 2016, at 3:39 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 4 May 2016 at 16:03, Robert Collins <robertc at robertcollins.net> wrote:
>> The edits I'd expect to make if the conclusions I suggested in
>> https://mail.python.org/pipermail/distutils-sig/2016-March/028437.html
>> are adopted are:
>> 
>> - change to a Python API
>> - BFDL call on the file format and name
>> 
>> There is no need to issue a new sdist thing, because sdists today are
>> *already* documented across PEPs 241, 314 and 345.
> 
> I already +1'ed using a Python API, but on the file name & format
> side, we have the following candidates and prior art floating around:
> 
> pypa.json in PEP 516
> pypackage.json in PEP 517
> pydist.json in PEP 426
> METADATA (Key: Value) in sdists and wheels
> WHEEL (Key: Value) in wheels
> 
> My impression is that we're generally agreed on wanting to move from
> Key:Value to JSON as the baseline for interoperability formats, so my
> suggestion is to use the name "pybuild.json”.

I'd actually prefer not using JSON for something that is human
editable/writable because I think it's a pretty poor format for that case. It
is overly restrictive in what it allows (for instance, no trailing comma gets
me every time) and the lack of comments I think make it a poor format for that.

I think JSON is great for what gets included *IN* a sdist or a wheel, but for
what sits inside of a VCS checkout that we expect human beings to edit, I think
not.

I'm +1 on tying this to a new extension because I feel like it fundamentally
changes what it means to be a Python sdist. It eliminates the setup.py which
is just about the only thing you actually can depend on existing inside of a
Python sdist and there are a lot of things out there that make the assumption
that Python + ".tar.gz/.zip/etc" == has a setup.py and are going to be broken
from it. A new extension means those tools will ignore it and we can bake in
versioning of the format itself right from the start (even if the new format
looks remarkably like the old format with a different name).

I also believe that we can't provide a replacement for setup.py without either
purposely declaring we no longer support something that people used from it or
providing a way to support that in the new, setup.py-less format.

One thing I remarked to Nataniel yesterday was that it might be a good idea to
drop the build system aspect of these for right now (since afaict all the
invested parties are currently overloaded and/or have a lack of time) and focus
soley on the part of the proposal that enables us to get a real setup_requires
that doesn't involve needing to do the tricky delayed import thing that the
current implementation of setup_requires needs. That would net a pretty huge
win I think since people would be able to use abstractions in their setup.py
(even if they still use setuptools) through a simple ``import statement``,
including the ability to specify what version of setuptools they need to build.
People could still implement non setuptools build systems by mimicing the
setup.py interface (though it will still continue to be less then amazingly
documented/defined) but some of the Numpy folks from one of the previous
threads stated that mimicing setup.py wasn't really the hard part of making
numpy.distutils anyways. The benefit of that is not only a smaller chunk, but
also the chunk that I think (and I could be wrong) that there's no real
disagreement on about how to go about doing it (besides some bikshedding things
like what the filename should be).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160504/ed2634f2/attachment-0001.sig>