On 5 May 2016 at 16:22, Nathaniel Smith <njs@pobox.com> wrote: ...
I'm sympathetic to the general approach, but on net I think I prefer a slightly different proposal.
Downsides to just standardizing [setup_requires]:
If I write a PEP, it won't be standardising setup_requires, it will be standardising bootstrap requirements. The distinction is nuance but critical: - we don't expect setuptools to ever need to honour it (of course, it could, should it choose to, and ease of adoption may suggest that it should) - as a new feature, making it opt in allows folk to adopt it when they are ready; if it was literally just a new home for setup_requires, we may see build systems auto-populating it, and the potential swamp you describe below would then apply.
- if projects have existing ones, that's actually kinda good / kinda bad -- pip has never been respecting these before, so if we suddenly start doing that then existing stuff could break. I don't know how likely this is, but experience suggests that *something* will break and make someone angry. (I'm still blinking at getting angry complaints arguing that uploading a wheel to pypi, where before there was only an sdist, should be treated as a compatibility-breaking change that requires a new version etc.)
Yes, things will break: anyone using this will need a new pip, by definition. Not everyone will be willing to wait 10 years before using it :).
- IMO an extremely valuable aspect of this new declarative setup-requirements thing is that it gives us an opportunity to switch to enforcing the accuracy of this metadata. Right now we have a swamp we need to drain, where there's really no way to know what environment any given setup.py needs to run. Sometimes there are setup_requires, sometimes not; if there are setup_requires then sometimes they're
Huh? I've not encountered any of this, ever. I'd love some examples to go look at. The only issue I've ever had with setup_requires is the easy_install stuff it ties into. ...
instead of punishing them -- is if we make the rule be "projects that use the declarative setup-requirements feature also get isolated build environments". (Then the messaging is: "see, this helps you check that you actually set it up right! if your new metadata works for you in testing, it'll also work for your users!) But this again would mean we can't reuse the existing [setup_requires] section.
I'm very much against forcing isolated build environments as part of this effort. I get where you are coming from, but really it conflates two entirely separate things, and you'll *utterly* break building anything with dependencies that are e.g. SWIG based unless you increase the size of the PEP by about 10-fold. (Thats not hyperbole, I think). Working around that will require a bunch of UX work, and its transitive: you have to expose how-to-workaround-the-fact-that-all-our-deps-are-not-installable all the way up the chain. That's probably a good thing to do (e.g. see bindep, or the aborted callout-to-system-packages we proposed after PyCon AU last year), but tying these two things together is not necessary, though I can certainly see the appeal; the main impact I see is that it will just impede adoption. The reality, AFAICT, is that most projects with undeclared build deps today get corrected fairly quickly: a bug is filed, folk fix it, and we move on. A robotic system that isolates everything such that folk *cannot* fix it is much less usable, and I'm very much in favour of pragmatism here.
- setup.cfg is kind of a terrible format for standardizing things because the only definition of the format is "read the ConfigParser source". You cannot parse setup.cfg from a non-Python language. And the *only* benefit is that it already exists; teaching pip to read pybuild.json or pybuild.toml instead would be completely trivial. So if we aren't going to try and grandfather in existing [setup_requires] sections, then we might as well switch to a better file format at the same time -- this is not the hard part.
The patch to read a single list-valued key out of setup.cfg is trivial and shallow. We've not managed to settle on consensus on a file format choice in a year of debate. I hold zero confidence we will going forward either. If the BDFL delegate makes a call - fine. I read Nick's earlier email in the thread as such a thing TBH :).
So I like the idea of splitting out the declarative setup-requirements PEP from the new build system hook interface PEP, but I think that the way we should do this is by defining a new pybuild.whatever file like the ones in PEP 516 / PEP 517 (they're identical in this regard) that *only* has schema-version and bootstrap-requirements, and then we can add the build backend key as a second step.
I think the risk there is that that presumes the answer, and without the abstract build effort actually moving forward we may be years before we actually see the file being used for anything else. If ever. -Rob