The setup.py of numpy looks big and complicated: https://github.com/numpy/numpy/blob/master/setup.py I guess there are even more complicated setup.py files. Do you think it will be possible in the future to make setup.py conditionless (no "if", no "else", no function calls)? I mean that the setuptools/distutils/whatever libraries will handle all needed conditions? This would mean setup.py would be a pure data structure which could be encoded in json or yaml. What do you think? Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/
Hi Thomas! Have you seen Pipfile? It's something along those lines - a declarative specification of project data. :) github.com/pypa/pipfile Regards, Pradyun Gedam On Fri, Aug 25, 2017, 14:12 Thomas Güttler <guettliml@thomas-guettler.de> wrote:
The setup.py of numpy looks big and complicated: https://github.com/numpy/numpy/blob/master/setup.py
I guess there are even more complicated setup.py files.
Do you think it will be possible in the future to make setup.py conditionless (no "if", no "else", no function calls)?
I mean that the setuptools/distutils/whatever libraries will handle all needed conditions?
This would mean setup.py would be a pure data structure which could be encoded in json or yaml.
What do you think?
Regards, Thomas Güttler
-- Thomas Guettler http://www.thomas-guettler.de/ _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 25 August 2017 at 09:36, Thomas Güttler <guettliml@thomas-guettler.de> wrote:
The setup.py of numpy looks big and complicated: https://github.com/numpy/numpy/blob/master/setup.py
I guess there are even more complicated setup.py files.
Do you think it will be possible in the future to make setup.py conditionless (no "if", no "else", no function calls)?
I mean that the setuptools/distutils/whatever libraries will handle all needed conditions?
This would mean setup.py would be a pure data structure which could be encoded in json or yaml.
What do you think?
The goal (under PEP 517, setuptools is merely one possible "backend") is that some backends will allow purely declarative descriptions of build requirements. Flit already does this. Whether setuptools will ever switch to a purely declarative form I don't know (but I doubt it - it's more likely that someone will write an alternative backend to replace some or all of setuptools, as flit did). And whether projects with requirements as complex as numpy and similar will be able to manage with a purely declarative build description is even more questionable, I suspect. But certainly, once PEP 517 is implemented and as flit gains popularity, I fully expect more and more projects to use a static data structure for their metadata (flit.ini, specifically). Paul
On 2017-08-25 10:50:11 +0100 (+0100), Paul Moore wrote: [...]
once PEP 517 is implemented and as flit gains popularity, I fully expect more and more projects to use a static data structure for their metadata (flit.ini, specifically).
This has also been possible for years already using either PBR or distutils2. For example, hundreds of Python packages produced by the OpenStack community use a branchless boilerplate setup.py which declares a setup_requires on the pbr package, and then everything else goes into an INI-formatted setup.cfg file (except for install_requires which are drawn from requirements.txt instead). -- Jeremy Stanley
Except that OpenStack frequently rejects outside use cases as I learned as an OpenStack developer who tried to improve PBR. Sadly it will never be seen as a global solution as long as that continues On Aug 25, 2017 6:32 AM, "Jeremy Stanley" <fungi@yuggoth.org> wrote:
On 2017-08-25 10:50:11 +0100 (+0100), Paul Moore wrote: [...]
once PEP 517 is implemented and as flit gains popularity, I fully expect more and more projects to use a static data structure for their metadata (flit.ini, specifically).
This has also been possible for years already using either PBR or distutils2. For example, hundreds of Python packages produced by the OpenStack community use a branchless boilerplate setup.py which declares a setup_requires on the pbr package, and then everything else goes into an INI-formatted setup.cfg file (except for install_requires which are drawn from requirements.txt instead). -- Jeremy Stanley
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 2017-08-25 07:37:23 -0500 (-0500), Ian Stapleton Cordasco wrote:
Except that OpenStack frequently rejects outside use cases as I learned as an OpenStack developer who tried to improve PBR. Sadly it will never be seen as a global solution as long as that continues
Many projects frequently reject use cases regardless of their proponents, if they don't fit with the scope as identified by their maintainers. No project can be all things to all users and expect to remain manageable in the long term. I've not seen patches rejected out of hand because of who/where the use case came from, and while I'm curious to know which change(s) this was it's probably off-topic for this ML. -- Jeremy Stanley
It's important to realize that build is a Turing-complete activity. We write programs to build programs. Where setup.py gets in trouble is by making both the metadata and the build script the same thing, making it harder to reason about that metadata. That is why we are working towards being able to express the metadata in pyproject.toml, while letting you have any kind of build script you please. We expect that many more projects will be able to deal with this limitation compared to e.g. always having to express the whole build with static configuration of something like distutils2 or PBR.
On 25 August 2017 at 12:14, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2017-08-25 10:50:11 +0100 (+0100), Paul Moore wrote: [...]
once PEP 517 is implemented and as flit gains popularity, I fully expect more and more projects to use a static data structure for their metadata (flit.ini, specifically).
This has also been possible for years already using either PBR or distutils2. For example, hundreds of Python packages produced by the OpenStack community use a branchless boilerplate setup.py which declares a setup_requires on the pbr package, and then everything else goes into an INI-formatted setup.cfg file (except for install_requires which are drawn from requirements.txt instead).
Cool. I'm not that familiar with those tools, but if they enable that sort of use then that's great. I did get the impression that they were for more complex/specialised use cases, though, whereas flit (with PEP 517) is much more about simple configuration for the majority of (pure Python) projects that don't need complex behaviour. But that's mostly about target audiences than capabilities. One thought - are the PBR and/or distutils2 teams looking at providing PEP 517 support? Assuming they are, have they had a change to review the PEP to ensure it suits their needs? And if they aren't, what is it about the PEP that makes them unwilling to do so? Paul
On 2017-08-25 14:00:25 +0100 (+0100), Paul Moore wrote: [...]
I'm not that familiar with those tools, but if they enable that sort of use then that's great. I did get the impression that they were for more complex/specialised use cases, though, whereas flit (with PEP 517) is much more about simple configuration for the majority of (pure Python) projects that don't need complex behaviour. [...]
Not really, no; pretty much everything in OpenStack is pure Python as well and relies on PBR. I can't personally think of any non-pure package I've seen using PBR, so it may not even be usable outside pure Python packaging for all I know (I've certainly never tried anyway). PBR basically started with the primary goals of reducing copied code by simplifying common setup.py file patterns and integrating features to help avoid additional setup_requires which couldn't easily be versioned (especially back in those days) without causing all manner of breakage when they might end up conflicting with transitive install_requires. With PEP 517 on the way now I consider these earlier attempts at declarative package definitions will be more of a historical footnote soon enough, but the intent was pretty similar. Here's an example setup.cfg from a reasonably simple command-line utility: https://git.openstack.org/cgit/openstack-infra/bindep/tree/setup.cfg Anyway, my point wasn't to advertise these tools specifically, but rather to point out that there's nothing stopping projects who want to extract their package metadata into static configuration and bundle their setuptools logic into a separate reusable module (for example by leveraging a setuptools entrypoint the way PBR does) without having to wait around for PEP 517. Then they can always revisit the design once PEP 517 support is more generally rolled out in standard packaging tools.
One thought - are the PBR and/or distutils2 teams looking at providing PEP 517 support? Assuming they are, have they had a change to review the PEP to ensure it suits their needs? And if they aren't, what is it about the PEP that makes them unwilling to do so?
As far as I know, Distutils2 has essentially been dead upstream for ~5 years, so I wouldn't expect it to receive any updates in that regard. I've had conversations with the primary maintainers of PBR about what would need to be done to coexist with and eventually take advantage of PEP 517 paradigms. On-the-fly generation of pyproject.toml files during or, if necessary, preceding the sdist build phase is probably the way they'll be looking to go there, though the details have yet to be completely hashed out. (The community around it is sensitive to gender diversity issues and wants to avoid acquiring more of a "brogrammer" image, so some of us worry that any conspicuous TOML files checked into revision control repositories could be seen as a tacit endorsement of the author's alleged behavior at GH a few years ago.) -- Jeremy Stanley
On Fri, Aug 25, 2017 at 1:00 PM, Jeremy Stanley <fungi@yuggoth.org> wrote:
(The community around it is sensitive to gender diversity issues and wants to avoid acquiring more of a "brogrammer" image, so some of us worry that any conspicuous TOML files checked into revision control repositories could be seen as a tacit endorsement of the author's alleged behavior at GH a few years ago.)
I was one of the folks championing TOML during the original discussions, and this is an issue that also worried me a lot. In case it's a useful data point: I actually contacted several of the main rust/cargo developers, since they were the major users of TOML and are also well known to be sensitive to these issues, to ask if they've had any issues with this, and they said that they haven't heard any complaints. Obviously there's a difference between "no-one complained" and "no-one was bothered", and I suspect the community's existing reputation may affect how this is interpreted as well, but... maybe useful as a data point. Between this and the way the TOML spec appears to have been abandoned at v0.4 (with the admonition "you should assume that is is unstable and act accordingly") I've wondered if we should fork it, rename it "the obvious minimal language", and release our own 1.0. -n -- Nathaniel J. Smith -- https://vorpus.org
I personally do not understand the aversion to YAML. I mean yes, the specification is more complicated, but it's also more popular and the YAML files will not be complex enough for a C library to help that much. And since it's more popular, people might even prefer specifying package metadata in a pyproject.yaml. pip could even cache a wheel of the pyyaml package between builds that could be imported at build time with a zipimporter rather than vendoring the package. And as a plus it's not named after an alleged sexist. Honestly this is not an issue that interests me very much but this rant is because I was surprised that toml was chosen when I first found out about it. 2017-08-25 18:16 GMT-05:00 Nathaniel Smith <njs@pobox.com>:
On Fri, Aug 25, 2017 at 1:00 PM, Jeremy Stanley <fungi@yuggoth.org> wrote:
(The community around it is sensitive to gender diversity issues and wants to avoid acquiring more of a "brogrammer" image, so some of us worry that any conspicuous TOML files checked into revision control repositories could be seen as a tacit endorsement of the author's alleged behavior at GH a few years ago.)
I was one of the folks championing TOML during the original discussions, and this is an issue that also worried me a lot. In case it's a useful data point: I actually contacted several of the main rust/cargo developers, since they were the major users of TOML and are also well known to be sensitive to these issues, to ask if they've had any issues with this, and they said that they haven't heard any complaints.
Obviously there's a difference between "no-one complained" and "no-one was bothered", and I suspect the community's existing reputation may affect how this is interpreted as well, but... maybe useful as a data point.
Between this and the way the TOML spec appears to have been abandoned at v0.4 (with the admonition "you should assume that is is unstable and act accordingly") I've wondered if we should fork it, rename it "the obvious minimal language", and release our own 1.0.
-n
-- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On Fri, Aug 25, 2017 at 5:46 PM, xoviat <xoviat@gmail.com> wrote:
I personally do not understand the aversion to YAML. I mean yes, the specification is more complicated, but it's also more popular and the YAML files will not be complex enough for a C library to help that much. And since it's more popular, people might even prefer specifying package metadata in a pyproject.yaml. pip could even cache a wheel of the pyyaml package between builds that could be imported at build time with a zipimporter rather than vendoring the package. And as a plus it's not named after an alleged sexist.
Honestly this is not an issue that interests me very much but this rant is because I was surprised that toml was chosen when I first found out about it.
If you want to know why it was chosen then there's lots of discussion in the list archives. I don't think this is a great place to relitigate it. -n -- Nathaniel J. Smith -- https://vorpus.org
Me too. YAML is *so much* more widely used, and the complicated edge cases can simply be ignored for this requirement. Maybe it's just because I've never heard of whatever improper behavior the author engaged in, but I don't think a data format needs to answer to the actions of its creator(s). If G. Klyne or C. Newman were to do something dreadful I wouldn't want dates to stop following ISO-8601.[*] [*] I don't even know the 8601 authors' first names, and assume they are good and honorable people. Just making an analogy. On Aug 25, 2017 5:46 PM, "xoviat" <xoviat@gmail.com> wrote:
I personally do not understand the aversion to YAML. I mean yes, the specification is more complicated, but it's also more popular and the YAML files will not be complex enough for a C library to help that much. And since it's more popular, people might even prefer specifying package metadata in a pyproject.yaml. pip could even cache a wheel of the pyyaml package between builds that could be imported at build time with a zipimporter rather than vendoring the package. And as a plus it's not named after an alleged sexist.
Honestly this is not an issue that interests me very much but this rant is because I was surprised that toml was chosen when I first found out about it.
2017-08-25 18:16 GMT-05:00 Nathaniel Smith <njs@pobox.com>:
On Fri, Aug 25, 2017 at 1:00 PM, Jeremy Stanley <fungi@yuggoth.org> wrote:
(The community around it is sensitive to gender diversity issues and wants to avoid acquiring more of a "brogrammer" image, so some of us worry that any conspicuous TOML files checked into revision control repositories could be seen as a tacit endorsement of the author's alleged behavior at GH a few years ago.)
I was one of the folks championing TOML during the original discussions, and this is an issue that also worried me a lot. In case it's a useful data point: I actually contacted several of the main rust/cargo developers, since they were the major users of TOML and are also well known to be sensitive to these issues, to ask if they've had any issues with this, and they said that they haven't heard any complaints.
Obviously there's a difference between "no-one complained" and "no-one was bothered", and I suspect the community's existing reputation may affect how this is interpreted as well, but... maybe useful as a data point.
Between this and the way the TOML spec appears to have been abandoned at v0.4 (with the admonition "you should assume that is is unstable and act accordingly") I've wondered if we should fork it, rename it "the obvious minimal language", and release our own 1.0.
-n
-- Nathaniel J. Smith -- https://vorpus.org _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
On 2017-08-25 16:16:39 -0700 (-0700), Nathaniel Smith wrote: [...]
I've wondered if we should fork it, rename it "the obvious minimal language", and release our own 1.0.
And make it a toggleable option to configparser? ;) The basic format is pretty similar, granted TOML has a lot more flexibility over configparser's classic loader. -- Jeremy Stanley
participants (9)
-
Daniel Holth -
David Mertz -
Ian Stapleton Cordasco -
Jeremy Stanley -
Nathaniel Smith -
Paul Moore -
Pradyun Gedam -
Thomas Güttler -
xoviat