Mailman 3 September 2016 - Distutils-SIG

Re: [Distutils] What is the official position on distutils?
by Sylvain Corlay 04 Sep '16

04 Sep '16

Hi Steve, - I posted an on setuptools github https://github.com/pypa/ setuptools/issues/771 to ask the setuptools folks (and especially Jason) whether they would want such a feature. I still find it baffling that the preferred way to add distutils features would be to monkey-patch it via setuptools first and then to add it into distutils later, while my proposed approach was to add it to distutils and monkey patch for earlier versions of python. - Regarding the policy that you just nominated, does this apply to things that have been in pip for a long time? In the initial email that triggered this discussion, it was also question of adding the `pip.location.distutils_scheme` convenience function to distutils itself. Cf https://bugs.python.org/issue26955 and pip's implementation at https://github.com/pypa/pip/blob/281eb61b09d87765d7c2b92f6982b3fe76ccb0af/p… Indeed, the problem with certain distutils commands like `install_headers` is that the only way to retrieve the directory where things were installed is to instantiate a `distutils.dist.Distribution` object, which is not trivial to do. pip's distutils_scheme convenience function does it. This function is *about* distutils, and IMO should really be *in* distutils. Thanks, Sylvain On Sun, Sep 4, 2016 at 4:02 PM, Steve Dower <steve.dower(a)python.org> wrote: > "add it to setuptools first, and then add things to distutils where we > feel they're sufficiently stable to not need the benefit of the faster > setuptools update cycle" > > I nominate this as the official policy on API changes for distutils, with > regular maintenance mode policies applying to anything else. > > Top-posted from my Windows Phone > ------------------------------ > From: Nick Coghlan <ncoghlan(a)gmail.com> > Sent: ‎9/‎4/‎2016 2:19 > To: Sylvain Corlay <sylvain.corlay(a)gmail.com> > Cc: distutils sig <distutils-sig(a)python.org> > Subject: Re: [Distutils] What is the official position on distutils? > > On 4 September 2016 at 06:44, Sylvain Corlay <sylvain.corlay(a)gmail.com> > wrote: > > Hi Brett, > > > > On Sat, Sep 3, 2016 at 8:05 PM, Brett Cannon <brett(a)python.org> wrote: > >> > >> > >> If Jason is up for the responsibility that seems like a reasonable > >> approach to take. It also helps test out features in setuptools first > before > >> upstreaming it. > >> > > > > How do you see `has_flag` get into setuptools? By monkey-patching > distutils' > > ccompiler to have it aside of `has_function` when setuptools is imported? > > > > I find really weird the idea of adding this in a convoluted fashion > instead > > of allowing incremental improvement of distutils. > > The change to distutils would still be a plain patch to distutils, it > would just be accepted at the API design level in setuptools first. > > The problem you're running into right now isn't a technical one - it's > that there isn't anyone that currently feels like they have sufficient > design authority over the distutils API to accept your proposal, hence > Brett starting this thread to address that underlying recurring > question, rather than the specifics of your change. > > Jason *definitely* has that design authority over setuptools though, > and will be tasked with making any API additions available on older > versions of Python via setuptools regardless of what policy we adopt > for distutils maintenance, so if he's amenable to the idea, it makes > sense to me to invert the order we ask those questions: add it to > setuptools first, and then add things to distutils where we feel > they're sufficiently stable to not need the benefit of the faster > setuptools update cycle. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia > _______________________________________________ > Distutils-SIG maillist - Distutils-SIG(a)python.org > https://mail.python.org/mailman/listinfo/distutils-sig >

2 1

setup_requires: the obvious option(?)
by Antony Lee 03 Sep '16

03 Sep '16

Hi all, The `setup_requires` option to `setup()` is well-known to suffer from multiple issues. Most importantly, as it is a keyword argument to `setup()`, it appears too late for modules that may need to be imported for the build to occur (e.g., Cython, for which support must explicitly provided by setuptools itself rather than by letting Cython hook into it); additionally, there are various contorsions that people go to to avoid some `setup_requires` when not building the package (such as checking the value of `sys.argv`). `setup_requires` also uses `easy_install` rather than `pip`, but I do not see why this could not be fixed; let's focus on the first issue instead. If `setup_requires` appears too late to be useful, the obvious(?) option is to move it earlier: provide a function, say, `setuptools.setup_requires()`, that should be called *before* `setup()` itself, e.g.: from setuptools import setup, setup_requires setup_requires("numpy", needed_for=["build_ext"]) try: import numpy as np except ImportError: np = None setup(..., include_dirs=[np.get_include()] if np else []) When `setup.py` is invoked, either directly or by pip, upon the call to `setup_requires()`, if `sys.argv[0]` is in the `needed_for` kwarg, and at least one requirement is missing, `setup_requires()` calls asks pip to install the required packages (similarly to ` https://bitbucket.org/dholth/setup-requires`) in a temporary directory, and the whole Python process replaces itself (in the `os.execv()` sense) by a new call to `python setup.py` with this temporary directory prepended to the PYTHONPATH. In this new process, the arguments to `setup_requires()` are now available and we can proceed to the rest of `setup.py`. I feel like this idea is natural enough that someone must already have come up with it... but I may be missing something :-) Best, Antony

6 10

Accepting PEP 527 (reducing the variety of file types supported by PyPI)
by Nick Coghlan 02 Sep '16

02 Sep '16

Having reviewed the comments on the PEP 527 thread and the latest draft of the PEP, I'm now accepting the PEP. This means that: * supported sdist types will be reduced to .tar.gz and .zip * projects will need to choose one or the other for future releases * supported binary formats will be reduced to bdist_wheel and bdist_egg The problem that Maurits van Rees pointed out with ez_setup.py currently still specifically looking for a zip file and processing it accordingly can be resolved by setuptools settling on zip as its sdist publication format (unless/until ez_setup.py is changed to handle tar.gz). Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 1