[Distutils] comparison of configuration languages

[Brett Cannon]

Based on this email and Nathaniel's evaluation I've gone ahead and taken it
upon myself to start writing a PEP so we have something concrete to work
from. I'm hoping to have it done some time this week.

On Sun, 8 May 2016 at 08:43 Donald Stufft <donald at stufft.io> wrote:

>
> > On May 8, 2016, at 9:13 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > So let's reduce our scope to: "We want *current users* of d2to1 and
> > pbr to be able to declare those dependencies to pip and other
> > installation tools in a way that avoids the implicit invocation of
> > easy_install by setuptools"
>
> I think it's less pbr and d2to1 that we're designing for here (although
> they'll
> benefit from it) but more like numpy.distutils. The reasoning is that pbr
> and
> such have already been able to architecture themselves so that they work
> within
> the current system. One of the goals here is to enable projects that
> haven't
> been able to (or can't reasonably) be written to work as a setuptools
> extension
> to be used here without requiring a manual pre-installation step.
>
> The other side of this is anytime you have dependencies that aren't
> installed
> by pip (such as setup_requires being installed by setuptools) you end up
> with
> a confusing situation where settings don't get passed down into the "inner"
> installer (like ``--index-url``) or when they have different resolution
> algorithms (like setuptools supporting pre-releases by default) or when
> they
> support different formats (like pip not supporting eggs, but setuptools not
> supporting wheels).
>
> However, while I think that a new format is more work just to make sure we
> don't back ourselves into a corner, I also don't think it's so much more
> work
> that it's really worth it to skip doing it now. Their isn't a whole lot of
> ways
> to solve the very narrow problem of "we need some dependencies installed
> prior
> to building". We need a field in a file that takes a list of PEP 508 style
> specifiers. We can skip any of the build system support right now and add
> it in
> later (and when we add it in, we can just make it so that a lack of a
> declaration is an implicit setuptool). So the only real things we need to
> decide on are:
>
> 1) What format to use? For this I think that it's been pretty clearly that
> TOML
>    has had the greatest amount of support here and I think it represents
> the
>    best set of trade offs for us.
>
> 2) What do we want to call this file? This one is pure bikeshed but also a
> bit
>    important since this is one of the things that will be the hardest to
> change
>    once we actually pick the name. Ideas I've seen so far (using the toml
>    extension) are:
>
>    * pypa.toml - This one is specific to us which is good, but it's also a
> bit
>      bad in that I don't think it's very widely known what the PyPA even
> is and
>      I think that the PyPA works better if it's just sort of the thing in
> the
>      background.
>
>    * pybuild.toml - This one might be a bit too oriented towards building
>      rather than all of the possible uses of this file, but the bigger
> problem
>      with it I think is that hte name clashes with pybuild on Debian which
> is
>      their tool used for building Python packages.
>
>    * pip.toml - Also specific to us which is good, but I think it's a bit
> too
>      overly specific maybe? One of our goals has been to make this stuff
> not
>      dependent on a specific implementation so that we can replace it if we
>      need to (as we more or less replaced distutils with setuptools, or
>      easy_install with pip) however this also isn't exactly depending on an
>      implementation-- just the name of a particular implementation. It does
>      have the benefit that for a lot of people they associate "pip" with
>      everything packaging related (e.g. I see people calling them
>      "pip packages" now).
>
>    * pymeta.toml - This one might be reasonable, it's a bit generic but at
>      least it has the ``py`` prefix to tie it to us. Not much to say about
> it
>      otherwise.
>
>    Overall from those names, I think I probably like pymeta.toml the best
> (or
>    maybe just ``meta.toml`` if we don't like the py prefix) but maybe other
>    people have ideas/opinions that I haven't seen yet.
>
> 3) How do we structure the file and what keys do we use for just this part
> of
>    the overall feature we're hoping to eventually get (and what semantics
> do
>    we give these keys). We could just call it ``setup_requires``, but I
> think
>    that's a bad name for something we're not planning on getting rid of
> since
>    one of the eventual goals is to make ``setup.py`` itself optional.
> Another
>    option is ``build_requires`` but that isn't quite right because we don't
>    *just* need these dependencies for the build step (``setup.py
> bdist_wheel``)
>    but also for the metadata step (``setup.py egg_info``). Although
> perhaps it
>    doesn't really matter. Another option is to call them
> ``bootstrap_requires``
>    though it's a bit wonky to cram *build* requirements into that.
>
>    I guess if I were deciding this I would just call it build requirements
>    because the nature of ``setup.py`` is that pretty much anything you
> need to
>    execute ``setup.py`` is going to be a requirement for all ways you
> invoke
>    ``setup.py`` (metadata and build and the legacyish direct to install).
> We
>    could then say that if a project is missing this new file, then there
> is an
>    implicit ``build_requires`` of ``["setuptools", "wheel"]`` but that as
> soon
>    as someone defines this file that they are expected to accurately list
> all
>    of their build dependencies in it.
>
>
> Overall, my suggestion here would be to have a file called ``pymeta.toml``
> (or
> ``meta.toml``) and have it look like::
>
>     [dependencies]
>     build = [
>         "setuptools",
>         "wheel>=1.0",
>     ]
>
> If at some point we decide we need to add a bootstrap_requires (and then
> the
> ability to add dynamic build requirements) we can do that by just saying
> that
> if you plan on having dynamic build requirements, you need to omit the
> build
> key under the [dependencies] section. This same thing could be used for
> other
> kinds of dependencies too (I've come around to the idea that you can't
> always
> declare a static set of dependencies in a sdist, but you can in a wheel) so
> that at some point in the future we could add additional keys like
> ``runtime``
> to this where, if declared, will be assumed to be a static set of
> dependencies
> for that type of dependency.
>
> This also doesn't prevent us from moving more of the metadata to this file
> in
> the future either. For an example, we could (at some point in the future,
> not
> now-- we shouldn't get bogged down in this now, it's just an example of
> this
> not tying us down too much) add something like::
>
>     [package]
>     name = "mycoolpackage"
>     version = "1.0"
>
>     [dependencies]
>     build = ["setuptools", "wheel>=1.0"]
>     runtime = ["requests==2.*"]
>
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160508/42e68fe0/attachment-0001.html>