> On May 8, 2016, at 9:13 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
>
> So let's reduce our scope to: "We want *current users* of d2to1 and
> pbr to be able to declare those dependencies to pip and other
> installation tools in a way that avoids the implicit invocation of
> easy_install by setuptools"
I think it's less pbr and d2to1 that we're designing for here (although they'll
benefit from it) but more like numpy.distutils. The reasoning is that pbr and
such have already been able to architecture themselves so that they work within
the current system. One of the goals here is to enable projects that haven't
been able to (or can't reasonably) be written to work as a setuptools extension
to be used here without requiring a manual pre-installation step.
The other side of this is anytime you have dependencies that aren't installed
by pip (such as setup_requires being installed by setuptools) you end up with
a confusing situation where settings don't get passed down into the "inner"
installer (like ``--index-url``) or when they have different resolution
algorithms (like setuptools supporting pre-releases by default) or when they
support different formats (like pip not supporting eggs, but setuptools not
supporting wheels).
However, while I think that a new format is more work just to make sure we
don't back ourselves into a corner, I also don't think it's so much more work
that it's really worth it to skip doing it now. Their isn't a whole lot of ways
to solve the very narrow problem of "we need some dependencies installed prior
to building". We need a field in a file that takes a list of PEP 508 style
specifiers. We can skip any of the build system support right now and add it in
later (and when we add it in, we can just make it so that a lack of a
declaration is an implicit setuptool). So the only real things we need to
decide on are:
1) What format to use? For this I think that it's been pretty clearly that TOML
has had the greatest amount of support here and I think it represents the
best set of trade offs for us.
2) What do we want to call this file? This one is pure bikeshed but also a bit
important since this is one of the things that will be the hardest to change
once we actually pick the name. Ideas I've seen so far (using the toml
extension) are:
* pypa.toml - This one is specific to us which is good, but it's also a bit
bad in that I don't think it's very widely known what the PyPA even is and
I think that the PyPA works better if it's just sort of the thing in the
background.
* pybuild.toml - This one might be a bit too oriented towards building
rather than all of the possible uses of this file, but the bigger problem
with it I think is that hte name clashes with pybuild on Debian which is
their tool used for building Python packages.
* pip.toml - Also specific to us which is good, but I think it's a bit too
overly specific maybe? One of our goals has been to make this stuff not
dependent on a specific implementation so that we can replace it if we
need to (as we more or less replaced distutils with setuptools, or
easy_install with pip) however this also isn't exactly depending on an
implementation-- just the name of a particular implementation. It does
have the benefit that for a lot of people they associate "pip" with
everything packaging related (e.g. I see people calling them
"pip packages" now).
* pymeta.toml - This one might be reasonable, it's a bit generic but at
least it has the ``py`` prefix to tie it to us. Not much to say about it
otherwise.
Overall from those names, I think I probably like pymeta.toml the best (or
maybe just ``meta.toml`` if we don't like the py prefix) but maybe other
people have ideas/opinions that I haven't seen yet.
3) How do we structure the file and what keys do we use for just this part of
the overall feature we're hoping to eventually get (and what semantics do
we give these keys). We could just call it ``setup_requires``, but I think
that's a bad name for something we're not planning on getting rid of since
one of the eventual goals is to make ``setup.py`` itself optional. Another
option is ``build_requires`` but that isn't quite right because we don't
*just* need these dependencies for the build step (``setup.py bdist_wheel``)
but also for the metadata step (``setup.py egg_info``). Although perhaps it
doesn't really matter. Another option is to call them ``bootstrap_requires``
though it's a bit wonky to cram *build* requirements into that.
I guess if I were deciding this I would just call it build requirements
because the nature of ``setup.py`` is that pretty much anything you need to
execute ``setup.py`` is going to be a requirement for all ways you invoke
``setup.py`` (metadata and build and the legacyish direct to install). We
could then say that if a project is missing this new file, then there is an
implicit ``build_requires`` of ``["setuptools", "wheel"]`` but that as soon
as someone defines this file that they are expected to accurately list all
of their build dependencies in it.
Overall, my suggestion here would be to have a file called ``pymeta.toml`` (or
``meta.toml``) and have it look like::
[dependencies]
build = [
"setuptools",
"wheel>=1.0",
]
If at some point we decide we need to add a bootstrap_requires (and then the
ability to add dynamic build requirements) we can do that by just saying that
if you plan on having dynamic build requirements, you need to omit the build
key under the [dependencies] section. This same thing could be used for other
kinds of dependencies too (I've come around to the idea that you can't always
declare a static set of dependencies in a sdist, but you can in a wheel) so
that at some point in the future we could add additional keys like ``runtime``
to this where, if declared, will be assumed to be a static set of dependencies
for that type of dependency.
This also doesn't prevent us from moving more of the metadata to this file in
the future either. For an example, we could (at some point in the future, not
now-- we shouldn't get bogged down in this now, it's just an example of this
not tying us down too much) add something like::
[package]
name = "mycoolpackage"
version = "1.0"
[dependencies]
build = ["setuptools", "wheel>=1.0"]
runtime = ["requests==2.*"]
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig