[Distutils] setup_requires for dev environments

[Donald Stufft]

> On Mar 16, 2015, at 2:53 PM, Daniel Holth <dholth at gmail.com> wrote:
> 
> No one should be asked to learn how to extend distutils, and in
> practice no one knows how.

Some people know how, pytest figured it out, pbr figured it out. There’s
some documentation at https://pythonhosted.org/setuptools/setuptools.html#extending-and-reusing-setuptools

It is true though that it’s not particularly great documentation and
actually doing it is somewhat of an annoyance.

> 
> People have been begging for years for working setup_requires, far
> longer than I've been interested in it, and all they want to do is
> 
> import fetch_version
> setup(version=fetch_version(), ...)
> 
> Then they will eventually notice setup_requires has never worked the
> way most people expect. As a result there are too few setup.py
> abstractions.
> 
> The other proposal is a little bit interesting. Parse setup.py without
> running it, extract setup_requires, and pip install locally before
> running the file? It would be easy as long as the setup_requires were
> defined as a literal list in the setup() call, but you would have to
> tell people they were not allowed to write normal clever Python code.
> I think the gotchas would be severe…

I wasn’t going to try and parse the setup.py, I was going to execute it.

Here’s a proof of concept I started on awhile back to try and validate
the overall idea: https://github.com/pypa/pip/compare/develop...dstufft:eldritch-horror

Since then I’ve thought about it more and decided it’d probably be better
to instead of trying to shuffle arguments into the subprocess, have the
subprocess write out into a file or stdout or something what all of the
setup_requires are. This would require executing the setup.py 3x instead
of 2x like pip is currently doing.

This would also enable people do something like:

try:
    import fetch_version
except ImportError:
    def fetch_version():
        return “UNKNOWN”

setup(version=fetch_version(), …)

If they are happy with mandating that their thing can only be installed from
sdist with pip newer than X, because given a three pass installation (once
to discover setup_requires, once to write egg_info, once to actually install)
as long as the setup_requires list doesn’t rely on anything installed then
the first pass can have no real information except the setup_requires.

It actually wouldn’t even really be completely broken, it’d just have a nonsense
version number (or whatever piece of metadata can’t be located).

> 
> Release a setuptools command class that actually works with regular
> setup_requires, and parses setup_requires out of a side file? But
> fails fetch_version case...
> 
> The main reason the installer should handle setup_requires instead of
> setup.py is related to one of the major goals of packaging, which is
> to get setup.py out of the business of installing (it is OK if it is a
> build script).
> 
> Would you be interested in a JSON-format metadata that you were
> willing to support long term, with a quick version 0.1 release that
> only adds the setup_requires feature?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150316/3a28c481/attachment-0001.sig>