[Distutils] How to specify dependencies in Python

[Glyph Lefkowitz]

> On Jan 16, 2017, at 7:59 AM, Thomas Güttler <guettliml at thomas-guettler.de> wrote:
> 
> I think requirements.txt should be the result of some kind of Continous-Integration run.
> If all tests are successful, then requirements.txt should be created with "pip freeze".
> 
> This means, that the Continous-Integration run does not use requirements.txt to
> build its environment.
> 
> Next question: Where is the best place to store requirements.txt?
> 
> I think it should not be in the repo of a library. It should be somehow outside.

I think I understand what you're trying to say here, but I think you have it backwards.

The best example I have made of the "right" way to do this is in this project: https://github.com/rackerlabs/mimic/ <https://github.com/rackerlabs/mimic/>

The project has both a setup.py and (several versions of) requirements.txt.

setup.py gives the abstract requirements for the project.  This is what should work, but has not necessarily been exactly tested to.

requirements.txt is the concrete requirements.  This is generated (more or less) by freezing an environment where one does `pip install .` to trigger the setup.py.

However, all continuous integration tests are run against the versions listed in requirements.txt (the details are here: https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/install.sh#L101-L105 <https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/install.sh#L101-L105> and here https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/run.sh#L18 <https://github.com/rackerlabs/mimic/blob/5fae30d9e9a45c15f3f0a51fa436a7f25502b742/.travis/run.sh#L18>) to ensure that new contributors always have a stable set of versions to test against, and their PR won't end up randomly having to fix some upgrade issue.

Finally, we use https://requires.io <https://requires.io/> to submit PRs every time one of the dependencies upgrades.  On a regular basis (every 6 months or so), these do cause errors; when they do, development unrelated to the version upgrade can continue on unimpeded, and the version-compatibility fix lives neatly in its own PR that solves just that problem.

It's a bit subtle to understand the distinction between setup.py and requirements.txt (https://caremad.io/posts/2013/07/setup-vs-requirement/ <https://caremad.io/posts/2013/07/setup-vs-requirement/> is a good attempt, but I think stumbles over explaining some nuances that are obvious to dstufft and not to anyone else), but if you get them lined up correctly then a bunch of edge-cases work very nicely, and reasoning about deployment is much easier.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170116/d08c4f6c/attachment-0001.html>