[Distutils] Library instability on PyPI and impact on OpenStack

[Reinout van Rees]

On 03-03-13 17:07, Mark McLoughlin wrote:
>> >I consider it good form for a setup.py to declare as loose
>> >dependencies as possible (no version qualifier or a >= version
>> >qualifier) and for an application to provide a requires.txt or a
>> >buildout that has stricter requirements.
> Interesting!
>
> I feel like I'm missing some context on the latter part - mostly because
> I hadn't come across buildout, so more reading for me! - but if the idea
> is that a buildout/requires.txt specifies the versions that a developer
> should use when working on the project ... how do you avoid a situation
> where developers are happily working on one stack of libraries and the
> app either no longer works with the minimum versions specified in
> setup.py or the latest versions published upstream?

Unless you explicitly test for this, you cannot really know.

- If you always unpin the vesions and grab the latest one, you cannot be 
sure it still works with the minimum version.

- If you pin on the lowest version, you cannot know if it breaks with 
the latest.

You could test for this explicitly by having two different buildouts (or 
requirements.txt files) and test both.


Perhaps some input from my practice can help. In reality it isn't so 
much of a problem.

- If I encounter a problem ("hey, I now should use djangorestframework 
2.x instead of the 0.3.x I was using because some package uses the new 
imports"), I fix up my setup.py. A colleague switched to the latest 
djangorestframework with an incompatible API, so my code broke. I added 
the ">= 2.0" requirement to my setup.py, ensuring I'm at least getting a 
good explicit error message instead of a unclear ImportError when 
running my site.

- Sites are conservative and are pinned down solid.

- Libraries are more loose. Often certain packages aren't pinned, which 
makes you note errors earlier.

The 2nd and 3rd point provide a bit of certainty at both ends of the 
spectrum. The 1st is a handy method to ensure nothing goes terribly 
wrong. Ok, it is reactive instead of proactive, but you should encounter 
errors early enough (when developing features) so you can update your 
min/max versions before you do a release.


Reinout

-- 
Reinout van Rees                    http://reinout.vanrees.org/
reinout at vanrees.org             http://www.nelen-schuurmans.nl/
"If you're not sure what to do, make something. -- Paul Graham"