Python Packages : A looming problem? packages might no longer work? (well not on your platform or python version anyway)

[Steven D'Aprano]

On Wed, 22 Apr 2009 23:47:08 -0700, Daniel Fetchinson wrote:

>>>> Why? Why should every package on PyPI need to support all those
>>>> Python versions? That should be the decision of the package
>>>> maintainer. If they want to support every version of Python back to
>>>> 1.0, they can, and if they want to only support version 2.5 that's
>>>> fine too.
>>>
>>> Why shouldn't packages support more than one python version?
>>
>> They can. That depends on the developer of the package.
> 
> And if the developer decides to support multiple versions currently
> there is no infrastructural help that would be available to him. On the
> other hand, there is infrastructural help available to every developer
> for distributing their apps: it's PyPI itself. You could ask, why do we
> need PyPI? Let the developers distribute their code in any way they
> like, python.org doesn't need to support this.

They don't *need* to support distribution. That they do is a nice-to-
have, not a must-have -- there are other solutions, like SourceForge. We 
can be grateful for PyPI without believing that it's a must-have.


> The OP is just thinking out loud that it would be great if developers
> could count on some help for testing various platforms and versions. And
> I agree, it would indeed be great.

To me, the OP doesn't seem to be asking "Wouldn't it be good if 
developers could get access to whatever systems they wanted?" and more 
like "Somebody needs to develop this amazing technology that will just 
magically make every package on PyPI work on all these different 
platforms, otherwise the Sky Will Fall". Just look at the subject line: 
the issue is described as "a looming problem", it's suggested that 
packages "might no longer work".

He's also worried about the combinational explosion of packages, 
platforms and versions, which to my mind is only a problem if you think 
that every package must work on every platform with every version of 
Python.

I suspect you're primed to read the OP's suggestion in the most positive 
light, because you're already aware of snakebite, and so you're seeing 
his posts in terms of what snakebite is offering. I had never even heard 
of snakebite before now, so I was seeing it in a very different light.


[...]
>>>> What's the dire problem you are trying to solve?
>>>
>>> Backward and forward compatability of python package resources.
>>
>> That's not a problem, that's a feature.
>>
>> What's the *problem* that you are trying to solve? What bad thing will
>> happen if we don't build your proposed system?
> 
> I fail to understand what is so mysterious about this. The problem is
> the following: developer X wants to support python versions a, b, c, d
> on platforms U, V, W. Developer X has no access to platforms V and W and
> also has no access to python versions b, c and d. Developer X can not
> run tests on these platforms and python versions although he really
> wants to. This situation is what is commonly referred to as a problem.

That's *a* problem. It has many solutions, snakebite being just one. Is 
it the "looming" problem the OP is concerned about?


> This testing infrastructure should be available to developers who choose
> to opt in. Once they run their code and notice that there are errors,
> they will go back and fix these errors that otherwise would be
> undetected until a real person runs into them. Or they could clearly
> state what platform and python versions the code runs on for sure and on
> what platforms and python versions the code fails. This would be a great
> advantage to developers.

It certainly would. 

But the opt-in arrangement you are talking about doesn't sound much like 
the OP's suggestion that some cross-platform build-bot goes out and pulls 
in every single package on PyPI and tests them all on every combination 
of platforms and versions.


> Actually, the OP's suggestion is a really good one, the people behind
> snakebite.org realized this already some time ago. To me the idea is
> very clear and obviously would make the life of developers a whole lot
> easier and consequently the life of users as well.

Snakebite certainly sounds beneficial for development of Python. I'm not 
sure that it would scale to every developer with a package on PyPI, but 
maybe someday.



-- 
Steven