Python Packages : A looming problem? packages might no longer work? (well not on your platform or python version anyway)

[Daniel Fetchinson]

>>>>> Why? Why should every package on PyPI need to support all those
>>>>> Python versions? That should be the decision of the package
>>>>> maintainer. If they want to support every version of Python back to
>>>>> 1.0, they can, and if they want to only support version 2.5 that's
>>>>> fine too.
>>>>
>>>> Why shouldn't packages support more than one python version?
>>>
>>> They can. That depends on the developer of the package.
>>
>> And if the developer decides to support multiple versions currently
>> there is no infrastructural help that would be available to him. On the
>> other hand, there is infrastructural help available to every developer
>> for distributing their apps: it's PyPI itself. You could ask, why do we
>> need PyPI? Let the developers distribute their code in any way they
>> like, python.org doesn't need to support this.
>
> They don't *need* to support distribution. That they do is a nice-to-
> have, not a must-have -- there are other solutions, like SourceForge. We
> can be grateful for PyPI without believing that it's a must-have.

Nobody believes PyPI is a must-have and for sure I didn't say it is a
must-have. PyPI is nice to have not only for developers but also for
users. A multiplatform/multiversion test farm would also be nice for
developers and users alike.

>> The OP is just thinking out loud that it would be great if developers
>> could count on some help for testing various platforms and versions. And
>> I agree, it would indeed be great.
>
> To me, the OP doesn't seem to be asking "Wouldn't it be good if
> developers could get access to whatever systems they wanted?" and more
> like "Somebody needs to develop this amazing technology that will just
> magically make every package on PyPI work on all these different
> platforms, otherwise the Sky Will Fall". Just look at the subject line:
> the issue is described as "a looming problem", it's suggested that
> packages "might no longer work".

You and I probably have a different approach to posts to c.l.p. I try
to interpret things in the best possible light and get the most out of
a suggestion. You are right, the OP was not formulating his proposal
clearly and didn't use a sophisticated enough language but what is
clear is that at the core of the proposal there is something valuable.
Having a test farm would be very useful and would benefit the python
community.

> He's also worried about the combinational explosion of packages,
> platforms and versions, which to my mind is only a problem if you think
> that every package must work on every platform with every version of
> Python.

I don't think every package should work on every platform and with
every version of python. But I do think that many developers want to
support more platforms and versions than what they have access to.
Having a test farm would be beneficial to these developers and their
users.

> I suspect you're primed to read the OP's suggestion in the most positive
> light, because you're already aware of snakebite, and so you're seeing
> his posts in terms of what snakebite is offering. I had never even heard
> of snakebite before now, so I was seeing it in a very different light.

I swear I didn't read this paragraph when I wrote above that I usually
try to interpret posts in the best possible light! This is actually
very funny, you used the exact same phrase :) But even if you have
never heard of snakebite it is simply beyond my understanding how a
developer can argue against a testing farm like this. You seriously
think it would not be a good idea?


>>>>> What's the dire problem you are trying to solve?
>>>>
>>>> Backward and forward compatability of python package resources.
>>>
>>> That's not a problem, that's a feature.
>>>
>>> What's the *problem* that you are trying to solve? What bad thing will
>>> happen if we don't build your proposed system?
>>
>> I fail to understand what is so mysterious about this. The problem is
>> the following: developer X wants to support python versions a, b, c, d
>> on platforms U, V, W. Developer X has no access to platforms V and W and
>> also has no access to python versions b, c and d. Developer X can not
>> run tests on these platforms and python versions although he really
>> wants to. This situation is what is commonly referred to as a problem.
>
> That's *a* problem.

Yes, a problem, that is why I wrote, a problem.

> It has many solutions, snakebite being just one.

Yes, something integrated into PyPI would be another one.

> Is  it the "looming" problem the OP is concerned about?

What are we worried about, the use of the English language or the
technical merit of a proposal?

>> This testing infrastructure should be available to developers who choose
>> to opt in. Once they run their code and notice that there are errors,
>> they will go back and fix these errors that otherwise would be
>> undetected until a real person runs into them. Or they could clearly
>> state what platform and python versions the code runs on for sure and on
>> what platforms and python versions the code fails. This would be a great
>> advantage to developers.
>
> It certainly would.

Great, so we agree!

> But the opt-in arrangement you are talking about doesn't sound much like
> the OP's suggestion that some cross-platform build-bot goes out and pulls
> in every single package on PyPI and tests them all on every combination
> of platforms and versions.

Yes, an automated process for all packages and all platforms and all
versions would be foolish.


>> Actually, the OP's suggestion is a really good one, the people behind
>> snakebite.org realized this already some time ago. To me the idea is
>> very clear and obviously would make the life of developers a whole lot
>> easier and consequently the life of users as well.
>
> Snakebite certainly sounds beneficial for development of Python. I'm not
> sure that it would scale to every developer with a package on PyPI, but
> maybe someday.

That is why the proposal is an important one. Let's say snakebite is
not suitable for PyPI. What can be done on PyPI that helps developers
with multiplatform/multiversion testing?

Cheers,
Daniel


-- 
Psss, psss, put it down! - http://www.cafepress.com/putitdown