[Python-Dev] Status of packaging in 3.3

[Dag Sverre Seljebotn]

On 06/21/2012 01:56 PM, Tarek Ziadé wrote:
> On 6/21/12 11:08 AM, Dag Sverre Seljebotn wrote:
>> ...
>> David Cournapeau's Bento project takes the opposite approach,
>> everything is explicit and without any magic.
>>
>> http://cournape.github.com/Bento/
>>
>> It had its 0.1.0 release a week ago.
>>
>> Please, I don't want to reopen any discussions about Bento here --
>> distutils2 vs. Bento discussions have been less than constructive in
>> the past -- I just wanted to make sure everybody is aware that
>> distutils2 isn't the only horse in this race. I don't know if there
>> are others too?
>>
> That's *exactly* the kind of approach that has made me not want to
> continue.
>
> People are too focused on implementations, and 'how distutils sucks'
> 'how setuptools sucks' etc 'I'll do better' etc
>
> Instead of having all the folks involved in packaging sit down together
> and try to fix the issues together by building PEPs describing what
> would be a common set of standards, they want to create their own tools
> from scratch.

Guido was asked about build issues and scientific software at PyData 
this spring, and his take was that "if scientific users have concerns 
that are that special, perhaps you just need to go and do your own 
thing". Which is what David is doing.

Trailing Q&A session here: http://www.youtube.com/watch?v=QjXJLVINsSA

Generalizing a bit I think it's "web developers" and "scientists" 
typically completely failing to see each others' usecases. I don't know 
if that bridge can be crossed through mailing list discussion alone. I 
know that David tried but came to a point where he just had to 
unsubscribe to distutils-sig.

Sometimes design by committee is just what you want, and sometimes 
design by committee doesn't work. ZeroMQ, for instance, is a great piece 
of software resulting from dropping out of the AQMP committee.

>
> That will not work. And I will say here again what I think we should do
> imho:
>
> 1/ take all the packaging PEPs and rework them until everyone is happy
> (compilation sucks in distutils ? write a PEP !!!)

I think the only way of making scientists happy is to make the build 
tool choice arbitrary (and allow the use of waf, scons, cmake, jam, ant, 
etc. for the build). After all, many projects contains more C++ and 
Fortran code than Python code. (Of course, one could make a PEP saying 
that.)

Right now things are so horribly broken for the scientific community 
that I'm not sure if one *can* sanely specify PEPs. It's more a question 
of playing around and throwing things at the wall and see what sticks -- 
5 years from now one is perhaps in a position where the problem is 
really understood and one can write PEPs.

Perhaps the "web developers" are at the PEP-ing stage already. Great for 
you. But the usecases are really different.

Anyway: I really don't want to start a flame-war here. So let's accept 
up front that we likely won't agree here; I just wanted to clarify my 
position.

(Some context: I might have funding to work 2 months full-time on 
distributing Python software on HPC clusters this autumn. It's not 
really related to Bento (or distutils though, more of a client tool 
using those libraries)

Dag Sverre Seljebotn