On 06/21/2012 03:23 PM, Tarek Ziadé wrote:
On 6/21/12 2:45 PM, Dag Sverre Seljebotn wrote:
Guido was asked about build issues and scientific software at PyData this spring, and his take was that "if scientific users have concerns that are that special, perhaps you just need to go and do your own thing". Which is what David is doing.
Trailing Q&A session here: http://www.youtube.com/watch?v=QjXJLVINsSA
if you know what you want and have a tool that does it, why bother using distutils ?
But then, what your community will do with the guy that create packages with distutils ? just tell him he suck ?
The whole idea is *interoperability*, not the tool used.
Generalizing a bit I think it's "web developers" and "scientists" typically completely failing to see each others' usecases. I don't know if that bridge can be crossed through mailing list discussion alone. I know that David tried but came to a point where he just had to unsubscribe to distutils-sig.
I was there, and sorry to be blunt, but he came to tell us we had to drop distutils because it sucked, and left because we did not follow that path
Sometimes design by committee is just what you want, and sometimes design by committee doesn't work. ZeroMQ, for instance, is a great piece of software resulting from dropping out of the AQMP committee.
That will not work. And I will say here again what I think we should do imho:
1/ take all the packaging PEPs and rework them until everyone is happy (compilation sucks in distutils ? write a PEP !!!)
I think the only way of making scientists happy is to make the build tool choice arbitrary (and allow the use of waf, scons, cmake, jam, ant, etc. for the build). After all, many projects contains more C++ and Fortran code than Python code. (Of course, one could make a PEP saying that.)
Right now things are so horribly broken for the scientific community that I'm not sure if one *can* sanely specify PEPs. It's more a question of playing around and throwing things at the wall and see what sticks -- 5 years from now one is perhaps in a position where the problem is really understood and one can write PEPs.
Perhaps the "web developers" are at the PEP-ing stage already. Great for you. But the usecases are really different.
If you sit down and ask your self: "what are the information a python project should give me so I can compile its extensions ?" I think this has nothing to do with the tools/implementations.
I'm not sure if I understand. A project can't "give the information needed to build it". The build system is an integrated piece of the code and package itself. Making the build of library X work on some ugly HPC setup Y is part of the development of X. To my mind a solution looks something like (and Bento is close to this): Step 1) "Some standard" to do configuration of a package (--prefix and other what-goes-where options, what libraries to link with, what compilers to use...) Step 2) Launch the package's custom build system (may be Unix shell script or makefile in some cases (sometimes portability is not a goal), may be a waf build) Step 3) "Some standard" to be able to cleanly install/uninstall/upgrade the product of step 2) An attempt to do Step 2) in a major way in the packaging framework itself, and have the package just "declare" its C extensions, would not work. It's fine to have a way in the packaging framework that works for trivial cases, but it's impossible to create something that works for every case.
And if we're able to write down in a PEP this, e.g. the information a compiler is looking for to do its job, then any tool out there waf, scons, cmake, jam, ant, etc, can do the job, no ?
Anyway: I really don't want to start a flame-war here. So let's accept up front that we likely won't agree here; I just wanted to clarify my position.
After 4 years I still don't understand what "we won't agree" means in this context. *NO ONE* ever ever came and told me : here's what I want a Python project to describe for its extensions.
That's unfortunate. To be honest, it's probably partly because it's easier to say what won't work than come with a constructive suggestion. A lot of people (me included) just use waf/cmake/autotools, and forget about making the code installable through PyPI or any of the standard Python tools. Just because that works *now* for us, but we don't have any good ideas for how to make this into something that works on a wider scale. I think David is one of the few who has really dug into the matter and tried to find something that can both do builds and work through standard install mechanisms. I can't answer for why you haven't been able to understand one another. It may also be an issue with how much one can constructively do on mailing lists. Perhaps the only route forward is to to bring people together in person and walk distutils2 people through some hairy scientific HPC builds (and vice versa).
Just "we won't agree" or "distutils sucks" :)
Gosh I hope we will overcome this lock one day, and move forward :D
Well, me too. Dag