Re: [Numpy-discussion] Meta: too many numerical libraries doing the same thing?

"Christos Siopis <siopis@umich.edu>" <siopis@umich.edu> writes:
I agree that sounds nice in theory. But even if it were technically feasible (which I doubt) given the language differences, it would be a development project that is simply too big for scientists to handle as a side job, even if they were willing (which again I doubt). My impression is that the organizational aspects of software development are often neglected. Some people are good programmers but can't work well in teams. Others can work in teams, but are not good coordinators. A big project requires at least one, if not several, people who are good scientist and programmers, have coordinator skills, and a job description that permits them to take up the task. Plus a larger number of people who are good scientists and programmers and can work in teams. Finally, all of these have to agree on languages, design principles, etc. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

Another factor that complicates things is open source philosophy and the licenses that go with it. The GSL project looks very promising, and the ultimate goals of that project appear to be to create a coherent and complete numerical library. This kind of thing NEEDS to be open source, and the GSL folks have chosen a license (GPL) that guarantees that it remains that way. That is a good thing. The license also make it impossible to use the library in closed source projects, which is a deal killer for a lot of people, but it is also an important attribute for many folks that don't think there should be closed source projects at all. I believe that that will greatly stifle the potential of the project, but it fits with the philosophy iof it's creators. Personally I think the LGPL would have guaranteed the future openness of the source, and allowed a much greater user (and therefor contributer) base. BTW, IANAL either, but my reading of the GPL and Python's "GPL compatable" license, is that GSL could be used with Python, but the result would have to be released under the GPL. That means it could not be imbedded in a closed source project. As a rule, Python itself and most of the libraries I have seen for it (Numeric, wxPython, etc.) are released under licences that allow propriatary use, so we probably don't want to make Numeric, or SciPy GPL. too bad. On another note, it looks like the blitz++ library might be a good basis for a general Numerical library (and NumPy 3) as well. It does come with a flexible license. Any thoughts? -Chris -- Christopher Barker, Ph.D. ChrisHBarker@home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------

Chris Barker <chrishbarker@home.net> writes:
I think the major question is whether we are willing to move to C++. And if we want to keep up any pretentions for Numeric becoming part of the Python core, this translates into whether Guido will accept C++ code in the Python core. pragmatically, is blitz++ reasonably efficient with g++? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

Konrad Hinsen wrote:
Actually, It's worse than that. Blitz++ makes heavy use of templates, and thus only works with compilers that support that well. The current Python core can compile under a very wide variety of compilers. I doubt that Guido would want to change that. Personally, I'm torn. I would very much like to see NumPy arrays become part of the core Python, but don't want to have to compromise what it could be to do that. Another idea is to extend the SciPy project to become a complete Python distribution, that would clearly include Numeric. One download, and you have all you need.
I know g++ is supported (and I think it is their primary development platform). From the web site: Is there a way to soup up C++ so that we can keep the advanced language features but ditch the poor performance? This is the goal of the Blitz++ project: to develop techniques which will enable C++ to rival -- and in some cases even exceed -- the speed of Fortran for numerical computing, while preserving an object-oriented interface. The Blitz++ Numerical Library is being constructed as a testbed for these techniques. Recent benchmarks show C++ encroaching steadily on Fortran's high-performance monopoly, and for some benchmarks, C++ is even faster than Fortran! These results are being obtained not through better optimizing compilers, preprocessors, or language extensions, but through the use of template techniques. By using templates cleverly, optimizations such as loop fusion, unrolling, tiling, and algorithm specialization can be performed automatically at compile time. see: http://www.oonumerics.org/blitz/whatis.html for more info. I havn't messed with it myself, but from the web page, it seems the answer is yes, C++ can produce high performance code. -- Christopher Barker, Ph.D. ChrisHBarker@home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------

I'm currently testing the SciPy Blitz++ features with FDTD. Should have some comparisons soon. Right now my statements are compiling, but not giving the right answers :( I think they might have it fixed soon. Rob. Chris Barker wrote:
-- The Numeric Python EM Project www.members.home.net/europax

Another factor that complicates things is open source philosophy and the licenses that go with it. The GSL project looks very promising, and the ultimate goals of that project appear to be to create a coherent and complete numerical library. This kind of thing NEEDS to be open source, and the GSL folks have chosen a license (GPL) that guarantees that it remains that way. That is a good thing. The license also make it impossible to use the library in closed source projects, which is a deal killer for a lot of people, but it is also an important attribute for many folks that don't think there should be closed source projects at all. I believe that that will greatly stifle the potential of the project, but it fits with the philosophy iof it's creators. Personally I think the LGPL would have guaranteed the future openness of the source, and allowed a much greater user (and therefor contributer) base. BTW, IANAL either, but my reading of the GPL and Python's "GPL compatable" license, is that GSL could be used with Python, but the result would have to be released under the GPL. That means it could not be imbedded in a closed source project. As a rule, Python itself and most of the libraries I have seen for it (Numeric, wxPython, etc.) are released under licences that allow propriatary use, so we probably don't want to make Numeric, or SciPy GPL. too bad. On another note, it looks like the blitz++ library might be a good basis for a general Numerical library (and NumPy 3) as well. It does come with a flexible license. Any thoughts? -Chris -- Christopher Barker, Ph.D. ChrisHBarker@home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------

Chris Barker <chrishbarker@home.net> writes:
I think the major question is whether we are willing to move to C++. And if we want to keep up any pretentions for Numeric becoming part of the Python core, this translates into whether Guido will accept C++ code in the Python core. pragmatically, is blitz++ reasonably efficient with g++? Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------

Konrad Hinsen wrote:
Actually, It's worse than that. Blitz++ makes heavy use of templates, and thus only works with compilers that support that well. The current Python core can compile under a very wide variety of compilers. I doubt that Guido would want to change that. Personally, I'm torn. I would very much like to see NumPy arrays become part of the core Python, but don't want to have to compromise what it could be to do that. Another idea is to extend the SciPy project to become a complete Python distribution, that would clearly include Numeric. One download, and you have all you need.
I know g++ is supported (and I think it is their primary development platform). From the web site: Is there a way to soup up C++ so that we can keep the advanced language features but ditch the poor performance? This is the goal of the Blitz++ project: to develop techniques which will enable C++ to rival -- and in some cases even exceed -- the speed of Fortran for numerical computing, while preserving an object-oriented interface. The Blitz++ Numerical Library is being constructed as a testbed for these techniques. Recent benchmarks show C++ encroaching steadily on Fortran's high-performance monopoly, and for some benchmarks, C++ is even faster than Fortran! These results are being obtained not through better optimizing compilers, preprocessors, or language extensions, but through the use of template techniques. By using templates cleverly, optimizations such as loop fusion, unrolling, tiling, and algorithm specialization can be performed automatically at compile time. see: http://www.oonumerics.org/blitz/whatis.html for more info. I havn't messed with it myself, but from the web page, it seems the answer is yes, C++ can produce high performance code. -- Christopher Barker, Ph.D. ChrisHBarker@home.net --- --- --- http://members.home.net/barkerlohmann ---@@ -----@@ -----@@ ------@@@ ------@@@ ------@@@ Oil Spill Modeling ------ @ ------ @ ------ @ Water Resources Engineering ------- --------- -------- Coastal and Fluvial Hydrodynamics -------------------------------------- ------------------------------------------------------------------------

I'm currently testing the SciPy Blitz++ features with FDTD. Should have some comparisons soon. Right now my statements are compiling, but not giving the right answers :( I think they might have it fixed soon. Rob. Chris Barker wrote:
-- The Numeric Python EM Project www.members.home.net/europax
participants (3)
-
Chris Barker
-
Konrad Hinsen
-
Rob