Re: [Numpy-discussion] Meta: too many numerical libraries doing the same thing?

I am reviving the 'too many numerical libraries doing the same thing?' thread, with apologies for the long delay to respond to the list's comments. Thanks to Joe Harrington, Paul Dubois, Konrad Hinsen and Chris Barker who took the time to respond to my posting. The funny thing is that, as i see it, even though they were mostly trying to provide reasons why such a project has not, and probably will not, be done, their reasons sound to me as reasons why it *should* be done! In essence, what i am 'proposing' is for a big umbrella organization (NSF, NASA and IEEE come to mind) to sponsor the development of this uber-library for numerical scientific and engineering applications. This would be 'sold' as an infrastructure project: creating the essential functionality that is needed in order to build most kinds of scientific and engineering applications. It would save lots of duplication effort and improve productivity and quality at government labs, academia and the private sector alike. The end product would have some sort of open-source license (this can be a thorny issue, but i am sure a mutually satisfactory solution can be found). Alternatively (or in addition), it might be better to specify the API and leave the implementation part (open source and/or commercial) to others. Below I give some reasons, based mostly on the feedback from the list, why i think it is appropriate that this effort be pursued at such a high level (both bureaucratically and talent-wise). - The task is too complex to be carried out by a single group of people (e.g., by a single government lab): * Not only does it involve 'too much work', but the expertise and the priorities of the group won't necessarily reflect those of the scientific and engineering community at large. * The resources of a single lab/group do not suffice. Even if they did, project managers primarily care to produce what is needed for their project. If it has beneficial side-effects for the community, so much the better, but this is not a priority. Alas, since the proposed task is too general to fit in the needs of any single research group, noone will probably ever carry it out. - The end product should be something that the 'average' numerical scientist will feel compelled to use. For instance, if the majority of the community does not feel at home with object-oriented techniques, there should be a layer (if technically possible) that would allow access to the library via more traditional languages (C/Fortran), all the while helping people to make the transition to OO, when appropriate --i can hear Paul saying "but it's *always* appropriate" :) - There is a number of issues for which it is not presently clear that there is a good answer. Paul brought up some of these problems. For example, is it possible to build the library in layers such that it can be accessed at different levels as object components, as objects, and as 'usual', non-OO functions all at the same time? As Paul said, "Eiffel or C++ versions of some NAG routines typically have methods with one or two arguments while the C or Fortran ones have 15 or more". Does NAG use the same code-base for both its non-OO and OO products? - It is hard to find people who are good scientists, good numerical scientists, good programmers, have good coordinator skills and can work in a team. This was presented as a reason why a project such as this proposed cannot be done. But in fact i think these are all arguments why a widely-publicised and funded project such as this would have more chances to succeed than small-scale, individual efforts. I would also imagine that numerical analysts might feel more compelled to 'publicize' their work to the masses if there were single, respected, quality resource that provides the context in which they can deposit their work (or a draft thereof, anyway). - Some companies and managers are hesitant to use open-source products due to requirements for 'fiduciary responsibility', as Paul put it. I think that a product created under the conditions that i described would probably pass this requirement --not in the sense that there will be an entity to sue, but in the sense that it would be a reputable product sponsored by some of the top research organizations or even the government. *************************************************************** / Christos Siopis | Tel : 734-615-1585 \ / Postdoctoral Research Fellow | \ / Department of Astronomy | FAX : 734-763-6317 \ / University of Michigan | \ / Ann Arbor, MI 48109-1090 | E-mail : siopis@umich.edu \ / U.S.A. _____________________| \ / / http://www.astro.lsa.umich.edu/People/siopis.html \ ***************************************************************

Chris wrote in part: -----Original Message----- From: numpy-discussion-admin@lists.sourceforge.net [mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Christos Siopis <siopis@umich.edu> Sent: Thursday, December 06, 2001 11:53 PM To: numpy-discussion@lists.sourceforge.net Subject: Re: [Numpy-discussion] Meta: too many numerical libraries doing the same thing? <snip> In essence, what i am 'proposing' is for a big umbrella organization (NSF, NASA and IEEE come to mind) to sponsor the development of this uber-library for numerical scientific and engineering applications. This would be 'sold' as an infrastructure project: creating the essential functionality that is needed in order to build most kinds of scientific and engineering applications. It would save lots of duplication effort and improve productivity and quality at government labs, academia and the private sector alike. The end product would have some sort of open-source license (this can be a thorny issue, but i am sure a mutually satisfactory solution can be found). ----------- Those who do not know history, etc. LLNL, LANL, and Sandia had such a project in the 70s called the SLATEC library for mathematical software. It was pretty successful for the Fortran era. However, the funding agencies are unable to maintain interest in infrastructure very long. If there came a day when the vast majority of scientific programmers shared a platform and a language, there is now the communications infrastructure so that they could do a good open-source library, given someone to lead it with some vision. Linus Mathguy.
participants (2)
-
Christos Siopis <siopis@umich.edu>
-
Paul F. Dubois