RE: [Numpy-discussion] Status of Numeric
Perry Greenfield writes:
Peter J. Verveer writes:
I was under the impression that Numarray was intended to be a replacement for Numeric, also as a building block for larger packages such as SciPy. Was Numarray not intended to be an "improved Numeric" in the first place? I chose to develop for Numarray rather than Numeric because of its improvements, under the assumption that eventually my code would also become available to the users of such packages as SciPy. (I wrote the nd_image extension that is now distributed with Numarray. I also contributed some improvements to RandomArray extension that are not in the Numeric version.)
It has been our intention to port scipy to use numarray soon. This work has been delayed somewhat since our current focus is on plotting. We do still intend to see that scipy works with numarray.
That this discussion is happening NOW really surprises me. I have been following this list for a couple of years now, with the intention of eventually using numerical Python as the main teaching toolbox for numerical analysis, and possibly for the migration small research codes as well. The possibility of doing numerics in Phython has always intrigued me. Right now I am primarily using Matlab. It's very powerful, but not free and the language is horrible; Octave is trying to play catch up but has mostly lost steam. So a good scientific Phython environment (of any sort) would be a really cool thing to have. However, two things have always held me back (apart from coding small examples on a few occasions): 1. Numerical Phython has been in a limbo for too long (I had even assumed a few times that both Numeric and Numarray were dead for all practical purposes). If there are two incompatible version for years and no clear indication where the whole thing is going, I am very hesitant to invest any time into writing substantial code, or recommend it for class room use. 2. Plotting is a major issue. There are a couple of semi-functional packages, but neither a comprehensive solution nor a clear direction for the plotting architecture. Short, I see a lot of potential, unused mainly because the numerical Python community seems to lack clear direction and leadership. This is a real showstopper for someone who is primarily interested in building on top. I am still hopeful that something will come of all this - any progress will be very much appreciated. Best regards, Marcel --------------------------------------------------------------------- Marcel Oliver Phone: +49-421-200-3212 School of Engineering and Science Fax: +49-421-200-3103 International University Bremen m.oliver@iu-bremen.de Campus Ring 1 oliver@member.ams.org 28759 Bremen, Germany http://math.iu-bremen.de/oliver ---------------------------------------------------------------------
Perry Greenfield writes:
It has been our intention to port scipy to use numarray soon. This work has been delayed somewhat since our current focus is on plotting.
That is good news. What plotting package are you working on? Last I heard Chaco had turned into Enthought's (and STSci) in-house Windows only package. (Not because they want it that way, but because they don't have funding to make it work on other platforms, and support the broader community). I don't see anything new on the SciPy page after August '03. Frankly, weak plotting is a bigger deal to me than array performance. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Tuesday, January 20, 2004, at 06:08 PM, Chris Barker wrote:
Perry Greenfield writes:
It has been our intention to port scipy to use numarray soon. This work has been delayed somewhat since our current focus is on plotting.
That is good news. What plotting package are you working on? Last I heard Chaco had turned into Enthought's (and STSci) in-house Windows only package. (Not because they want it that way, but because they don't have funding to make it work on other platforms, and support the broader community).
I don't see anything new on the SciPy page after August '03.
Frankly, weak plotting is a bigger deal to me than array performance.
Yes, I agree completely (and why we are giving plotting higher priority than scipy integration). I really was hoping to raise this issue later, but I might as well address it since the Numeric/numarray issue has raised it indirectly. Chaco had been the focus of our plotting efforts for more than a year. The effort started with our funding Enthought to start the effort. We had a number of requirements for a plotting package that weren't met by any existing package, and it didn't appear that any would be easily modified to our needs. The requirements we had (off the top of my head) included: 1) easy portability to graphics devices *and* different windowing systems. 2) it had to run on all major platforms including Solaris, Linux, Macs, and Windows. 3) the graphics had to be embedable within gui widgets. 4) it had to allow cursor interactions, at least to the point of being able to read cursor positions from python. 5) it had to be open source and preferably not gpl (though the latter was probably not a show stopper for us) 6) It also had to be customizable to the point of being able to produce very high quality hardcopy plots suitable for publication. 7) object oriented plotting framework capable of sensible composition. 8) command line interface akin to that available in matlab or IDL to make producing quick interactive plots very, very easy. Developing something that satisfies these is not at all trivial. In the process Enthought has expended much energy developing chaco, kiva and traits (and lately they are working on yet more extensions); easily much more of the effort has come from sources other than STScI. Kiva is the back end that presents a uniform api for different graphics devices. Traits handles many of the user interface issues for plot parameters, and handling the relationships of these parameters between plot components. Chaco is the higher level plotting software that provides the traditional plotting capabilities for 2-d data. Much has been invested in chaco. It is with some regret that we (STScI) have concluded that chaco is not suitable for our needs and that we need to take a different approach (or at least give it a try). I'll take some space to explain why. The short answer is that in the end we think it was too ambitious. We still aim to achieve the goals I listed above. The problem we think is that chaco was also tasked to try to achieve extra goals with regard interactive capabilities that were in the end, not really important to STScI and it's community, but were important to Enthought (and presumably its clients, and the scipy community). More specifically, a lot of thought and work went into making many aspects of the plots could be interactively modified. That is, by clicking on various aspects of plots, one could bring up editors for the attributes of that plot element, such as color, line style, font, size, etc. Many other interactive aspects have been enhanced as well. Much recent work by Enthought is going into extending the capabilities even further by adding gui kinds of features (e.g., widgets of all sorts). Unfortunately these capabilities have come at a price, namely complexity. We have found it difficult to track the ongoing changes to chaco to become proficient enough to contribute significantly by adding capabilities we have needed. Perhaps that argues that we aren't competent to do so. To a certain degree, that is probably is true. There is no doubt that Enthought has some very talented software engineers working on chaco and related products. On the other hand, our goal is to have this software be accessible by scientists in general, and particularly astronomers. Chaco is complex enough that we think that is a serious problem. Customizing it's behavior requires a very large investment of time understanding how it works, far beyond what most astronomers are willing to tackle (at least that's my impression). Much of this complexity (and many of its ongoing changes) is to support the interactive capabilities, and to make it responsive enough that plots can update themselves quickly enough not to lead to annoying lags. But frankly, we just want something to render plots on the screen and on hardcopy. Outside of being able to obtain cursor coordinates, we find many of the interactive capabilities as secondary in importance. When most astronomers want to tune a plot (either for publication quality, or for batch processing), they usually want to be able to reproduce the adjustments for new data, for which the interactive attribute editing capability is of little use. Generally they would like to script the the more customized plots so that they can be easily modified and reused. So it seems that it is too difficult to accomplish all these aims within one package. We would like to develop a different plotting package (using many of ideas from chaco, and some code) based on kiva and the traits package. We have started on this over the past month, and hope to have some simple functionality available within a month (though when we make it public may take a bit longer). It will be open source and we hope significantly simpler than chaco. It will not focus on speed (well, we want fairly fast display times for plots of a reasonable number of points, but we don't need video refresh rates). If your interest in plotting matches ours, then this may be for you. We will welcome contributions and comments once we get it off the ground. (We are calling it pyxis by the way). Enthought is continuing to work on chaco and at some point that will be mature, and will be capable of some sophisticated things. That may be more appropriate for some than what we are working on. Perry Greenfield
On Tuesday, January 20, 2004, at 05:53 PM, Marcel Oliver wrote:
That this discussion is happening NOW really surprises me. I have been following this list for a couple of years now, with the intention of eventually using numerical Python as the main teaching toolbox for numerical analysis, and possibly for the migration small research codes as well.
The possibility of doing numerics in Phython has always intrigued me. Right now I am primarily using Matlab. It's very powerful, but not free and the language is horrible; Octave is trying to play catch up but has mostly lost steam. So a good scientific Phython environment (of any sort) would be a really cool thing to have.
However, two things have always held me back (apart from coding small examples on a few occasions):
1. Numerical Phython has been in a limbo for too long (I had even assumed a few times that both Numeric and Numarray were dead for all practical purposes). If there are two incompatible version for
I don't know why you assumed that. Both have regularly been updated more than once in the past two years.
years and no clear indication where the whole thing is going, I am very hesitant to invest any time into writing substantial code, or recommend it for class room use.
That's your right of course. You have to remember that neither we (STScI) nor Enthought (who has funded virtually all the scipy work) are getting paid to do the work we are doing for the general community. In our case, we do much of it for our own purposes, and it would certainly be to our advantage if numarray were adopted by the general community so we invest resources in it. If you don't feel it is ready for your purposes, don't use numarray (or Numeric). We have only so many resources and while we wish we could do everything immediately, we can't. We are committed to making Python a good scientific environment, but we don't promise that it has everything that everyone would need now (and it certainly doesn't).
2. Plotting is a major issue. There are a couple of semi-functional packages, but neither a comprehensive solution nor a clear direction for the plotting architecture.
I agree completely. A later (tonight) message will discuss the current situation at more length. Perry Greenfield
participants (3)
-
Chris Barker
-
Marcel Oliver
-
Perry Greenfield