Python's various numeric packages
Is there some place where there is an overview of the various packages, their relationships and their present stages of development? I have downloaded SciPy: SciPy-0.2.0_alpha_202.4187.win32-py2.1.exe The install fails. It does not permit the install directory to be entered. It does not detect the registered (XP) Python installation to suggest site-packages as a likely install directory. Does SciPy depend on Numeric? Numeric appears to be fully operational, but no longer being developed as it is being replaced eventually by numarray. The intended time frame for this evolution is not clear. numarray appears to be largely operable, but speed problems have been reported. NDArray and NumArray are the basic classes of numarray, but the design decision seems to have been made to deprecate the use of class methods in favour of factory functions. This appears to inhibit the development of sub-classes to NumArray. MatPy is an effort to facilitate matrix operations. It seems to have resulted from a lot of work by Huaiyu Zhu. An extended discussion took place during 2000 to 2001, but there seems to be little current interest. The project was originally based on Numeric, but appears to be in transition to numarray. Again, the decision to use factory functions, rather than exploiting the full capability of the Python class structure, appears to hav been made. Source Forge advertises the availability of version 0.4 as a downloadable Win 32 package, but it seems to have regressed to version 0.3. I would appreciate any advice. Colin W.
On Saturday 12 July 2003 13:20, Colin J. Williams wrote:
Does SciPy depend on Numeric?
Yes. To quote from http://www.scipy.org/site_content/tutorials/build_instructions "Numeric 20 or higher - This is the core to Numeric computing in Python. SciPy relies heavily on it. SciPy expects the source version of this library to live in a directory named 'Numerical' within the scipy directory. " Cheers, J
Hi Colin, thanks for asking these questions. I also thing the situation isn't very clear. For what I know the latest version on numarray (0.5) is supposed to have fixed lot's of speed problems; that is: for small arrays, since for arrays larger than about 10000 it (reportedly) always was faster. My specific question is: WHY does SciPy claim (for a long time !) that it can work with both Numeric and with numarry. But instructions or even further comments on that are nowhere to be found ???? My impression is that currently Numeric is still supported if not further developed (as python's main numeric library) and the numarray is still being hold back - even though it shouldn't, ... <wink> [ some kind of Roadmap would be nice ] For the data-structures of numarray: NDArray was used (until) 0.3.6 but is nowhere to be found since 0.4. The reason is a political one: They what to make it as close to Numeric as possible to simplify (encourage) the transition [[ the doc says (said) that there is now one main structure, called PyArrayObject, and so it a"first class citizen" - just like it is in Numeric --- what is a "first class citizen" anyway?? ]] Hope this thread continues (and then get's copy&pasted to www.python.org/cgi-bin/moinmoin/NumArray -- once they have space again ;-) Cheers, Sebastian Haase
Is there some place where there is an overview of the various packages, their relationships and their present stages of development?
I have downloaded SciPy: SciPy-0.2.0_alpha_202.4187.win32-py2.1.exe The install fails. It does not permit the install directory to be entered. It does not detect the registered (XP) Python installation to suggest site-packages as a likely install directory.
Does SciPy depend on Numeric?
Numeric appears to be fully operational, but no longer being developed as it is being replaced eventually by numarray. The intended time frame for this evolution is not clear.
numarray appears to be largely operable, but speed problems have been reported. NDArray and NumArray are the basic classes of numarray, but the design decision seems to have been made to deprecate the use of class methods in favour of factory functions. This appears to inhibit the development of sub-classes to NumArray.
MatPy is an effort to facilitate matrix operations. It seems to have resulted from a lot of work by Huaiyu Zhu. An extended discussion took place during 2000 to 2001, but there seems to be little current interest.
The project was originally based on Numeric, but appears to be in transition to numarray. Again, the decision to use factory functions, rather than exploiting the full capability of the Python class structure, appears to hav been made.
Source Forge advertises the availability of version 0.4 as a downloadable Win 32 package, but it seems to have regressed to version 0.3.
I would appreciate any advice.
Colin W.
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.net http://www.scipy.net/mailman/listinfo/scipy-user
My specific question is: WHY does SciPy claim (for a long time !) that it can work with both Numeric and with numarry. But instructions or even further comments on that are nowhere to be found ????
I have had problems with Numeric... in particular, a bug that I found and reported, but seeing as Numeric is no longer being developed, I'm not holding my breath as to when it's going to be fixed. I had a look at fixing it myself, but the code is extremely hairy, and I'm totally lost in it. Which leads me to the following extension of your question: has anybody here had experience in using Numarray with SciPy? Andrew.
My specific question is: WHY does SciPy claim (for a long time !)
Hey Andrew, that it
can work with both Numeric and with numarry. But instructions or even further comments on that are nowhere to be found ????
I have had problems with Numeric... in particular, a bug that I found and reported, but seeing as Numeric is no longer being developed, I'm not holding my breath as to when it's going to be fixed.
Bugs are periodically fixed by Travis Oliphant, but he is pretty much the only one pitching in on this now.
I had a look at fixing it myself, but the code is extremely hairy, and I'm totally lost in it.
Hence, numarray. :-)
Which leads me to the following extension of your question: has anybody here had experience in using Numarray with SciPy?
Not I. It isn't ported yet. eric
Hi Colin, thanks for asking these questions. I also thing the situation isn't very clear.
For what I know the latest version on numarray (0.5) is supposed to have fixed lot's of speed problems; that is: for small arrays, since for arrays larger than about 10000 it (reportedly) always was faster.
It is much better now than it was even a few months ago -- Perry presented results at OSCON last week. It is still slower, but not cripplingly so as it was before.
My specific question is: WHY does SciPy claim (for a long time !) that it can work with both Numeric and with numarry. But instructions or even further comments on that are nowhere to be found ????
I wasn't aware of this claim, so I apologize if I made it. <wink...> Where is this information posted? Anyway, numarray support will come as the STSci people have time to work on the port. F2py support apparently exists now, which is a huge step. Perry mentioned that the special function library is one of the harder things remaining with all of its ufunc support. After that, it will just be working through the issues that come up. Generally though, the API, both Python and C level, for Numeric and Numarray are fairly compatible.
My impression is that currently Numeric is still supported if not
further
developed (as python's main numeric library) and the numarray is still being hold back - even though it shouldn't, ... <wink> [ some kind of Roadmap would be nice ]
The numarray folks are doing periodic releases and CVS is available to anyone wishing to tinker with it. eric
-----Original Message----- From: scipy-user-admin@scipy.net [mailto:scipy-user-admin@scipy.net]On Behalf Of eric jones Sent: Tuesday, July 15, 2003 2:30 AM To: scipy-user@scipy.net Subject: RE: [SciPy-user] Python's various numeric packages
My specific question is: WHY does SciPy claim (for a long time !) that it can work with both Numeric and with numarry. But instructions or even further comments on that are nowhere to be found ????
I wasn't aware of this claim, so I apologize if I made it. <wink...> Where is this information posted?
I wasn't aware of this claim either. There is a module in kiva that implies this (num_compat if I recall correctly), but I wasn't aware of any explicit claim to this effect.
Anyway, numarray support will come as the STSci people have time to work on the port. F2py support apparently exists now, which is a huge step. Perry mentioned that the special function library is one of the harder things remaining with all of its ufunc support. After that, it will just be working through the issues that come up. Generally though, the API, both Python and C level, for Numeric and Numarray are fairly compatible.
We will start working on this when we get to v0.8.
Hey Colin,
Is there some place where there is an overview of the various packages, their relationships and their present stages of development?
Numeric is the only required prerequisite. wxPython for plotting and a C++ compiler for weave are helpful but not required.
I have downloaded SciPy: SciPy-0.2.0_alpha_202.4187.win32-py2.1.exe The install fails. It does not permit the install directory to be
entered. > It does not detect the registered (XP) Python installation to suggest site-
packages as a likely install directory.
I am not sure why it doesn't work on your machine. I just installed the 2.2 version of the same file, and it found my Python2.2 installation and installed fine. Are others having problem with the 2.1 file? The easiest way to get a windows installation is to use the full python installation found here: http://www.enthought.com/python/
Does SciPy depend on Numeric?
Numeric appears to be fully operational, but no longer being developed as it is being replaced eventually by numarray. The intended time
Yes. I believe that 20.3 or beyond should work. Most people are probably using 22 or 23 currently. frame for
this evolution is not clear.
Numarray will be the successor to Numeric. I talked with Perry Greenfield from STSci at OSCON last week. STSci is developing numarray. He said that pretty much the only things left before numarray is a "feature complete" replacement for Numeric are PyObject array support and Paul Dubois Masked Array (MA) classes. He wants a full replacement before releasing 1.0. I'll let him pronounce the official release dates, but it is not so far off. Numarray and Numeric will co-exist for a long while. Numarray is new and therefore less well tested. It also has slower performance for small arrays by about a factor of 3, but is faster for large arrays. There are also many packages that rely on Numeric out in the wild, and the transition will be slow. At some point, SciPy will cut over to using Numarray. STSci folks are working on getting the major portions of the port to work. The process and time for doing this hasn't been decided. If it is easy to support both Numeric and numarray, we will. If it isn't, we'll have to think hard about when and how to do the transition.
numarray appears to be largely operable, but speed problems have been reported. NDArray and NumArray are the basic classes of numarray, but
the
design decision seems to have been made to deprecate the use of class methods in favour of factory functions. This appears to inhibit the development of sub-classes to NumArray.
Not sure what this is about because I haven't followed it that closely, but I will say that sub-classing arrays isn't all that it is cracked up to be in practice. One of the nicest things about arrays is there array arithmetic. They allow you to write mathematic formula-like code for numerical operations. When you sub-class an array, say with an Image class, all mathematic operations work, but they return arrays instead of images -- not so nice. You end up having to do a lot of work to get the behavior you want from the sub-classing. I'm not saying it isn't occasionally useful, it just isn't as nice as it first sounds...
MatPy is an effort to facilitate matrix operations. It seems to have resulted from a lot of work by Huaiyu Zhu. An extended discussion took place during 2000 to 2001, but there seems to be little current interest.
Don't know much about this.
The project was originally based on Numeric, but appears to be in transition to numarray. Again, the decision to use factory functions, rather than exploiting the full capability of the Python class structure, appears to hav been made.
Source Forge advertises the availability of version 0.4 as a downloadable Win 32 package, but it seems to have regressed to version 0.3.
I would appreciate any advice.
Colin W.
_______________________________________________ SciPy-user mailing list SciPy-user@scipy.net http://www.scipy.net/mailman/listinfo/scipy-user
-----Original Message----- From: scipy-user-admin@scipy.net [mailto:scipy-user-admin@scipy.net]On Behalf Of eric jones Sent: Tuesday, July 15, 2003 2:19 AM To: scipy-user@scipy.net Subject: RE: [SciPy-user] Python's various numeric packages
Numeric appears to be fully operational, but no longer being developed as it is being replaced eventually by numarray. The intended time frame for this evolution is not clear.
Numarray will be the successor to Numeric. I talked with Perry Greenfield from STSci at OSCON last week. STSci is developing numarray. He said that pretty much the only things left before numarray is a "feature complete" replacement for Numeric are PyObject array support and Paul Dubois Masked Array (MA) classes. He wants a full replacement before releasing 1.0. I'll let him pronounce the official release dates, but it is not so far off.
No dates fixed in stone, but here is the basic plan 0.6 to be released this week, this reorganizes numarray as a package 0.7 (few weeks?) Will include PyObject support 0.8 (2 months?) Will include MA support. At this point we hope to replicate all Numeric functionality (other than the planned changes). v1.0 will come out when 0.8 has been tested by users for a little while (with perhaps a 0.9 if many changes and fixes are needed)
Numarray and Numeric will co-exist for a long while. Numarray is new and therefore less well tested. It also has slower performance for small arrays by about a factor of 3, but is faster for large arrays. There are also many packages that rely on Numeric out in the wild, and the transition will be slow.
There may be some rough corners where performance of numarray needs optimization that haven't yet been handled (e.g. printing? slicing?)
At some point, SciPy will cut over to using Numarray. STSci folks are working on getting the major portions of the port to work. The process and time for doing this hasn't been decided. If it is easy to support both Numeric and numarray, we will. If it isn't, we'll have to think hard about when and how to do the transition.
yep Perry Greenfield
I have a perfectly functional system on WinXP, and a serviceable one on Linux. So of course my new employer is Mac centric. I have no religious attachement to anything, but I am all thumbs on the Mac. Never touched one. it's OS X, 10.1 I have the fink version of python running, and the fink version of Numeric. and X11. Idle runs. Numeric works. How the heck do I get scipy up? I started with the fink version because I want to use the visual package, which claims to work only with the fink python. There's a fink package called scipy-2002xxxxx in the unstable area. As an example of my expertise with Mac, I can't figure out how to copy the .info file from the "unstable" directory to the "local" directory (as directed by some instructions somewhere). Any help? Once I get this nailed down, I'll start looking for a mouse with three buttons ... -gary
participants (7)
-
Andrew Nesbit -
Colin J. Williams -
eric jones -
Gary Pajer -
J Analytis -
Perry Greenfield -
Sebastian Haase