[Numpy-discussion] Numeric 24.0
oliphant at ee.byu.edu
Tue Apr 5 22:26:38 EDT 2005
Michiel Jan Laurens de Hoon wrote:
> Travis Oliphant wrote:
>> I'd like to release a Numeric 24.0 to get the array interface out
>> there. There are also some other bug fixes in Numeric 24.0
> About the array interface, my feeling is that while it may be helpful
> in the short run, it is likely to damage SciPy in the long run.
Well, I guess we'll just have to see. Again, I see the array interface
as important for talking to other modules that may not need or want the
"full power" of a packed array module like scipy.base is.
> The array interface allows different array implementations to move in
> different directions. These different implementations will be
> compatible with respect to the array interface, but incompatible
> otherwise (depending on the level of self-restraint of the developers
> of the different array implementations). So in the end, extension
> modules will be written for a specific array implementation anyway. At
> this point, Numerical Python is the most established and has most
> users. Numarray, as far as I can tell, keeps closer to the Numerical
> Python tradition, so maybe extension modules can work with either one
> without further modification (e.g., pygist seems to work with both
> Numerical Python and numarray).
> But SciPy has been moving away (e.g. by replacing functions by methods).
Michiel, you seem to want to create this impression that "SciPy" is
"moving away." I'm not sure of your motivations. But, since this is a
public forum, I have to restate emphatically, that "SciPy" is not
"moving away from Numeric." It is all about bringing together the
communities. For the 5 years that scipy has been in development, it has
always been about establishing a library of common routines that we
could all share. It has built on Numeric from the beginning. Now,
there is another "library" of routines that is developing around
numarray. It is this very real break that I'm trying to help fix. I
have no other "desire" to "move away" or "create a break" or any other
such notions that you seem to want to spread.
That is precisely why I have publically discussed practically every step
of my work. You seem to be the only vocal one who thinks that
scipy.base is not just a replacement for Numeric, but something else
entirely. So, I repeat: **scipy.base is just a new version of
Numeric with a few minor compatibility issues and a lot of added
functionality and features**
For example, despite your claims, I have not "replaced" functions by
methods. The functions are still all there just like before. I've
simply noticed that numarray has a lot of methods and so I've added
similar methods to the Numeric object to help numarray users make the
Everything else that I've changed, I've done to bring Numeric up-to-date
with modern Python versions, and to fix old warts that have sat around
for years. If there are problems with my changes, speak up. Tell me
what to do to make the new Numeric better.
> As extension module writers are usually busy people, they may not be
> willing to modify their code so that it works with SciPy, and even
> less to maintain two versions of their code, one for Numerical
> Python/numarray and one for SciPy.
It's comments like this that make me wonder what you are thinking. It
seems to me that you are the only one I've talked to that wants to
maintain the notion of a "split". Everybody else, I'm in contact with
is in full support of merging the two communities behind a single
scientific array object.
Every extension module that compiles for Numeric should compile for
scipy.base. Notice that full scipy already has a huge number of
extension modules that needs to compile for scipy.base. So, I have
every motivation to make that a painless process.
> Users who could previously choose to install SciPy as an addition to
> Numerical Python, now find that they have to choose between SciPy and
> Numerical Python. As Numerical Python has many more extension
> packages, I expect that SciPy will end up losing users.
Again, scipy.base should *replace* Numerical Python for all users
(except the most adamant who don't seem to want to go with the rest of
the community). scipy.base is a new version of Numeric. On the
C-level I don't know of any incompatibilities, on the Python level
there are a very few (most of them rarely-used typecode character issues
which a simple search and replace will fix).
I should emphasize this next point, since I don't seem to be coming
across very clearly to some people. As head Numeric developer, I'm
stating that **Numeric 24 is the last release that will be called
Numeric**. New releases of Numeric will be called scipy.base.
Of course, I realize that people can do whatever they want with the old
Numeric code base, but then they will be the ones responsible for
continuing a "split," because the Numerical Python project at
sourceforge will point people to install scipy.base.
Help me make the transition as painless as possible, that's all I'm
asking. People transitioning from Numeric should have no trouble at
all as I repeatedly point out. People transitioning from numarray will
have a *little* harder time which is why the array interface should help
out during that process. It is helping people transition back from
numarray that is 90% of the reason I've made any changes to the
internals of Numeric.
I've been a happy and quiet Numeric user and developer for years, but I
respect the problems that Perry, Rick, Paul, and Todd have pointed out
with their numarray implementation, and I saw a way to support their
needs inside of Numeric. That is the whole reason for my efforts. I
wish people would stop trying to make it seem to casual readers of this
forum that I'm trying to create a "whole new" incompatible system.
Help me fix the obviously unnecessary incompatibilites where they may
exist, and help me make automatic transistion scripts to help people
upgrade painlessly to the newer Numeric.
I very much appreciate all who voice your concerns. Michiel, you are
particularly appreciated because you are voice from a solid Numeric
user. I just think that such concerns would be more productive in the
context of accepting the fact that an upgrade from Numeric to scipy.base
is going to happen, rather than trying to make it look like some new
"split" is occurring. I've received a lot of offline support for the
Numeric/numarray unification effort that scipy.base is. It would help
if more people could provide public support on this forum so that
others can see that I'm not just some outsider pushing some random
ideas, but I am simply someone who decided to sacrifice some time for
what I think is a very important effort. It would also help if other
people who have concerns would voice them (I'm very grateful for those
who have expressed their concerns) so that we can all address them and
get on the same page for future development.
Right now, the CVS version of Numeric3 works reasonably. It compiles
and uses the old ufunc objects (which have only been extended to support
the new types). I could use a lot of help in finding bugs. You can
also try out the new array scalars to see how they work (math works on
them now) and also see what may still be missing in their implementation.
> Personally I use Numerical Python, and I plan to continue to use it
> for years to come, so it doesn't matter much to me. I'm just warning
> that the array interface may be a Trojan horse for the SciPy project.
As long as you realize that as far as I know the other developers of
Numerical Python are going to be moving to scipy.base, and so you will
be using obsolete technology, you are free to do as you wish. But, I
really hope we can persuade you to join us. It is much better if we
More information about the NumPy-Discussion