To all who were waiting: I've finished adding the methods to the array object so that Numeric3 in CVS now compiles (at least for me on Linux). I will be away for at least a day so it is a good time to play... -Travis
I've downloaded the latest Numeric3 and tried to compile it. I found one error in setup.py that I put in myself a couple of days ago; I fixed this in CVS. On Cygwin, the compilation and installation runs without problems, other than the warning messages. However, when I try to compile Numeric3 for Windows, an error windows pops up with the following message: 16 bit MS-DOS Subsystem ~/Numeric3 The NTVDM CPU has encountered an illegal instruction. CS:071d IP:210f OP:63 69 66 69 65 Choose 'Close' to terminate the application It seems that this message is due to the section labeled "#Generate code" in setup.py, where python is being run with a call to os.system. What does this do? Is there a need to generate code automatically in setup.py, rather than include the generated code with the Numeric3 source code? When using Numeric3, I found that the zeros function now returns a float array by default, whereas Numeric returns an integer array: $ python2.4 Python 2.4 (#1, Dec 5 2004, 20:47:03) [GCC 3.3.3] on cygwin Type "help", "copyright", "credits" or "license" for more information.
from ndarray import * zeros(5) array([0.0, 0.0, 0.0, 0.0, 0.0], 'd') array([1,2,3]) array([1, 2, 3], 'l')
mdehoon@ginseng ~ $ python2.4 Python 2.4 (#1, Dec 5 2004, 20:47:03) [GCC 3.3.3] on cygwin Type "help", "copyright", "credits" or "license" for more information.
from Numeric import * zeros(5) array([0, 0, 0, 0, 0]) array([1,2,3]) array([1, 2, 3]) array([1,2,3]).typecode() 'l'
Is there a reason to change the default behavior of zeros? Existing code may assume that zeros returns an integer array, and may behave incorrectly with Numeric3. Such bugs would be very hard to find. Finally, I tried to compile an extension module with Numeric3. The warning messages concerning the type of PyArrayObject->dimensions no longer occur, now that intp is typedef'd as an int instead of a long. But I agree with David Cooke that using Py_intptr_t in pyport.h would be better:
Why not use Py_intptr_t? It's defined by the Python C API already (in pyport.h).
When compiling the extension module, I get warning messages about PyArray_Cast, which now takes a PyObject* instead of a PyArrayObject* as in Numerical Python. Is this a typo in Src/arrayobject.c? Also, the PyArray_Cast function has the comment /* For backward compatibility */ in Src/arrayobject.c. Why is that? Linking the extension module fails due to the undefined reference to _PyArray_API. --Michiel. Travis Oliphant wrote:
To all who were waiting:
I've finished adding the methods to the array object so that Numeric3 in CVS now compiles (at least for me on Linux).
I will be away for at least a day so it is a good time to play... -Travis
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon
Hi Travis, On Fri, 25 Mar 2005, Travis Oliphant wrote:
To all who were waiting:
I've finished adding the methods to the array object so that Numeric3 in CVS now compiles (at least for me on Linux).
Compilation is fine for me as well (linux). I played around a bit - Obviously, addition of arrays, multiplication etc. don't work yet (as expected, if I remember your mails correctly). One thing which confused me is the following In [1]:from ndarray import * In [2]:x=arange(10.0) In [3]:scalar=x[3] In [4]:print scalar+1 --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) TypeError: unsupported operand type(s) for +: 'double_arrtype' and 'int' In [5]:print type(scalar) <type 'double_arrtype'> OK, this has been discussed up-and-down on this mailing list. At the moment I don't know how this will affect my Numeric(3) life, so I will wait until the other operations are implemented and see if there are any consequences for my programs at all ... ;-) A couple of things seem to be a bit unusual, e.g.: In [9]:x=arange(10.0) In [10]:x Out[10]:array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0], 'd') In [11]:x.argmax() --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) /home/scratch/abaecker/INSTALL_SOFT/TEST_NUMERIC3/<console> TypeError: function takes exactly 1 argument (0 given) In [12]:x.argmax(None) Out[12]:9 In [13]:t=x.argmax(None) In [14]:type(t) Out[14]:<type 'int_arrtype'> So argmax also returns an array type, but I would have really thought that this is an integer index?! Also a couple of attributes (E.g. x.sum() are not yet implemented) or lack documention (I know this comes last ;-). Best, Arnd
I'm a bit confused about where Numeric3 is heading. Originally, the idea was that Numeric3 should go in Python core. Are we still aiming for that? More recently, another goal was to integrate Numeric and numarray, which I fully support. However, from looking at the Numeric3 source code, and from looking at the errors found by Arnd, it looks like that Numeric3 is a complete rewrite of Numeric. This goes well beyond integrating Numeric and numarray. Now I realize that sometimes it is necessary to rethink and rewrite some code. However, Numerical Python has served us very well over the years, and I'm worried that rewriting the whole thing will break more than it fixes. So where is Numeric3 going? --Michiel. Arnd Baecker wrote:
Hi Travis,
On Fri, 25 Mar 2005, Travis Oliphant wrote:
To all who were waiting:
I've finished adding the methods to the array object so that Numeric3 in CVS now compiles (at least for me on Linux).
Compilation is fine for me as well (linux). I played around a bit - Obviously, addition of arrays, multiplication etc. don't work yet (as expected, if I remember your mails correctly). One thing which confused me is the following
In [1]:from ndarray import * In [2]:x=arange(10.0) In [3]:scalar=x[3] In [4]:print scalar+1 --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) TypeError: unsupported operand type(s) for +: 'double_arrtype' and 'int' In [5]:print type(scalar) <type 'double_arrtype'>
OK, this has been discussed up-and-down on this mailing list. At the moment I don't know how this will affect my Numeric(3) life, so I will wait until the other operations are implemented and see if there are any consequences for my programs at all ... ;-)
A couple of things seem to be a bit unusual, e.g.:
In [9]:x=arange(10.0) In [10]:x Out[10]:array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0], 'd') In [11]:x.argmax() --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last)
/home/scratch/abaecker/INSTALL_SOFT/TEST_NUMERIC3/<console>
TypeError: function takes exactly 1 argument (0 given) In [12]:x.argmax(None) Out[12]:9 In [13]:t=x.argmax(None) In [14]:type(t) Out[14]:<type 'int_arrtype'>
So argmax also returns an array type, but I would have really thought that this is an integer index?!
Also a couple of attributes (E.g. x.sum() are not yet implemented) or lack documention (I know this comes last ;-).
Best,
Arnd
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon
Michiel Jan Laurens de Hoon wrote:
I'm a bit confused about where Numeric3 is heading. Originally, the idea was that Numeric3 should go in Python core. Are we still aiming for that? More recently, another goal was to integrate Numeric and numarray, which I fully support.
I would prefer to re-integrate the numarray people "back" into the Numeric community, by adding the features to Numeric that they need.
However, from looking at the Numeric3 source code, and from looking at the errors found by Arnd
These errors are all due to the fact that umath functions are not available yet. Please don't judge too hastily. At this point, I'm really just looking for people who are going to pitch in and help with coding or to offer suggestions about where the code base should go. I have done this completely in the open as best I can. I have no other "hidden" plans. All of the scalar types will get their math from the umath functions too. So, of course they don't work either yet (except for those few that inherit from the basic Python types).
it looks like that Numeric3 is a complete rewrite of Numeric.
I really don't understand how you can make this claim. All I've done is add significantly to Numeric's code base and re-inserted some code-generators (code-generators are critical for maintainability --- in answer to your previous question).
This goes well beyond integrating Numeric and numarray. Now I realize that sometimes it is necessary to rethink and rewrite some code. However, Numerical Python has served us very well over the years, and I'm worried that rewriting the whole thing will break more than it fixes. So where is Numeric3 going?
You really need to justify this idea you are creating that I am "re-writing the whole thing" I disagree wholeheartedly with your assessment. Certain changes were made to accomodate numarray features, new types were added, indexing was enhanced. When possible, I've deferred to Numeric's behavior. But, you can't bring two groups back to a common array type by not changing the array type at all. Numeric3 is going wherever the community takes it. It is completely open. Going into the Python core is going to have to wait until things settle down in our own community. It's not totally abandoned just put on hold (again). That was the decision thought best by Guido, Perry, myself, and Paul at our lunch together. We thought it best to instead suggest interoperability strategies. Numeric3 is NOT a re-write of Numeric. I've re-used a great deal of the Numeric code. The infrastructure is exactly the same. I started the project from the Numeric code base. I've just added a great deal more following the discussions on this list. At worst, I've just expanded quite a few things and moved a lot into C. Little incompatibilties are just a sign of alpha code not a "direction." Some have thought that zeros returning ints was a bug in the past which is why the change occured. But, it is an easy change, and I tend to agree that it should stay returning the Numeric default of Intp.
In [1]:from ndarray import * In [2]:x=arange(10.0) In [3]:scalar=x[3] In [4]:print scalar+1 ---------------------------------------------------------------------------
exceptions.TypeError Traceback (most recent call last) TypeError: unsupported operand type(s) for +: 'double_arrtype' and 'int' In [5]:print type(scalar) <type 'double_arrtype'>
All scalar types by default will do math operations as rank-0 arrays (which haven't been brought back in yet). That is why the error. I believe that right now I am inheriting first from the Generic Scalar Type instead of the double type (so it will use Scalar Arithmetic first). Later, special methods for each scalar type could be used (for optimization).
A couple of things seem to be a bit unusual, e.g.:
In [9]:x=arange(10.0) In [10]:x Out[10]:array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0], 'd') In [11]:x.argmax() ---------------------------------------------------------------------------
exceptions.TypeError Traceback (most recent call last)
This is an error. I'll look into it. Notice all method calls that take an axis argument are defaulting to None (which means ravel the whole array). The function calls won't change for backward compatibility.
/home/scratch/abaecker/INSTALL_SOFT/TEST_NUMERIC3/<console>
TypeError: function takes exactly 1 argument (0 given) In [12]:x.argmax(None) Out[12]:9 In [13]:t=x.argmax(None) In [14]:type(t) Out[14]:<type 'int_arrtype'>
So argmax also returns an array type, but I would have really thought that this is an integer index?!
Remember, array operations always return array scalars!
Also a couple of attributes (E.g. x.sum() are not yet implemented) or lack documention (I know this comes last ;-).
Hmm. x.sum should be there. Believe me, it was a bit of a pain to convert all the Python code to C. Have to check this. -Travis
In [1]:from ndarray import * In [2]:x=arange(10.0) In [3]:scalar=x[3] In [4]:print scalar+1 ---------------------------------------------------------------------------
exceptions.TypeError Traceback (most recent call last) TypeError: unsupported operand type(s) for +: 'double_arrtype' and 'int' In [5]:print type(scalar) <type 'double_arrtype'>
All scalar types by default will do math operations as rank-0 arrays (which haven't been brought back in yet). That is why the error. I believe that right now I am inheriting first from the Generic Scalar Type instead of the double type (so it will use Scalar Arithmetic first). Later,
special methods for each scalar type could be used (for optimization).
To clarify, it is the umath operations that have not been 'brought back in', or activated, yet. The rank-0 arrays are of course there as they have always been. -Travis
Travis Oliphant wrote:
Michiel Jan Laurens de Hoon wrote:
I'm a bit confused about where Numeric3 is heading. Originally, the idea was that Numeric3 should go in Python core. Are we still aiming for that? More recently, another goal was to integrate Numeric and numarray, which I fully support.
I would prefer to re-integrate the numarray people "back" into the Numeric community, by adding the features to Numeric that they need.
However, from looking at the Numeric3 source code, and from looking at the errors found by Arnd
These errors are all due to the fact that umath functions are not available yet. Please don't judge too hastily. At this point, I'm
OK. Fair enough. Maybe I did judge too hastily. From your comments, it looks like we agree on where Numeric3 is going.
It seems that this message is due to the section labeled "#Generate code" in setup.py, where python is being run with a call to os.system. What does this do? Is there a need to generate code automatically in setup.py, rather than include the generated code with the Numeric3 source code?
(code-generators are critical for maintainability --- in answer to your previous question).
As far as I can tell from their setup.py, neither Numerical Python nor numarray currently does code generation on the fly from setup.py. (This was one of the reasons that I started to worry if Numeric3 is more than Numerical Python + numarray). --Michiel. -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon
On Mar 27, 2005, at 3:47 AM, Michiel Jan Laurens de Hoon wrote:
As far as I can tell from their setup.py, neither Numerical Python nor numarray currently does code generation on the fly from setup.py. (This was one of the reasons that I started to worry if Numeric3 is more than Numerical Python + numarray).
Numarray definitely does code generation (and so did Numeric originally, eventually the generated code was hand-edited). Code generation is the way to go (with C anyway). Perry
I have made one change to setup.py and Include/ndarray/arrayobject.h to make the definition of intp, uintp consistent with what is in pyport.h (which gets #included via Python.h). This shouldn't change anything, but since this affects the compilation almost everywhere, I thought I should let everybody know. If it causes any problems, feel free to change it back. --Michiel. Travis Oliphant wrote:
To all who were waiting:
I've finished adding the methods to the array object so that Numeric3 in CVS now compiles (at least for me on Linux).
I will be away for at least a day so it is a good time to play... -Travis
------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon
participants (4)
-
Arnd Baecker -
Michiel Jan Laurens de Hoon -
Perry Greenfield -
Travis Oliphant