Mailman 3 June 2004 - NumPy-Discussion

Review your account 555418212
by OEMcd--Cheap--software--Inc. June 19, 2004

June 19, 2004

1 0

Re: [Numpy-discussion] Using dotblas
by Fernando Perez June 17, 2004

June 17, 2004

Robert Kern wrote: > Check the last few lines of Numeric.py . Argh. Could've saved myself the embarrasment :) In my meager defense, the README.html file for doblas then needs to be updated, since that's what I followed: # quote Finally, if you don't know whether dotblas will be available on all the machines where you whish to run your code, you can simply use a try-except statement like this: import Numeric try: import dotblas Numeric.dot = dotblas.dot if we can't find the fast dotblas, just use the normal dot except ImportError: pass # /quote Thanks, though. I'll clean up my codes f

1 0

Using dotblas
by Fernando Perez June 17, 2004

June 17, 2004

Hi all, I've built NumPy 23.1 with dotblas enabled, and as per the readme I'm starting to pepper my code with things like: # External packages import Numeric as N # Use BLAS-optimized versions when available try: import dotblas N.dot = dotblas._dotblas.matrixproduct N.multiarray.innerproduct = dotblas._dotblas.innerproduct except ImportError: print '*** dotblas not available, using regular Numeric functions' I wonder if this is really necessary, and if so, why. Is there any reason _not_ to use the dotblas versions when available? If not, shouldn't Numeric itself pick them up internally so that it always exposes the fastest possible version? I can always send in a patch to do this ( a trivial 5-liner like the above code), but I may just be missing something, as it seems too obvious not to be already the default. Regards, f.

1 0

RE: [Numpy-discussion] Subclassing NumArray: _PROTOTYPE flag issue
by Nadav Horesh June 16, 2004

June 16, 2004

OK it is my mistake, I did not implement the buffer argument in the constructor of my derived class. Still I have some minor problems, but nothing to bother about for the time being. It is very compelling to subclass NumArray, I wish that it will be documented when version 1.0 will be released. Nadav -----Original Message----- From: Perry Greenfield [mailto:perry@stsci.edu] Sent: Mon 14-Jun-04 21:27 To: Nadav Horesh; numpy Cc: Subject: RE: [Numpy-discussion] Subclassing NumArray: _PROTOTYPE flag issue Unfortunately Todd is away this week. I can try to help if you can illustrate what is being tried in the more complex script that you are referring to that is failing. Offhand, I didn't believe that _PROTOTYPE should ever be set to 0 unless it was done so explicitly for testing or debugging purposes (but perhaps I misremember). Thanks, Perry Nadav Horesh wrote: > For a simulation project I am working on I've subclasses ArrayType. I > was able to do much of my intentions until in one place when I tried to > make an array from a list of arrays I got an error message: > > . > . > . > File > "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line > 325, in array > return fromlist(sequence, type, shape) > File > "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line > 212, in fromlist > a = a.astype(type) > File > "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line > 630, in astype > retarr = self.__class__(buffer=None, shape=self._shape, type=type) > TypeError: __init__() got an unexpected keyword argument 'buffer' > > The analysis of the code showed that: > > 1. The NumArray class method definitions depends on the _PROTOTYPE flag > 2. The post-mortem debugging showed that when the error flagged, the > value of the variable _PROTOTYPE was 0 > > In a stand alone script there was no problem to do the list-> array > conversion: > > >>> import numarray as N > >>> import NumImage as NI # My module with the derived class > >>> a = N.arange(4) > >>> ia = NI.Cimage(N.arange(4)) # CImage is a derivative of NumImage > >>> a > array([0, 1, 2, 3]) > >>> ia > Cimage([0, 1, 2, 3]) > >>> N.array([a+i for i in range(3)]) > array([[0, 1, 2, 3], > [1, 2, 3, 4], > [2, 3, 4, 5]]) > >>> N.array([ia+i for i in range(3)]) # OK here, but failed as a part > of a complex script > Cimage([[0, 1, 2, 3], > [1, 2, 3, 4], > [2, 3, 4, 5]]) > > > My questions are: > > 1. Is this flag is in use? If I set it to 0 will I be able to derive > a class from the "C code"? > 2. Any intelligent solution? > > Nadav. > > > ------------------------------------------------------- > This SF.Net email is sponsored by the new InstallShield X. > From Windows to Linux, servers to mobile, InstallShield X is the > one installation-authoring solution that does it all. Learn more and > evaluate today! http://www.installshield.com/Dev2Dev/0504 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion >

1 0

numarray.sum should raise exception for wrong axis argument
by Sebastian Haase June 14, 2004

June 14, 2004

Hi, please take a look at this: >>> na.sum( na.zeros((2,6)) ) [0 0 0 0 0 0] >>> na.sum( na.zeros((2,6)) , 0) [0 0 0 0 0 0] >>> na.sum( na.zeros((2,6)) , 1) [0 0] >>> na.sum( na.zeros((2,6)) , 2) [0 0] >>> na.sum( na.zeros((2,6)) , 3) [0 0] >>> na.sum( na.zeros((2,6)) , 4) [0 0] >>> na.sum( na.zeros((2,6)) , -1) [0 0] >>> na.sum( na.zeros((2,6)) , -2) [0 0 0 0 0 0] >>> na.sum( na.zeros((2,6)) , -3) [0 0] >>> na.sum( na.zeros((2,6)) , -4) [0 0] >>> I think here should be a ValueError exception thrown rather than defaulting to the '-1'-axis. Comments ? Thanks, Sebastian Haase

1 0

Re: [Numpy-discussion] ObjectArray
by Todd Miller June 14, 2004

June 14, 2004

On Fri, 2004-06-11 at 10:53, Colin J. Williams wrote: > Todd, > > I'm trying out objects.py from the CVS as of June 11, with PROTOTYPE= > 1 > > Suggestions and comments based on that. > > 1. change the order in ObjectsArray so the __init__ becomes: > def __init__(self, objects=None, shape=None, > rank=None): > The thinking is that, in the absence of others, objects is the > argument most likely to be specified. I think this can go either way. For people that care, there are already array() and fromlist(). Subclasses can override the order. Am I missing something? > 2. add a check in __init__ to ensure that the shape is in fact > shape-like. Right now it raises a TypeError, and although it's ugly, it does at least trap the error and give a clue in the right direction. > 3. real doc strings would be helpful. Currently, the doc strings are > monopolized by the doctest stuff, > which was not the original intent of doc strings. True enough. Maybe for 1.1. > 4. I wonder about the need for both rank and shape. NumArray gets > along fine with just shape. This is a new feature, and is useful in the context of object arrays where a sequence object is an ambiguous thing: should the sequence be seen as containing elements of an object array, or itself an element of the object array. rank lets you specify the rank of the resulting object array explicitly. It is also computed implicitly if not specified based on the first object which doesn't match the type of the original sequence. This is new for 1.0. > 5. It is assumed that objects (in __init__) is a sequence, but no > check to make sure that it is. Actually, for 1.0 I don't even think it has to be a sequence anymore. It's possible to make a rank-0 object array. > 6. _byteorder is specified for NumArray, but not for ObjectArray, > this upsets a utility which > looks at array characteristics. _byteorder is a private attribute which doesn't make sense for ObjectArray. Add exception handling to the utility if it must be used with ObjectArrays. > 7. How is an ObjectArray better than a nested list of objects? It > seems to provide speedier access > to object pointers for larger n. It may offer the opportunity > to present data in a more organized > tabular manner. Dictionaries are better, in that they provide > access by means of an arbitrary key, > for the cost of hashing. > > NumArray appears to offer storage saving, in addition to the > above, but not here. > > Am I missing something? ObjectArrays can be indexed, manipulated, and printed like numarrays. ObjectArrays also work with object array ufuncs, so for instance, adding two ObjectArrays together implicitly applies the add builtin to each pair of elements. These ufuncs even support reductions and accumulations. This *is* a good question, but I still think ObjectArrays are useful in some contexts. Thanks for the feedback. Regards, Todd

2 1

Numarray problem on Opteron system, Suse 9.1
by Jason Ruiter June 11, 2004

June 11, 2004

Greetings, I'm running Suse 9.1 on a dual opteron system with 16GB of RAM. I'm using Python 2.3.3, Numeric 23.1 and numarray 0.9. I'm trying to allocate large (>4GB) arrays. Under Numeric, I get: Python 2.3.3 (#1, Apr 6 2004, 09:45:08) [GCC 3.3.3 (SuSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from Numeric import * >>> a=ones(2000000000,Int16); Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib64/python2.3/site-packages/Numeric/Numeric.py", line 578, in ones a=zeros(shape, typecode, savespace) MemoryError: can't allocate memory for array >>> Under numarray: Python 2.3.3 (#1, Apr 6 2004, 09:45:08) [GCC 3.3.3 (SuSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from numarray import * >>> a=ones(2000000000,Int16) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib64/python2.3/site-packages/numarray/numarraycore.py", line 1111, in ones retarr = _fillarray(_gen.product(shape), 1, 0, type) File "/usr/lib64/python2.3/site-packages/numarray/numarraycore.py", line 144, in _fillarray outarr = NumArray((size,), outtype) ValueError: new_memory: invalid region size: -294967296. >>> I've verified that python, Numeric, and numarray are built and linked against the 64bit libraries. I've also verified that it's possible to allocate a >16GB Array with the following program I found on one of the debian mailling lists. :q:q:q! #include <stdio.h> #include <stdlib.h> int main() { size_t n; void *p; double gb; for(gb=20;gb>.3;gb-=.5) { n= 1024L * 1024L * 1024L * gb; p = malloc( n ); printf("%12lu %4.1lfGb %p\n",n,n/1024./1024./1024.,p); free(p); } return 0; } 21474836480 20.0Gb (nil) 20937965568 19.5Gb (nil) 20401094656 19.0Gb (nil) 19864223744 18.5Gb (nil) 19327352832 18.0Gb (nil) 18790481920 17.5Gb (nil) 18253611008 17.0Gb (nil) 17716740096 16.5Gb 0x3995a02010 17179869184 16.0Gb 0x3995a02010 16642998272 15.5Gb 0x3995a02010 16106127360 15.0Gb 0x3995a02010 15569256448 14.5Gb 0x3995a02010 15032385536 14.0Gb 0x3995a02010 14495514624 13.5Gb 0x3995a02010 13958643712 13.0Gb 0x3995a02010 13421772800 12.5Gb 0x3995a02010 12884901888 12.0Gb 0x3995a02010 12348030976 11.5Gb 0x3995a02010 11811160064 11.0Gb 0x3995a02010 11274289152 10.5Gb 0x3995a02010 10737418240 10.0Gb 0x3995a02010 10200547328 9.5Gb 0x3995a02010 9663676416 9.0Gb 0x3995a02010 9126805504 8.5Gb 0x3995a02010 8589934592 8.0Gb 0x3995a02010 8053063680 7.5Gb 0x3995a02010 7516192768 7.0Gb 0x3995a02010 6979321856 6.5Gb 0x3995a02010 6442450944 6.0Gb 0x3995a02010 5905580032 5.5Gb 0x3995a02010 5368709120 5.0Gb 0x3995a02010 4831838208 4.5Gb 0x3995a02010 4294967296 4.0Gb 0x3995a02010 3758096384 3.5Gb 0x3995a02010 3221225472 3.0Gb 0x3995a02010 2684354560 2.5Gb 0x3995a02010 2147483648 2.0Gb 0x3995a02010 1610612736 1.5Gb 0x3995a02010 1073741824 1.0Gb 0x3995a02010 536870912 0.5Gb 0x3995a02010 Can someone point me in the right direction? Thanks Jason -- Jason Ruiter Research Scientist Environment and Emerging Technologies Division Altarum Institute Ann Arbor, Michigan, USA (v)734.302.4724 (f)734.302.4991 (m)248.345.4598 Jason.Ruiter(a)Altarum.org

2 1

ObjectArray
by Colin J. Williams June 10, 2004

June 10, 2004

ObjectArray, in objects.py, seems to offer a neat packaging capability. The examples focus on sting data, but it appears capable of handling all simple types and some container types. What is the intent? Dictionaries seem OK, tuples are OK when the shape is not specified, but an instance of ObjectArray fails. Colin W.

1 0

Re: [Numpy-discussion] RandomArray.random()
by Bruce Southey June 9, 2004

June 9, 2004

Hi, I would be prepared to at least look at it with numarray if you are prepared to share the scripts or rather the relevant part of them. On the surface, it would appear that this is a 32 vs 64 bit problem in defining the type precision. Regards Bruce ---- Original message ---- >Date: Thu, 3 Jun 2004 14:18:56 +0300 >From: Janne Sinkkonen <janne.sinkkonen(a)hut.fi> >Subject: [Numpy-discussion] RandomArray.random() >To: numpy-discussion(a)lists.sourceforge.net > >Related to a recent discussion on random numbers: > >People using RandomArray.random() on 64-bit architectures should also be aware >that sometimes the function returns exactly zero or one, which in turn causes >problems in many other routines which generate e.g. multinomial or dirichlet >variates. > >I'm not absolutely sure that the problem persists in the newest version of >Numeric, and I have not tested it on numarray. Anyway, I have seen it both in >Alphas and now later in an AMD Opteron machine - over the years. > >Unfortunately, as we do not have had time to dig any deeper around here, we >have just used wrapper with a test and a loop. > >-- >Janne > > > >------------------------------------------------------- >This SF.Net email is sponsored by the new InstallShield X. >From Windows to Linux, servers to mobile, InstallShield X is the one >installation-authoring solution that does it all. Learn more and >evaluate today! http://www.installshield.com/Dev2Dev/0504 >_______________________________________________ >Numpy-discussion mailing list >Numpy-discussion(a)lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

rank-0 arrays: Something useful
by Nadav Horesh June 8, 2004

June 8, 2004

One can use rank 0 arrays instead of scalars to automatically promote type of scalar-array operations: >>> from numarray import * >>> a = arange(10, type=Int16) >>> a * 10 array([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90], type=Int16) # same type >>> a * array(10, type=Int32) array([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90]) # Type promoted to Int32 Is there any way to economically set the type of the operation result? something like: multiply(a,b, type=xxx) # Doesn't work instead of: (a*b).astype(xxx) Nadav.

1 0