Mailman 3 May 2007 - NumPy-Discussion

subclassing ndarray and recarray
by Bernhard Voigt 09 May '07

09 May '07

I'm trying to subclass ndarray or recarray to build a record array that has a dictionary with a mapping of keys to array indexes and vice versa. I come across the problem, that depending on the field access method I get different types back: # a is a sublcass of record array >> print type(a) <class 'myrecarray'> >> print type(a['x']) <class 'myrecarray'> >> print type(a.x) <type 'numpy.ndarray'> I guess this has something to do with the call to the view method in the recarray implementation but I can't figure out how to solve this. Hence I switched back to subclass the ndarray and implemented the field access via a.x by myself. In order to update the mapping of the array instance when slicing is used, I would like to get the selected indexes in the __array_finalize__ method. Is there a way to access the selected indexes, or do I have to overwrite the __getitem__ method to get indexes from the slice object. This could be rather complicated due to the many different slice specifications. Here's an incomplete code example from my subclass, maybe this helps to understand the problem: def __array_finalize__(self, obj): if not hasattr(self, 'mapping'): # create a mapping for the selected indexes from the mapping of the original array given by obj # reverse_mapping maps indexes to keys, __selected_indexes is the field I would like to have self.reverse_mapping = dict((obj.reverse_mapping[i], i for i n self.__selected_indexes) self.mapping = dict((k,i) for i,k in self.reverse_mapping.iteritems()) def __getitem__(self, key): # fill the __selected_indexes field self.__selected_indexes = foo(key) # pass the selection to ndarray - calls __array_finalilze__ and where the mapping is performed return numpy.ndarray.__getitem__(self, key) Thanks for help! Bernhard

2 1

Memory Error in Search Sorted
by Tom Denniston 09 May '07

09 May '07

So searchsorted is supposed to take a list. The arguments I am using are not correct. But it still seems strange to me that these incorrect params trigger a memory error. Is a check simply missing or is this possibly a sign of a larger bug? I can simply correct my params so this is no big deal, but I just wanted to report it as it seemed like a subtle bug. --Tom In [2]: numpy.searchsorted(numpy.array(['1', '2', '3', '4']), [1]) Out[2]: array([0]) In [3]: numpy.searchsorted(numpy.array(['1', '2', '3', '4']), 1) --------------------------------------------------------------------------- <type 'exceptions.MemoryError'> Traceback (most recent call last) /code/src/<ipython console> in <module>() /lib/python2.5/site-packages/numpy/core/fromnumeric.py in searchsorted(a, v, side) 378 except AttributeError: 379 return _wrapit(a, 'searchsorted', v, side) --> 380 return searchsorted(v, side) 381 382 def resize(a, new_shape): <type 'exceptions.MemoryError'>:

2 1

numpy.distutils
by Mike Beachy 09 May '07

09 May '07

Hi all - I've been trying to set up configuration files to standardize a local 1.02numpy installation and have run into a problem with the intel compiler package. If I try python setup.py config_fc --fcompiler=intel build_ext (from the top level numpy-1.0.2 directory) I get the following failures (with ifort 8.1): ifort: Command line warning: extension 'M' not supported ignored in option '-x' ifort: Command line error: Unrecognized keyword 'SSE2' for option '-arch' ifort: Command line warning: extension 'M' not supported ignored in option '-x' ifort: Command line error: Unrecognized keyword 'SSE2' for option '-arch' I assumed that the 'noarch' option to the config_fc command would get me around this, but it does not. This seems to be because the get_flags_linker_so method in the numpy.distutils.fcompiler.intel.IntelFCompiler class always extends the options with the result of self.get_flags_arch(). Removing that call fixes the problem. I'm asking about this here rather than simply filing a bug because I wonder if there's a way of overriding the get_flags_linker_so method that I've simply overlooked. I'm not sure that I've discovered all the ways to modify a numpy installation yet. (So far I've found site.cfg, setup.cfg, and Setup. Am I missing anything? Also, is there a recommended documentation source for this stuff?) Thanks, Mike

1 0

Type conversion weirdness in numpy-1.0.2.win32-py2.4 binary
by Martin Spacek 09 May '07

09 May '07

In linux and win32 (numpy 1.0.1 release compiled from source, and 1.0.3dev3726 respectively), I get the following normal behaviour: >>> import numpy as np >>> np.array([1.0, 2.0, 3.0, 4.0]) array([ 1., 2., 3., 4.]) >>> np.int32(np.array([1.0, 2.0, 3.0, 4.0])) array([ 1, 2, 3, 4]) But on three separate Windows machines running the binary numpy-1.0.2.win32-py2.4 install, I get this: >>> import numpy as np >>> np.array([1.0, 2.0, 3.0, 4.0]) array([ 1., 2., 3., 4.]) >>> np.int32(np.array([1.0, 2.0, 3.0, 4.0])) 31195176 >>> np.int32(np.array([1.0, 2.0, 3.0, 4.0])) 30137880 >>> np.int32(np.array([1.0, 2.0, 3.0, 4.0])) 31186080 >>> np.int32(np.array([1.0, 2.0, 3.0, 4.0])) 31186080 >>> np.float64(np.array([1.0, 2.0, 3.0, 4.0])) 2.1359481850412033e-314 >>> np.int32(np.array([1, 2, 3, 4])) 28729424 One of them was running the numpy-1.0.1 binary, which showed normal behaviour, until I upgraded it to 1.0.2. Test come out fine: >>> np.test() Found 5 tests for numpy.distutils.misc_util Found 31 tests for numpy.core.numerictypes Found 32 tests for numpy.linalg Found 4 tests for numpy.lib.index_tricks Found 4 tests for numpy.core.scalarmath Found 9 tests for numpy.lib.arraysetops Found 42 tests for numpy.lib.type_check Found 198 tests for numpy.core.multiarray Found 3 tests for numpy.lib.getlimits Found 36 tests for numpy.core.ma Found 2 tests for numpy.lib.polynomial Found 1 tests for numpy.fft.fftpack Found 13 tests for numpy.lib.twodim_base Found 10 tests for numpy.core.defmatrix Found 13 tests for numpy.core.umath Found 1 tests for numpy.lib.ufunclike Found 4 tests for numpy.ctypeslib Found 43 tests for numpy.lib.function_base Found 9 tests for numpy.core.records Found 59 tests for numpy.core.numeric Found 3 tests for numpy.fft.helper Found 48 tests for numpy.lib.shape_base Found 0 tests for __main__ ............................................................................................................... ............................................................................................................... ............................................................................................................... ............................................................................................................... ............................................................................................................... ............... ---------------------------------------------------------------------- Ran 570 tests in 1.125s OK <unittest.TextTestRunner object at 0x0114B810> >>> numpy.version.version '1.0.2' I'm running python 2.4.4 in all cases (except 2.4.3 on linux). Is this a problem with the numpy-1.0.2.win32-py2.4 binary? I could try building a new 1.0.2 binary, if that would help. Martin

2 2

matlab vs. python question
by Neal Becker 08 May '07

08 May '07

I'm interested in this comparison (not in starting yet another flame fest). I actually know nothing about matlab, but almost all my peers use it. One of the things I recall reading on this subject is that matlab doesn't support OO style programming. I happened to look on the matlab vendor's website, and found that it does have classes. OTOH, I've seen at least some matlab code, and never saw anyone use these features. I'm perfectly happy with python myself, but I wonder if anyone has good arguments for why to prefer python over matlab?

19 31

building pynetcdf-0.7 on OS X 10.4 intel
by James Boyle 08 May '07

08 May '07

OS X 10.4.9, python 2.5, numpy 1.0.3.dev3726, netcdf 3.6.2 I am trying to build pynetcdf-0.7, so as to be able to read netCDF files from python 2.5/numpy . So far I have not had success. Enclosed is the dump of my most recent failure - the complaint comes from the linker since the linker has both a powerPC (ppc) and an intel (i386) flag. I guess that what is being attempted is a universal build ( both ppc and i386) but failing. Apparently my netcdf build was just i386 - although I did nothing special. I am at a loss as to what to do next - this might be a question for an Mac specialist - but I figured I'd try here first. --Jim [krait:~/Desktop/pynetcdf-0.7] boyle5% python setup.py build running build running config_fc running build_src building extension "pynetcdf._netcdf" sources building data_files sources running build_py creating build creating build/lib.macosx-10.3-fat-2.5 creating build/lib.macosx-10.3-fat-2.5/pynetcdf copying ./__init__.py -> build/lib.macosx-10.3-fat-2.5/pynetcdf copying ./NetCDF.py -> build/lib.macosx-10.3-fat-2.5/pynetcdf copying ./netcdf_demo.py -> build/lib.macosx-10.3-fat-2.5/pynetcdf running build_ext customize UnixCCompiler customize UnixCCompiler using build_ext building 'pynetcdf._netcdf' extension compiling C sources C compiler: gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/ MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fPIC -fno-common -dynamic -DNDEBUG -g -O3 -Wall - Wstrict-prototypes creating build/temp.macosx-10.3-fat-2.5 compile options: '-I/Users/boyle5/netcdf-3.6.2/include -I/Library/ Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/ numpy/core/include -I/Library/Frameworks/Python.framework/Versions/ 2.5/include/python2.5 -c' gcc: ./_netcdf.c ./_netcdf.c: In function 'PyNetCDFFile_Close':./_netcdf.c: In function 'PyNetCDFFile_Close': ./_netcdf.c:965: warning: passing argument 2 of 'PyDict_Next' from incompatible pointer type./_netcdf.c:965: warning: passing argument 2 of 'PyDict_Next' from incompatible pointer type ./_netcdf.c: In function 'PyNetCDFVariableObject_subscript': ./_netcdf.c:1822: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1822: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1822: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1842: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1842: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1842: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c: In function 'PyNetCDFVariableObject_subscript': ./_netcdf.c:1822: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1822: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1822: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1842: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1842: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1842: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c: In function 'PyNetCDFVariableObject_ass_subscript': ./_netcdf.c:1944: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1944: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1944: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1964: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1964: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1964: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c: In function 'PyNetCDFVariableObject_ass_subscript': ./_netcdf.c:1944: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1944: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1944: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1964: warning: passing argument 3 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1964: warning: passing argument 4 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c:1964: warning: passing argument 5 of 'PySlice_GetIndices' from incompatible pointer type ./_netcdf.c: At top level: ./_netcdf.c:2009: warning: initialization from incompatible pointer type ./_netcdf.c:2011: warning: 'intargfunc' is deprecated ./_netcdf.c:2011: warning: initialization from incompatible pointer type ./_netcdf.c:2012: warning: 'intargfunc' is deprecated ./_netcdf.c:2012: warning: initialization from incompatible pointer type ./_netcdf.c:2013: warning: 'intintargfunc' is deprecated ./_netcdf.c:2013: warning: initialization from incompatible pointer type ./_netcdf.c:2014: warning: initialization from incompatible pointer type ./_netcdf.c:2015: warning: initialization from incompatible pointer type ./_netcdf.c:2019: warning: initialization from incompatible pointer type ./_netcdf.c: At top level: ./_netcdf.c:2009: warning: initialization from incompatible pointer type ./_netcdf.c:2011: warning: 'intargfunc' is deprecated ./_netcdf.c:2011: warning: initialization from incompatible pointer type ./_netcdf.c:2012: warning: 'intargfunc' is deprecated ./_netcdf.c:2012: warning: initialization from incompatible pointer type ./_netcdf.c:2013: warning: 'intintargfunc' is deprecated ./_netcdf.c:2013: warning: initialization from incompatible pointer type ./_netcdf.c:2014: warning: initialization from incompatible pointer type ./_netcdf.c:2015: warning: initialization from incompatible pointer type ./_netcdf.c:2019: warning: initialization from incompatible pointer type ./_netcdf.c: In function 'PyNetCDFVariable_WriteArray': ./_netcdf.c:1626: warning: 'lastloop' may be used uninitialized in this function ./_netcdf.c: In function 'PyNetCDFVariable_WriteArray': ./_netcdf.c:1626: warning: 'lastloop' may be used uninitialized in this function gcc -arch i386 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk - bundle -undefined dynamic_lookup build/temp.macosx-10.3-fat-2.5/ _netcdf.o -L/Users/boyle5/netcdf-3.6.2/lib -lnetcdf -o build/ lib.macosx-10.3-fat-2.5/pynetcdf/_netcdf.so /usr/bin/ld: for architecture ppc /usr/bin/ld: warning /Users/boyle5/netcdf-3.6.2/lib/libnetcdf.a archive's cputype (7, architecture i386) does not match cputype (18) for specified -arch flag: ppc (can't load from it) [krait:~/Desktop/pynetcdf-0.7] boyle5% python ActivePython 2.5.0.0 (ActiveState Software Inc.) based on Python 2.5 (r25:51908, Mar 9 2007, 17:40:37) [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.0.3.dev3726'

5 4

Should numpy.allclose return True for arrays of different shape ?
by David Cournapeau 08 May '07

08 May '07

Hi, I wanted to know if the following behaviour is a bug or intended behaviour: """ import numpy as N N.allclose(N.array([[1., 1.]]), N.array([1.])) """ eg should allclose return True if arrays have different shape ? cheers, David

2 3

Searching object arrays
by Martin Spacek 08 May '07

08 May '07

I want to find the indices of all the None objects in an object array: >> import numpy as np >> i = np.array([0, 1, 2, None, 3, 4, None]) >> np.where(i == None) () Using == doesn't work the same way on object arrays as it does on, say, an array of int32. Any suggestions? Do I have to use a loop or list comprehension, like this? >> [ ii for (ii, id) in enumerate(i) if id == None ] Thanks, Martin

2 2

howto check arrays for equal (numpy 1.0b, array_equal is absent)? & best way to copy?
by dmitrey 07 May '07

07 May '07

hi all, I have some troubles with Python2.5+matplotlib, so now I'm using Python2.4.3. I failed to compile both numpy1.0.1 and 1.0.2 (Mandrake2007) so currently I'm using 1.0b. Howto check if numpy.array instances x and y are equal? (i.e. all elements are same) Please answer one more question - what is the best (fastest) way to create copy of numpy.array instance? WBR, D.

2 1

svd headers and diagonals
by dpn 06 May '07

06 May '07

Hi, i have two questions, both loosely related to SVD. I've seen this post: http://thread.gmane.org/gmane.comp.python.numeric.general/4575 >>> u,s,v = numpy.linalg.svd(numpy.array([[4,2],[2,4]])) # symmetric matrix u == v >>> u array([[-0.70710678, -0.70710678], [-0.70710678, 0.70710678]]) >>> v array([[-0.70710678, -0.70710678], [-0.70710678, 0.70710678]]) >>> s.shape (2,) since my data matrix is symmetrical, i'd expect USV = X, but I don't get that: >>> u * s * v array([[ 3., 1.], [ 3., 1.]]) matrixmultiply doesnt help either >>> from numpy.core import matrixmultiply as mm >>> mm(u,mm(s,v)) array([ 6., 2.]) Question 2. I'm relativly new to linealg, so i could be way off here. In applications such as LSA, the dimensions of a matrix are either documents or term identifiers, I noticed in PDL ( http://pdl.perl.org/ ), you can set the headers of row or columns. I havent found a way to do this in numpy, which means when the dimensions get sorted by their singular value, I lose the ordering I may have recorded externally. I there a way to store row and column headers? Cheers David Novakovic

2 2