Mailman 3 May 2010 - NumPy-Discussion

dtype and array creation
by Keith Goodman 21 May '10

21 May '10

Why do the follow expressions give different dtype? >> np.array([1, 2, 3], dtype=str) array(['1', '2', '3'], dtype='|S1') >> np.array(np.array([1, 2, 3]), dtype=str) array(['1', '2', '3'], dtype='|S8')

3 5

question about creating numpy arrays
by Darren Dale 20 May '10

20 May '10

I have a question about creation of numpy arrays from a list of objects, which bears on the Quantities project and also on masked arrays: >>> import quantities as pq >>> import numpy as np >>> a, b = 2*pq.m,1*pq.s >>> np.array([a, b]) array([ 12., 1.]) Why doesn't that create an object array? Similarly: >>> m = np.ma.array([1], mask=[True]) >>> m masked_array(data = [--], mask = [ True], fill_value = 999999) >>> np.array([m]) array([[1]]) This has broader implications than just creating arrays, for example: >>> np.sum([m, m]) 2 >>> np.sum([a, b]) 13.0 Any thoughts? Thanks, Darren

5 10

Runtime error in numpy.polyfit
by William Carithers 20 May '10

20 May '10

I'm trying to do a simple 2nd degree polynomial fit to two arrays of 5 entries. I get a runtime error: RuntimeError: more argument specifiers than keyword list entries (remaining format:'|:calc_lwork.gelss') in the lstsq module inside numpy.polyfit. Here's the code snippet: def findPeak(self, ydex, xdex): # take a vertical slice vslice = [] for i in range(-1,10,1) : vslice.append(arcImage[ydex+i][xdex]) vdex = n.where(vslice == max(vslice)) ymax = ydex -1 + vdex[0][0] # approximate gaussian fit by parabolic fit to logs yvalues = n.array([ymax-2, ymax-1, ymax, ymax+1, ymax+2]) svalues=n.array([arcImage[ymax-2][xdex],arcImage[ymax-1][xdex],arcImage[ymax ][xdex],arcImage[ymax+1][xdex], arcImage[ymax+2][xdex]]) avalues = n.log(svalues) ypoly = n.polyfit(yvalues, avalues, 2) And the traceback: File "/Users/williamcarithers/BOSS/src/calibrationModel.py", line 345, in findPeak ypoly = n.polyfit(yvalues, avalues, 2) File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/ numpy/lib/polynomial.py", line 503, in polyfit c, resids, rank, s = _lstsq(v, y, rcond) File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/ numpy/lib/polynomial.py", line 46, in _lstsq return lstsq(X, y, rcond) File "/Library/Python/2.6/site-packages/scipy-0.7.1-py2.6-macosx-10.6-universal.e gg/scipy/linalg/basic.py", line 545, in lstsq lwork = calc_lwork.gelss(gelss.prefix,m,n,nrhs)[1] RuntimeError: more argument specifiers than keyword list entries (remaining format:'|:calc_lwork.gelss') This is such a simple application of polyfit and the error occurs in the guts of lstsq, so I'm completely stumped. Any help would be greatly appreciated. Thanks, Bill Carithers

4 10

Converting None to NULL using ndpointer
by Lutz Maibaum 18 May '10

18 May '10

Hello, I am trying to use a C library function from Python using numpy and ctypes. Assume the function has the signature void foo(int* bar) so it usually takes an integer array as a parameter. In python, I define foo.argtypes=[ndpointer(dtype="intc",flags="C_CONTIGUOUS")] and can then call the function foo(x), where x is an appropriate ndarray. This works very well. The problem is that I need to be able to call the function foo with a NULL pointer as an argument. Ideally, I would like to be able to use foo(None), but that doesn't work because the argument conversion throws a TypeError. Therefore my questions: 1. Is there a ndarray that is converted to NULL by ndpointer? I was hoping that maybe ndarray(None) or an array of size 0 would work, but they don't. 2. Is there a way to tell ndpointer that None is an acceptable parameter, which should be converted to NULL? 3. I could also change the argument specification in python to foo.argtypes=[POINTER(c_int)], but then I would lose the type checking features of ndpointer. Can I invoke the ndpointer conversion manually? For example, does a ndpointer object have a method that takes a ndarray as an argument and returns a POINTER(c_int) if appropriate? Any suggestions would be much appreciated. Thanks, Lutz

1 0

Saving an array on disk to free memory - Pickling
by Jean-Baptiste Rudant 18 May '10

18 May '10

Hello, I tried to create an object : - which behave just like a numpy array ; - which can be saved on disk in an efficient way (numpy.save in my example but with pytables in my real program) ; - which can be "unloaded" (if it is saved) to free memory : it can exsit has an empty stuff which knows how to retrieve real values ; it will be loaded only when we need to work with it ; - which unloads itself before being pickled (values are already saved and don't have to be pickled). It can't, at least I think so, inherit from ndarray because sometimes (for example juste after being unpickled and before being used) it is juste an empty shell. I don't think memmap can be helpful (I want to use pytables to save it on disk and I want it to be flexible : if I use it in a temporary way, I just need it in memory and I will never save it on disk). My problems are : - this code is ugly ; - I have to define explicitely all special methods (__add__, __mul__...) of ndarrays because: * __getattr__ don't retrieve them ; * even if it does, I have to define explicitely the type of the return value (if I well understand, if it inherits from ndarray __array_wrap__ do all the stuff). Thank you for the help. Regards. import numpy import numpy class PersistentArray(object): def __init__(self, values): ''' values is a numpy array ''' self.values = values self.filename = None self.is_loaded = True self.is_saved = False def save(self, filename): self.filename = filename numpy.save(self.filename, self.values) self.is_saved = True def load(self): self.values = numpy.load(self.filename) self.is_loaded = True def unload(self): if not self.is_saved: raise Exception, "PersistentArray must be saved before being unloaded" del self.values self.is_loaded = False def __getitem__(self, index): return self.values[index] def __getattr__(self, key): if key == 'values': if not self.is_loaded: self.load() return self.values elif key == '__array_interface__': #I can't remember why I wrote this code, but I think it's necessary to make pickling work properly raise AttributeError, key else: try: #to emulate ndarray inheritance return self.values.__getattribute__(key) except AttributeError: raise AttributeError, key def __setstate__(self, dict): self.__dict__.update(dict) if self.is_loaded and self.is_saved: self.load() def __getstate__(self): if not self.is_saved: raise Exception, "persistent array must be saved before being pickled" odict = self.__dict__.copy() if self.is_saved: if self.is_loaded: odict['is_loaded'] = False del odict['values'] return odict filename = 'persistent_test.npy' a = PersistentArray(numpy.arange(10e6)) a.save(filename) a.sum() a.unload() # a still exists, knows how to retrieve values if needed, but don't use space in memory

3 4

Wrong Eigenvalue (Approximation?)
by Gabriel Mihalache 18 May '10

18 May '10

Hello, all! I'm new to Numpy and Python so please tolerate by ignorance on this but I'm having problems with some weird behavior. Consider the session: >>> import numpy as np >>> import numpy.linalg as la >>> x = np.array([[0.3, 0.2, 0.5], [0.2, 0.1, 0.7], [0.9, 0.05, 0.05]]).transpose() >>> esystem = la.eig(x) >>> esystem[0] array([ 1. , -0.3 , -0.25]) >>> esystem[0][0] 1.0000000000000004 The eigenvalue should be 1 exactly. In fact, later on I want to be able to do np.where(x == 1) which fails. The way I set up the matrix, I know for sure that there must be one eigenvalue exactly equal to 1. Any help is greatly appreciated! This is all on Python 2.6.5 (r265:79096, Mar 19 2010, 21:48:26) [MSC v.1500 32 bit (Intel)] on win32 Regards, Gabriel

6 5

savetxt not working with python3.1
by Nadav Horesh 18 May '10

18 May '10

in module npyio.py lines 794,796 "file" should be replaced by "_file" Nadav

3 2

faster code
by Davide Lasagna 17 May '10

17 May '10

Hi all, What is the fastest and lowest memory consumption way to compute this? y = np.arange(2**24) bases = y[1:] + y[:-1] Actually it is already quite fast, but i'm not sure whether it is occupying some temporary memory is the summation. Any help is appreciated. Cheers Davide

5 8

newbie: convert recarray to floating-point ndarray with mixed types
by Gregory, Matthew 17 May '10

17 May '10

Apologies for what is likely a simple question and I hope it hasn't been asked before ... Given a recarray with a dtype consisting of more than one type, e.g. >>> import numpy as n >>> a = n.array([(1.0, 2), (3.0, 4)], dtype=[('x', float), ('y', int)]) >>> b = a.view(n.recarray) >>> b rec.array([(1.0, 2), (3.0, 4)], dtype=[('x', '<f8'), ('y', '<i4')]) Is there a simple way to convert 'b' to a floating-point ndarray, casting the integer field to a floating-point? I've tried the naïve: >>> c = b.view(dtype='float').reshape(b.size,-1) but that fails with: ValueError: new type not compatible with array. I understand why this would fail (as it is a view and not a copy), but I'm lost on a method to do this conversion simply. thanks, matt

3 2

[sympy] EuroScipy abstract submission deadline extended
by Gael Varoquaux 17 May '10

17 May '10

Given that we have been able to turn on registration only very late, the EuroScipy conference committee is extending the deadline for abstract submission for the 2010 EuroScipy conference. On Thursday May 20th, at midnight Samoa time, we will turn off the abstract submission on the conference site. Up to then, you can modify the already-submitted abstract, or submit new abstracts. We are very much looking forward to your submissions to the conference. Gaël Varoquaux Nicolas Chauvat -- EuroScipy 2010 is the annual European conference for scientists using Python. It will be held July 8-11 2010, in ENS, Paris, France. Links: Conference website: http://www.euroscipy.org/conference/euroscipy2010 Call for papers: http://www.euroscipy.org/card/euroscipy2010_call_for_papers Practical information: http://www.euroscipy.org/card/euroscipy2010_practical_information

1 0