NumPy-Discussion January 2007

numpy-discussion@python.org

96 participants
115 discussions

newbie: attempt at data frame
by Vincent Nijs 03 Jan '07

03 Jan '07

I just started working on a time-series module/class in scipy/numpy and it seemed useful to have some of the R data-frame functionality (i.e., select columns of data based on variable names). I tried rec-arrays but couldn't get them to work the way I wanted. I also looked at the Dataframe class by Andrew Straw but at over 400 lines of code that seemed pretty complicated, to me at least. I searched the mailing-list archives and found a discussion on 'Table like array' (see exert below). To get the minimal functionality discussed, I wrote a simple class (see attached) to try and implement X.get('a','c') where 'a' and 'c' are variables names linked to columns of data in X. I added some test code so that if you run the code in the attachment you will see that is seems to work. However, since this is my first class I'd appreciate your input on the approach I used and any suggestions on how to improve the class (or use something else). I'd like to read the data and variable names directly from a single csv file. I tried this through the python csv module but it would read all data as strings and I couldn't figure out how to easily separate the variable names and the data. Thanks, Vincent > [Numpy-discussion] Re: [SciPy-user] Table like array > Paul Barrett pebarrett at gmail.com > Wed Mar 1 06:45:02 CST 2006 > > On 3/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote: >> >> >> How many people would like to see x['f1','f2','f5'] return a new array >> with a new data-type descriptor constructed from the provided fields? >> >> > > I'm surprised that it's not already available. > > -- Paul

3 7

The risks of empty()
by Bock, Oliver BGI SYD 03 Jan '07

03 Jan '07

Some of my arrays are not fully populated. (I separately record which entries are valid.) I want to use numpy.empty() to speed up the creation of these arrays, but I'm worried about what will happen if I apply operations to the entire contents of these arrays. E.g. a + b I care about the results where valid entries align, but not otherwise. Given that numpy.empty() creates an ndarray using whatever junk it finds on the heap, it seems to me that there is the possibility that this could include bit patterns that are not valid floating point representations, which might raise floating point exceptions if used in operations like the one above (if they are "signalling" NaNs). Will this be a problem, or will the results of operations on invalid floating point numbers yield NaN? Or to put it another way: do I need to ensure that array data is initialised before using it? Oliver -- This message and any attachments are confidential, proprietary, and may be privileged. If this message was misdirected, Barclays Global Investors (BGI) does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this e-mail message are the author's own and may not reflect the views and opinions of BGI, unless the author is authorized by BGI to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by BGI. Although BGI operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed.

4 3

argmax()
by Niklas Saers 02 Jan '07

02 Jan '07

Hi, I'm quite new to numpy. My understanding of argmax() is that it gives the index of the highest number in the array, just like: from numpy import * a=array(range(4096)) a=a*15 argmax(a) gives 4095. But, when I use this within my program I have a little function: def localMaxima(set): maxPos = argmax(set) maxVal = set[maxPos] return maxPos, maxVal that I have had to extend a little bit to: def localMaxima(set): maxPos = argmax(set) try: maxVal = set[maxPos] print "ok" except: print "maxPos = " + str(maxPos) + ", while len(set) is " + str(len (set)) print set maxVal = 0.0 return maxPos, maxVal The problem is that set is an array of 2048 numbers. However, when I run this function I get: "maxPos = 7925760, while len(set) is 2048" I do not understand why I get such a high index. Do you have any suggestions? Cheers Nik

2 3

Scalar values with a matrix
by Colin J. Williams 02 Jan '07

02 Jan '07

What is the best way to treat scalar values from a matrix? It seems that it's best to treat them as a simple Python values and not to leave them wrapped up in the ndarray structure. I would welcome advice. The problem is illustrated below. Colin W. # tMul.py import numpy.core as _n a= _n.arange(12).reshape((3, 4)) b= _n.arange(12).reshape((4, 3)) c= _n.dot(a, b) print `c` amat= _n.mat(a) bmat= _n.mat(b) cmat= amat*bmat print cmat print _n.dot(amat, _n.mat(5)) Output: >>> array([[ 42, 48, 54], [114, 136, 158], [186, 224, 262]]) [[ 42 48 54] [114 136 158] [186 224 262]] Traceback (most recent call last): File "<string>", line 74, in run_nodebug File "C:\Documents and Settings\cjw\My Documents\PyDev\PyMatrix\tMiul.py", line 11, in <module> print _n.dot(amat, _n.mat(5)) ValueError: objects are not aligned >>>

3 2

subclassing recarray
by Matthew Koichi Grimes 02 Jan '07

02 Jan '07

Is it hard to subclass recarrays? I'm currently working on a subclass that doesn't add much; just some methods that sets all the values to zero and other similarly trivial things. It'd be nice to make it work, but if I have to use external functions on a plain recarray instead of this subclass, I won't be found weeping in the corner. The constructor of this subclass just calls recarray's __init__ method, but when it does this, I get an error. The class is currently just the following: class nnvalue(N.recarray): def __init__(self, shape): N.recarray.__init__(self, shape, dtype=N.dtype([('x', 'f8'), ('dx', 'f8'), ('delta', 'f8')])) def clearGradients(self): """Fills all members other than x with zeros.""" self.dx[:] = 0. self.deltas[:] = 0. The error message reads: Traceback (most recent call last): File "/home/mkg/projects/physlearn/src/pl/modules.py", line 287, in ? mod = DynModule(states.shape[1:], controls.shape[1:], states.shape[0]) File "/home/mkg/projects/physlearn/src/pl/modules.py", line 62, in __init__ self.params = nnvalue(pSize) File "/usr/lib/python2.4/site-packages/numpy/core/records.py", line 175, in __new__ descr = format_parser(formats, names, titles, aligned, byteorder)._descr File "/usr/lib/python2.4/site-packages/numpy/core/records.py", line 44, in __init__ self._parseFormats(formats, aligned) File "/usr/lib/python2.4/site-packages/numpy/core/records.py", line 52, in _parseFormats raise ValueError, "Need formats argument" ValueError: Need formats argument Any help would be much appreciated. -- Matt

2 1

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion January 2007