I just started working on a time-series module/class in scipy/numpy and it
seemed useful to have some of the R data-frame functionality (i.e., select
columns of data based on variable names).
I tried rec-arrays but couldn't get them to work the way I wanted. I also
looked at the Dataframe class by Andrew Straw but at over 400 lines of code
that seemed pretty complicated, to me at least.
I searched the mailing-list archives and found a discussion on 'Table like
array' (see exert below). To get the minimal functionality discussed, I
wrote a simple class (see attached) to try and implement X.get('a','c')
where 'a' and 'c' are variables names linked to columns of data in X.
I added some test code so that if you run the code in the attachment you
will see that is seems to work. However, since this is my first class I'd
appreciate your input on the approach I used and any suggestions on how to
improve the class (or use something else). I'd like to read the data and
variable names directly from a single csv file. I tried this through the
python csv module but it would read all data as strings and I couldn't
figure out how to easily separate the variable names and the data.
Thanks,
Vincent
> [Numpy-discussion] Re: [SciPy-user] Table like array
> Paul Barrett pebarrett at gmail.com
> Wed Mar 1 06:45:02 CST 2006
>
> On 3/1/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>>
>>
>> How many people would like to see x['f1','f2','f5'] return a new array
>> with a new data-type descriptor constructed from the provided fields?
>>
>>
>
> I'm surprised that it's not already available.
>
> -- Paul
Some of my arrays are not fully populated. (I separately record which
entries are valid.) I want to use numpy.empty() to speed up the
creation of these arrays, but I'm worried about what will happen if I
apply operations to the entire contents of these arrays. E.g.
a + b
I care about the results where valid entries align, but not otherwise.
Given that numpy.empty() creates an ndarray using whatever junk it finds
on the heap, it seems to me that there is the possibility that this
could include bit patterns that are not valid floating point
representations, which might raise floating point exceptions if used in
operations like the one above (if they are "signalling" NaNs). Will
this be a problem, or will the results of operations on invalid floating
point numbers yield NaN?
Or to put it another way: do I need to ensure that array data is
initialised before using it?
Oliver
--
This message and any attachments are confidential, proprietary, and may be privileged. If this message was misdirected, Barclays Global Investors (BGI) does not waive any confidentiality or privilege. If you are not the intended recipient, please notify us immediately and destroy the message without disclosing its contents to anyone. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. The views and opinions expressed in this e-mail message are the author's own and may not reflect the views and opinions of BGI, unless the author is authorized by BGI to express such views or opinions on its behalf. All email sent to or from this address is subject to electronic storage and review by BGI. Although BGI operates anti-virus programs, it does not accept responsibility for any damage whatsoever caused by viruses being passed.
Hi,
I'm quite new to numpy. My understanding of argmax() is that it gives
the index of the highest number in the array, just like:
from numpy import *
a=array(range(4096))
a=a*15
argmax(a)
gives 4095.
But, when I use this within my program I have a little function:
def localMaxima(set):
maxPos = argmax(set)
maxVal = set[maxPos]
return maxPos, maxVal
that I have had to extend a little bit to:
def localMaxima(set):
maxPos = argmax(set)
try:
maxVal = set[maxPos]
print "ok"
except:
print "maxPos = " + str(maxPos) + ", while len(set) is " + str(len
(set))
print set
maxVal = 0.0
return maxPos, maxVal
The problem is that set is an array of 2048 numbers. However, when I
run this function I get:
"maxPos = 7925760, while len(set) is 2048"
I do not understand why I get such a high index. Do you have any
suggestions?
Cheers
Nik
What is the best way to treat scalar values from a matrix?
It seems that it's best to treat them as a simple Python values and not
to leave them wrapped up in the ndarray structure. I would welcome advice.
The problem is illustrated below.
Colin W.
# tMul.py
import numpy.core as _n
a= _n.arange(12).reshape((3, 4))
b= _n.arange(12).reshape((4, 3))
c= _n.dot(a, b)
print `c`
amat= _n.mat(a)
bmat= _n.mat(b)
cmat= amat*bmat
print cmat
print _n.dot(amat, _n.mat(5))
Output:
>>>
array([[ 42, 48, 54],
[114, 136, 158],
[186, 224, 262]])
[[ 42 48 54]
[114 136 158]
[186 224 262]]
Traceback (most recent call last):
File "<string>", line 74, in run_nodebug
File "C:\Documents and Settings\cjw\My
Documents\PyDev\PyMatrix\tMiul.py", line 11, in <module>
print _n.dot(amat, _n.mat(5))
ValueError: objects are not aligned
>>>
Is it hard to subclass recarrays? I'm currently working on a subclass
that doesn't add much; just some methods that sets all the values to
zero and other similarly trivial things. It'd be nice to make it work,
but if I have to use external functions on a plain recarray instead of
this subclass, I won't be found weeping in the corner.
The constructor of this subclass just calls recarray's __init__ method,
but when it does this, I get an error. The class is currently just the
following:
class nnvalue(N.recarray):
def __init__(self, shape):
N.recarray.__init__(self, shape, dtype=N.dtype([('x', 'f8'),
('dx', 'f8'), ('delta', 'f8')]))
def clearGradients(self):
"""Fills all members other than x with zeros."""
self.dx[:] = 0.
self.deltas[:] = 0.
The error message reads:
Traceback (most recent call last):
File "/home/mkg/projects/physlearn/src/pl/modules.py", line 287, in ?
mod = DynModule(states.shape[1:], controls.shape[1:], states.shape[0])
File "/home/mkg/projects/physlearn/src/pl/modules.py", line 62, in
__init__
self.params = nnvalue(pSize)
File "/usr/lib/python2.4/site-packages/numpy/core/records.py", line
175, in __new__
descr = format_parser(formats, names, titles, aligned, byteorder)._descr
File "/usr/lib/python2.4/site-packages/numpy/core/records.py", line
44, in __init__
self._parseFormats(formats, aligned)
File "/usr/lib/python2.4/site-packages/numpy/core/records.py", line
52, in _parseFormats
raise ValueError, "Need formats argument"
ValueError: Need formats argument
Any help would be much appreciated.
-- Matt
