[Numpy-discussion] ValueError: objects are not aligned

[Rui DaCosta]

rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()


did the trick, 1d is perfect

Thankyou very much!

Rui





________________________________
From: "josef.pktd at gmail.com" <josef.pktd at gmail.com>
To: Discussion of Numerical Python <numpy-discussion at scipy.org>
Sent: Wed, September 8, 2010 3:30:54 PM
Subject: Re: [Numpy-discussion] ValueError: objects are not aligned

On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta <ruidc at yahoo.com> wrote:
> Thanks again,
>
> I was trying to
>follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>3
> but with the key difference being, that instead of the data for the three
> arrays being constructed from literals, that i'd be sourcing from a
> relational database. The data is quite simple, in that I have 199 recorded
> days for 8 different variables, but am just trying to do a proof-of-concept
> first with three, when I had this problem.
> The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
> =CORREL(rf1,rf1)
> =CORREL(rf1,rf2)
> =CORREL(rf1,rf3)
> =CORREL(rf2,rf1)
> =CORREL(rf2,rf2)
> =CORREL(rf2,rf3)
> =CORREL(rf3,rf1)
> =CORREL(rf3,rf2)
> =CORREL(rf3,rf3)
> I tried to simplify and isolate the problem, and hence the confusion.
> So In sourcing from the DB, I have 3 different cursors (taking a subset of 3
> of the 8), each 199 rows in length.
> From reading the lists and blogs, the way to put these into numpy seemed to
> be:
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
> rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
> rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
> Then to try and produce a correlation matrix:
> print (np.corrcoef([rf1, rf2, rf3]))
> But this is where I get the original error.
> From looking at column_stack, this doesn't seem like what I want to achieve,
> or am I looking at this the wrong way somehow?.

column_stack and rowvar=0 should work, but it might be easier to get
rid of the extra dimension that the cursor gives you

e.g.
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()

depends on whether you want to work with 2d or 1d data arrays

Josef

> Thanks again,
> Rui
>
> ________________________________
> From: "josef.pktd at gmail.com" <josef.pktd at gmail.com>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Sent: Wed, September 8, 2010 2:34:42 PM
> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>
> On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <ruidc at yahoo.com> wrote:
>> Thanks for your assistance,
>> I was following this example:
>>
>>http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>>3
>
> I didn't know,
> If arrays are 1d, then numpy does the stacking for you., but it
> doesn't work with column-vectors or 2d arrays.
>>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
> array([[ 1.,  1.],
>       [ 1.,  1.]])
>>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
> array([[ NaN,  NaN,  NaN],
>       [ NaN,  NaN,  NaN],
>       [ NaN,  NaN,  NaN]])
>>
>> I intend to use different arrays once I have resolved this problem.
>> Trying your suggestion:
>> ...
>> print (rf1)
>> print (np.corrcoef(rf1, rf1, rowvar=0))
>> results in:
>> [[-0.00641625]
>> [-0.00498411]
>> [-0.0038015 ]]
>> [[ 1. 1.]
>> [ 1. 1.]]
>>
>> which should have been a 3x3 matrix
>
> I'm not sure what you want here, you only have three values
> variance would also be zero, and corrcoef wouldn't be defined if you
> only have 1 observation per variable
>
>>>> np.corrcoef(np.arange(3)[:,None])
> array([[ NaN,  NaN,  NaN],
>       [ NaN,  NaN,  NaN],
>       [ NaN,  NaN,  NaN]])
>
>>>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
>>>> np.arange(5)**3]), rowvar=0)
> array([[ 1.        ,  0.9589266 ,  0.90588235],
>       [ 0.9589266 ,  1.        ,  0.98713033],
>       [ 0.90588235,  0.98713033,  1.        ]])
>
> examples are in docstring for np.cov
>
>
>
>
>> as for "stacking rf1 rf1 into a single array first" how would I do this?
>
>>>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
> array([[0, 0],
>       [1, 1],
>       [2, 2]])
>>>> np.column_stack([np.arange(3)[:,None], np.arange(3)])  #doesn't required
>>>> 2d
> array([[0, 0],
>       [1, 1],
>       [2, 2]])
>
>>>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
> array([[0, 0],
>       [1, 1],
>       [2, 2]])
>>>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
> array([[0, 0],
>       [1, 1],
>       [2, 2]])
>
>
> Josef
>
>> Regards,
>> Rui
>> ________________________________
>> From: "josef.pktd at gmail.com" <josef.pktd at gmail.com>
>> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
>> Sent: Wed, September 8, 2010 1:09:46 PM
>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>
>> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc at yahoo.com> wrote:
>>>
>>> I'm getting this error, which must be a simple error in getting the
>>> result
>>> from the cursor in the right shape, how do I get the cursor into the
>>> array
>>> as a single dimension?:
>>>
>>>    import numpy as np
>>>    import sqlite3
>>>    conn = sqlite3.connect("simpledb")
>>>    cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>>>    rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>>>    print (rf1)
>>>    print (np.corrcoef([rf1, rf1]))
>>
>> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>>
>> try
>> np.corrcoef(rf1, rf1, rowvar=0)  no [ ]
>> or
>> stacking rf1 rf1 into a single array first
>>
>> Josef
>>
>>>
>>> result:
>>> [[-0.00641625]
>>>  [-0.00498411]
>>>  [-0.0038015 ]]
>>> Traceback (most recent call last):
>>>  File "C:\Documents and
>>> Settings\rui\workspace\numpytest\src\numpytest.py",
>>> line 38, in <module>
>>>    print (np.corrcoef([rf1, rf1]))
>>>  File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>>> 2003, in corrcoef
>>>    c = cov(x, y, rowvar, bias, ddof)
>>>  File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>>> 1953, in cov
>>>    return (dot(X, X.T.conj()) / fact).squeeze()
>>> --
>>> View this message in context:
>>>
>>>http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
>>>l
>>> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100908/5d67147f/attachment.html>