[Numpy-discussion] ValueError: objects are not aligned

Wed Sep 8 09:21:55 EDT 2010

Thanks again,

I was trying to 
follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53

but with the key difference being, that instead of the data for the three arrays 
being constructed from literals, that i'd be sourcing from a relational 
database. The data is quite simple, in that I have 199 recorded days for 8 
different variables, but am just trying to do a proof-of-concept first with 
three, when I had this problem. 

The functionality I am seeking to reproduce is in Excel as a 3x3 grid:

=CORREL(rf1,rf1)
=CORREL(rf1,rf2)
=CORREL(rf1,rf3)
=CORREL(rf2,rf1)
=CORREL(rf2,rf2)
=CORREL(rf2,rf3)
=CORREL(rf3,rf1)
=CORREL(rf3,rf2)
=CORREL(rf3,rf3)

I tried to simplify and isolate the problem, and hence the confusion.

So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of 
the 8), each 199 rows in length.
From reading the lists and blogs, the way to put these into numpy seemed to be:
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)

Then to try and produce a correlation matrix:
print (np.corrcoef([rf1, rf2, rf3]))

But this is where I get the original error. 
From looking at column_stack, this doesn't seem like what I want to achieve, or 
am I looking at this the wrong way somehow?.

Thanks again,
Rui

________________________________
From: "josef.pktd at gmail.com" <josef.pktd at gmail.com>
To: Discussion of Numerical Python <numpy-discussion at scipy.org>
Sent: Wed, September 8, 2010 2:34:42 PM
Subject: Re: [Numpy-discussion] ValueError: objects are not aligned

On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <ruidc at yahoo.com> wrote:
> Thanks for your assistance,
> I was following this example:
>http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>3

I didn't know,
If arrays are 1d, then numpy does the stacking for you., but it
doesn't work with column-vectors or 2d arrays.
>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
array([[ 1.,  1.],
       [ 1.,  1.]])
>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
array([[ NaN,  NaN,  NaN],
       [ NaN,  NaN,  NaN],
       [ NaN,  NaN,  NaN]])
>
> I intend to use different arrays once I have resolved this problem.
> Trying your suggestion:
> ...
> print (rf1)
> print (np.corrcoef(rf1, rf1, rowvar=0))
> results in:
> [[-0.00641625]
> [-0.00498411]
> [-0.0038015 ]]
> [[ 1. 1.]
> [ 1. 1.]]
>
> which should have been a 3x3 matrix

I'm not sure what you want here, you only have three values
variance would also be zero, and corrcoef wouldn't be defined if you
only have 1 observation per variable

>>> np.corrcoef(np.arange(3)[:,None])
array([[ NaN,  NaN,  NaN],
       [ NaN,  NaN,  NaN],
       [ NaN,  NaN,  NaN]])

>>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, np.arange(5)**3]), 
>>>rowvar=0)
array([[ 1.        ,  0.9589266 ,  0.90588235],
       [ 0.9589266 ,  1.        ,  0.98713033],
       [ 0.90588235,  0.98713033,  1.        ]])

examples are in docstring for np.cov

> as for "stacking rf1 rf1 into a single array first" how would I do this?

>>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
array([[0, 0],
       [1, 1],
       [2, 2]])
>>> np.column_stack([np.arange(3)[:,None], np.arange(3)])  #doesn't required 2d
array([[0, 0],
       [1, 1],
       [2, 2]])

>>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
array([[0, 0],
       [1, 1],
       [2, 2]])
>>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
array([[0, 0],
       [1, 1],
       [2, 2]])

Josef

> Regards,
> Rui
> ________________________________
> From: "josef.pktd at gmail.com" <josef.pktd at gmail.com>
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Sent: Wed, September 8, 2010 1:09:46 PM
> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>
> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc at yahoo.com> wrote:
>>
>> I'm getting this error, which must be a simple error in getting the result
>> from the cursor in the right shape, how do I get the cursor into the array
>> as a single dimension?:
>>
>>    import numpy as np
>>    import sqlite3
>>    conn = sqlite3.connect("simpledb")
>>    cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>>    rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>>    print (rf1)
>>    print (np.corrcoef([rf1, rf1]))
>
> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>
> try
> np.corrcoef(rf1, rf1, rowvar=0)  no [ ]
> or
> stacking rf1 rf1 into a single array first
>
> Josef
>
>>
>> result:
>> [[-0.00641625]
>>  [-0.00498411]
>>  [-0.0038015 ]]
>> Traceback (most recent call last):
>>  File "C:\Documents and
>> Settings\rui\workspace\numpytest\src\numpytest.py",
>> line 38, in <module>
>>    print (np.corrcoef([rf1, rf1]))
>>  File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>> 2003, in corrcoef
>>    c = cov(x, y, rowvar, bias, ddof)
>>  File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>> 1953, in cov
>>    return (dot(X, X.T.conj()) / fact).squeeze()
>> --
>> View this message in context:
>>http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
>>l
>> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100908/6446834b/attachment.html>