[Numpy-discussion] ValueError: objects are not aligned

Bruce Southey bsouthey at gmail.com
Wed Sep 8 09:39:56 EDT 2010 

  On 09/08/2010 08:30 AM, josef.pktd at gmail.com wrote:
> On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta<ruidc at yahoo.com>  wrote:
>> Thanks again,
>>
>> I was trying to
>> follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>> but with the key difference being, that instead of the data for the three
>> arrays being constructed from literals, that i'd be sourcing from a
>> relational database. The data is quite simple, in that I have 199 recorded
>> days for 8 different variables, but am just trying to do a proof-of-concept
>> first with three, when I had this problem.
>> The functionality I am seeking to reproduce is in Excel as a 3x3 grid:
>> =CORREL(rf1,rf1)
>> =CORREL(rf1,rf2)
>> =CORREL(rf1,rf3)
>> =CORREL(rf2,rf1)
>> =CORREL(rf2,rf2)
>> =CORREL(rf2,rf3)
>> =CORREL(rf3,rf1)
>> =CORREL(rf3,rf2)
>> =CORREL(rf3,rf3)
>> I tried to simplify and isolate the problem, and hence the confusion.
>> So In sourcing from the DB, I have 3 different cursors (taking a subset of 3
>> of the 8), each 199 rows in length.
>>  From reading the lists and blogs, the way to put these into numpy seemed to
>> be:
>> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>> rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64)
>> rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64)
>> Then to try and produce a correlation matrix:
>> print (np.corrcoef([rf1, rf2, rf3]))
>> But this is where I get the original error.
>>  From looking at column_stack, this doesn't seem like what I want to achieve,
>> or am I looking at this the wrong way somehow?.
> column_stack and rowvar=0 should work, but it might be easier to get
> rid of the extra dimension that the cursor gives you
>
> e.g.
> rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
>
> depends on whether you want to work with 2d or 1d data arrays
>
> Josef
>
>> Thanks again,
>> Rui
>>
>> ________________________________
>> From: "josef.pktd at gmail.com"<josef.pktd at gmail.com>
>> To: Discussion of Numerical Python<numpy-discussion at scipy.org>
>> Sent: Wed, September 8, 2010 2:34:42 PM
>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>
>> On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta<ruidc at yahoo.com>  wrote:
>>> Thanks for your assistance,
>>> I was following this example:
>>>
>>> http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee9d68a4a53
>> I didn't know,
>> If arrays are 1d, then numpy does the stacking for you., but it
>> doesn't work with column-vectors or 2d arrays.
>>>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=1)
>> array([[ 1.,  1.],
>>        [ 1.,  1.]])
>>>>> np.corrcoef([np.arange(3), np.arange(3)], rowvar=0)
>> array([[ NaN,  NaN,  NaN],
>>        [ NaN,  NaN,  NaN],
>>        [ NaN,  NaN,  NaN]])
>>> I intend to use different arrays once I have resolved this problem.
>>> Trying your suggestion:
>>> ...
>>> print (rf1)
>>> print (np.corrcoef(rf1, rf1, rowvar=0))
>>> results in:
>>> [[-0.00641625]
>>> [-0.00498411]
>>> [-0.0038015 ]]
>>> [[ 1. 1.]
>>> [ 1. 1.]]
>>>
>>> which should have been a 3x3 matrix
>> I'm not sure what you want here, you only have three values
>> variance would also be zero, and corrcoef wouldn't be defined if you
>> only have 1 observation per variable
>>
>>>>> np.corrcoef(np.arange(3)[:,None])
>> array([[ NaN,  NaN,  NaN],
>>        [ NaN,  NaN,  NaN],
>>        [ NaN,  NaN,  NaN]])
>>
>>>>> np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2,
>>>>> np.arange(5)**3]), rowvar=0)
>> array([[ 1.        ,  0.9589266 ,  0.90588235],
>>        [ 0.9589266 ,  1.        ,  0.98713033],
>>        [ 0.90588235,  0.98713033,  1.        ]])
>>
>> examples are in docstring for np.cov
>>
>>
>>
>>
>>> as for "stacking rf1 rf1 into a single array first" how would I do this?
>>>>> np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]])
>> array([[0, 0],
>>        [1, 1],
>>        [2, 2]])
>>>>> np.column_stack([np.arange(3)[:,None], np.arange(3)])  #doesn't required
>>>>> 2d
>> array([[0, 0],
>>        [1, 1],
>>        [2, 2]])
>>
>>>>> np.c_[np.arange(3)[:,None], np.arange(3)[:,None]]
>> array([[0, 0],
>>        [1, 1],
>>        [2, 2]])
>>>>> np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]])
>> array([[0, 0],
>>        [1, 1],
>>        [2, 2]])
>>
>>
>> Josef
>>
>>> Regards,
>>> Rui
>>> ________________________________
>>> From: "josef.pktd at gmail.com"<josef.pktd at gmail.com>
>>> To: Discussion of Numerical Python<numpy-discussion at scipy.org>
>>> Sent: Wed, September 8, 2010 1:09:46 PM
>>> Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
>>>
>>> On Wed, Sep 8, 2010 at 5:42 AM, RuiDC<ruidc at yahoo.com>  wrote:
>>>> I'm getting this error, which must be a simple error in getting the
>>>> result
>>>> from the cursor in the right shape, how do I get the cursor into the
>>>> array
>>>> as a single dimension?:
>>>>
>>>>     import numpy as np
>>>>     import sqlite3
>>>>     conn = sqlite3.connect("simpledb")
>>>>     cursor1 = conn.execute("select LogReturn from tblReturns limit 3")
>>>>     rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64)
>>>>     print (rf1)
>>>>     print (np.corrcoef([rf1, rf1]))
>>> numpy.corrcoef(x, y=None, rowvar=1, bias=0)
>>>
>>> try
>>> np.corrcoef(rf1, rf1, rowvar=0)  no [ ]
>>> or
>>> stacking rf1 rf1 into a single array first
>>>
>>> Josef
>>>
>>>> result:
>>>> [[-0.00641625]
>>>>   [-0.00498411]
>>>>   [-0.0038015 ]]
>>>> Traceback (most recent call last):
>>>>   File "C:\Documents and
>>>> Settings\rui\workspace\numpytest\src\numpytest.py",
>>>> line 38, in<module>
>>>>     print (np.corrcoef([rf1, rf1]))
>>>>   File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>>>> 2003, in corrcoef
>>>>     c = cov(x, y, rowvar, bias, ddof)
>>>>   File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line
>>>> 1953, in cov
>>>>     return (dot(X, X.T.conj()) / fact).squeeze()
>>>> --
>>>> View this message in context:
>>>>
>>>> http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641661.html
>>>> Sent from the Numpy-discussion mailing list archive at Nabble.com.
>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion

Your errors suggest that you are not getting what expect from the 
database query. Please check the dtype and shape of your arrays created 
from the database query. Then check the contents to ensure the values 
make sense especially for unexpected values. You should be able to find 
the mean and variance of each of your arrays.

Of course, providing the actual data from the query and/or the arrays is 
also a plus.

Bruce

More information about the NumPy-Discussion mailing list