ValueError: objects are not aligned
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?: import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1])) result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context: http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... Sent from the Numpy-discussion mailing list archive at Nabble.com.
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0) try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context: http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks for your assistance, I was following this example: http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... I intend to use different arrays once I have resolved this problem. Trying your suggestion: ... print (rf1) print (np.corrcoef(rf1, rf1, rowvar=0)) results in: [[-0.00641625] [-0.00498411] [-0.0038015 ]] [[ 1. 1.] [ 1. 1.]] which should have been a 3x3 matrix as for "stacking rf1 rf1 into a single array first" how would I do this? Regards, Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 1:09:46 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0) try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context: http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641...
Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <ruidc@yahoo.com> wrote:
Thanks for your assistance, I was following this example: http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee...
I didn't know, If arrays are 1d, then numpy does the stacking for you., but it doesn't work with column-vectors or 2d arrays.
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1) array([[ 1., 1.], [ 1., 1.]]) np.corrcoef([np.arange(3), np.arange(3)], rowvar=0) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
I intend to use different arrays once I have resolved this problem. Trying your suggestion: ... print (rf1) print (np.corrcoef(rf1, rf1, rowvar=0)) results in: [[-0.00641625] [-0.00498411] [-0.0038015 ]] [[ 1. 1.] [ 1. 1.]]
which should have been a 3x3 matrix
I'm not sure what you want here, you only have three values variance would also be zero, and corrcoef wouldn't be defined if you only have 1 observation per variable
np.corrcoef(np.arange(3)[:,None]) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, np.arange(5)**3]), rowvar=0) array([[ 1. , 0.9589266 , 0.90588235], [ 0.9589266 , 1. , 0.98713033], [ 0.90588235, 0.98713033, 1. ]])
examples are in docstring for np.cov
as for "stacking rf1 rf1 into a single array first" how would I do this?
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d array([[0, 0], [1, 1], [2, 2]])
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]] array([[0, 0], [1, 1], [2, 2]]) np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]])
Josef
Regards, Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 1:09:46 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first
Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context: http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks again, I was trying to follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... but with the key difference being, that instead of the data for the three arrays being constructed from literals, that i'd be sourcing from a relational database. The data is quite simple, in that I have 199 recorded days for 8 different variables, but am just trying to do a proof-of-concept first with three, when I had this problem. The functionality I am seeking to reproduce is in Excel as a 3x3 grid: =CORREL(rf1,rf1) =CORREL(rf1,rf2) =CORREL(rf1,rf3) =CORREL(rf2,rf1) =CORREL(rf2,rf2) =CORREL(rf2,rf3) =CORREL(rf3,rf1) =CORREL(rf3,rf2) =CORREL(rf3,rf3) I tried to simplify and isolate the problem, and hence the confusion. So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of the 8), each 199 rows in length. From reading the lists and blogs, the way to put these into numpy seemed to be: rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64) rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64) Then to try and produce a correlation matrix: print (np.corrcoef([rf1, rf2, rf3])) But this is where I get the original error. From looking at column_stack, this doesn't seem like what I want to achieve, or am I looking at this the wrong way somehow?. Thanks again, Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 2:34:42 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <ruidc@yahoo.com> wrote:
Thanks for your assistance, I was following this example: http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... 3
I didn't know, If arrays are 1d, then numpy does the stacking for you., but it doesn't work with column-vectors or 2d arrays.
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1) array([[ 1., 1.], [ 1., 1.]]) np.corrcoef([np.arange(3), np.arange(3)], rowvar=0) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
I intend to use different arrays once I have resolved this problem. Trying your suggestion: ... print (rf1) print (np.corrcoef(rf1, rf1, rowvar=0)) results in: [[-0.00641625] [-0.00498411] [-0.0038015 ]] [[ 1. 1.] [ 1. 1.]]
which should have been a 3x3 matrix
I'm not sure what you want here, you only have three values variance would also be zero, and corrcoef wouldn't be defined if you only have 1 observation per variable
np.corrcoef(np.arange(3)[:,None]) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, np.arange(5)**3]), rowvar=0) array([[ 1. , 0.9589266 , 0.90588235], [ 0.9589266 , 1. , 0.98713033], [ 0.90588235, 0.98713033, 1. ]])
examples are in docstring for np.cov
as for "stacking rf1 rf1 into a single array first" how would I do this?
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d array([[0, 0], [1, 1], [2, 2]])
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]] array([[0, 0], [1, 1], [2, 2]]) np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]])
Josef
Regards, Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 1:09:46 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first
Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context: http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... l Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta <ruidc@yahoo.com> wrote:
Thanks again,
I was trying to follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... but with the key difference being, that instead of the data for the three arrays being constructed from literals, that i'd be sourcing from a relational database. The data is quite simple, in that I have 199 recorded days for 8 different variables, but am just trying to do a proof-of-concept first with three, when I had this problem. The functionality I am seeking to reproduce is in Excel as a 3x3 grid: =CORREL(rf1,rf1) =CORREL(rf1,rf2) =CORREL(rf1,rf3) =CORREL(rf2,rf1) =CORREL(rf2,rf2) =CORREL(rf2,rf3) =CORREL(rf3,rf1) =CORREL(rf3,rf2) =CORREL(rf3,rf3) I tried to simplify and isolate the problem, and hence the confusion. So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of the 8), each 199 rows in length. From reading the lists and blogs, the way to put these into numpy seemed to be: rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64) rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64) Then to try and produce a correlation matrix: print (np.corrcoef([rf1, rf2, rf3])) But this is where I get the original error. From looking at column_stack, this doesn't seem like what I want to achieve, or am I looking at this the wrong way somehow?.
column_stack and rowvar=0 should work, but it might be easier to get rid of the extra dimension that the cursor gives you e.g. rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel() depends on whether you want to work with 2d or 1d data arrays Josef
Thanks again, Rui
________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 2:34:42 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <ruidc@yahoo.com> wrote:
Thanks for your assistance, I was following this example:
http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee...
I didn't know, If arrays are 1d, then numpy does the stacking for you., but it doesn't work with column-vectors or 2d arrays.
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1) array([[ 1., 1.], [ 1., 1.]]) np.corrcoef([np.arange(3), np.arange(3)], rowvar=0) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
I intend to use different arrays once I have resolved this problem. Trying your suggestion: ... print (rf1) print (np.corrcoef(rf1, rf1, rowvar=0)) results in: [[-0.00641625] [-0.00498411] [-0.0038015 ]] [[ 1. 1.] [ 1. 1.]]
which should have been a 3x3 matrix
I'm not sure what you want here, you only have three values variance would also be zero, and corrcoef wouldn't be defined if you only have 1 observation per variable
np.corrcoef(np.arange(3)[:,None]) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, np.arange(5)**3]), rowvar=0) array([[ 1. , 0.9589266 , 0.90588235], [ 0.9589266 , 1. , 0.98713033], [ 0.90588235, 0.98713033, 1. ]])
examples are in docstring for np.cov
as for "stacking rf1 rf1 into a single array first" how would I do this?
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d array([[0, 0], [1, 1], [2, 2]])
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]] array([[0, 0], [1, 1], [2, 2]]) np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]])
Josef
Regards, Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 1:09:46 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first
Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context:
http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 09/08/2010 08:30 AM, josef.pktd@gmail.com wrote:
On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta<ruidc@yahoo.com> wrote:
Thanks again,
I was trying to follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... but with the key difference being, that instead of the data for the three arrays being constructed from literals, that i'd be sourcing from a relational database. The data is quite simple, in that I have 199 recorded days for 8 different variables, but am just trying to do a proof-of-concept first with three, when I had this problem. The functionality I am seeking to reproduce is in Excel as a 3x3 grid: =CORREL(rf1,rf1) =CORREL(rf1,rf2) =CORREL(rf1,rf3) =CORREL(rf2,rf1) =CORREL(rf2,rf2) =CORREL(rf2,rf3) =CORREL(rf3,rf1) =CORREL(rf3,rf2) =CORREL(rf3,rf3) I tried to simplify and isolate the problem, and hence the confusion. So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of the 8), each 199 rows in length. From reading the lists and blogs, the way to put these into numpy seemed to be: rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64) rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64) Then to try and produce a correlation matrix: print (np.corrcoef([rf1, rf2, rf3])) But this is where I get the original error. From looking at column_stack, this doesn't seem like what I want to achieve, or am I looking at this the wrong way somehow?.
column_stack and rowvar=0 should work, but it might be easier to get rid of the extra dimension that the cursor gives you
e.g. rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel()
depends on whether you want to work with 2d or 1d data arrays
Josef
Thanks again, Rui
________________________________ From: "josef.pktd@gmail.com"<josef.pktd@gmail.com> To: Discussion of Numerical Python<numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 2:34:42 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta<ruidc@yahoo.com> wrote:
Thanks for your assistance, I was following this example:
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1) array([[ 1., 1.], [ 1., 1.]]) np.corrcoef([np.arange(3), np.arange(3)], rowvar=0) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]]) I intend to use different arrays once I have resolved this problem. Trying your suggestion: ...
http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... I didn't know, If arrays are 1d, then numpy does the stacking for you., but it doesn't work with column-vectors or 2d arrays. print (rf1) print (np.corrcoef(rf1, rf1, rowvar=0)) results in: [[-0.00641625] [-0.00498411] [-0.0038015 ]] [[ 1. 1.] [ 1. 1.]]
which should have been a 3x3 matrix I'm not sure what you want here, you only have three values variance would also be zero, and corrcoef wouldn't be defined if you only have 1 observation per variable
np.corrcoef(np.arange(3)[:,None]) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, np.arange(5)**3]), rowvar=0) array([[ 1. , 0.9589266 , 0.90588235], [ 0.9589266 , 1. , 0.98713033], [ 0.90588235, 0.98713033, 1. ]])
examples are in docstring for np.cov
as for "stacking rf1 rf1 into a single array first" how would I do this?
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d array([[0, 0], [1, 1], [2, 2]])
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]] array([[0, 0], [1, 1], [2, 2]]) np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]])
Josef
Regards, Rui ________________________________ From: "josef.pktd@gmail.com"<josef.pktd@gmail.com> To: Discussion of Numerical Python<numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 1:09:46 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC<ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1])) numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first
Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in<module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context:
http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Your errors suggest that you are not getting what expect from the database query. Please check the dtype and shape of your arrays created from the database query. Then check the contents to ensure the values make sense especially for unexpected values. You should be able to find the mean and variance of each of your arrays. Of course, providing the actual data from the query and/or the arrays is also a plus. Bruce
rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel() did the trick, 1d is perfect Thankyou very much! Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 3:30:54 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned On Wed, Sep 8, 2010 at 9:21 AM, Rui DaCosta <ruidc@yahoo.com> wrote:
Thanks again,
I was trying to follow http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... 3 but with the key difference being, that instead of the data for the three arrays being constructed from literals, that i'd be sourcing from a relational database. The data is quite simple, in that I have 199 recorded days for 8 different variables, but am just trying to do a proof-of-concept first with three, when I had this problem. The functionality I am seeking to reproduce is in Excel as a 3x3 grid: =CORREL(rf1,rf1) =CORREL(rf1,rf2) =CORREL(rf1,rf3) =CORREL(rf2,rf1) =CORREL(rf2,rf2) =CORREL(rf2,rf3) =CORREL(rf3,rf1) =CORREL(rf3,rf2) =CORREL(rf3,rf3) I tried to simplify and isolate the problem, and hence the confusion. So In sourcing from the DB, I have 3 different cursors (taking a subset of 3 of the 8), each 199 rows in length. From reading the lists and blogs, the way to put these into numpy seemed to be: rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) rf2 = np.asarray(cursor2.fetchall(), dtype=np.float64) rf3 = np.asarray(cursor3.fetchall(), dtype=np.float64) Then to try and produce a correlation matrix: print (np.corrcoef([rf1, rf2, rf3])) But this is where I get the original error. From looking at column_stack, this doesn't seem like what I want to achieve, or am I looking at this the wrong way somehow?.
column_stack and rowvar=0 should work, but it might be easier to get rid of the extra dimension that the cursor gives you e.g. rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64).ravel() depends on whether you want to work with 2d or 1d data arrays Josef
Thanks again, Rui
________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 2:34:42 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 8:08 AM, Rui DaCosta <ruidc@yahoo.com> wrote:
Thanks for your assistance, I was following this example:
http://www.scipy.org/Numpy_Example_List#head-779283aaa3cc20a3786ad33c2ee1fee... 3
I didn't know, If arrays are 1d, then numpy does the stacking for you., but it doesn't work with column-vectors or 2d arrays.
np.corrcoef([np.arange(3), np.arange(3)], rowvar=1) array([[ 1., 1.], [ 1., 1.]]) np.corrcoef([np.arange(3), np.arange(3)], rowvar=0) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
I intend to use different arrays once I have resolved this problem. Trying your suggestion: ... print (rf1) print (np.corrcoef(rf1, rf1, rowvar=0)) results in: [[-0.00641625] [-0.00498411] [-0.0038015 ]] [[ 1. 1.] [ 1. 1.]]
which should have been a 3x3 matrix
I'm not sure what you want here, you only have three values variance would also be zero, and corrcoef wouldn't be defined if you only have 1 observation per variable
np.corrcoef(np.arange(3)[:,None]) array([[ NaN, NaN, NaN], [ NaN, NaN, NaN], [ NaN, NaN, NaN]])
np.corrcoef(np.column_stack([np.arange(5), np.arange(5)**2, np.arange(5)**3]), rowvar=0) array([[ 1. , 0.9589266 , 0.90588235], [ 0.9589266 , 1. , 0.98713033], [ 0.90588235, 0.98713033, 1. ]])
examples are in docstring for np.cov
as for "stacking rf1 rf1 into a single array first" how would I do this?
np.column_stack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]]) np.column_stack([np.arange(3)[:,None], np.arange(3)]) #doesn't required 2d array([[0, 0], [1, 1], [2, 2]])
np.c_[np.arange(3)[:,None], np.arange(3)[:,None]] array([[0, 0], [1, 1], [2, 2]]) np.hstack([np.arange(3)[:,None], np.arange(3)[:,None]]) array([[0, 0], [1, 1], [2, 2]])
Josef
Regards, Rui ________________________________ From: "josef.pktd@gmail.com" <josef.pktd@gmail.com> To: Discussion of Numerical Python <numpy-discussion@scipy.org> Sent: Wed, September 8, 2010 1:09:46 PM Subject: Re: [Numpy-discussion] ValueError: objects are not aligned
On Wed, Sep 8, 2010 at 5:42 AM, RuiDC <ruidc@yahoo.com> wrote:
I'm getting this error, which must be a simple error in getting the result from the cursor in the right shape, how do I get the cursor into the array as a single dimension?:
import numpy as np import sqlite3 conn = sqlite3.connect("simpledb") cursor1 = conn.execute("select LogReturn from tblReturns limit 3") rf1 = np.asarray(cursor1.fetchall(), dtype=np.float64) print (rf1) print (np.corrcoef([rf1, rf1]))
numpy.corrcoef(x, y=None, rowvar=1, bias=0)
try np.corrcoef(rf1, rf1, rowvar=0) no [ ] or stacking rf1 rf1 into a single array first
Josef
result: [[-0.00641625] [-0.00498411] [-0.0038015 ]] Traceback (most recent call last): File "C:\Documents and Settings\rui\workspace\numpytest\src\numpytest.py", line 38, in <module> print (np.corrcoef([rf1, rf1])) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 2003, in corrcoef c = cov(x, y, rowvar, bias, ddof) File "C:\Python26\lib\site-packages\numpy\lib\function_base.py", line 1953, in cov return (dot(X, X.T.conj()) / fact).squeeze() -- View this message in context:
http://old.nabble.com/ValueError%3A-objects-are-not-aligned-tp29641661p29641... l Sent from the Numpy-discussion mailing list archive at Nabble.com.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
Bruce Southey -
josef.pktd＠gmail.com -
Rui DaCosta -
RuiDC