<div dir="ltr">This looks familiar, I ran into this a few years ago, and if I recall correctly, there is an option to loadmat to reduce array dimensions appropriately. There is a "squeeze_me" option (unfortunately named... should probably be deprecated in favor of "squeeze") which I think does this.<div><br></div><div><a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html">https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html</a><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer <span dir="ltr"><<a href="mailto:gregor.thalhammer@gmail.com" target="_blank">gregor.thalhammer@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><div><div class="h5"><blockquote type="cite"><div>Am 22.02.2017 um 12:02 schrieb Propadovic Nenad <<a href="mailto:npropadovic@gmail.com" target="_blank">npropadovic@gmail.com</a>>:</div><br class="m_8755324769002528865Apple-interchange-newline"><div><div dir="ltr">Hello,<br><br>bear with me for the long post that follows: it took me more than a week to get this far, and I tried to compress all the relevant information into the post.<br><br>There seems to be a bug in scipy.io.loadmat; I'll present it by a short piece of code and it's output.<br><br>I create file x.mat with the following:<br><br>import <a href="http://scipy.io/" target="_blank">scipy.io</a><br><br>d = {'CanData':<br> {<br> 'msg': {<br> 'RPDO2': {<br> 'timest': [0.0, 0.0039679999899817631, 0.0079779999941820279],<br> 'sig': {<br> 'VelAct': {<br> 'Values': [-0.050000000000000003, -0.10000000000000001, 0.29999999999999999, ],<br> 'Name': 'VelAct'<br> },<br> 'PosAct': {<br> 'Values': [61.960000000000001, 61.960000000000001, 61.960000000000001, ],<br> 'Name': 'PosAct'<br> }<br> }<br> }<br> }<br> }<br>}<br>scipy.io.savemat("x.mat", d)<br><br>Matlab is happy with the file and handles it the way I expect.<br><br>When I read in the data stored in the file and print it out:<br><br>import <a href="http://scipy.io/" target="_blank">scipy.io</a><br>y = scipy.io.loadmat("x.mat")<br># print y<br>cd = y['CanData']<br>msg = cd['msg']<br>print msg<br>print msg.dtype<br>print msg.dtype.names<br><br>The output is:<br>>C:\Anaconda2\pythonw -u "test1.py"<br>[[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), array([u'PosAct'],<br> dtype='<U6'))]], [[(array([[-0.05, -0.1 , 0.3 ]]), array([u'VelAct'],<br> dtype='<U6'))]])]],<br> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , 0.003968, 0.007978]]))]],)]],<br> dtype=[('RPDO2', 'O')])]]<br>object<br>None<br><br>Now I've read the manual, and as I see it I have no way for me to access the deeper layers of data I just put in the file x.mat, although they are obviously right there in the data read in. Access via msg['RPDO2'] gives:<br>IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices.<br></div></div></blockquote><div><br></div></div></div><div>For historic reasons, in Matlab everything is at least a 2D array, even scalars. By sprinkling some [0,0] in your code you should get what you want, e.g.</div><div><br></div><div>msg[0,0]['RPDO2'][0,0]['<wbr>timest'][0,0]</div><div><span class=""><pre style="box-sizing:border-box;overflow:auto;font-size:14px;padding:0px;margin-top:0px;margin-bottom:0px;line-height:inherit;word-break:break-all;word-wrap:break-word;border:0px;border-top-left-radius:0px;border-top-right-radius:0px;border-bottom-right-radius:0px;border-bottom-left-radius:0px;white-space:pre-wrap;vertical-align:baseline">array([[ 0. , 0.003968, 0.007978]])</pre><div><br></div></span><div>Gregor</div><div><br></div></div><br><blockquote type="cite"><div><div><div class="h5"><div dir="ltr"><br>If I use parameter squeeze_me=True:<br><br>scipy.io.savemat("x.mat", d)<br>y = scipy.io.loadmat("x.mat", squeeze_me=True)<br># print y<br>cd = y['CanData']<br>msg = cd['msg']<br>print msg<br>print msg.dtype<br>print msg.dtype.names<br><br>I get output:<br>>C:\Anaconda2\pythonw -u "test1.py"<br>((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , 0.3 ]), u'VelAct')),<br> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , 0.003968, 0.007978])),)<br>object<br>None<br>>Exit code: 0<br><br>All well, but the name 'RPDO2' disapeared from the data!<br><br>Now I need this information; in future I won't handle what's put into x.mat, so I need a way to access through the data all the way down (and handle the variations that will come).<br><br>I have found a workaround at:<br><a href="http://stackoverflow.com/questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/" target="_blank">http://stackoverflow.com/<wbr>questions/7008608/scipy-io-<wbr>loadmat-nested-structures-i-e-<wbr>dictionaries/</a><br><br>The problem is, the workaround uses struct_as_record=False in loadmat, and which boils down to using scipy.io.matlab.mio5_params.<wbr>mat_struct, and when you read the docstring of class mat_struct, it says:<br><br>'''<br>...<br>We deprecate this method of holding struct information, and will<br>soon remove it, in favor of the recarray method (see loadmat<br>docstring)<br>'''<br>So my questions:<br>1) Did I miss something? Is there a way to access the data in 'RPDO2' by using this name, without using parameter struct_as_record=False in loadmat?<br>2) If not, where do I file a bug? The workaround is five years old, so the issue seems to be in scipy for ages...<br><br>(For the records, I use scipy within Anaconda2 1.4.1, under Windows, but this does not seem to matter).<br><br>Thanks a lot for the answers, in advance.<br><br>Nenad<br><br><br></div></div></div>
______________________________<wbr>_________________<br>SciPy-User mailing list<br><a href="mailto:SciPy-User@python.org" target="_blank">SciPy-User@python.org</a><br><a href="https://mail.python.org/mailman/listinfo/scipy-user" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scipy-user</a><br></div></blockquote></div><br></div><br>______________________________<wbr>_________________<br>
SciPy-User mailing list<br>
<a href="mailto:SciPy-User@python.org">SciPy-User@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scipy-user" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scipy-user</a><br>
<br></blockquote></div><br></div>