R: Re: "ValueError: total size of new array must be unchanged" only on Windows
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there. = Olivier Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 * From: * Olivier Delalleau <shish@keba.be>; * To: * Discussion of Numerical Python <numpydiscussion@scipy.org>; * Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
This is my code: matrix="".join(f.readlines()) matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T Il 05/02/2012 17:21, Olivier Delalleau ha scritto:
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.
= Olivier
Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it <mailto:p.zaffino@yahoo.it>> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 *From: * Olivier Delalleau <shish@keba.be <mailto:shish@keba.be>>; *To: * Discussion of Numerical Python <numpydiscussion@scipy.org <mailto:numpydiscussion@scipy.org>>; *Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows *Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Sun, Feb 5, 2012 at 12:39 PM, Paolo <p.zaffino@yahoo.it> wrote:
This is my code:
matrix="".join(f.readlines())
my guess would be, that you have to strip the line endings \n versus \r\n Josef
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T
Il 05/02/2012 17:21, Olivier Delalleau ha scritto:
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.
= Olivier
Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 * From: * Olivier Delalleau <shish@keba.be>; * To: * Discussion of Numerical Python <numpydiscussion@scipy.org>; * Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
How I can do this? Il 05/02/2012 18:47, josef.pktd@gmail.com ha scritto:
On Sun, Feb 5, 2012 at 12:39 PM, Paolo <p.zaffino@yahoo.it <mailto:p.zaffino@yahoo.it>> wrote:
This is my code:
matrix="".join(f.readlines())
my guess would be, that you have to strip the line endings \n versus \r\n
Josef
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T
Il 05/02/2012 17:21, Olivier Delalleau ha scritto:
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.
= Olivier
Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it <mailto:p.zaffino@yahoo.it>> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 *From: * Olivier Delalleau <shish@keba.be <mailto:shish@keba.be>>; *To: * Discussion of Numerical Python <numpydiscussion@scipy.org <mailto:numpydiscussion@scipy.org>>; *Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows *Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org <mailto:NumPyDiscussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Sun, Feb 5, 2012 at 12:52 PM, Paolo <p.zaffino@yahoo.it> wrote:
How I can do this?
I'm not sure without trying, numpy.loadtxt might be the easier choice matrix="".join((i.strip() for i in f.readlines())) I think strip() also removes newlines besides other whitespace otherwise more explicitly matrix="".join((i.strip(f.newlines) for i in f.readlines())) or open the file with mode 'rU' and strip('\n') Josef
Il 05/02/2012 18:47, josef.pktd@gmail.com ha scritto:
On Sun, Feb 5, 2012 at 12:39 PM, Paolo <p.zaffino@yahoo.it> wrote:
This is my code:
matrix="".join(f.readlines())
my guess would be, that you have to strip the line endings \n versus \r\n
Josef
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T
Il 05/02/2012 17:21, Olivier Delalleau ha scritto:
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.
= Olivier
Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 * From: * Olivier Delalleau <shish@keba.be>; * To: * Discussion of Numerical Python <numpydiscussion@scipy.org>; * Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing listNumPyDiscussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Sun, Feb 5, 2012 at 12:06 PM, <josef.pktd@gmail.com> wrote:
On Sun, Feb 5, 2012 at 12:52 PM, Paolo <p.zaffino@yahoo.it> wrote:
How I can do this?
I'm not sure without trying, numpy.loadtxt might be the easier choice
matrix="".join((i.strip() for i in f.readlines()))
I think strip() also removes newlines besides other whitespace otherwise more explicitly matrix="".join((i.strip(f.newlines) for i in f.readlines()))
or open the file with mode 'rU' and strip('\n')
Josef
This code: matrix="".join(f.readlines()) matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T implies that the data in f is binary, because the 'sep' keyword is not used in the call to np.fromstring. If that is the case, you should not use f.readlines() to read the data. Instead, read it as a single string with f.read(). (Or perhaps read the file with a single call to np.fromfile()). Also be sure that the file was opened in binary mode (i.e. f = open(filename, 'rb')). Warren
Il 05/02/2012 18:47, josef.pktd@gmail.com ha scritto:
On Sun, Feb 5, 2012 at 12:39 PM, Paolo <p.zaffino@yahoo.it> wrote:
This is my code:
matrix="".join(f.readlines())
my guess would be, that you have to strip the line endings \n versus \r\n
Josef
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T
Il 05/02/2012 17:21, Olivier Delalleau ha scritto:
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.
= Olivier
Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 * From: * Olivier Delalleau <shish@keba.be>; * To: * Discussion of Numerical Python <numpydiscussion@scipy.org>; * Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing listNumPyDiscussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
I solved using 'rb' instead of 'r' option in the open file task. Thank you very much. Il 05/02/2012 19:13, Warren Weckesser ha scritto:
On Sun, Feb 5, 2012 at 12:06 PM, <josef.pktd@gmail.com <mailto:josef.pktd@gmail.com>> wrote:
On Sun, Feb 5, 2012 at 12:52 PM, Paolo <p.zaffino@yahoo.it <mailto:p.zaffino@yahoo.it>> wrote:
How I can do this?
I'm not sure without trying, numpy.loadtxt might be the easier choice
matrix="".join((i.strip() for i in f.readlines()))
I think strip() also removes newlines besides other whitespace otherwise more explicitly matrix="".join((i.strip(f.newlines) for i in f.readlines()))
or open the file with mode 'rU' and strip('\n')
Josef
This code:
matrix="".join(f.readlines()) matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T
implies that the data in f is binary, because the 'sep' keyword is not used in the call to np.fromstring. If that is the case, you should not use f.readlines() to read the data. Instead, read it as a single string with f.read(). (Or perhaps read the file with a single call to np.fromfile()). Also be sure that the file was opened in binary mode (i.e. f = open(filename, 'rb')).
Warren
Il 05/02/2012 18:47, josef.pktd@gmail.com <mailto:josef.pktd@gmail.com> ha scritto:
On Sun, Feb 5, 2012 at 12:39 PM, Paolo <p.zaffino@yahoo.it <mailto:p.zaffino@yahoo.it>> wrote:
This is my code:
matrix="".join(f.readlines())
my guess would be, that you have to strip the line endings \n versus \r\n
Josef
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T
Il 05/02/2012 17:21, Olivier Delalleau ha scritto:
It means there is some of your code that is not entirely platformindependent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpyrelated. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.
= Olivier
Le 5 février 2012 11:16, Paolo Zaffino <p.zaffino@yahoo.it <mailto:p.zaffino@yahoo.it>> a écrit :
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!
 *From: * Olivier Delalleau <shish@keba.be <mailto:shish@keba.be>>; *To: * Discussion of Numerical Python <numpydiscussion@scipy.org <mailto:numpydiscussion@scipy.org>>; *Subject: * Re: [Numpydiscussion] "ValueError: total size of new array must be unchanged" only on Windows *Sent: * Sun, Feb 5, 2012 3:02:44 PM
It should mean that matrix.size != a * b * c.
= Olivier
Le 5 février 2012 09:32, Paolo <p.zaffino@yahoo.it> a écrit :
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:
matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged
Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org <mailto:NumPyDiscussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org <mailto:NumPyDiscussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org <mailto:NumPyDiscussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org <mailto:NumPyDiscussion@scipy.org> http://mail.scipy.org/mailman/listinfo/numpydiscussion
_______________________________________________ NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Sun, Feb 5, 2012 at 10:41 AM, Paolo <p.zaffino@yahoo.it> wrote:
I solved using 'rb' instead of 'r' option in the open file task.
that would do it, if it's binary data, but you might as well so it "right":
matrix="".join(f.readlines())
readlines is giving you a list of the data, as separated by newline charactors ("\n")  it was broken on Windows, because opening the file in text mode translated Windows newlines ("\r\n") to *nix style ones  opening it in binary fixed that, but why use readlines at all? That's for text  use read. However, even better is to use fromfile(), which creates an array form binary data in a file without puttin git in a string, first:
matrix = np.fromfile(f, dtype=np.int16)
by the way  be careful of endian issues here  if you are moving data among different machines. You could specify the endianness, for instance: dt = numpy.dtype('<i2') for a 16 bit littleendian integer. CHB Chris  Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 5266959 voice 7600 Sand Point Way NE (206) 5266329 fax Seattle, WA 98115 (206) 5266317 main reception Chris.Barker@noaa.gov
participants (6)

Chris Barker

josef.pktd＠gmail.com

Olivier Delalleau

Paolo

Paolo Zaffino

Warren Weckesser