Mailman 3 R: Re: "ValueError: total size of new array must be unchanged" only on Windows - NumPy-Discussion

R: Re: "ValueError: total size of new array must be unchanged" only on Windows

Paolo Zaffino

5 Feb 2012 5 Feb '12

9:46 p.m.

Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

Attachments:

attachment.htm (text/html — 3.5 KB)

Show replies by date

Olivier Delalleau

5 Feb 5 Feb

9:51 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there. -=- Olivier Le 5 février 2012 11:16, Paolo Zaffino a écrit :

...

Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------ * From: * Olivier Delalleau ; * To: * Discussion of Numerical Python ; * Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

...
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Paolo

11:09 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

This is my code: matrix="".join(f.readlines()) matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T Il 05/02/2012 17:21, Olivier Delalleau ha scritto:

...

It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.

-=- Olivier

Le 5 février 2012 11:16, Paolo Zaffino mailto:p.zaffino@yahoo.it> a écrit :

Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------------------------------------------------ *From: * Olivier Delalleau mailto:shish@keba.be>; *To: * Discussion of Numerical Python mailto:numpy-discussion@scipy.org>; *Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows *Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

josef.pktd＠gmail.com

11:17 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

On Sun, Feb 5, 2012 at 12:39 PM, Paolo wrote:

...

This is my code:

matrix="".join(f.readlines())

my guess would be, that you have to strip the line endings \n versus \r\n Josef

...

matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T

Il 05/02/2012 17:21, Olivier Delalleau ha scritto:

It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.

-=- Olivier

Le 5 février 2012 11:16, Paolo Zaffino a écrit :

...
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------ * From: * Olivier Delalleau ; * To: * Discussion of Numerical Python ; * Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

...
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Paolo

11:22 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

How I can do this? Il 05/02/2012 18:47, josef.pktd@gmail.com ha scritto:

...

On Sun, Feb 5, 2012 at 12:39 PM, Paolo mailto:p.zaffino@yahoo.it> wrote:

This is my code:

matrix="".join(f.readlines())

my guess would be, that you have to strip the line endings \n versus \r\n

Josef

matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T

Il 05/02/2012 17:21, Olivier Delalleau ha scritto:

...
It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.

-=- Olivier

Le 5 février 2012 11:16, Paolo Zaffino mailto:p.zaffino@yahoo.it> a écrit :

Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------------------------------------------------ *From: * Olivier Delalleau mailto:shish@keba.be>; *To: * Discussion of Numerical Python mailto:numpy-discussion@scipy.org>; *Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows *Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

josef.pktd＠gmail.com

11:36 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

On Sun, Feb 5, 2012 at 12:52 PM, Paolo wrote:

...

How I can do this?

I'm not sure without trying, numpy.loadtxt might be the easier choice matrix="".join((i.strip() for i in f.readlines())) I think strip() also removes newlines besides other whitespace otherwise more explicitly matrix="".join((i.strip(f.newlines) for i in f.readlines())) or open the file with mode 'rU' and strip('\n') Josef

...

Il 05/02/2012 18:47, josef.pktd@gmail.com ha scritto:

On Sun, Feb 5, 2012 at 12:39 PM, Paolo wrote:

...
This is my code:

matrix="".join(f.readlines())

my guess would be, that you have to strip the line endings \n versus \r\n

Josef

...
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T

Il 05/02/2012 17:21, Olivier Delalleau ha scritto:

It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.

-=- Olivier

Le 5 février 2012 11:16, Paolo Zaffino a écrit :

...
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------ * From: * Olivier Delalleau ; * To: * Discussion of Numerical Python ; * Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

...
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Warren Weckesser

11:43 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

On Sun, Feb 5, 2012 at 12:06 PM, wrote:

...

On Sun, Feb 5, 2012 at 12:52 PM, Paolo wrote:

...
How I can do this?

I'm not sure without trying, numpy.loadtxt might be the easier choice

matrix="".join((i.strip() for i in f.readlines()))

I think strip() also removes newlines besides other whitespace otherwise more explicitly matrix="".join((i.strip(f.newlines) for i in f.readlines()))

or open the file with mode 'rU' and strip('\n')

Josef

This code: matrix="".join(f.readlines()) matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T implies that the data in f is binary, because the 'sep' keyword is not used in the call to np.fromstring. If that is the case, you should not use f.readlines() to read the data. Instead, read it as a single string with f.read(). (Or perhaps read the file with a single call to np.fromfile()). Also be sure that the file was opened in binary mode (i.e. f = open(filename, 'rb')). Warren

...

...
Il 05/02/2012 18:47, josef.pktd@gmail.com ha scritto:

On Sun, Feb 5, 2012 at 12:39 PM, Paolo wrote:

...
This is my code:

matrix="".join(f.readlines())

my guess would be, that you have to strip the line endings \n versus \r\n

Josef

...
matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T

Il 05/02/2012 17:21, Olivier Delalleau ha scritto:

It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.

-=- Olivier

Le 5 février 2012 11:16, Paolo Zaffino a écrit :

...
Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------ * From: * Olivier Delalleau ; * To: * Discussion of Numerical Python ; * Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows * Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

...
Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing listNumPy-Discussion@scipy.orghttp://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Paolo

6 Feb 6 Feb

12:11 a.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

I solved using 'rb' instead of 'r' option in the open file task. Thank you very much. Il 05/02/2012 19:13, Warren Weckesser ha scritto:

...

On Sun, Feb 5, 2012 at 12:06 PM, mailto:josef.pktd@gmail.com> wrote:

On Sun, Feb 5, 2012 at 12:52 PM, Paolo mailto:p.zaffino@yahoo.it> wrote:

How I can do this?

I'm not sure without trying, numpy.loadtxt might be the easier choice

matrix="".join((i.strip() for i in f.readlines()))

I think strip() also removes newlines besides other whitespace otherwise more explicitly matrix="".join((i.strip(f.newlines) for i in f.readlines()))

or open the file with mode 'rU' and strip('\n')

Josef

This code:

matrix="".join(f.readlines()) matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T

implies that the data in f is binary, because the 'sep' keyword is not used in the call to np.fromstring. If that is the case, you should not use f.readlines() to read the data. Instead, read it as a single string with f.read(). (Or perhaps read the file with a single call to np.fromfile()). Also be sure that the file was opened in binary mode (i.e. f = open(filename, 'rb')).

Warren

Il 05/02/2012 18:47, josef.pktd@gmail.com mailto:josef.pktd@gmail.com ha scritto:

...
On Sun, Feb 5, 2012 at 12:39 PM, Paolo mailto:p.zaffino@yahoo.it> wrote:

This is my code:

matrix="".join(f.readlines())

my guess would be, that you have to strip the line endings \n versus \r\n

Josef

matrix=np.fromstring(matrix, dtype=np.int16) matrix=matrix.reshape(siz[2],siz[1],siz[0]).T

Il 05/02/2012 17:21, Olivier Delalleau ha scritto:

...
It means there is some of your code that is not entirely platform-independent. It's not possible to tell you which part because you didn't provide your code. The problem may not even be numpy-related. So you should first look at the current shape of 'matrix', and what are the values of a, b and c, then see where the discrepancy is, and work from there.

-=- Olivier

Le 5 février 2012 11:16, Paolo Zaffino mailto:p.zaffino@yahoo.it> a écrit :

Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

------------------------------------------------------------------------ *From: * Olivier Delalleau mailto:shish@keba.be>; *To: * Discussion of Numerical Python mailto:numpy-discussion@scipy.org>; *Subject: * Re: [Numpy-discussion] "ValueError: total size of new array must be unchanged" only on Windows *Sent: * Sun, Feb 5, 2012 3:02:44 PM

It should mean that matrix.size != a * b * c.

-=- Olivier

Le 5 février 2012 09:32, Paolo a écrit :

Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error:

matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged

Why? Do anyone have an idea about this? Thank you very much. _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org mailto:NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Chris Barker

10:11 p.m.

New subject: "ValueError: total size of new array must be unchanged" only on Windows

On Sun, Feb 5, 2012 at 10:41 AM, Paolo wrote:

...

I solved using 'rb' instead of 'r' option in the open file task.

that would do it, if it's binary data, but you might as well so it "right":

...

matrix="".join(f.readlines())

readlines is giving you a list of the data, as separated by newline charactors ("\n") -- it was broken on Windows, because opening the file in text mode translated Windows newlines ("\r\n") to *nix style ones -- opening it in binary fixed that, but why use readlines at all? That's for text -- use read. However, even better is to use fromfile(), which creates an array form binary data in a file without puttin git in a string, first:

...

matrix = np.fromfile(f, dtype=np.int16)

by the way -- be careful of endian issues here -- if you are moving data among different machines. You could specify the endian-ness, for instance: dt = numpy.dtype('

4461

Age (days ago)

4462

Last active (days ago)

List overview

Download

8 comments

6 participants

participants (6)

Chris Barker
josef.pktd＠gmail.com
Olivier Delalleau
Paolo
Paolo Zaffino
Warren Weckesser

R: Re: "ValueError: total size of new array must be unchanged" only on Windows

Olivier Delalleau

tags

participants (6)