Execution time difference between 2.7 and 3.2 using numpy
Without much detailed knowledge of the topic, I would expect both versions to give very similar timing, as it is essentially a call to ATLAS function, not much is done in Python. Given this, maybe the difference is in ATLAS itself. How have you installed it? When you compile ATLAS, it will do some machine-specific optimisation, but if you have installed a binary chances are that your version is optimised for a machine quite different from yours. So, two different installations could have been compiled in different machines and so one is more suited for your machine. If you want to be sure, I would try to compile ATLAS (this may be difficult) or check the same on a very different machine (like an AMD processor, different architecture...). Just for reference, on Linux Python 2.7 64 bits can deal with these matrices easily. %timeit mat=np.random.random((6143,6143)); matinv= np.linalg.inv(mat); res = np.dot(mat, matinv); diff= res-np.eye(6143); print np.sum(np.abs(diff)) 2.41799631031e-05 1.13955868701e-05 3.64338191541e-05 1.13484781021e-05 1 loops, best of 3: 156 s per loop Intel i5, 4 GB of RAM and SSD. ATLAS installed from Fedora repository (I don't run heavy stuff on this computer). On 20 March 2013 14:46, Colin J. Williams <cjw@ncf.ca> wrote:
I have a small program which builds random matrices for increasing matrix orders, inverts the matrix and checks the precision of the product. At some point, one would expect operations to fail, when the memory capacity is exceeded. In both Python 2.7 and 3.2 matrices of order 3,071 area handled, but not 6,143.
Using wall-clock times, with win32, Python 3.2 is slower than Python 2.7. The profiler indicates a problem in the solver.
Done on a Pentium, with 2.7 GHz processor, 2 GB of RAM and 221 GB of free disk space. Both Python 3.2.3 and Python 2.7.3 use numpy 1.6.2.
The results are show below.
Colin W.
aaaa_ssss 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.097 Time elapsed (seconds)= 0.004143 order= 5 measure ofimprecision= 2.207 Time elapsed (seconds)= 0.001514 order= 11 measure ofimprecision= 2.372 Time elapsed (seconds)= 0.001455 order= 23 measure ofimprecision= 3.318 Time elapsed (seconds)= 0.001608 order= 47 measure ofimprecision= 4.257 Time elapsed (seconds)= 0.002339 order= 95 measure ofimprecision= 4.986 Time elapsed (seconds)= 0.005747 order= 191 measure ofimprecision= 5.788 Time elapsed (seconds)= 0.029974 order= 383 measure ofimprecision= 6.765 Time elapsed (seconds)= 0.145339 order= 767 measure ofimprecision= 7.909 Time elapsed (seconds)= 0.841142 order= 1535 measure ofimprecision= 8.532 Time elapsed (seconds)= 5.793630 order= 3071 measure ofimprecision= 9.774 Time elapsed (seconds)= 39.559540 order= 6143 Process terminated by a MemoryError
Above: 2.7.3 Below: Python 3.2.3
bbb_bbb 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.000 Time elapsed (seconds)= 0.113930 order= 5 measure ofimprecision= 1.807 Time elapsed (seconds)= 0.001373 order= 11 measure ofimprecision= 2.395 Time elapsed (seconds)= 0.001468 order= 23 measure ofimprecision= 3.073 Time elapsed (seconds)= 0.001609 order= 47 measure ofimprecision= 5.642 Time elapsed (seconds)= 0.002687 order= 95 measure ofimprecision= 5.745 Time elapsed (seconds)= 0.013510 order= 191 measure ofimprecision= 5.866 Time elapsed (seconds)= 0.061560 order= 383 measure ofimprecision= 7.129 Time elapsed (seconds)= 0.418490 order= 767 measure ofimprecision= 8.240 Time elapsed (seconds)= 3.815713 order= 1535 measure ofimprecision= 8.735 Time elapsed (seconds)= 27.877270 order= 3071 measure ofimprecision= 9.996 Time elapsed (seconds)=212.545610 order= 6143 Process terminated by a MemoryError
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, Could also be that they are linked to different libs such as atlas and standart Blas. What is the output of numpy.show_config() in the two different python versions. Jens On Wed, Mar 20, 2013 at 2:14 PM, Daπid <davidmenhur@gmail.com> wrote:
Without much detailed knowledge of the topic, I would expect both versions to give very similar timing, as it is essentially a call to ATLAS function, not much is done in Python.
Given this, maybe the difference is in ATLAS itself. How have you installed it? When you compile ATLAS, it will do some machine-specific optimisation, but if you have installed a binary chances are that your version is optimised for a machine quite different from yours. So, two different installations could have been compiled in different machines and so one is more suited for your machine. If you want to be sure, I would try to compile ATLAS (this may be difficult) or check the same on a very different machine (like an AMD processor, different architecture...).
Just for reference, on Linux Python 2.7 64 bits can deal with these matrices easily.
%timeit mat=np.random.random((6143,6143)); matinv= np.linalg.inv(mat); res = np.dot(mat, matinv); diff= res-np.eye(6143); print np.sum(np.abs(diff)) 2.41799631031e-05 1.13955868701e-05 3.64338191541e-05 1.13484781021e-05 1 loops, best of 3: 156 s per loop
Intel i5, 4 GB of RAM and SSD. ATLAS installed from Fedora repository (I don't run heavy stuff on this computer).
On 20 March 2013 14:46, Colin J. Williams <cjw@ncf.ca> wrote:
I have a small program which builds random matrices for increasing matrix orders, inverts the matrix and checks the precision of the product. At some point, one would expect operations to fail, when the memory capacity is exceeded. In both Python 2.7 and 3.2 matrices of order 3,071 area handled, but not 6,143.
Using wall-clock times, with win32, Python 3.2 is slower than Python 2.7. The profiler indicates a problem in the solver.
Done on a Pentium, with 2.7 GHz processor, 2 GB of RAM and 221 GB of free disk space. Both Python 3.2.3 and Python 2.7.3 use numpy 1.6.2.
The results are show below.
Colin W.
aaaa_ssss 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.097 Time elapsed (seconds)= 0.004143 order= 5 measure ofimprecision= 2.207 Time elapsed (seconds)= 0.001514 order= 11 measure ofimprecision= 2.372 Time elapsed (seconds)= 0.001455 order= 23 measure ofimprecision= 3.318 Time elapsed (seconds)= 0.001608 order= 47 measure ofimprecision= 4.257 Time elapsed (seconds)= 0.002339 order= 95 measure ofimprecision= 4.986 Time elapsed (seconds)= 0.005747 order= 191 measure ofimprecision= 5.788 Time elapsed (seconds)= 0.029974 order= 383 measure ofimprecision= 6.765 Time elapsed (seconds)= 0.145339 order= 767 measure ofimprecision= 7.909 Time elapsed (seconds)= 0.841142 order= 1535 measure ofimprecision= 8.532 Time elapsed (seconds)= 5.793630 order= 3071 measure ofimprecision= 9.774 Time elapsed (seconds)= 39.559540 order= 6143 Process terminated by a MemoryError
Above: 2.7.3 Below: Python 3.2.3
bbb_bbb 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.000 Time elapsed (seconds)= 0.113930 order= 5 measure ofimprecision= 1.807 Time elapsed (seconds)= 0.001373 order= 11 measure ofimprecision= 2.395 Time elapsed (seconds)= 0.001468 order= 23 measure ofimprecision= 3.073 Time elapsed (seconds)= 0.001609 order= 47 measure ofimprecision= 5.642 Time elapsed (seconds)= 0.002687 order= 95 measure ofimprecision= 5.745 Time elapsed (seconds)= 0.013510 order= 191 measure ofimprecision= 5.866 Time elapsed (seconds)= 0.061560 order= 383 measure ofimprecision= 7.129 Time elapsed (seconds)= 0.418490 order= 767 measure ofimprecision= 8.240 Time elapsed (seconds)= 3.815713 order= 1535 measure ofimprecision= 8.735 Time elapsed (seconds)= 27.877270 order= 3071 measure ofimprecision= 9.996 Time elapsed (seconds)=212.545610 order= 6143 Process terminated by a MemoryError
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, win32 do not mean it is a 32 bits windows. sys.platform always return win32 on 32bits and 64 bits windows even for python 64 bits. But that is a good question, is your python 32 or 64 bits? Fred On Wed, Mar 20, 2013 at 10:14 AM, Daπid <davidmenhur@gmail.com> wrote:
Without much detailed knowledge of the topic, I would expect both versions to give very similar timing, as it is essentially a call to ATLAS function, not much is done in Python.
Given this, maybe the difference is in ATLAS itself. How have you installed it? When you compile ATLAS, it will do some machine-specific optimisation, but if you have installed a binary chances are that your version is optimised for a machine quite different from yours. So, two different installations could have been compiled in different machines and so one is more suited for your machine. If you want to be sure, I would try to compile ATLAS (this may be difficult) or check the same on a very different machine (like an AMD processor, different architecture...).
Just for reference, on Linux Python 2.7 64 bits can deal with these matrices easily.
%timeit mat=np.random.random((6143,6143)); matinv= np.linalg.inv(mat); res = np.dot(mat, matinv); diff= res-np.eye(6143); print np.sum(np.abs(diff)) 2.41799631031e-05 1.13955868701e-05 3.64338191541e-05 1.13484781021e-05 1 loops, best of 3: 156 s per loop
Intel i5, 4 GB of RAM and SSD. ATLAS installed from Fedora repository (I don't run heavy stuff on this computer).
On 20 March 2013 14:46, Colin J. Williams <cjw@ncf.ca> wrote:
I have a small program which builds random matrices for increasing matrix orders, inverts the matrix and checks the precision of the product. At some point, one would expect operations to fail, when the memory capacity is exceeded. In both Python 2.7 and 3.2 matrices of order 3,071 area handled, but not 6,143.
Using wall-clock times, with win32, Python 3.2 is slower than Python 2.7. The profiler indicates a problem in the solver.
Done on a Pentium, with 2.7 GHz processor, 2 GB of RAM and 221 GB of free disk space. Both Python 3.2.3 and Python 2.7.3 use numpy 1.6.2.
The results are show below.
Colin W.
aaaa_ssss 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.097 Time elapsed (seconds)= 0.004143 order= 5 measure ofimprecision= 2.207 Time elapsed (seconds)= 0.001514 order= 11 measure ofimprecision= 2.372 Time elapsed (seconds)= 0.001455 order= 23 measure ofimprecision= 3.318 Time elapsed (seconds)= 0.001608 order= 47 measure ofimprecision= 4.257 Time elapsed (seconds)= 0.002339 order= 95 measure ofimprecision= 4.986 Time elapsed (seconds)= 0.005747 order= 191 measure ofimprecision= 5.788 Time elapsed (seconds)= 0.029974 order= 383 measure ofimprecision= 6.765 Time elapsed (seconds)= 0.145339 order= 767 measure ofimprecision= 7.909 Time elapsed (seconds)= 0.841142 order= 1535 measure ofimprecision= 8.532 Time elapsed (seconds)= 5.793630 order= 3071 measure ofimprecision= 9.774 Time elapsed (seconds)= 39.559540 order= 6143 Process terminated by a MemoryError
Above: 2.7.3 Below: Python 3.2.3
bbb_bbb 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.000 Time elapsed (seconds)= 0.113930 order= 5 measure ofimprecision= 1.807 Time elapsed (seconds)= 0.001373 order= 11 measure ofimprecision= 2.395 Time elapsed (seconds)= 0.001468 order= 23 measure ofimprecision= 3.073 Time elapsed (seconds)= 0.001609 order= 47 measure ofimprecision= 5.642 Time elapsed (seconds)= 0.002687 order= 95 measure ofimprecision= 5.745 Time elapsed (seconds)= 0.013510 order= 191 measure ofimprecision= 5.866 Time elapsed (seconds)= 0.061560 order= 383 measure ofimprecision= 7.129 Time elapsed (seconds)= 0.418490 order= 767 measure ofimprecision= 8.240 Time elapsed (seconds)= 3.815713 order= 1535 measure ofimprecision= 8.735 Time elapsed (seconds)= 27.877270 order= 3071 measure ofimprecision= 9.996 Time elapsed (seconds)=212.545610 order= 6143 Process terminated by a MemoryError
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
Hi,
win32 do not mean it is a 32 bits windows. sys.platform always return win32 on 32bits and 64 bits windows even for python 64 bits.
But that is a good question, is your python 32 or 64 bits? 32 bits.
Colin W.
Fred
On Wed, Mar 20, 2013 at 10:14 AM, Daπid <davidmenhur@gmail.com> wrote:
Without much detailed knowledge of the topic, I would expect both versions to give very similar timing, as it is essentially a call to ATLAS function, not much is done in Python.
Given this, maybe the difference is in ATLAS itself. How have you installed it? When you compile ATLAS, it will do some machine-specific optimisation, but if you have installed a binary chances are that your version is optimised for a machine quite different from yours. So, two different installations could have been compiled in different machines and so one is more suited for your machine. If you want to be sure, I would try to compile ATLAS (this may be difficult) or check the same on a very different machine (like an AMD processor, different architecture...).
Just for reference, on Linux Python 2.7 64 bits can deal with these matrices easily.
%timeit mat=np.random.random((6143,6143)); matinv= np.linalg.inv(mat); res = np.dot(mat, matinv); diff= res-np.eye(6143); print np.sum(np.abs(diff)) 2.41799631031e-05 1.13955868701e-05 3.64338191541e-05 1.13484781021e-05 1 loops, best of 3: 156 s per loop
Intel i5, 4 GB of RAM and SSD. ATLAS installed from Fedora repository (I don't run heavy stuff on this computer).
On 20 March 2013 14:46, Colin J. Williams <cjw@ncf.ca> wrote:
I have a small program which builds random matrices for increasing matrix orders, inverts the matrix and checks the precision of the product. At some point, one would expect operations to fail, when the memory capacity is exceeded. In both Python 2.7 and 3.2 matrices of order 3,071 area handled, but not 6,143.
Using wall-clock times, with win32, Python 3.2 is slower than Python 2.7. The profiler indicates a problem in the solver.
Done on a Pentium, with 2.7 GHz processor, 2 GB of RAM and 221 GB of free disk space. Both Python 3.2.3 and Python 2.7.3 use numpy 1.6.2.
The results are show below.
Colin W.
aaaa_ssss 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.097 Time elapsed (seconds)= 0.004143 order= 5 measure ofimprecision= 2.207 Time elapsed (seconds)= 0.001514 order= 11 measure ofimprecision= 2.372 Time elapsed (seconds)= 0.001455 order= 23 measure ofimprecision= 3.318 Time elapsed (seconds)= 0.001608 order= 47 measure ofimprecision= 4.257 Time elapsed (seconds)= 0.002339 order= 95 measure ofimprecision= 4.986 Time elapsed (seconds)= 0.005747 order= 191 measure ofimprecision= 5.788 Time elapsed (seconds)= 0.029974 order= 383 measure ofimprecision= 6.765 Time elapsed (seconds)= 0.145339 order= 767 measure ofimprecision= 7.909 Time elapsed (seconds)= 0.841142 order= 1535 measure ofimprecision= 8.532 Time elapsed (seconds)= 5.793630 order= 3071 measure ofimprecision= 9.774 Time elapsed (seconds)= 39.559540 order= 6143 Process terminated by a MemoryError
Above: 2.7.3 Below: Python 3.2.3
bbb_bbb 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.000 Time elapsed (seconds)= 0.113930 order= 5 measure ofimprecision= 1.807 Time elapsed (seconds)= 0.001373 order= 11 measure ofimprecision= 2.395 Time elapsed (seconds)= 0.001468 order= 23 measure ofimprecision= 3.073 Time elapsed (seconds)= 0.001609 order= 47 measure ofimprecision= 5.642 Time elapsed (seconds)= 0.002687 order= 95 measure ofimprecision= 5.745 Time elapsed (seconds)= 0.013510 order= 191 measure ofimprecision= 5.866 Time elapsed (seconds)= 0.061560 order= 383 measure ofimprecision= 7.129 Time elapsed (seconds)= 0.418490 order= 767 measure ofimprecision= 8.240 Time elapsed (seconds)= 3.815713 order= 1535 measure ofimprecision= 8.735 Time elapsed (seconds)= 27.877270 order= 3071 measure ofimprecision= 9.996 Time elapsed (seconds)=212.545610 order= 6143 Process terminated by a MemoryError
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The python3 version is compiled without any optimised library and is falling back on a slow version. Where did you get this installation from? Jens On Wed, Mar 20, 2013 at 3:01 PM, Colin J. Williams <cjwilliams43@gmail.com>wrote:
On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
Hi,
win32 do not mean it is a 32 bits windows. sys.platform always return win32 on 32bits and 64 bits windows even for python 64 bits.
But that is a good question, is your python 32 or 64 bits? 32 bits.
Colin W.
Fred
On Wed, Mar 20, 2013 at 10:14 AM, Daπid <davidmenhur@gmail.com> wrote:
Without much detailed knowledge of the topic, I would expect both versions to give very similar timing, as it is essentially a call to ATLAS function, not much is done in Python.
Given this, maybe the difference is in ATLAS itself. How have you installed it? When you compile ATLAS, it will do some machine-specific optimisation, but if you have installed a binary chances are that your version is optimised for a machine quite different from yours. So, two different installations could have been compiled in different machines and so one is more suited for your machine. If you want to be sure, I would try to compile ATLAS (this may be difficult) or check the same on a very different machine (like an AMD processor, different architecture...).
Just for reference, on Linux Python 2.7 64 bits can deal with these matrices easily.
%timeit mat=np.random.random((6143,6143)); matinv= np.linalg.inv(mat); res = np.dot(mat, matinv); diff= res-np.eye(6143); print np.sum(np.abs(diff)) 2.41799631031e-05 1.13955868701e-05 3.64338191541e-05 1.13484781021e-05 1 loops, best of 3: 156 s per loop
Intel i5, 4 GB of RAM and SSD. ATLAS installed from Fedora repository (I don't run heavy stuff on this computer).
On 20 March 2013 14:46, Colin J. Williams <cjw@ncf.ca> wrote:
I have a small program which builds random matrices for increasing
matrix
orders, inverts the matrix and checks the precision of the product. At some point, one would expect operations to fail, when the memory capacity is exceeded. In both Python 2.7 and 3.2 matrices of order 3,071 area handled, but not 6,143.
Using wall-clock times, with win32, Python 3.2 is slower than Python 2.7. The profiler indicates a problem in the solver.
Done on a Pentium, with 2.7 GHz processor, 2 GB of RAM and 221 GB of free disk space. Both Python 3.2.3 and Python 2.7.3 use numpy 1.6.2.
The results are show below.
Colin W.
aaaa_ssss 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.097 Time elapsed (seconds)= 0.004143 order= 5 measure ofimprecision= 2.207 Time elapsed (seconds)= 0.001514 order= 11 measure ofimprecision= 2.372 Time elapsed (seconds)= 0.001455 order= 23 measure ofimprecision= 3.318 Time elapsed (seconds)= 0.001608 order= 47 measure ofimprecision= 4.257 Time elapsed (seconds)= 0.002339 order= 95 measure ofimprecision= 4.986 Time elapsed (seconds)= 0.005747 order= 191 measure ofimprecision= 5.788 Time elapsed (seconds)= 0.029974 order= 383 measure ofimprecision= 6.765 Time elapsed (seconds)= 0.145339 order= 767 measure ofimprecision= 7.909 Time elapsed (seconds)= 0.841142 order= 1535 measure ofimprecision= 8.532 Time elapsed (seconds)= 5.793630 order= 3071 measure ofimprecision= 9.774 Time elapsed (seconds)= 39.559540 order= 6143 Process terminated by a MemoryError
Above: 2.7.3 Below: Python 3.2.3
bbb_bbb 3.2.3 (default, Apr 11 2012, 07:15:24) [MSC v.1500 32 bit (Intel)] order= 2 measure ofimprecision= 0.000 Time elapsed (seconds)= 0.113930 order= 5 measure ofimprecision= 1.807 Time elapsed (seconds)= 0.001373 order= 11 measure ofimprecision= 2.395 Time elapsed (seconds)= 0.001468 order= 23 measure ofimprecision= 3.073 Time elapsed (seconds)= 0.001609 order= 47 measure ofimprecision= 5.642 Time elapsed (seconds)= 0.002687 order= 95 measure ofimprecision= 5.745 Time elapsed (seconds)= 0.013510 order= 191 measure ofimprecision= 5.866 Time elapsed (seconds)= 0.061560 order= 383 measure ofimprecision= 7.129 Time elapsed (seconds)= 0.418490 order= 767 measure ofimprecision= 8.240 Time elapsed (seconds)= 3.815713 order= 1535 measure ofimprecision= 8.735 Time elapsed (seconds)= 27.877270 order= 3071 measure ofimprecision= 9.996 Time elapsed (seconds)=212.545610 order= 6143 Process terminated by a MemoryError
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Wed, Mar 20, 2013 at 11:01 AM, Colin J. Williams <cjwilliams43@gmail.com> wrote:
On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
Hi,
win32 do not mean it is a 32 bits windows. sys.platform always return win32 on 32bits and 64 bits windows even for python 64 bits.
But that is a good question, is your python 32 or 64 bits?
32 bits.
That explain why you have memory problem but not other people with 64 bits version. So if you want to work with bigger input, change to a python 64 bits. Fred
On Fri, Mar 22, 2013 at 2:39 PM, Colin J. Williams <cjwilliams43@gmail.com> wrote:
I have updated to numpy 1.7.0 for each of the Pythons 2.7.3, 3.2.3 and 3.3.0. ... The tests, which are available here(http://web.ncf.ca/cjw/FP%20Summary%20over%20273-323-330.txt), show that 3.2 is slower, but not to the same degree reported before.
Have posted your test code anywhere? Anyway, depending on how you did your timings, that looks to me like 3.* is a bit faster with small data, and pretty much within measurement error for the large datasets. And if the large ones are doing things with really big arrays (I'm assuming pretty big, as you're getting close to 32 bit memory limits...), then it's really hard to imagine how python version could make a noticeable difference -- the real work would be in the numpy code, and that's exactly the same on all python versions. If you are using BLAS or LAPACK stuff, then there might be some differences with the different builds, though I wouldn't expect so if you ar getting them from the same source. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Fri, Mar 22, 2013 at 10:39 PM, Colin J. Williams <cjwilliams43@gmail.com>wrote:
On 20/03/2013 11:12 AM, Frédéric Bastien wrote:
On Wed, Mar 20, 2013 at 11:01 AM, Colin J. Williams<cjwilliams43@gmail.com> <cjwilliams43@gmail.com> wrote:
On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
Hi,
win32 do not mean it is a 32 bits windows. sys.platform always return win32 on 32bits and 64 bits windows even for python 64 bits.
But that is a good question, is your python 32 or 64 bits?
32 bits.
That explain why you have memory problem but not other people with 64 bits version. So if you want to work with bigger input, change to a python 64 bits.
Fred
Thanks to the people who responded to my report that numpy, with Python 3.2 was significantly slower than with Python 2.7.
I have updated to numpy 1.7.0 for each of the Pythons 2.7.3, 3.2.3 and 3.3.0.
The Pythons came from python.org and the Numpys from PyPi. The SciPy site still points to Source Forge, I gathered from the responses that Source Forge is no longer recommended for downloads.
That's not the case. The official binaries for NumPy and SciPy are on SourceForge. The Windows installers on PyPI are there to make easy_install work, but they're likely slower than the SF installers (no SSE2/SSE3 instructions). Ralf
On Sat, Mar 23, 2013 at 12:21 PM, Ralf Gommers <ralf.gommers@gmail.com>wrote:
That's not the case. The official binaries for NumPy and SciPy are on SourceForge. The Windows installers on PyPI are there to make easy_install work, but they're likely slower than the SF installers (no SSE2/SSE3 instructions).
Ralf
Is there a reason why the same binaries can't be used for both?
On Sat, Mar 23, 2013 at 12:23 PM, Todd <toddrjen@gmail.com> wrote:
On Sat, Mar 23, 2013 at 12:21 PM, Ralf Gommers <ralf.gommers@gmail.com>wrote:
That's not the case. The official binaries for NumPy and SciPy are on SourceForge. The Windows installers on PyPI are there to make easy_install work, but they're likely slower than the SF installers (no SSE2/SSE3 instructions).
Ralf
Is there a reason why the same binaries can't be used for both?
The SF .exe superpack installers contains three installers: plain, SSE2 and SSE3 support. easy_install doesn't know what to do with such an installer. See http://thread.gmane.org/gmane.comp.python.numeric.general/29395/focus=29582f... the discussion on why things are as they are now. Ralf
I am a bit worried about the differences in results. Just to be sure you are comparing apples with apples, it may be a good idea to set the seed at the beginning: np.random.seed( SEED ) where SEED is an int. This way, you will be inverting always the same matrix, regardless of the Python version. I think, even if the timing is different, the results should be the same. http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.seed.html#n... David. On 23 March 2013 15:39, Colin J. Williams <cjwilliams43@gmail.com> wrote:
On 23/03/2013 7:21 AM, Ralf Gommers wrote:
On Fri, Mar 22, 2013 at 10:39 PM, Colin J. Williams <cjwilliams43@gmail.com> wrote:
On 20/03/2013 11:12 AM, Frédéric Bastien wrote:
On Wed, Mar 20, 2013 at 11:01 AM, Colin J. Williams <cjwilliams43@gmail.com> wrote:
On 20/03/2013 10:30 AM, Frédéric Bastien wrote:
Hi,
win32 do not mean it is a 32 bits windows. sys.platform always return win32 on 32bits and 64 bits windows even for python 64 bits.
But that is a good question, is your python 32 or 64 bits?
32 bits.
That explain why you have memory problem but not other people with 64 bits version. So if you want to work with bigger input, change to a python 64 bits.
Fred
Thanks to the people who responded to my report that numpy, with Python 3.2 was significantly slower than with Python 2.7.
I have updated to numpy 1.7.0 for each of the Pythons 2.7.3, 3.2.3 and 3.3.0.
The Pythons came from python.org and the Numpys from PyPi. The SciPy site still points to Source Forge, I gathered from the responses that Source Forge is no longer recommended for downloads.
That's not the case. The official binaries for NumPy and SciPy are on SourceForge. The Windows installers on PyPI are there to make easy_install work, but they're likely slower than the SF installers (no SSE2/SSE3 instructions).
Ralf
Thanks, I'll read over Robert Kern's comments. PyPi is the simpler process, but, if the result is unoptimized code, then easy_install is not the way to go.
The code is available here(http://web.ncf.ca/cjw/testFPSpeed.py) and the most recent test results are here(http://web.ncf.ca/cjw/FP%2023-Mar-13%20Test%20Summary.txt). These are using PyPi, I'll look into SourceForge.
Colin W.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (8)
-
Chris Barker - NOAA Federal -
Colin J. Williams -
Colin J. Williams -
Daπid -
Frédéric Bastien -
Jens Nielsen -
Ralf Gommers -
Todd