Hi All, I'm pleased to announce that Numpy 1.11.0b1 <http://sourceforge.net/projects/numpy/files/NumPy/1.11.0b1/> is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;) Chuck
Hi Chuck,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-) adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources. executing numpy/core/code_generators/generate_numpy_api.py error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'
tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes -rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h -rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10. Cheers, Derek
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier < derek@astro.physik.uni-goettingen.de> wrote:
Hi Chuck,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-)
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources. executing numpy/core/code_generators/generate_numpy_api.py error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'
Grr, yes indeed, `paver sdist` doesn't do the right thing, none of the `multiarray/*.c.src` files are included, but it works fine in 1.10.x. The changes are minimal, the only thing that would seem to matter is the removal of setupegg.py. Ralf, any ideas.
tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes -rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h -rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream` Chuck
On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier < derek@astro.physik.uni-goettingen.de> wrote:
Hi Chuck,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-)
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources. executing numpy/core/code_generators/generate_numpy_api.py error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'
Grr, yes indeed, `paver sdist` doesn't do the right thing, none of the `multiarray/*.c.src` files are included, but it works fine in 1.10.x. The changes are minimal, the only thing that would seem to matter is the removal of setupegg.py. Ralf, any ideas.
tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes -rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h -rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream`
setupegg.py doesn't seem to matter... Chuck
On Tue, Jan 26, 2016 at 7:13 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier < derek@astro.physik.uni-goettingen.de> wrote:
Hi Chuck,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-)
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources. executing numpy/core/code_generators/generate_numpy_api.py error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'
Grr, yes indeed, `paver sdist` doesn't do the right thing, none of the `multiarray/*.c.src` files are included, but it works fine in 1.10.x. The changes are minimal, the only thing that would seem to matter is the removal of setupegg.py. Ralf, any ideas.
tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes -rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h -rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream`
setupegg.py doesn't seem to matter...
OK, it is the changes in the root setup.py file, probably the switch to setuptools. Chuck
On Tue, Jan 26, 2016 at 7:45 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Tue, Jan 26, 2016 at 7:13 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier < derek@astro.physik.uni-goettingen.de> wrote:
Hi Chuck,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-)
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources. executing numpy/core/code_generators/generate_numpy_api.py error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'
Grr, yes indeed, `paver sdist` doesn't do the right thing, none of the `multiarray/*.c.src` files are included, but it works fine in 1.10.x. The changes are minimal, the only thing that would seem to matter is the removal of setupegg.py. Ralf, any ideas.
tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes -rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h -rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream`
setupegg.py doesn't seem to matter...
OK, it is the changes in the root setup.py file, probably the switch to setuptools.
Yep, it's setuptools. If I import sdist from distutils instead, everything works fine. Chuck
On Wed, Jan 27, 2016 at 4:47 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Tue, Jan 26, 2016 at 7:45 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Jan 26, 2016 at 7:13 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Jan 26, 2016 at 6:58 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Jan 26, 2016 at 4:48 PM, Derek Homeier < derek@astro.physik.uni-goettingen.de> wrote:
Hi Chuck,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
the tarball seems to be incomplete, hope that does not bode ill ;-)
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources. executing numpy/core/code_generators/generate_numpy_api.py error: [Errno 2] No such file or directory: 'numpy/core/code_generators/../src/multiarray/arraytypes.c.src'
Grr, yes indeed, `paver sdist` doesn't do the right thing, none of the `multiarray/*.c.src` files are included, but it works fine in 1.10.x. The changes are minimal, the only thing that would seem to matter is the removal of setupegg.py. Ralf, any ideas.
tar tvf /sw/src/numpy-1.11.0b1.tar.gz |grep arraytypes -rw-rw-r-- charris/charris 62563 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/include/numpy/ndarraytypes.h -rw-rw-r-- charris/charris 981 2016-01-21 20:38 numpy-1.11.0b1/numpy/core/src/multiarray/arraytypes.h
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream`
setupegg.py doesn't seem to matter...
OK, it is the changes in the root setup.py file, probably the switch to setuptools.
Yep, it's setuptools. If I import sdist from distutils instead, everything works fine.
Setuptools starts build_src, don't know why yet but it doesn't look to me like it should be doing that. Issue for more detailed discussion: https://github.com/numpy/numpy/issues/7127 Ralf
On 27 Jan 2016, at 2:58 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
FWIW, the maintenance/1.11.x branch (there is no tag for the beta?) builds and passes all tests with Python 2.7.11 and 3.5.1 on Mac OS X 10.10.
You probably didn't fetch the tags, if they can't be reached from the branch head they don't download automatically. Try `git fetch --tags upstream`
Thanks, that did it. Successfully tested v1.11.0b1 on 10.11 and with Python 2.7.8 and 3.4.1 on openSUSE 13.2 as well. Derek
Why the dot function/method is slower than @ on python 3.5.1? Tested from the latest 1.11 maintenance branch. np.__version__ Out[39]: '1.11.0.dev0+Unknown' %timeit A @ c 10000 loops, best of 3: 185 µs per loop %timeit A.dot(c) 1000 loops, best of 3: 526 µs per loop %timeit np.dot(A,c) 1000 loops, best of 3: 527 µs per loop A.dtype, A.shape, A.flags Out[43]: (dtype('float32'), (100, 100, 3), C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False) c.dtype, c.shape, c.flags Out[44]: (dtype('float32'), (3, 3), C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False) From: NumPy-Discussion <numpy-discussion-bounces@scipy.org> on behalf of Charles R Harris <charlesr.harris@gmail.com> Sent: 26 January 2016 22:49 To: numpy-discussion; SciPy Developers List; SciPy Users List Subject: [Numpy-discussion] Numpy 1.11.0b1 is out Hi All, I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;) Chuck
On Mi, 2016-01-27 at 11:19 +0000, Nadav Horesh wrote:
Why the dot function/method is slower than @ on python 3.5.1? Tested from the latest 1.11 maintenance branch.
The explanation I think is that you do not have a blas optimization. In which case the fallback mode is probably faster in the @ case (since it has SSE2 optimization by using einsum, while np.dot does not do that). Btw. thanks for all the work getting this done Chuck! - Sebastian
np.__version__ Out[39]: '1.11.0.dev0+Unknown'
%timeit A @ c 10000 loops, best of 3: 185 µs per loop
%timeit A.dot(c) 1000 loops, best of 3: 526 µs per loop
%timeit np.dot(A,c) 1000 loops, best of 3: 527 µs per loop
A.dtype, A.shape, A.flags Out[43]: (dtype('float32'), (100, 100, 3), C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False)
c.dtype, c.shape, c.flags Out[44]: (dtype('float32'), (3, 3), C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False)
From: NumPy-Discussion <numpy-discussion-bounces@scipy.org> on behalf of Charles R Harris <charlesr.harris@gmail.com> Sent: 26 January 2016 22:49 To: numpy-discussion; SciPy Developers List; SciPy Users List Subject: [Numpy-discussion] Numpy 1.11.0b1 is out
Hi All,
I'm pleased to announce that Numpy 1.11.0b1 is now available on sourceforge. This is a source release as the mingw32 toolchain is broken. Please test it out and report any errors that you discover. Hopefully we can do better with 1.11.0 than we did with 1.10.0 ;)
Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
On 27 Jan 2016, at 1:10 pm, Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Mi, 2016-01-27 at 11:19 +0000, Nadav Horesh wrote:
Why the dot function/method is slower than @ on python 3.5.1? Tested from the latest 1.11 maintenance branch.
The explanation I think is that you do not have a blas optimization. In which case the fallback mode is probably faster in the @ case (since it has SSE2 optimization by using einsum, while np.dot does not do that).
I am a bit confused now, as A @ c is just short for A.__matmul__(c) or equivalent to np.matmul(A,c), so why would these not use the optimised blas? Also, I am getting almost identical results on my Mac, yet I thought numpy would by default build against the VecLib optimised BLAS. If I build explicitly against ATLAS, I am actually seeing slightly slower results. But I also saw these kind of warnings on the first timeit runs: %timeit A.dot(c) The slowest run took 6.91 times longer than the fastest. This could mean that an intermediate result is being cached and when testing much larger arrays, the discrepancy between matmul and dot rather increases, so perhaps this is more an issue of a less memory-efficient implementation in np.dot? Cheers, Derek
On Sa, 2016-01-30 at 20:27 +0100, Derek Homeier wrote:
On 27 Jan 2016, at 1:10 pm, Sebastian Berg < sebastian@sipsolutions.net> wrote:
On Mi, 2016-01-27 at 11:19 +0000, Nadav Horesh wrote:
Why the dot function/method is slower than @ on python 3.5.1? Tested from the latest 1.11 maintenance branch.
The explanation I think is that you do not have a blas optimization. In which case the fallback mode is probably faster in the @ case (since it has SSE2 optimization by using einsum, while np.dot does not do that).
I am a bit confused now, as A @ c is just short for A.__matmul__(c) or equivalent to np.matmul(A,c), so why would these not use the optimised blas? Also, I am getting almost identical results on my Mac, yet I thought numpy would by default build against the VecLib optimised BLAS. If I build explicitly against ATLAS, I am actually seeing slightly slower results. But I also saw these kind of warnings on the first timeit runs:
%timeit A.dot(c) The slowest run took 6.91 times longer than the fastest. This could mean that an intermediate result is being cached
and when testing much larger arrays, the discrepancy between matmul and dot rather increases, so perhaps this is more an issue of a less memory -efficient implementation in np.dot?
Sorry, I missed the fact that one of the arrays was 3D. In that case I am not even sure which if the functions call into blas or what else they have to do, would have to check. Note that `np.dot` uses a different type of combinging high dimensional arrays. @/matmul broadcasts extra axes, while np.dot will do the outer combination of them, so that the result is: As = A.shape As.pop(-1) cs = c.shape cs.pop(-2) # if possible result_shape = As + cs which happens to be identical if only A.ndim > 2 and c.ndim <= 2. - Sebastian
Cheers, Derek
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
On 31 Jan 2016, at 9:48 am, Sebastian Berg <sebastian@sipsolutions.net> wrote:
On Sa, 2016-01-30 at 20:27 +0100, Derek Homeier wrote:
On 27 Jan 2016, at 1:10 pm, Sebastian Berg < sebastian@sipsolutions.net> wrote:
On Mi, 2016-01-27 at 11:19 +0000, Nadav Horesh wrote:
Why the dot function/method is slower than @ on python 3.5.1? Tested from the latest 1.11 maintenance branch.
The explanation I think is that you do not have a blas optimization. In which case the fallback mode is probably faster in the @ case (since it has SSE2 optimization by using einsum, while np.dot does not do that).
I am a bit confused now, as A @ c is just short for A.__matmul__(c) or equivalent to np.matmul(A,c), so why would these not use the optimised blas? Also, I am getting almost identical results on my Mac, yet I thought numpy would by default build against the VecLib optimised BLAS. If I build explicitly against ATLAS, I am actually seeing slightly slower results. But I also saw these kind of warnings on the first timeit runs:
%timeit A.dot(c) The slowest run took 6.91 times longer than the fastest. This could mean that an intermediate result is being cached
and when testing much larger arrays, the discrepancy between matmul and dot rather increases, so perhaps this is more an issue of a less memory -efficient implementation in np.dot?
Sorry, I missed the fact that one of the arrays was 3D. In that case I am not even sure which if the functions call into blas or what else they have to do, would have to check. Note that `np.dot` uses a different type of combinging high dimensional arrays. @/matmul broadcasts extra axes, while np.dot will do the outer combination of them, so that the result is:
As = A.shape As.pop(-1) cs = c.shape cs.pop(-2) # if possible result_shape = As + cs
which happens to be identical if only A.ndim > 2 and c.ndim <= 2.
Makes sense now; with A.ndim = 2 both operations take about the same time (and are ~50% faster with VecLib than with ATLAS) and yield identical results, while any additional dimension in A adds more overhead time to np.dot, and the results are np.allclose, but not exactly identical. Thanks, Derek
participants (5)
-
Charles R Harris -
Derek Homeier -
Nadav Horesh -
Ralf Gommers -
Sebastian Berg