Mailman 3 February 2013 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker 11 Jul '23

11 Jul '23

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem 18 Feb '23

18 Feb '23

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell 22 Jul '21

22 Jul '21

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

C-API: multidimensional array indexing?
by Johann Bauer 31 Mar '16

31 Mar '16

Dear experts, is there a C-API function for numpy which implements Python's multidimensional indexing? Say, I have a 2d-array PyArrayObject * M; and an index int i; how do I extract the i-th row or column M[i,:] respectively M[:,i]? I am looking for a function which gives again a PyArrayObject * and which is a view to M (no copied data; the result should be another PyArrayObject whose data and strides points to the correct memory portion of M). I searched the API documentation, Google and mailing lists for quite a long time but didn't find anything. Can you help me? Thanks, Johann

3 3

SVD errors
by mtrumpis＠berkeley.edu 04 Jun '13

04 Jun '13

Hello list.. I've run into two SVD errors over the last few days. Both errors are identical in numpy/scipy. I've submitted a ticket for the 1st problem (numpy ticket #990). Summary is: some builds of the lapack_lite module linking against system LAPACK (not the bundled dlapack_lite.o, etc) give a "LinAlgError: SVD did not converge" exception on my matrix. This error does occur using Mac's Accelerate framework LAPACK, and a coworker's Ubuntu LAPACK version. It does not seem to happen using ATLAS LAPACK (nor using Octave/Matlab on said Ubuntu) Just today I've come across a negative singular value cropping up in an SVD of a different matrix. This error does occur on my ATLAS LAPACK based numpy, as well as on the Ubuntu setup. And once again, it does not happen in Octave/Matlab. I'm using numpy 1.3.0.dev6336 -- don't know what the Ubuntu box is running. Here are some npy files for the two different cases: https://cirl.berkeley.edu/twiki/pub/User/MikeTrumpis/noconverge_operator.npy https://cirl.berkeley.edu/twiki/pub/User/MikeTrumpis/negsval_operator.npy Mike

6 7

pyhdf packaging
by Andreas Hilboll 23 May '13

23 May '13

Hi, I would like to package pyhdf for Ubuntu and make the package publicly available. Since the license is not totally clear to me (I cannot find any information in the sources, and the cheeseshop says "public", which doesn't mean anything to me), I tried to contact the maintainer, Andre Gosselin, however the email bounces, so I guess he's gone. Can anyone point me to how to proceed from here? Cheers, Andreas.

2 2

Any plans for windows 64-bit installer for 1.7?
by Matthew Brett 06 May '13

06 May '13

Hi, I see there is no Windows 64 bit installer for the 1.7 rc1. Is there any prospect of a 64 bit installer for the full release? Can I help? I have a Windows 7 64 bit machine that I use as a build slave; I am happy to give access. Cheers, Matthew

13 57

GSOC 2013
by Todd 06 Mar '13

06 Mar '13

Is numpy planning to participate in GSOC this year, either on their own or as a part of another group? If so, should we start trying to get some project suggestions together?

10 14

a question about freeze on numpy 1.7.0
by Gelin Yan 06 Mar '13

06 Mar '13

Hi All When I used numpy 1.7.0 with cx_freeze 4.3.1 on windows, I quickly found out even a simple "import numpy" may lead to program failed with following exception: "AttributeError: 'module' object has no attribute 'sys' After a poking around some codes I noticed /numpy/core/__init__.py has a line 'del sys' at the bottom. After I commented this line, and repacked the whole program, It ran fine. I also noticed this 'del sys' didn't exist on numpy 1.6.2 I am curious why this 'del sys' should be here and whether it is safe to omit it. Thanks. Regards gelin yan

4 7

Adding .abs() method to the array object
by Till Stensitzki 04 Mar '13

04 Mar '13

Hello, i know that the array object is already crowded, but i would like to see the abs method added, especially doing work on the console. Considering that many much less used functions are also implemented as a method, i don't think adding one more would be problematic. greetings Till

14 28

[ANN] SciPy2013 Sponsorships: Apply Now!
by Jonathan Rocher 28 Feb '13

28 Feb '13

[Apologies for cross-posts] Dear all, The SciPy2013 conference <http://conference.scipy.org/scipy2013/> will continue the tradition of offering sponsorships to attend the conference. These sponsorships provide funding for airfare, lodging, and conference registration. This year, these sponsorships will be *open to community members rather than just students*. Applications will be judged both on merit as well as need. If you would like to apply for yourself or a worthy candidate <http://conference.scipy.org/scipy2013/reg_fin_aid.php>, please note our application due date of *Monday, March 25th*. Winners will be announced on April 22nd. Looking forward to a very exciting conference! The SciPy 2013 Financial Aid Chairs Jeff Daily, Pacific Northwest National Lab. John Wiggins, Enthought Inc. -- Jonathan Rocher, PhD Scientific software developer Co-chair of SciPy2013 Conference Enthought, Inc. jrocher(a)enthought.com 1-512-536-1057 http://www.enthought.com

1 0

[ANN] SciPy2013 Tutorials: Call for Submissions
by Jonathan Rocher 28 Feb '13

28 Feb '13

[Apologies for cross-posts] Dear all, We are excited to kick off the SciPy2013 conference<http://conference.scipy.org/scipy2013/>with two days of tutorials. This year we are proud to expand the session to include *THREE parallel tracks*: introductory, intermediate and advanced. Teachers will receive a stipend for their service. We are accepting tutorial proposals from individuals or teams until *April 1st*. Click here for more details and to submit applications<http://conference.scipy.org/scipy2013/tutorial_overview.php> . Looking forward to a very exciting conference! The SciPy 2013 Tutorial Chairs Francesc Alted, Continuum Analytics Inc. Dharhas Pothina, Texas Water Development Board -- Jonathan Rocher, PhD Scientific software developer Co-chair of SciPy2013 Conference Enthought, Inc. jrocher(a)enthought.com 1-512-536-1057 http://www.enthought.com

1 0

scipy build errors
by raghu330 28 Feb '13

28 Feb '13

Dear All, I am trying to install iris, which needs scipy0.10 or above. My machine is RHEL6 with gfortran and g95 installed. I was able to install numpy without any trouble, but have issues with scipy. I have installed ATLAS, BLAS and LAPACK, but I still get this error. Kindly let me know how to go ahead.. Thanks in advance & Regards Raghu ----------------- /opt/python2.7.3/bin/python setup.py build --fcompiler=gfortran Running from scipy source directory. blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in /opt/python2.7.3/lib libraries mkl,vml,guide not found in /usr/local/lib64 libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /opt/lib libraries mkl,vml,guide not found in /usr/lib64 libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /opt/python2.7.3/lib libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64 libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries ptf77blas,ptcblas,atlas not found in /opt/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/atlas libraries ptf77blas,ptcblas,atlas not found in /usr/lib64/sse2 libraries ptf77blas,ptcblas,atlas not found in /usr/lib64 libraries ptf77blas,ptcblas,atlas not found in /usr/lib NOT AVAILABLE atlas_blas_info: libraries f77blas,cblas,atlas not found in /opt/python2.7.3/lib libraries f77blas,cblas,atlas not found in /usr/local/lib64 libraries f77blas,cblas,atlas not found in /usr/local/lib libraries f77blas,cblas,atlas not found in /opt/lib libraries f77blas,cblas,atlas not found in /usr/lib64/atlas libraries f77blas,cblas,atlas not found in /usr/lib64/sse2 libraries f77blas,cblas,atlas not found in /usr/lib64 libraries f77blas,cblas,atlas not found in /usr/lib NOT AVAILABLE /opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/system_info.py:1425 : UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: libraries blas not found in /opt/python2.7.3/lib libraries blas not found in /usr/local/lib64 libraries blas not found in /usr/local/lib libraries blas not found in /opt/lib libraries blas not found in /usr/lib64 libraries blas not found in /usr/lib NOT AVAILABLE /opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/system_info.py:1434 : UserWarning: Blas (http://www.netlib.org/blas/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [blas]) or by setting the BLAS environment variable. warnings.warn(BlasNotFoundError.__doc__) blas_src_info: NOT AVAILABLE /opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/system_info.py:1437 : UserWarning: Blas (http://www.netlib.org/blas/) sources not found. Directories to search for the sources can be specified in the numpy/distutils/site.cfg file (section [blas_src]) or by setting the BLAS_SRC environment variable. warnings.warn(BlasSrcNotFoundError.__doc__) Traceback (most recent call last): File "setup.py", line 208, in <module> setup_package() File "setup.py", line 199, in setup_package configuration=configuration ) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/core.py", l ine 152, in setup config = configuration() File "setup.py", line 136, in configuration config.add_subpackage('scipy') File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/misc_util.p y", line 1002, in add_subpackage caller_level = 2) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/misc_util.p y", line 971, in get_subpackage caller_level = caller_level + 1) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/misc_util.p y", line 908, in _get_configuration_from_setup_py config = setup_module.configuration(*args) File "scipy/setup.py", line 8, in configuration config.add_subpackage('integrate') File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/misc_util.p y", line 1002, in add_subpackage caller_level = 2) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/misc_util.p y", line 971, in get_subpackage caller_level = caller_level + 1) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/misc_util.p y", line 908, in _get_configuration_from_setup_py config = setup_module.configuration(*args) File "scipy/integrate/setup.py", line 10, in configuration blas_opt = get_info('blas_opt',notfound_action=2) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/system_info .py", line 321, in get_info return cl().get_info(notfound_action) File "/opt/python2.7.3/lib/python2.7/site-packages/numpy/distutils/system_info .py", line 472, in get_info raise self.notfounderror(self.notfounderror.__doc__) numpy.distutils.system_info.BlasNotFoundError: Blas (http://www.netlib.org/blas/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [blas]) or by setting the BLAS environment variable. ------------------------------------ EOM -------------------------------------- -- View this message in context: http://numpy-discussion.10968.n7.nabble.com/scipy-build-errors-tp33032.html Sent from the Numpy-discussion mailing list archive at Nabble.com.

1 0

interactive distutils
by Charles R Harris 27 Feb '13

27 Feb '13

It's there, it uses raw_input, does anyone use it? Chuck

1 0

embeding numpy in c++
by Sunghwan Choi 27 Feb '13

27 Feb '13

Hi, I am now trying to embedding numpy in my c++ code (more correctly ase module, which is dependent to numpy). My test code is simple test.cpp #include "Python.h" #include <stdio.h> #include <iostream> extern "C" void Py_Initialize(); extern "C" void PyErr_Print(); using namespace std; int main(int argc, char* argv[]) { double answer = 0; PyObject *modname, *mod, *mdict, *func, *stringarg, *args, *rslt; Py_Initialize(); modname = PyString_FromString("numpy"); mod = PyImport_Import(modname); PyErr_Print(); cout << mod << endl; return 0; } shchoi@aims0:~/tmp/test2$ vi test.cpp shchoi@aims0:~/tmp/test2$ icpc -c test.cpp -I ~/program/epd/include/python2.7/ shchoi@aims0:~/tmp/test2$ icpc -o exe test.o -L ~/program/epd/lib/ -lpython2.7 shchoi@aims0:~/tmp/test2$ ./exe Traceback (most recent call last): File "/home/shchoi/program/epd/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module> import add_newdocs File "/home/shchoi/program/epd/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module> from numpy.lib import add_newdoc File "/home/shchoi/program/epd/lib/python2.7/site-packages/numpy/lib/__init__.py" , line 4, in <module> from type_check import * File "/home/shchoi/program/epd/lib/python2.7/site-packages/numpy/lib/type_check.p y", line 8, in <module> import numpy.core.numeric as _nx File "/home/shchoi/program/epd/lib/python2.7/site-packages/numpy/core/__init__.py ", line 5, in <module> import multiarray ImportError: /home/shchoi/program/epd/lib/python2.7/site-packages/numpy/core/multiarray.s o: undefined symbol: PyUnicodeUCS2_AsASCIIString How to solve this problem? Can anyone help me? If anyone know the solution, please let me know as soon as possible Sincerely Sunghwan Choi

1 0

polyfit in NumPy v1.7
by David Pine 27 Feb '13

27 Feb '13

As of NumPy v1.7, numpy.polyfit includes an option for providing weighting to data to be fit. It's a welcome addition, but the implementation seems a bit non-standard, perhaps even wrong, and I wonder if someone can enlighten me. 1. The documentation does not specify what the weighting array "w" is supposed to be. After some fooling around I figured out that it is 1/sigma, where sigma is the standard deviation uncertainty "sigma" in the y data. A better implementation, which would be consistent with how weighting is done in scipy.optimize.curve_fit, would be to specify "sigma" directly, rather than "w". This is typically what the user has at hand, this syntax is more transparent, and it would make polyfit consistent with the nonlinear fitting routine scipy.optimize.curve_fit. 2. The definition of the covariance matrix in polyfit is peculiar (or maybe wrong, I'm not sure). Towards the end of the code for polyfit, the standard covariance matrix is calculated and given the variable name "Vbase". However, instead of returning Vbase as the covariance matrix, polyfit returns the quantity Vbase * fac, where fac = resids / (len(x) - order - 2.0). fac scales as N, the number of data points, so the covariance matrix is about N times larger than it should be for large values of N; the increase is smaller for small N (of order 10). Perhaps the authors of polyfit want to use a more conservative error estimate, but the references given in the poyfit code make no mention of it. I think it would be much better to return the standard definition of the covariance matrix (see, for example, the book Numerical Recipes). David Pine

4 6

Apply transform to many small matrices
by Jorge Scandaliaris 27 Feb '13

27 Feb '13

Hi, First of all excuse me if this is a trivial question. I have the feeling it is, but searching and looking through the docs has proven unsuccesful so far. I have an ndarray A of shape (M,2,2) representing M 2 x 2 matrices. Now I want to apply a transform T of shape (2,2) to each of matrix. The way I do this now is by iterating over all rows of A multiplying the matrices using numpy.dot(): for row in np.arange(A.shape[0]): A[row] = np.dot(A[row],T) but this seems to be slow when M is large and I have the feeling there must be a way of doing it better. Thanks, jorges

3 4

swig interface file (numpy.i) warning
by Tom Krauss 27 Feb '13

27 Feb '13

Hi, I've been happy to use numpy.i for generating SWIG interfaces to C++. For a while, I've noticed this warning while compiling: /Users/tkrauss/projects/dev_env/lib/python2.7/site-packages/numpy-1.8.0.dev_f2f0ac0_20120725-py2.7-macosx-10.8-x86_64.egg/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" and today tried to get rid of the warning. So, in numpy.i, I followed the warning's advice. I added the # def here: %{ #ifndef SWIG_FILE_WITH_INIT # define NO_IMPORT_ARRAY #endif #include "stdio.h" *#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION* #include <numpy/arrayobject.h> %} SWIG was happy, but when compiling the C++ wrapper, there were many warnings followed by many errors. The warnings were for redefinition of NPY_MIN_BYTE and similar. The errors were for all kinds of stuff, excerpt here: native_wrap.cpp:3632: error: ‘PyArray_NOTYPE’ was not declared in this scope native_wrap.cpp:3633: error: cannot convert ‘PyObject*’ to ‘const PyArrayObject*’ for argument ‘1’ to ‘int PyArray_TYPE(const PyArrayObject*)’ native_wrap.cpp: At global scope: native_wrap.cpp:3877: error: ‘intp’ has not been declared native_wrap.cpp: In function ‘int require_fortran(PyArrayObject*)’: native_wrap.cpp:3929: error: ‘struct tagPyArrayObject’ has no member named ‘nd’ native_wrap.cpp:3933: error: ‘struct tagPyArrayObject’ has no member named ‘flags’ native_wrap.cpp:3933: error: ‘FARRAY’ was not declared in this scope native_wrap.cpp:20411: error: ‘struct tagPyArrayObject’ has no member named ‘data’ It looks like there is a new C API for numpy, and the version of numpy.i that I have doesn't use it. Is there a new version of numpy.i available (or in development) that works with the new API? Short term it will just get rid of a warning but I am interested in a good long term solution in case I need to upgrade numpy. Thanks, Tom

4 8

[ANN] SciPy2013: Call for abstracts
by Jonathan Rocher 27 Feb '13

27 Feb '13

[Apologies for cross-posts] Dear all, The annual SciPy Conference (Scientific Computing with Python)<http://conference.scipy.org/scipy2013/about.php> allows participants from academic, commercial, and governmental organizations to showcase their latest projects, learn from skilled users and developers, and collaborate on code development. *The deadline for abstract submissions is March 20th, 2013. * Submissions are welcome that address general Scientific Computing with Python, one of the two special themes for this years conference (machine learning & reproducible science), or the domain-specific mini-symposia<http://conference.scipy.org/scipy2013/about.php>held during the conference (Meteorology, climatology, and atmospheric and oceanic science, Astronomy and astrophysics, Medical imaging, Bio-informatics). Please submit your abstract at the SciPy 2013 website abstract submission form <http://conference.scipy.org/scipy2013/speaking_submission.php>. Abstracts will be accepted for posters or presentations. Optional papers to be published in the conference proceedings will be requested following abstract submission. This year the proceedings will be made available prior to the conference to help attendees navigate the conference. We look forward to an exciting and interesting set of talks, posters, and discussions and hope to see you at the conference. The SciPy 2013 Program Committee Chairs Matt McCormick, Kitware, Inc. Katy Huff, University of Wisconsin-Madison and Argonne National Laboratory

1 0

Plans for missing data
by Lluís 27 Feb '13

27 Feb '13

Hi there, I haven't been able to find an answer after a quick search, but are there any plans regarding the (re)inclusion of missing data support into some future version of numpy? Thanks a lot! Lluis -- "And it's much the same thing with knowledge, for whenever you learn something new, the whole world becomes that much richer." -- The Princess of Pure Reason, as told by Norton Juster in The Phantom Tollbooth

2 2

flatnonzero / nonzero returned indices order
by Gabor Kovacs 27 Feb '13

27 Feb '13

Hello, for 1-D arrays, can one assume that the indices returned by flatnonzero is ordered? It looks so, but no such statement anywhere in the docs. (In general, order of indices in case of higher dimensions?) Thanks Gabor

1 0

1.7.1
by Charles R Harris 26 Feb '13

26 Feb '13

When should we put out 1.7.1? Discuss ;) Chuck

4 7

numpy.scipy.org giving 404 error
by Peter Cock 26 Feb '13

26 Feb '13

Hello all, http://numpy.scipy.org is giving a GitHub 404 error. As this used to be a widely used URL for the project, and likely appears in many printed references, could it be fixed to point to or redirect to the (relatively new) http://www.numpy.org site please? Thanks, Peter

1 0

What should np.ndarray.__contains__ do
by Sebastian Berg 26 Feb '13

26 Feb '13

Hello all, currently the `__contains__` method or the `in` operator on arrays, does not return what the user would expect when in the operation `a in b` the `a` is not a single element (see "In [3]-[4]" below). The first solution coming to mind might be checking `all()` for all dimensions given in argument `a` (see line "In [5]" for a simplistic example). This does not play too well with broadcasting however, but one could maybe simply *not* broadcast at all (i.e. a.shape == b.shape[b.ndim-a.ndim:]) and raise an error/return False otherwise. On the other hand one could say broadcasting of `a` onto `b` should be "any" along that dimension (see "In [8]"). The other way should maybe raise an error though (see "In [9]" to understand what I mean). I think using broadcasting dimensions where `a` is repeated over `b` as the dimensions to use "any" logic on is the most general way for numpy to handle this consistently, while the other way around could be handled with an `all` but to me makes so little sense that I think it should be an error. Of course this is different to a list of lists, which gives False in these cases, but arrays are not list of lists... As a side note, since for loop, etc. use "for item in array", I do not think that vectorizing along `a` as np.in1d does is reasonable. `in` should return a single boolean. I have opened an issue for it: https://github.com/numpy/numpy/issues/3016#issuecomment-14045545 Regards, Sebastian In [1]: a = np.array([0, 2]) In [2]: b = np.arange(10).reshape(5,2) In [3]: b Out[3]: array([[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]]) In [4]: a in b Out[4]: True In [5]: (b == a).any() Out[5]: True In [6]: (b == a).all(0).any() # the 0 could be multiple axes Out[6]: False In [7]: a_2d = a[None,:] In [8]: a_2d in b # broadcast dimension means "any" -> True Out[8]: True In [9]: [0, 1] in b[:,:1] # should not work (or be False, not True) Out[9]: True

3 6

another discussion on numpy correlate (and convolution)
by Pierre Haessig 26 Feb '13

26 Feb '13

Hi everybody, (just coming from a discussion on the performance of Matplotlib's (x)corr function which uses np.correlate) There have been already many discussions on how to compute (cross-)correlations of time-series in Python (like http://stackoverflow.com/questions/6991471/computing-cross-correlation-func…). The discussion is spread between the various stakeholders (just to name some I've in mind : scipy, statsmodel, matplotlib, ...). There are basically 2 implementations : time-domain and frequency-domain (using fft + multiplication). My discussion is only on time-domain implementation. The key usecase which I feel is not well adressed today is when computing the (cross)correlation of two long timeseries on only a few lagpoints. For time-domain, one can either write its own implementation or rely on numpy.correlate. The latter rely on the fast C implementation _pyarray_correlate() in multiarraymodule.c (https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/multia…) Now, the signature of this function is meant to accept one of three *computation modes* ('valid', 'same', 'full'). Thoses modes make a lot of sense when using this function in the context of convoluting two signals (say an "input array" and a "impulse response array"). In the context of computing the (cross)correlation of two long timeseries on only a few lagpoints, those computation modes are unadapted and potentially leads to huge computational and memory wastes (for example : https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/axes.py…). For some time, I was thinking the solution was to write a dedicated function in one of the stakeholder modules (but which ? statsmodel ?) but now I came to think numpy is the best place to put a change and this would quickly benefit to all stackeholders downstream. Indeed, I looked more carefully at the C _pyarray_correlate() function, and I've come to the conclusion that these three computation modes are an unnecessary restriction because the actual computation relies on a triple for loop (https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/multia…) which boundaries are governed by two integers `n_left` and `n_right` instead of the three modes. Maybe I've misunderstood the inner-working of this function, but I've the feeling that the ability to set these two integers directly instead of just the mode would open up the possibility to compute the correlation on only a few lagpoints. I'm fully aware that changing the signature of such a core numpy is out-of-question but I've got the feeling that a reasonably small some code refactor might lift the longgoing problem of (cross-)correlation computation. The Python np.correlate would require two additional keywords -`n_left` and `n_right`)which would override the `mode` keyword. Only the C function would need some more care to keep good backward compatibility in case it's used externally. What do other people think ? Best, Pierre

4 5