Mailman 3 November 2007 - NumPy-Discussion

how to assign to a "sub set" of an array
by Sebastian Haase Nov. 14, 2007

Nov. 14, 2007

Hi, First here is some test I ran, where I think the last command shows a bug: >>> a = N.arange(4); a.shape=2,2; a [[0 1] [2 3]] >>> aa = N.array((a,a,a)); aa [[[0 1] [2 3]] [[0 1] [2 3]] [[0 1] [2 3]]] >>> N.nonzero(a==0) ([0], [0]) >>> aa[N.nonzero(a==0)] = 5,5; aa [[[ 5 544434034] [ 2 3]] [[ 0 1] [ 2 3]] [[ 0 1] [ 2 3]]] What happend here ? The value 544434034 is different every time..... Background: I have a gray-scale 2D image (in this examples it's a 2x2 pixel image named 'a') I would like to show this image using grey-values EXCEPT I would like to show 0-values as blue. For use with pyOpenGL textures, I convert the image into an RGB image, in the example called 'aa', with shape 3,2,2. This is where I'm getting lost, and I tried assigning "something" (5,5) to aa "only at certain pixels".... Maybe someone can teach what the proper indexing-constuct, that I would need to do what I want. Did I find a bug ? Thanks, Sebastian Haase

2 2

Proposing Ubiquity
by dieter h. Nov. 14, 2007

Nov. 14, 2007

Hi all, Would it make sense for all functionality in Numpy/Scipy to have ubiquitous returns? In that I'm proposing that every func/method (where appropriate) have a flag in its arg list establishing a return for either [INDEX, BOOLEAN, VALUE=default]. this could be by a singular Enum flag, I believe. Obvioulsy, this proposal gives the option for a return of Index/Bool to be used for Indexing arrays, or the actual values. Since I mosty index existing arrays, this would be a huge plus for me. Thanks -Dieter

2 2

Warnings as exceptions?
by Michael McNeil Forbes Nov. 13, 2007

Nov. 13, 2007

Why are numpy warnings printed rather than issued using the standard warnings library? I know that the behaviour can be controlled by seterr(), but it seem rather unpythonic not to use the warnings library. Is there an explicit reason for this choice? (It seems like a pretty trivial modification of util.py will catch most of the cases.) If performance is an issue, one could add a 'print' mode which behaves as the current 'warn'. Michael. (P.S. I am sure this has been discussed somewhere before, but I can't seem to find a reference.)

3 4

Using arr.dtype.type to check byteorder-independed dtype fails for bool
by Sebastian Haase Nov. 13, 2007

Nov. 13, 2007

Hi, I need to check the array dtype in a way that it is ignoring differences coming only from big-endian vs. little-endian. So far I was using a test like this successfully: if arr.dtype.type == N.uint8: .... This would test positively for both byte-orders independent of the system's byteorder. I just noticed however, that if arr.dtype.type == N.bool: ... always fails (evaluates to False) Testing arr.dtype == N.bool, on the other hand works (Sitting at my Windows-PC cannot test the byte-order problem though !) While writing this email, I just found that if arr.dtype.type == N.bool_: ... works. (Note the trailing underscore !). What is this ? Thanks for help,hints and comments, Sebastian Haase

3 3

f2py cannot find gfortran compiler on OSX 10.5
by Chris Nov. 13, 2007

Nov. 13, 2007

I've just upgraded my OSX system to Leopard, and have successfully build numpy from scratch. I am trying to build some code, which includes f2py extensions, that build successfully on 10.4. I have the gfortran compiler installed, and am building my code using: python setup.py config_fc --fcompiler gnu95 build However, when I run this, I get the following error when the Fortran code compiles: AttributeError: 'NoneType' object has no attribute 'compile' Is there any reason that gnu95 no longer would find gfortran? I have confirmed that the compiler is in my path. Thanks.

2 1

Merging new (optional) build system in the trunk ?
by David Cournapeau Nov. 13, 2007

Nov. 13, 2007

Hi, I would appreciate to get some comment on whether there is any chance to get my numpy.scons branch merge into the trunk at some near future. I feel to have reached the point where the only big thing missing is more testing. I tried to test it on many platforms, but there is a limit to what I can test just by myself. The branch has been conceived such as by default, the current numpy.distutils is used to build, and the scons-based build is used only by explicit request (another setup.py), so this does not force any use now: except the work related to numpyconfig (which can be tested separately, since it was done in a different branch, and is only a few lines of code), everything else is exactly the same than before. Thanks, David P.S: Just as a reminder, here are some of the already implemented goals currently: - all the facilities of scons are available to build extensions (parallel builds, automatic dependency, easy extension through of new builders and configuration checks, etc...) - ability to build shared libraries and Ctypes-based extension in a cross platform way. - optimization flags logic cleanly separated from the other flags: you can now easily replace, add, remove optimization flags without having to handle platform specific flags such as -fPIC, etc... - facility replacing system_info which is easier to use (one function call to support most common cases) - performance library checks are stronger, for more reliable build results. MKL, Sunperf, ATLAS, vecLib and Accelerate are already supported; new libraries can generally be added in a few lines, as they all use the same infrastructure. - the current (successfully) tested platforms are : Mac OS X Intel (gfortran + gcc), Linux (g77 + gcc, gfortran + gcc, intel compilers), MS Windows (mingw, VS 2003 + g77, VS 2005 + g77), Solaris (with sun compilers). - Generally, I tried to follow the autoconf philosophy (do not check for version, check for facilities). The fortran runtime, for example, is found automatically, as is the fortran name mangling. - easier maintenance: no more distutils monkey patching because upstream distutils does not have a clean API, the current codebase is a bit below 3000 lines of code, not counting scons (as a comparison, system_info.py is almost 2000 lines by itself), most of the code is totally cross platform, and platform specific are encapsulated; there are a few hacks, but they are implementation-only hacks. So concretely, I believe that most of the things stated in http://projects.scipy.org/scipy/numpy/wiki/DistutilsRevamp are available *now*. More concrete info here: http://projects.scipy.org/scipy/numpy/wiki/NumpyScons

3 3

I removed some duplicate code from weave
by Jarrod Millman Nov. 13, 2007

Nov. 13, 2007

Hey, I just noticed that dumb_shelve.py and dumbdbm_patched.py were in both scipy.io and scipy.weave. Since I assume that this was just an oversight, I went ahead and cleaned it up. First, I made sure that the newest code was in scipy.io: http://projects.scipy.org/scipy/scipy/changeset/3521 http://projects.scipy.org/scipy/scipy/changeset/3522 And then I removed the version in scipy.weave: http://projects.scipy.org/scipy/scipy/changeset/3524 Please let me know if this change causes any major problems. Cheers, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 0

how to convert btw rgb and pixel value
by devnew＠gmail.com Nov. 12, 2007

Nov. 12, 2007

hi i wish to convert an rgb image into an array of double values..is there a method for that in numpy? also i want to create an array of doubles into corresponding rgb tuples of an image can anyone guide me? dn

4 3

numpy-1.0.5.dev4445.egg-info
by Keith Goodman Nov. 12, 2007

Nov. 12, 2007

I noticed that python setup.py install --prefix=/usr/local installs a new file called numpy-1.0.5.dev4445.egg-info. It used to be that files were only installed in /usr/local/lib/python2.4/site-packages/numpy. What is the file used for? $ cat /usr/local/lib/python2.4/site-packages/numpy-1.0.5.dev4445.egg-info Metadata-Version: 1.0 Name: numpy Version: 1.0.5.dev4445 Summary: NumPy: array processing for numbers, strings, records, and objects. Home-page: http://numeric.scipy.org Author: NumPy Developers Author-email: numpy-discussion(a)lists.sourceforge.net License: BSD Download-URL: http://sourceforge.net/project/showfiles.php?group_id=1369&package_id=175103 Description: NumPy is a general-purpose array-processing package designed to efficiently manipulate large multi-dimensional arrays of arbitrary records without sacrificing too much speed for small multi-dimensional arrays. NumPy is built on the Numeric code base and adds features introduced by numarray as well as an extended C-API and the ability to create arrays of arbitrary type which also makes NumPy suitable for interfacing with general-purpose data-base applications. There are also basic facilities for discrete fourier transform, basic linear algebra and random number generation. Platform: Windows Platform: Linux Platform: Solaris Platform: Mac OS-X Platform: Unix Classifier: Development Status :: 4 - Beta Classifier: Intended Audience :: Science/Research Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved Classifier: Programming Language :: C Classifier: Programming Language :: Python Classifier: Topic :: Software Development Classifier: Topic :: Scientific/Engineering Classifier: Operating System :: Microsoft :: Windows Classifier: Operating System :: POSIX Classifier: Operating System :: Unix Classifier: Operating System :: MacOS

2 1

Getting an array interface and using it
by Matthieu Brucher Nov. 12, 2007

Nov. 12, 2007

Hi, I have an object that exposes an array interface. I want to modify the data it contains, but using numpy.array(myObject) seems to copy the data and thus my object is not modified. Am I mistaken or did I make a mistake in my array interface ? Matthieu -- French PhD student Website : http://miles.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher

3 3

Numpy book and PyArray_CastToType
by Matthieu Brucher Nov. 12, 2007

Nov. 12, 2007

Hi, I'm trying to use PyArray_CastToType(), but according to the book, there are two arguments, a PyArrayObject* and a PyArrayDescr*, but according to the header file, there are three arguments, an additional int. Does someone know its use ? Matthieu -- French PhD student Website : http://miles.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher

2 3

Re: [Numpy-discussion] Problem with numpy.linalg.eig?
by Lou Pecora Nov. 12, 2007

Nov. 12, 2007

Works fine on my computer (Mac OS X 10.4), Python 2.4. Runs in a second or so. -- Lou Pecora ---Peter wrote: Hi all, The following code calling numpy v1.0.4 fails to terminate on my machine, which was not the case with v1.0.3.1 from numpy import arange, float64 from numpy.linalg import eig a = arange(13*13, dtype = float64) a.shape = (13,13) a = a%17 eig(a) Regards, Peter __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

weibull distribution has only one parameter?
by D.Hendriks (Dennis) Nov. 12, 2007

Nov. 12, 2007

According to (for instance) http://en.wikipedia.org/wiki/Weibull_distribution the Weibull distribution has two parameters: lambda > 0 is the scale parameter (real) and k > 0 is the shape parameter (real). However, the numpy.random.weibull function has only a single 'a' parameter (except for the size parameter which indicates the size of the array to fill with values - this is NOT a parameter of the distribution itself). My question is how this 'a' parameter translates to the Weibull distribution as it 'normally' is and how to sample the distribution when I have the lambda and k parameters?

4 6

Compilation with Visual Studio 2003
by Matthieu Brucher Nov. 11, 2007

Nov. 11, 2007

Hi, I want to add some support to ACML, but currently, I can't even compile numpy (Visual Studio and g77), the current SVN fails with : running build_ext No module named msvccompiler in numpy.distutils; trying from distutils customize MSVCCompiler customize MSVCCompiler using build_ext building 'numpy.core.multiarray' extension compiling C sources D:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.exe /c /nologo /Ox /MD /W3 /GX /DNDEBUG -Ibuild\src.win32-2.5\numpy\core\src-Inumpy\core\include -Ibuild\src.win32-2.5\numpy\core -Inumpy\core\src -Inumpy\core\include -ID:\Pyth on25\include -ID:\Python25\PC /Tcnumpy\core\src\multiarraymodule.c /Fobuild\temp .win32-2.5\Release\numpy\core\src\multiarraymodule.obj multiarraymodule.c d:\Travail\numpy\numpy\core\src\arrayobject.c(662) : warning C4244: '=' : conver sion de 'npy_longlong' en 'long', perte possible de données numpy\core\src\multiarraymodule.c(7592) : error C2065: 'NPY_ALLOW_THREADS' : ide ntificateur non déclaré error: Command "D:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\bin\cl.ex e /c /nologo /Ox /MD /W3 /GX /DNDEBUG -Ibuild\src.win32-2.5\numpy\core\src-Inum py\core\include -Ibuild\src.win32-2.5\numpy\core -Inumpy\core\src -Inumpy\core\i nclude -ID:\Python25\include -ID:\Python25\PC /Tcnumpy\core\src\multiarraymodule .c /Fobuild\temp.win32-2.5\Release\numpy\core\src\multiarraymodule.obj" failed w ith exit status 2 Is there something I forgot to add in the site.cfg ? Matthieu -- French PhD student Website : http://miles.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher

3 4

numpy 1.04 numpy.test() hang
by Geoffrey Zhu Nov. 10, 2007

Nov. 10, 2007

Good morning. I just installed the Windows binary of numpy 1.04. When I ran numpy.test() in IDLE (the Python shell that comes with Python), the program hang (or at least is running for half an hour). I am using Windows XP, duel core intel CPU. Does anyone know what is going on? Thanks, Geoffrey

6 12

Unnecessarily bad performance of elementwise operators with Fortran-arrays
by Hans Meine Nov. 10, 2007

Nov. 10, 2007

Hi! I wonder why simple elementwise operations like "a * 2" or "a + 1" are not performed in order of increasing memory addresses in order to exploit CPU caches etc. - as it is now, their speed drops by a factor of around 3 simply by transpose()ing. Similarly (but even less logical), copy() and even the constructor are affected (yes, I understand that copy() creates contiguous arrays, but shouldn't it respect/retain the order nevertheless?): ### constructor ### In [89]: %timeit -r 10 -n 1000000 numpy.ndarray((3,3,3)) 1000000 loops, best of 10: 1.19 s per loop In [90]: %timeit -r 10 -n 1000000 numpy.ndarray((3,3,3), order="f") 1000000 loops, best of 10: 2.19 s per loop ### copy 3x3x3 array ### In [85]: a = numpy.ndarray((3,3,3)) In [86]: %timeit -r 10 a.copy() 1000000 loops, best of 10: 1.14 s per loop In [87]: a = numpy.ndarray((3,3,3), order="f") In [88]: %timeit -r 10 -n 1000000 a.copy() 1000000 loops, best of 10: 3.39 s per loop ### copy 256x256x256 array ### In [74]: a = numpy.ndarray((256,256,256)) In [75]: %timeit -r 10 a.copy() 10 loops, best of 10: 119 ms per loop In [76]: a = numpy.ndarray((256,256,256), order="f") In [77]: %timeit -r 10 a.copy() 10 loops, best of 10: 274 ms per loop ### fill ### In [79]: a = numpy.ndarray((256,256,256)) In [80]: %timeit -r 10 a.fill(0) 10 loops, best of 10: 60.2 ms per loop In [81]: a = numpy.ndarray((256,256,256), order="f") In [82]: %timeit -r 10 a.fill(0) 10 loops, best of 10: 60.2 ms per loop ### power ### In [151]: a = numpy.ndarray((256,256,256)) In [152]: %timeit -r 10 a ** 2 10 loops, best of 10: 124 ms per loop In [153]: a = numpy.asfortranarray(a) In [154]: %timeit -r 10 a ** 2 10 loops, best of 10: 458 ms per loop ### addition ### In [160]: a = numpy.ndarray((256,256,256)) In [161]: %timeit -r 10 a + 1 10 loops, best of 10: 139 ms per loop In [162]: a = numpy.asfortranarray(a) In [163]: %timeit -r 10 a + 1 10 loops, best of 10: 465 ms per loop ### fft ### In [146]: %timeit -r 10 numpy.fft.fft(vol, axis=0) 10 loops, best of 10: 1.16 s per loop In [148]: %timeit -r 10 numpy.fft.fft(vol0, axis=2) 10 loops, best of 10: 1.16 s per loop In [149]: vol.flags Out[149]: C_CONTIGUOUS : True F_CONTIGUOUS : False OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [150]: vol0.flags Out[150]: C_CONTIGUOUS : False F_CONTIGUOUS : True OWNDATA : False WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In [9]: %timeit -r 10 numpy.fft.fft(vol0, axis=0) 10 loops, best of 10: 939 ms per loop ### mean ### In [173]: %timeit -r 10 vol.mean() 10 loops, best of 10: 272 ms per loop In [174]: %timeit -r 10 vol0.mean() 10 loops, best of 10: 683 ms per loop ### max ### In [175]: %timeit -r 10 vol.max() 10 loops, best of 10: 63.8 ms per loop In [176]: %timeit -r 10 vol0.max() 10 loops, best of 10: 475 ms per loop ### min ### In [177]: %timeit -r 10 vol.min() 10 loops, best of 10: 63.8 ms per loop In [178]: %timeit -r 10 vol0.min() 10 loops, best of 10: 476 ms per loop ### rot90 ### In [10]: %timeit -r 10 numpy.rot90(vol) 100000 loops, best of 10: 6.97 s per loop In [12]: %timeit -r 10 numpy.rot90(vol0) 100000 loops, best of 10: 6.92 s per loop -- Ciao, / / /--/ / / ANS

8 30

NumPy 1.04, MKL 10.0, & Intel 10.1 icc & ifort
by rex Nov. 9, 2007

Nov. 9, 2007

Build was successful after a change in distutils. Core 2 Duo, Debian Etch-32, Python 2.5, icc 10.1, ifort 10.1, & mkl 10.0. MKL & the compilers were installed to their default locations: /opt/intel/mkl/, /opt/intel/cc/, /opt/intel/fc/ Installation will not interfere with earlier versions. NumPy site.cfg for MKL 10.0 ========================================== [DEFAULT] library_dirs=/opt/intel/mkl/10.0.011/lib/32 include_dirs=/opt/intel/mkl/10.0.011/include [mkl] libraries=mkl,guide [lapack_info] libraries=mkl_lapack,mkl,guide ========================================== Changes in distutils: ========================================================= In MKL 9.1 & 10.0, mkl_lapack32 & mkl_lapack64 no longer exist. They were replaced by /32/mkl_lapack and /64/mkl_lapack. As a result, numpy-1.04/numpy/distutils/system_info.py needs to be changed: #lapack_libs = self.get_libs('lapack_libs',['mkl_lapack32','mkl_lapack64']) lapack_libs = self.get_libs('lapack_libs',['mkl_lapack']) If the above change is not made numpy builds, but without using MKL. In numpy-1.04/numpy/distutils/ccompiler.py change: #cc_exe = 'icc' #works, but is suboptimal cc_exe = 'icc -msse3 -fast' #adjust to suit your cpu ========================================================= Running the command below (all one line) from the numpy-1.04 directory python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install > inst.log Gave this inst.log: ===================================================== F2PY Version 2_4412 blas_opt_info: blas_mkl_info: FOUND: libraries = ['mkl', 'pthread', 'mkl', 'guide'] library_dirs = ['/opt/intel/mkl/10.0.011/lib/32'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/mkl/10.0.011/include'] FOUND: libraries = ['mkl', 'pthread', 'mkl', 'guide'] library_dirs = ['/opt/intel/mkl/10.0.011/lib/32'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/mkl/10.0.011/include'] lapack_opt_info: lapack_mkl_info: mkl_info: FOUND: libraries = ['mkl', 'pthread', 'mkl', 'guide'] library_dirs = ['/opt/intel/mkl/10.0.011/lib/32'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/mkl/10.0.011/include'] FOUND: libraries = ['mkl_lapack', 'mkl', 'pthread', 'mkl', 'guide', 'mkl', 'guide'] library_dirs = ['/opt/intel/mkl/10.0.011/lib/32'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/mkl/10.0.011/include'] FOUND: libraries = ['mkl_lapack', 'mkl', 'pthread', 'mkl', 'guide', 'mkl', 'guide'] library_dirs = ['/opt/intel/mkl/10.0.011/lib/32'] define_macros = [('SCIPY_MKL_H', None)] include_dirs = ['/opt/intel/mkl/10.0.011/include'] running config running build_clib running build_ext running build_src building py_modules sources creating build [...] =========================================================== For reference, here are the respective lib/32 files for MKL 9.1 and 10.0: c2d0:/opt/intel/mkl/9.1/lib/32# ls libguide.a libmkl_gfortran.a libmkl_ias.so libmkl_p3.so libmkl_p4.so libmkl_vml_def.so libmkl_vml_p4p.so libguide.so libmkl_gfortran.so libmkl_lapack.a libmkl_p4m.so libmkl.so libmkl_vml_p3.so libmkl_vml_p4.so libmkl_def.so libmkl_ia32.a libmkl_lapack.so libmkl_p4p.so libmkl_solver.a libmkl_vml_p4m.so libvml.so c2d0:/opt/intel/mkl/10.0.011/lib/32# ls libguide.a libmkl_cdft_core.a libmkl_intel.a libmkl_p4.so libmkl_vml_ia.so libguide.so libmkl_core.a libmkl_intel.so libmkl_scalapack.a libmkl_vml_p3.so libiomp5.a libmkl_core.so libmkl_intel_thread.a libmkl_scalapack_core.a libmkl_vml_p4m2.so libiomp5.so libmkl_def.so libmkl_intel_thread.so libmkl_sequential.a libmkl_vml_p4m.s o libmkl_blacs.a libmkl_gf.a libmkl_lapack.a libmkl_sequential.so libmkl_vml_p4p.so libmkl_blacs_intelmpi20.a libmkl_gf.so libmkl_lapack.so libmkl.so libmkl_vml_p4.so libmkl_blacs_intelmpi.a libmkl_gnu_thread.a libmkl_p3.so libmkl_solver.a libmkl_blacs_openmpi.a libmkl_gnu_thread.so libmkl_p4m.so libmkl_solver_sequential.a libmkl_cdft.a libmkl_ia32.a libmkl_p4p.so libmkl_vml_def.so Thanks to all who have helped me with earlier versions. -rex

1 0

Suggestions on the best way to customize compilation flags
by David Cournapeau Nov. 9, 2007

Nov. 9, 2007

Hi, I would like know the opinion from other people on the best way to customize compilation flags for building numpy. I am talking about the end-user point of view (e.g. I want to use the --optimize--so-aggressively-that-my-cpu-burns-into-flames flags, but without breaking build of python extensions because I forgot -fPIC). One of the goal of using scons within distutils was to separate the logic between compilation options which are necessary to make the binary work at all (basically, what are the options to build a python extension on supported platforms, how to build a dll for ctypes, etc...) and the ones which are 'secondary' (optimization, debug, warnings). I have started working in this direction, using compiler configuration objects, which are just dictionaries: http://projects.scipy.org/scipy/numpy/browser/branches/numpy.scons/numpy/di… The idea is that: - those configurations can be easily changed during the configure step of a package (for example, use sse unit instead of x87, etc...) - those configuration are not used directly when building projects: they are first copied to the actually used flags. The idea is that if necessary, you could request build environments without some warning flags (think pyrex/swig compilation), and only add them on user request. - those configurations are totally independent from the options necessary to build python extensions (those are not user accessible; I have not yet cleaned everything, but the goal is to have those flags set outside the extension builder logic, to avoid the distutils mess, which put and modifying flags everywhere deep down in the code). thanks, David

1 0

Warning or error on conversion from complex to float.
by Michael McNeil Forbes Nov. 8, 2007

Nov. 8, 2007

Hi, Is it possible or easy to add a warning and/or error when array assignments are made that lose information? I just got caught with the following type of code: def f(): return numpy.array([1j,2.0]) x = numpy.empty((2,),dtype=float) x[:] = f() I am pre-allocating arrays for speed, but made a change to f() resulting in complex results. My code now silently ignores the imaginary part. It would have been very helpful if the assignment check to make sure that the conversion was not going to lose information (presumably with an assert so that performance is not affected outside of debugging). Any simple suggestions on how to avoid this problem in the future? How hard would it be to add such a check in numpy? Thanks, Michael.

2 1

Improving numpy.interp
by LB Nov. 8, 2007

Nov. 8, 2007

I often need to make a linear interpolation for a single scalar value but this is not very simple with numpy.interp : >>> import numpy as n >>> n.__version__ '1.0.5.dev4420' >>> xp = n.arange(10) >>> yp = 2.5 + xp**2 -x >>> x = 3.2 >>> n.interp(x, xp, yp) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: object of too small depth for desired array So I use the following trick (which is really ugly) : >>> n.interp([x], xp, yp)[0] 7.7000000000000011 Did I miss an obvious way to do it ? If not, I'd be tempted to patch interp to let it accept scalar values as first argument as follow : >>> def interp(x, *args, **kwds): ... if type(x) in (float, int): ... return n.interp([x], *args, **kwds).item() ... else : ... return n.interp(x, *args, **kwds) ... >>> >>> interp(x, xp, yp) 7.7000000000000011 >>> interp([x,2*x], xp, yp) array([ 7.7, 38.5]) -- LB

1 0

saving and loading as PNG
by Martin Teichmann Nov. 8, 2007

Nov. 8, 2007

Hi all, i am working with numpy and mostly 16 bit matrices, and so I was looking for a standard way of saving them. I found that PNG actually supports that, but I could not find a way to use this feature from within python. I thought that it actually is a cool way of storing matrices and thus I wrote a little pyrex program that does the job. I figured this might be interesting to a broader audience, so I posted it here, maybe for inclusion in numpy? Some more thoughts: * Other implementations: There is other people who have done such a thing. the PIL knows how to read and write PNG, but only 8 bit. The same holds for matplotlib. * Problems: until now, numpy does not link agains libpng. This should not be a problem on linux-like systems, as those normally all have libpng installed. Windows however, I don't know. Same thing with pyrex. Numpy doesn't use pyrex until now. * other places to put: numpy could be considered to basic to have such a functionality, one could put it in scipy. Well, but png is not really a _scientific_ application (even though I personally use it to do science). Matplotlib would be another nice place to put this feature, big advantage is that matplotlib already links against libpng. So, now I shared my thoughts with you, comments? Greetings, Martin And here the code (188 lines are hopefully ok for a mailinglist?) cimport c_numpy import numpy cdef extern from "stdlib.h": cdef void *malloc(int) cdef void free(void *) cdef extern from "stdio.h": cdef struct file: pass ctypedef file FILE cdef FILE *fopen(char *, char *) cdef int fclose(FILE *) cdef extern from "errno.h": cdef int errno cdef extern from "png.h": cdef struct jmp_buf: pass cdef int setjmp(jmp_buf) cdef struct png_struct: jmp_buf jmpbuf ctypedef png_struct *png_structp cdef struct png_info: int width int height int bit_depth int color_type int channels ctypedef png_info *png_infop cdef struct png_text: int compression char *key char *text int text_length ctypedef png_text *png_textp cdef png_structp png_create_read_struct(char *, void *, void *, void*) cdef void png_init_io(png_structp, FILE *) cdef void png_set_sig_bytes(png_structp, int) cdef png_infop png_create_info_struct(png_structp) cdef void png_read_info(png_structp, png_infop) cdef int png_set_interlace_handling(png_structp) cdef void png_read_update_info(png_structp, png_infop) cdef void png_read_image(png_structp, unsigned char **) cdef void png_read_png(png_structp, png_infop, int, void *) cdef void png_read_end(png_structp, png_infop) cdef void png_destroy_read_struct(png_structp *, png_infop *, void *) cdef png_structp png_create_write_struct(char *, void *, void *, void*) cdef void png_set_IHDR(png_structp, png_infop, unsigned int, unsigned int, int, int, int, int, int) cdef void png_write_info(png_structp, png_infop) cdef void png_write_image(png_structp, unsigned char **) cdef void png_write_end(png_structp, png_infop) cdef void png_destroy_write_struct(png_structp *,png_infop *) cdef void png_set_swap(png_structp) cdef void png_set_rows(png_structp, png_infop, unsigned char **) cdef void png_write_png(png_structp, png_infop, int, void *) cdef void png_set_text(png_structp, png_infop, png_textp, int) cdef int PNG_COLOR_TYPE_GRAY cdef int PNG_COLOR_TYPE_PALETTE cdef int PNG_COLOR_TYPE_RGB cdef int PNG_COLOR_TYPE_RGB_ALPHA cdef int PNG_COLOR_TYPE_GRAY_ALPHA cdef int PNG_INTERLACE_NONE cdef int PNG_COMPRESSION_TYPE_BASE cdef int PNG_FILTER_TYPE_BASE cdef int PNG_TRANSFORM_IDENTITY cdef int PNG_TRANSFORM_SWAP_ENDIAN cdef char *PNG_LIBPNG_VER_STRING def load(fn): """ load a PNG file into a NumPy array. This function can read all kinds of PNG images in a senseful way: Depending on the number of bits per pixel, we chose numpy.bool, numpy.unit8 or numpy.uint16 as base type. For gray scale images or pallettes, this returns a 2-D array, for everything else a 3-D array, where the third dimension has a size of 2 for gray+alpha images, 3 for RGB and 4 for RGBA images. """ cdef FILE *f f = fopen(fn, "rb") if f == NULL: raise IOError(errno) cdef png_structp p p = png_create_read_struct(PNG_LIBPNG_VER_STRING, NULL, NULL, NULL) if setjmp(p.jmpbuf): fclose(f) raise RuntimeError("libpng reported error") cdef png_infop i i = png_create_info_struct(p) png_init_io(p, f) png_read_info(p, i) if i.color_type == PNG_COLOR_TYPE_GRAY: shape = (i.height, i.width) elif i.color_type == PNG_COLOR_TYPE_GRAY_ALPHA: shape = (i.height, i.width, 2) elif i.color_type == PNG_COLOR_TYPE_PALETTE: shape = (i.height, i.width) elif i.color_type == PNG_COLOR_TYPE_RGB: shape = (i.height, i.width, 3) elif i.color_type == PNG_COLOR_TYPE_RGB_ALPHA: shape = (i.height, i.width, 4) if i.bit_depth == 1: dtype = numpy.bool elif i.bit_depth == 8: dtype = numpy.uint8 elif i.bit_depth == 16: dtype = numpy.uint16 png_set_interlace_handling(p) png_read_update_info(p, i) r = numpy.zeros(shape, dtype) cdef c_numpy.ndarray s s = r cdef unsigned char **rp rp = <unsigned char **> malloc(sizeof(char *) * i.height) cdef int j for j from 0 <= j < i.height: rp[j] = <unsigned char *> &s.data[(j * i.channels * i.width * i.bit_depth) / 8] png_set_swap(p) png_read_image(p, rp) free(rp) png_destroy_read_struct(&p, &i, NULL) fclose(f) return r def save(fn, data): """ Save a NumPy array to PNG This function saves the NumPy array in data to the file fn. It saves the data the best way it can: 2-D arrays are saved as gray scale images, 3-D arrays are saved as gray+alpha, rgb or rgba depending on the size of the 3rd dimension. """ cdef c_numpy.ndarray d d = numpy.ascontiguousarray(data) cdef int j cdef int width if data.itemsize > 2: raise ValueError("PNG cannot handle BPP > 16") if data.ndim == 2: width = data.shape[1] * data.itemsize elif data.ndim == 3: width = data.shape[2] * data.shape[1] * data.itemsize else: raise ValueError("Can only save 2-D or 3-D arrays to PNG") cdef FILE *f f = fopen(fn, "wb") cdef png_structp p p = png_create_write_struct(PNG_LIBPNG_VER_STRING, NULL, NULL, NULL) cdef unsigned char **rp rp = NULL if setjmp(p.jmpbuf): free(rp) fclose(f) raise RuntimeError("libpng reported error") cdef png_infop i i = png_create_info_struct(p) png_init_io(p, f) if data.ndim == 2: color = PNG_COLOR_TYPE_GRAY elif data.shape[2] == 2: color = PNG_COLOR_TYPE_GRAY_ALPHA elif data.shape[2] == 3: color = PNG_COLOR_TYPE_RGB elif data.shape[2] == 4: color = PNG_COLOR_TYPE_RGB_ALPHA else: raise ValueError("Third dimension must be <= 4 for saving to PNG") png_set_IHDR(p, i, data.shape[1], data.shape[0], data.itemsize*8, color, PNG_INTERLACE_NONE, PNG_COMPRESSION_TYPE_BASE, PNG_FILTER_TYPE_BASE) rp = <unsigned char **> malloc(sizeof(char *) * data.shape[0]) for j from 0 <= j < data.shape[0]: rp[j] = <unsigned char *> &d.data[j * width] png_set_rows(p, i, rp) png_write_png(p, i, PNG_TRANSFORM_SWAP_ENDIAN, NULL) free(rp) png_destroy_write_struct(&p, &i) fclose(f)

2 1

SWIGed function does not like my boolean array
by Sebastian Haase Nov. 8, 2007

Nov. 8, 2007

Hi, I have a templated function written in C++. My SWIG typemap instantiates this for many argument array types (such as unit8, int16, uint16, int32, float32,float64,...) All works well, except when feeding in a boolean array as in seb.mmms(a>6) I get this error: NotImplementedError: No matching function for overloaded 'mmms' What should I do ? Preferably I would avoid having to add another type-instantiation into the library (it looks already quite bloated having 6+ versions of every function). Isn't bool just a synonym for int32 ? Thanks Sebastian Haase

3 9

Why is g++ used for link with mingw ?
by David Cournapeau Nov. 8, 2007

Nov. 8, 2007

Hi, I was wondering why the linking was done with g++ with mingw in numpy.distutils ? (I got some trouble because some function in math library are only defined for gcc and not g++, for example atanhf). This also means that g++ is required to build numpy, which is a bit strange since there is not a single C++ line; is there an issue I am missing, or is this legacy (the mingw tools in numpy.distutils seems to be 6 years old from the comments). cheers, David

1 0

ANN: NumPy 1.0.4
by Jarrod Millman Nov. 7, 2007

Nov. 7, 2007

I'm pleased to announce the release of NumPy 1.0.4. NumPy is the fundamental package needed for scientific computing with Python. It contains: * a powerful N-dimensional array object * sophisticated (broadcasting) functions * basic linear algebra functions * basic Fourier transforms * sophisticated random number capabilities * tools for integrating Fortran code. Besides it's obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide-variety of databases. This is largely a bug fix release with one notable improvements: - The NumPy and SciPy developers have decided to adopt the Python naming convention for classes. So as of this release, TestCase classes may now be prefixed with either 'test' or 'Test'. This will allow us to write TestCase classes using the CapCase words, while still accepting the old style names. For information, please see the release notes: http://sourceforge.net/project/shownotes.php?release_id=552568&group_id=1369 Thank you to everybody who contributed to the recent release. Enjoy, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 0

[f2py] mailing lists, bug, and .f2py_f2cmap, Oh My!
by Peter Schmitt Nov. 7, 2007

Nov. 7, 2007

Hi, I have one short question, a possible bug report, and one longer question: 1. (short question) Which mailing list is appropriate for f2py discussion: numpy-discussion or f2py-users? 2. (bug report?) I'm using numpy from svn (revision 4410). When using f2py to compile, I got an error: File "/home/pschmitt/usr/local/lib/python2.5/site-packages/numpy/f2py/rules.py", line 1228, in buildmodule for l in '\n\n'.join(funcwrappers2)+'\n'.split('\n'): TypeError: cannot concatenate 'str' and 'list' objects I made what I think should be a fix: $ svn diff numpy/numpy/f2py/rules.py Index: numpy/numpy/f2py/rules.py =================================================================== --- numpy/numpy/f2py/rules.py (revision 4410) +++ numpy/numpy/f2py/rules.py (working copy) @@ -1219,7 +1219,7 @@ f.write('! This file is autogenerated with f2py (version:%s)\n'%(f2py_version)) f.write('! It contains Fortran 90 wrappers to fortran functions.\n') lines = [] - for l in '\n\n'.join(funcwrappers2)+'\n'.split('\n'): + for l in ('\n\n'.join(funcwrappers2)+'\n').split('\n'): if len(l)>72 and l[0]==' ': lines.append(l[:72]+'&\n &') l = l[72:] but I'm not sure if this is what Pearu wanted with that block of code.... Should I submit a bug report??? 3. (Longer question) I have 3 source files: a) types_QG.f90, which specifies the "types_QG" module: "integer, parameter :: WP=4, intdim=selected_int_kind(8)" b) fftpack.f which is a f77 program to perform fourier transforms in 1D c) fft_lib.f90 which is a f90 program (a f90 module, actually) that uses fftpack.f to do 1,2 or 3D fourier transforms. I want access to access the functions in fft_lib.f90 via Python... So I use f2py! :) I just learned about ".f2py_f2cmap" to handle fortran90 <type spec> (kind=KIND(..)) statements and I'm trying to implement it into my code [http://cens.ioc.ee/projects/f2py2e/FAQ.html#q-what-if-fortran-90-code-uses-…]. f2py successfully applies the f2py_f2cmap: make fft gfortran -fPIC -c types_QG.f90 gfortran -fPIC -c fftpack.f f2py2.5 --fcompiler=gfortran -m fft_lib -c types_QG.o fftpack.o fft_lib.f90 Reading .f2py_f2cmap ... Mapping "real(kind=wp)" to "float" Mapping "real(kind=WP)" to "float" Mapping "integer(kind=intdim)" to "int" Succesfully applied user defined changes from .f2py_f2cmap ... However, f2py complains about the "kind=" syntax in every single function I have defined in the module within fft_lib.f90. The errors look something like this: > /tmp/tmpeP_IaX/src.linux-i686-2.5/fft_lib-f2pywrappers2.f90:9.16: > > real(kind=wp) a(2,2,2) > 1 > Error: Parameter 'wp' at (1) has not been declared or is a variable, which does not reduce to a constant expression If I remove every function from the fft_lib.f90 source code, f2py succesfully compiles the Python module, and I can call all the subroutines in fft_lib. Any ideas on what's going on with f2py, the "kind" statement, .f2py_f2cmap, and functions within a f90 module? Thanks, Pete

1 1