Mailman 3 February 2008 - NumPy-Discussion

FORTRAN compiler detection
by Christopher Hanley 27 Feb '08

27 Feb '08

Greetings, I was wondering if within the last 8 - 10 weeks anyone has made changes to the way FORTRAN compilers are detected. In the past I was able to specify which compiler was used by the F77 system variable. However, I am now having a f90 compiler that exists on my Solaris system detected regardless of the F77 value. Even unsetting the F77 variable leads to the use of the f90 compiler. I was going to look though the distutil change logs but I was hoping that someone might remember changing something off the top of their heads. Thank you for your time and help, Chris -- Christopher Hanley Systems Software Engineer Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21218 (410) 338-4338

4 7

#error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)
by Manos Pontikakis 27 Feb '08

27 Feb '08

Hello, I am trying to install numpy 1.0.4 to the following machine: $ uname -a Linux myhostname 2.6.9-55.ELsmp #1 SMP Sat Apr 21 11:16:24 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux and I am getting the error that appears in the subject of this email. It appears that BLAS and LAPACK libraries cannot be found as well, but I think numpy still shouldn't have problem with that. There must be some problem with 64 bit. Please see the complete log. ------------------------------------------------------------------------------------------- Running from numpy source directory. F2PY Version 2_4422 blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2 libraries ptf77blas,ptcblas,atlas not found in /usr/lib NOT AVAILABLE atlas_blas_info: libraries f77blas,cblas,atlas not found in /usr/local/lib libraries f77blas,cblas,atlas not found in /usr/lib/sse2 libraries f77blas,cblas,atlas not found in /usr/lib NOT AVAILABLE /.../numpy-1.0.4/numpy/distutils/system_info.py:1340: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) blas_info: libraries blas not found in /usr/local/lib FOUND: libraries = ['blas'] library_dirs = ['/usr/lib'] language = f77 FOUND: libraries = ['blas'] library_dirs = ['/usr/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide not found in /usr/local/lib libraries mkl,vml,guide not found in /usr/lib NOT AVAILABLE NOT AVAILABLE atlas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/local/lib libraries ptf77blas,ptcblas,atlas not found in /usr/lib/sse2 libraries lapack_atlas not found in /usr/lib/sse2 libraries ptf77blas,ptcblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib numpy.distutils.system_info.atlas_threads_info NOT AVAILABLE atlas_info: libraries f77blas,cblas,atlas not found in /usr/local/lib libraries lapack_atlas not found in /usr/local/lib libraries f77blas,cblas,atlas not found in /usr/lib/sse2 libraries lapack_atlas not found in /usr/lib/sse2 libraries f77blas,cblas,atlas not found in /usr/lib libraries lapack_atlas not found in /usr/lib numpy.distutils.system_info.atlas_info NOT AVAILABLE /.../numpy-1.0.4/numpy/distutils/system_info.py:1247: UserWarning: Atlas (http://math-atlas.sourceforge.net/) libraries not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [atlas]) or by setting the ATLAS environment variable. warnings.warn(AtlasNotFoundError.__doc__) lapack_info: libraries lapack not found in /usr/local/lib FOUND: libraries = ['lapack'] library_dirs = ['/usr/lib'] language = f77 FOUND: libraries = ['lapack', 'blas'] library_dirs = ['/usr/lib'] define_macros = [('NO_ATLAS_INFO', 1)] language = f77 running install running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler options running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options running build_src building py_modules sources building extension "numpy.core.multiarray" sources Generating build/src.linux-x86_64-2.5/numpy/core/config.h customize GnuFCompiler Could not locate executable g77 Could not locate executable f77 customize IntelFCompiler Could not locate executable ifort Could not locate executable ifc customize LaheyFCompiler Could not locate executable lf95 customize PGroupFCompiler Could not locate executable pgf90 Could not locate executable pgf77 customize AbsoftFCompiler Could not locate executable f90 customize NAGFCompiler Could not locate executable f95 customize VastFCompiler customize GnuFCompiler customize CompaqFCompiler Could not locate executable fort customize IntelItaniumFCompiler Could not locate executable efort Could not locate executable efc customize IntelEM64TFCompiler customize Gnu95FCompiler Found executable /usr/bin/gfortran customize G95FCompiler Could not locate executable g95 don't know how to compile Fortran code on platform 'posix' C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC compile options: '-I/.../python2.5 -Inumpy/core/src -Inumpy/core/include -I/.../python2.5 -c' gcc: _configtest.c In file included from /.../python2.5/Python.h:57, from _configtest.c:2: */.../python2.5/pyport.h:730:2: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."* _configtest.c: In function `main': _configtest.c:50: warning: int format, different type arg (arg 3) _configtest.c:57: warning: int format, different type arg (arg 3) _configtest.c:72: warning: int format, different type arg (arg 3) In file included from /.../python2.5/Python.h:57, from _configtest.c:2: */.../python2.5/pyport.h:730:2: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)."* _configtest.c: In function `main': _configtest.c:50: warning: int format, different type arg (arg 3) _configtest.c:57: warning: int format, different type arg (arg 3) _configtest.c:72: warning: int format, different type arg (arg 3) failure. removing: _configtest.c _configtest.o Traceback (most recent call last): File "setup.py", line 89, in <module> setup_package() File "setup.py", line 82, in setup_package configuration=configuration ) File "/.../numpy-1.0.4/numpy/distutils/core.py", line 176, in setup return old_setup(**new_attr) File "/.../python2.5/distutils/core.py", line 151, in setup dist.run_commands() File "/.../python2.5/distutils/dist.py", line 974, in run_commands self.run_command(cmd) File "/.../python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/.../numpy-1.0.4/numpy/distutils/command/install.py", line 16, in run r = old_install.run(self) File "/.../python2.5/distutils/command/install.py", line 506, in run self.run_command('build') File "/.../python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/.../python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/.../python2.5/distutils/command/build.py", line 112, in run self.run_command(cmd_name) File "/.../python2.5/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/.../python2.5/distutils/dist.py", line 994, in run_command cmd_obj.run() File "/.../numpy-1.0.4/numpy/distutils/command/build_src.py", line 130, in run self.build_sources() File "/.../numpy-1.0.4/numpy/distutils/command/build_src.py", line 147, in build_sources self.build_extension_sources(ext) File "/.../numpy-1.0.4/numpy/distutils/command/build_src.py", line 250, in build_extension_sources sources = self.generate_sources(sources, ext) File "/.../numpy-1.0.4/numpy/distutils/command/build_src.py", line 307, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 53, in generate_config_h raise SystemError,"Failed to test configuration. "\ SystemError: Failed to test configuration. See previous error messages for more information. ------------------------------------------------------------------------------------------- Is there a specific configuration that I should apply for 64 bit machines? Thank you, Manos

2 1

masked_array/matplotlib issue with memmaps
by Christopher Burns 27 Feb '08

27 Feb '08

If I initialize an AxesImage using a np.zeros array and then set the axes data later to a np.memmap array, I get a RuntimeError when matplotlib tries to autoscale the image. The errors continue to fill my console and I'm forced to close the shell. This bug was introduced when I switched from numpy v1.0.3.1 to the trunk v1.0.5.dev4815 The two hacks to get around this are: 1) Setting any array element to something other than zero fixes the error: zdata[0,0] = 1 2) Specify the extent and max/min values when creating the image: imgaxes = pylab.imshow(zdata, extent=(0, data_shape[1], data_shape[0], 0), vmin=0, vmax=1) Unfortunately, due to the way this errors I'm having a difficult time debugging it. I'm hoping someone with in-depth knowledge of masked_arrays will have some insight. Code and output are below. Thanks! Chris ---- script to reproduce the bug ---- import pylab import numpy as np def printinfo(imgaxes): a = imgaxes.get_array() print '\nimgaxes array info:' print 'type', type(a) print 'shape', a.shape print 'dtype', a.dtype print 'has _mmap', hasattr(a, '_mmap') data_type = 'float32' data_shape = (30, 40) zdata = np.zeros(data_shape, dtype=data_type) #zdata[0,0] = 1 # No exception raised if this line is executed imgaxes = pylab.imshow(zdata) printinfo(imgaxes) mmdata = np.memmap('foo.dat', dtype=zdata.dtype, shape=zdata.shape, mode='w+') imgaxes.set_data(mmdata) printinfo(imgaxes) # imgaxes array now has a _mmap pylab.show() ---- version info ---- In [2]: pylab.matplotlib.__version__ Out[2]: '0.91.2' In [4]: numpy.version.version Out[4]: '1.0.5.dev4817' ---- error ---- In [26]: run memmap_reassign.py imgaxes array info: type <class 'numpy.ma.core.MaskedArray'> shape (30, 40) dtype float32 has _mmap False imgaxes array info: type <class 'numpy.ma.core.MaskedArray'> shape (30, 40) dtype float32 has _mmap True Exception exceptions.RuntimeError: 'maximum recursion depth exceeded' in ignored ERROR: An unexpected error occurred while tokenizing input The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (10, 0)) --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) /Users/cburns/local/lib/python2.5/site-packages/matplotlib/backends/backend_wx.pyc in _onPaint(self, evt) 1079 self.realize() 1080 # Render to the bitmap -> 1081 self.draw(repaint=False) 1082 # Update the display using a PaintDC 1083 self.gui_repaint(drawDC=wx.PaintDC(self)) /Users/cburns/local/lib/python2.5/site-packages/matplotlib/backends/backend_wxagg.pyc in draw(self, repaint) 59 """ 60 DEBUG_MSG("draw()", 1, self) ---> 61 FigureCanvasAgg.draw(self) 62 63 self.bitmap = _convert_agg_to_wx_bitmap(self.get_renderer(), None) /Users/cburns/local/lib/python2.5/site-packages/matplotlib/backends/backend_agg.pyc in draw(self) 356 357 self.renderer = self.get_renderer() --> 358 self.figure.draw(self.renderer) 359 360 def get_renderer(self): /Users/cburns/local/lib/python2.5/site-packages/matplotlib/figure.pyc in draw(self, renderer) 622 623 # render the axes --> 624 for a in self.axes: a.draw(renderer) 625 626 # render the figure text /Users/cburns/local/lib/python2.5/site-packages/matplotlib/axes.pyc in draw(self, renderer, inframe) 1303 mag = renderer.get_image_magnification() 1304 ims = [(im.make_image(mag),0,0) -> 1305 for im in self.images if im.get_visible()] 1306 1307 /Users/cburns/local/lib/python2.5/site-packages/matplotlib/image.pyc in make_image(self, magnification) 129 im.is_grayscale = False 130 else: --> 131 x = self.to_rgba(self._A, self._alpha) 132 im = _image.fromarray(x, 0) 133 if len(self._A.shape) == 2: /Users/cburns/local/lib/python2.5/site-packages/matplotlib/cm.pyc in to_rgba(self, x, alpha, bytes) 74 x = ma.asarray(x) 75 x = self.norm(x) ---> 76 x = self.cmap(x, alpha=alpha, bytes=bytes) 77 return x 78 /Users/cburns/local/lib/python2.5/site-packages/matplotlib/colors.pyc in __call__(self, X, alpha, bytes) 431 vtype = 'array' 432 xma = ma.asarray(X) --> 433 xa = xma.filled(0) 434 mask_bad = ma.getmask(xma) 435 if xa.dtype.char in npy.typecodes['Float']: /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in filled(self, fill_value) 1542 m = self._mask 1543 if m is nomask or not m.any(): -> 1544 return self._data 1545 # 1546 if fill_value is None: /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) /Users/cburns/local/lib/python2.5/site-packages/numpy/core/memmap.pyc in __array_finalize__(self, obj) 204 self._mmap = obj._mmap 205 else: --> 206 raise ValueError, 'Cannot create a memmap from object %s'%obj 207 else: 208 self._mmap = None /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in __str__(self) 1614 m = self._mask 1615 if m is nomask: -> 1616 res = self._data 1617 else: 1618 if m.shape == (): /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) /Users/cburns/local/lib/python2.5/site-packages/numpy/core/memmap.pyc in __array_finalize__(self, obj) 204 self._mmap = obj._mmap 205 else: --> 206 raise ValueError, 'Cannot create a memmap from object %s'%obj 207 else: 208 self._mmap = None /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in __str__(self) 1614 m = self._mask 1615 if m is nomask: -> 1616 res = self._data 1617 else: 1618 if m.shape == (): /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) /Users/cburns/local/lib/python2.5/site-packages/numpy/core/memmap.pyc in __array_finalize__(self, obj) 204 self._mmap = obj._mmap 205 else: --> 206 raise ValueError, 'Cannot create a memmap from object %s'%obj 207 else: 208 self._mmap = None .... [snip] /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) RuntimeError: maximum recursion depth exceeded Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in ignored Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in ignored [snip] Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])> ignored Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])> ignored [snip]

2 1

numpy.random.randint() inconsistent with plain random.randint()
by Christopher Kerr 26 Feb '08

26 Feb '08

I don't know if this is the right place to report bugs, but I couldn't find anywhere else on the website... random.randint(min,max) from python core returns an integer between min and max inclusive. The documentation on the website says that numpy.random.randint(min,max [,size]) does this too, but it in fact only ever returns numbers strictly less than the max, and gives an error if min is equal to max Python 2.4.4 (#1, Dec 20 2007, 08:43:49) [GCC 4.1.2 20070214 ( (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import random >>> import numpy.random >>> numpy.version.version '1.0.4' >>> random.randint(3,3) 3 >>> numpy.random.randint(3,3) Traceback (most recent call last): File "<stdin>", line 1, in ? File "mtrand.pyx", line 600, in mtrand.RandomState.randint ValueError: low >= high >>> numpy.random.randint(1,3,(2,10)) array([[2, 1, 2, 1, 2, 1, 2, 1, 2, 1], [1, 2, 1, 2, 1, 1, 1, 2, 1, 2]])

2 2

py2exe, pyinstaller and numpy
by Francesco Anselmo 25 Feb '08

25 Feb '08

Hi all! I have just subscribed to this mailing list and would like to say hello to everybody and a big thank you to all the people involved in the development of numpy and scipy. My first question is about packaging numpy scripts into executables. I have googled around and read mailing-list posts, but I haven't found the definitive answer, yet: is there a working method of "compiling" numpy and scipy scripts into binaries on Windows? In the best occasion, I manage to create the executable, but get errors like this one: No scipy-style subpackage 'dft' found in Z:\Projects\lightinglib\dist \library.zi p\numpy. Ignoring: cannot import name deprecate Traceback (most recent call last): File "uv2lmap.py", line 5, in ? File "zipextimporter.pyc", line 82, in load_module File "threedee\__init__.pyc", line 5, in ? File "zipextimporter.pyc", line 82, in load_module File "threedee\mapping.pyc", line 14, in ? File "zipextimporter.pyc", line 82, in load_module File "numpy\__init__.pyc", line 35, in ? File "numpy\_import_tools.pyc", line 173, in __call__ File "numpy\_import_tools.pyc", line 68, in _init_info_modules File "<string>", line 1, in ? File "zipextimporter.pyc", line 82, in load_module File "numpy\random\__init__.pyc", line 3, in ? File "zipextimporter.pyc", line 98, in load_module File "numpy.pxi", line 32, in mtrand AttributeError: 'module' object has no attribute 'dtype' Many thanks in advance. Francesco

1 0

How do I change endianess?
by debl2＠verizon.net 25 Feb '08

25 Feb '08

I would like to change the Endianess of a large of data written on PC so I can process it on a Solaris box. I see that the dtype.str attribute is read-only. TIA David Lees

3 2

Closing tickets
by Charles R Harris 25 Feb '08

25 Feb '08

I believe there are a number of tickets that can be closed. Will the relevant folks please comment on the following: #632: numpy.histogram fails with bin=<list> #568: Aligned allocator for numpy #558: 'axis' support for numpy.median() #555: setting random seed does not work with a numpy.int64 #551: numpy.ndarray messed up after unpickling #69: Why numpy headers are installed using add_data_dir and not add_headers? #637: typo in scalarmathmodule.c.src for ctype_negative Chuck

5 5

build issue on Solaris System
by Christopher Hanley 25 Feb '08

25 Feb '08

Good morning, I've just done a fresh checkout from SVN and attempted to build numpy on a Solaris 10 system. The build actually completes normally. However, when attempting to import numpy I get the following error: Python 2.5.1 (r251:54863, Jan 21 2008, 11:03:00) [C] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/data/basil5/site-packages/lib/python/numpy/__init__.py", line 51, in <module> import linalg File "/data/basil5/site-packages/lib/python/numpy/linalg/__init__.py", line 4, in <module> from linalg import * File "/data/basil5/site-packages/lib/python/numpy/linalg/linalg.py", line 28, in <module> from numpy.linalg import lapack_lite ImportError: ld.so.1: python: fatal: relocation error: file /data/basil5/site-packages/lib/python/numpy/linalg/lapack_lite.so: symbol s_cat: referenced symbol not found >>> I haven't had these issues with lapack in the past. Has anyone made changes to linalg dependencies lately that I must have missed? Thank you for your time and help, Chris -- Christopher Hanley Systems Software Engineer Space Telescope Science Institute 3700 San Martin Drive Baltimore MD, 21218 (410) 338-4338

1 0

numpy 1:1.0.4: numpy.average() returns the wrong result with weights
by Ondrej Certik 24 Feb '08

24 Feb '08

Hi, more details in this bug report. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=467095 The bug report offers a fix for this problem. It seems to me this is not fixed even in the latest svn. Thanks, Ondrej

5 6

Matching 0-d arrays and NumPy scalars
by Travis E. Oliphant 23 Feb '08

23 Feb '08

Hi everybody, In writing some generic code, I've encountered situations where it would reduce code complexity to allow NumPy scalars to be "indexed" in the same number of limited ways, that 0-d arrays support. For example, 0-d arrays can be indexed with * Boolean masks * Ellipses x[...] and x[..., newaxis] * Empty tuple x[()] I think that numpy scalars should also be indexable in these particular cases as well (read-only of course, i.e. no setting of the value would be possible). This is an easy change to implement, and I don't think it would cause any backward compatibility issues. Any opinions from the list? Best regards, -Travis O.

12 38